| Time |
Nickname |
Message |
|
04:40
🔗
|
|
tech234a has joined #webroasting |
|
06:50
🔗
|
|
tech234a has quit IRC (Quit: Connection closed for inactivity) |
|
15:03
🔗
|
|
tech234a has joined #webroasting |
|
17:23
🔗
|
|
tech234a has quit IRC (Quit: Connection closed for inactivity) |
|
18:25
🔗
|
|
kiska is now known as kiskan |
|
18:29
🔗
|
|
kiskan is now known as kiskablah |
|
18:30
🔗
|
|
kiskablah is now known as kiska3 |
|
19:29
🔗
|
|
kiska3 is now known as kiska |
|
20:58
🔗
|
eythian |
OK, the custom search stuff looked more trouble than it's worth, but I was able to get a couple of hundred URLs out pretty quickly by using https://serpapi.com/demo |
|
21:19
🔗
|
eythian |
<JAA> Bing's results are ridiculously bad, and there is a lot of duplication in there. |
|
21:19
🔗
|
eythian |
wow, you're not kidding. |
|
22:36
🔗
|
eythian |
when running grab-site and it stops, but doesn't seem to terminate, does that still mean it's done? |
|
22:38
🔗
|
JAA |
eythian: My theory is that Bing only tolerates bots because nobody else wants to use it due to how bad the results are. ;-) |
|
22:39
🔗
|
eythian |
that seems plausible :) |
|
22:39
🔗
|
JAA |
That way they can boost their usage numbers. |
|
22:40
🔗
|
JAA |
"stops but doesn't terminate" sounds like the wpull bug we've been seeing a lot on ArchiveBot pipelines. But I don't know since I've never used grab-site. You might want to ask ivan over in -ot. |
|
22:40
🔗
|
eythian |
ta will do |