#newsgrabber 2017-11-04,Sat

Logs of this channel are not protected. You can protect them by a password.

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)


WhoWhatWhen
***figpucker has joined #newsgrabber
figpucker has quit IRC (Read error: Connection reset by peer)
figpucker has joined #newsgrabber
figpucker has quit IRC (Client Quit)
[09:01]
..................................................................................... (idle for 7h1mn)
JensRexI have a sneaking suspicion that the precompiled wpull is kind of poopy. [16:11]
............................ (idle for 2h17mn)
"ERROR Fetching \u2018https://site.com/example.js\u2019 encountered an error: Connect timed out."
That's never going to work. Invalid URL.
[18:28]
JAALooks valid to me? \u2018 and \u2019 are the markers around the URL, ‘URL’.
Something's broken about those two characters, obviously.
[18:30]
JensRexI know, but it looks like someone used it as an example url.
I don't know if the job failed because of that.
[18:32]
Kaznot much anyone can do about that, that URL will be tried a few times then fail, and wpull will move on to the next one
people put all sorts of bizarre shit into their sites, nothing to worry about too much
[18:32]
JensRexOkay good. It just got hung up there for a long while. Thought it might fail out. [18:35]
JAAI'm not really familiar with NewsGrabber, is it using wpull 2.0.x? [18:36]
Kazcan't remember what the timeout is for stuff like that, could be up to like 5 minutes [18:36]
JensRexNo, 1.2.3. [18:36]
JAAAh ok [18:36]
Kazconsidering site.com does have a valid A record [18:36]
JensRexpipeline.py is Python2, so it can't use regular wpull because that's Python3 only. And the precompiled wpull is weird. [18:37]
JAAhttps://github.com/ArchiveTeam/NewsGrabber-Warrior/blob/31b29fc0ba61a05b081c06852d9ae9413421cab5/pipeline.py#L254-L259
Ah right, same as the Flickr project.
We really need to move this to Python 3.
It's been almost 10 years since that was released...
[18:37]
JensRexHmm, investigating youtube-dl failure further: https://bpaste.net/show/73eb42a23506
I'm not sure what to make of that.
Newsgrabber fails on every video job here.
[18:40]
JAAPython 2 vs. 3?
That paste is clearly using Python 3...
[18:54]
JensRexThat's just using the precompiled wpull.
Uninstalled local youtube-dl --> https://bpaste.net/show/a7fe1552962c
[18:58]
JAAHmm
So that "precompiled" wpull is Python 3.4, as far as I can tell.
I wonder though, why can't you run the normal wpull with Python 3? pipeline.py doesn't "import wpull", it just executes a subprocess.
[19:01]
JensRexI tried that yesterday in a test VM. It failed spectacularly. [19:12]
.................... (idle for 1h36mn)
***ErkDog_ has joined #newsgrabber
ErkDog_ has quit IRC (Remote host closed the connection!)
[20:48]
.......... (idle for 49mn)
figpucker has joined #newsgrabber
figpucker has quit IRC (Client Quit)
[21:37]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)