#newsgrabber 2018-05-23,Wed

Logs of this channel are not protected. You can protect them by a password.

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)


WhoWhatWhen
JensActually, disregard that. I'm a clown and should not own a computer. [01:44]
............. (idle for 1h2mn)
***CoolCanuk has quit IRC (Quit: Connection closed for inactivity) [02:46]
.......... (idle for 45mn)
qw3rty114 has joined #newsgrabber [03:31]
qw3rty113 has quit IRC (Read error: Operation timed out) [03:37]
.... (idle for 15mn)
odemg has quit IRC (Ping timeout: 246 seconds) [03:52]
odemg has joined #newsgrabber [04:06]
............................................................................................... (idle for 7h52mn)
Spydar007 has quit IRC (se.hub irc.efnet.nl)
Spydar007 has joined #newsgrabber
[11:58]
...... (idle for 29mn)
JensIt appears that even with --concurrent set to somethin absurd, the dedupe only runs one thread at a time, stalling the entire pipeline. [12:29]
....... (idle for 30mn)
KazThere's a way around that iirc
I think we merged the fix in, anyway
Jens: https://github.com/ArchiveTeam/NewsGrabber-Warrior/commit/51c1b822937f200755ace841cd8582718c42533a#diff-89b367242c85104b205a8bb4557e972cR274
in theory you can specify a value for shared:dedupe_threads when running it, and it'll respect that
(though that is separate from --concurrent, and I don't remember the correct format for this one)
[12:59]
....................................................... (idle for 4h31mn)
***blitzed has quit IRC (Quit: Leaving) [17:32]
.......................... (idle for 2h6mn)
ivanJens: you can use pyenv to make a Python 3.4.8, that's what the grab-site setup advises [19:38]
............. (idle for 1h4mn)
***odemg has quit IRC (Read error: Operation timed out) [20:42]
.... (idle for 16mn)
odemg has joined #newsgrabber [20:58]
blitzed has joined #newsgrabber [21:04]
........... (idle for 52mn)
Aoede has quit IRC (se.hub efnet.portlane.se)
Aoede has joined #newsgrabber
[21:56]
....... (idle for 33mn)
Spydar007 has quit IRC (se.hub irc.efnet.nl) [22:31]
............ (idle for 57mn)
SmileyG_ has joined #newsgrabber
SmileyG has quit IRC (Read error: Operation timed out)
[23:28]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)