[00:02] *** killsushi has quit IRC (Quit: Leaving) [00:06] *** BlueMax has joined #archiveteam-ot [03:34] *** odemg has quit IRC (Ping timeout: 615 seconds) [03:40] *** odemg has joined #archiveteam-ot [04:09] *** dhyan_nat has joined #archiveteam-ot [05:53] *** svchfoo1 has left [05:53] *** deevious has joined #archiveteam-ot [06:09] *** BnAboyZ has quit IRC (Read error: Operation timed out) [06:11] *** legoktm has quit IRC (Ping timeout: 268 seconds) [06:11] *** VoynichCr has quit IRC (Ping timeout: 268 seconds) [06:11] *** VoynichCr has joined #archiveteam-ot [06:14] *** noirscape has quit IRC (Ping timeout: 268 seconds) [06:14] *** BnAboyZ has joined #archiveteam-ot [06:15] *** dhyan_nat has quit IRC (Ping timeout: 268 seconds) [06:15] *** legoktm has joined #archiveteam-ot [06:16] *** VoynichCr has quit IRC (Ping timeout: 268 seconds) [06:16] *** Mateon1 has quit IRC (Ping timeout: 268 seconds) [06:16] *** Mateon1 has joined #archiveteam-ot [06:17] *** asie has quit IRC (Ping timeout: 268 seconds) [06:17] *** VoynichCr has joined #archiveteam-ot [06:19] *** BnAboyZ has quit IRC (Ping timeout: 268 seconds) [06:19] *** Ravenloft has quit IRC (Read error: Connection reset by peer) [06:19] *** BlueMaxim has joined #archiveteam-ot [06:20] *** BlueMax has quit IRC (Read error: Connection reset by peer) [06:21] *** noirscape has joined #archiveteam-ot [06:21] *** dashcloud has quit IRC (Ping timeout: 268 seconds) [06:22] *** dashcloud has joined #archiveteam-ot [06:22] *** asie has joined #archiveteam-ot [06:22] *** BnAboyZ has joined #archiveteam-ot [06:23] *** dhyan_nat has joined #archiveteam-ot [06:31] *** dhyan_nat has quit IRC (Read error: Operation timed out) [07:04] *** ColdIce has quit IRC (Remote host closed the connection) [07:04] *** ColdIce has joined #archiveteam-ot [07:04] *** Madbrad has joined #archiveteam-ot [07:28] *** Madbrad has quit IRC (Ping timeout: 615 seconds) [08:11] *** Zerote has joined #archiveteam-ot [09:02] VoynichCr: Sorting seems to work correctly now, thanks. :-) [09:15] \o/ [09:20] *** Verified_ has quit IRC (Ping timeout: 252 seconds) [09:26] *** deevious has quit IRC (Quit: deevious) [09:40] *** BlueMaxim has quit IRC (Quit: Leaving) [09:50] *** Verified_ has joined #archiveteam-ot [10:17] *** Verified_ has quit IRC (Ping timeout: 252 seconds) [10:24] *** Verified_ has joined #archiveteam-ot [10:47] *** deevious has joined #archiveteam-ot [12:08] *** icedice has joined #archiveteam-ot [12:11] *** icedice has quit IRC (Client Quit) [12:31] *** dhyan_nat has joined #archiveteam-ot [12:55] *** icedice has joined #archiveteam-ot [13:03] Wiki pages by country help to keep a log of small projects: [13:04] - Brazil https://www.archiveteam.org/index.php?title=Brazil [13:04] - Sri Lanka https://www.archiveteam.org/index.php?title=Sri_Lanka [13:04] - Venezuela https://www.archiveteam.org/index.php?title=Venezuela [13:11] *** icedice has quit IRC (Remote host closed the connection) [13:13] *** icedice has joined #archiveteam-ot [13:16] *** katocala has joined #archiveteam-ot [13:31] *** Oddly has joined #archiveteam-ot [13:46] *** icedice2 has joined #archiveteam-ot [13:50] *** icedice has quit IRC (Ping timeout: 252 seconds) [14:01] *** icedice2 has quit IRC (Read error: Connection reset by peer) [14:01] *** icedice2 has joined #archiveteam-ot [14:05] *** Oddly has quit IRC (Remote host closed the connection) [14:05] *** icedice has joined #archiveteam-ot [14:06] *** icedice has quit IRC (Remote host closed the connection) [14:07] *** Oddly has joined #archiveteam-ot [14:09] *** Oddly has quit IRC (Remote host closed the connection) [14:13] *** Zerote has quit IRC (Ping timeout: 260 seconds) [14:14] *** icedice2 has quit IRC (Read error: Operation timed out) [14:43] *** Zerote has joined #archiveteam-ot [15:21] *** cfarquhar has quit IRC (Read error: Operation timed out) [15:22] *** cfarquhar has joined #archiveteam-ot [16:45] *** Zerote has quit IRC (Ping timeout: 261 seconds) [17:06] *** godane has quit IRC (Quit: Leaving.) [17:30] *** schbirid has quit IRC (Remote host closed the connection) [17:54] *** chferfa has quit IRC (Quit: Connection closed for inactivity) [18:04] Trying to get my highly parallelised archival tool that I've been talking about so many times into shape now. (For the unaware, that's the code I used to retrieve Storify (~100 million URLs), Outpost (same), AMO (~50 million), and others within just a few days from a single machine.) [18:05] I still don't have a good name, but I'm thinking of epat or epwat right now, short for "embarassingly parallel (web) archival tool". [18:05] Any thoughts? [18:10] its a good name [18:14] fastcrawl [18:15] ivan: It's not a crawler though, just a framework for managing work items and making requests/writing them to WARC. There is no recursion or link extraction whatsoever in it. [18:17] qwarc [18:17] fark [18:18] *** Stiletto has joined #archiveteam-ot [18:18] fark archiver ¯\_(ツ)_/¯ [18:18] fast* [18:18] hell what do i know i'm bad with naming projects lol [18:18] *** Stiletto has quit IRC (Client Quit) [18:18] *** Stiletto has joined #archiveteam-ot [18:25] I like qwarc because it sounds like quark and I like modern physics. I also like fark because that's what you'll probably say when you realise just how large the site you're trying to archive is. [18:28] fark's not exactly unique though. Unsurprisingly, there are several projects on GitHub with that name already. [18:29] casting a vote for qwarc [18:30] it's also fairly unique on google, which is impressive for five characters that are pronounceable [18:30] Yeah indeed. [18:31] #archiveteam-branding [18:31] Haha [18:33] *** JAA sets mode: +oo astrid ivan [18:37] I fully expect this channel to be a thing [18:43] i don't ... [20:04] does it do useful things for people who aren't targeting .warc output? [20:07] Nope, because creating WARCs is literally what I wrote it for, and 90 % of the code complexity comes from that. [20:13] *** tuluu has quit IRC (Ping timeout: 265 seconds) [20:16] qwarc is appropriate then, and cute [20:17] it might end up referred to as JAA's archival tool, if you don't end up naming it [20:18] *** dhyan_nat has quit IRC (Read error: Operation timed out) [20:18] *sigh* Lots of election propaganda, and its 1 month from the federal election(in Australia) [20:19] I guess this is why they call it a campaign trail, it leaves ads EVERYWHERE [20:20] the US is 1.5 years away [20:20] Yes, I'm using qwarc. Sorry, thought I had mentioned that earlier. Thanks for the suggestion, ivan. (Also, after naming both snscrape and qwarc, you know I'll ping you whenever I need a name now. :-) ) [20:22] kiska: Yeah, look at https://en.wikipedia.org/wiki/2019_national_electoral_calendar . There are so many more elections to do... :-/ [20:23] * kiska bashes head against wall [20:24] more dictatorships would simplify this [20:25] Isn't the campaign trail for the US started?4 [20:25] Also you guys don't have mandatory voting we do. And the fine for not voting is $150 and it grows exponentially [20:26] I guess thats revenue for the government xD [20:28] Of course it is. Trump started his reelection campaign pretty much the moment he took office I believe, and various Democrats have announced their candidacy already as well. [21:00] *** Kaz has quit IRC (Ping timeout: 265 seconds) [21:01] *** revi has quit IRC (Read error: Connection reset by peer) [21:04] *** Medowar has quit IRC (Ping timeout: 615 seconds) [21:07] *** Kaz has joined #archiveteam-ot [21:08] *** Medowar has joined #archiveteam-ot [21:09] *** revi has joined #archiveteam-ot [21:23] *** marked has quit IRC (Read error: Operation timed out) [22:55] *** BlueMax has joined #archiveteam-ot