#archiveteam-bs 2020-03-14,Sat

↑back Search

Time Nickname Message
00:11 🔗 OrIdow6 JAA: Nope - I've only seen stickies in the page itseldf
00:11 🔗 OrIdow6 *itself
00:28 🔗 JAA Oh, another fun bug of the boards: https://boards.na.leagueoflegends.com/api/PEr1qIcT/discussions?sort_type=recently_replied&num_loaded=1377400 claims to return 21 results but the HTML string is empty.
00:29 🔗 JAA (It's supposed to return the 21 discussions that were least recently active ever.)
00:51 🔗 godane has joined #archiveteam-bs
02:04 🔗 JAA Oh, fantastic. After finally working around all that rubbish, I did a first load test. Guess what, under load, the API returns an empty list instead of data, still with HTTP 200 though. (╯°□°)╯︵ ┻━┻
02:37 🔗 RichardG_ has quit IRC (Quit: Keyboard not found, press F1 to continue)
02:38 🔗 RichardG has joined #archiveteam-bs
03:24 🔗 wp494 has quit IRC (Quit: LOUD UNNECESSARY QUIT MESSAGES)
03:26 🔗 JAA Running okay now. Estimated 400 GB and ~50 hours for the NA EN boards (presumably the largest ones). So assuming it goes okay now, it should finish on time.
03:35 🔗 OrIdow6 Ok
03:45 🔗 wp494 has joined #archiveteam-bs
04:15 🔗 qw3rty_ has joined #archiveteam-bs
04:23 🔗 qw3rty__ has quit IRC (Read error: Operation timed out)
04:24 🔗 katocala has joined #archiveteam-bs
04:24 🔗 katocala has left
04:43 🔗 JAA Discovered more edge cases, trying to solve them now and will have to restart then.
05:01 🔗 ephemer0l has quit IRC (Read error: Connection reset by peer)
05:11 🔗 ephemer0l has joined #archiveteam-bs
05:47 🔗 JAA Discussion counts on each board according to the API: https://transfer.notkiska.pw/inline/av40k/lol-boards-discussion-counts
05:48 🔗 JAA Apparently the English EU forums are available under both eune and euw.
06:52 🔗 JAA This is fun. ~2k req/s, only a few dozen errors per minute :-)
06:59 🔗 DFJustin has quit IRC (Remote host closed the connection)
06:59 🔗 DFJustin has joined #archiveteam-bs
07:16 🔗 HP_Archiv has quit IRC (Quit: Leaving)
10:22 🔗 ShellyRol has quit IRC (Read error: Connection reset by peer)
10:24 🔗 ShellyRol has joined #archiveteam-bs
11:54 🔗 PlsHelp has joined #archiveteam-bs
11:56 🔗 PlsHelp Also, regarding xbooru.com purging photos: comments at Xbooru is exclusive content so that's another reason to archive it.
11:56 🔗 PlsHelp Please read http://blog.booru.org/?p=209
12:12 🔗 mtntmnky has quit IRC (Remote host closed the connection)
12:13 🔗 mtntmnky has joined #archiveteam-bs
12:26 🔗 PlsHelp FOCUS: https://closeup.booru.org/index.php?page=post&s=list&pid=26440
12:27 🔗 PlsHelp FOCUS: https://cheesecake.booru.org/index.php?page=post&s=list
12:34 🔗 godane latest digitize tapes : https://www.patreon.com/posts/digitize-tapes-34874417
12:35 🔗 PlsHelp FOCUS: https://vulva.booru.org/index.php?page=post&s=list&tags=all - these websites which are part of the booru.org network will be PURGED soon!
12:38 🔗 PlsHelp https://joy2.booru.org/index.php?page=post&s=list&tags=all
12:49 🔗 PlsHelp https://celeb-fake-nude.booru.org/index.php?page=post&s=list&pid=5940
12:50 🔗 BlueMax has quit IRC (Read error: Connection reset by peer)
13:11 🔗 zhongfu has quit IRC (Ping timeout: 745 seconds)
13:12 🔗 zhongfu has joined #archiveteam-bs
13:16 🔗 PlsHelp has quit IRC (Quit: Page closed)
15:49 🔗 MaximeleG has joined #archiveteam-bs
16:10 🔗 godane SketchCow: so i found this looking for pdfs on twitter : https://twitter.com/rohweroutpost
16:11 🔗 godane its a magazine/newspaper from Rohwer Arkansas Japanese Relocation Camp
17:12 🔗 JAA All of the LoL forums except the NA ones are done (save for errors to be handled soon). The smaller boards (eune_ro, eune_hu, eune_cs, eune_el, euw_de, jp_ja-jp, pbe_en, oce_en) are also done. The other boards are still running.
17:13 🔗 JAA On the forums, I'm saving the thread pages and the forum pagination. I'm bruteforcing thread IDs up to {'na': 4924000, 'euw': 2106000, 'eune': 807000, 'ru': 30000, 'br': 382000, 'lan': 96000, 'las': 183000, 'oce': 89000, 'tr': 137000}
17:14 🔗 JAA On the boards, I'm saving the homepage pagination, discussions including pagination in flat view, and user profiles.
17:14 🔗 JAA As mentioned before, no images, outlinks, etc.
17:30 🔗 Smiley has joined #archiveteam-bs
17:33 🔗 SmileyG has quit IRC (Read error: Operation timed out)
18:41 🔗 idk has joined #archiveteam-bs
18:42 🔗 Ryz Hey JAA, after this, I assume it would be ideal to run the non-targeted images, outlinks, etc in AB after the League of Legends forums grab is finished?
18:45 🔗 idk has quit IRC (Ping timeout: 260 seconds)
18:47 🔗 JAA Ryz: If anyone wants to extract that information from the WARCs, sure. I'm not planning to do so.
19:02 🔗 arkiver_ has quit IRC (Read error: Connection reset by peer)
19:02 🔗 arkiver_ has joined #archiveteam-bs
19:06 🔗 DogsRNice has joined #archiveteam-bs
19:07 🔗 MaximeleG has quit IRC (Ping timeout: 745 seconds)
19:30 🔗 DogsRNice has quit IRC (Read error: Connection reset by peer)
19:37 🔗 SynMonger has quit IRC (Ping timeout: 276 seconds)
19:40 🔗 SynMonger has joined #archiveteam-bs
19:50 🔗 Craigle has quit IRC (Ping timeout: 276 seconds)
19:50 🔗 anarcat has quit IRC (Ping timeout: 276 seconds)
19:50 🔗 benjins has quit IRC (Ping timeout: 276 seconds)
19:50 🔗 fredgido has quit IRC (Ping timeout: 276 seconds)
19:50 🔗 pew has quit IRC (Ping timeout: 276 seconds)
19:50 🔗 klg has quit IRC (Ping timeout: 276 seconds)
19:50 🔗 logchfoo2 has quit IRC (Ping timeout: 276 seconds)
19:51 🔗 logchfoo3 starts logging #archiveteam-bs at Sat Mar 14 19:51:55 2020
19:51 🔗 logchfoo3 has joined #archiveteam-bs
19:52 🔗 logchfoo2 has quit IRC (Ping timeout: 276 seconds)
19:52 🔗 Dallas has quit IRC (Ping timeout: 276 seconds)
19:52 🔗 Hooloovoo has quit IRC (Ping timeout: 276 seconds)
19:52 🔗 arkiver_ has quit IRC (Ping timeout: 276 seconds)
19:52 🔗 dxrt has quit IRC (Ping timeout: 276 seconds)
19:52 🔗 Fionera has quit IRC (Ping timeout: 276 seconds)
19:53 🔗 pew has joined #archiveteam-bs
19:54 🔗 godane has quit IRC (Ping timeout: 276 seconds)
19:54 🔗 coderobe has quit IRC (Ping timeout: 276 seconds)
19:56 🔗 synm0nger has joined #archiveteam-bs
19:58 🔗 Xibalba has quit IRC (Ping timeout: 276 seconds)
19:59 🔗 Xibalba has joined #archiveteam-bs
20:01 🔗 SynMonger has quit IRC (Read error: Operation timed out)
20:03 🔗 OrIdow6 has quit IRC (Ping timeout: 276 seconds)
20:03 🔗 VoynichCr has quit IRC (Ping timeout: 276 seconds)
20:05 🔗 mtntmnky_ has joined #archiveteam-bs
20:05 🔗 is-_ has joined #archiveteam-bs
20:06 🔗 pew has quit IRC (se.hub irc.underworld.no)
20:06 🔗 jmtd has quit IRC (se.hub irc.underworld.no)
20:06 🔗 is- has quit IRC (se.hub irc.underworld.no)
20:06 🔗 Frogging has quit IRC (se.hub irc.underworld.no)
20:06 🔗 purplebot has quit IRC (se.hub irc.underworld.no)
20:07 🔗 Jon| has joined #archiveteam-bs
20:21 🔗 mtntmnky has quit IRC (Remote host closed the connection)
20:23 🔗 dxrt has joined #archiveteam-bs
20:24 🔗 svchfoo1 sets mode: +o dxrt
20:26 🔗 klg has joined #archiveteam-bs
20:28 🔗 VoynichCr has joined #archiveteam-bs
20:30 🔗 Samizdat has joined #archiveteam-bs
20:41 🔗 pew has joined #archiveteam-bs
21:00 🔗 Samizdat has quit IRC (Read error: Connection reset by peer)
21:32 🔗 opticnerv has joined #archiveteam-bs
21:50 🔗 ephemer0l has quit IRC (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.)
21:54 🔗 ephemer0l has joined #archiveteam-bs
22:07 🔗 Smiley has quit IRC (Read error: Connection reset by peer)
22:07 🔗 Smiley has joined #archiveteam-bs
22:07 🔗 antomati_ has joined #archiveteam-bs
22:09 🔗 antomatic has quit IRC (Read error: Operation timed out)
22:15 🔗 HP_Archiv has joined #archiveteam-bs
22:28 🔗 OrIdow6 has joined #archiveteam-bs
22:39 🔗 Craigle has quit IRC (Quit: The Lounge - https://thelounge.chat)
22:48 🔗 opticnerv has quit IRC (Quit: Leaving)
22:50 🔗 Craigle has joined #archiveteam-bs
23:13 🔗 BlueMax has joined #archiveteam-bs

irclogger-viewer