[00:08] *** oldcad has quit IRC (Quit: Leaving.) [00:17] *** Aranje has quit IRC (Read error: Connection reset by peer) [00:17] *** Aranje has joined #archiveteam [00:28] *** mistym has quit IRC (Remote host closed the connection) [00:35] *** Aranje has quit IRC (Ping timeout: 240 seconds) [00:41] *** BlueMaxim has joined #archiveteam [00:42] *** mistym has joined #archiveteam [00:44] *** kyan has quit IRC (Quit: This computer has gone to sleep) [00:47] *** Aranje has joined #archiveteam [00:52] *** jmtd has quit IRC (Ping timeout: 240 seconds) [00:52] *** jmtd has joined #archiveteam [01:36] *** kyan has joined #archiveteam [01:58] *** mistym has quit IRC (Remote host closed the connection) [02:08] WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD [02:09] The secret word is "yahoosucks", m'lord [02:10] *** mistym has joined #archiveteam [02:10] Thanks. I just realized I could have searched the logs. Oh well [02:11] *** bzc6p_ has joined #archiveteam [02:11] *** swebb sets mode: +o bzc6p_ [02:14] *** bzc6p has quit IRC (Read error: Operation timed out) [02:31] *** khaoohs__ has joined #archiveteam [02:33] *** WinterFox has quit IRC (Ping timeout: 483 seconds) [02:33] *** xk_id has quit IRC (Remote host closed the connection) [02:34] *** khaoohs_ has quit IRC (Read error: Operation timed out) [02:35] *** Aranje has quit IRC (Read error: Connection reset by peer) [02:35] *** xk_id has joined #archiveteam [02:36] *** Aranje has joined #archiveteam [02:43] *** WinterFox has joined #archiveteam [02:49] *** primus104 has quit IRC (Leaving.) [02:50] *** Ymgve has quit IRC () [02:55] *** Ymgve has joined #archiveteam [02:58] *** mistym has quit IRC (Remote host closed the connection) [03:01] *** Ungstein has joined #archiveteam [03:03] *** Ungstein1 has quit IRC (Ping timeout: 265 seconds) [03:08] *** Asparagir has joined #archiveteam [03:09] *** robink has joined #archiveteam [03:15] *** xk_id has quit IRC (Remote host closed the connection) [03:23] *** mistym has joined #archiveteam [03:24] *** seanrife_ has joined #archiveteam [03:27] *** bzc6p_ is now known as bzc6p [03:29] *** Asparagir has quit IRC (Asparagir) [03:33] *** Aranje has quit IRC (Read error: Connection reset by peer) [03:34] *** Aranje has joined #archiveteam [03:34] *** Froggypwn has quit IRC (Write error: Connection reset by peer) [03:35] *** Aranje has quit IRC (Read error: Connection reset by peer) [03:35] *** Aranje has joined #archiveteam [03:36] *** Froggypwn has joined #archiveteam [03:37] *** Zero_Dogg has quit IRC (Ping timeout: 252 seconds) [03:37] *** Zero_Dogg has joined #archiveteam [03:38] *** Aranje has quit IRC (Read error: Connection reset by peer) [03:39] *** godane has quit IRC (Ping timeout: 306 seconds) [03:46] *** Aranje has joined #archiveteam [03:48] *** godane has joined #archiveteam [04:06] *** lukeman has quit IRC (Quit: ZNC - http://znc.in) [04:07] *** mistym has quit IRC (Remote host closed the connection) [04:13] *** lukeman has joined #archiveteam [04:22] *** seanrife_ has quit IRC (Quit: Page closed) [04:22] *** Asparagir has joined #archiveteam [04:36] *** bzc6p sets mode: +oooo chfoo godane garyrh Infreq [04:36] *** bzc6p sets mode: +oo Kazzy Kenshin [04:36] *** bzc6p sets mode: +ooo midas Start wp494 [04:37] *** aaaaaaaaa has quit IRC (Leaving) [04:39] *** mistym has joined #archiveteam [04:46] *** xk_id has joined #archiveteam [04:57] *** bzc6p has left [05:08] *** xk_id has quit IRC (Read error: Operation timed out) [05:09] *** Ravenloft has quit IRC (Ping timeout: 240 seconds) [05:28] *** xk_id has joined #archiveteam [05:39] *** Start_ has joined #archiveteam [05:39] *** Start has quit IRC (Read error: Connection reset by peer) [05:54] *** xk_id has quit IRC (Read error: Operation timed out) [05:55] *** bsmith093 has quit IRC (Read error: Operation timed out) [06:09] *** Aranje has quit IRC (Read error: Connection reset by peer) [06:11] *** Start has joined #archiveteam [06:12] *** Start_ has quit IRC (Read error: Connection reset by peer) [06:15] *** Aranje has joined #archiveteam [06:16] *** Aranje has quit IRC (Read error: Connection reset by peer) [06:18] *** habi has joined #archiveteam [06:19] *** habi has left [06:24] *** mistym has quit IRC (Remote host closed the connection) [07:10] *** Asparagir has quit IRC (Asparagir) [07:25] *** mistym has joined #archiveteam [07:25] *** aliz has joined #archiveteam [07:28] *** yipdw has quit IRC (Quit: No Ping reply in 180 seconds.) [07:30] *** yipdw has joined #archiveteam [07:34] *** mistym has quit IRC (Read error: Operation timed out) [07:34] *** primus104 has joined #archiveteam [07:40] *** schbirid has joined #archiveteam [07:44] *** WinterFox has quit IRC (Remote host closed the connection) [07:44] *** Froggypwn has quit IRC (Read error: Connection reset by peer) [07:44] *** WinterFox has joined #archiveteam [07:45] *** Froggypwn has joined #archiveteam [07:54] atomicgamer is shutting down [07:55] "July(Date TBD)" [07:55] this is an emergency, lots of old game files [07:55] i am trying to get in contact, should succeed unless things are burning [07:56] #atomicgamer [07:58] *** schbirid is now known as schbiridw [07:58] *** schbiridw is now known as schbiwork [07:59] they might be cooperative, in the past they did not mind me archiving parts, now they might appreciate it [07:59] off to work, more later [08:06] *** toad2 has joined #archiveteam [08:08] *** toad1 has quit IRC (Read error: Operation timed out) [08:29] *** primus104 has quit IRC (Leaving.) [08:42] *** atomotic has joined #archiveteam [08:43] *** xk_id has joined #archiveteam [08:44] *** MMovie has joined #archiveteam [08:45] *** MMovie2 has quit IRC (Read error: Operation timed out) [08:49] *** habi has joined #archiveteam [08:54] *** habi has left [09:01] *** schbird has joined #archiveteam [09:08] *** PepsiMax has quit IRC (Ping timeout: 252 seconds) [09:19] *** jmc_ has quit IRC () [09:22] *** xk_id has quit IRC (Read error: Operation timed out) [09:41] *** dx has quit IRC (Read error: Operation timed out) [09:42] *** primus104 has joined #archiveteam [09:52] *** PepsiMax has joined #archiveteam [09:54] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [10:09] *** dx has joined #archiveteam [10:09] *** db48x has quit IRC (Read error: Connection reset by peer) [10:12] *** godane has quit IRC (Ping timeout: 265 seconds) [10:25] *** godane has joined #archiveteam [10:57] *** godane has quit IRC (Quit: Leaving.) [11:06] *** godane has joined #archiveteam [11:16] *** vOYtEC has quit IRC (Ping timeout: 369 seconds) [11:23] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [11:24] *** xk_id has joined #archiveteam [11:30] *** xk_id has quit IRC (Remote host closed the connection) [11:35] *** matthusby has quit IRC (ZNC - http://znc.in) [11:38] *** xf2e has quit IRC (Ping timeout: 483 seconds) [11:48] *** vOYtEC has joined #archiveteam [11:57] *** ohhdemgir has quit IRC (Read error: Connection reset by peer) [12:00] *** balrog has quit IRC (Read error: Operation timed out) [12:01] *** ripvanwin has quit IRC (Read error: Operation timed out) [12:02] *** ohhdemgir has joined #archiveteam [12:02] *** atomotic has joined #archiveteam [12:05] *** dashcloud has quit IRC (Read error: Operation timed out) [12:08] *** balrog has joined #archiveteam [12:08] *** swebb sets mode: +o balrog [12:11] *** dashcloud has joined #archiveteam [12:30] *** xk_id has joined #archiveteam [12:46] *** Stilett0 has joined #archiveteam [12:48] *** xk_id has quit IRC (Read error: Operation timed out) [12:50] *** Stiletto has quit IRC (Ping timeout: 370 seconds) [13:11] *** primus104 has quit IRC (Leaving.) [13:19] *** khaoohs__ has quit IRC (Read error: Connection reset by peer) [13:38] *** xk_id has joined #archiveteam [13:43] *** K4k has joined #archiveteam [13:43] *** Panasonic has joined #archiveteam [13:43] *** Panasonic is now known as Ravenloft [13:45] *** MMovie2 has joined #archiveteam [13:46] *** MMovie has quit IRC (Read error: Operation timed out) [13:48] *** xk_id has quit IRC (Read error: Operation timed out) [13:48] *** WinterFox has quit IRC (Remote host closed the connection) [13:51] *** K4k has quit IRC (Ping timeout: 240 seconds) [14:13] *** xk_id has joined #archiveteam [14:20] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [14:30] *** mistym has joined #archiveteam [14:32] *** khaoohs has joined #archiveteam [14:36] *** K4k has joined #archiveteam [14:39] *** K4k_ has joined #archiveteam [14:40] *** mistym has quit IRC (Remote host closed the connection) [14:42] *** K4k has quit IRC (Ping timeout: 369 seconds) [15:00] chfoo: can you please add 'atomicgamer' to projects.json? and add a FOS rsync? [15:00] We're moving fast on that site [15:00] shutting download this month [15:00] files go back 10+ years [15:05] *** schbird has quit IRC (Quit: Leaving) [15:07] *** mistym has joined #archiveteam [15:55] *** Emcy has quit IRC (Ping timeout: 265 seconds) [15:58] *** T31M has joined #archiveteam [16:01] *** MMovie2 has quit IRC (Read error: Operation timed out) [16:01] *** MMovie has joined #archiveteam [16:03] *** K4k_ has quit IRC (Ping timeout: 370 seconds) [16:08] *** primus104 has joined #archiveteam [16:09] *** atomotic has joined #archiveteam [16:29] *** aliz has quit IRC (Remote host closed the connection) [16:31] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [16:43] *** goekesmi_ has quit IRC (Remote host closed the connection) [16:44] *** K4k_ has joined #archiveteam [16:45] *** Ravenloft has quit IRC (Ping timeout: 606 seconds) [16:50] *** mistym has quit IRC (Remote host closed the connection) [16:55] *** goekesmi has joined #archiveteam [17:05] *** mistym has joined #archiveteam [17:06] *** Emcy has joined #archiveteam [17:08] i (and others) am in slow contact with atomicgamer. generally they are supportive. [17:08] everything (including all the hosted sites) is ~8.5TB. where could we stuff that? [17:09] in the worst case i would just rent a server for it but if we have the capacity it would help [17:09] *** primus104 has quit IRC (Leaving.) [17:10] *** habi has joined #archiveteam [17:15] *** habi has left [17:16] /dev/sd3a 3.6T 514G 2.9T 15% /var/www/htdocs/mirror [17:16] 1 Gbps connection [17:16] Bad news is it's OpenBSD and doesn't support Warrior [17:16] But I assume you're talking about the staging server [17:25] *** Jogie has quit IRC (Ping timeout: 492 seconds) [17:25] *** Jogie has joined #archiveteam [17:31] WubTheCap: where does it fail? [17:33] *** Emcy has quit IRC (Ping timeout: 483 seconds) [17:34] balrog: I believe the dependency on bash and other things. Not directly related to Warrior (since that's an ISO to boot), but the wget grab scripts. [17:35] WubTheCap: get bash and use #!/bin/bash? [17:35] balrog: Not so easy, also depends on GNU Make (which I can install as gmake and patch in source), but even so then the dependencies for wget lua fail [17:36] pretty sure I've build wget-lua on OS X [17:39] *** dashcloud has quit IRC (Read error: Operation timed out) [17:44] *** aaaaaaaaa has joined #archiveteam [17:44] *** swebb sets mode: +o aaaaaaaaa [17:45] *** dashcloud has joined #archiveteam [17:56] *** whattheha has joined #archiveteam [17:56] *** K4k_ has quit IRC (Ping timeout: 252 seconds) [17:56] WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD! [17:56] yahoosucks [17:57] what is your quest [17:57] to find great treasure [18:00] Good answer. [18:05] *** ripvanwin has joined #archiveteam [18:08] *** Elegance_ has quit IRC (Ping timeout: 252 seconds) [18:08] *** Elegance has joined #archiveteam [18:09] *** MMovie has quit IRC (Ping timeout: 306 seconds) [18:12] *** MMovie has joined #archiveteam [18:13] *** K4k_ has joined #archiveteam [18:13] *** whattheha has quit IRC (Quit: leaving) [18:18] *** WubTheCap has quit IRC (Quit: Leaving) [18:20] *** K4k__ has joined #archiveteam [18:21] *** K4k_ has quit IRC (Read error: Operation timed out) [18:26] *** K4k__ has quit IRC (Quit: WeeChat 1.2) [18:28] *** K4k has joined #archiveteam [18:29] *** dashcloud has quit IRC (Read error: Operation timed out) [18:33] arkiver: ok, added [18:35] *** dashcloud has joined #archiveteam [18:38] *** gwern has joined #archiveteam [18:39] is this an ok place to ask about Internet Archive torrent problems? (there seems to be no official IA channel) [18:39] there are people here who might be able to help you [18:42] I see. well, my issue is that I finally finished uploading https://archive.org/details/dnmarchives last night and the current torrent file https://archive.org/download/dnmarchives/dnmarchives_archive.torrent is half the size which it should be, and is missing files like 'outlawmarket.tar.xz' (missing even in the raw .torrent when I look through it) which are definitely listed in... [18:42] ...https://archive.org/download/dnmarchives and https://archive.org/download/dnmarchives/outlawmarket.tar.xz downloads fine when I try a non-torrent download. I looked at the IA faq on torrents but nothing mentions an issue like this where files in the full archive simply aren't included in the torrent [18:43] (when I was uploading, the torrent seemed to regularly update with each new file so now that I finished uploading with 'ia' last night, I don't know what more I can do. there are no options I can see to 'force' a recreation of the .torrent file to try and hope that would fix the issue) [18:43] i just searched 20 seconds for the edit button, god i LOVE the new design [18:43] *** dashcloud has quit IRC (Read error: Operation timed out) [18:44] try https://en.wikisource.org/wiki/Help:Internet_Archive#Deriving [18:44] schbiwork: you mean https://archive.org/edit/dnmarchives ? I looked at that but I didn't see anythign that struck me as relevant to the problem [18:45] go to files, click update at the bottom (i think) [18:45] *** Emcy has joined #archiveteam [18:45] hm [18:45] if you change any of the metadata even a little bit, it should queue a new derive [18:45] hm. I only ever looked at the metadata editor [18:45] but you might be able to make that happen without it [18:47] * xmc hits rederive button [18:47] nope i can't [18:47] you should go to [18:47] https://archive.org/item-mgr.php?identifier=dnmarchives [18:47] and hit the "derive" button (first button down, right side) [18:48] I just tried adding some per-file metadata [18:49] wait, I just found something [18:49] 'SKIPPING UPDATE to dnmarchives_archive.torrent IN /12/items/dnmarchives... item (49116 MB) exceeds maximum size (25600 MB)' https://catalogd.archive.org/log/418144456 [18:49] it didn't queue a derive https://archive.org/history/dnmarchives [18:49] that'll do it [18:49] whaaaa [18:49] ahhhh [18:49] email info@archive.org, they can change things [18:49] it's a sanity limit put in a while ago [18:50] @_@ the faq could've fucking mentioned that [18:50] good point [18:50] or it could've threw me a warning [18:50] *** K4k has quit IRC (Ping timeout: 186 seconds) [18:50] the systems are so decoupled from each other ... [18:50] 'hey there you uploaded all those gigabytes of stuff and you probably would like to download them at some point! but you can't because of this limit. you should do something about that.' [18:51] maaaaybe [18:51] archive.org is a lot of pieces that somehow magically work :) [18:52] you can always download through https [18:52] DFJustin: yeah, but I was trying to be *nice* to the IA by telling everyone to use the torrent [18:52] another 17.5 hours chaps [18:59] *** dashcloud has joined #archiveteam [19:01] xmc: ok, I emailed info@ explaining the problem and asking for the limit to be removed [19:02] *** primus104 has joined #archiveteam [19:02] gwern: that is a great archive btw [19:03] thanks. it was a ridiculous amount of work. scraping is hard enough on its own, but getting data from the researchers has been like pulling teeth [19:03] some of them just lie to you [19:03] :( [19:03] 'oh, I signed an NDA for that dataset' [19:04] (no, you lying asshole, you fucked up the data completely and your paper is 100% wrong, every last bit of it, and now you're trying to cover it up) [19:06] * gwern is still quite bitter about that one [19:18] *** aaaaaaaaa has quit IRC (Leaving) [19:22] *** Emcy has quit IRC (Ping timeout: 483 seconds) [19:25] *** Emcy has joined #archiveteam [19:33] *** aaaaaaaaa has joined #archiveteam [19:33] *** swebb sets mode: +o aaaaaaaaa [19:40] omg people linking https://en.wikisource.org/wiki/Help:Internet_Archive#Deriving as the bible [19:41] And I never revised that page. Whatever [19:41] gwern: come on, it's not so unconceivable that a 50 GB item may hit some restrictions, even though we're used to abuse of IA [19:42] Usually we just ping SketchCow with a "please force torrent for https://archive.org/details/dnmarchives thank you so much" [19:42] Nemo_bis: I always heard of you archive team guys slinging around terabytes of stuff and the ia being cool with that; and anyway, I am surprised to run into a limit which is not documented in the interface or the faq all about torrent problems [19:43] ok [19:43] yes, I might find it a little unusual that iA lets met upload 52gb but a) I checked the docs in the first place and b) this sort of thing is exactly why I donate to the IA and chose it for the upload [19:43] sure [19:45] * Nemo_bis wonders how to fix. If people use Wikisource docs for IA... of course the docs can't talk of 25+ GB items [19:45] Nemo_bis: early hit for "archive.org force derive" ;) [19:54] *** aaaaaaaaa has quit IRC (Leaving) [19:58] gwern: ok I'm sprinkling some knowledge on the forums now [20:15] Fwaro [20:27] *** mistym has quit IRC (Remote host closed the connection) [20:35] *** dashcloud has quit IRC (Read error: Operation timed out) [20:38] *** dashcloud has joined #archiveteam [20:40] *** mistym has joined #archiveteam [20:56] *** Froggypwn has quit IRC (Read error: Connection reset by peer) [20:57] *** Froggypwn has joined #archiveteam [20:58] http://www.digitaltrends.com/gaming/playstation-mobile-shuts-down-tomorrow/ [21:00] *** mistym has quit IRC (Remote host closed the connection) [21:16] *** mistym has joined #archiveteam [21:54] *** RichardG has quit IRC (Remote host closed the connection) [21:58] *** RichardG has joined #archiveteam [21:59] alright, I think the torrent has been fixed. thanks for the advice, I'm not sure it would've occurred to me to check the logs... [21:59] *** gwern has left [22:01] uh oh https://www.twitter.com/robots.txt [22:02] ? [22:04] *** kyan has quit IRC (Quit: This computer has gone to sleep) [22:04] oh nice [22:28] *** aaaaaaaaa has joined #archiveteam [22:28] *** swebb sets mode: +o aaaaaaaaa [22:40] *** xf2e has joined #archiveteam [22:41] *** dashcloud has quit IRC (Remote host closed the connection) [22:43] *** dashcloud has joined #archiveteam [22:53] *** dashcloud has quit IRC (Read error: Operation timed out) [22:58] *** dashcloud has joined #archiveteam [23:25] *** Asparagir has joined #archiveteam [23:26] *** w0rp has quit IRC (Read error: Operation timed out) [23:37] twitter's robots.txt seems to be back to normal [23:39] *** WinterFox has joined #archiveteam [23:45] *** Jonimus has quit IRC (Ping timeout: 370 seconds) [23:47] http://www.gwern.net/Black-market%20archives [23:56] *** Jonimus has joined #archiveteam