[00:21] *** bsmith093 has joined #archiveteam [00:22] iv'e never been very clear on this. if IA gets hit with a dmca for something, do they delete it or just dark it for a few decades? [00:23] SketchCow, yipdw [00:24] *** c_b2 has joined #archiveteam [00:27] *** c_b has quit IRC (Read error: Operation timed out) [00:27] *** c_b2 is now known as c_b [00:31] bsmith093: dark it indefinitely, afaik [00:32] *** mistym has quit IRC (Remote host closed the connection) [00:34] Based on some of the interviews, I get the impression that it might be available if you show up at the physical site. [00:36] *** godane has quit IRC (Quit: Leaving.) [00:37] *** xmc has joined #archiveteam [00:51] *** godane has joined #archiveteam [01:08] *** godane has quit IRC (Ping timeout: 370 seconds) [01:21] *** philpem has quit IRC (Ping timeout: 260 seconds) [02:03] *** godane has joined #archiveteam [02:10] *** ohhdemgir has quit IRC (Read error: Operation timed out) [02:11] *** ohhdemgir has joined #archiveteam [02:18] *** nertzy has quit IRC (Leaving) [02:23] *** nertzy has joined #archiveteam [02:35] *** c_b has quit IRC (Quit: c_b) [02:35] *** Ymgve has quit IRC () [02:40] *** xmc has quit IRC (Quit: [X]) [02:42] *** xmc has joined #archiveteam [02:51] *** primus104 has quit IRC (Leaving.) [02:52] *** schbirid has quit IRC (Read error: Operation timed out) [02:58] *** xmc has quit IRC (Quit: [X]) [03:02] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [03:02] *** schbirid has joined #archiveteam [03:30] *** xmc has joined #archiveteam [04:08] *** aaaaaaaaa has quit IRC (Leaving) [04:12] *** balrog has quit IRC (Ping timeout: 512 seconds) [04:44] *** jmathai has joined #archiveteam [04:45] Anyone around who can throttle one of the warriors? [04:50] If someone comes by that can…could you pause the warriors that are downloading from Trovebox? Our load balancers are returning a 500 “over capacity” error. [04:55] it was already paused for a while. the scripts went awry on some items [04:55] there isn't a remote kill switch unfortunately [04:57] "The warriors...can be programmed for exponential growth." [04:58] chfoo: so are they currently crawling the site? [04:59] A couple days ago Arkiver2 had cut them to 1/2 capacity for us [04:59] the tracker limit is to 0. nothing is supposed to be crawling right now [04:59] but we think some scripts are just stuck in a loop [05:00] until the process is oom killed or the warrior is rebooted [05:07] *** mistym has joined #archiveteam [05:09] chfoo: okay, thanks. [05:09] Hopefully tomorrow I can let Arkiver2 know to resume at a lower load. [05:11] jmathai: if the load keep going up, perhaps any request with "ArchiveTeam" user-agent should be served with 429 to get the scripts to quit. we can use a different user-agent knowing that you know it is not malicious [05:12] chfoo: thanks. good tip. [05:13] it feels weird suggesting a user-agent ban on archiveteam [05:14] :/ :\ :/ :\ [05:22] *** jmathai has quit IRC (Read error: Connection reset by peer) [05:41] *** BlueMaxim has quit IRC (Quit: Leaving) [05:43] *** balrog_ has joined #archiveteam [05:48] sad trombone [05:49] does the useragent have the script version in it? [05:49] like ArchiveTeam-Warrior-STIENAME-DATESTAMP_OF_CODE [05:56] All of the scripts are open and available at https://github.com/ArchiveTeam/ [05:56] so have a dig [06:01] k [06:03] *** balrog_ is now known as balrog [06:03] *** SketchCow sets mode: +oooo antomatic balrog bsmith093 Cameron_D [06:03] *** SketchCow sets mode: +oooo closure Coderjoe dcmorton DFJustin [06:03] *** SketchCow sets mode: +oo chfoo xmc [06:03] looks like this uses default wpull UA https://github.com/ArchiveTeam/example-wpull-seesaw-project/blob/master/pipeline.py [06:13] *** rejon has quit IRC (Read error: Operation timed out) [06:16] Ctrl-S: in general, no [06:16] well [06:17] a lot of the projects either imitate a browser user-agent or cycle through a list. some of them set user-agent to just "ArchiveTeam" [06:18] the pipeline version is often found in the first WARC record [06:19] typically the [PROJECT]-dld-script-version header [06:33] *** BlueMaxim has joined #archiveteam [07:08] *** mistym has quit IRC (Remote host closed the connection) [07:52] *** JMC has joined #archiveteam [07:53] *** BlueMaxim has quit IRC (Ping timeout: 512 seconds) [07:53] hello? [07:53] *** BlueMaxim has joined #archiveteam [08:01] *** spacetom has joined #archiveteam [08:02] *** spacetom has left [08:02] Hi?? [08:17] ??? [08:24] ah, there we go [08:24] so, weird question [08:25] can I link to 4chan? It's not a "oh lol" kinf od stuff, but rather, I made a thread there. [08:25] I can explain here... it will just take, you know, longer [08:26] you can link to 4chan, but I can't open it for some hours as I at work [08:26] *I'm [08:27] what's the thread about? [08:27] fair enough [08:27] So, there used to be this things back in the early internet days called DeskMates. Some were dull, some were kind of hawt. They were done by a company named oska. [08:28] http://i.4cdn.org/g/1426837548560s.jpg <- you know, the ones with this chick (SFW) [08:28] Problem is, oska is gone and idiots had their demos downloading directly from their servers (which are now gone) so its impossible to download the demos, and that's what you need so a popular torrent that's still hanging around works. (won't link to the torrent because that's dum) [08:28] Problem is, without the demos the torrent doesn't work. [08:28] So, does anybody still has the original demo files somewhere, or knows where to get it? I know its a long shot, but it doesn't hurt to ask. [08:29] The main problem is that there's a certain file called "Manager.exe" that apparently ran everything [08:29] even if you get all the files, and the only working demo out there (Oska 2.0) the thing doesn't work. [08:30] so, in my head, a solution might be to get ALL the files from the demo AND THEN go from there [08:30] but, you know, this stuff is quite old (90s old) so... [08:30] the thread pointed me here, actually [08:36] I know its an INSANE longshot (and for equally insanely obscure stuff) but hey, why not? :P [08:41] *** Emcy_ has joined #archiveteam [08:43] *** Start has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** Emcy has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** xtr-201 has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** Froggypwn has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:43] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [08:44] yikes. Netsplit? [08:44] uhu [08:46] *** primus104 has joined #archiveteam [08:49] anybody has any idea what I'm talking about, btw? [09:02] *** Start has joined #archiveteam [09:02] *** SadDM has joined #archiveteam [09:02] *** Froggypwn has joined #archiveteam [09:02] *** useretail has joined #archiveteam [09:02] *** matthusby has joined #archiveteam [09:02] *** Jonimus has joined #archiveteam [09:02] *** will has joined #archiveteam [09:03] wb [09:26] *** primus104 has quit IRC (Leaving.) [10:16] *** schbirid has quit IRC (Read error: Operation timed out) [10:19] *** schbirid has joined #archiveteam [10:50] *** Ymgve has joined #archiveteam [10:50] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [10:51] *** dashcloud has quit IRC (Read error: Operation timed out) [10:54] *** dashcloud has joined #archiveteam [11:09] *** Morbus has quit IRC (Quit: http://www.disobey.com/) [11:14] *** Morbus has joined #archiveteam [11:42] JMC: there's no problem with linking to torrents here, or anything really :P [11:44] (in fact, you should probably make sure that the contents of the torrent are archived - torrents are not good archival media) [13:06] its a real bitch youtube keeps blocking ovh downloading of videos [13:06] want to keep everything on my box and now its a pain constantly [13:14] really, do they? :/ [13:15] ya i get a captcha like all the time and never used to [13:16] see the problem on youtube-dl github issues page also [13:54] *** scyther has joined #archiveteam [14:04] *** VADemon has joined #archiveteam [14:15] *** johtso has joined #archiveteam [14:23] *** hive-mind has quit IRC (Ping timeout: 260 seconds) [14:35] *** mistym has joined #archiveteam [15:17] *** hive-mind has joined #archiveteam [15:40] op please. [15:41] *** signius__ has quit IRC (Read error: Operation timed out) [15:43] http://www.csmonitor.com/Books/chapter-and-verse/2015/0320/Vanuatu-asks-the-world-please-send-books [15:44] *** mistym has quit IRC (Remote host closed the connection) [15:50] *** signius__ has joined #archiveteam [15:57] *** mistym has joined #archiveteam [16:04] *** primus104 has joined #archiveteam [16:23] joepie91_: here's what's going on in a nutshell [16:23] http://boards.4chan.org/g/thread/47104359 [16:30] more specific info in the torrent right here: [16:30] https://thepiratebay.se/torrent/3430667/ [16:34] *** aaaaaaaaa has joined #archiveteam [17:19] *** mistym_ has joined #archiveteam [17:23] *** Arkiver2 is now known as arkiver [17:24] *** mistym has quit IRC (Read error: Operation timed out) [17:37] *** lag has joined #archiveteam [17:57] *** habi has joined #archiveteam [18:02] *** habi has left [18:13] arkiver: could you start on last.fm after we begin rapidshare discovery? [18:13] SketchCow mentioned that they'll be switching codebases in april, and that some user generated material may die [18:18] *** Stiletto has joined #archiveteam [18:25] last.fm going away?? O.o [18:25] ah ok [18:25] I'll get last.fm going [18:27] *** nertzy has joined #archiveteam [18:31] *** Ravenloft has joined #archiveteam [18:33] whoa, what? [18:34] oh [18:39] *** russss has quit IRC (Ping timeout: 260 seconds) [18:39] *** fresco___ has quit IRC (Ping timeout: 260 seconds) [19:03] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [19:15] *** VADemon has quit IRC (Quit: left4dead) [19:17] *** VADemon has joined #archiveteam [19:29] *** lytv has quit IRC (Quit: Leaving) [19:39] *** lytv has joined #archiveteam [19:42] *** chfoo sets mode: +o swebb [19:42] *** swebb sets mode: +o SadDM [19:43] *** svchfoo1 has joined #archiveteam [19:43] *** chfoo sets mode: +o svchfoo1 [19:45] *** russss has joined #archiveteam [19:45] *** fresco___ has joined #archiveteam [19:47] *** SN4T14__ has joined #archiveteam [19:48] *** godane has quit IRC (Ping timeout: 512 seconds) [19:54] *** SN4T14_ has quit IRC (Ping timeout: 512 seconds) [20:02] *** jmathai has joined #archiveteam [20:02] *** nertzy has joined #archiveteam [20:02] *** jmathai has quit IRC (Connection closed) [20:15] *** sandrine_ has joined #archiveteam [20:17] *** sandrine_ has quit IRC (Client Quit) [20:18] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [20:29] *** godane has joined #archiveteam [20:35] *** mistym_ has quit IRC (Remote host closed the connection) [20:39] *** svchfoo2 has joined #archiveteam [20:39] *** chfoo sets mode: +o svchfoo2 [20:43] *** schbirid has quit IRC (Leaving) [20:46] *** Peetz0r_ is now known as Peetz0r [21:00] *** BlueMaxim has joined #archiveteam [21:06] *** johtso has quit IRC (Quit: Connection closed for inactivity) [21:19] *** mistym has joined #archiveteam [21:33] *** Start has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:33] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:33] *** Froggypwn has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:33] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:33] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:33] *** Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:33] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:35] *** Start has joined #archiveteam [21:35] *** SadDM has joined #archiveteam [21:35] *** Froggypwn has joined #archiveteam [21:35] *** useretail has joined #archiveteam [21:35] *** matthusby has joined #archiveteam [21:35] *** Jonimus has joined #archiveteam [21:35] *** will has joined #archiveteam [21:35] *** irc.shaw.ca sets mode: +o SadDM [21:35] *** swebb sets mode: +o SadDM [21:36] *** scyther has quit IRC (Leaving) [21:40] *** Start has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:40] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:40] *** Froggypwn has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:40] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:40] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:40] *** Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:40] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [21:47] *** MorbusIff has joined #archiveteam [21:50] *** RKenshin has joined #archiveteam [21:50] *** Morbus has quit IRC (hub.efnet.us irc.Prison.NET) [21:50] *** bsmith093 has quit IRC (hub.efnet.us irc.Prison.NET) [21:50] *** Zebranky has quit IRC (hub.efnet.us irc.Prison.NET) [21:50] *** Kenshin has quit IRC (hub.efnet.us irc.Prison.NET) [21:50] *** patrickod has quit IRC (hub.efnet.us irc.Prison.NET) [21:50] *** oli has quit IRC (hub.efnet.us irc.Prison.NET) [21:51] *** oli_ has joined #archiveteam [21:52] *** ben__ has joined #archiveteam [21:52] *** Start has joined #archiveteam [21:52] *** SadDM has joined #archiveteam [21:52] *** Froggypwn has joined #archiveteam [21:52] *** useretail has joined #archiveteam [21:52] *** matthusby has joined #archiveteam [21:52] *** Jonimus has joined #archiveteam [21:52] *** will has joined #archiveteam [21:52] *** irc.shaw.ca sets mode: +o SadDM [21:52] *** swebb sets mode: +o SadDM [21:54] *** Zebranky_ has joined #archiveteam [22:00] *** patricko- has joined #archiveteam [22:05] *** oli_ is now known as oli [22:06] *** RKenshin is now known as Kenshin [22:18] *** kyan has joined #archiveteam [22:38] are there plans to archive the japanese geocities? [22:41] not that i'm aware of, although we probably should start soon [22:41] i'm somewhat amazed that yahoo hasn't killed it yet [22:42] as a side note, does anybody know of a browser extension that helps see if a site is on archive.org? [22:43] especially for 404s, parked domains, or just pure dead sites [22:55] *** patricko- is now known as patrickod [22:58] *** patrickod is now known as patricko- [22:58] *** patricko- is now known as patrickod [23:01] yahoo japan is run pretty independently I think [23:04] *** patrickod is now known as patricko- [23:07] you know what I've also been wanting for a long time? an extension that would warc every single page I visit [23:08] https://webrecorder.io/ is close [23:12] yeah, but it's manual [23:12] and also a proxy [23:14] if you stretched out an average persons skin over a football field, you would be arrested and no one would like you or trust you anymore [23:34] *** patricko- is now known as patrickod [23:40] *** primus_ has quit IRC (Quit: Leaving) [23:52] *** lag has quit IRC (Quit: Leaving)