[00:03] *** godane has joined #archiveteam [00:04] *** godane has quit IRC (Client Quit) [00:06] *** godane has joined #archiveteam [00:12] *** MMovie has quit IRC (Read error: Operation timed out) [00:15] *** MMovie has joined #archiveteam [00:17] Yeah, grab it [00:17] Awesome, we'll get it [00:18] Yay! [00:27] This is one of the fotolog profiles we saved https://web.archive.org/web/20160208221850/http://www.fotolog.com/fotoescritura/ [00:27] Looks awesome [00:28] Also, fotolog should have shut down 9 days ago, but is still running. Maybe they saw the archiving efforts and decided to wait :) [00:34] Thats good [00:46] *** MMovie has quit IRC (Read error: Operation timed out) [00:47] *** MMovie has joined #archiveteam [01:04] *** MMovie has quit IRC (Read error: Operation timed out) [01:05] *** MMovie has joined #archiveteam [01:21] *** MMovie has quit IRC (Read error: Operation timed out) [01:22] *** MMovie has joined #archiveteam [01:37] *** MMovie has quit IRC (Read error: Operation timed out) [01:38] *** Froggypwn has quit IRC (Quit: ~ Trillian Astra - www.trillian.im ~) [01:38] *** ndiddy has quit IRC (Read error: Operation timed out) [01:39] *** MMovie has joined #archiveteam [01:49] *** Boppen has quit IRC (Ping timeout: 200 seconds) [01:49] *** Boppen has joined #archiveteam [01:50] *** MMovie has quit IRC (Read error: Operation timed out) [01:51] *** MMovie has joined #archiveteam [01:53] *** philpem has quit IRC (Ping timeout: 260 seconds) [01:59] *** Froggypwn has joined #archiveteam [02:04] yeah, grab livejournal [02:04] def [02:06] *** Boppen has quit IRC (hub.se irc.du.se) [02:06] *** Froggypwn has quit IRC (Ping timeout: 252 seconds) [02:09] *** Froggypwn has joined #archiveteam [02:21] *** MMovie has quit IRC (Read error: Operation timed out) [02:22] *** MMovie has joined #archiveteam [02:30] *** Atom-- has quit IRC (Ping timeout: 252 seconds) [02:35] *** Atom__ has joined #archiveteam [02:36] *** MMovie has quit IRC (Read error: Operation timed out) [02:37] *** MMovie has joined #archiveteam [02:41] *** thx_ has joined #archiveteam [02:41] *** thx_ has quit IRC (Client Quit) [02:43] *** Famicoman has joined #archiveteam [02:44] *** dashcloud has quit IRC (Read error: Operation timed out) [02:48] *** dashcloud has joined #archiveteam [02:53] *** JetBalsa has joined #archiveteam [03:12] *** bwn has quit IRC (Ping timeout: 492 seconds) [03:12] *** Jonimus has quit IRC (Read error: Operation timed out) [03:13] *** remsen has quit IRC (Read error: Operation timed out) [03:13] *** vegbrasil has quit IRC (Read error: Operation timed out) [03:13] *** remsen has joined #archiveteam [03:13] *** nwf has quit IRC (Read error: Operation timed out) [03:14] *** Kenshin has quit IRC (Read error: Operation timed out) [03:14] *** Kenshin has joined #archiveteam [03:14] *** mismatch has joined #archiveteam [03:30] *** logchfoo1 starts logging #archiveteam at Mon Feb 29 03:30:35 2016 [03:30] *** logchfoo1 has joined #archiveteam [03:32] *** vitzli has joined #archiveteam [03:33] *** SimpBrain has joined #archiveteam [03:34] *** toad1 has joined #archiveteam [03:35] *** MMovie has quit IRC (Read error: Operation timed out) [03:37] *** MMovie has joined #archiveteam [03:39] *** godane has quit IRC (Ping timeout: 961 seconds) [03:40] *** vegbrasil has joined #archiveteam [03:40] *** godane has joined #archiveteam [03:43] *** aMunster has joined #archiveteam [03:50] *** MMovie has quit IRC (Read error: Operation timed out) [03:51] *** MMovie has joined #archiveteam [03:51] *** bsmith095 has quit IRC (Read error: Connection reset by peer) [03:52] *** mhazinsk has joined #archiveteam [03:54] *** Froggypwn has quit IRC (Ping timeout: 252 seconds) [03:54] *** Jonimus has joined #archiveteam [03:54] *** bsmith094 has joined #archiveteam [03:57] *** rossdylan has joined #archiveteam [04:01] *** nwf has joined #archiveteam [04:11] *** MMovie has quit IRC (Read error: Operation timed out) [04:11] *** Boppen has joined #archiveteam [04:11] *** MMovie has joined #archiveteam [04:12] *** bsmith094 has quit IRC (Read error: Operation timed out) [04:17] *** bsmith094 has joined #archiveteam [04:21] *** Laverne has quit IRC (Ping timeout: 362 seconds) [04:21] *** Laverne has joined #archiveteam [04:22] *** acridAxid has quit IRC (Ping timeout: 362 seconds) [04:23] *** acridAxid has joined #archiveteam [04:23] *** JetBalsa has quit IRC (Quit: - nbs-irc 2.39 - www.nbs-irc.net -) [04:29] *** Froggypwn has joined #archiveteam [04:35] *** Emcy_ has joined #archiveteam [04:35] *** antomati_ has joined #archiveteam [04:36] *** antomatic has quit IRC (Read error: Connection reset by peer) [04:36] *** Emcy has quit IRC (Read error: Connection reset by peer) [04:38] *** Fletcher_ has quit IRC (Ping timeout: 250 seconds) [04:38] *** Deewiant has quit IRC (Ping timeout: 250 seconds) [04:39] *** midas has quit IRC (Ping timeout: 250 seconds) [04:39] *** DFJustin has quit IRC (Read error: Connection reset by peer) [04:39] *** karendoes has quit IRC (Ping timeout: 250 seconds) [04:40] *** karen has joined #archiveteam [04:42] *** DFJustin has joined #archiveteam [04:46] *** dashcloud has quit IRC (Read error: Operation timed out) [04:47] *** LordNigh2 has joined #archiveteam [04:47] *** dashcloud has joined #archiveteam [04:49] *** midas has joined #archiveteam [04:51] *** Sk2d has joined #archiveteam [04:51] *** Lord_Nigh has quit IRC (Ping timeout: 250 seconds) [04:51] *** Sk1d has quit IRC (Ping timeout: 250 seconds) [04:51] *** LordNigh2 is now known as Lord_Nigh [04:51] *** gibigiana has quit IRC (Remote host closed the connection) [04:51] *** Deewiant has joined #archiveteam [04:51] *** Fletcher_ has joined #archiveteam [04:51] *** filippo__ has joined #archiveteam [04:51] *** Sk2d is now known as Sk1d [04:58] *** gibigiana has joined #archiveteam [05:08] *** MMovie has quit IRC (Read error: Operation timed out) [05:09] Last chance - we got the Eve Onine wiki, right? [05:09] By any means necessary? [05:10] *** MMovie has joined #archiveteam [05:10] ArchiveBot is chipping away at it. [05:19] I think we've got most of it between the two grabs.. [05:20] *** Sk1d has quit IRC (Ping timeout: 200 seconds) [05:22] I should have a full copy [05:22] From my local scrape [05:22] I'll upload it to the IA [05:27] *** Sk1d has joined #archiveteam [05:35] *** MMovie has quit IRC (Read error: Operation timed out) [05:37] *** MMovie has joined #archiveteam [05:44] As soon as it finishes uploading it will be at https://archive.org/details/evewiki_2016-02-27 [05:49] *** yipdw_ has quit IRC (Remote host closed the connection) [05:50] *** yipdw has joined #archiveteam [05:50] *** Boppen has quit IRC (hub.se irc.du.se) [05:51] *** WinterFox has joined #archiveteam [06:03] *** MMovie has quit IRC (Read error: Operation timed out) [06:04] *** MMovie has joined #archiveteam [06:17] *** Fletcher sets mode: +o Fletcher_ [06:32] *** wp494 has quit IRC (Read error: Operation timed out) [06:37] *** MMovie has quit IRC (Read error: Operation timed out) [06:39] *** MMovie has joined #archiveteam [06:41] BACK [06:41] Everyone who has one, excellent. [06:42] *** wp494 has joined #archiveteam [06:43] *** dserodio has quit IRC (Read error: Operation timed out) [06:52] *** dserodio has joined #archiveteam [06:53] I wrote to the author of Scriblio (the project that bought a copy of the LoC's MARC records and passed them along to OpenLibrary.org) about the scriblio.net website being down -- and he's going to put it back up! Yay! [07:02] *** MMovie has quit IRC (Read error: Operation timed out) [07:03] Argh, the IA site says it's down for "maintenance." I hope it didn't eat my upload [07:03] *** MMovie has joined #archiveteam [07:09] weird, and archive.is also seems to be down. What's going on? [07:14] archive.is is up for me [07:15] it shows "Loading" [07:17] vitzli: yes, forever. That doesn't count as "up" to me. :-/ [07:18] yeah, "Loading" for 3 minutes already [07:19] IA is back [07:20] Fuck, it ate my upload! [07:21] It created the item but with only one of the files. I'll try uploading again [07:21] ;-( [07:22] Of course it had to save the *smallest* file (by far) [07:22] Oh well [07:25] LOL, and now one of the files failed with "Server IO Error" (fortunately it's not the main file) [07:32] *** megaminxw has joined #archiveteam [07:45] *** metalcamp has joined #archiveteam [07:51] *** MMovie has quit IRC (Read error: Operation timed out) [07:52] *** MMovie has joined #archiveteam [08:02] *** Jonimus has quit IRC (Read error: Operation timed out) [08:02] *** nwf has quit IRC (Read error: Operation timed out) [08:03] Heads up, Wiki for EVE goes down in 2 hours [08:03] *** beardicus has quit IRC (Read error: Operation timed out) [08:03] *** rossdylan has quit IRC (Read error: Operation timed out) [08:04] *** aMunster has quit IRC (Read error: Operation timed out) [08:04] *** vegbrasil has quit IRC (Read error: Operation timed out) [08:04] *** aMunster has joined #archiveteam [08:04] *** vegbrasil has joined #archiveteam [08:04] *** beardicus has joined #archiveteam [08:05] *** Jonimus has joined #archiveteam [08:05] *** nwf has joined #archiveteam [08:14] *** rossdylan has joined #archiveteam [08:17] *** MMovie has quit IRC (Read error: Operation timed out) [08:18] *** JesseW has quit IRC (Quit: Leaving.) [08:18] *** MMovie has joined #archiveteam [08:21] *** dxrt- has quit IRC (Ping timeout: 260 seconds) [08:22] *** sivoais has quit IRC (Ping timeout: 252 seconds) [08:23] *** espes__ has quit IRC (Ping timeout: 252 seconds) [08:23] *** chazchaz has quit IRC (Remote host closed the connection) [08:23] *** chazchaz has joined #archiveteam [08:24] *** metalcamp has quit IRC (Ping timeout: 252 seconds) [08:25] *** schbirid has joined #archiveteam [08:33] *** MMovie has quit IRC (Read error: Operation timed out) [08:33] *** MMovie has joined #archiveteam [08:34] *** vitzli has quit IRC (Leaving) [08:35] *** wp494 has quit IRC (LOUD UNNECESSARY QUIT MESSAGES) [08:37] *** scyther has joined #archiveteam [08:40] SketchCow: I uploaded my crawl of the EVE wiki to the IA at https://archive.org/details/evewiki_2016-02-27 [08:43] *** wp494 has joined #archiveteam [08:44] *** bwn has joined #archiveteam [08:45] *** sivoais has joined #archiveteam [08:55] *** khaoohs_ has joined #archiveteam [09:00] *** atomotic has joined #archiveteam [09:00] Thanks. Integrated. [09:05] *** khaoohs has quit IRC (Read error: Operation timed out) [09:09] *** espes__ has joined #archiveteam [09:10] *** MMovie has quit IRC (Read error: Operation timed out) [09:11] *** MMovie has joined #archiveteam [09:28] *** MMovie has quit IRC (Read error: Operation timed out) [09:30] *** MMovie has joined #archiveteam [09:48] *** MMovie has quit IRC (Read error: Operation timed out) [09:49] *** MMovie has joined #archiveteam [09:53] *** vtyl has quit IRC (Read error: Operation timed out) [09:55] *** vtyl has joined #archiveteam [10:04] *** MMovie has quit IRC (Read error: Operation timed out) [10:06] *** MMovie has joined #archiveteam [10:24] *** MMovie has quit IRC (Read error: Operation timed out) [10:26] *** MMovie has joined #archiveteam [10:43] *** MMovie has quit IRC (Read error: Operation timed out) [10:45] *** MMovie has joined #archiveteam [11:03] *** MMovie has quit IRC (Read error: Operation timed out) [11:05] *** MMovie has joined #archiveteam [11:22] *** MMovie has quit IRC (Read error: Operation timed out) [11:23] *** MMovie has joined #archiveteam [11:27] *** scyther has quit IRC (Quit: Leaving) [11:37] *** MMovie has quit IRC (Read error: Operation timed out) [11:38] *** MMovie has joined #archiveteam [11:51] *** morbus_ has joined #archiveteam [11:51] *** Morbus has quit IRC (Ping timeout: 258 seconds) [11:52] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [12:10] *** metalcamp has joined #archiveteam [12:10] *** MMovie has quit IRC (Read error: Operation timed out) [12:11] *** MMovie has joined #archiveteam [12:11] *** arkiver2 has joined #archiveteam [12:23] *** metalcamp has quit IRC (Ping timeout: 252 seconds) [12:27] *** MMovie has quit IRC (Read error: Operation timed out) [12:28] *** MMovie has joined #archiveteam [12:28] *** WinterFox has quit IRC (Remote host closed the connection) [12:42] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [12:46] *** atomotic has joined #archiveteam [13:01] *** MMovie has quit IRC (Read error: Operation timed out) [13:02] *** MMovie has joined #archiveteam [13:05] *** scyther has joined #archiveteam [13:07] People, since i got a idle ded [13:07] Any active projevts? [13:07] Projects? [13:07] 200Mbit is waitingz [13:08] *** bauruine has quit IRC (Ping timeout: 260 seconds) [13:13] *** bauruine has joined #archiveteam [13:14] *** vOYtEC has quit IRC (Ping timeout: 258 seconds) [13:14] *** Atluxity has quit IRC (Ping timeout: 260 seconds) [13:16] *** PotcFdk has quit IRC (Ping timeout: 506 seconds) [13:19] *** altlabel has quit IRC (Ping timeout: 506 seconds) [13:19] *** MMovie has quit IRC (Read error: Operation timed out) [13:20] *** mafrasi2 has quit IRC (Ping timeout: 506 seconds) [13:20] *** MMovie has joined #archiveteam [13:21] *** PotcFdk has joined #archiveteam [13:21] *** PurpleSym has quit IRC (*) [13:21] *** ppsym has joined #archiveteam [13:22] *** ppsym is now known as PurpleSym [13:22] *** mafrasi2 has joined #archiveteam [13:22] *** vOYtEC has joined #archiveteam [13:24] *** Atluxity has joined #archiveteam [13:27] *** altlabel has joined #archiveteam [13:28] *** morbus_ has quit IRC (Quit: http://www.disobey.com/) [13:31] *** Morbus has joined #archiveteam [13:35] *** megaminxw has quit IRC (Quit: Leaving.) [13:35] *** megaminxw has joined #archiveteam [13:36] *** MMovie has quit IRC (Read error: Operation timed out) [13:36] *** megaminxw has quit IRC (Client Quit) [13:37] *** MMovie has joined #archiveteam [13:38] *** Morbus has quit IRC (Quit: http://www.disobey.com/) [13:44] *** Morbus has joined #archiveteam [13:45] *** noahc_ has joined #archiveteam [13:51] *** MMovie has quit IRC (Read error: Operation timed out) [13:53] *** MMovie has joined #archiveteam [13:56] *** arkiver2 has joined #archiveteam [14:09] *** MMovie has quit IRC (Read error: Operation timed out) [14:09] *** MMovie has joined #archiveteam [14:11] limebyte, FTP [14:11] https://github.com/ArchiveTeam/ftp-grab [14:11] WOW FTP [14:11] Dat protocol is dyingz? [14:12] where do i need to send the URL to my ftp server? [14:12] lots of stuff on there [14:13] needs backup [14:13] read the repo [14:15] limebyte: how much data are we talking about? archivebot can handle a lot [14:21] Well [14:22] need set a bandwidth limit [14:22] as soon i gonna exeed more as 25MB/s [14:22] Online gonna rape me [14:22] does that client support such? [14:23] *** metalcamp has joined #archiveteam [14:23] no. [14:23] Dam [14:23] archivebot grabs, at any speed [14:24] also, archivebot isnt a backup service fyi :p [14:24] i was kidding btw [14:25] however i will find a way to limit it [14:31] *** MMovie has quit IRC (Read error: Operation timed out) [14:33] *** MMovie has joined #archiveteam [14:44] *** beardicus has quit IRC (Read error: Operation timed out) [14:44] *** beardicus has joined #archiveteam [14:51] *** Start has quit IRC (Quit: Disconnected.) [14:51] *** noahc_ has quit IRC (Ping timeout: 255 seconds) [14:55] *** aMunster has quit IRC (Read error: Operation timed out) [14:56] *** nwf has quit IRC (Read error: Operation timed out) [14:57] *** metalcamp has quit IRC (Ping timeout: 252 seconds) [15:01] *** mhazinsk has quit IRC (Ping timeout: 633 seconds) [15:03] *** Jonimus has quit IRC (Ping timeout: 633 seconds) [15:06] *** rossdylan has quit IRC (Ping timeout: 633 seconds) [15:06] *** mhazinsk has joined #archiveteam [15:06] *** vegbrasil has quit IRC (Ping timeout: 605 seconds) [15:06] *** MMovie has quit IRC (Read error: Operation timed out) [15:07] *** vegbrasil has joined #archiveteam [15:07] *** Jonimus has joined #archiveteam [15:08] *** closure has quit IRC (Ping timeout: 605 seconds) [15:08] *** MMovie has joined #archiveteam [15:09] *** beardicus has quit IRC (Read error: Operation timed out) [15:17] *** vegbrasil has quit IRC (Ping timeout: 633 seconds) [15:17] *** mhazinsk has quit IRC (Ping timeout: 633 seconds) [15:18] *** Jonimus has quit IRC (Ping timeout: 633 seconds) [15:26] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:37] *** closure has joined #archiveteam [15:37] *** vegbrasil has joined #archiveteam [15:38] *** beardicus has joined #archiveteam [15:38] *** MMovie has quit IRC (Read error: Operation timed out) [15:38] *** szalwia has joined #archiveteam [15:38] EVE online official wiki went down today [15:38] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [15:38] they published a database dump here http://web.ccpgamescdn.com/common/wiki/Evelopedia_Database_Copy_2016-02-17.zip [15:39] *** MMovie has joined #archiveteam [15:39] Yep, we archived it [15:40] cool. great job, guys [15:40] You should be able to browse the whole thing on the Internet Archive's Wayback Machine [15:40] yeah. i've been directing my corpmates there [15:41] *** Jonimus has joined #archiveteam [15:41] *** aMunster has joined #archiveteam [15:43] they're also shutting down dust 514 (their f2p fps shooter in the eve universe) on May 30 [15:43] Unfortunately I'm not sure there's much we can do for that :/ [15:44] we could grab data from the official api [15:44] characters sheets, killmails, etc. [15:45] szalwia: if you have data on the api someone here might be able to help you with that so we can archive it [15:53] afaik dust 514 shares the same api endpoints with eve https://eveonline-third-party-documentation.readthedocs.org/en/latest/ [15:53] *** rossdylan has joined #archiveteam [15:53] *** mhazinsk has joined #archiveteam [15:55] there are sites that already have a lot historical data for dust like zkillboard https://zkillboard.com/kills/dust/ [15:55] *** nwf has joined #archiveteam [15:56] but i'm not sure what's going to happen to that once the game shuts down [15:56] zkillboard has its own api https://github.com/zKillboard/zKillboard/wiki/API-(Killmails) [16:03] i'll try to get in touch with the zkillboard guys [16:04] maybe they can get us a database dump [16:12] *** atomotic has joined #archiveteam [16:16] maybe someone could contact ccp (the developers) to get us more data? [16:19] *** Start has joined #archiveteam [16:28] *** mismatch has quit IRC (Ping timeout: 633 seconds) [16:28] *** MMovie has quit IRC (Read error: Operation timed out) [16:30] *** MMovie has joined #archiveteam [16:34] *** Jogie has quit IRC (Ping timeout: 252 seconds) [16:38] *** Tomcat_ has joined #archiveteam [16:41] sent the zkillboard guy this in-game message https://paste.fedoraproject.org/331216/56764064/ [16:48] *** mismatch has joined #archiveteam [16:53] *** mismatch has quit IRC (Ping timeout: 260 seconds) [16:53] *** mismatch_ has joined #archiveteam [16:54] *** jut has joined #archiveteam [16:55] *** Start has quit IRC (Read error: Connection reset by peer) [16:56] *** Start has joined #archiveteam [16:58] *** MMovie has quit IRC (Read error: Operation timed out) [16:59] *** MMovie has joined #archiveteam [17:02] *** JesseW has joined #archiveteam [17:07] *** Start has quit IRC (Quit: Disconnected.) [17:15] *** MMovie has quit IRC (Read error: Operation timed out) [17:16] *** MMovie has joined #archiveteam [17:27] *** JesseW has quit IRC (Quit: Leaving.) [17:32] *** LastNinja has quit IRC (Ping timeout: 260 seconds) [17:36] *** metalcamp has joined #archiveteam [17:43] *** philpem has joined #archiveteam [18:02] Fotolog is still going strong! [18:03] *** MMovie has quit IRC (Read error: Operation timed out) [18:04] *** MMovie has joined #archiveteam [18:06] Would adding another machine to that be helpful? [18:11] last I checket we are at tracker limit [18:11] *checked [18:24] How long do you think fotolog will stay up? [18:29] They said they would close down 9 days ago, so who knows [18:30] *** MMovie has quit IRC (Read error: Operation timed out) [18:30] *** MMovie has joined #archiveteam [18:34] someone forgot to make the change-ticket, and now it is up at the next CAB? [18:42] *** Start has joined #archiveteam [18:49] How big do we think Fotolog is going to be? [18:57] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [19:00] *** MMovie has quit IRC (Read error: Operation timed out) [19:02] *** MMovie has joined #archiveteam [19:05] *** atomotic has joined #archiveteam [19:11] we've now done 825000 users [19:11] and according to the site they have 33 million user [19:11] users* [19:11] If the average size for each user stays at this level we'll have a total of 176 TB [19:12] I doubt fotolog will stay online long enough to get all 33 million users though [19:12] Can you raise the limit? [19:12] we're not on the limit [19:12] I assumed you focused on the oldest accounts first. [19:12] How much time is left for Fotolog [19:13] We didn't have any list of users [19:13] so we're just adding and discovering users as the grab goes [19:13] *** Start has quit IRC (Quit: Disconnected.) [19:14] arkiver: Do we need emergency resources to throw at this project for a few hours? I can spin up a couple hundred DO instances and get us speeding along if necessary. [19:14] I think the main bottleneck is their very slow servers [19:15] phuzion: the site is already very slow, I'm not sure how much more they can handle [19:15] Gotcha. [19:15] *** n00b962 has joined #archiveteam [19:16] Hey, is the tracker dead? URLTeam project is erroring [19:16] SketchCow: as for time left, that would be like over 1.5 years at this speed [19:16] (and the server can't handle a lot more, so we'll probably stay at this speed) [19:16] "The tracker needs an operator for manual maintenance. Try again later. for url: http://tracker.archiveteam.org:1337/api/get." [19:16] Let me check n00b962 [19:18] paging jessew OH WAIT he's not here [19:18] xmc: I'm a urlteam tracker admin now too, FYI [19:18] ah cool [19:18] There it goes [19:18] Cool. [19:18] Thanks, phuzion [19:19] We just had a large backlog of errors, the tracker shuts down when it detects it has a lot. [19:19] Ahh [19:19] *** arkiver2 has joined #archiveteam [19:20] *** n00b962 has quit IRC (Client Quit) [19:20] Where's my Staples button, because "That was easy!" [19:20] *** Start has joined #archiveteam [19:21] arkiver: I mean how long before they shut down Fotolog [19:21] -9 days [19:21] they shuld have shut down 9 days ago [19:21] should* [19:23] Perhaps they are staying up until their servers expire [19:23] Maybe [19:24] SketchCow: on the 26th of januari they announced they were going offline the 20th of february [19:24] We started the grab the 9th of february [19:25] Basically it was always impossible to get even a few percent before they'd shut down [19:25] Unless we'd have started archiving them preemptively [19:25] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [19:26] *** arkiver2 has joined #archiveteam [19:26] *** MMovie has quit IRC (Read error: Operation timed out) [19:28] *** MMovie has joined #archiveteam [19:35] *** jut has quit IRC (Read error: Connection reset by peer) [19:41] We can definitely use some more resourced on the FTP project! [19:44] *** Start has quit IRC (Ping timeout: 260 seconds) [19:48] The mass of data is just getting archive.org's attention, this is why I'm asking [19:52] *** Start has joined #archiveteam [20:01] arkiver: we´ll point some machines at the FTP project later today. Is the discussion for this project in #effteepee ? [20:01] yes [20:01] That'd be awesome! [20:02] BnA-Robin, we will have something else for you to throw machines at soon [20:03] *** schbirid has quit IRC (Quit: Leaving) [20:08] HCross: Awesome, we have some machines that are still pointing at some projects that are ended recently. We´ll have to do some scripting soon to make it a bit easier to reconfigure these machines in the future. [20:09] ah, this one is a very long term one, that is quite custom [20:14] *** ndiddy has joined #archiveteam [20:14] *** MMovie has quit IRC (Read error: Operation timed out) [20:15] *** MMovie has joined #archiveteam [20:15] *** megaminxw has joined #archiveteam [20:21] BnA-Robin, what is your throughput to France like? [20:26] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [20:30] *** metalcamp has quit IRC (Ping timeout: 252 seconds) [20:31] *** metalcamp has joined #archiveteam [20:35] *** bwn has quit IRC (Ping timeout: 246 seconds) [20:43] *** MMovie has quit IRC (Read error: Operation timed out) [20:44] *** MMovie has joined #archiveteam [20:44] *** megaminxw has quit IRC (Quit: Leaving.) [20:45] *** megaminxw has joined #archiveteam [20:45] *** megaminxw has quit IRC (Client Quit) [20:45] *** Start has quit IRC (Quit: Disconnected.) [20:54] *** SN4T14 has quit IRC (Remote host closed the connection) [20:55] *** RedType has joined #archiveteam [21:00] *** SN4T14 has joined #archiveteam [21:03] *** bwn has joined #archiveteam [21:06] xmc: when does urlteam scraper upload its grabs? [21:09] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [21:10] *** MMovie has quit IRC (Read error: Operation timed out) [21:11] *** MMovie has joined #archiveteam [21:12] *** Tomcat_ has quit IRC (Remote host closed the connection) [21:16] arkiver: How much horsepower do you want on ftp? [21:16] you can put anything you want on it [21:16] If we go too fast I'll limit the crawl [21:16] Ok [21:17] arkiver: is FOS the target for this project right now? [21:17] yes [21:18] Alright. I'm getting really slow upload to FOS atm. [21:18] FOS seems to be as slow as s snail [21:18] HCross and I with the same thought [21:18] I'm getting 150-200KB/s [21:18] We can use an other target for the FTP grab [21:19] same here [21:19] How much space are you looking at? [21:20] Depends on how fast you can get it up to IA [21:20] or well [21:20] If you can get it up as fast as it's uploaded to the server that 1 TB would be fine [21:20] else I'd suggest some more [21:21] ah, if we can multithread it, like the youtube test we did. Then I suppose we can use that server while newsbuddy is re written. 3.8TB there [21:22] But not immediatly, as that server is dead [21:22] you can't run two scripts at the same time? [21:23] SketchCow: any recent changes to FOS that would explain the speed to it? [21:32] *** MMovie has quit IRC (Read error: Operation timed out) [21:33] *** MMovie has joined #archiveteam [21:36] *** Famicoma1 has quit IRC (Read error: Connection reset by peer) [21:36] *** tomwsmf-a has joined #archiveteam [21:38] *** Famicoma1 has joined #archiveteam [21:40] Oh who the fuck knows [21:40] It's a glory hole combined with a sexual assault dungeon [21:41] Machine is doing some copying over of the oldfriends collection which is mostly a bunch of mv's to the same drive - etc. [21:41] load average: 4.18, 5.22, 4.73 [21:47] *** metalcamp has quit IRC (Ping timeout: 252 seconds) [21:51] I smell a new fos motd [21:57] *** MMovie has quit IRC (Read error: Operation timed out) [21:57] *** MMovie has joined #archiveteam [22:00] yeah my crawler is going to be bottlenecked by FOS speeds in a couple days likely [22:03] *** Famicoma1 has quit IRC (Read error: Connection reset by peer) [22:03] arkiver, i couild [22:03] wacha need [22:04] *** scyther has quit IRC (Quit: Leaving) [22:05] *** dxrt- has joined #archiveteam [22:05] *** dxrt sets mode: +o dxrt- [22:06] FOS, your home for 100% uncensored large hard disks always ready for more [22:10] for ftp-grab [22:10] how much storage is enought? [22:11] Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | FOS DOLLA DOLLA BILLS Y'ALL [22:12] *** SketchCow changes topic to: Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | FOS DOLLA DOLLA BILLS Y'ALL [22:23] *** bzc6p has joined #archiveteam [22:24] *** bzc6p has quit IRC (Client Quit) [22:25] *** MMovie has quit IRC (Read error: Operation timed out) [22:26] *** MMovie has joined #archiveteam [22:44] *** MMovie has quit IRC (Read error: Operation timed out) [22:45] *** MMovie has joined #archiveteam [22:45] *** Famicoma1 has joined #archiveteam [23:01] *** MMovie has quit IRC (Read error: Operation timed out) [23:02] *** MMovie has joined #archiveteam [23:20] *** MMovie has quit IRC (Read error: Operation timed out) [23:21] *** MMovie has joined #archiveteam [23:36] *** JW_work has joined #archiveteam [23:44] *** xXx_ndidd has joined #archiveteam [23:46] *** brayden_ has joined #archiveteam [23:47] Well Guys, launched a 100Mbit unmetered VM on my Dedi, pulling well [23:47] hopefully 250GB Storage are enought [23:47] for ftp-grab [23:49] *** brayden has quit IRC (Read error: Operation timed out) [23:52] *** ndiddy has quit IRC (Read error: Operation timed out) [23:59] It's enough to ru ftp-grab [23:59] it's not enough to be a rsync target for ftp-grab