[00:14] SketchCow: ping [00:14] SketchCow: See PM please [00:14] It's semi-urgent [00:15] The thing, yes. [00:15] How much data [00:17] Like 100 gigs [00:18] It's the starwars forum stuff [00:18] Sent you the password [00:18] Got it, thanks [00:18] Put in subdirectory called STARWARS-FORUMS [00:18] Still gives me an auth failed message [00:19] Then you're doing it wrong. [00:19] Paste in the channel and stop blowing it into this one [00:19] @ERROR: auth failed on module gv_ [00:19] Password: [00:19] alex@datalove:~$ rsync -avP starwars.tar.maybebad blindtiger.textfiles.com::gv_5 [00:19] Nobody needs to see this. [00:19] Oh, sorry [00:19] Excellent, that is precisely the opposite. [00:20] ? [00:34] howdy all [00:36] Yo [00:40] SketchCow: could you check to see which slot I'm assigned to? [00:44] I need to stop uploads to those machines [00:44] Give me a day or two [00:46] ok [00:47] SketchCow: Where are we moving? [00:54] New machine, 36tb [00:54] We were only using 20, but with everyone with something 20mb-2tb in random slots [00:54] now it'll pool [00:56] Oh, that's nifty [00:57] Same thing, write-only rsync? [01:15] Yes [01:45] SketchCow: "That awesome time..." is amazing [01:45] I just watched it in its entirety for the first time [02:20] the friendster answer to what happened to my stuff is really ugly- they've could've at least dropped the "we had to" part and just said we removed instead [02:42] I love the we had to. [04:08] friendster answer? where's that? [04:35] Hey, gang. [04:35] OK, so I am moving data around like crazy on the servers. [04:36] did you find Secretz~ [05:28] SketchCow, Jeff wrote me to reupload that file, will it work now? [05:28] He is the expert. [05:30] ï¿½Just realized the file you mention is not there. if it is less than 2GB you can upload it to the item at http://ia600602.us.archive.org/edit.php?identifier=openwetware.org. if it's over 2GB there are options that include using the ias3 interface, an auto submit or a wget if it is on a server.ï¿½ [05:31] * Nemo_bis doesn't understand if he means under another item [05:45] He's saying the item needs to be uploaded with an s3 command [05:53] I already did so [05:53] There's no way you could upload a 17.5 GiB file otherwise [12:17] root@teamarchive-0:/3/FRIENDSTER# cat friendster.00220xxxx.tar.bz2 | bunzip2 - | tar vtf - | tail [12:18] That... that is going to take a while. [12:21] i found that using tar v can be much slower than not having it output stuff [12:29] I'm just trying to verify if that file truly has 002200000-002299999 [12:49] Maybe it's just me, but it looks like tr.im finally bought the farm? I can't get it to resolve here... [12:51] Same here. [13:40] Spirit_: No wonder, since it has to buffer up the text [13:40] Spirit_: Anything that outputs to a display takes lots of time more than if it wouldn't [13:55] Ooooh, dear, this could get large. I'm currently spidering the moribund Soundshock forums (FM Synth music community). [13:56] Predictably, it looks like there's going to contain a lot of mp3s, pdf datasheets, and images (schematics and the like). [14:20] Awesome :) [14:54] wicked, http://geocities.yahoo.co.jp/ [14:56] Yup. I've been wondering about geogities.jp, actually... [15:07] Sort of in the same vein as Angelfire and Tripod. I hit a site and express surprise to myself that they're still around. [15:08] Has anyone done any sort of initial surveying of any of them to determine archivability? [15:08] No [15:08] If you want it, it's yours. [15:10] Hmm, I may at that. Not /super/ sure how to approach it, but I don't believe the Geocities effort started with better info. [15:14] It's the big reckoning, where I am funelling all the material into the archiveteam machine [15:14] That's... it's just a lot of friendster. [15:19] Friendster really was much larger than I expected for a network I'd barely heard of. [15:19] It's not THAT large. [15:19] It's the photos. [15:22] The dumping will be going all night. [15:22] Probably a few days, it'll be exciting. [15:23] Something like a terabyte, just for one of these directories. [15:23] oh, i should totally run a local dns server for my robots.txt downloading [15:23] how stupid of me [15:24] yahoo is a lot more popular in japan for whatever reason and is basically run separately iirc, I don't know that geocities.jp is in any special danger [15:25] what blew my mind the other day is homestead.com is still up [15:25] Well in any case, I'll probably look into one of those three once I've finished soundshock. Regardless of Yahoo Japan's health, losing it would be suboptimal. [15:25] for sure [15:27] In the mean time, since I'm apparently not going to be employed by a hosting company, I need to figure out a good way of moving a Friendsterball. [15:27] ugh. [15:28] I have 988GB (compressed [15:28] ) myself [15:28] Oh bloody hell, man. [15:29] what spooks me japanese internet-wise is pixiv, there is SO much stuff on there and it's all behind a login barrier so wayback will have 0% of it [15:29] I think the best way is going to be sneakernet via ups/fedex [15:30] Yeah, we have to get cranking on the friendster. [15:30] Shortly, I'll be rejiggering things so we can figure out next moves. [15:38] DFJustin: Ooooh Pixiv. I'm going to declare right now that it's worse than you think, even. [15:38] (Well, unless you've done any research on it, in which case it's probably only just as bad.) [17:52] ndurner_o: How we doing with ggroups? [17:53] I've been getting a lot of 501s [17:59] Discovery by topic is done [17:59] I'm currently preferring downloads over discovery [18:00] All in all, we're doing pretty good. [18:31] http://archiveteam.org/index.php?title=Special:Contributions/Alepa [18:31] O_o [18:46] Is there a way to tell httrack to ignore a certain set of URI (like, say, profile.php or privmsg.php)? [19:00] Okay, I think I got it. Needed to use minus to filter. [20:51] 2.6T . [20:51] root@teamarchive-0:/3/FRIENDSTER# du -sh . [20:56] Jeez [20:57] Long way to go. [21:08] SketchCow: is that compressed or raw? [22:10] Ugh, a lot of the forum links here are for googlepages domains. Is there a simple format for rewriting all of those? [22:30] They say they "migrated" everything, but... well, it looks like Adlib Underground is a lost cause.