| Time |
Nickname |
Message |
|
00:14
🔗
|
underscor |
SketchCow: ping |
|
00:14
🔗
|
underscor |
SketchCow: See PM please |
|
00:14
🔗
|
underscor |
It's semi-urgent |
|
00:15
🔗
|
SketchCow |
The thing, yes. |
|
00:15
🔗
|
SketchCow |
How much data |
|
00:17
🔗
|
underscor |
Like 100 gigs |
|
00:18
🔗
|
underscor |
It's the starwars forum stuff |
|
00:18
🔗
|
SketchCow |
Sent you the password |
|
00:18
🔗
|
underscor |
Got it, thanks |
|
00:18
🔗
|
SketchCow |
Put in subdirectory called STARWARS-FORUMS |
|
00:18
🔗
|
underscor |
Still gives me an auth failed message |
|
00:19
🔗
|
SketchCow |
Then you're doing it wrong. |
|
00:19
🔗
|
SketchCow |
Paste in the channel and stop blowing it into this one |
|
00:19
🔗
|
underscor |
@ERROR: auth failed on module gv_ |
|
00:19
🔗
|
underscor |
Password: |
|
00:19
🔗
|
underscor |
alex@datalove:~$ rsync -avP starwars.tar.maybebad blindtiger.textfiles.com::gv_5 |
|
00:19
🔗
|
SketchCow |
Nobody needs to see this. |
|
00:19
🔗
|
underscor |
Oh, sorry |
|
00:19
🔗
|
SketchCow |
Excellent, that is precisely the opposite. |
|
00:20
🔗
|
underscor |
? |
|
00:34
🔗
|
db48x |
howdy all |
|
00:36
🔗
|
SketchCow |
Yo |
|
00:40
🔗
|
db48x |
SketchCow: could you check to see which slot I'm assigned to? |
|
00:44
🔗
|
SketchCow |
I need to stop uploads to those machines |
|
00:44
🔗
|
SketchCow |
Give me a day or two |
|
00:46
🔗
|
db48x |
ok |
|
00:47
🔗
|
underscor |
SketchCow: Where are we moving? |
|
00:54
🔗
|
SketchCow |
New machine, 36tb |
|
00:54
🔗
|
SketchCow |
We were only using 20, but with everyone with something 20mb-2tb in random slots |
|
00:54
🔗
|
SketchCow |
now it'll pool |
|
00:56
🔗
|
underscor |
Oh, that's nifty |
|
00:57
🔗
|
underscor |
Same thing, write-only rsync? |
|
01:15
🔗
|
SketchCow |
Yes |
|
01:45
🔗
|
underscor |
SketchCow: "That awesome time..." is amazing |
|
01:45
🔗
|
underscor |
I just watched it in its entirety for the first time |
|
02:20
🔗
|
dashcloud |
the friendster answer to what happened to my stuff is really ugly- they've could've at least dropped the "we had to" part and just said we removed instead |
|
02:42
🔗
|
SketchCow |
I love the we had to. |
|
04:08
🔗
|
Coderjoe |
friendster answer? where's that? |
|
04:35
🔗
|
SketchCow |
Hey, gang. |
|
04:35
🔗
|
SketchCow |
OK, so I am moving data around like crazy on the servers. |
|
04:36
🔗
|
RedType |
did you find Secretz~ |
|
05:28
🔗
|
Nemo_bis |
SketchCow, Jeff wrote me to reupload that file, will it work now? |
|
05:28
🔗
|
SketchCow |
He is the expert. |
|
05:30
🔗
|
Nemo_bis |
�Just realized the file you mention is not there. if it is less than 2GB you can upload it to the item at http://ia600602.us.archive.org/edit.php?identifier=openwetware.org. if it's over 2GB there are options that include using the ias3 interface, an auto submit or a wget if it is on a server.� |
|
05:31
🔗
|
* |
Nemo_bis doesn't understand if he means under another item |
|
05:45
🔗
|
SketchCow |
He's saying the item needs to be uploaded with an s3 command |
|
05:53
🔗
|
Nemo_bis |
I already did so |
|
05:53
🔗
|
Nemo_bis |
There's no way you could upload a 17.5 GiB file otherwise |
|
12:17
🔗
|
SketchCow |
root@teamarchive-0:/3/FRIENDSTER# cat friendster.00220xxxx.tar.bz2 | bunzip2 - | tar vtf - | tail |
|
12:18
🔗
|
SketchCow |
That... that is going to take a while. |
|
12:21
🔗
|
Spirit_ |
i found that using tar v can be much slower than not having it output stuff |
|
12:29
🔗
|
SketchCow |
I'm just trying to verify if that file truly has 002200000-002299999 |
|
12:49
🔗
|
Wyatt |
Maybe it's just me, but it looks like tr.im finally bought the farm? I can't get it to resolve here... |
|
12:51
🔗
|
SketchCow |
Same here. |
|
13:40
🔗
|
ersi |
Spirit_: No wonder, since it has to buffer up the text |
|
13:40
🔗
|
ersi |
Spirit_: Anything that outputs to a display takes lots of time more than if it wouldn't |
|
13:55
🔗
|
Wyatt |
Ooooh, dear, this could get large. I'm currently spidering the moribund Soundshock forums (FM Synth music community). |
|
13:56
🔗
|
Wyatt |
Predictably, it looks like there's going to contain a lot of mp3s, pdf datasheets, and images (schematics and the like). |
|
14:20
🔗
|
ersi |
Awesome :) |
|
14:54
🔗
|
Spirit_ |
wicked, http://geocities.yahoo.co.jp/ |
|
14:56
🔗
|
Wyatt |
Yup. I've been wondering about geogities.jp, actually... |
|
15:07
🔗
|
Wyatt |
Sort of in the same vein as Angelfire and Tripod. I hit a site and express surprise to myself that they're still around. |
|
15:08
🔗
|
Wyatt |
Has anyone done any sort of initial surveying of any of them to determine archivability? |
|
15:08
🔗
|
SketchCow |
No |
|
15:08
🔗
|
SketchCow |
If you want it, it's yours. |
|
15:10
🔗
|
Wyatt |
Hmm, I may at that. Not /super/ sure how to approach it, but I don't believe the Geocities effort started with better info. |
|
15:14
🔗
|
SketchCow |
It's the big reckoning, where I am funelling all the material into the archiveteam machine |
|
15:14
🔗
|
SketchCow |
That's... it's just a lot of friendster. |
|
15:19
🔗
|
Wyatt |
Friendster really was much larger than I expected for a network I'd barely heard of. |
|
15:19
🔗
|
SketchCow |
It's not THAT large. |
|
15:19
🔗
|
SketchCow |
It's the photos. |
|
15:22
🔗
|
SketchCow |
The dumping will be going all night. |
|
15:22
🔗
|
SketchCow |
Probably a few days, it'll be exciting. |
|
15:23
🔗
|
SketchCow |
Something like a terabyte, just for one of these directories. |
|
15:23
🔗
|
Spirit_ |
oh, i should totally run a local dns server for my robots.txt downloading |
|
15:23
🔗
|
Spirit_ |
how stupid of me |
|
15:24
🔗
|
DFJustin |
yahoo is a lot more popular in japan for whatever reason and is basically run separately iirc, I don't know that geocities.jp is in any special danger |
|
15:25
🔗
|
DFJustin |
what blew my mind the other day is homestead.com is still up |
|
15:25
🔗
|
Wyatt |
Well in any case, I'll probably look into one of those three once I've finished soundshock. Regardless of Yahoo Japan's health, losing it would be suboptimal. |
|
15:25
🔗
|
DFJustin |
for sure |
|
15:27
🔗
|
Wyatt |
In the mean time, since I'm apparently not going to be employed by a hosting company, I need to figure out a good way of moving a Friendsterball. |
|
15:27
🔗
|
Coderjoe |
ugh. |
|
15:28
🔗
|
Coderjoe |
I have 988GB (compressed |
|
15:28
🔗
|
Coderjoe |
) myself |
|
15:28
🔗
|
Wyatt |
Oh bloody hell, man. |
|
15:29
🔗
|
DFJustin |
what spooks me japanese internet-wise is pixiv, there is SO much stuff on there and it's all behind a login barrier so wayback will have 0% of it |
|
15:29
🔗
|
Coderjoe |
I think the best way is going to be sneakernet via ups/fedex |
|
15:30
🔗
|
SketchCow |
Yeah, we have to get cranking on the friendster. |
|
15:30
🔗
|
SketchCow |
Shortly, I'll be rejiggering things so we can figure out next moves. |
|
15:38
🔗
|
Wyatt |
DFJustin: Ooooh Pixiv. I'm going to declare right now that it's worse than you think, even. |
|
15:38
🔗
|
Wyatt |
(Well, unless you've done any research on it, in which case it's probably only just as bad.) |
|
17:52
🔗
|
underscor |
ndurner_o: How we doing with ggroups? |
|
17:53
🔗
|
underscor |
I've been getting a lot of 501s |
|
17:59
🔗
|
ndurner_o |
Discovery by topic is done |
|
17:59
🔗
|
ndurner_o |
I'm currently preferring downloads over discovery |
|
18:00
🔗
|
ndurner_o |
All in all, we're doing pretty good. |
|
18:31
🔗
|
Coderjoe |
http://archiveteam.org/index.php?title=Special:Contributions/Alepa |
|
18:31
🔗
|
Coderjoe |
O_o |
|
18:46
🔗
|
Wyatt |
Is there a way to tell httrack to ignore a certain set of URI (like, say, profile.php or privmsg.php)? |
|
19:00
🔗
|
Wyatt |
Okay, I think I got it. Needed to use minus to filter. |
|
20:51
🔗
|
SketchCow |
2.6T . |
|
20:51
🔗
|
SketchCow |
root@teamarchive-0:/3/FRIENDSTER# du -sh . |
|
20:56
🔗
|
ersi |
Jeez |
|
20:57
🔗
|
SketchCow |
Long way to go. |
|
21:08
🔗
|
Coderjoe |
SketchCow: is that compressed or raw? |
|
22:10
🔗
|
Wyatt |
Ugh, a lot of the forum links here are for googlepages domains. Is there a simple format for rewriting all of those? |
|
22:30
🔗
|
Wyatt |
They say they "migrated" everything, but... well, it looks like Adlib Underground is a lost cause. |