| Time |
Nickname |
Message |
|
00:01
🔗
|
brayden |
Rebooting things is a mistake we all make. |
|
00:01
🔗
|
brayden |
I'll check out Warrior this afternoon. |
|
00:02
🔗
|
brayden |
I was already rather impressed with the interface.. if it isn't better though they're fired. |
|
00:03
🔗
|
GLaDOS |
DoubleJ: using that method, how would you prevent any further pipeline downloads? |
|
00:06
🔗
|
GLaDOS |
Wait, nevermind. |
|
00:20
🔗
|
DoubleJ |
GLaDOS: Glad you figured it out, 'cause I don't really know what you were asking :) |
|
00:20
🔗
|
DoubleJ |
if you meant "touch STOP" that works like always. |
|
00:52
🔗
|
SketchCow |
root@teamarchive-1:/1# rm -rf PICPLZ |
|
00:52
🔗
|
SketchCow |
yay |
|
00:52
🔗
|
SketchCow |
done |
|
00:52
🔗
|
* |
BlueMax pops the champagne |
|
01:05
🔗
|
S[h]O[r]T |
awesome |
|
01:06
🔗
|
S[h]O[r]T |
thnx to alard and doublej instructions got my cinch running on non warrior |
|
01:06
🔗
|
S[h]O[r]T |
might want to jsut remove the https from git clone and pip otherwise it complains about the cert |
|
01:16
🔗
|
SketchCow |
18G . |
|
01:16
🔗
|
SketchCow |
root@teamarchive-1:/1/CINCH# du -sh . |
|
01:22
🔗
|
godane |
i'm getting the last episode of making the most of micro |
|
01:22
🔗
|
godane |
i'm also not banned from thebox.wz |
|
01:22
🔗
|
SketchCow |
We're likely to kill the entire list of cinch sites we have so far in one night. |
|
01:22
🔗
|
SketchCow |
That'd be nice. |
|
01:27
🔗
|
swebb |
Just messing with the archiveteam warrior image now for the first time. It says that there are no active projects. Is that expected? |
|
01:33
🔗
|
S[h]O[r]T |
i dont think it is swebb but there might be some update command you need to run to get it to download the latest projects |
|
01:36
🔗
|
swebb |
git must not be updated because I'm following the menu.sh script and it's going a git pull successfully, so I'm guessing that the git repo for the archiveteam warrior hasn't been updated yet for this cinch pull. |
|
01:45
🔗
|
S[h]O[r]T |
im not sure but im pretty sure people here are running the warrior but yeah maybe they had to do something manual |
|
01:48
🔗
|
swebb |
I'm going to get it working manually using the archiveteam warrior iso as a base. |
|
01:48
🔗
|
swebb |
We'll see if I can get it running before the whole job is done. :) |
|
01:50
🔗
|
swebb |
Yea! I'm grabbing! |
|
01:54
🔗
|
S[h]O[r]T |
:D |
|
01:54
🔗
|
S[h]O[r]T |
nice |
|
01:55
🔗
|
Coderjoe |
swebb: were you using the older warrior, or the newer python-based one? |
|
01:56
🔗
|
swebb |
I just downloaded the warrior image today. The menu says that it was updated 12 days ago. |
|
01:56
🔗
|
swebb |
I had to install a bunch of stuff to get the python stuff working. Most notably lua. |
|
01:57
🔗
|
Coderjoe |
swebb: this one? Warrior v2 beta version: http://archive.org/download/archiveteam-warrior/archiveteam-warrior-v2-20120813.ova |
|
01:57
🔗
|
swebb |
nope |
|
01:57
🔗
|
swebb |
I'm using v1. |
|
02:02
🔗
|
swebb |
Anyone know anything about the pipeline.py script? I wasn't able to start two of them on the same machine due to a "port in use" error. Is there a way to get two running at once? |
|
02:05
🔗
|
Coderjoe |
pass --help |
|
02:05
🔗
|
Coderjoe |
it has a parameter for how many concurrent downloads to run |
|
02:05
🔗
|
Coderjoe |
also, check out http://localhost:8080/ |
|
02:05
🔗
|
swebb |
Ahh, thanks. |
|
02:06
🔗
|
Coderjoe |
er |
|
02:06
🔗
|
Coderjoe |
make that port 8001 |
|
02:06
🔗
|
swebb |
Yup. Just saw that. Thanks! |
|
02:06
🔗
|
Coderjoe |
(looking at http://archive.org/details/archiveteam-warrior which also links to the most recent v2) |
|
02:36
🔗
|
S[h]O[r]T |
swebb |
|
02:36
🔗
|
S[h]O[r]T |
if you kill your single one and then run it again with the concurrent it will work |
|
02:40
🔗
|
swebb |
Yea, I've used some of the older ones and I know about the STOP file, so I stopped and restarted with more workers. |
|
02:50
🔗
|
underscor |
Received item 'cumiscuteonyrfce' from tracker |
|
02:51
🔗
|
underscor |
SAVED |
|
02:51
🔗
|
swebb |
Wow! underscore just started like 100 workers. :) |
|
02:52
🔗
|
underscor |
16 :D |
|
02:52
🔗
|
Nintendud |
what's going on, guys? |
|
02:52
🔗
|
underscor |
Nintendud: cinch.fm is closing |
|
02:52
🔗
|
Nintendud |
idk what that is, but it sounds bad |
|
02:53
🔗
|
Nintendud |
is this one of those things that I can use the archiveteam warrior on? |
|
02:53
🔗
|
SketchCow |
yes |
|
02:53
🔗
|
Nintendud |
cool cool. I'll see if I can finally get that set up. |
|
02:55
🔗
|
underscor |
the warrior is so awesome |
|
02:55
🔗
|
swebb |
I'm using the v1 version of the archiveteam warrior and I didn't see it on the list of active projects. Do I need v2 to see it in the menu? |
|
02:55
🔗
|
Nintendud |
the only sad part is that it's going to be running as a VM in a VM |
|
02:56
🔗
|
swebb |
Nintendud: it doesn't take hardly any CPU. |
|
02:56
🔗
|
Nintendud |
it's not CPU |
|
02:56
🔗
|
Nintendud |
disk IO is slow already :) |
|
02:56
🔗
|
Nintendud |
it's going to have even slower disk io |
|
02:56
🔗
|
Nintendud |
I'm too lazy to move it closer to the machine |
|
02:57
🔗
|
swebb |
underscore doubled my efforts in just 5 minutes. :) |
|
02:57
🔗
|
underscor |
Nintendud: Why double-vm'd? |
|
02:58
🔗
|
underscor |
swebb: maybe? dunno if the old one still works. should though |
|
02:58
🔗
|
Nintendud |
underscor: I don't feel like repartitioning the disk / memory / cpu among three VMs |
|
02:59
🔗
|
Nintendud |
I share this server with a friend |
|
02:59
🔗
|
underscor |
why not run it on the host? |
|
02:59
🔗
|
Nintendud |
the host machine is VMWare ESXi. Doesn't have any hard drive space allocated to it |
|
02:59
🔗
|
Nintendud |
well, a bare minimum :) |
|
03:00
🔗
|
Nintendud |
no biggie though |
|
03:00
🔗
|
Nintendud |
should be interesting to kick off this VM from the command line. usually I use a GUI. |
|
03:00
🔗
|
swebb |
do the workers wait while an upload (sync) is running? It seems like no downloads are going on during the sync when using concurrentcy. |
|
03:01
🔗
|
swebb |
concurrency |
|
03:01
🔗
|
underscor |
Nintendud: oic |
|
03:01
🔗
|
underscor |
swebb: each thread is synchronous |
|
03:01
🔗
|
underscor |
so if all 6 threads are "uploading" (or however many you're running), then yeah, nothing will download |
|
03:02
🔗
|
swebb |
ok, but each thread is independent, so it doesn't deadlock or something during an rsync. |
|
03:09
🔗
|
Nintendud |
underscor / swebb: Is there a dashboard showing current projects / activity? |
|
03:09
🔗
|
Nintendud |
wondering how you knew that underscor spun up workers |
|
03:09
🔗
|
S[h]O[r]T |
http://tracker.archiveteam.org/cinch/ for cinch |
|
03:09
🔗
|
underscor |
Nintendud: http://warriorhq.archiveteam.org/ to see what we're saving and who's participating |
|
03:09
🔗
|
underscor |
and http://tracker.archiveteam.org/cinch/ to see people in action |
|
03:09
🔗
|
S[h]O[r]T |
http://warriorhq.archiveteam.org/ |
|
03:09
🔗
|
Nintendud |
awesome. why is this not more prominent on archiveteam.org? |
|
03:09
🔗
|
underscor |
s/who's participating/where people are participating from/ |
|
03:10
🔗
|
underscor |
it's still in beta-ish, iirc |
|
03:10
🔗
|
Nintendud |
nice font |
|
03:10
🔗
|
Nintendud |
is that droid? |
|
03:10
🔗
|
underscor |
also the front page is superprotected afaik |
|
03:11
🔗
|
underscor |
no idea |
|
03:11
🔗
|
underscor |
I think it's a google web font, if that helps |
|
03:11
🔗
|
Nintendud |
yeah, I think it's droid |
|
03:11
🔗
|
Nintendud |
Firefox just fubared trying to inspect element :3 |
|
03:12
🔗
|
underscor |
so cool to watch warriorhq |
|
03:12
🔗
|
underscor |
lots of eastcoasters |
|
03:12
🔗
|
S[h]O[r]T |
id be on there in dallas if i was running warrior |
|
03:12
🔗
|
S[h]O[r]T |
maybe ill load it up here at home |
|
03:14
🔗
|
swebb |
For cinch, is there more discovery crawling needed to be done? |
|
03:14
🔗
|
godane |
i'm uploading all my remainer gamesmaster season 5 to the root (/) of my items-uploads.archive.org account |
|
03:15
🔗
|
godane |
this way i can just move them the item id folder when i need to |
|
03:15
🔗
|
godane |
i just want to know will this cause problems? |
|
03:15
🔗
|
Nintendud |
importing vm :3 |
|
03:16
🔗
|
S[h]O[r]T |
swebb i believe everything in the tracker is what cinch publishes on their sitemap which is *supposed* to be everything on the site |
|
03:20
🔗
|
Nintendud |
damn; looks like I'll have to do this through the vbox gui |
|
03:23
🔗
|
S[h]O[r]T |
dang the warrior is nice |
|
03:29
🔗
|
S[h]O[r]T |
think i might have had the biggest one so far. 1.3gb? |
|
03:30
🔗
|
S[h]O[r]T |
i think the concurrent does deadlock the other threads during an rsync |
|
03:31
🔗
|
S[h]O[r]T |
at least not in warrior |
|
03:38
🔗
|
BlueMax |
underscor, you big fat cheater. :| |
|
03:39
🔗
|
underscor |
howso? |
|
03:39
🔗
|
underscor |
:D |
|
03:41
🔗
|
Nintendud |
woot. got the warrior working |
|
03:42
🔗
|
Nintendud |
why is the maximum number of downloads 6? :3 |
|
03:44
🔗
|
BlueMax |
underscor, because you have 2600 items because your e-peen is huge |
|
03:44
🔗
|
swebb |
I'm predicting that underscore will grab > 50% of the whole site. |
|
03:50
🔗
|
BlueMax |
lawl |
|
03:51
🔗
|
S[h]O[r]T |
trying to get stuff setup on a centos box but stupid lib errors all over the place |
|
03:52
🔗
|
S[h]O[r]T |
Nintendud i wonder if 6 is a limit on the ajax interface that or prob since its only running with 400mb of ram |
|
03:54
🔗
|
swebb |
tracker just died for me. |
|
03:55
🔗
|
chronomex |
really? works fine for me. |
|
03:55
🔗
|
Nintendud |
back |
|
03:55
🔗
|
Nintendud |
yeah |
|
03:55
🔗
|
Nintendud |
it doesn't seem to be uploading |
|
03:55
🔗
|
Nintendud |
or accepting uploads* |
|
03:55
🔗
|
swebb |
My uploads have stopped and the tracker page comes up, but no traffic on it. |
|
03:56
🔗
|
Nintendud |
yeah |
|
03:56
🔗
|
Nintendud |
same here |
|
03:56
🔗
|
swebb |
something died somewhere. |
|
03:56
🔗
|
chronomex |
odd. |
|
03:56
🔗
|
Nintendud |
there's no 'single box' that we're uploading to, right? |
|
03:56
🔗
|
chronomex |
the tracker is a single box, but I'm logged in to it right now |
|
03:57
🔗
|
swebb |
Yea, we're rsyncing to a single box. |
|
03:57
🔗
|
Nintendud |
ah |
|
03:57
🔗
|
Nintendud |
that doesn't seem like it would scale |
|
03:57
🔗
|
swebb |
My downloads are still flowing for existing jobs |
|
03:57
🔗
|
swebb |
but uploads are hung. |
|
03:57
🔗
|
chronomex |
ok |
|
03:57
🔗
|
Nintendud |
my downloads are hung because the one upload thread is blocked |
|
03:58
🔗
|
Nintendud |
if I could spawn more download threads, I could be making progress :> |
|
03:58
🔗
|
Nintendud |
(with that said, the warrior is one fine piece of software) |
|
03:58
🔗
|
swebb |
I got an rsync timeout. |
|
03:59
🔗
|
swebb |
failed connect to fos.textfiles.com: Connection times out. |
|
03:59
🔗
|
S[h]O[r]T |
yeah me 2. foss is different from the tracker |
|
04:01
🔗
|
S[h]O[r]T |
i imagine SketchCow must already know :P |
|
04:01
🔗
|
swebb |
I'm seeing life again on the tracker now. |
|
04:01
🔗
|
swebb |
I'm guessing that it has recovered. I'm uploading again. |
|
04:01
🔗
|
Nintendud |
yup |
|
04:03
🔗
|
BlueMax |
underscor needs to stop downloading and let everyone catch up |
|
04:04
🔗
|
BlueMax |
What sort of bloody pipe is he working with |
|
04:04
🔗
|
swebb |
I don't know, but it's *way* bigger than what I've got. |
|
04:05
🔗
|
Nintendud |
I've got a big one... 175 urls so far.. |
|
04:05
🔗
|
DFJustin |
he has access to archive.org boxen |
|
04:05
🔗
|
swebb |
Ahh. |
|
04:06
🔗
|
swebb |
archive.org has a crap-ton of hardware and bandwidth. |
|
04:06
🔗
|
Nintendud |
250... |
|
04:08
🔗
|
swebb |
ok, we're past the 1/2-way mark. |
|
04:08
🔗
|
Nintendud |
50... |
|
04:08
🔗
|
Nintendud |
350* even |
|
04:14
🔗
|
Nintendud |
this guy really loves using cinch... http://cinch.fm/travisdecker |
|
04:14
🔗
|
S[h]O[r]T |
think he has some mental illness..not joking or being funny seems like he used it to send some messages |
|
04:15
🔗
|
Nintendud |
I'm in public with no headphones |
|
04:15
🔗
|
Nintendud |
so I can't check it out yet |
|
04:15
🔗
|
Nintendud |
625 urls so far |
|
04:15
🔗
|
swebb |
I'm downloading scobleizer now! :) |
|
04:15
🔗
|
swebb |
up to 1500 files so-far |
|
04:15
🔗
|
Nintendud |
or wow |
|
04:15
🔗
|
Nintendud |
oh wow* |
|
04:16
🔗
|
swebb |
http://cinch.fm/scobleizer |
|
04:19
🔗
|
Nintendud |
long tail is loooooooooong |
|
04:20
🔗
|
Nintendud |
925... I'm really curious how large his file is going to be |
|
04:21
🔗
|
Nintendud |
too bad I'll have to head home soon. I'll miss that upload. |
|
04:23
🔗
|
BlueMax |
what's the name Nintendud |
|
04:23
🔗
|
BlueMax |
oh wait, travis decker |
|
04:23
🔗
|
BlueMax |
I'll keep my eyes open for it |
|
04:23
🔗
|
Nintendud |
cool cool |
|
04:23
🔗
|
Nintendud |
1050 |
|
04:24
🔗
|
Nintendud |
considering that he has something like 100 files over scobleizer, I assume it'll be 1500 or more files? |
|
04:28
🔗
|
swebb |
scoble had 1700+ urls on cinch. |
|
04:28
🔗
|
Nintendud |
Any way to estimate from the user summary? |
|
04:29
🔗
|
Nintendud |
I dunno what it counts as 'pages' |
|
04:29
🔗
|
Nintendud |
er, URLs |
|
04:29
🔗
|
swebb |
1.3GB for just his stuff. :) |
|
04:29
🔗
|
Nintendud |
1350 so far and counting for Mr. travis decker |
|
04:29
🔗
|
Nintendud |
OH it's uploading |
|
04:30
🔗
|
Nintendud |
looks like ~90 MB |
|
04:30
🔗
|
Nintendud |
disappointing |
|
04:30
🔗
|
Nintendud |
Yup. 84MB. |
|
04:32
🔗
|
Nintendud |
when this is over, I'm going to want stats on stats |
|
04:34
🔗
|
Nintendud |
welp, going home. see you guys later |
|
04:35
🔗
|
chronomex |
cya |
|
04:35
🔗
|
chronomex |
the annoying thing about munin is you have to wait a long time to see interesting graphs. |
|
04:36
🔗
|
chronomex |
http://176.58.114.30/localdomain/localhost.localdomain/index.html#archiveteam |
|
04:37
🔗
|
chronomex |
(spoiler: come back in a few hours.) |
|
04:44
🔗
|
swebb |
cacti is the same way. |
|
04:48
🔗
|
chronomex |
yeah, I would think so |
|
04:53
🔗
|
underscor |
DFJustin: these are all on my home network and on various free OVH vks's |
|
04:53
🔗
|
underscor |
http://www.ovh.com/us/vks/index.xml |
|
04:54
🔗
|
underscor |
hahahaha |
|
04:54
🔗
|
underscor |
\o/ |
|
04:54
🔗
|
underscor |
first place |
|
04:55
🔗
|
swebb |
Dude, you trounced us all! |
|
04:56
🔗
|
underscor |
http://maelstrom.foxfag.com/munin/foxfag.com/ponder.foxfag.com/fw_conntrack.html hahaha |
|
04:59
🔗
|
swebb |
My stats are much wimpier: https://skitch.com/scumola/epa9g/cacti |
|
05:04
🔗
|
chronomex |
that's nice |
|
05:05
🔗
|
S[h]O[r]T |
underscor watch your back |
|
05:05
🔗
|
S[h]O[r]T |
you pulled one over on me though :( |
|
05:07
🔗
|
Nintendud |
oh. now that I'm home... yeah, S[h]O[r]T... travisdecker.. :( |
|
05:08
🔗
|
Nintendud |
probably has a mental illness |
|
05:08
🔗
|
S[h]O[r]T |
good on him tho using the service for whatever his needs were |
|
05:09
🔗
|
Nintendud |
yeah |
|
05:09
🔗
|
Nintendud |
he has 9 followers. I wonder if he was leaving those for a loved one? |
|
05:10
🔗
|
* |
Daevien watches the counter spin |
|
05:14
🔗
|
chronomex |
Lord_Nigh: I misread that as Lord_High |
|
05:14
🔗
|
chronomex |
keep on truckin |
|
05:14
🔗
|
underscor |
wheeeee |
|
05:14
🔗
|
underscor |
we're almost done |
|
05:14
🔗
|
Daevien |
how many boxes you got going underscor? :p |
|
05:14
🔗
|
underscor |
only 8 :( |
|
05:15
🔗
|
Daevien |
heh. i've got two, one is in london. and then my crappy home connection |
|
05:15
🔗
|
underscor |
aha, you're one of the londoners |
|
05:16
🔗
|
underscor |
http://warriorhq.archiveteam.org/ |
|
05:16
🔗
|
Daevien |
i'm not in london, but have a vps there yeah |
|
05:16
🔗
|
underscor |
(the points are purposefully obfuscated somewhat) |
|
05:16
🔗
|
underscor |
sorry, that's what I meant |
|
05:16
🔗
|
Daevien |
the london one is a highly bored linode 512mb vps, figured i may as well do something with it :p |
|
05:17
🔗
|
underscor |
cool :D |
|
05:17
🔗
|
underscor |
yeah, most of mine are OVH vKSes that have mostly just been idling |
|
05:17
🔗
|
Daevien |
er, i also appear to be the only one for sure in canada as well lol |
|
05:18
🔗
|
underscor |
I mean, they do stuff like static file hosting and irc client |
|
05:18
🔗
|
underscor |
but nothing taxing |
|
05:18
🔗
|
underscor |
:o cool! |
|
05:18
🔗
|
underscor |
:D |
|
05:18
🔗
|
Daevien |
i'm trying to smack darkstar and get another position before it's done :p |
|
05:19
🔗
|
Nintendud |
I think my box is in france |
|
05:20
🔗
|
Daevien |
somewhere in the cinch datacenter someone is cursing us probably |
|
05:20
🔗
|
Nintendud |
THe IP I'm behind is in Ireland though |
|
05:20
🔗
|
underscor |
hahaha |
|
05:20
🔗
|
underscor |
they're on ec2 |
|
05:20
🔗
|
underscor |
so they likely won't notice until they get their usage summary |
|
05:21
🔗
|
underscor |
:D |
|
05:21
🔗
|
Daevien |
heh yeah and then will go wtf happened? |
|
05:21
🔗
|
chronomex |
lul |
|
05:21
🔗
|
chronomex |
ec2++ |
|
05:21
🔗
|
Daevien |
where is jason's pic of the rubber glove that he's used for his avatar? :p |
|
05:22
🔗
|
underscor |
I'm kinda amazed we'll have this done inside of 6 hours |
|
05:22
🔗
|
underscor |
I dunno |
|
05:22
🔗
|
underscor |
I was looking for it before |
|
05:22
🔗
|
Daevien |
shodul see if we can get a number of users to change their avatar to that :) |
|
05:22
🔗
|
underscor |
t SketchCow where's your rubberglove picture? |
|
05:22
🔗
|
Daevien |
yeah i looked for it to send to a friend that was liek wtf are you doing? :p |
|
05:22
🔗
|
underscor |
lol |
|
05:23
🔗
|
Daevien |
since then he's been checkign the counter while he programs lol |
|
05:24
🔗
|
Daevien |
has SketchCow noticed how fast it's going yet or will he be surprised when he gets back to a connection? :p |
|
05:25
🔗
|
underscor |
he doesn't know |
|
05:25
🔗
|
underscor |
but he probably won't be too surprised |
|
05:25
🔗
|
Daevien |
heh it's still a ton of data in this time period |
|
05:25
🔗
|
underscor |
archiveteam tends to do things like this :D |
|
05:25
🔗
|
Daevien |
i usually miss the stuff when its going on or too busy |
|
05:26
🔗
|
Nintendud |
Can we have a bat signal |
|
05:26
🔗
|
Nintendud |
batsignal@archiveteam.org mailing list? |
|
05:26
🔗
|
underscor |
we have a mailing list now! |
|
05:26
🔗
|
Nintendud |
:O |
|
05:26
🔗
|
underscor |
see topic |
|
05:26
🔗
|
chronomex |
o_o |
|
05:26
🔗
|
Nintendud |
oh right |
|
05:26
🔗
|
Nintendud |
<-- blind |
|
05:29
🔗
|
underscor |
https://groups.google.com/d/msg/archive-team/VloK-lYGA9Q/5GvMs9khL1gJ |
|
05:29
🔗
|
underscor |
better late than never |
|
05:33
🔗
|
Daevien |
nearing 500 left |
|
05:34
🔗
|
chronomex |
lame, I had to reset the munin bytes & items graphs |
|
05:38
🔗
|
Daevien |
hmm, crap. guess i'm about out of thigns to do and shodul go to bed here in a sec when this finishes :p |
|
05:39
🔗
|
Daevien |
er, negative counter. interesting |
|
05:39
🔗
|
chronomex |
?? |
|
05:39
🔗
|
chronomex |
oh |
|
05:40
🔗
|
chronomex |
yeah, that happens |
|
05:40
🔗
|
DFJustin |
and it's a cinch |
|
05:40
🔗
|
chronomex |
ha |
|
05:40
🔗
|
DFJustin |
didn't even have time to get a full gigabyte :( |
|
05:41
🔗
|
chronomex |
Daevien: the negative is because one counter is "number unclaimed" and the other is "number finished", but a finished event decrements the unclaimed counter in the ui |
|
05:41
🔗
|
Daevien |
got one goign with 500mb and still at 70% lol |
|
05:44
🔗
|
S[h]O[r]T |
underscor :( |
|
05:44
🔗
|
S[h]O[r]T |
i thought i could make a come back and beat u |
|
05:44
🔗
|
chronomex |
think again! |
|
05:44
🔗
|
chronomex |
underscor generally wins |
|
05:44
🔗
|
Daevien |
heh you'd have had to open some big pipes, he was cruising |
|
05:44
🔗
|
underscor |
:D |
|
05:44
🔗
|
underscor |
<3 you all though |
|
05:44
🔗
|
S[h]O[r]T |
i did open big pipes |
|
05:45
🔗
|
S[h]O[r]T |
i just didnt notice and get everything going until too late |
|
05:45
🔗
|
S[h]O[r]T |
had 30 on one server, 100 on another, 200 on another, and 500 on another :P |
|
05:45
🔗
|
BlueMax |
underscor is a big fat cheat :| |
|
05:45
🔗
|
underscor |
S[h]O[r]T: the problem with that is that you get cpustarved |
|
05:46
🔗
|
underscor |
:P |
|
05:46
🔗
|
underscor |
because you have 450 runnable threads |
|
05:46
🔗
|
S[h]O[r]T |
not really |
|
05:46
🔗
|
S[h]O[r]T |
not running on a vps |
|
05:46
🔗
|
S[h]O[r]T |
all corequads except 1 machine |
|
05:46
🔗
|
underscor |
what limit do you hit then? |
|
05:46
🔗
|
S[h]O[r]T |
nothing |
|
05:46
🔗
|
underscor |
well, probably upstream to fos actually |
|
05:46
🔗
|
S[h]O[r]T |
the limit of rsyncing to foss i guess since i dont get more than 1-3MB/s |
|
05:47
🔗
|
S[h]O[r]T |
but its not because pipe is full |
|
05:47
🔗
|
underscor |
sinec you can only have one running at a time |
|
05:47
🔗
|
underscor |
since* |
|
05:47
🔗
|
S[h]O[r]T |
yup |
|
05:47
🔗
|
Daevien |
689mb file landed heh. someone had a lot to talk about |
|
05:47
🔗
|
S[h]O[r]T |
so if i hit a big file with this it stops |
|
05:47
🔗
|
underscor |
you can only have one rsync across all workthreads at a time |
|
05:47
🔗
|
underscor |
yep |
|
05:47
🔗
|
underscor |
:( |
|
05:47
🔗
|
S[h]O[r]T |
not a huge deal its done :p but yeah |
|
05:47
🔗
|
S[h]O[r]T |
i beat you on picplz and tabloo i think though |
|
05:48
🔗
|
chronomex |
haha |
|
05:49
🔗
|
underscor |
probably |
|
05:49
🔗
|
underscor |
but I didn't have a lot of spare boxen for them |
|
05:49
🔗
|
underscor |
:) |
|
05:49
🔗
|
underscor |
this was just good timing |
|
05:53
🔗
|
Daevien |
underscor: you flipped over to 50gb :p |
|
05:53
🔗
|
underscor |
:D |
|
05:53
🔗
|
* |
BlueMax throws a ball at underscor |
|
05:53
🔗
|
* |
underscor catches, returns |
|
05:56
🔗
|
Dark_Star |
hmm... still >2400 to do (according to the tracker) but I keep getting "no item received"... |
|
05:56
🔗
|
Dark_Star |
ah. |
|
05:57
🔗
|
Dark_Star |
now the tracker shows 0 left. |
|
05:57
🔗
|
Nintendud |
I have a HUGE upload going |
|
05:57
🔗
|
Nintendud |
this is like over 1.5GB |
|
05:57
🔗
|
chronomex |
yow |
|
05:57
🔗
|
chronomex |
cool |
|
05:57
🔗
|
Daevien |
Dark_Star: it seems to be peopel with a lot of processes sending the last bits in |
|
05:57
🔗
|
Nintendud |
wwwpubliceyeimagingcom |
|
05:57
🔗
|
Dark_Star |
after a refresh. I think something in the tracker's code is broken :) |
|
05:57
🔗
|
chronomex |
it's not broken, it works fine |
|
05:57
🔗
|
chronomex |
it has weirdness though |
|
05:58
🔗
|
Daevien |
woot, i managed to just push above you in gb Dark_Star heh |
|
05:58
🔗
|
Dark_Star |
well I had the window open since yesterday and it kept showing 2300 items left, still counting down. when I pressed F5, it now shows 0 items and no progress |
|
05:58
🔗
|
Dark_Star |
@Davien :-P |
|
06:00
🔗
|
Dark_Star |
well that was an easy project... what's next? :) |
|
06:00
🔗
|
Daevien |
once it got going and i noticed where i was int he list, i wanted to stay in the top 10 hehe, stupid things i do late at night when bored :p |
|
06:01
🔗
|
Dark_Star |
yeah, well, I couldn't do much against it, my upload speed is only about 5% of my download speed... while I could have grabbed more, it would have taken a few days to upload it all :) |
|
06:02
🔗
|
Cameron_D |
Same here, I upload at about 30kb/s tops :( |
|
06:02
🔗
|
Daevien |
yeah my own connection is usually about 1mbit max even though isp claims 2 |
|
06:02
🔗
|
Nintendud |
I'm using a 100mbit box, but... 6 threads :( |
|
06:02
🔗
|
Cameron_D |
I used to get 1mbit, I now really miss that |
|
06:02
🔗
|
Daevien |
i had a ton of bandwidth quota on two vps doing nothing that rolls over in a weekish so put it to some use :p |
|
06:03
🔗
|
Dark_Star |
I have 100mbit downstream as well, and I ran 2 VMs with 6 threads each, but it doesn't help when all the downloads are constantly finished and waiting for the uploads :-( |
|
06:03
🔗
|
Dark_Star |
I want something more symmetrical. FTTH would be nice |
|
06:03
🔗
|
Nintendud |
this account I'm uploading is like.. 2GB |
|
06:04
🔗
|
Nintendud |
yeah |
|
06:04
🔗
|
Daevien |
i can get 100 down, 30 up if i paid the money with my isp.. but all packages besides my 20/2 (20/1 really) are with bandwidth caps... my 20 is not. so i stick with it ;) |
|
06:04
🔗
|
Cameron_D |
symmetrical fibre would be nice |
|
06:05
🔗
|
Daevien |
thats another option in my area, but i haven't heard that great of feedback about it here |
|
06:05
🔗
|
Cameron_D |
Well, anything that is more than .3mbps up would be nice |
|
06:05
🔗
|
Dark_Star |
there's no fiber here, sadly. at least no affordable fiber |
|
06:05
🔗
|
Cameron_D |
Australia's national fibre network is kinda promising |
|
06:06
🔗
|
Daevien |
plus the company that ffers teh fibre is one of the countries biggest pro-bandwidth capping supporters so i don't like them |
|
06:06
🔗
|
Cameron_D |
But the data caps, not so much |
|
06:06
🔗
|
Cameron_D |
I think the best plan at the moment is 100/40mbps with a 1tb cap |
|
06:08
🔗
|
Nintendud |
I can apparently only upload at 1MB/s to the box that;s hosting the cinch.fm stuff |
|
06:08
🔗
|
Nintendud |
in one connection |
|
06:09
🔗
|
Daevien |
gah, my isp nuked their big package.. 100 down, 5 up, 250gb cap. bleh. |
|
06:09
🔗
|
Daevien |
and "only" $165 canadian a month :p |
|
06:10
🔗
|
Daevien |
other company offering 250 down, 30 up, with supposedly no cap but liek i said they are in favor of them so makes me leery.. $250/month |
|
06:15
🔗
|
S[h]O[r]T |
1 box left finishing up |
|
06:17
🔗
|
Daevien |
https://twitter.com/textfiles/status/237790179090374657 |
|
06:19
🔗
|
Nintendud |
that's me |
|
06:19
🔗
|
Nintendud |
2GB+ upload |
|
06:20
🔗
|
Nintendud |
13 mins or so remaining |
|
06:20
🔗
|
Daevien |
Nintendud: slowass :p |
|
06:21
🔗
|
Nintendud |
1MB/s :3 |
|
06:21
🔗
|
Nintendud |
from france or something |
|
06:22
🔗
|
Daevien |
heh not sure where your server is even? :p |
|
06:24
🔗
|
chronomex |
hey, it happens |
|
06:25
🔗
|
Nintendud |
Well, I'm billed from France, IP is Ireland |
|
06:25
🔗
|
Nintendud |
I think the main IP is french, |
|
06:25
🔗
|
Nintendud |
:3 |
|
06:26
🔗
|
Nintendud |
cheapest dedicated server I could get. works as a minecraft server. |
|
06:26
🔗
|
Daevien |
ovh or one of the resellers? |
|
06:27
🔗
|
chronomex |
Daevien: http://www.qdb.us/5273 |
|
06:27
🔗
|
Nintendud |
ovh |
|
06:28
🔗
|
Nintendud |
yeahhh 2443MB |
|
06:28
🔗
|
Daevien |
chronomex: heh i've had that happen before. stuffed an old box in the closet and then stuff got piled onto it over time.. it was my router at the time, hd finally blew up one day and took me a bit to find it |
|
06:28
🔗
|
chronomex |
yep |
|
06:28
🔗
|
Daevien |
i finally gave up and chased it down via network cable :p |
|
06:29
🔗
|
BlueMax |
Bahahahaha |
|
06:29
🔗
|
BlueMax |
http://i.imgur.com/9zVkp.png |
|
06:29
🔗
|
Daevien |
haha |
|
06:38
🔗
|
Daevien |
looks like bsmith094 got the last one, hasn't moved in a bit |
|
06:43
🔗
|
underscor |
dammit |
|
06:43
🔗
|
underscor |
Guess it's going in -bs |
|
06:45
🔗
|
chronomex |
heh |
|
07:43
🔗
|
Coderjoe |
i've heard stories of a long-running server that wound up lost in some hidden room where the door had been plastered/drywalled over in a remodel |
|
07:43
🔗
|
chronomex |
classic story, tat |
|
07:43
🔗
|
chronomex |
*that |
|
07:46
🔗
|
ersi |
yeah, urban legend |
|
07:52
🔗
|
Coderjoe |
here's one instance: http://www.theregister.co.uk/2001/04/12/missing_novell_server_discovered_after/ |
|
08:19
🔗
|
godane |
uploaded: http://archive.org/details/filesharefreak.com-20120820-mirror |
|
08:31
🔗
|
kennethre |
can my services be of assistance with cinch.fm? |
|
08:47
🔗
|
alard |
kennethre: You're too late (as am I). They've eaten everything already. |
|
08:47
🔗
|
kennethre |
hehe, a good problem to have :) |
|
08:47
🔗
|
kennethre |
maybe there should be a mailing list? |
|
08:48
🔗
|
alard |
There's a little bit of cleanup left, but that's all. |
|
08:48
🔗
|
chronomex |
more work units to come tomorrow, apparently |
|
08:48
🔗
|
alard |
Oh, where from? |
|
08:48
🔗
|
chronomex |
idk, I think SketchCow said something |
|
08:55
🔗
|
chronomex |
I could be wrong thoug |
|
08:55
🔗
|
alard |
He may have meant the requeue-what's-not-returned phase. |
|
08:56
🔗
|
chronomex |
yeah |
|
08:56
🔗
|
alard |
And we could always do a second run, of course. |
|
08:57
🔗
|
alard |
By the way, I've started made notes of some of the warrior feature requests. If there's anything missing, please add it here: https://github.com/ArchiveTeam/seesaw-kit/issues |
|
09:23
🔗
|
GLaDOS |
alard: did you reset the users that I horribly failed at downloading? |
|
09:24
🔗
|
alard |
GLaDOS: You had quite a few, yes. :) I've requeued everything, they're almost gone now. |
|
09:24
🔗
|
GLaDOS |
Yeah, turns out I forgot to install liblua on that VPS. |
|
09:40
🔗
|
SmileyG |
Awesome, I was right about using TC :) |
|
10:47
🔗
|
chronomex |
alard: I made up a unified munin thing for several of my hosts today |
|
10:47
🔗
|
chronomex |
new home of graphs for shilling is at http://zeppelin.xrtc.net/corp.xrtc.net/shilling.corp.xrtc.net/index.html |
|
10:49
🔗
|
chronomex |
the old data is still cascading into the new graphs, so things are not completely settled yet |
|
10:49
🔗
|
chronomex |
also goddamnit today I was not going to stay up to 4am |
|
10:49
🔗
|
chronomex |
next week, perhaps I will get some kind of vpn system up. |
|
11:47
🔗
|
ersi |
chronomex: #archiveteam-bs man |
|
11:47
🔗
|
ersi |
default there |
|
12:11
🔗
|
alard |
We have completed the first phase of Cinch.fm: the queue is empty, all outstanding tasks are finished. |
|
12:33
🔗
|
SmileyG |
:O |
|
12:33
🔗
|
SmileyG |
was that via warrior? |
|
12:36
🔗
|
SketchCow |
Just wanted to say...... |
|
12:36
🔗
|
SketchCow |
So we have that warrior, that consistent warrior |
|
12:37
🔗
|
SketchCow |
I'd prefer people not cobble together six pieces of rebar and a side of ham and some twist ties and join in the fun |
|
12:37
🔗
|
SketchCow |
And then we have to redo it because the environment isn't consistent. |
|
12:37
🔗
|
SketchCow |
That's no different than when we had the Windows do-gooders with their uppercase and their broken date strings |
|
12:40
🔗
|
SketchCow |
Just saying. |
|
12:40
🔗
|
SketchCow |
Now, as for phases, I mostly assumed we had the usual situation of there being low-hanging fruit, and then efforts to find the deeper, not-obvious-though-the previous-means users. |
|
12:41
🔗
|
SketchCow |
But are you really convinced cinch had 27,000 users? |
|
12:41
🔗
|
alard |
What we have now is the list of all users who had at least one recording in the sitemap. |
|
12:41
🔗
|
SketchCow |
OK. |
|
12:42
🔗
|
SketchCow |
So there's a chance more are out there. |
|
12:43
🔗
|
alard |
Yes. There may be users without recordings (they're less interesting to archive, of course) and there may be users who are not in the sitemap. |
|
12:54
🔗
|
SketchCow |
Could the whole site be 27,000 users and 178gb? That'd be sad. |
|
12:55
🔗
|
SketchCow |
Maybe do some google searches? |
|
12:56
🔗
|
alard |
79792 usernames-20120821.txt |
|
12:56
🔗
|
alard |
That's all the users in the sitemaps. |
|
12:56
🔗
|
SketchCow |
Huh. |
|
12:56
🔗
|
SketchCow |
The thing last night looked like it was looking at 27,000 things to download. |
|
12:56
🔗
|
SketchCow |
Was it not? |
|
12:56
🔗
|
SketchCow |
Was it 79,000? |
|
12:57
🔗
|
alard |
Yes, no. I made the first list by listing the recording urls, then extracting the usernames from those urls. So the first batch included only users with content. |
|
12:57
🔗
|
alard |
This second list is a list of every user. |
|
12:57
🔗
|
alard |
I've checked a few, every user we haven't done has zero recordings. |
|
12:58
🔗
|
alard |
I'll add the list, but I expect we won't find more recordings. |
|
12:58
🔗
|
SketchCow |
But, but the Scobelizer said Cinch was as important as twitter |
|
12:59
🔗
|
alard |
Did he? Okay, there we go: 50382 items in the queue. |
|
13:02
🔗
|
balrog_ |
I wish there was a way to participate without having to run a VM :/ |
|
13:02
🔗
|
alard |
There are only 561412 recordings/messages on Cinch. (Assuming they started counting at 1.) |
|
13:02
🔗
|
balrog_ |
(since while the VM is nice, it won't run on low power hardware that isn't x86) |
|
13:09
🔗
|
altlabel |
guys, I'm trying to upload some MobileMe data with the upload-finished script, but it's returning me an error: |
|
13:09
🔗
|
altlabel |
Uploading anitafranken |
|
13:09
🔗
|
altlabel |
An rsync error. Scary! |
|
13:09
🔗
|
altlabel |
@ERROR: Unknown module 'mobileme' |
|
13:09
🔗
|
altlabel |
rsync error: error starting client-server protocol (code 5) at main.c(1524) [sender=3.0.7] |
|
13:10
🔗
|
altlabel |
...what can it be? |
|
13:11
🔗
|
alard |
That's probably because the upload space was closed down. |
|
13:15
🔗
|
altlabel |
ah, ops. ..will it be re-opened? |
|
13:18
🔗
|
alard |
No, I don't think so. It's probably easiest if you can make a tar file (don't gzip) of the files you have, then ask SketchCow where to upload that tar file. |
|
13:19
🔗
|
altlabel |
ok, thanks alard |
|
13:19
🔗
|
alard |
We're more or less done mobileme, so a lot of the automated things have stopped. But every bit helps. |
|
13:22
🔗
|
alard |
Okay, there is at least one unlisted recording on Cinch. |
|
13:23
🔗
|
alard |
This http://cinch.fm/jeremy-millers/100013 is not in the sitemap, and not listed on the user's page (the user is in the sitemap). |
|
13:24
🔗
|
alard |
The sitemap lists 182841 recordings. We know there are less than 561412. |
|
13:29
🔗
|
alard |
Another one: http://cinch.fm/lingonaut/123sesame/240001 with an apparently unlisted album http://cinch.fm/lingonaut/123sesame |
|
13:36
🔗
|
swebb |
I made an archiveteam-batsignal mailing list if people want to sign up: http://badcheese.com/cgi-bin/mailman/listinfo/archiveteam-batsignal |
|
14:09
🔗
|
SketchCow |
OK, I'm out in the cube for a while this morning. |
|
14:21
🔗
|
alard |
We should probably do a second run of Cinch.fm, number-based this time. |
|
14:28
🔗
|
Schbirid |
Nintendud: you gonna up "The [INCOMPLETE] PC Gamer Demo/Sampler Collection [Demo/Sampler/2001]" to archive.org? |
|
14:34
🔗
|
DFJustin |
it's all (or almost all) up already isn't it |
|
14:39
🔗
|
SmileyG |
Wooo, it shows activity in the webpage now! |
|
14:40
🔗
|
SmileyG |
zomg how many uploads is it gonna try and do? |
|
14:40
🔗
|
SmileyG |
Holy cow this is cool :) I'd not seen it working on anything but tumblr which obviously took forever... |
|
14:42
🔗
|
DFJustin |
all these new users seem to be 1mb |
|
14:48
🔗
|
SmileyG |
Always 25 urls... |
|
14:53
🔗
|
swebb |
How do I get to a console on the archiveteam warrior v2? |
|
15:02
🔗
|
SmileyG |
Ooo I got a 50url one. |
|
15:02
🔗
|
SmileyG |
swebb: why do you want a console? |
|
15:03
🔗
|
swebb |
Just curious. I wanted to know the IP that the machine was running on. An ssh connection would also be good. I don't like not being able to see what's running on my machine. |
|
15:03
🔗
|
swebb |
Also, it seems to be crashing every now and then. |
|
15:04
🔗
|
alard |
swebb: Give focus to the virtual machine, then press Alt+F3. |
|
15:04
🔗
|
swebb |
I tried that. |
|
15:05
🔗
|
alard |
And? |
|
15:05
🔗
|
swebb |
(rebooting now) |
|
15:05
🔗
|
SmileyG |
lol |
|
15:05
🔗
|
swebb |
I'll try again in a sec. |
|
15:05
🔗
|
swebb |
The VM crashed, so I had to reboot it. |
|
15:05
🔗
|
SmileyG |
swebb: Reason I asked btw was to check you'd read the instructions. |
|
15:05
🔗
|
alard |
You should see a normal login screen, saying "the root password is archiveteam" |
|
15:05
🔗
|
swebb |
Ahh, I got it now. |
|
15:06
🔗
|
swebb |
and yes, I read the instructions |
|
15:06
🔗
|
swebb |
thanks for belittling me though. |
|
15:06
🔗
|
alard |
Alt+F2 should show output from the system. |
|
15:06
🔗
|
alard |
Any error messages might pop up there. |
|
15:06
🔗
|
alard |
The thing doesn't have an ssh server. |
|
15:07
🔗
|
alard |
(But you can apt-get install that, of course.) |
|
15:07
🔗
|
SmileyG |
swebb: Ok, I do lots of support, lots of users don't read instructions and say they've done things when they haven't. Don't hate for trying to make both mine and your life easier :( |
|
15:08
🔗
|
swebb |
Hung again. No error on tty #2. |
|
15:08
🔗
|
alard |
What happens on tty #1? |
|
15:09
🔗
|
alard |
Does it get to the splash screen image? |
|
15:09
🔗
|
swebb |
Yea, it got there, and it ran 51 fetches (I'm running 4 concurrent processes). |
|
15:10
🔗
|
alard |
But when does it hang, exactly? |
|
15:10
🔗
|
swebb |
It started again. |
|
15:10
🔗
|
swebb |
The web browser says "lost connection to the warrior" in a red bar along the top of the web page. |
|
15:11
🔗
|
swebb |
The console still works but the web guy has that red bar after crawling for a short while. |
|
15:11
🔗
|
swebb |
s/guy/gui/g |
|
15:11
🔗
|
alard |
That happens as soon as you restart the warrior. It may not reconnect immediately when it comes back up. The guy checks every now and then, but you should refresh the page if you want it to go faster. |
|
15:12
🔗
|
alard |
Maybe this is something for #warrior, if there's more. |
|
15:13
🔗
|
swebb |
Screenshot: https://skitch.com/scumola/ep4in/skitch |
|
15:13
🔗
|
swebb |
I'm seeing errors now on tty #2 |
|
15:14
🔗
|
swebb |
File "/usr/local/lib/python2.6/dist-packages/tornado/websocket.py", line 258, in wrapper ... |
|
15:14
🔗
|
swebb |
HTTPError(401, "Invalid Session") |
|
15:34
🔗
|
DoubleJ |
SketchCow: re 8:37, I diasgree. At some point we'll have another large project that will require those of us with Linux boxes but who can't/won't run the warrior. I'd rather shake out any problems with things like Cinch, where we can re-download everything in udner a day, than lose time on a project that has none. |
|
15:40
🔗
|
SketchCow |
RE 8:37 is pretty oof |
|
15:41
🔗
|
SketchCow |
I'd rather just recruit more warriors and let the linux people be doing other things than just straight up/download |
|
15:42
🔗
|
swebb |
I made an archiveteam-batsignal mailing list if people want to sign up: http://badcheese.com/cgi-bin/mailman/listinfo/archiveteam-batsignal |
|
15:43
🔗
|
SketchCow |
What is its purpose |
|
15:43
🔗
|
SketchCow |
That the IRC channel doesn't solve |
|
15:43
🔗
|
swebb |
Send an email to the mailing list as a "call to action" to notify people if/when they're not on the irc channel. People were talking about it, so I made one. |
|
15:43
🔗
|
SketchCow |
OK. |
|
15:44
🔗
|
swebb |
aka, "new project - start your downloaders" |
|
15:44
🔗
|
SketchCow |
Let's go further on the linux vs. warrior |
|
15:44
🔗
|
SketchCow |
It's the windows argument ALL over again |
|
15:44
🔗
|
SketchCow |
it's MUCH better to have a consistent environment. We can have one. We can just get more people involved on that side. |
|
15:44
🔗
|
SketchCow |
The side itself is the most unthinking part of archiveteam - running a program that uploads. |
|
15:45
🔗
|
SketchCow |
It's better it be produced consistently, so down the line we don't discover someone forgot to install lksdhsiodufhsdaioufhasdfhdf.h and it produces stuff that's "just a little different" |
|
15:46
🔗
|
* |
SketchCow is firing up the warrior before heading out to do more infocube |
|
15:46
🔗
|
SketchCow |
Wh aren't any of you fuckers near Fishkill NY |
|
15:46
🔗
|
SketchCow |
Totally need help |
|
15:46
🔗
|
* |
SketchCow waves fist |
|
15:47
🔗
|
SketchCow |
I'm so old, and here I am lifting boxes |
|
15:47
🔗
|
* |
SketchCow calls amnesy international |
|
15:47
🔗
|
SketchCow |
amnesy |
|
15:49
🔗
|
SketchCow |
Loving the warrior |
|
15:49
🔗
|
SketchCow |
By the way |
|
15:49
🔗
|
SketchCow |
It's tv/movie quality |
|
15:56
🔗
|
ersi |
The only thing that's "bad" with the warrior is that it's hard to scale. It's really great for distributing ONE machine to everyone - but I can't spin one up everywhere. Scripts I can fire in a screen anywhere |
|
16:02
🔗
|
swebb |
ersi: agreed |
|
16:02
🔗
|
swebb |
screen is the shit. |
|
16:04
🔗
|
SmileyG |
can't you grab the collection of scripts which make up warrior and use them? |
|
16:04
🔗
|
SmileyG |
but don't tell anyone? |
|
16:04
🔗
|
SmileyG |
Oh wait, that kilsl the clean system idea. |
|
16:04
🔗
|
* |
SmileyG goes off to think |
|
16:04
🔗
|
swebb |
having the scripts in github is a good start and seems to work for most people who don't run the warrior image. |
|
16:05
🔗
|
swebb |
Most people run ubuntu anyway. |
|
16:05
🔗
|
SmileyG |
yeah but it negates the whole point. |
|
16:05
🔗
|
swebb |
I understand the idea of the black box. It's a great idea. |
|
16:05
🔗
|
SmileyG |
Not just being a blackbox though, its a _Stable_ blackbox. |
|
16:06
🔗
|
swebb |
but for people with available resources, it's easy to pull a script onto 10 machines here and there and just start them all up. With the warrior image, I need a VM to run it on somewhere. |
|
16:07
🔗
|
SmileyG |
yeah |
|
16:07
🔗
|
SmileyG |
make a binary... that way theres no "zomg wrong lib" issues... |
|
16:07
🔗
|
swebb |
Yea, a single binary would be a good idea. |
|
16:07
🔗
|
swebb |
statically linked |
|
16:08
🔗
|
SmileyG |
indeed. |
|
16:08
🔗
|
swebb |
The whole linux/warc/rsync combination should be a standard-enough platform though. |
|
16:10
🔗
|
SmileyG |
well warc is only in some versions of wget... |
|
16:10
🔗
|
swebb |
Yea, I know. |
|
16:11
🔗
|
swebb |
Don't get me wrong, I think that the tracker and warrior stuff is really well done and I admire the work, but personally, I can only run it on a few machines with VirtualBox on them. I can't run it on any of my servers. |
|
16:15
🔗
|
balrog_ |
SmileyG: warc is in all latest versions of wget |
|
16:15
🔗
|
balrog_ |
it's a standard part of wget 1.14, which was released |
|
16:17
🔗
|
Deewiant |
Two weeks ago, so most distributions probably don't have it yet(?) |
|
16:17
🔗
|
balrog_ |
most do, though some in unstable repos |
|
16:18
🔗
|
Deewiant |
Well yeah, I guess I should have said "most users" |
|
16:18
🔗
|
balrog_ |
ubuntu doesn't though, but of course there's probably a PPA |
|
16:30
🔗
|
alard |
So you'd need a statically linked Wget+Lua (in this case), plus the Python stuff in an easy to handle form. |
|
16:32
🔗
|
swebb |
Perhaps a package (dpkg or rpm) that will handle all of the dependancies to ensure a compatible environment? |
|
16:33
🔗
|
alard |
Yes. (Although that still gives you a lot of different distributions and versions to deal with.) |
|
17:37
🔗
|
nitro2k01 |
http://aws.amazon.com/glacier/ |
|
17:39
🔗
|
nitro2k01 |
Long term storage for $0.01/GB/mo. Access time, hours. Amazon are probably trying tapes handled by robots. |
|
17:40
🔗
|
Ymgve |
eh, $10 per terabyte per month is still a bit |
|
17:45
🔗
|
swebb |
and there's a penalty if you delete the data in less than 3 months. |
|
17:46
🔗
|
swebb |
"Amazon is encouraging customers to use Glacier for archival, rather than short-term, storage. If information is deleted within three months of being uploaded, there is an "early deletion fee", it said." |
|
17:47
🔗
|
swebb |
upload is free. storage is $0.01/GB/mo. download is $0.12/GB |
|
17:50
🔗
|
Ymgve |
literally holding your data prisoner |
|
18:01
🔗
|
yipdw |
Ymgve: only if they force you to delete your other copies :P |
|
18:02
🔗
|
* |
yipdw will probably be using Glacier as off-site backup for a photo archive |
|
18:02
🔗
|
yipdw |
it's not easy to efficiently store 200 gigabytes of images |
|
18:03
🔗
|
yipdw |
well |
|
18:03
🔗
|
yipdw |
it is easy. it's not *cheap* :P |
|
18:04
🔗
|
yipdw |
and $2/month is an order of magnitude less than what I'm paying now for an off-site backup for that |
|
18:05
🔗
|
SketchCow |
Hug hug |
|
18:09
🔗
|
Ymgve |
yipdw: yeah, then it's not too bad |
|
18:10
🔗
|
yipdw |
I guess the other solution is "prune your damn archive" |
|
18:10
🔗
|
yipdw |
which works, but is really a short stopgap solution |
|
18:10
🔗
|
yipdw |
frickin 5D mkII generates huge outputs no matter what you do, unless you shoot in sRAW/small JPEG or something |
|
18:11
🔗
|
swebb |
Flicker pro account is $30/yr which is just a little more than $2/mo |
|
18:12
🔗
|
SmileyG |
gah, now your making me tempted to write a "warrior-script" ebuild for gentoo ¬_¬ |
|
18:12
🔗
|
yipdw |
Flickr's a possibility -- I just have more trust in Amazon than Yahoo, which is important whilst I only have one off-site backup |
|
19:19
🔗
|
swebb |
SmileyG: re: Gentoo, geek points +1! I've not built a Gentoo box for over 5 years. Is Gentoo still maintained and releasing new stuff still? |
|
19:19
🔗
|
ersi |
Fuck, how hard is it to remember #archiveteam-bs? |
|
19:19
🔗
|
* |
ersi gives nitro2k01 the cold stare |
|
19:33
🔗
|
SmileyG |
swebb: oh yes, and if you want to discusss this futher lets go to #archiveteam-bs :D |
|
19:44
🔗
|
underscor |
it would be cool if there was an openphoto connector for glacier |
|
20:00
🔗
|
chronomex |
woop woop woop off-topic siren |
|
20:03
🔗
|
underscor |
D: |
|
20:41
🔗
|
swebb |
I'm getting "Tracker returned status code 455" now from my client. |
|
20:43
🔗
|
alard |
Yes. I've just started the second round of Cinch.fm. That requires a new version of the scripts. If you're running a warrior, it should update itself (it probably has done that by now). |
|
20:44
🔗
|
alard |
If you're not running a warrior, you should do the update yourself: |
|
20:44
🔗
|
alard |
1. install the latest version of the seesaw-kit with pip |
|
20:44
🔗
|
alard |
2. git pull on the cinch-grab repository |
|
20:46
🔗
|
alard |
455 is code for "you're too old, please upgrade". (The new version of the seesaw-kit has a better error message.) |
|
20:46
🔗
|
chronomex |
is an auto-updating seesaw-kit in the roadmap? |
|
20:48
🔗
|
alard |
No, but feel free to look into it. It's auto-updating on the warrior, but not standalone. It probably requires extensive sudo permissions, for instance, and it depends on how you've installed the thing. |
|
20:48
🔗
|
chronomex |
ah, an auto-updating warrior is all I was asking about |
|
20:48
🔗
|
alard |
(Of the two upgrade steps #2, the project-specific upgrade, is probably easier and more frequent than #1.) |
|
20:49
🔗
|
alard |
Your warrior is fine. It might take up to an hour for the warrior to realize it's out of date, but then it will reboot. |
|
20:56
🔗
|
case_ |
anyone have experience using BagIt? http://www.dcc.ac.uk/resources/external/bagit-library |
|
21:30
🔗
|
godane |
http://arstechnica.com/gaming/2012/08/source-nintendo-power-magazine-to-cease-publication/ |
|
21:30
🔗
|
godane |
we maybe able to get all of nintendo power pdfs up in a year |
|
21:42
🔗
|
Nemo_bis |
does the deriver digest even WebEx arf format? |
|
22:04
🔗
|
Nemo_bis |
no it doesn't |
|
22:05
🔗
|
chronomex |
aw fuck I just got a google result pointing to splinder |
|
22:05
🔗
|
chronomex |
why do these fucks keep breaking the internet |
|
22:38
🔗
|
underscor |
alard: http://p.defau.lt/?nkK_A5hDzvu_VGi7KrfVLg |
|
22:39
🔗
|
underscor |
oh, seesaw-kit needs updating too |
|
22:39
🔗
|
underscor |
damn |
|
22:39
🔗
|
underscor |
nevermind |
|
22:42
🔗
|
* |
Daevien ducks for cover, underscor has started up again :p |
|
22:42
🔗
|
underscor |
:D |
|
22:49
🔗
|
Daevien |
gah, my mind has converted fully to linux.. command prompt on windows and it took me 2 tries to realize why ls wasn't working |
|
22:53
🔗
|
balrog_ |
lol |
|
22:53
🔗
|
balrog_ |
happens to me all the time |
|
22:54
🔗
|
Daevien |
doesn't help that with my laptop having a ssd, i can flip completely from one to another within seconds and forget which i'm in until i pull up the taskbar :p |
|
22:56
🔗
|
underscor |
haha |
|
22:56
🔗
|
underscor |
solution is to not use windows at all |
|
22:56
🔗
|
underscor |
:D |
|
22:57
🔗
|
Daevien |
i mostly have windows for a game or two but sometimes it's useful |
|
22:58
🔗
|
chronomex |
woop woop woop off-topic siren |
|
22:59
🔗
|
Daevien |
:p |