#archiveteam 2011-11-26,Sat

↑back Search

Time Nickname Message
00:01 🔗 Coderjoe I see a number of 500 errors on italian profiles
00:01 🔗 Coderjoe though I suppose those are blocked by the .incomplete check
00:02 🔗 Coderjoe hmm
00:02 🔗 Coderjoe data/it/m/ma/man/mani1985/wget-phase-3-0f08s3.splinder.com.log:2011-11-22 04:03:01 ERROR 504: Gateway Time-out.
00:02 🔗 Coderjoe data/it/m/ma/man/mani1985/wget-phase-3-0f08s3.splinder.com.log:2011-11-22 04:38:35 ERROR 502: Bad Gateway.
00:04 🔗 Coderjoe possibly also masked by a .incomplete file
00:10 🔗 Coderjoe yipdw^: I added a file to my gist with the times of the noconn pages
00:27 🔗 Coderjoe hmm. should the script only check italian profiles for splinder_noconn.html?
00:29 🔗 closure Wyatt: in case you see this, you still need to upload your Belios grab; get a rsync slot from SketchCow
00:46 🔗 Wyatt|Wor closure: I'm aware. Also have some google groups stuff. I'll bring my external to work tomorrow
00:47 🔗 closure thanks. need to finish berlios soon
00:52 🔗 db48x alard: ./dld-me-com.sh: line 324: 6893 Segmentation fault (core dumped) $WGET_WARC -U "$USER_AGENT"
00:52 🔗 db48x alard: probably when my disk filled up
00:53 🔗 Wyatt|Wor closure: Yeah, meant to get that up two weeks ago before I left town. Sorry I've been behind on that.
01:00 🔗 SketchCow winr4r: Glad to hear I could help your buddy.
01:01 🔗 db48x 254M categories
01:01 🔗 db48x 4.0K letter.txt
01:01 🔗 db48x 53G poems
01:01 🔗 db48x [db48x@celebdil unified]$ du -chs *
01:01 🔗 db48x 4.0K robots.txt
01:01 🔗 db48x 120M site
01:01 🔗 db48x
01:04 🔗 db48x oooh: http://aktuell.ruhr-uni-bochum.de/pm2011/pm00386.html.en
01:08 🔗 Wyatt|Wor I need to study various MitM techniques more; I feel like there's a lot of ground to cover there.
01:20 🔗 db48x yea, it's the main way cryptographic protocols are broken
01:56 🔗 SketchCow Blasting older magazines into the archive now.
01:59 🔗 Wyatt|Wor SketchCow: So is this still the phase where you're aggregating things that are available from elsewhere?
02:00 🔗 closure Telling tracker that 'it:the_Godfather' is done.
02:18 🔗 SketchCow Yes, still DEEP in that.
02:18 🔗 SketchCow There's so many left to do.
02:19 🔗 NotGLaDOS Oh wow, it's still got to wind down on 3 users.
02:20 🔗 Wyatt|Wor That's really impressive how many disparate full magazine runs are floating around.
02:28 🔗 Wyatt|Wor What's the copyright status of these things? Has anyone come forward to C&D their magazine archive?
02:28 🔗 SketchCow Ask me another one
02:31 🔗 NotGLaDOS SketchCow: I have somewhere around, I don't know, a couple of hundred users.
02:32 🔗 NotGLaDOS I need an rsync slot.
02:32 🔗 Wyatt|Wor Ah, I see. ;)
02:32 🔗 NotGLaDOS wait, you CAN rsync while the script's wind- nevermind.
02:32 🔗 NotGLaDOS there's literally 3 users left before it's finished winding down.
02:33 🔗 Coderjoe i've been on 3 to 7 users for two days
02:33 🔗 Coderjoe (two different machines)
02:37 🔗 NotGLaDOS Crap.
02:38 🔗 NotGLaDOS Well, time to clean incomplete users up.
02:38 🔗 Coderjoe be careful. the ones still running are also incomplete
02:39 🔗 SketchCow Finish it, then we'll go
02:44 🔗 NotGLaDOS Right.
02:52 🔗 SketchCow http://www.archive.org/search.php?query=collection%3Acomputer-magazine-rack&sort=-publicdate
02:54 🔗 SketchCow This is going to be quite a spetacular set of variant magazines
03:12 🔗 SketchCow http://batcave.textfiles.com/spec/HappyComputer/Issue8405/Pages/HappyComputer840500001.jpg
03:13 🔗 PatC :D
03:18 🔗 db48x mmm, pepperoni
03:22 🔗 SketchCow Happy Computer was pretty awesome, loading them in now.
03:22 🔗 PatC You're on a roll today :D
03:23 🔗 SketchCow I'm on a roll every day.
03:23 🔗 SketchCow But not everyone see's the day's roll.
03:24 🔗 db48x 2453122
03:24 🔗 db48x [db48x@celebdil poems]$ time find -name '*.html' | wc -l
03:24 🔗 db48x real 134m16.059s
03:24 🔗 db48x sys 5m33.791s
03:24 🔗 db48x user 0m32.564s
03:25 🔗 db48x SketchCow: do you have anything more from lulupoetry, that you know of?
03:25 🔗 SketchCow Might show up
03:26 🔗 db48x ok, then I'll wrap this up
03:27 🔗 db48x will I be able to amend the existing item on archive.org, or do you have to do that since you created it?
03:30 🔗 SketchCow I can add anything
03:30 🔗 SketchCow I can change anything
03:30 🔗 SketchCow rename anything
03:30 🔗 SketchCow rederive anything
03:30 🔗 SketchCow BUT I CANNOT LOVE
03:30 🔗 SketchCow WHY CAN'T I LOVE
03:30 🔗 db48x aww
03:31 🔗 SketchCow ls
03:31 🔗 SketchCow WHY CAN'T I LS
03:31 🔗 DFJustin . ..
03:31 🔗 db48x 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014
03:34 🔗 db48x SketchCow: listen to Miserere by Gregorio Allegri; that will help with the love thing
03:35 🔗 db48x on the other hand, you would have to spend 10 minutes not archiving anything in order to listen to it
03:35 🔗 db48x so it's a trade-off
03:36 🔗 NotGLaDOS db48x: not really worth it.
03:36 🔗 NotGLaDOS Oh hey, it finished grabbing another user!
03:36 🔗 NotGLaDOS Approximately 20 minutes ago.
03:36 🔗 db48x heh
03:37 🔗 db48x alard: hey, are you about?
04:16 🔗 SketchCow http://www.archive.org/details/whichmicro-1983-04
04:33 🔗 yipdw^ git status
04:34 🔗 yipdw^ er
04:34 🔗 yipdw^ whoops
04:37 🔗 SketchCow So, dreamhost just took an enormous shit
04:39 🔗 underscor <Wyatt|Wor> What's the copyright status of these things? Has anyone come forward to C&D their magazine archive?
04:39 🔗 underscor USER WAS BANNED FOR THIS POST
04:39 🔗 Wyatt|Wor Dreamhost did?
04:40 🔗 Cameron_D They do that a lot
04:40 🔗 Cameron_D I don't think my VPS has had more than a week of uptime in the past 3 months
04:40 🔗 DFJustin my dreamhost site is up
04:40 🔗 Cameron_D Mine is most of the time but it jsut seems that the server crashes itself every few days
04:41 🔗 SketchCow A lot of mine are on the same server, which means down.
04:41 🔗 underscor Dreamhost is pretty fucked
04:41 🔗 underscor Their shit is pwned
04:42 🔗 Wyatt|Wor Care to elucidate? pwned you say?
05:13 🔗 SketchCow Yep, that machine is down
05:13 🔗 SketchCow Good job, boys
05:13 🔗 SketchCow I'll just keep doing my other tasks until then
05:14 🔗 SketchCow Down to 60 items in mailbox, this is a big deal
05:16 🔗 NotGLaDOS http://tools.ietf.org/html/rfc2324 we need to get this implemented.
05:17 🔗 Wyatt|Wor Weird, I thought they were using ceph storage nodes for everything.
05:17 🔗 NotGLaDOS I don't care if it's an aprils fools joke.
05:19 🔗 NotGLaDOS "418 I'm a teapot"
05:20 🔗 Wyatt|Wor Huh, that one may even be possible. To be sure, if people can implement RFC 1149 and put NetBSD on a toaster, surely this is within reach!
05:21 🔗 NotGLaDOS It must be!
05:22 🔗 Wyatt|Wor Huh, missed that one: "In an attempt to anthropomorphize the bit streams on countless physical layer networks throughout the world, we propose a TCP option to express packet mood [DSM-IV]."
05:25 🔗 underscor Wyatt|Wor: Just sites get hacked a lot on there
05:26 🔗 underscor cf archiveteam wiki
05:26 🔗 Cameron_D Never try and use RFC 1149 over IEEE 1394 as it results in dead carrier
05:27 🔗 SketchCow Yeah, they splatted this thing
05:28 🔗 Wyatt|Wor underscor: "sites get hacked" can mean a lot of things, though. People think they've been hacked when they install a malignant theme and it defaces their homepage.
05:28 🔗 underscor Right. I don't mean that kind
05:29 🔗 underscor I mean the "people installed scripts/modified shit in my public_html" from within the server
05:30 🔗 Coderjoe "OTHER people installed scripts/modified shit in my public_html"
05:33 🔗 Wyatt|Wor Ah, interesting. I wonder how they're doing that? Perhaps a variation on the symlink attacks we've been trying to find a good containment strategy for here.
05:35 🔗 Coderjoe cf: archiveteam wiki (seen here: http://multimedia.cx/eggs/cloaked-archive-wiki/ )
05:42 🔗 Wyatt|Wor And the compromise vector?
05:44 🔗 Coderjoe i don't know the vector.
05:45 🔗 Coderjoe but the start and location of the compromise:
05:45 🔗 Coderjoe May 13 11 22:17:23 <SketchCow> Ah, found it.
05:45 🔗 Coderjoe May 13 11 22:17:32 <SketchCow> WebStart.php:<? eval(gzuncompress(base64_decode('eF6lVttu2kAQ/ZU+VCKRqsrYJapV8ZA+YGqlqUJVX7aqIq8NoWTtoiQkmK/vztmF0DDrIPVlxrBn7mfHfhtdfPt8fvH9Z
05:50 🔗 underscor Wyatt|Wor: OH, do you work at dreamhost?
05:50 🔗 Wyatt|Wor underscor: No, but I work for a hosting company.
05:50 🔗 underscor Oh, good
05:51 🔗 underscor I was like, "Whoops, I'm a dick"
05:51 🔗 underscor hahaha
05:53 🔗 Coderjoe i don't use dreamhost, so I don't know what's going on there. pretty sure it is other people that have accounts with them, though.
05:53 🔗 Wyatt|Wor If I did I probably wouldn't be offended anyway. Security isn't the sort of thing you can bluff, bluster, and deny until it stops nagging you.
05:54 🔗 underscor Some companies think otherwise
05:55 🔗 Wyatt|Wor There's this interesting ongoing problem where +FollowSymlinks allows any user to view / and family.
05:56 🔗 underscor Yeah, I usually disable that
05:57 🔗 underscor Why not chroot it?
05:57 🔗 underscor I do that with my thttpd processes to shield that attack
05:58 🔗 underscor I think it's called mod_security
05:58 🔗 Coderjoe +FollowSymlinksIfOwnersMatch
05:59 🔗 underscor Wyatt|Wor: Jumpline, or a reseller of Jumpline?
05:59 🔗 Wyatt|Wor underscor: A few things. It can be reenabled in a local .htaccess. Symlinks can be uploaded already in place. Symlinks are provided by the kernel; ln isn't the only way to make them.
05:59 🔗 underscor <Coderjoe> +FollowSymlinksIfOwnersMatch
06:00 🔗 underscor Also, chroots fix that
06:00 🔗 underscor ln -s / would just link to the chroot
06:00 🔗 Coderjoe chroot is a kernel api. a process in a chroot cannot access stuff outside it (not inclding any kernel vulns that might be discovered)
06:00 🔗 Wyatt|Wor Coderjoe: http://httpd.apache.org/docs/2.2/mod/core.html#options
06:02 🔗 Coderjoe i know the options. check out AllowOverride
06:02 🔗 Wyatt|Wor There are patches for apache that turn FollowSymlinks into SymlinksIfOwnersMatch, but I haven't gotten clearance to deploy that yet
06:02 🔗 db48x and in a shared hosting environment like dreamhost, all the users on a machine share an apache instance
06:02 🔗 underscor I LOVE SECURITY LIKE THAT
06:02 🔗 db48x yea
06:03 🔗 Wyatt|Wor Yeah, it's not exactly tenable, but this is the reality I walked into.
06:03 🔗 db48x the option to make it check the permsissions is the way to go
06:04 🔗 Wyatt|Wor underscor: Jumpline itself. (Also Hostican, Christian Web Host, HostingRails, GemStream, OpenSourceHost, 100MegsWeb Hosting, Web Intellects, WebHost Giant, and possibly a couple more I've forgotten).
06:05 🔗 underscor That's...
06:05 🔗 underscor A lot of brands
06:06 🔗 Wyatt|Wor Yeah, tell me about it. :/
06:07 🔗 Wyatt|Wor I think the Gemstream and HostingRails brands are getting killed soon.
06:08 🔗 Wyatt|Wor Cannibalism as a business model has its drawbacks.
06:14 🔗 no2pencil That's quote worthy
06:14 🔗 underscor Do they do different things, or is it just all rebranded shared hosting?
06:15 🔗 no2pencil love it
06:16 🔗 Wyatt|Wor underscor: They're all formerly separate hosting companies. The Man shops around, buys a company, and we migrate them in.
06:16 🔗 Wyatt|Wor Apparently it's quite lucrative.
06:16 🔗 underscor Aha
06:16 🔗 underscor That's interesting
06:17 🔗 Wyatt|Wor Yeah, I was surprised too. But it's more common than I would have guessed; I hear Bluehost and Dreamhost both do a lot of that.
06:17 🔗 underscor I suppose it makes sense
06:18 🔗 underscor I wouldn't want my hosting transferred to either of those companies though
06:18 🔗 * underscor buys hosting from jumpline so he can directly complain to Wyatt|Wor in here
06:18 🔗 db48x heh
06:20 🔗 Wyatt|Wor Oh right, Coderjoe, AllowOverride would work, but apparently it would break a LOT of Joomla installs.
06:30 🔗 godane underscor: i can't get js-wikireader working
06:34 🔗 underscor http://imgur.com/gallery/Thedr
06:56 🔗 * SketchCow is down to 48 letters in inbox.
06:56 🔗 SketchCow Hurrrrrraaahhhhhhhhhhhhhh
06:57 🔗 DFJustin time to mailbomb sketchcow
06:58 🔗 Coderjoe http://imgur.com/gallery/53uLJ
06:59 🔗 SketchCow I just got rid of an e-mail
06:59 🔗 SketchCow I wrote this "here's what I'm up to"
07:00 🔗 SketchCow Someone just wrote, essentially, "So, I am hijacking your announcement to say I would love your talents in pointing out to me a bunch of historical textfiles related to bandits and being considered heroes".
07:00 🔗 SketchCow And he is a history student.
07:00 🔗 SketchCow Well, I decided at this late hour that my spending 20 years collecting all sorts of materials connected to these subjects to then have them all done by a search engine is PERHAPS ENOUGH WORK
07:01 🔗 SketchCow I sat on it a while before deciding.
07:01 🔗 underscor My suggestion to Jason is to delegate, delegate, delegate. Find someone else to build/dupe the torrent. Coordinate, but clearly point out who is actually in charge of each effort.
07:01 🔗 underscor For the donation/DVD stuff, I humbly recommend a part-time office helper. Maybe just a friend or family member, but having some help could make the difference. Of course, my expectations were always low since I know he does this work as a side project, but it would be nice to have a slightly better response rate.
07:01 🔗 SketchCow WHat where what.
07:02 🔗 * underscor wouldn't mind being a Jason Office Monkey™
07:02 🔗 underscor http://ascii.textfiles.com/archives/3395
07:05 🔗 underscor Acceptance, 2011-11-21, 11:43:00, LANSDOWNE, PA, 19050
07:05 🔗 underscor Arrive USPS Sort Facility, 2011-11-23, 05:33:00, KANSAS CITY, MO, 64121
07:05 🔗 underscor Processed through USPS Sort Facility, 2011-11-21, 21:17:00, PHILADELPHIA, PA, 19176
07:05 🔗 underscor what the fuck
07:05 🔗 underscor why would you go from pennsylvania to missouri to get to northern virginia?
07:06 🔗 chronomex question not the ways of shippers
07:06 🔗 kennethre All of this over a 1GB disk?
07:06 🔗 Wyatt|Wor 1TB, no?
07:06 🔗 kennethre (re: the escalation post)
07:06 🔗 kennethre ah
07:07 🔗 kennethre "I'd be mailing a 1gig external usb"
07:07 🔗 underscor He typoed
07:07 🔗 kennethre gotcha
07:07 🔗 chronomex yeah geocities = 0.9 gigabytes.
07:08 🔗 kennethre SketchCow: would love to hear if he responds
07:18 🔗 SketchCow He doesn't know about the post.
07:18 🔗 SketchCow He sent me mail saying if I didn't respond by Monday, he'd pull the trigger
07:21 🔗 kennethre just getting the the threats
07:21 🔗 SketchCow So wait
07:21 🔗 SketchCow You guys get a site if you go to ascii.textfiles.com?
07:21 🔗 chronomex I do
07:21 🔗 SketchCow Good
07:21 🔗 kennethre yes
07:22 🔗 SketchCow I will see if I can find the problem here
07:25 🔗 underscor 0 2:25AM:alex@alex-desktop:~/emcore 156 π curl -I ascii.textfiles.com
07:25 🔗 underscor HTTP/1.1 200 OK
07:25 🔗 underscor Kinda a shitty route to it though
07:34 🔗 DFJustin okay I got this js-wikireader thing, I produced a dump file, I installed the extension in chrome, I'm at the "add dumps" page, now how in the name of fuck do I add my dump
07:34 🔗 underscor Dunno, I'll try and figure it out with kevin tomorrow
07:34 🔗 underscor s/kevin/antimatter15/
07:45 🔗 Coderjoe what is js-wikireader?
07:49 🔗 instence_ is it possible to use the -k option (--convert-links) outside of the original execution? Like on a site you have already archived?
07:49 🔗 instence_ this is for wget
07:50 🔗 db48x nope
08:29 🔗 DFJustin https://github.com/antimatter15/js-wikireader
08:35 🔗 NotGLaDOS Great.
08:36 🔗 NotGLaDOS Aurora froze when i went to open a new tab, so now I have the right click menu stuck on my sc-
08:36 🔗 NotGLaDOS oh, just went.
10:12 🔗 Nemo_bis is someone able to fix this wget problem, please? http://archiveteam.org/index.php?title=Talk:Blogger
10:13 🔗 Coderjoe --exclude-domains=LIST comma-separated list of rejected domains.
10:13 🔗 Coderjoe -D, --domains=LIST comma-separated list of accepted domains.
10:13 🔗 Coderjoe also, seeing the logs or some sample page would help
10:13 🔗 Coderjoe and level=2 could be a problem
10:14 🔗 Nemo_bis I tried to accept those domains but they weren't followed
10:14 🔗 Nemo_bis I'm trying -m now
10:14 🔗 Coderjoe -m, --mirror shortcut for -N -r -l inf --no-remove-listing.
10:14 🔗 Nemo_bis yep
10:15 🔗 Nemo_bis which shows -l 2 is a problem, probably
10:15 🔗 Coderjoe all that does differently is turn on timestamps and set level to infinity
10:15 🔗 Nemo_bis yes
10:16 🔗 Nemo_bis but a thing I don't understand is: you can use --page-requisites, but how can you follow and download all and only the links from an embedded image, which usually go to the larger version of it?
10:16 🔗 Nemo_bis listing all the domains of such larger versions is not very handy
10:19 🔗 Nemo_bis perhaps one can play with --ignore-tags but I'm not able to
11:39 🔗 Wyatt|Wor Wow, so everything got really interesting at the end of this shift! The New Guy used passwd on a shared server. The carnage...rootkits everywhere.
11:42 🔗 NotGLaDOS Whee!
11:49 🔗 chronomex that fast, eh
11:51 🔗 * chronomex zzzz
11:53 🔗 Wyatt|Wor No, he changed it sometime yesterday from what we've been able to tell.
11:53 🔗 Wyatt|Wor Things exploded about an hour and a half ago
11:53 🔗 Wyatt|Wor From an IP in Lebanon.
12:10 🔗 Wyatt|Wor Ugh, I'm going to go attempt sleeping. Night all.
15:14 🔗 kennethre NotGLaDOS: http://httpbin.org/status/418
15:56 🔗 underscor t Wyatt|Wor You guys should scan for that stuff daily, if not every few hours :V
17:04 🔗 SketchCow Back
17:06 🔗 winr4r morning!
17:57 🔗 dnova is there a place for splinder to go when we're ready to upload it?
18:24 🔗 SketchCow Yes
20:54 🔗 winr4r SketchCow: my suggestion is to find the person who decided to erase the apollo tapes, interview them, then eat them
20:59 🔗 bsmith095 WHO DARES ERASE THE APOLLO TAPES?!?!?
20:59 🔗 winr4r bsmith095: they actually did
20:59 🔗 bsmith095 no seriously, who?, and also, what are they?
20:59 🔗 winr4r the original telemetry and TV tapes from apollo 11 were erased and re-used, probably in the 1980s
20:59 🔗 bsmith095 OMFG y9our kidding me
21:00 🔗 bsmith095 this is nasa, here, they keep everything how could they erase those?!
21:00 🔗 winr4r bsmith095: for the same reason that history is often destroyed because of very-short-term expediency
21:00 🔗 winr4r they don't keep everything
21:01 🔗 winr4r for example, they don't even have most of the design documents for the apollo spacecraft
21:03 🔗 bsmith095 again AFAIK, nasa has kept nearly every physical part, of every spacecraft they have, i a bone yard somewhere
21:04 🔗 bsmith095 can i time an rsync after its already started, automatically
21:04 🔗 kennethre Hopefully the digital age solves this.
21:04 🔗 winr4r bsmith095: the parts of the spacecraft themselves that weren't sent into solar orbit or didn't burn up on re-entry, yes
21:04 🔗 bsmith095 well yeah thats what i meant
21:04 🔗 winr4r bsmith095: we're still missing lots and lots of things
21:05 🔗 winr4r as in pretty much all the design documents, what we have now is a rounding error on what has been destroyed
21:05 🔗 bsmith095 are these important? sending incremental file list
21:05 🔗 bsmith095 rsync: link_stat "/home/ben/splinder-grab/data/it/a/al/alb/albe^" failed: No such file or directory (2)
21:05 🔗 bsmith095 rsync: link_stat "/home/ben/splinder-grab/data/it/g/gy/gyp/gypsy!" failed: No such file or directory (2)
21:06 🔗 bsmith095 seems like the special chars, killed the dump
21:06 🔗 * winr4r doesn't know
21:09 🔗 bsmith095 155,381 items, totalling 8.3 GB in splinder-grab
21:11 🔗 winr4r :/
21:11 🔗 winr4r good that you saved it, ":/" that stuff is just being destroyed
21:12 🔗 bsmith095 well *I* think its alot
21:12 🔗 winr4r yes, it is
21:14 🔗 bsmith095 i remember this one part of distributed preservation of service attack, jason's recent defcon talk, where he detailed all the project yahoo killed recently, one of them, yahoo briefcase, 25mb of space, probably less than 2 gb total, shutdown, and purged, "why, no spare usb key?!"
21:17 🔗 winr4r haha mhm
21:18 🔗 bsmith095 its on youtube if you havent seen it
21:19 🔗 winr4r i have!
21:19 🔗 winr4r i'm sure i saved a copy somewhere, too
21:34 🔗 bsmith095 yeah me, too, he's great when he's not reading off a script
21:55 🔗 winr4r does he ever read from a script?
21:57 🔗 winr4r his speeches are awesome though, i pointed one of my friends to one of them last night, he mentioned geocities, she was like "oh i had some stuff on geocities that i can't get back" and i was like "yeah, talk to jason"
21:58 🔗 winr4r she ended up getting back a poem that she wrote about her son who was killed by a drunk driver
21:59 🔗 goekesmi once again, archive team scores.
22:00 🔗 winr4r archive team = internet superheroes
22:00 🔗 * winr4r can only sit back and admire from afar, since he has a 20gb/month bandwidth cap and 384kbit/s upstream
22:10 🔗 Coderjoe only one profile left on the ec2. been spinning down for almost 5 days now. (7 hours short of that)
22:14 🔗 Wyatt underscor: Re your earlier comment, Yup. And now everyone who ISN'T me also knows this.
22:38 🔗 godane i'm backing up the screen savers
22:38 🔗 godane off of youtube
22:39 🔗 godane or more of what was not in techtv torrent
22:41 🔗 alard Hi all: I added a Splinder verification script to the git repository. It should to more/better checks than the check and fix scripts.
22:42 🔗 winr4r hiya alard
22:42 🔗 alard I think I covered the errors that I know of, but it may be useful if someone to check if anything is missing. https://github.com/ArchiveTeam/splinder-grab/blob/master/verify-splinder-profile.py
22:42 🔗 yipdw^ heh. so, like a dumbass, I erased my EC2 ssh keys
22:42 🔗 yipdw^ probably still got a copy on a desktop at home
22:44 🔗 Coderjoe you can download them from ec2, iirc
22:45 🔗 yipdw^ I've not seen that option
22:45 🔗 yipdw^ alard: is it correct to say that that script supercedes check-dld.sh?
22:46 🔗 Coderjoe i've been thinking we might want to start using python for the downloader scripts... aside from wget, it reduces the number of things to worry about (like bash being new enough for associative arrays, proper expr support, etc)
22:46 🔗 yipdw^ I agree
22:47 🔗 Coderjoe alard: does this also check for the stuff check-dld.sh checked?
22:47 🔗 yipdw^ I never knew there were so many differences in bash versions until I started doing AT stuff
22:48 🔗 yipdw^ to say nothing of the differences between BSD and GNU versions of utilities
22:50 🔗 Coderjoe i'm not seeing a check for splinder_noconn.html
22:50 🔗 alard https://github.com/ArchiveTeam/splinder-grab/blob/master/verify-splinder-profile.py#L171
22:50 🔗 yipdw^ Coderjoe: https://github.com/ArchiveTeam/splinder-grab/blob/master/verify-splinder-profile.py#L171-173
22:50 🔗 yipdw^ er, yeah
22:51 🔗 Coderjoe ok. irblind
22:52 🔗 Coderjoe eek. needs hanzo. planning on distributing it with the script?
22:52 🔗 alard I agree with bash/python point, by the way. (I originally thought the advantage of bash was that it's more or less the same everywhere and you don't have to install anything, but that turns out to be false. What's left is a somewhat clumsy scripting language.)
22:52 🔗 alard The hanzo files should be there.
22:53 🔗 yipdw^ Coderjoe: it's in the source tree
22:53 🔗 Coderjoe i haven't looked outside this file so I didn't know they were also checked in
22:54 🔗 alard Coderjoe: Yes, this could be a replacement for check-dld.sh. (Although check-dld.sh is much faster.)
22:54 🔗 Coderjoe this doesn't check for a .incomplete file... are you leaving it to the caller to not pass profiles that are still being downloaded?
22:55 🔗 alard This was more or less intended as a after-uploading check to run on batcave.
22:55 🔗 alard If there is an .incomplete file, the tests should not pass. The WARC files are incomplete, for example.
22:55 🔗 yipdw^ if you've got an incomplete download, you shouldn't have the wget metadata resources
22:55 🔗 yipdw^ -s +records
22:57 🔗 Coderjoe yipdw: btw, go to aws.amazon.com, drop down the account box and pick "security credentials"
22:58 🔗 Coderjoe I think you can get the keys from there
22:58 🔗 Coderjoe hmm
22:58 🔗 Coderjoe ok, that directs you to the ec2 management console
22:58 🔗 Coderjoe hm
22:58 🔗 Coderjoe bush
22:59 🔗 Coderjoe bust
22:59 🔗 yipdw^ "For your protection, AWS does not retain your private key."
22:59 🔗 yipdw^ it's fine
22:59 🔗 Coderjoe yeah, I just saw that
22:59 🔗 yipdw^ I should have a copy of it on my desktop at home
22:59 🔗 yipdw^ it just means I can't access the instance from here
22:59 🔗 yipdw^ so it'll have to wait until I get back
23:26 🔗 yipdw^ ok, this is sad
23:26 🔗 yipdw^ JSON::ParserError: 710: unexpected token at '"2011-11-26 16:26:55 -0700"'
23:26 🔗 yipdw^ ruby-1.9.2-p290 :004 > JSON.parse(Time.now.to_json)
23:27 🔗 yipdw^ er, wait
23:27 🔗 yipdw^ never mind, that was me being dumb
23:34 🔗 DFJustin in today's apple II floppy dumping stack: http://interbutt.com/temp/drruth.png
23:34 🔗 DFJustin oddly enough on a bootleg disk from a store which was apparently in saudi arabia
23:37 🔗 winr4r DFJustin: cool!
irclogger-viewer