[00:00] *** arkiver2 has joined #archiveteam-bs [00:09] *** wyatt8740 has quit IRC (Read error: Operation timed out) [00:14] *** arkiver2 has quit IRC (Ping timeout: 260 seconds) [00:16] *** wyatt8740 has joined #archiveteam-bs [00:24] I am nearly jealous of the way Bowie went out. [00:24] It appears he got the diagnosis, and then decides to immediately do the best fucking album ever and finish it on his birthday [00:25] In one pair of weeks, he gets worldwide attention for the single, the release of the album, and then he dies [00:25] And it turns out all the videos and stuff he was doing were made to work with the knowledge the audience knows the performer is dying [00:25] brava, bravo [00:32] *** Start has joined #archiveteam-bs [00:36] SketchCow: you may get some david bowie bootlegs [01:29] Appreciated [01:51] *** JesseW has joined #archiveteam-bs [02:53] Springer opened the door a little larger. [02:54] They went from 124 english books to 1,135 [02:54] So, I'm going to go turn on my thing again [03:14] Uploading 220 issues of Linux Format [03:32] *** Coderjoe_ has quit IRC (Read error: Operation timed out) [03:33] *** RichardG has quit IRC (Ping timeout: 252 seconds) [03:36] *** Coderjoe has joined #archiveteam-bs [03:45] *** acridAxid has joined #archiveteam-bs [03:46] SketchCow: i also got a copy of linux format from the same torrent [03:46] for my private archive collection [03:47] Nice. [03:51] i did question the +800mb file size [03:51] only cause even byte magazine are more like 200 to 300mb range [03:51] also there not full scans like byte magazine [03:53] *** Coderjoe has quit IRC (Read error: Connection reset by peer) [03:53] *** Coderjoe has joined #archiveteam-bs [03:55] SketchCow: How are you getting a list of the Springer books? [04:04] I search-blort [04:04] http://link.springer.com/search?facet-content-type=%22Book%22&showAll=false&facet-language=%22En%22 [04:04] "All english Books with no preview-only" [04:05] I'm about to tune the thing to pull them down, guarantee I've not already snagged, and onward [04:11] Ah, nice [04:13] Hm, there's another 1000 or so non-English ones. [04:15] archiveteam.org is giving me a bunch of php errors.. Warning: preg_match() [function.preg-match]: Compilation failed: group name must start with a non-digit at offset 8 in /home/archivet/public_html/includes/MagicWord.php on line 860 [04:16] I'll do english then do german [04:17] The host updated PHP [04:20] *** Coderjoe has quit IRC (Read error: Operation timed out) [04:35] i have past 600k [04:35] items [04:35] *** JesseW has quit IRC (Read error: Operation timed out) [04:35] *** Coderjoe has joined #archiveteam-bs [04:38] congrats godane [04:51] *** JesseW has joined #archiveteam-bs [05:00] *** Stiletto has quit IRC (Read error: Operation timed out) [05:00] *** is- has quit IRC (Read error: Operation timed out) [05:00] *** brayden_ has joined #archiveteam-bs [05:00] *** swebb sets mode: +o brayden_ [05:00] *** Stilett0 has joined #archiveteam-bs [05:01] *** is- has joined #archiveteam-bs [05:01] *** Stilett0 is now known as Stiletto [05:04] [17:49] Everything has VAT. But maybe, if you're a farmer and are using this ticket in an agricultural endeavour, VAT may be 13,56% Except on sundays. Or if you _ate_ the ticket, then it's 7% (but not if you're an animal or a public library) [05:04] [17:49] easy! [05:05] [17:51] but beware! an edible ticket is only 7% if you take it home to cook it and it is not ready to eat... if you can eat it on the premises, it is still 19% because fuck you that's why [05:05] (discussing 32c3 conference ticket VAT) [05:05] *** brayden has quit IRC (Read error: Operation timed out) [05:06] *** swebb has quit IRC (Read error: Operation timed out) [05:06] *** xmc has quit IRC (Read error: Operation timed out) [05:06] *** SN4T14 has quit IRC (Read error: Operation timed out) [05:06] *** chazchaz has quit IRC (Read error: Operation timed out) [05:07] *** xmc has joined #archiveteam-bs [05:07] *** tephra has quit IRC (Read error: Operation timed out) [05:07] *** hawc145 has joined #archiveteam-bs [05:07] *** atlogbot has quit IRC (Ping timeout: 369 seconds) [05:07] *** swebb has joined #archiveteam-bs [05:07] *** HCross has quit IRC (Read error: Operation timed out) [05:07] *** chfoo has quit IRC (Read error: Operation timed out) [05:08] *** DFJustin has quit IRC (Read error: Operation timed out) [05:08] *** SimpBrain has quit IRC (Read error: Operation timed out) [05:08] *** chfoo- has quit IRC (Read error: Operation timed out) [05:08] *** DFJustin has joined #archiveteam-bs [05:08] *** chfoo- has joined #archiveteam-bs [05:08] *** SimpBrain has joined #archiveteam-bs [05:08] *** dcmorton has quit IRC (Read error: Operation timed out) [05:08] *** chfoo has joined #archiveteam-bs [05:10] *** mistym- has quit IRC (Ping timeout: 369 seconds) [05:10] *** zenguy has quit IRC (Ping timeout: 369 seconds) [05:10] *** dxrt has quit IRC (Ping timeout: 369 seconds) [05:11] *** atlogbot has joined #archiveteam-bs [05:11] *** zenguy has joined #archiveteam-bs [05:11] *** SketchCow has quit IRC (Read error: Connection reset by peer) [05:11] *** dxrt has joined #archiveteam-bs [05:12] *** Start has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** Kazzy has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** ohhdemgir has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** Baljem has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** mr-b has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** Nertsy has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** closure has quit IRC (ircd.choopa.net irc.teksavvy.ca) [05:12] *** BlueMaxim has quit IRC (Read error: Operation timed out) [05:12] *** mistym has joined #archiveteam-bs [05:13] *** dcmorton has joined #archiveteam-bs [05:13] *** tephra has joined #archiveteam-bs [05:13] *** Smiley has quit IRC (west.us.hub irc.Prison.NET) [05:13] *** midas1 has quit IRC (west.us.hub irc.Prison.NET) [05:13] *** Zebranky has quit IRC (west.us.hub irc.Prison.NET) [05:13] *** lbft has quit IRC (west.us.hub irc.Prison.NET) [05:13] *** Kazzy_ has joined #archiveteam-bs [05:14] *** BlueMaxim has joined #archiveteam-bs [05:16] *** SN4T14 has joined #archiveteam-bs [05:17] *** ivan` has quit IRC (Read error: Operation timed out) [05:17] *** Sanqui has quit IRC (Read error: Operation timed out) [05:18] *** beardicus has quit IRC (Write error: Broken pipe) [05:18] *** JesseW has quit IRC (Read error: Operation timed out) [05:18] *** chazchaz has joined #archiveteam-bs [05:19] *** Joins: BlueMaxim (~BlueMaxim@[redacted]) [05:19] *** Joins: Kazzy_ (~Kaz@[redacted]) [05:19] *** Quits: acridAxid (~acridAxid@[redacted]) (Ping timeout: 312 seconds) [05:21] *** Joins: SN4T14 (~SN4T14@[redacted]) [05:21] *** Joins: chazchaz (~chazchaz@[redacted]) [05:21] *** Quits: JesseW (~jesse@[redacted]) (Read error: Operation timed out) [05:21] *** Quits: Sanqui (~Sanky_R@[redacted]) (Read error: Operation timed out) [05:21] *** Quits: beardicus (~beardicus@[redacted]) (Write error: Broken pipe) [05:21] *** Quits: ivan` (~marvinw@[redacted]) (Read error: Operation timed out) [05:21] *** Quits: zenguy (~zenguy@[redacted]) (Ping timeout: 310 seconds) [05:23] *** Joins: chfoo- (~chfooZnc@[redacted]) [05:23] *** Server sets mode: +stn [05:23] *** Joins: Sanqui (~Sanky_R@[redacted]) [05:24] *** Quits: BlueMaxim (~BlueMaxim@[redacted]) (Read error: Operation timed out) [05:24] *** GLaDOS sets mode: +o SketchCow [05:24] *** Joins: SketchCow (~jscott@[redacted]) [05:26] *** Joins: BlueMaxim (~BlueMaxim@[redacted]) [05:26] *** Quits: Boppen (~Boppen@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Ctrl-S___ (sid86077@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Fusl (Fusl@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: JSharp___ (sid4580@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Kazzy_ (~Kaz@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Kenshin (~rurouni@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Rickster (~Ricky@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Rotab (~Rotab@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: SadDM (~SadDM@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: SmileyG (~Smiley@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Stiletto (Stiletto@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: X1011 (uid138367@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: _desu___ (sid73031@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: arkiver (~arkiver@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: bauruine (~bauruine@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: deathy___ (sid2210@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: godane (~slacker@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: joepie91 (~joepie91@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: mistym (~mistym@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: mutoso (~alastair@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: pikhq (~pikhq@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: sigkell (~sigkell@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: signius (~signius@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: turnkit|2 (~kvirc@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: yipdw (~yipdw@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: zhongfu (~zhongfu@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: zyphlar_ (sid28450@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Apathy (~Grey@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: DFJustinZ (~justin@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Dark-Star (~darkstar@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Famicoman (~famicoman@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Fletcher (~F@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Infreq (~Infreq@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Lord_Nigh (Lord_Nigh@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Mayonaise (~kornpops@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Rye (~riley@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Silvan (~quassel@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: Start_ (~Start@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: afics (~afics@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: antomatic (~antomatic@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: brayden_ (~brayden@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: dan- (~d@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: dashcloud (~quassel@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: ersi (~ersi@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: espes___ (~espes@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: jk[SVP] (~efnet@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: jspiros (jspiros@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: matthusb- (~matthusby@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: rduser (~rduser@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: useretai- (useretail@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: wednesday (~wednesday@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: will (will@[redacted]) (hub.efnet.us hub.dk) [05:26] *** Quits: zerkalo_ (myricae@[redacted]) (hub.efnet.us hub.dk) [05:29] *** Joins: Darkstar (~darkstar@[redacted]) [05:30] *** Joins: matthusb| (~matthusby@[redacted]) [05:52] *** Quits: acridAxid (~acridAxid@[redacted]) (marauder) [05:53] *** Joins: acridAxid (~acridAxid@[redacted]) [05:56] *** Joins: Mayonaise (~kornpops@[redacted]) [05:56] *** Joins: afics (~afics@[redacted]) [05:56] *** Joins: dan- (~d@[redacted]) [05:57] *** Joins: Apathy (~Grey@[redacted]) [06:01] *** Joins: FAMAS (67c68b35@[redacted]) [06:05] *** Joins: JesseW (~jesse@[redacted]) [06:06] Anyone else having a lot of trouble finding a working EFnet server? [06:08] *** Quits: acridAxid (~acridAxid@[redacted]) (Ping timeout: 246 seconds) [06:12] *** Joins: Boppen (~Boppen@[redacted]) [06:12] *** Joins: Ctrl-S___ (sid86077@[redacted]) [06:12] *** Joins: DFJustinZ (~justin@[redacted]) [06:12] *** Joins: Famicoma1 (~famicoman@[redacted]) [06:12] *** Joins: Fletcher (~F@[redacted]) [06:12] *** Joins: Fusl (Fusl@[redacted]) [06:12] *** Joins: Infreq (~Infreq@[redacted]) [06:12] *** Joins: JSharp___ (sid4580@[redacted]) [06:12] *** Joins: Kazzy (~Kaz@[redacted]) [06:12] *** Joins: Kenshin (~rurouni@[redacted]) [06:12] *** Joins: Lord_Nigh (Lord_Nigh@[redacted]) [06:12] *** Joins: Rickster (~Ricky@[redacted]) [06:12] *** Joins: Rotab (~Rotab@[redacted]) [06:12] *** Joins: Rye (~riley@[redacted]) [06:12] *** Joins: SadDM (~SadDM@[redacted]) [06:12] *** Joins: SilSte (~quassel@[redacted]) [06:12] *** Joins: SmileyG (~Smiley@[redacted]) [06:12] *** Joins: Stiletto (Stiletto@[redacted]) [06:12] *** Joins: X1011 (uid138367@[redacted]) [06:12] *** Joins: _desu___ (sid73031@[redacted]) [06:12] *** Joins: antomatic (~antomatic@[redacted]) [06:12] *** Joins: arkiver (~arkiver@[redacted]) [06:12] *** Joins: bauruine (~bauruine@[redacted]) [06:12] *** Joins: dashcloud (~quassel@[redacted]) [06:12] *** Joins: deathy___ (sid2210@[redacted]) [06:12] *** Joins: espes___ (~espes@[redacted]) [06:12] *** Joins: godane (~slacker@[redacted]) [06:12] *** Joins: jk[SVP] (~efnet@[redacted]) [06:12] *** Joins: joepie91 (~joepie91@[redacted]) [06:12] *** Joins: jspiros (jspiros@[redacted]) [06:12] *** Joins: marvinw (~marvinw@[redacted]) [06:12] *** Joins: midas2 (~midas@[redacted]) [06:12] *** Joins: mistym (~mistym@[redacted]) [06:12] *** Joins: mutoso (~alastair@[redacted]) [06:12] *** Joins: pikhq (~pikhq@[redacted]) [06:12] *** Joins: rduser (~rduser@[redacted]) [06:12] *** Joins: sigkell (~sigkell@[redacted]) [06:12] *** Joins: signius (~signius@[redacted]) [06:12] *** Joins: turnkit|2 (~kvirc@[redacted]) [06:12] *** Joins: useretai- (useretail@[redacted]) [06:12] *** Joins: will (will@[redacted]) [06:12] *** Joins: yipdw (~yipdw@[redacted]) [06:12] *** Joins: zerkalo_ (myricae@[redacted]) [06:12] *** Joins: zhongfu (~zhongfu@[redacted]) [06:12] *** Joins: zyphlar_ (sid28450@[redacted]) [06:12] *** hub.se sets mode: +oo SadDM antomatic [06:12] *** hub.se sets mode: +oooo Kenshin godane dashcloud arkiver [06:14] it is requested towards the archiveteam staff to contemplate their antagonistic views and response towards this user and the possible consequential impediments towards the cause of archiving data [06:16] reiteration: it is requested towards the archiveteam staff to contemplate their antagonistic views and response towards this user and the possible consequential impediments towards the cause of archiving data [06:18] non sequitur: your facts are uncoordinated [06:19] *** logchfoo1 starts logging #archiveteam-bs at Tue Jan 12 06:19:30 2016 [06:19] *** logchfoo1 has joined #archiveteam-bs [06:20] dfjustin: an analysis of the archiveteam channel logs will reveal the nature of the behavior that this user is perpetrating within the claims [06:22] *** beardicus has joined #archiveteam-bs [06:26] *** SketchCow sets mode: +b *!*67c68b35@103.198.139.* [06:26] *** FAMAS was kicked by SketchCow (FAMAS) [06:36] *** brayden has joined #archiveteam-bs [06:49] wow, that's some splits [06:52] *** Baljem has joined #archiveteam-bs [06:52] *** mr-b has joined #archiveteam-bs [06:52] *** irc.teksavvy.ca sets mode: +o Baljem [06:53] *** Nertsy has joined #archiveteam-bs [06:55] *** ersi has joined #archiveteam-bs [06:55] *** Zebranky has joined #archiveteam-bs [07:07] Mega-splits [07:08] My springer book uploads are blasting it [07:11] Tech News 2Night is full uploaded now [07:14] Congrads to both of you. [07:16] I kinda wish had some suggestions of *where* to send people who are not welcome to participate in the #archiveteam channels... I mean, we can send them to /r/datahorders, and presumably the various 'chan archiving efforts have somewhere they organize and discuss their work -- but it'd be good to have a list. [07:17] * JesseW is a fan of multiple, mutually-distrustful archiving efforts, if necessary or preferred [07:17] hm, missed a word above: "wish *we* had" [07:54] *** Start has joined #archiveteam-bs [08:04] *** w0rp has joined #archiveteam-bs [08:25] *** JesseW has quit IRC (Ping timeout: 246 seconds) [08:43] I'm going to bed with three different robots uploading books from Springer at the rate (int total) of one every 9 seconds. [08:50] *** JesseW has joined #archiveteam-bs [08:51] *** schbirid has joined #archiveteam-bs [08:51] *** GLaDOS has quit IRC (Read error: Operation timed out) [08:58] *** GLaDOS has joined #archiveteam-bs [09:23] *** RichardG has joined #archiveteam-bs [09:40] *** JesseW has quit IRC (Ping timeout: 246 seconds) [09:49] *** GLaDOS has quit IRC (Ping timeout: 260 seconds) [09:50] *** GLaDOS has joined #archiveteam-bs [09:50] *** turnkit has joined #archiveteam-bs [09:52] *** dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.) [09:52] *** mutoso has quit IRC (Ping timeout: 252 seconds) [09:52] *** turnkit|2 has quit IRC (Ping timeout: 252 seconds) [09:52] *** schbirid has quit IRC (Read error: Operation timed out) [09:53] *** zenguy has joined #archiveteam-bs [09:55] *** schbirid has joined #archiveteam-bs [09:56] *** dashcloud has joined #archiveteam-bs [09:58] *** mutoso has joined #archiveteam-bs [10:04] *** arkiver2 has joined #archiveteam-bs [10:09] *** arkiver2 has quit IRC (Ping timeout: 260 seconds) [10:19] *** arkiver2 has joined #archiveteam-bs [11:17] *** midas2 is now known as midas [11:20] *** arkiver2 has quit IRC (Quit: Nettalk6 - www.ntalk.de) [11:53] *** dashcloud has quit IRC (Read error: Operation timed out) [11:56] *** vitzli has joined #archiveteam-bs [11:57] *** dashcloud has joined #archiveteam-bs [12:00] *** mr-b has quit IRC (Ping timeout: 369 seconds) [12:04] *** mr-b has joined #archiveteam-bs [12:06] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [12:38] *** Muad-Dib has joined #archiveteam-bs [14:34] They finished. Some came out weird but 1,500 new books it is [14:34] Now doing 1,500 german-language books from same [14:35] Pile of SGI CD-ROMs coming to me for no effort here: https://archive.org/details/cdromsoftware?sort=-publicdate [14:36] sehr gut SketchCow [14:36] (my german is very rusty) [14:37] sehr rostig [14:44] SN4T14: my icelandic is frozen, what in short does that pdf say? [14:49] wow, SGI CDs. I also have a couple of them lying around here somewhere [14:50] upload them :( [14:50] :) [14:51] hm. I probably should [14:51] I already dumped and scanned them some time ago for another site [14:53] they're in mdf+mds though, and the scans are TIFF instead of PNG. I don't suppose the archive.org robots will convert these automatically? [14:53] (the TIFFs are probably fine but the MDF+MDS?) [14:53] also can I just oload a ZIP file and it will be automatically uncompressed? or do I need to upload everything individually? [15:04] *** Start has quit IRC (Quit: Disconnected.) [15:05] also I'm wondering if there is a better way than uploading through the web interface. If I want to upload multiple gigabytes of huge ISOs, I'd prefer some automated way. [15:05] s3 [15:05] like a tool where I can prepare the uploads beforehand, like 1 directory per archive.org item, with a metadata.txt file of a specific format, and the ability to resume interrupted transfers [15:06] http://archive.org/help/abouts3.txt [15:06] (this is the old info tho, currently at work so not that much time) [15:06] yeah, I know about the S3 interface but manually typing 200-character curl commands is not what I'm looking for ;-) [15:07] someone, somewhere, must have automated this with a script or something. right? [15:07] well, write a frontend that does that for you :) [15:07] yeah, thanks, I was asking if anyone knows if something like this already exists. but it seems the answer is "no" :( [15:08] it probably exists, just not known to me :) [15:12] *** closure has joined #archiveteam-bs [15:13] I'm a little cautious with playing around with the S3 interface, I don't want to pollute archive.org with tons of "test"-items that I am unable to delete afterwards, for example :) [15:21] This is something I know an answer to, it's not the brightest idea, but I found that it's the best way to upload stuff to the IA: [15:24] 1. python tool to generate metadata from the item's name -> produces .yaml file (this: http://pastebin.com/7HemYyma) - does not do any checks, just assumes that everything is right [15:27] input for that program is a file with one item per row, boring, like this http://pastebin.com/a9e2Nu44 [15:29] 2. Tool to upload the stub file, generally item image or text file or checksum file. Small enough to initiate the item with metadata and not to bother with s3cmd headers (I got feeling that automating s3cmd with cmd headers is quite unpleasant). Here is the script, it takes .yaml file from step 1 and uploads it to the IA: http://pastebin.com/vu2pF87H . It requires 'completed' directory to exist and moves the 'stub' file there. [15:31] all files are in the directory that has same name as the item [15:32] hm so you're basically first creating an empty ia item (s3 bucket) without any metadata and then add each item with its metadata one by one to this bucket? [15:33] no wait the metadata is per-item, not per-file [15:33] or is it? hmm [15:33] ok you have to have the correct metadata on the first PUT command and then each additional file you add doesn't need any metadata [15:33] 3. Upload the rest of the files to the IA, using s3cmd (because, apparently, ia has problems with 10+ GB files), by default s3cmd uses 15MB chuncks, use config to change it, I use 256MB. Script: http://pastebin.com/Jf2Lmnrx [15:35] yeah, the first item should have metadata, correct. Apparently, doing 'no-derive' for large uploads is a good thing, but I cannot make it work with s3cmd for some reason (probably because I'm retarded and passing wrong header to it) [15:35] yeah I think I would have done it similar, although I would probably write a shell script and use curl :) [15:36] still it'd be great if there were a "s3 test server" for IA that just discarded everything after a day or two [15:37] that's the reason for me to use internetarchive library - parse item name and fill .yaml with text editor when necessary [15:37] *** dashcloud has quit IRC (Read error: Operation timed out) [15:38] There is 'Test collection', but I don't know if it works, all items in it should be deleted after a month. I've tried it but did something wrong and item stayed there [15:38] yeah, still I wonder what happens when the upload aborts halfway through a huge file. can I just re-start the upload of the same file and overwrite the existing file on IA? can I resume? is there a way to see which files are already in the IA bucket (like an "ls") to see what to resume? [15:39] with s3cmd and multipart uploading it is possible to abort or resume the upload [15:40] I don't know if ia tool allows this [15:41] *** dashcloud has joined #archiveteam-bs [15:42] ah apparently I can just GET the bucket and I see which files are there and how big they are [15:42] that will be useful [15:43] maybe I will indeed write a small little client tool for this [15:44] just to make sure: the foo_files.xml file, the foo_meta.* files and the torrent are all autogenerated, right? [15:46] *** Start has joined #archiveteam-bs [15:47] I have no idea how xml files are generated [15:48] yeah but I don't have to generate and upload them myself, IA does that. right? [15:49] correct, but, iirc, except for the torrent file - there is an option/http header somewhere that disables the torrent generation [15:51] yeah, that's fine, I'm just trying to come up with a detailed description on what an upload script needs to do to not fuck up everything on IA :) [15:57] midas, that PDF is from some accounting course, seemingly intended for the first class [16:19] *** RichardG has quit IRC (Read error: Connection reset by peer) [16:22] *** RichardG has joined #archiveteam-bs [16:26] Okay I came up with a high-level overview over what a simple upload script should do: http://pastebin.com/ULVxyMZ5 ... Ideas/comments anyone? :) [16:30] *** RichardG has quit IRC (Read error: Connection reset by peer) [16:32] *** RichardG has joined #archiveteam-bs [16:44] *** slyphic has joined #archiveteam-bs [17:07] *** Start has quit IRC (Quit: Disconnected.) [17:12] *** Start has joined #archiveteam-bs [17:18] *** dashcloud has quit IRC (Read error: Operation timed out) [17:22] *** dashcloud has joined #archiveteam-bs [17:43] *** acridAxid has joined #archiveteam-bs [17:54] The Springer english books are added! [17:54] Now doing a slight metadata fix [17:54] Now grabbing german. [17:57] *** bwn has joined #archiveteam-bs [18:00] *** hawc145 is now known as HCross [18:05] *** JesseW has joined #archiveteam-bs [18:12] *** JesseW has quit IRC (Leaving.) [18:21] Can I get ops? [18:21] Can I get ops? [18:21] not from me [18:25] *** arkiver sets mode: +o swebb [18:25] *** swebb sets mode: +o brayden [18:25] *** swebb sets mode: +o ersi [18:25] *** swebb sets mode: +o xmc [18:36] *** Start has quit IRC (Quit: Disconnected.) [18:43] *** Start has joined #archiveteam-bs [18:58] *** vitzli has quit IRC (Leaving) [19:05] midas: can you check if "Scio - Non fa per me" is in the jamendo archive and sent me the license and id? [19:06] fuck jamendo for not keeping an archive of removed songs [19:06] got a copyright claim on youtube on that track [19:06] and youtube does not let me see my fucking own description where the license and url are [19:18] *** Start has quit IRC (Quit: Disconnected.) [19:23] *** Start has joined #archiveteam-bs [19:45] SketchCow: may 2007 of kpfa is getting uploaded [19:46] i ran like 4 days with 8 proxy ip addresses [19:48] *** Start_ has joined #archiveteam-bs [19:48] *** Start has quit IRC (Read error: Connection reset by peer) [20:03] *** arkiver has quit IRC (Ping timeout: 260 seconds) [20:04] *** arkiver has joined #archiveteam-bs [20:34] *** SketchCow has quit IRC (Read error: Connection reset by peer) [20:39] *** SketchCow has joined #archiveteam-bs [20:39] *** swebb sets mode: +o SketchCow [20:44] *** Start_ has quit IRC (Quit: Disconnected.) [20:48] *** Start has joined #archiveteam-bs [21:12] *** Darkstar has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** unstable has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** PotcFdk has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** mksplg has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** altlabel has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** PurpleSym has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** limebyte has quit IRC (hub.efnet.us irc.homelien.no) [21:12] *** i0npulse has quit IRC (hub.efnet.us irc.homelien.no) [21:14] *** PotcFdk has joined #archiveteam-bs [21:14] *** mksplg has joined #archiveteam-bs [21:14] *** altlabel has joined #archiveteam-bs [21:14] *** PurpleSym has joined #archiveteam-bs [21:14] *** limebyte has joined #archiveteam-bs [21:14] *** i0npulse has joined #archiveteam-bs [21:15] *** Darkstar has joined #archiveteam-bs [21:20] *** unstable has joined #archiveteam-bs [21:22] *** wyatt8740 has joined #archiveteam-bs [21:42] *** tephra has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** DFJustin has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** is- has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** Coderjoe has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** lytv has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** winr5r has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** goekesmi_ has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** alard has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** wp494 has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** lysobit has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** no2penci1 has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** MrRadar has quit IRC (ircd.choopa.net irc.mzima.net) [21:42] *** fie has quit IRC (ircd.choopa.net irc.mzima.net) [21:47] *** tephra has joined #archiveteam-bs [21:47] *** DFJustin has joined #archiveteam-bs [21:47] *** is- has joined #archiveteam-bs [21:47] *** Coderjoe has joined #archiveteam-bs [21:47] *** lytv has joined #archiveteam-bs [21:47] *** winr5r has joined #archiveteam-bs [21:47] *** goekesmi_ has joined #archiveteam-bs [21:47] *** alard has joined #archiveteam-bs [21:47] *** wp494 has joined #archiveteam-bs [21:47] *** lysobit has joined #archiveteam-bs [21:47] *** no2penci1 has joined #archiveteam-bs [21:47] *** MrRadar has joined #archiveteam-bs [21:47] *** fie has joined #archiveteam-bs [21:47] *** irc.mzima.net sets mode: +o alard [21:47] *** swebb sets mode: +o alard [21:55] *** Start has quit IRC (Quit: Disconnected.) [22:52] idea: a websocket interface for uBlock to pass URLs to an isolated Web browser to facilitate click fraud [23:19] *** Start has joined #archiveteam-bs [23:22] *** Ravenloft has joined #archiveteam-bs [23:42] *** logan has joined #archiveteam-bs [23:45] *** JetBalsa has joined #archiveteam-bs [23:52] *** dashcloud has quit IRC (Read error: Operation timed out) [23:56] *** dashcloud has joined #archiveteam-bs