[00:12] what does the internet archive use to download google books [00:13] i'm looking at a google books downloader so i can complete my infoworld download [00:24] the google book grab is not working [00:25] ok looks like it may just be that book [01:06] http://pastebin.com/B6RiEqev [01:51] arkiver: yes; for individual pages, use a browser engine and a proxy to capture the request/responses [01:51] https://github.com/odie5533/WarcMITMProxy is one such proxy [02:52] I finally finished my grab of www.dreamlandbbs.com , and when I started it, I had each .warc file stop at 1 GB- is that going to be a problem if I upload 73 warcs? [03:43] no [03:44] i have seen items that have a few 100 files without any problems [03:45] and the only derive is making is cdx and idx files of the warc.gz [03:51] yeah, deriver can chew on as many files as you want [04:31] thanks! [06:05] ON AVERAGE, one of the Dogster packs has 30,000 users [06:53] http://boltagain.ning.com/page/bolt3-closing-down-1 [06:53] Bolt is closing down. [06:53] Wow, they are doing it QUICK. [07:06] i feel like they're thinking "And nothing of value was lost." [07:06] because of all of the trolls [07:09] never heard of Bolt [07:11] good community if your first memory after 5 years is "two separate moderator civil wars" [07:19] yipdw: Thank you, I tried that progra but every time I want to install MITMProxy I get an error about ssleay32.dll [07:19] :/ [07:19] I tried a lot and nothing worked... [07:19] did you install 32-bit and 64-bit OpenSSL for Windows? [07:19] (don't know which one you need for that) [07:20] well [07:20] ehm I never installed openssl? [07:21] I didn't knew that was needed... [07:21] I'll do it now [07:28] no... Still now working... [07:28] still getting the ssleay32.dll is issing error [07:28] but it is in the openssl bin folder [07:28] lib folder* [09:22] what I'doing now: [09:22] go to facebook/twitter page [09:22] load as any scroll down pages as possible [09:22] go to the cache [09:23] get all the urls fro the scroll down pages, example: https://twitter.com/i/profiles/show/Polare/timeline?include_available_features=1&include_entities=1&last_note_ts=0&max_id=263340143895801855 [09:24] facebook example: https://www.facebook.com/ajax/pagelet/generic.php/ProfileTimelineSectionPagelet?no_script_path=1&data=%7B%22profile_id%22%3A535305456508224%2C%22start%22%3A1357027200%2C%22end%22%3A1388563199%2C%22query_type%22%3A8%2C%22section_pagelet_id%22%3A%22pagelet_timeline_year_last%22%2C%22load_immediately%22%3Afalse%2C%22force_no_friend_activity%22%3Afalse%7D&__user=0&__a=1&__dyn=7wKzS10Ax-7o8UhACGeGEmBWpU&__req=jsonp_3&__rev=1119147&__adt=3 [09:24] Then I'm putting all those links into Heritrix [09:24] and Heritrix downloads them together with all the nearest links... [09:24] Hopefully that will work... [11:28] It's working! https://twitter.com/YahooVictims [11:40] What IS that about, by the way. [11:41] we got an arch-enemy [11:44] SketchCow: you asked to monitor some wiki pages and news... [11:47] Suggestions for name and whatever appreciated, I just set up some RSS -> Twitter feeds because it's easier done than said [11:51] No, it's fine, it's just kind of aggressive, just wanted to know what we're up to. [11:51] I'm obviously fine with aggression [11:56] :P [12:31] have to agree, dont use firefox then [12:35] I work with all the browsers because of JSMESS [12:38] i cant even use javascript now, my laptop has broken filesystem and i am booted from LAN to ms dos with ssh emulation, so im using text based stuff only now [13:06] does anyone have list of what iles are in the The Devil's Doorknob BBS from 1996? [13:06] files* [13:07] Really. [13:07] You're really asking that. [13:07] So, back in 1983, there was a girl I liked. How's she doing? [13:07] theres no txt file with it here https://archive.org/details/Devils_Doorknob_BBS_Capture_1996_2006 [13:08] the others mostly have txt files with the file list [13:09] https://archive.org/details/cdrom-devilsdoorknobbbscapture1996-2003 [13:09] https://archive.org/details/Devils_Doorknob_BBS_Capture_1996_2003 [13:09] https://archive.org/download/Devils_Doorknob_BBS_Capture_1996_2003/Devils_Doorknob_BBS_Capture_1996_2003.zip/ gives you the list. [13:13] In related news: [13:13] We had someone who's been on here for 3 years storm off because in a private message he gave me a question, and I gave a Jason Scott answer. [13:13] And he announced he was blocking me and fuuuuuuck you. [13:14] Anyway, the moral of the story is: come the fuck on [13:14] I only mention because I see he quit here. [13:15] to be fair, you *can* be pretty abraisive at times [13:15] "Jason Scott answer" in 3... 2... 1... [13:15] :-) [13:15] SUPER abraisve [13:15] World fuckin' class [13:16] Top ten michelin stars [13:16] * SadDM chuckles [13:16] the fact that you're so self-aware helps though [13:16] It helps nothing. [13:16] Still stings you in the little dainty man-purse [13:17] err k [13:18] isnt archiveteam a "Fuck you, we are going to do it anyway" state of mind? [13:18] Yes. [13:18] But we attract two types of people. [13:18] Thick-skinned powernerds dedicated to preservation at all costs [13:19] so if it's going to be fuck you and we are going to do it, you better have that mindset and dont be a tool about it [13:19] layabout aspies who are seeking to be a part of a greater thing without really committing [13:19] The second ones usually wash out within 2 hours [13:19] This one took longer. [13:20] my guess it took you 3 years to check if he/she had that mindset ;-) [13:21] most of the time it kinda works quick if they get a Jason (tm) anwser in the 2 hours they are here [13:21] Others do it too [13:21] As it should be. [13:21] We're getting shit done [13:22] btw, do you actually sleep sometimes? you are here 24/7. [13:22] Maybe this guy can start an archiveteam ladies auxillary [13:22] burnnnnn ;-) [13:22] I've got FOS downloading thingiverse, a massive portugese FTP site, grabbing the dogster crap, taking in the archivebot, and shoving in databases. [13:24] Also, it'd be more of a tragedy if he did anything. [13:24] He did not. [13:24] huh? who are you talking about [13:25] Shia Lebouf [13:25] worst actor EVER [13:25] ... [13:25] and the most shitty skywriter, cant archive sky [13:26] Skychive [13:26] tried it once, almost suffocated in the amount of work... [13:27] http://gadgetsin.com/uploads/2011/04/airship_elizabeth_steampunk_handbag_1.jpg [13:27] cUh http://www.independent.co.uk/news/people/news/shia-labeouf-announces-retirement-from-public-life-after-plagiarism-scandal-sparks-bizarre-apology-spree-9051553.html [13:27] however, you can compress it pritty well, it's a breeze [13:28] ok, im going to stop this, it's not -bs here and i want to keep the sky clean here. [14:17] We all just want a hug. [15:04] "free hugs, but only if you archive your website!" [17:02] midas: when SketchCow sleeps, FOS is being awake in his place [17:13] someone left me a message: http://archiveteam.org/index.php?title=User_talk:Chfoo&oldid=18518 . also http://archiveteam.org/index.php?title=Talk:Current_Projects&oldid=18516 [17:17] chfoo: people are really in love with mediafire, I'll never understand why. Can you download the file from it? [17:18] i never looked at it yet. i was hoping someone would do it for me [17:37] chfoo, it looks like it's a httrack crawl for ms.nintendo-europe.com/dkc [17:44] hmm, alright. i guess i will take a look at it later and upload it on behalf. not sure why they didn't upload it themselves though. [18:03] most people are astonished when they learn they can upload stuff to archive.org [18:03] After all, most people are astonished even when they learn they can edit a wiki OOOOOH [18:03] no [18:04] that's a very different thing [18:04] I'm not saying it's the same, only that I see the same pattern when I tell people "do it yourself" [18:06] i was surprised myself tht you can upload stuff these days like the shareware cd dumps, in fact when i told that to some people that it's there they were very very surprised, especially ones that used to be active in the warez scene etc [18:06] but editing wiki is a given i think, i mean thats the point of the wiki [18:07] Sure. Technically, uploading the stuff ArchiveTeam uploads is prohibited in clear letters by the IA terms of use. :) [18:08] But that's not what stops most people, copyright paranoia is not so widespread outside Wikimedia. :P [18:09] yea i guess so, well i hope people continue uploading some rare cds still, waiting for the day the voodoo cds and number fate will be there i guess especially [18:11] One would think that in this period of general economical crisis many would get tons of such stuff out of their cellars to get some dollars at the local flea market [18:12] i guess most of the stuff contained is "trash" or useless, but there are sometimes rarities among it, aside those early leaks of stuff like unreal, some other programs and tools that got lost meanwhile, i know of few other people looking for that [18:12] hmm [18:12] i used to buy in the 90s pirated stuff from the vietnamese [18:13] I spent a few hundreds euros on the online flea markets of Italy but they're not as cheap as I'd like [18:13] Tons of idiots going "hey I paid this issue 4 € now 20 years later I want at least 1" [18:14] lol [18:15] well but thanks to those guys i got acess to some carttirdges etc that werent sold in europe, games on the famicon/nes and some rare games/hacks [18:15] they even had that "somari" thing, yes, i did buy it [18:15] look it up if you want [18:15] all the old cds are worthwhile imo, maybe when they came out they were worthless cause you could get all the same stuff online, but now 15+ years later a lot of stuff is hard to find or not available [18:16] i believe there were even some videos on ytube i saw years back, with guys cursing this game out [18:19] Leo_TCK: you ripped them yourself? [18:22] afraid not, most of my cartridges are with my grandparent place [18:22] stored [18:22] i didnt hae the equipment to rip them anyway [18:23] though my version of somari differed a bit from the rom that is avail around [18:23] so perhaps one day when i get back there [18:23] i can do something about it [20:40] i found a cd back some time ago, got 15 hours of internets on it. [20:40] i think joepie91 archived one with 60 hours of internet on it [20:40] in .nl it was normal to ship internets to someone by cd [20:41] (we smoke alot of pot) [20:45] midas: I did not [20:45] but you should archive that CD [20:46] wow, i thought you already archived that [20:46] anyway, yeah. it's here somewhere [20:46] wanadoo cd i think [20:47] most people are astonished when they learn they can upload stuff to archive.org [20:47] I have noticed this also [20:48] midas: I'm _looking_ for these kind of CDs [20:48] but haven't found any [20:48] and I destroyed / repurposed my old stack of them as frisbees years ago [20:48] :( [20:48] even better when people understand they can see stuff at archive.org, it's like they are reborn [20:49] joepie91: ill start searching for cd's this weekend i think [20:49] midas: :D [20:49] also, the usual offer is still open [20:49] i hope.., you saw the jason room pictures right? [20:49] if you have old CDs with crap on it and you're too lazy to image them, send them to me :P [20:49] well if you're in NL anyway [20:49] because for the price of international shipping you might as well rent a private jet and come deliver them in person [20:49] midas: yes [20:49] yeah :p [20:50] well, think of that room, and put a bomb in it. [20:50] joepie91: oh, can I? [20:50] and fill the bomb with CD's, keycords, businesscards, random stuff [20:50] joepie91: do you also scan magazines? [20:51] Nemo_bis: yes, but that is more or less on hiatus until I get a decent A3 scanner [20:51] I do scan them, but very very slowly [20:51] and occasionally [20:52] midas: and then you have your room? :D [20:52] Hm. Well let me know if you run out of material. [20:52] joepie91: how fast are you with CDs? [20:52] Nemo_bis: will do, but I also have a habit of roaming around the streets on paper trash collection day and taking home boxes of magazines... :P [20:52] joepie91: more or less yeah [20:52] Nemo_bis: fast [20:52] I've optimized/automated it quite a bit [20:53] i had friends who compared my office space to 5 seconds after the hiroshima bomb. [20:53] home-made CD label scanning sleeve for faster scanning, both scanning and imaging scripted - effectively a while True: # wait for return key loop [20:53] joepie91: sounds cost effective, wouldn't work here because we have per-address bins and weekly collection [20:54] yeah, that'd be a problem [20:54] the place where I live - the city center anyway - just has off-the-street collection [20:54] on collection day you put boxes with paper / cardboard crap outside [20:54] and they drive by and pick it up [20:54] which is a sensible strategy when you consider that most houses here don't have space for a front lawn, let alone a collection bin [20:55] but it also makes it very easy to take home boxes of reading materials :) [20:55] and, as opposed to grocery store dumpsterdiving and large trash scouring, nobody gives a damn [20:55] ah yes, our famous kliko system [20:55] kliko's suck, i fell in one once. [20:56] lol [20:56] life long hatred against kliko's [20:56] yeah [20:56] midas: we don't have those here in the city center though... just electronic RFID card (wtf?) central collection bins for general trash [20:56] and off-the-street pickup for paper and plastic (wtf?) [20:57] plastic pickup is like twice a month, too (??!???!?!) [20:57] fyi, i smoke. yes, people hate me for it and think i should be shot for smoking. i dont care. what happened is that someone threw away my carton of smokes. [20:57] perhaps you can tell that we have a very wacky waste collection system here... [20:57] i like the plastic pickup system. [20:58] too bad i have a garage full of stuff + plasic bags full of plastic now [20:58] i hate those RFID trash cans it once got stuck and i kicked it and stuff and [20:59] we have RFID trashcans? [20:59] tried to open it multiple times with the card and wasted four times on nothing, and it can only take so much [21:00] midas: sadly, yes, we do [21:00] and you have to do per one big plastic bag [21:00] in Dordrecht we do [21:00] fucking awful things [21:00] broken half of the time [21:00] full the other half [21:00] and you get fined when you put bags next to the bin [21:00] yes like i'm saying too, since i started living in.nl [21:00] where the fuck do you need to put them otherwise, seriously [21:01] (they will actually cut open your bags, and look for anything with your address on it, to figure out who put the bag there, and then deliver a fine to you... I am not kidding) [21:01] and they also see each time you use the trash can in the municipal building [21:01] Leo_TCK: they do not -yet- charge by the use here (the collection bins), but it's almost certain that that's going to happen in not too long [21:01] there is no other reason to replace the cardless bins with card-requiring bins [21:02] this is card requiring one [21:02] (of course the cardless bins -never- broke down - they were fully mechanical) [21:02] the one i use [21:02] Leo_TCK: where do you live? [21:02] but they do subtract from it the money per use [21:02] woop woop woop off-topic siren [21:02] take it to #archiveteam-bs [21:02] ok [21:03] oh, I thought this was -bs, haha [21:03] sorry [21:03] Yeah, I was going to say.