#archiveteam-bs 2015-11-29,Sun

↑back Search

Time Nickname Message
00:06 🔗 RedType has quit IRC (Quit: leaving)
00:07 🔗 RedType has joined #archiveteam-bs
00:11 🔗 schbirid has quit IRC (Quit: Leaving)
00:26 🔗 RedType has quit IRC (Quit: leaving)
00:34 🔗 RedType has joined #archiveteam-bs
00:39 🔗 HCross SketchCow, can we archive 19TB of mouse genomesand send them onto FOS and the IA?
00:40 🔗 arkiver With the FTP project that is ^
00:40 🔗 arkiver SketchCow: some more info https://www.sanger.ac.uk/sanger/Mouse_SnpViewer/rel-1303
00:40 🔗 HCross http://www.sanger.ac.uk/science/data/mouse-genomes-project
00:41 🔗 aaaaaaaaa has joined #archiveteam-bs
00:48 🔗 arkiver To be exact, that FTP ^ is 19422781428266 bytes
01:47 🔗 bwn has quit IRC (Read error: Operation timed out)
03:20 🔗 primus104 has quit IRC (Leaving.)
03:21 🔗 BlueMaxim has joined #archiveteam-bs
03:22 🔗 chazchaz has quit IRC (Read error: Operation timed out)
03:23 🔗 espes__ has quit IRC (Read error: Operation timed out)
03:23 🔗 espes__ has joined #archiveteam-bs
03:32 🔗 chazchaz has joined #archiveteam-bs
03:33 🔗 joepie91 https://twitter.com/uppfinnarn/status/670229025445519360
03:33 🔗 joepie91 computers, ladies and gentlemen
03:46 🔗 godane first 10 issues of metropop that i got are getting uploaded
03:46 🔗 godane issues 400 to 410
04:05 🔗 godane https://archive.org/details/metropop-400
04:05 🔗 godane will fix metadata later
04:20 🔗 aaaaaaaaa has quit IRC (Leaving)
04:32 🔗 chazchaz has quit IRC (Read error: Operation timed out)
04:32 🔗 espes__ has quit IRC (Read error: Operation timed out)
04:34 🔗 espes__ has joined #archiveteam-bs
04:39 🔗 chazchaz has joined #archiveteam-bs
05:33 🔗 bwn has joined #archiveteam-bs
05:59 🔗 Sk1d has quit IRC (Ping timeout: 252 seconds)
06:58 🔗 vitzli has joined #archiveteam-bs
08:08 🔗 bwn has quit IRC (Read error: Connection reset by peer)
08:09 🔗 bwn has joined #archiveteam-bs
08:10 🔗 remsen has quit IRC (Read error: Operation timed out)
08:21 🔗 primus104 has joined #archiveteam-bs
08:37 🔗 schbirid has joined #archiveteam-bs
09:16 🔗 cvb has joined #archiveteam-bs
09:20 🔗 bwn has quit IRC (Read error: Operation timed out)
09:29 🔗 godane so i'm at 519k items now
09:53 🔗 primus104 has quit IRC (Leaving.)
10:01 🔗 vitzli has quit IRC (Quit: Leaving)
10:59 🔗 BlueMaxim has quit IRC (Read error: Connection reset by peer)
11:16 🔗 godane uploaded: https://archive.org/details/russian-Katalog_Lego-1994
11:25 🔗 remsen has joined #archiveteam-bs
12:10 🔗 primus104 has joined #archiveteam-bs
12:19 🔗 vitzli has joined #archiveteam-bs
12:20 🔗 schbirid godane: holy nostalgia, i remember that catalogue (in german) from my lego heydays :)
12:35 🔗 VADemon has joined #archiveteam-bs
12:36 🔗 godane i figure people will like that
12:43 🔗 godane btw i'm starting to get 2400k TheBlaze TV videos again
12:44 🔗 godane whats funny is it just started happening when i was getting 500k copies of Glenn Beck and Dana
12:44 🔗 godane then 2015-11-10 of dana when into 2400k
12:45 🔗 godane anyways i'm grabbing some of the documentary at 2400k since i only have 500k copies of those
13:20 🔗 godane so looks like i may have missed adding the 4 hour mp3 to some of glenn beck insider podcast
13:21 🔗 godane i believe the 4 hour started in july 2010
13:27 🔗 primus104 has quit IRC (Leaving.)
13:49 🔗 cvb has quit IRC (Quit: Leaving)
13:56 🔗 arkiver2 has joined #archiveteam-bs
14:04 🔗 Lord_Nigh has quit IRC (Ping timeout: 252 seconds)
14:05 🔗 Lord_Nigh has joined #archiveteam-bs
14:09 🔗 arkiver2 has quit IRC (Ping timeout: 252 seconds)
16:08 🔗 joepie91 well then
16:08 🔗 joepie91 http://pastebin.com/scraping
16:08 🔗 joepie91 pretty sure this page wasn't there before
16:08 🔗 joepie91 lol
16:08 🔗 joepie91 interesting
16:09 🔗 joepie91 so pastebin offers a lifetime pro account for $30 at the moment
16:09 🔗 joepie91 which gives you access to their scraping API
16:09 🔗 joepie91 do we care about scraping pastebin enough to drop $30on it?
16:14 🔗 SimpBrain meh
16:23 🔗 HCross Sure we could split it up if needs be
16:29 🔗 * joepie91 ping phuzion
16:30 🔗 joepie91 so, they require you to use one whitelisted IP
16:30 🔗 joepie91 I can probably set up scraping
16:30 🔗 phuzion Yeah, that's the only thing I particularly noticed about it was the single IP restriction
16:30 🔗 joepie91 if there's no risk of them changing the layout or banning me for it, then it should be pretty much zero-maintenance
16:30 🔗 joepie91 phuzion: I can kind of understand it, though
16:31 🔗 phuzion Yeah, otherwise they'd get shared all over the place
16:31 🔗 joepie91 yep
16:31 🔗 phuzion Like we were planning to do :)
16:31 🔗 joepie91 especially given that it's lifetime
16:31 🔗 joepie91 heh
16:31 🔗 joepie91 I can always just set up a proxy
16:31 🔗 phuzion Huh
16:31 🔗 joepie91 to which I can add credentials for archiveteam people
16:31 🔗 phuzion Right
16:31 🔗 joepie91 and that internally rate limits
16:31 🔗 phuzion But what's the point if you can just do the scraping yourself?
16:31 🔗 joepie91 phuzion: we might have, for example, people who need to scrape specific pastes
16:31 🔗 joepie91 to fetch URLs from
16:31 🔗 joepie91 stuff like that
16:32 🔗 phuzion Ahhhh, gotcha.
16:32 🔗 joepie91 and if we're going to pool $ for a pastebin account, then we might as well make it a collective one
16:32 🔗 phuzion Right
16:33 🔗 phuzion That's shockingly cheap for essentially unfettered access to all of pastebin though, isn't it?
16:33 🔗 joepie91 yes
16:33 🔗 joepie91 ish
16:33 🔗 joepie91 compared to other paid APIs, yes, it's cheap
16:33 🔗 joepie91 but the idea of charging for an API is a bit stupid to begin with
16:33 🔗 joepie91 :p
16:33 🔗 phuzion Depends on what the API does, IMO.
16:34 🔗 joepie91 not really
16:34 🔗 joepie91 see, the thing is
16:34 🔗 joepie91 an API call is cheaper than a frontend call for the same data
16:34 🔗 joepie91 to the provider
16:34 🔗 joepie91 so why limit API calls but not frontend calls?
16:34 🔗 joepie91 sure, I can understand limits for eg. complex search queries
16:34 🔗 phuzion frontend calls are assumed to also be generating ad revenue.
16:34 🔗 joepie91 but those should be enforced in the frontend also
16:34 🔗 joepie91 not if scrapers use them
16:34 🔗 joepie91 that's my point
16:35 🔗 joepie91 if somebody wants to scrape, they can either use the API or the frontend
16:35 🔗 joepie91 neither is going to bring in ad rev
16:35 🔗 joepie91 but the API is cheaper for them to provide
16:35 🔗 joepie91 so why make that the one with the limit? :P
16:35 🔗 phuzion So give them access to the API so they can do so in a less costly manner, gotcha
16:35 🔗 joepie91 basically
16:35 🔗 joepie91 it benefits both provider and consumer to have a limitless API (or equivalent limits to the frontend)
16:36 🔗 joepie91 anyhow, phuzion, HCross, ideas on $?
16:36 🔗 joepie91 my current budget remains 0 until hopefully-soon, but idk how long that deal is going to be availabl for
16:36 🔗 joepie91 available*
16:36 🔗 joepie91 they call it a "black friday deal" but it's not like pastebin isn't _constantly_ running promos
16:37 🔗 phuzion I can chip in $10 or something, but I don't have paypal.
16:37 🔗 HCross they always seem to be doing promos. My budget is 0 until the start of Dec when I get paid
16:37 🔗 joepie91 phuzion: what do you have?
16:38 🔗 HCross Where in the world are you?
16:38 🔗 joepie91 who, me? or phuzion?
16:38 🔗 phuzion Unless you take credit cards directly, I probably can't give you money over the internet now that I think about it.
16:38 🔗 phuzion lol
16:38 🔗 HCross phuzion
16:38 🔗 phuzion I'm in the US, btw.
16:38 🔗 joepie91 phuzion: you can go through my donation form... :P
16:38 🔗 joepie91 creditcard-via-paypal
16:40 🔗 phuzion joepie91: let me know once you've got a few other people to chip in on this and I'll send a few bucks your way for it.
16:41 🔗 joepie91 noted :P I will await responses from others...
16:41 🔗 joepie91 so, because it has scrolled out of view: pooling together $30 for a pastebin pro lifetime account, which gives API access for one(?) IP which means we can scrape properly without blocks
16:41 🔗 joepie91 anybody who wants to chip in, please maketh thyself known
16:42 🔗 HCross joepie91, you going to hold the Pastebin acc? Or let someone like SketchCow own it
16:42 🔗 joepie91 HCross: I don't mind either way, but it's probably most practical if the one doing the scraping has access to the account (because of the IP whitelisting)
16:42 🔗 joepie91 even if that's just an `archiveteam` account changing hands when needed
16:42 🔗 HCross ye, just wondering if they will pick up on account sharing
16:43 🔗 joepie91 shouldn't matter
16:43 🔗 joepie91 it's just going to be one IP using it
16:44 🔗 primus104 has joined #archiveteam-bs
16:48 🔗 dashcloud joepie91: how do you accept money? I can probably give you the $30 depending on if I can get you the money
16:48 🔗 HCross PayPal I think dashcloud
16:49 🔗 HCross or CC via PP
16:49 🔗 ndiddy has joined #archiveteam-bs
16:49 🔗 dashcloud got a link to your site joepie91?
16:50 🔗 joepie91 dashcloud: http://cryto.net/~joepie91/donate.html
16:50 🔗 joepie91 dashcloud: plus SEPA and such
16:55 🔗 achip joepie91 I can pitch the $30 for the lifetime account, if the funding isn't there now
16:55 🔗 HCross achip, looks like dashcloud has it. You could go halves?
16:55 🔗 achip if dashcloud would like, sure
16:58 🔗 * joepie91 will await some kind of agreement
16:59 🔗 joepie91 (I'll just register an 'archiveteam' account, although with a less obvious name - sleeping dogs etc)
16:59 🔗 dashcloud sure- I can't seem to edit my selection from $30 though without cancelling this transaction though
16:59 🔗 joepie91 (and I'll CC credentials to SketchCow so that the bus factor is 2)
17:01 🔗 dashcloud okay- the money should be on its way to you now
17:02 🔗 joepie91 dashcloud: St**** Za*****?
17:02 🔗 joepie91 (idk if your name is public)
17:02 🔗 dashcloud yep
17:02 🔗 joepie91 okay :P
17:02 🔗 joepie91 you actually sent euro
17:02 🔗 joepie91 heh
17:02 🔗 dashcloud that's what I was offered
17:02 🔗 joepie91 though I don't think it differs much after transaction fees
17:02 🔗 joepie91 yeah
17:02 🔗 joepie91 right
17:03 🔗 joepie91 28.48 post-transaction-fees
17:03 🔗 joepie91 == 30.17 $
17:03 🔗 dashcloud apparently the difference isn't that much right nwo
17:03 🔗 joepie91 lol
17:03 🔗 joepie91 yeah, exchange rate is shit at the moment
17:03 🔗 joepie91 dashcloud: can you think of a fun non-obvious username?
17:03 🔗 dashcloud for this project?
17:03 🔗 joepie91 dashcloud: for the pastebin account
17:03 🔗 joepie91 for archiveteam
17:04 🔗 joepie91 :p
17:04 🔗 dashcloud is f451 too obvious?
17:04 🔗 joepie91 I.. have no idea what that means
17:04 🔗 dashcloud fahrenheit 451, the famous Ray Bradbury story
17:04 🔗 joepie91 ahh
17:04 🔗 joepie91 worksforme
17:04 🔗 joepie91 :P
17:05 🔗 joepie91 achip: dashcloud already paid the full amount
17:06 🔗 dashcloud if you want to donate anyway, it won't be a waste
17:06 🔗 joepie91 will leave that decision up to achip :P I can send back the $ if wanted
17:07 🔗 SN4T14 has quit IRC (Read error: Operation timed out)
17:07 🔗 joepie91 mmk, it should activate pro status in a few mins
17:08 🔗 joepie91 ..
17:08 🔗 joepie91 dashcloud: I typoed the username
17:08 🔗 joepie91 gg me
17:08 🔗 joepie91 ah well, less obvious I guses
17:08 🔗 joepie91 guess*
17:10 🔗 joepie91 okay, so
17:10 🔗 joepie91 if I get hit by a bus, poke SketchCow - just sent him a copy of the account credentials
17:10 🔗 joepie91 if SketchCow gets hit by a bus, we're probably all doomed
17:13 🔗 * ersi hops on the Magic School bus
17:27 🔗 xmc joepie91: can you op swebb
17:37 🔗 joepie91 sets mode: +o swebb
17:37 🔗 swebb sets mode: +o DFJustin
17:37 🔗 swebb sets mode: +o SadDM
17:37 🔗 swebb sets mode: +o antomatic
17:37 🔗 swebb sets mode: +o balrog
17:37 🔗 swebb sets mode: +o xmc
17:37 🔗 joepie91 xmc: there you go
17:37 🔗 xmc thanks :)
17:37 🔗 joepie91 ops-spreader engaged
17:37 🔗 joepie91 :p
17:37 🔗 joepie91 looks like housemates may not be coming back home tonight
17:38 🔗 joepie91 went to a protest, and 20 arrests have been reported...
17:40 🔗 xmc :|
17:41 🔗 remsen has quit IRC (Read error: Operation timed out)
17:41 🔗 joepie91 xmc: they'll probably be released in the morning
17:41 🔗 joepie91 as usually happens
17:41 🔗 joepie91 "oh we didn't actually have a reason to arrest you, sorry, out you go"
17:41 🔗 joepie91 pretty much a standard tactic for defusing protests in NL
17:54 🔗 SN4T14 has joined #archiveteam-bs
18:09 🔗 xmc same thing here, but with more beatings and fewer apologies
18:10 🔗 vitzli no apologies here
18:14 🔗 vitzli good night all
18:14 🔗 vitzli has quit IRC (Leaving)
18:14 🔗 joepie91 I kind of imagined the apology, this isn't Canada :)
18:17 🔗 schbirid thanks gnome3 for crashing whenyou try to run your fucking stupid new file dialog
18:42 🔗 Microguru has quit IRC (Quit: Microguru)
18:45 🔗 joepie91 btw
18:45 🔗 joepie91 http://www.polygon.com/2015/8/18/9173621/ryan-north-stuck-hole-twitter
18:45 🔗 joepie91 Twitter Plays Stuck-In-A-Hole
18:45 🔗 joepie91 (cc SketchCow )
18:53 🔗 Sanqui I think we got that when it happened P:
18:57 🔗 ndiddy joepie91, it's like that episode of it's always sunny
19:03 🔗 primus104 has quit IRC (Leaving.)
19:11 🔗 nomadpeng has joined #archiveteam-bs
19:33 🔗 joepie91 tracker not listing anything?
19:34 🔗 primus104 has joined #archiveteam-bs
19:42 🔗 kyan joepie91, what are the protests? Couldn't find anything on google news...
19:42 🔗 joepie91 kyan: Pegida == neonazis, essentially
19:43 🔗 joepie91 unfortunately it's being imported to NL
19:43 🔗 joepie91 they were doing a protest today, and there was a corresponding anti-protest
19:43 🔗 Start has quit IRC (Ping timeout: 310 seconds)
19:44 🔗 kyan aah :(
19:44 🔗 Start has joined #archiveteam-bs
19:44 🔗 kyan thanks for enlightening me :)
19:45 🔗 joepie91 kyan: I should clarify
19:45 🔗 joepie91 neonazis pretending not to be neonazis
19:45 🔗 joepie91 doing the entire "concerned citizen" spiel
19:45 🔗 joepie91 you know the drill, probably
19:45 🔗 joepie91 it's fairly telling that the founder of the organization is an arms dealer
19:45 🔗 kyan I've heard of similar, but never run across them in real life thanks goodnes
19:46 🔗 * kyan is lucky to live in an area without too many nutcases
19:48 🔗 ersi I'm sure there's a few actual concerned citizens in those groups as well
19:48 🔗 joepie91 sure, they're just hitched up by the shady folks
19:48 🔗 joepie91 something something Hitler
19:55 🔗 aaaaaaaaa has joined #archiveteam-bs
19:55 🔗 swebb sets mode: +o aaaaaaaaa
19:59 🔗 godane has quit IRC (Leaving.)
20:01 🔗 godane has joined #archiveteam-bs
20:15 🔗 wyatt8750 is now known as wyatt8749
20:15 🔗 wyatt8749 is now known as wyatt8740
20:44 🔗 kyan Is DocumentCloud mirrored to IA?
20:48 🔗 joepie91 kyan: it was until two years ago, it seems: https://archive.org/details/documentcloud
20:48 🔗 kyan Hmm.
20:48 🔗 ersi Cool, gwern is fiddling about with the AT datasets
20:48 🔗 joepie91 ersi: oh?
20:49 🔗 ersi Saw some PR's to warcat on Github from gwern
20:49 🔗 ersi no, not PRs - opened issues
20:49 🔗 ersi They're well written (of course, it's gwern :) )
20:49 🔗 joepie91 I seem to keep running into gwern everywhere
20:49 🔗 joepie91 lol
20:50 🔗 godane so this is the lion king magazine i'm uploading: http://lionking.wikia.com/wiki/The_Lion_King:_A_Nature_Fun_and_Learn_Series
20:51 🔗 aaaaaaaaa has quit IRC (Read error: Connection reset by peer)
20:52 🔗 aaaaaaaaa has joined #archiveteam-bs
20:52 🔗 swebb sets mode: +o aaaaaaaaa
20:54 🔗 aaaaaaaaa sets mode: +oooo chfoo closure godane midas
20:54 🔗 aaaaaaaaa sets mode: +oo nico_32 yipdw
20:56 🔗 schbirid has quit IRC (Quit: Leaving)
20:57 🔗 godane uploaded: https://archive.org/details/The_Lion_King_-_A_Nature_Fun_and_Learn_Series-01
21:15 🔗 VADemon has quit IRC (Read error: Connection reset by peer)
21:16 🔗 godane has quit IRC (Quit: Leaving.)
21:17 🔗 aaaaaaaaa has quit IRC (Read error: Connection reset by peer)
21:17 🔗 aaaaaaaaa has joined #archiveteam-bs
21:17 🔗 swebb sets mode: +o aaaaaaaaa
21:18 🔗 godane has joined #archiveteam-bs
21:25 🔗 nomadpeng has quit IRC (Quit: Leaving)
21:45 🔗 Start has quit IRC (Read error: Connection reset by peer)
21:45 🔗 Start has joined #archiveteam-bs
21:46 🔗 ens has joined #archiveteam-bs
21:59 🔗 balrog has quit IRC (Bye)
22:13 🔗 balrog has joined #archiveteam-bs
22:13 🔗 swebb sets mode: +o balrog
22:23 🔗 bwn has joined #archiveteam-bs
22:36 🔗 chazchaz has quit IRC (Read error: Operation timed out)
22:37 🔗 espes__ has quit IRC (Read error: Operation timed out)
22:42 🔗 chazchaz has joined #archiveteam-bs
22:50 🔗 BlueMaxim has joined #archiveteam-bs
22:57 🔗 ParkerR has joined #archiveteam-bs
22:57 🔗 espes__ has joined #archiveteam-bs
22:57 🔗 ParkerR ndiddy, Ok master. whatever you say
22:58 🔗 ndiddy ...
22:58 🔗 ParkerR :)
23:01 🔗 ndiddy so... if you want to do stuff there's the archive team warrior which is a virtualbox appliance that you run and automatically backs up websites
23:02 🔗 ndiddy http://www.archiveteam.org/index.php?title=Main_Page
23:17 🔗 Marcelo has joined #archiveteam-bs
23:36 🔗 ParkerR has quit IRC (Remote host closed the connection)
23:37 🔗 ParkerR has joined #archiveteam-bs
23:44 🔗 nightpool has joined #archiveteam-bs
23:47 🔗 DMackey has joined #archiveteam-bs
23:56 🔗 Marcelo has quit IRC (Quit: Page closed)

irclogger-viewer