#archiveteam 2015-06-19,Fri

↑back Search

Time Nickname Message
00:01 πŸ”— Start has joined #archiveteam
00:05 πŸ”— SketchCow WHat
00:06 πŸ”— SketchCow Huh, FOS rebooted
00:06 πŸ”— SketchCow Must have been a system issue
00:08 πŸ”— SketchCow rsync is back.
00:08 πŸ”— SketchCow Archivebot uploader is back.
00:08 πŸ”— SketchCow Web server is back.
00:09 πŸ”— SketchCow Freeze sourceforge project to Monday
00:09 πŸ”— SketchCow I am sure it will take a couple days to talk to these people.
00:09 πŸ”— arkiver SketchCow: is it ok if I write them a mail tomorrow?
00:10 πŸ”— arkiver Or are you going to talk to them (might be better)
00:11 πŸ”— Sanqui a lot of ArchiveBot jobs seem to have terminated.
00:11 πŸ”— SketchCow I'm going to writeto them
00:12 πŸ”— arkiver Ok
00:13 πŸ”— SketchCow Sent him something.
00:13 πŸ”— SketchCow Bowing and scraping not needed. Want to find out what speed needs to be done.
00:14 πŸ”— arkiver Ok, good
00:14 πŸ”— Apathy_ how do you politely word something that says "you're fucking everything up and we want to archive it all before it goes to shit"
00:14 πŸ”— arkiver I hope they are cooperative
00:15 πŸ”— SketchCow I just said I heard we were downloading too fast, what's a good speed
00:15 πŸ”— Apathy_ sounds good
00:15 πŸ”— SketchCow If they're not coorperative, I'll make noise.
00:15 πŸ”— SketchCow This is not my first rodeo, and they are not my first clowns
00:16 πŸ”— arkiver I remember twitpic
00:16 πŸ”— SketchCow I'm sure we blew up their shit infrastructure
00:16 πŸ”— SketchCow Well, twitpic guy's an ass
00:16 πŸ”— arkiver SketchCow: actually I don't think we blew up anything
00:16 πŸ”— arkiver it was running fine
00:17 πŸ”— arkiver I didn't really notice any slowdowns or such things
00:17 πŸ”— arkiver Zoocasa started!
00:17 πŸ”— arkiver not in the warrior currently
00:18 πŸ”— arkiver #zoohouse
00:18 πŸ”— arkiver https://github.com/ArchiveTeam/zoocasa-grab
00:19 πŸ”— SketchCow Andover is obviously a company on the outs. Reduced resources, small staff. Of course their network would be the least expensive options.
00:20 πŸ”— SketchCow I'm sure they saw a spike and freaked.
00:23 πŸ”— mistym has quit IRC (Remote host closed the connection)
00:27 πŸ”— Ungstein has quit IRC (Ping timeout: 265 seconds)
00:29 πŸ”— koo5 has quit IRC (Read error: Operation timed out)
00:50 πŸ”— SketchCow Started Halo uploading again, hopefully that will blow out soon
01:03 πŸ”— Boltsie has quit IRC (Ping timeout: 370 seconds)
01:15 πŸ”— Ungstein has joined #archiveteam
01:28 πŸ”— username1 has joined #archiveteam
01:29 πŸ”— JesseW has joined #archiveteam
01:30 πŸ”— schbirid2 has quit IRC (Read error: Operation timed out)
01:44 πŸ”— mistym has joined #archiveteam
02:03 πŸ”— aschmitz has quit IRC (Remote host closed the connection)
02:17 πŸ”— primus104 has quit IRC (Leaving.)
02:28 πŸ”— trill_ has joined #archiveteam
03:01 πŸ”— zenguy_pc has quit IRC (Read error: Connection reset by peer)
03:04 πŸ”— JRWR has quit IRC (Remote host closed the connection)
03:07 πŸ”— mistym has quit IRC (Remote host closed the connection)
03:18 πŸ”— zenguy_pc has joined #archiveteam
03:25 πŸ”— mistym has joined #archiveteam
03:40 πŸ”— JRWR has joined #archiveteam
03:53 πŸ”— antomati_ has joined #archiveteam
03:53 πŸ”— swebb sets mode: +o antomati_
03:54 πŸ”— zhongfu has quit IRC (Remote host closed the connection)
03:57 πŸ”— antomatic has quit IRC (Ping timeout: 370 seconds)
03:58 πŸ”— zhongfu has joined #archiveteam
04:12 πŸ”— Muad-Dib has joined #archiveteam
04:30 πŸ”— aaaaaaaaa has quit IRC (Leaving)
04:37 πŸ”— cjp__ has joined #archiveteam
04:39 πŸ”— cjp__ has quit IRC (Client Quit)
04:41 πŸ”— Boltsie has joined #archiveteam
04:42 πŸ”— Stiletto sounds like Toshiba is starting to kill off support for old models: http://www.vogons.org/viewtopic.php?f=46&t=43805
04:51 πŸ”— TheLovina has joined #archiveteam
05:02 πŸ”— Start at a first glance, everything seems to be on cdgenp01.csd.toshiba.com
05:03 πŸ”— Start googling site:csd.toshiba.com also shows some relics of their older site
05:10 πŸ”— Start download pages appear to be sequential: http://support.toshiba.com/support/viewContentDetail?contentId=4006772
05:12 πŸ”— Start their robots.txt prevents any downloads from going into wayback: http://cdgenp01.csd.toshiba.com/robots.txt
05:17 πŸ”— Famicoman has quit IRC (Ping timeout: 512 seconds)
05:21 πŸ”— JesseW has quit IRC (Read error: Operation timed out)
05:29 πŸ”— Start i created a wiki page for it: http://archiveteam.org/index.php?title=Toshiba_Support
05:29 πŸ”— * Start is afk for the night
05:31 πŸ”— Famicoman has joined #archiveteam
05:35 πŸ”— bzc6p_ has joined #archiveteam
05:35 πŸ”— swebb sets mode: +o bzc6p_
05:41 πŸ”— bzc6p has quit IRC (Ping timeout: 600 seconds)
05:49 πŸ”— signius What is the irc channel name for the zoocasa grab?
05:51 πŸ”— Elegance #zoohouse
05:59 πŸ”— signius thanks
06:18 πŸ”— WubTheCap has quit IRC (Quit: Leaving)
06:35 πŸ”— mistym has quit IRC (Remote host closed the connection)
07:35 πŸ”— mistym has joined #archiveteam
07:41 πŸ”— mistym has quit IRC (Ping timeout: 252 seconds)
07:46 πŸ”— jmc_ has joined #archiveteam
07:47 πŸ”— khaoohs_ has joined #archiveteam
07:48 πŸ”— primus104 has joined #archiveteam
07:49 πŸ”— wp494_ has joined #archiveteam
07:49 πŸ”— Start has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— kisspunch has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— wp494 has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— xtr-201 has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— jmc has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— kniffy has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— Riviera has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— goekesmi has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— useretail has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— khaoohs has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— sb057 has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— DFJustin has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— DFJustinZ has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— rduser has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— yuvadm_ has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— mr-b has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— w0rp has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— wacky has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— Sanqui has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— warthurto has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— chfoo- has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— dx- has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— thefinn93 has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— jk[SVP] has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— offby1 has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:49 πŸ”— matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca)
07:51 πŸ”— Start has joined #archiveteam
07:51 πŸ”— kisspunch has joined #archiveteam
07:51 πŸ”— xtr-201 has joined #archiveteam
07:51 πŸ”— kniffy has joined #archiveteam
07:51 πŸ”— Riviera has joined #archiveteam
07:51 πŸ”— goekesmi has joined #archiveteam
07:51 πŸ”— useretail has joined #archiveteam
07:51 πŸ”— SadDM has joined #archiveteam
07:51 πŸ”— Jonimus has joined #archiveteam
07:51 πŸ”— khaoohs has joined #archiveteam
07:51 πŸ”— sb057 has joined #archiveteam
07:51 πŸ”— rduser has joined #archiveteam
07:51 πŸ”— yuvadm_ has joined #archiveteam
07:51 πŸ”— wacky has joined #archiveteam
07:51 πŸ”— Sanqui has joined #archiveteam
07:51 πŸ”— warthurto has joined #archiveteam
07:51 πŸ”— dx- has joined #archiveteam
07:51 πŸ”— thefinn93 has joined #archiveteam
07:51 πŸ”— offby1 has joined #archiveteam
07:51 πŸ”— matthusby has joined #archiveteam
07:51 πŸ”— irc.shaw.ca sets mode: +o SadDM
07:51 πŸ”— swebb sets mode: +o SadDM
07:51 πŸ”— SadDM_ has joined #archiveteam
07:51 πŸ”— swebb sets mode: +o SadDM_
07:51 πŸ”— wacky has quit IRC (Read error: Connection reset by peer)
07:51 πŸ”— yuvadm_ has quit IRC (Read error: Connection reset by peer)
07:51 πŸ”— wacky_ has joined #archiveteam
07:51 πŸ”— kisspunch has quit IRC (Ping timeout: 370 seconds)
07:51 πŸ”— SadDM has quit IRC (Ping timeout: 370 seconds)
07:51 πŸ”— goekesmi has quit IRC (Remote host closed the connection)
07:51 πŸ”— goekesmi has joined #archiveteam
07:52 πŸ”— xtr-201 has quit IRC (Ping timeout: 370 seconds)
07:52 πŸ”— khaoohs has quit IRC (Ping timeout: 370 seconds)
07:52 πŸ”— jk[[SVP]] has joined #archiveteam
07:53 πŸ”— mr-b has joined #archiveteam
07:53 πŸ”— DFJustinZ has joined #archiveteam
07:54 πŸ”— kisspunch has joined #archiveteam
07:54 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
07:54 πŸ”— chfoo- has joined #archiveteam
07:57 πŸ”— dashcloud has joined #archiveteam
07:58 πŸ”— w0rp_ has joined #archiveteam
08:03 πŸ”— yuvadm has joined #archiveteam
08:05 πŸ”— w0rp_ is now known as w0rp
08:05 πŸ”— jk[[SVP]] is now known as jk[SVP]
08:23 πŸ”— primus104 has quit IRC (Leaving.)
08:23 πŸ”— landshark has quit IRC (Read error: Operation timed out)
08:25 πŸ”— jmc_ has quit IRC (Ping timeout: 362 seconds)
08:37 πŸ”— trill_ has quit IRC (Quit: Page closed)
09:00 πŸ”— arkiver I'm going to look into downloading all tiles from Yahoo maps
09:08 πŸ”— arkiver Ok, got it. We'll download all tiles from yahoo maps
09:08 πŸ”— arkiver Will see about road data and streetview too in a bit
09:24 πŸ”— antomati_ is now known as antomatic
09:27 πŸ”— HCross Morning all. I am going to get on Zoohouse once the git downloads
09:50 πŸ”— signius HCross, i wouldnt bother i set it up earlier & its rate limited so hard it would be faster to archive it with a pencil & paper
09:51 πŸ”— HCross I can see that now
09:51 πŸ”— signius They told me they got it covered & to work on urlteam or something else instead
09:52 πŸ”— HCross I might run both at once
09:53 πŸ”— signius Its utter nonsense that SF reckon they was getting slammed too hard with the rsync.....i do not believe that 20 odd users with $5 VPS boxes was enough to bring them to a crawl
09:53 πŸ”— HCross tbh, I was using a dedi
09:53 πŸ”— signius yeah but that really wouldnt make any difference
09:54 πŸ”— HCross yeah, as it had the performance of a vps
09:54 πŸ”— arkiver SketchCow: we'll be getting all tiles from yahoo maps
09:54 πŸ”— arkiver We'll start at the highest level and go down
09:54 πŸ”— HCross is that what zoohouse is
09:54 πŸ”— signius But if they were bought to a crawl so easily they got bigger issues to worry about with their backend & network infrastructure
09:54 πŸ”— arkiver We might not get the lowest level, that would be tens of billions of tiles
09:56 πŸ”— signius HCross, zoohouse is an estate agents (i think)
09:56 πŸ”— HCross yeah, what is the yahoo maps one
09:57 πŸ”— signius arkiver, I am assuming the Yahoo Tiles grab will require some boxes with decent storage capacity
09:57 πŸ”— arkiver maybe, I'm not sure
09:58 πŸ”— signius arkiver, Well depending what happens with the sourceforge debacle the Yahoo grab is def another project i would be happy to throw some resources at
09:58 πŸ”— arkiver ok
09:58 πŸ”— HCross ditto signius
09:59 πŸ”— signius I do take issue with how Yahoo just kill off projects with little or no notice
09:59 πŸ”— signius I also have the same gripe with Google for doing the same * def want to be involved with the Google Code project when it starts
10:12 πŸ”— HCross seems that zoocasa doesnt like us, all my stuff is 503'ing from their end
10:32 πŸ”— HCross seems to be going again
10:40 πŸ”— mistym has joined #archiveteam
10:54 πŸ”— mistym has quit IRC (Read error: Operation timed out)
10:54 πŸ”— bzc6p__ has joined #archiveteam
10:54 πŸ”— swebb sets mode: +o bzc6p__
11:01 πŸ”— bzc6p_ has quit IRC (Ping timeout: 600 seconds)
11:22 πŸ”— arkiver scripts for the yahoo maps grab are created
11:23 πŸ”— arkiver testing and then we'll start
11:23 πŸ”— arkiver well, not yet
11:27 πŸ”— HCross whats the IRCD
11:27 πŸ”— HCross IRC I mean
11:28 πŸ”— arkiver SketchCow: what do you think of grabbing tiles from yahoo maps? We'll start with the highest level. The scripts are ready, so we can start immediatly
12:03 πŸ”— arkiver Every later of yahoomaps has (2^layernumber)^2 tiles
12:04 πŸ”— arkiver lowest layernumber = 0, highest layernumber = 20
12:06 πŸ”— arkiver and for tile we're going to download:
12:08 πŸ”— arkiver http://localhost:8090/replay/20150619112120/http://1.base.maps.api.here.com/maptile/2.1/maptile/187ddf591c/normal.day/16/18667/25000/256/png8?lg=ENG&token=TrLJuXVK62IQk0vuXFzaig%3D%3D&requestid=yahoo.prod&app_id=eAdkWGYRoc4RfxVo0Z4B
12:08 πŸ”— arkiver http://localhost:8090/replay/20150619112120/http://1.aerial.maps.api.here.com/maptile/2.1/maptile/187ddf591c/satellite.day/16/18667/25000/256/jpg?lg=ENG&token=TrLJuXVK62IQk0vuXFzaig%3D%3D&requestid=yahoo.prod&app_id=eAdkWGYRoc4RfxVo0Z4B
12:08 πŸ”— arkiver http://localhost:8090/replay/20150619112120/http://1.aerial.maps.api.here.com/maptile/2.1/maptile/187ddf591c/hybrid.day/16/18667/25000/256/jpg?lg=ENG&token=TrLJuXVK62IQk0vuXFzaig%3D%3D&requestid=yahoo.prod&app_id=eAdkWGYRoc4RfxVo0Z4B
12:08 πŸ”— arkiver oops
12:08 πŸ”— arkiver http://1.base.maps.api.here.com/maptile/2.1/maptile/187ddf591c/normal.day/16/18667/25000/256/png8?lg=ENG&token=TrLJuXVK62IQk0vuXFzaig%3D%3D&requestid=yahoo.prod&app_id=eAdkWGYRoc4RfxVo0Z4B
12:08 πŸ”— arkiver http://1.aerial.maps.api.here.com/maptile/2.1/maptile/187ddf591c/satellite.day/16/18667/25000/256/jpg?lg=ENG&token=TrLJuXVK62IQk0vuXFzaig%3D%3D&requestid=yahoo.prod&app_id=eAdkWGYRoc4RfxVo0Z4B
12:08 πŸ”— arkiver http://1.aerial.maps.api.here.com/maptile/2.1/maptile/187ddf591c/hybrid.day/16/18667/25000/256/jpg?lg=ENG&token=TrLJuXVK62IQk0vuXFzaig%3D%3D&requestid=yahoo.prod&app_id=eAdkWGYRoc4RfxVo0Z4B
12:09 πŸ”— arkiver there's also live traffic, but I guess we don't need live traffic saved. It won't be really live anymore by the time yahoo maps is gone
12:21 πŸ”— Lowfry has joined #archiveteam
12:21 πŸ”— Lowfry WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD
12:22 πŸ”— Kazzy Lowfry: yahoosucks
12:22 πŸ”— Lowfry true lol
12:32 πŸ”— Lowfry_ has joined #archiveteam
12:34 πŸ”— L0WFRY has joined #archiveteam
12:34 πŸ”— L0WFRY has quit IRC (Client Quit)
12:38 πŸ”— sankin has joined #archiveteam
12:40 πŸ”— Lowfry has quit IRC (Ping timeout: 512 seconds)
12:41 πŸ”— mistym has joined #archiveteam
12:42 πŸ”— Lowfry_ has quit IRC (Ping timeout: 512 seconds)
12:44 πŸ”— Fusl has quit IRC (Read error: Operation timed out)
12:45 πŸ”— mistym has quit IRC (Read error: Operation timed out)
12:50 πŸ”— Fusl has joined #archiveteam
13:06 πŸ”— BlueMaxim has quit IRC (Read error: Connection reset by peer)
13:18 πŸ”— signius arkiver, is there a channel for the yahoo map grab
13:29 πŸ”— bisko has joined #archiveteam
13:30 πŸ”— primus104 has joined #archiveteam
13:40 πŸ”— koo5 has joined #archiveteam
14:03 πŸ”— Start has quit IRC (Disconnected.)
14:11 πŸ”— lexicon has quit IRC (Read error: Operation timed out)
14:11 πŸ”— xk_id has joined #archiveteam
14:13 πŸ”— lexicon has joined #archiveteam
14:16 πŸ”— koo5 has quit IRC (Read error: Operation timed out)
14:24 πŸ”— arkiver signius: no
14:27 πŸ”— signius ok
14:31 πŸ”— mistym has joined #archiveteam
14:33 πŸ”— DFJustin has joined #archiveteam
14:33 πŸ”— swebb sets mode: +o DFJustin
14:35 πŸ”— sankin has quit IRC (Leaving.)
14:42 πŸ”— wacky_ has quit IRC (Ping timeout: 265 seconds)
14:47 πŸ”— sankin has joined #archiveteam
14:55 πŸ”— InAUGral has joined #archiveteam
14:57 πŸ”— bisko has quit IRC (Read error: Operation timed out)
15:08 πŸ”— JesseW has joined #archiveteam
15:19 πŸ”— InAUGral has quit IRC (Ping timeout: 265 seconds)
15:21 πŸ”— HCross Do you need any server help for the yahoo grab
15:24 πŸ”— mistym has quit IRC (Remote host closed the connection)
15:27 πŸ”— Ungstein has quit IRC (Quit: Leaving.)
15:28 πŸ”— Ungstein has joined #archiveteam
15:28 πŸ”— arkiver HCross: currently not, I do if we are going to run this project
15:28 πŸ”— HCross ok, let me know
15:28 πŸ”— arkiver I need a "Go" from SketchCow for the yahoomaps project, because alle this is going to be hosted on archive.org
15:28 πŸ”— Boltsie has quit IRC (Read error: Connection reset by peer)
15:29 πŸ”— HCross got a box that has direct peering with yahoo I thinj
15:29 πŸ”— HCross think
15:29 πŸ”— arkiver great.
15:29 πŸ”— xtr-201 has joined #archiveteam
15:29 πŸ”— arkiver SketchCow: second project is Xfire.
15:30 πŸ”— arkiver Xfire is a gaiming website with currently more then 35 million users. http://crash.xfire.com/
15:30 πŸ”— HCross isnt that time warner cable or am I confusing it with something
15:30 πŸ”— arkiver It hosts screenshots, videos (million of videos) and game information.
15:31 πŸ”— arkiver Xfire should have shutdown 12 june, but is still online. This means it could shut down any moment now
15:32 πŸ”— arkiver https://twitter.com/buckleyw/status/609513240624664576
15:33 πŸ”— arkiver SketchCow: if we are going to download all videos and screenshots from the website we are getting many TB's, probably more then 30T
15:33 πŸ”— arkiver What do you think?
15:33 πŸ”— arkiver Videos grab scripts is ready
15:33 πŸ”— Ungstein has quit IRC (Quit: Leaving.)
15:33 πŸ”— arkiver Getting the rest of the website ready too
15:33 πŸ”— Ungstein has joined #archiveteam
15:34 πŸ”— HCross just give me a shout when you are ready
15:34 πŸ”— arkiver ok
15:36 πŸ”— JesseW has quit IRC (Quit: Leaving.)
15:39 πŸ”— bzc6p__ has quit IRC (Read error: Operation timed out)
15:39 πŸ”— JRWR that sounds like a long project, we will need a ton of warriors for the xfire project
15:40 πŸ”— Ungstein has quit IRC (Quit: Leaving.)
15:43 πŸ”— primus104 has quit IRC (Leaving.)
15:46 πŸ”— DFJustin we usually have more warriors than a site can actually handle
15:46 πŸ”— Ungstein has joined #archiveteam
15:56 πŸ”— Start has joined #archiveteam
16:07 πŸ”— bzc6p__ has joined #archiveteam
16:07 πŸ”— swebb sets mode: +o bzc6p__
16:10 πŸ”— mistym has joined #archiveteam
16:11 πŸ”— bzc6p__ is now known as bzc6p
16:17 πŸ”— mistym has quit IRC (Remote host closed the connection)
16:18 πŸ”— mistym has joined #archiveteam
16:21 πŸ”— Start arkiver: we should do a warrior project for support.toshiba.com
16:22 πŸ”— Start they've recently been purging old support downloads
16:22 πŸ”— Start i created a page for it: http://archiveteam.org/index.php?title=Toshiba_Support
16:23 πŸ”— bzc6p has quit IRC (Read error: Operation timed out)
16:32 πŸ”— koo5 has joined #archiveteam
16:32 πŸ”— xk_id has quit IRC (Remote host closed the connection)
16:32 πŸ”— Stiletto Start: the vogons thread has been updated to say two things. 1. the info is still obtainable via some of the country TLDs (ex. toshiba.co.uk) 2. some people may or may not have archived it all in years past (ie. not helpful info LOL)
16:33 πŸ”— Stiletto 1. *some of the info
16:35 πŸ”— bzc6p has joined #archiveteam
16:35 πŸ”— swebb sets mode: +o bzc6p
16:45 πŸ”— Start has quit IRC (Disconnected.)
16:55 πŸ”— xmc arkiver: why are you downloading yahoo maps
16:55 πŸ”— xmc they just use openstreetmap
16:55 πŸ”— xmc or commercial sources, i forget
16:56 πŸ”— aaaaaaaaa has joined #archiveteam
16:56 πŸ”— swebb sets mode: +o aaaaaaaaa
17:04 πŸ”— signius has quit IRC (Quit: Leaving)
17:06 πŸ”— signius has joined #archiveteam
17:06 πŸ”— signius has quit IRC (Client Quit)
17:06 πŸ”— signius has joined #archiveteam
17:07 πŸ”— bzc6p_ has joined #archiveteam
17:07 πŸ”— swebb sets mode: +o bzc6p_
17:07 πŸ”— bzc6p has quit IRC (Read error: Connection reset by peer)
17:08 πŸ”— bzc6p_ is now known as bzc6p
17:15 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
17:16 πŸ”— dashcloud has joined #archiveteam
17:19 πŸ”— aaaaaaaaa I believe Yahoo uses HERE, which provides maps for yahoo, bing, Amazon and a couple others
17:19 πŸ”— bzc6p has quit IRC (Ping timeout: 601 seconds)
17:20 πŸ”— aaaaaaaaa https://developer.here.com/
17:23 πŸ”— xk_id has joined #archiveteam
17:25 πŸ”— bzc6p has joined #archiveteam
17:25 πŸ”— swebb sets mode: +o bzc6p
17:26 πŸ”— username1 there are tools for tile grabbing btw
17:28 πŸ”— xmc yeah
17:29 πŸ”— yipdw yeah, HERE is fine
17:29 πŸ”— yipdw it's like one of a few profitable parts of Nokia
17:29 πŸ”— xmc i don't think it's worth the time, effort, and disk space.
17:29 πŸ”— yipdw the map data is also utterly unusable anyway
17:29 πŸ”— yipdw you might as well just improve OSM
17:30 πŸ”— username1 map tile archiving would be awesome though since they change styles etc
17:30 πŸ”— yipdw that said if Yahoo stores custom layers then those are probably worth going after
17:30 πŸ”— username1 but it is HUGE data
17:31 πŸ”— username1 http://wiki.openstreetmap.org/wiki/Tile_disk_usage for osm
17:32 πŸ”— yipdw yeah, fetching map data via HTTP would be prerendering it
17:32 πŸ”— yipdw 54,000 GB into IA is an ass move
17:32 πŸ”— yipdw I know HERE isn't OSM but even so
17:32 πŸ”— SimpBrain o.O
17:34 πŸ”— xmc i hate to be the cranky suspenders-wearing old man, but that's not a good idea
17:34 πŸ”— yipdw custom layer data e.g. KMLs or whatnot are typically much smaller *and* are a lot more interesting
17:34 πŸ”— username1 not to a cartographer like me ;)
17:34 πŸ”— yipdw so use the USGS data
17:35 πŸ”— yipdw or etc.
17:35 πŸ”— username1 i wish we had google maps tiles from every year
17:35 πŸ”— username1 just to see how the style evolved
17:35 πŸ”— xmc username1: you'd rather have tiles than source data?
17:35 πŸ”— xmc hm
17:35 πŸ”— xmc well, you can sample it i guess
17:35 πŸ”— username1 and the usability but that is even harder to capture
17:35 πŸ”— username1 different aspects
17:35 πŸ”— xmc i'm a geographer, not a cartographer :P
17:35 πŸ”— username1 :)
17:41 πŸ”— username1 is now known as schbirid
17:49 πŸ”— schbirid2 has joined #archiveteam
17:49 πŸ”— joepie91 Cyphertite is closing; https://gist.github.com/joepie91/1c659fe7704e98520e17
17:49 πŸ”— schbirid2 has quit IRC (Read error: Connection reset by peer)
18:12 πŸ”— cjp_ has joined #archiveteam
18:17 πŸ”— garyrh http://www.waybackhn.com/
18:35 πŸ”— SketchCow HELLO HELLO
18:35 πŸ”— SketchCow Zoocasa has asked us to pull back a little
18:35 πŸ”— SketchCow Can we half the thing
18:38 πŸ”— oldcad has joined #archiveteam
18:39 πŸ”— arkiver ok
18:39 πŸ”— arkiver SketchCow: it's at 150 per minute now
18:39 πŸ”— arkiver however, we will not make it at that speed
18:39 πŸ”— J08nY has joined #archiveteam
18:41 πŸ”— mistym has quit IRC (Remote host closed the connection)
18:41 πŸ”— arkiver SketchCow: have you read what I wrote about Yahoo Maps and Xfire?
18:43 πŸ”— landshark has joined #archiveteam
18:43 πŸ”— arkiver Xfire should have closed june 12, 35 million users
18:44 πŸ”— bzc6p arkiver: not "only" 24?
18:45 πŸ”— arkiver bzc6p: 35,783,196
18:46 πŸ”— SketchCow Arkiver - make it a quarter for now. 40 per minute.
18:46 πŸ”— arkiver SketchCow: ok
18:46 πŸ”— SketchCow I realize that we wouldn't make it. I'm trying to work with this guy.
18:46 πŸ”— arkiver they banned our useragent, I'll have to requeue some thinigs
18:46 πŸ”— arkiver Ok, thank you
18:46 πŸ”— SketchCow He did it because users couldn't get in.
18:46 πŸ”— SketchCow We DPOS'd
18:46 πŸ”— SketchCow Let me know, I'll mail him, he'll turn it back up
18:47 πŸ”— arkiver ok
18:47 πŸ”— arkiver Meanwhile I'll get the xfire scripts more ready, they now only support videos.
18:48 πŸ”— arkiver We need to make a decisions on that. The website is half-dead and grabbing all videos will be 10s of TBs
18:49 πŸ”— SketchCow No to yahoo maps
18:49 πŸ”— arkiver ok
18:51 πŸ”— SketchCow Yes to xfire
18:51 πŸ”— arkiver ok
18:51 πŸ”— SketchCow Sourceforge, we're doing pre-emptive downloading, because we think they're cocks. So we can work it out.
18:51 πŸ”— SketchCow But I suspect we're going to have issues with Zoocasa.
18:52 πŸ”— SketchCow Apparently entire computing staff is fired to one guy
18:52 πŸ”— SketchCow G'day Archiveteam.
18:52 πŸ”— SketchCow Appreciate that you guys are archiving Zoocasa.com, but could you please throttle things back a little bit? We're getting a lot of traffic from you and archivebot simultaneously as well as the general public, but your crawlers are being too aggressive and are basically DDOSing U.S. and everyone was getting 503s while our app servers became saturated. I've had to add you to a block list at least temp
18:52 πŸ”— SketchCow orarily until you can ease up a bit. Archivebot is crawling at a rate of 1.2 req/s with 4 connections and a 500-1000 ms delay if that helps, but your crawlers were just ton aggressive for our servers this week. Because we're in the middle of a shutdown there's not a lot I can do to add more resources, so if you guys could tone down your crawling a bit, I can remove the blocking. We're running at a b
18:52 πŸ”— SketchCow it of a disadvantage here because of the shutdown so throwing more servers at the app cluster probably won't be happening. My hands are a little tied. :/
18:52 πŸ”— SketchCow Cheers and thanks.
18:52 πŸ”— SketchCow - Jay
18:53 πŸ”— arkiver I see
18:54 πŸ”— arkiver I guess there's not a lot we can do there
18:55 πŸ”— bzc6p Maybe they could keep it up a few days after the 22nd?
18:56 πŸ”— landshark has quit IRC (Read error: Operation timed out)
18:58 πŸ”— Start has joined #archiveteam
19:00 πŸ”— Start_ has joined #archiveteam
19:00 πŸ”— Start has quit IRC (Read error: Connection reset by peer)
19:09 πŸ”— koo5 has quit IRC (Read error: Operation timed out)
19:15 πŸ”— arkiver SketchCow: our useragent is unbanned from zoocasa. We're running at 40/min now
19:21 πŸ”— JRWR arkiver: Xfire is going to be a bitch to mirror, any help you need, let me know
19:21 πŸ”— arkiver JRWR: ok
19:23 πŸ”— Stiletto has quit IRC (Read error: Operation timed out)
19:25 πŸ”— Start_ has quit IRC (Disconnected.)
19:29 πŸ”— K4k has joined #archiveteam
19:29 πŸ”— SketchCow zoocasa has lifted the ban
19:29 πŸ”— SketchCow He says he can whip up a rule to increase bandwidth as it goes
19:29 πŸ”— SketchCow he says 23rd is last day
19:29 πŸ”— SketchCow I am asking about us getting some special post-shutdown. But assume it goes down
19:29 πŸ”— Start has joined #archiveteam
19:30 πŸ”— iamcold has joined #archiveteam
19:31 πŸ”— JRWR Your doing good work SketchCow
19:31 πŸ”— ruukasu what's with the multiple files here? https://archive.org/details/archiveteam_pomf
19:31 πŸ”— Stiletto has joined #archiveteam
19:32 πŸ”— primus104 has joined #archiveteam
19:34 πŸ”— jmc has joined #archiveteam
19:37 πŸ”— mistym has joined #archiveteam
19:41 πŸ”— bzc6p ruukasu: The ~3 TB of Pomf archive is stored in 22.5 GB chunks, in separate items.
19:42 πŸ”— bzc6p They are already available in the Wayback Machine, though.
19:44 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
19:47 πŸ”— dashcloud has joined #archiveteam
19:52 πŸ”— Froggypwn has quit IRC (Quit: ~ Trillian Astra - www.trillian.im ~)
20:10 πŸ”— iamcold has quit IRC (Quit: Page closed)
20:13 πŸ”— aaaaaaaaa has quit IRC (Ping timeout: 600 seconds)
20:16 πŸ”— aaaaaaaaa has joined #archiveteam
20:16 πŸ”— swebb sets mode: +o aaaaaaaaa
20:21 πŸ”— Start has quit IRC (Disconnected.)
20:23 πŸ”— wp494_ has quit IRC (Quit: LOUD UNNECESSARY QUIT MESSAGES)
20:23 πŸ”— wp494 has joined #archiveteam
20:32 πŸ”— SimpBrai1 has joined #archiveteam
20:35 πŸ”— SimpBrain has quit IRC (Ping timeout: 258 seconds)
20:46 πŸ”— ruukasu bzc6p: I tried going to a pomf link in wayback and it said it wasn't there, do I have to change the url at all?
20:47 πŸ”— bzc6p ruukasu: you don't need to change the URL for wayback
20:49 πŸ”— bzc6p Well, maybe the wayback importing hasn't finished yet...
20:49 πŸ”— ruukasu yeah I've tried like 3 files from different time periods and they're all getting "page not archived"
20:51 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
20:54 πŸ”— sankin has quit IRC (Leaving.)
21:00 πŸ”— dashcloud has joined #archiveteam
21:03 πŸ”— garyrh It can take a few days/weeks for the warcs to be indexed by wayback.
21:07 πŸ”— primus105 has joined #archiveteam
21:11 πŸ”— primus104 has quit IRC (Read error: Operation timed out)
21:17 πŸ”— K4k has quit IRC (Ping timeout: 370 seconds)
21:23 πŸ”— scyther has joined #archiveteam
21:30 πŸ”— scyther has quit IRC (Read error: Connection reset by peer)
21:52 πŸ”— lbft has quit IRC (Read error: Operation timed out)
21:54 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
21:58 πŸ”— lbft has joined #archiveteam
22:00 πŸ”— arkiver SketchCow: zoocasa is currently at 40/min. can we do 300/min? or whatever it takes to get everything before the end of the 23rd?
22:01 πŸ”— dashcloud has joined #archiveteam
22:04 πŸ”— xmc no
22:04 πŸ”— xmc scroll back about 4 hours
22:19 πŸ”— arkiver ok
22:19 πŸ”— arkiver #xfired for xfire!
22:24 πŸ”— J08nY has quit IRC (Quit: Page closed)
22:45 πŸ”— koo5 has joined #archiveteam
22:46 πŸ”— TheLovina has quit IRC (Read error: Connection reset by peer)
22:46 πŸ”— TheLovina has joined #archiveteam
22:48 πŸ”— arkiver We have started the Xfire grab!
22:48 πŸ”— arkiver #xfired
22:49 πŸ”— Sanqui \o/
22:49 πŸ”— Sanqui save all you can!
22:54 πŸ”— DopefishJ has joined #archiveteam
22:54 πŸ”— swebb sets mode: +o DopefishJ
22:55 πŸ”— wp494 has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— xtr-201 has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— DFJustin has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— chfoo- has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— kisspunch has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— DFJustinZ has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— mr-b has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— SadDM_ has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— kniffy has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— Riviera has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— useretail has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— sb057 has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— rduser has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— Sanqui has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— warthurto has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— dx- has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— thefinn93 has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— offby1 has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:55 πŸ”— matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca)
22:59 πŸ”— landshark has joined #archiveteam
23:01 πŸ”— BlueMaxim has joined #archiveteam
23:01 πŸ”— chfoo-_ has joined #archiveteam
23:02 πŸ”— arkiver SketchCow: Xfire is running!
23:03 πŸ”— SN4T14_ has joined #archiveteam
23:04 πŸ”— sb058 has joined #archiveteam
23:07 πŸ”— wp494_ has joined #archiveteam
23:07 πŸ”— SN4T14 has quit IRC (Ping timeout: 306 seconds)
23:09 πŸ”— kisspunc- has joined #archiveteam
23:10 πŸ”— kisspunc- is now known as kisspunch
23:10 πŸ”— TheLovina has quit IRC (Read error: Connection reset by peer)
23:12 πŸ”— TheLovina has joined #archiveteam
23:13 πŸ”— dx has joined #archiveteam
23:13 πŸ”— warthurto has joined #archiveteam
23:14 πŸ”— wp494_ is now known as wp494
23:15 πŸ”— xk_id has quit IRC (Remote host closed the connection)
23:21 πŸ”— JRWR I cant seem to get wget-lua to compile on ubuntu 15.04
23:22 πŸ”— Kazzy pastie an error log, JRWR
23:23 πŸ”— JRWR POD document had syntax errors at /usr/bin/pod2man line 71
23:23 πŸ”— JRWR recompiling now, ill pull a full log soon
23:23 πŸ”— Emcy_ has quit IRC (Read error: Connection reset by peer)
23:23 πŸ”— Kazzy JRWR: https://github.com/ArchiveTeam/xfire-grab#wget-lua-was-not-successfully-built
23:23 πŸ”— Emcy_ has joined #archiveteam
23:24 πŸ”— SketchCow Ramp up zoocasa
23:24 πŸ”— SketchCow Double it
23:24 πŸ”— SketchCow Seewhat happens
23:24 πŸ”— SketchCow He says they're absolutely deleting on 23rd
23:25 πŸ”— JRWR If I show up with my station wagon full from LTO-4 Tapes, ask him if I can get a backup
23:25 πŸ”— Peetz0r has joined #archiveteam
23:25 πŸ”— Peetz0r_ has quit IRC (Read error: Connection reset by peer)
23:26 πŸ”— primus104 has joined #archiveteam
23:27 πŸ”— oldcad can you do something about the rate limiting? i've been hitting the rate limit for hours now
23:27 πŸ”— oldcad i tseems it's the same users who get jobs
23:29 πŸ”— primus105 has quit IRC (Read error: Operation timed out)
23:31 πŸ”— mr-b has joined #archiveteam
23:31 πŸ”— kniffy has joined #archiveteam
23:31 πŸ”— Riviera has joined #archiveteam
23:31 πŸ”— useretail has joined #archiveteam
23:31 πŸ”— Jonimus has joined #archiveteam
23:31 πŸ”— rduser has joined #archiveteam
23:31 πŸ”— Sanqui has joined #archiveteam
23:31 πŸ”— thefinn93 has joined #archiveteam
23:31 πŸ”— matthusby has joined #archiveteam
23:33 πŸ”— SketchCow Regardomg zoocasa - turn it up to maximum on Monday afternoon
23:34 πŸ”— _0x2A has joined #archiveteam
23:34 πŸ”— koo5 has quit IRC (Read error: Operation timed out)
23:35 πŸ”— mr-b has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— kniffy has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— Riviera has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— useretail has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— rduser has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— Sanqui has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— thefinn93 has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:35 πŸ”— oldcad I got one job just now.... I'll leave it overnight
23:35 πŸ”— arkiver SketchCow: zoocasa is at 100/min now]
23:37 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
23:38 πŸ”— mr-b has joined #archiveteam
23:38 πŸ”— kniffy has joined #archiveteam
23:38 πŸ”— Riviera has joined #archiveteam
23:38 πŸ”— useretail has joined #archiveteam
23:38 πŸ”— Jonimus has joined #archiveteam
23:38 πŸ”— rduser has joined #archiveteam
23:38 πŸ”— Sanqui has joined #archiveteam
23:38 πŸ”— thefinn93 has joined #archiveteam
23:38 πŸ”— matthusby has joined #archiveteam
23:38 πŸ”— SketchCow Thank youuuuu
23:38 πŸ”— SketchCow What IS Zoocasa
23:40 πŸ”— garyrh It's a real estate brokerage website thing.
23:41 πŸ”— oldcad http://www.torontorealtyblog.com/archives/9091 says: Zoocasa is a website, backed by Rogers, and headed up by Lawrence Dale (previously associated with Realty Sellers and T.O. Solds), whose goal, according to theirΒ website,Β is to β€œHelp people make smarter home buying and selling decisions.”
23:43 πŸ”— dashcloud has joined #archiveteam
23:45 πŸ”— mr-b has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— kniffy has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— Riviera has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— useretail has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— rduser has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— Sanqui has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— thefinn93 has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:45 πŸ”— matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca)
23:51 πŸ”— rduser` has joined #archiveteam
23:55 πŸ”— joepie91 https://github.com/conformal
23:55 πŸ”— joepie91 these all need archiving
23:55 πŸ”— joepie91 as do these: https://github.com/btcsuite
23:56 πŸ”— joepie91 but I have to do a bunch of stuff, so would be good if somebody else could do that
23:56 πŸ”— joepie91 (cc godane)

irclogger-viewer