[00:01] *** Ravenloft has quit IRC (Ping timeout: 240 seconds) [00:19] *** Ymgve has quit IRC () [00:57] *** philpem has quit IRC (Ping timeout: 260 seconds) [01:00] *** mistym has joined #archiveteam [01:09] *** wp494_ has joined #archiveteam [01:09] *** wp494_ has quit IRC (Excess Flood) [01:09] *** Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** garyrh has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** DFJustin has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** trs80 has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** wp494 has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** gibigiana has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** Start has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** Jogie has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** xtr-107 has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** tephra has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** rduser has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** thefinn93 has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** twrist has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** will__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** dugo has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** wacky has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** goekesmi has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** dx has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:09] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:10] *** wp494_ has joined #archiveteam [01:10] *** NotGLaDOS has joined #archiveteam [01:11] *** tephra_ has joined #archiveteam [01:12] *** gibigian1 has joined #archiveteam [01:16] *** dx- has joined #archiveteam [01:19] *** Start_ has joined #archiveteam [01:20] *** dugo_ has joined #archiveteam [01:23] *** goekesmi_ has joined #archiveteam [01:23] *** Start_ has quit IRC (Client Quit) [01:24] *** wacky_ has joined #archiveteam [01:24] *** Start_ has joined #archiveteam [01:25] *** thefinn93 has joined #archiveteam [01:27] *** rduser has joined #archiveteam [01:27] *** torvik has joined #archiveteam [01:27] *** Jogie has joined #archiveteam [01:27] *** will has joined #archiveteam [01:27] *** matthusby has joined #archiveteam [01:27] *** SadDM has joined #archiveteam [01:31] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:31] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:31] *** Jogie has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:31] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:31] *** rduser has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:31] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:35] *** garyrh has joined #archiveteam [01:35] *** Start_ is now known as Start [01:36] *** Start has quit IRC (Quit: Disconnected.) [01:39] *** trs80 has joined #archiveteam [01:39] *** matthusby has joined #archiveteam [01:39] *** will has joined #archiveteam [01:39] *** Jogie has joined #archiveteam [01:39] *** rduser has joined #archiveteam [01:39] *** torvik has joined #archiveteam [01:42] *** rduser` has joined #archiveteam [01:42] *** trs80 has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:42] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:42] *** Jogie has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:42] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:42] *** rduser has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:42] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [01:47] *** londoncal has quit IRC (Remote host closed the connection) [01:50] *** cbb2 has joined #archiveteam [01:58] *** rduser` is now known as rduser [02:00] *** trs80 has joined #archiveteam [02:00] *** matthusby has joined #archiveteam [02:00] *** will has joined #archiveteam [02:00] *** Jogie has joined #archiveteam [02:00] *** torvik has joined #archiveteam [02:00] *** xtr-201 has joined #archiveteam [02:01] *** wacky_ is now known as wacky [02:04] *** xtr-201 has quit IRC (ircd.shaw.ca irc.shaw.ca) [02:04] *** trs80 has quit IRC (ircd.shaw.ca irc.shaw.ca) [02:04] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [02:04] *** Jogie has quit IRC (ircd.shaw.ca irc.shaw.ca) [02:04] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [02:04] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [02:08] *** Jogie_ has joined #archiveteam [02:22] *** cbb2 has quit IRC (Quit: cbb2) [02:33] *** torvik has joined #archiveteam [02:33] *** matthusby has joined #archiveteam [02:34] *** SadDM has joined #archiveteam [03:14] *** xtr-201 has joined #archiveteam [03:14] *** lytv has quit IRC (Read error: Connection reset by peer) [03:17] *** kyan has joined #archiveteam [03:17] *** lytv has joined #archiveteam [03:19] *** Start has joined #archiveteam [03:40] *** godane has quit IRC (Quit: Leaving.) [03:43] *** primus104 has quit IRC (Leaving.) [04:02] *** mistym has quit IRC (Remote host closed the connection) [04:02] *** mistym has joined #archiveteam [04:05] *** VADemon has quit IRC (Quit: left4dead) [04:56] *** cloudmons has joined #archiveteam [04:58] *** mistym has quit IRC (Remote host closed the connection) [05:10] *** DFJustin has joined #archiveteam [05:27] *** mistym has joined #archiveteam [05:42] *** godane has joined #archiveteam [05:45] Hey hey. [05:51] *** signius has quit IRC (Read error: Operation timed out) [06:03] Hi all, [06:03] *** dashcloud has quit IRC (Read error: Connection reset by peer) [06:03] Now that RS is going down is there any effort to preserve the [06:03] information that is on their website, which I assume will go down at [06:03] some point (I could be wrong though)? Apologies if this has been [06:03] discussed before. [06:03] Things like this for example (pinouts, specs, drivers etc) [06:03] http://support.radioshack.com/productinfo/DocumentResults.asp?sku_id=25-3539&Name=Tandy%20Laptops%20and%20Portable%20Computers&Reuse=N [06:04] *** dashcloud has joined #archiveteam [06:05] *** signius has joined #archiveteam [06:07] when i read RS i first thought you wore talking about RapidShare [06:21] *** chfoo has quit IRC (Ping timeout: 512 seconds) [06:27] *** chfoo has joined #archiveteam [06:34] We already have at least some of it covered via ArchiveBot: http://archive.fart.website/archivebot/viewer/?q=radioshack [06:34] How do we know if this coverage is good enough? [06:36] you can sample it in the Wayback Machine [06:36] *** johtso has quit IRC (Quit: Connection closed for inactivity) [06:36] or whatever tools you prefer by downloading WARCs [06:40] *** chfoo0 has joined #archiveteam [06:45] *** chfoo has quit IRC (Ping timeout: 512 seconds) [06:49] *** mistym has quit IRC (Remote host closed the connection) [07:14] *** bmcginty has joined #archiveteam [07:33] *** trs80 has joined #archiveteam [07:33] *** xtr-201 has quit IRC (Read error: Connection reset by peer) [07:37] *** Control-S has joined #archiveteam [07:41] Random question: If I upload my own Wpull-generated WARC to IA someplace, will they suck that into the Wayback Machine automatically or does it need to go into a special collection? [07:44] *** xtr-201 has joined #archiveteam [07:45] *** Ctrl-S has quit IRC (Read error: Operation timed out) [07:45] *** Control-S is now known as Ctrl-S [08:41] *** primus104 has joined #archiveteam [08:57] *** schbirid has joined #archiveteam [09:22] *** Jon has joined #archiveteam [09:30] *** londoncal has joined #archiveteam [09:31] cloudmons: you'll have to poke SketchCow about it, he can make it be ingested [09:32] it indeed needs to go into a specific collection, but you need admin perms to move stuff in there [09:38] *** primus104 has quit IRC (Leaving.) [09:56] /sb levelclear -level joins,quits,nicks,parts [09:56] bah [09:58] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [10:00] *** BlueMaxim has joined #archiveteam [10:11] *** BlueMaxim has quit IRC (Quit: Leaving) [10:33] *** Ymgve has joined #archiveteam [10:34] *** Coderjoe has quit IRC (Ping timeout: 260 seconds) [10:36] *** Coderjoe has joined #archiveteam [11:04] *** T31M has joined #archiveteam [11:16] *** Emcy_ has quit IRC (Ping timeout: 512 seconds) [11:38] *** Froggypwn has joined #archiveteam [12:05] *** primus104 has joined #archiveteam [12:28] *** Emcy has joined #archiveteam [12:39] *** Emcy has quit IRC (Ping timeout: 306 seconds) [13:21] *** wp494_ is now known as wp494 [13:46] *** Start has quit IRC (Disconnected.) [13:56] *** primus104 has quit IRC (Leaving.) [14:10] *** Start has joined #archiveteam [14:13] *** Start has quit IRC (Client Quit) [14:23] *** goekesmi_ has quit IRC (Remote host closed the connection) [14:24] *** scyther has joined #archiveteam [14:26] *** goekesmi has joined #archiveteam [14:32] *** mistym has joined #archiveteam [14:39] *** scyther has quit IRC (Read error: Connection reset by peer) [14:54] *** ionpulse has quit IRC (Remote host closed the connection) [15:02] *** Start has joined #archiveteam [15:31] *** dashcloud has quit IRC (Ping timeout: 265 seconds) [15:33] *** dashcloud has joined #archiveteam [15:38] *** johtso has joined #archiveteam [15:39] hmm, is the rate of urlteam tracker limited in some way? the total speed never really seems to break 1k/s, and I'm adding more nodes and not really seeing an increase in throughput.. [15:39] is there maybe an issue distributing work fast enough? [15:49] *** chfoo0 is now known as chfoo [15:52] *** Start has quit IRC (Disconnected.) [15:54] *** mistym has quit IRC (Remote host closed the connection) [15:55] *** Emcy has joined #archiveteam [15:57] *** Start has joined #archiveteam [15:58] *** Start_ has joined #archiveteam [15:58] *** Start has quit IRC (Read error: Connection reset by peer) [16:05] *** primus104 has joined #archiveteam [16:09] it doesn't always have much work to do [16:09] *** mistym has joined #archiveteam [16:20] hmm, maybe my rate is creeping up a bit.. [16:21] Smiley: ah, so that is a possibility then? Isn't there always plenty of new ranges to scrape? Why would there ever not be much work to do? [16:26] *** scyther has joined #archiveteam [16:29] I'm not sure to be quite honest [16:29] I think it's a case of theres a lot of scrapers running, so any new jobs coming in immediately clear out again. [16:29] maybe it's even limited in order not to overwhelm the url shortening services..? [16:30] that too [16:30] *** will has joined #archiveteam [16:31] I think the limiting factor is the latency of getting a response back from the url shortener, each response is very tiny so you're not building up a lot of speed [16:32] a few years back i was able to get a ~8x speedup by keeping my http connections open [16:32] but that was on a shortener with no ratelimit :) [16:33] but there could very well be a 100ms delay or something so you don't get banned instantly or ddos them [16:45] *** primus104 has quit IRC (Leaving.) [16:46] *** Start_ has quit IRC (Disconnected.) [16:51] *** Start has joined #archiveteam [16:52] ehh.. I suppose it is almost scaling linearly.. [16:53] was seeing about 4.4 urls/s with just a few workers, and with lots of workers it's now more around 3.2/s [16:55] *** Start has quit IRC (Read error: Connection reset by peer) [16:55] *** Start has joined #archiveteam [16:55] *** Start has quit IRC (Client Quit) [16:57] *** Atluxity has joined #archiveteam [16:58] *** Start has joined #archiveteam [16:59] *** Start_ has joined #archiveteam [16:59] *** Start has quit IRC (Read error: Connection reset by peer) [17:13] *** MMovie has quit IRC (Ping timeout: 306 seconds) [17:15] *** MMovie has joined #archiveteam [17:30] *** londoncal has quit IRC (Remote host closed the connection) [17:37] *** useretail has quit IRC (Ping timeout: 623 seconds) [17:40] *** useretail has joined #archiveteam [17:40] *** Start_ has quit IRC (Disconnected.) [17:50] *** scyther has quit IRC (Leaving) [17:50] There's also a project channel dedicated to URLTeam at #urlteam [17:51] likelihood that your thoughts are lost in the noise is waaay lower there [17:52] *** brayden__ has quit IRC (Ping timeout: 606 seconds) [18:03] *** philpem has joined #archiveteam [18:08] *** dashcloud has quit IRC (Read error: Operation timed out) [18:12] *** dashcloud has joined #archiveteam [18:36] *** schbirid has quit IRC (Read error: Operation timed out) [18:36] *** schbirid has joined #archiveteam [18:39] *** primus104 has joined #archiveteam [18:44] *** Start has joined #archiveteam [19:12] *** Start has quit IRC (Disconnected.) [19:21] *** Start has joined #archiveteam [19:26] *** Start has quit IRC (Client Quit) [19:31] *** Start has joined #archiveteam [19:40] *** mistym has quit IRC (Remote host closed the connection) [19:46] *** SN4T14__ has joined #archiveteam [19:51] *** SN4T14_ has quit IRC (Ping timeout: 306 seconds) [19:55] *** mistym has joined #archiveteam [19:56] is there flickr archiving going on? if so, https://www.flickr.com/photos/quakecon/sets/ please :) [19:58] schbirid: added to archivebot [19:59] thanks [19:59] does it do one warc per job by any chance? i'd love to have this for my own [20:02] *** Start has quit IRC (Disconnected.) [20:02] *** Start has joined #archiveteam [20:20] it's one warc per about 5GB [20:23] *** Start has quit IRC (Disconnected.) [20:24] *** ex-parrot has quit IRC (Quit: leaving) [20:26] *** Start has joined #archiveteam [20:27] *** dashcloud has quit IRC (Quit: No Ping reply in 240 seconds.) [20:33] *** mistym has quit IRC (Remote host closed the connection) [20:36] *** dashcloud has joined #archiveteam [20:39] *** dashcloud has quit IRC (Read error: Operation timed out) [20:47] *** dashcloud has joined #archiveteam [20:48] *** mistym has joined #archiveteam [20:48] *** sivoais has quit IRC (Read error: Connection reset by peer) [20:48] *** an has joined #archiveteam [20:48] anyone know what's up with imcute.yt ? [20:51] so looks like the 2015 launch festival are gone [20:51] i can't find the full streams anymore [20:59] *** sivoais has joined #archiveteam [21:03] *** mistym has quit IRC (Remote host closed the connection) [21:04] *** mistym has joined #archiveteam [21:06] *** Start has quit IRC (Disconnected.) [21:06] *** Start has joined #archiveteam [21:06] *** Start has quit IRC (Client Quit) [21:31] *** londoncal has joined #archiveteam [21:51] *** schbirid has quit IRC (Leaving) [21:52] *** habi has joined #archiveteam [22:02] *** mistym has quit IRC (Remote host closed the connection) [22:11] *** mistym has joined #archiveteam [22:24] *** habi has left [22:48] *** BlueMaxim has joined #archiveteam [23:04] http://www.theverge.com/2015/3/14/8209461/why-are-we-trying-to-create-ready-player-ones-terrifying-nostalgia [23:04] ahahahaha [23:11] *** an has quit IRC (Quit: Page closed) [23:14] is there an acronym for "excess verbiage from someone pretending to be more intelligent than they are, didn't read to the end?" [23:14] Kanye has done it again appently. [23:15] that article is nearly unreadable, if you think there's something of value in it i might read tomorrow [23:27] the comments are even better [23:28] it's incoherent bullshit piled on incoherent bullshit [23:28] right on the Verge of meaning [23:32] the comments are even more incomprehensible to me than the article [23:33] filing that under "don't drink and worthless degree" [23:35] *** brayden has joined #archiveteam [23:48] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [23:49] !a http://www.soup2nuts.tv/ [23:49] oops, wrong channel