[01:06] *** Start_ is now known as Start [01:27] *** JesseW has joined #archiveteam-bs [01:44] *** vitzli has joined #archiveteam-bs [02:23] JesseW, thanks, do you know what would be cheaper resource-wise for IA - to serve metadata from /metadata/ json requests or serve .xml files directly from the storage? [02:43] *** robink has quit IRC (Ping timeout: 190 seconds) [02:44] *** robink has joined #archiveteam-bs [02:44] i would guess that direct .xml servingwould be faster [02:47] isn't the metadata API supposedly super optimized? [02:51] vitzli: from what jake told me, use /metadata/ [02:51] got it [02:52] not direct download of the .xml. That's what iamine does, and he wrote it. [02:52] I mean, if for some reason you *want* the xml rather than json, it's probably fine to download the xml -- but there's no need to do so for performance reasons, AFAIK [02:54] regarding optimized -- I wouldn't be surprised if they didn't have various layers of caching in place for the JSON /metadata/ which wouldn't work for direct download of the XML. [02:54] argh, too many negatives -- let me rephrase that [02:55] JesseW: that's not too many non-negatives, nah [02:55] I understand, that's why I asked the question - I don't want to hit metadata servers hard [02:55] regarding optimized -- my semi-educated guess is that IA has various layers of caching in place for the JSON /metadata/, and fewer (or at least less specialized) caches for the direct download of the XML. [02:56] regarding system load -- IA wants you to do the reasonable thing. if they have a problem that's their fault, they'll work on it. [02:56] (ah, that wording makes much more sense (at least to me) [02:56] ) [02:56] :P [02:56] thanks. I can always count on archiveteam people to help close my parens. [02:59] thatt was the secondary mission of archiveteam, wasn't it? [02:59] closing the world's parens, after saving the world's heritage [03:00] is there some secret lisp conspiracy around? [03:00] (cdr 'secret-list-conspiracy) [03:13] *** JesseW has quit IRC (Leaving.) [03:20] please do not upload files large than 8 EiB to IA or you will break my database [04:07] *** JesseW has joined #archiveteam-bs [04:07] chfoo -- your logbot seems to have missed quite a bit lately [04:07] badcheese's as well [04:07] hm, maybe not [04:08] *** signius has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** chazchaz has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** Coderjoe has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** dcmorton has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** balrog has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** SN4T14 has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** tephra_ has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** Sanqui has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** arkiver has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** zenguy has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** logan has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** HCross has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** ohhdemgir has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** dxrt has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** slyphic has quit IRC (hub.efnet.us irc.servercentral.net) [04:08] *** Coderjoe_ has joined #archiveteam-bs [04:09] *** SN4T14_ has joined #archiveteam-bs [04:09] *** zenguy_pc has joined #archiveteam-bs [04:09] *** hawc145 has joined #archiveteam-bs [04:09] *** logan2 has joined #archiveteam-bs [04:10] *** dxrt- has joined #archiveteam-bs [04:11] *** Arkiver2 has joined #archiveteam-bs [04:11] *** Sanky has joined #archiveteam-bs [04:11] *** balrog_ has joined #archiveteam-bs [04:14] *** signius_ has joined #archiveteam-bs [04:15] *** chazchaz_ has joined #archiveteam-bs [04:18] *** tephra has joined #archiveteam-bs [04:24] *** zenguy_pc is now known as zenguy [04:24] *** balrog_ is now known as balrog [04:27] *** ohhdemgir has joined #archiveteam-bs [04:28] *** dcmorton has joined #archiveteam-bs [04:36] *** dxrt- is now known as dxrt [04:40] Ugh, twitter. Click on a tweet while scrolling through home page. Saw an interesting tweet below the one I clicked on, while the one I clicked on is loading. Click back button. Now the tweets are in a different order. #WTF #UX [04:44] *** chazchaz_ is now known as chazchaz [04:44] yeah, and the horrible habit of *SOMETIMES* loading things in the same window, and sometimes not [05:21] it's fine, you just have to purge yourself of such worldly expectations like sorting [05:32] *** Microguru has joined #archiveteam-bs [05:51] *** slyphic has joined #archiveteam-bs [05:57] *** phillipsj has joined #archiveteam-bs [05:57] * phillipsj finally makes it to the troll zone. [06:07] er, welcome? [06:07] what troll zone? [07:13] *** Microguru has quit IRC (Read error: Connection reset by peer) [07:20] *** wyatt8740 has quit IRC (Read error: Operation timed out) [07:23] *** wyatt8740 has joined #archiveteam-bs [07:30] *** wyatt8740 has quit IRC (Read error: Operation timed out) [07:34] *** wyatt8740 has joined #archiveteam-bs [07:35] *** phillipsj has quit IRC (Read error: Operation timed out) [07:36] *** phillipsj has joined #archiveteam-bs [08:20] *** JesseW has quit IRC (Ping timeout: 246 seconds) [08:26] this is *supposed* to be the noisier channel, no? [08:27] *** vitzli has quit IRC (Leaving) [09:04] *** schbirid has joined #archiveteam-bs [09:31] Well make some fucking noise then [09:31] Jeez [09:34] I should be sleeping... [09:38] *** kyan_ has joined #archiveteam-bs [09:43] *** kyan has quit IRC (Read error: Operation timed out) [09:47] *** kyan_ has quit IRC (Leaving) [10:03] ersi: stop talking to my fucking neighbors [10:08] heheh [10:14] *** BlueMaxim has quit IRC (Leaving) [11:14] *** balrog has quit IRC (Read error: Operation timed out) [11:14] *** MrRadar has quit IRC (Read error: Operation timed out) [11:14] *** balrog has joined #archiveteam-bs [11:14] *** SN4T14_ has quit IRC (Read error: Operation timed out) [11:14] *** phuzion has quit IRC (Read error: Operation timed out) [11:15] *** kvieta has quit IRC (Read error: Operation timed out) [11:15] *** acridAxid has quit IRC (Read error: Operation timed out) [11:15] *** acridAxid has joined #archiveteam-bs [11:20] *** logchfoo4 starts logging #archiveteam-bs at Wed Jan 27 11:20:40 2016 [11:20] *** logchfoo4 has joined #archiveteam-bs [11:24] *** SketchCo1 has joined #archiveteam-bs [11:25] *** logan has joined #archiveteam-bs [11:26] *** acridAxid has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** balrog has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** phillipsj has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** ohhdemgir has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** logan2 has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** hawc145 has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** zenguy has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** jspiros has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** mistym has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** rduser has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** swebb has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** chfoo has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** chfoo- has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** SketchCow has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** matthusb| has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** xmc has quit IRC (hub.efnet.us irc.colosolutions.net) [11:26] *** LordNigh2 has joined #archiveteam-bs [11:26] *** Lord_Nigh has quit IRC (Read error: Operation timed out) [11:26] *** LordNigh2 is now known as Lord_Nigh [11:28] *** chfoo has joined #archiveteam-bs [11:28] *** irc.colosolutions.net sets mode: +o chfoo [11:28] *** MrRadar has joined #archiveteam-bs [11:29] *** zenguy_pc has joined #archiveteam-bs [11:29] *** lbft_ has joined #archiveteam-bs [11:29] *** balrog_ has joined #archiveteam-bs [11:29] *** lbft has quit IRC (Read error: Operation timed out) [11:29] *** achip has quit IRC (Read error: Operation timed out) [11:29] *** chfoo has quit IRC (Read error: Operation timed out) [11:35] *** balrog_ has quit IRC (Read error: Operation timed out) [11:37] *** achip has joined #archiveteam-bs [11:39] *** logan has quit IRC (Read error: Operation timed out) [11:39] *** _acridAxd has joined #archiveteam-bs [11:39] *** SN4T14_ has quit IRC (Ping timeout: 360 seconds) [11:40] *** mistym- has joined #archiveteam-bs [11:40] *** dxrt has quit IRC (Ping timeout: 360 seconds) [11:41] *** logan has joined #archiveteam-bs [11:41] *** zenguy_pc is now known as zenguy [11:41] *** _acridAxd is now known as acridAxid [11:42] *** hawc145 has joined #archiveteam-bs [11:42] *** chfoo has joined #archiveteam-bs [11:43] *** phuzion has quit IRC (Ping timeout: 360 seconds) [11:43] *** chfoo- has joined #archiveteam-bs [11:43] *** phillipsj has joined #archiveteam-bs [11:45] *** ohhdemgir has joined #archiveteam-bs [11:45] *** dxrt has joined #archiveteam-bs [11:46] *** phuzion has joined #archiveteam-bs [11:46] *** atlogbot has joined #archiveteam-bs [11:46] *** swebb has joined #archiveteam-bs [11:49] *** SN4T14 has joined #archiveteam-bs [11:53] *** balrog has joined #archiveteam-bs [11:54] *** Sanqui has joined #archiveteam-bs [12:18] *** rduser has joined #archiveteam-bs [12:24] *** beardicus has joined #archiveteam-bs [12:28] *** matthusby has joined #archiveteam-bs [12:28] *** toad1 has quit IRC (Read error: Operation timed out) [12:29] *** kvieta has quit IRC (Read error: Operation timed out) [12:30] *** vitzli has joined #archiveteam-bs [12:30] *** beardicus has quit IRC (Read error: Operation timed out) [12:31] *** lbft_ has quit IRC (Read error: Operation timed out) [12:32] *** jspiros has joined #archiveteam-bs [12:34] *** lbft has joined #archiveteam-bs [12:34] *** xmc has joined #archiveteam-bs [12:35] *** phuzion has quit IRC (Read error: Operation timed out) [12:37] *** rolfb has joined #archiveteam-bs [12:39] *** bzc6p has joined #archiveteam-bs [12:41] *** phuzion has joined #archiveteam-bs [12:56] So this is the last message from my Fujitsu-Siemens Amilo A1650G laptop that has been my true companion in my archiving efforts in the last two years. Farewell old friend. [12:56] Goodbye old laptop [12:57] *** beardicus has joined #archiveteam-bs [12:57] *** megaminxw has joined #archiveteam-bs [12:57] *** megaminxw has left [12:58] *** kvieta has joined #archiveteam-bs [13:03] *** toad1 has joined #archiveteam-bs [13:04] *** bzc6p has quit IRC (Quit: bzc6p) [13:05] *** rolfb has quit IRC (Linkinus - http://linkinus.com) [13:12] *** hawc145 is now known as HCross [13:35] *** signius has joined #archiveteam-bs [13:36] *** signius_ has quit IRC (Ping timeout: 260 seconds) [14:19] how would i go about archiving a vbullitin site? [14:26] check wiki [14:52] *** jspiros has quit IRC (Read error: Operation timed out) [14:54] *** jspiros has joined #archiveteam-bs [16:17] *** dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.) [16:19] *** dashcloud has joined #archiveteam-bs [16:28] *** Fletcher has quit IRC (Ping timeout: 252 seconds) [16:50] *** Fletcher has joined #archiveteam-bs [16:55] [17:54] DID YOU KNOW: the Python SSL library does not check hostnames by default, because the maintainer claims that it's basically "too hard to do" and then tells application developers to implement it themselves [16:55] [17:54] ref https://bugs.python.org/issue1589 [16:56] so i'm close to 629k now [16:56] i'm at 628,992 currently [16:57] now at 629,002 [17:42] *** JesseW has joined #archiveteam-bs [17:50] *** JesseW has quit IRC (Leaving.) [17:54] Newsgrabber grabbed the attention of the wayback manager again. It was hil-arious [17:55] *** SketchCo1 is now known as SketchCow [17:55] oops [17:56] Do we need to do anything to keep him quiet? [18:00] No, no. Keep doing what you're doing. [18:01] Will do. :) [18:01] We are on track to cross the 10TB of news mark soon [18:01] that was fast! [18:02] Arkiver2: [18:02] 376G GOOGLECODE [18:02] 243G TELENOR [18:02] xmc: You just wait! [18:02] * xmc sits tight [18:02] More's comin [18:02] news is coming. [18:03] Stay classy, newsbuddy [18:04] The chat with the web archiving guy was about what we're doing, and how it's going, vs. GDELT [18:05] He let me know, by the way, that Archive Team is a significant portion of Internet Archive's web archiving now. [18:05] That is very nice to hear [18:05] Like, 20% [18:09] damn [18:09] okay then! [18:16] *** atlogbot has quit IRC (Read error: Operation timed out) [18:16] Fucking A [18:16] Fucking T! [18:17] *** slyphic has quit IRC (Read error: Operation timed out) [18:17] *** atlogbot has joined #archiveteam-bs [18:19] *** SN4T14 has quit IRC (Ping timeout: 360 seconds) [18:19] wait, was telenor uploaded to the IA? [18:19] *** swebb has quit IRC (Ping timeout: 360 seconds) [18:20] *** signius has quit IRC (Read error: Operation timed out) [18:20] *** slyphic has joined #archiveteam-bs [18:20] *** phuzion has quit IRC (Ping timeout: 360 seconds) [18:20] *** dxrt has quit IRC (Ping timeout: 360 seconds) [18:21] *** swebb has joined #archiveteam-bs [18:22] Yup, the whole operator [18:22] *** Sanqui has quit IRC (Read error: Operation timed out) [18:23] *** Sanqui has joined #archiveteam-bs [18:23] no, I mean home.online.no, telenor-grab stuff [18:23] *** atlogbot has quit IRC (Ping timeout: 360 seconds) [18:24] FYI: https://github.com/pugjs/jade/issues/2184#issuecomment-175763098 [18:26] *** schbirid has quit IRC (Read error: Operation timed out) [18:26] *** dxrt has joined #archiveteam-bs [18:27] *** phuzion has joined #archiveteam-bs [18:29] Shoving in telenor. [18:30] It was uploaded to FOS. It was not into the archive until right this moment. That 260gb is now going in. [18:30] awesome, glad it got home [18:31] *** vtyl has quit IRC (Ping timeout: 252 seconds) [18:32] *** phuzion has quit IRC (Read error: Operation timed out) [18:33] *** phuzion has joined #archiveteam-bs [18:34] *** slyphic has quit IRC (Read error: Operation timed out) [18:35] *** SN4T14 has joined #archiveteam-bs [18:36] *** slyphic has joined #archiveteam-bs [18:36] *** atlogbot has joined #archiveteam-bs [18:38] *** schbirid has joined #archiveteam-bs [18:40] *** balrog_ has joined #archiveteam-bs [18:42] *** atlogbot has quit IRC (Ping timeout: 360 seconds) [18:44] *** balrog has quit IRC (Read error: Operation timed out) [18:44] *** balrog_ is now known as balrog [18:48] *** atlogbot has joined #archiveteam-bs [18:51] *** lytv has joined #archiveteam-bs [18:55] *** phillipsj has quit IRC (Ping timeout: 506 seconds) [18:56] *** atlogbot has quit IRC (Ping timeout: 360 seconds) [18:56] *** vitzli has quit IRC (Leaving) [18:58] *** slyphic has quit IRC (Remote host closed the connection) [19:15] *** arkiver3 has joined #archiveteam-bs [19:17] *** schbirid has quit IRC (Read error: Operation timed out) [19:18] *** schbirid has joined #archiveteam-bs [19:20] https://pbs.twimg.com/media/CQ_0PzvWUAA2x7W.jpg:large [19:22] *** arkiver3 has quit IRC (Ping timeout: 252 seconds) [19:28] *** atlogbot has joined #archiveteam-bs [19:29] *** slyphic has joined #archiveteam-bs [19:31] *** schbirid has quit IRC (Quit: Leaving) [19:36] *** atlogbot has quit IRC (Read error: Operation timed out) [19:37] *** atlogbot has joined #archiveteam-bs [19:43] *** atlogbot has quit IRC (Ping timeout: 360 seconds) [19:46] *** slyphic has quit IRC (Read error: Operation timed out) [19:52] *** atlogbot has joined #archiveteam-bs [20:02] *** slyphic has joined #archiveteam-bs [20:44] *** RichardG has quit IRC (Quit: Keyboard not found, press F1 to continue) [20:49] *** RichardG has joined #archiveteam-bs [21:08] *** RichardG has quit IRC (Quit: Keyboard not found, press F1 to continue) [21:14] *** RichardG has joined #archiveteam-bs [21:22] *** kyan has joined #archiveteam-bs [21:34] *** Infreq has quit IRC (Read error: Operation timed out) [21:35] *** Infreq has joined #archiveteam-bs [21:39] *** Aad has joined #archiveteam-bs [22:06] *** VADemon has joined #archiveteam-bs [22:11] *** slyphic is now known as slyphic|a [22:34] *** yipdw has quit IRC (Read error: Operation timed out) [22:34] *** kyan_ has joined #archiveteam-bs [22:34] *** JetBalsa has joined #archiveteam-bs [22:38] *** DopefishJ has joined #archiveteam-bs [22:39] *** kyan__ has joined #archiveteam-bs [22:39] *** goekesmi has joined #archiveteam-bs [22:40] *** no2pencil has quit IRC (Read error: Operation timed out) [22:41] *** kyan has quit IRC (Read error: Operation timed out) [22:41] *** goekesmi_ has quit IRC (Ping timeout: 499 seconds) [22:41] *** yipdw has joined #archiveteam-bs [22:42] *** kyan_ has quit IRC (Read error: Operation timed out) [22:43] *** DFJustin has quit IRC (Ping timeout: 499 seconds) [22:45] *** winr4r has joined #archiveteam-bs [22:45] *** lysobit has quit IRC (Read error: Operation timed out) [22:46] *** no2pencil has joined #archiveteam-bs [22:48] *** winr5r has quit IRC (Ping timeout: 499 seconds) [22:50] *** lysobit has joined #archiveteam-bs [22:51] *** alard has quit IRC (Ping timeout: 730 seconds) [22:53] *** alard has joined #archiveteam-bs [22:53] *** tephra_ has joined #archiveteam-bs [22:54] *** is-_ has joined #archiveteam-bs [22:56] *** Zebranky has quit IRC (Ping timeout: 499 seconds) [22:56] *** mistym- has quit IRC (Ping timeout: 499 seconds) [22:57] *** is- has quit IRC (Ping timeout: 499 seconds) [22:57] *** mistym has joined #archiveteam-bs [22:59] *** kyan__ is now known as kyan [23:00] *** Zebranky has joined #archiveteam-bs [23:00] *** tephra has quit IRC (Ping timeout: 730 seconds) [23:14] *** DopefishJ is now known as DFJustin [23:22] *** slyphic|a is now known as slyphic