[00:39] *** mismatch has joined #archiveteam [00:55] *** ndiddy has quit IRC (Ping timeout: 250 seconds) [01:11] *** JesseW has joined #archiveteam [01:17] *** acridAxid has quit IRC (marauder) [01:18] *** acridAxid has joined #archiveteam [01:52] *** schbirid has quit IRC (Ping timeout: 258 seconds) [01:57] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [02:05] *** schbirid has joined #archiveteam [03:23] *** tomwsmf-a has joined #archiveteam [03:36] *** dashcloud has quit IRC (Read error: Operation timed out) [03:39] *** dashcloud has joined #archiveteam [03:42] *** vOYtEC has quit IRC (Read error: Connection reset by peer) [03:43] *** vOYtEC has joined #archiveteam [04:08] There is an argument for us grabbing the articles linked from Longform and other "long-journalism" link blogs [04:10] Also, Arlo on FOS (and therefore FOS) is pushed out, that means the machine is back down to 15% or so capacity, good times. [04:10] (It never gets lower than 11%, for various reasons of in-process projects or archivebot or so on) [04:14] *** dashcloud has quit IRC (Read error: Operation timed out) [04:15] *** balrog has quit IRC (Read error: Operation timed out) [04:15] *** balrog has joined #archiveteam [04:15] *** swebb sets mode: +o balrog [04:15] *** dashcloud has joined #archiveteam [04:31] *** Sk1d has quit IRC (Ping timeout: 194 seconds) [04:33] *** vitzli has joined #archiveteam [04:37] *** Sk1d has joined #archiveteam [05:01] *** kris33 has joined #archiveteam [05:21] *** dashcloud has quit IRC (Read error: Operation timed out) [05:25] *** dashcloud has joined #archiveteam [05:41] *** mismatch has quit IRC (Ping timeout: 633 seconds) [06:00] *** philpem has quit IRC (Ping timeout: 260 seconds) [06:09] *** JesseW has quit IRC (Read error: Operation timed out) [06:13] *** Honno_ has joined #archiveteam [06:24] *** vitzli has quit IRC (Quit: Leaving) [06:31] *** Honno_ has quit IRC (Read error: Operation timed out) [06:47] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [07:00] *** WinterFox has joined #archiveteam [07:06] *** metalcamp has joined #archiveteam [07:08] *** DFJustin has quit IRC (Ping timeout: 260 seconds) [07:25] *** vOYtEC has quit IRC (ny.us.hub west.us.hub) [07:25] *** fie has quit IRC (ny.us.hub west.us.hub) [07:25] *** redlob has quit IRC (ny.us.hub west.us.hub) [07:25] *** K4k has quit IRC (ny.us.hub west.us.hub) [07:25] *** edsu has quit IRC (ny.us.hub west.us.hub) [07:25] *** wp494 has quit IRC (ny.us.hub west.us.hub) [07:42] *** vOYtEC has joined #archiveteam [07:42] *** fie has joined #archiveteam [07:42] *** redlob has joined #archiveteam [07:42] *** K4k has joined #archiveteam [07:42] *** edsu has joined #archiveteam [07:42] *** wp494 has joined #archiveteam [07:42] *** west.us.hub sets mode: +oo edsu wp494 [07:42] *** swebb sets mode: +o edsu [08:07] *** vOYtEC has quit IRC (ny.us.hub west.us.hub) [08:07] *** fie has quit IRC (ny.us.hub west.us.hub) [08:07] *** redlob has quit IRC (ny.us.hub west.us.hub) [08:07] *** K4k has quit IRC (ny.us.hub west.us.hub) [08:08] *** edsu has quit IRC (ny.us.hub west.us.hub) [08:08] *** wp494 has quit IRC (ny.us.hub west.us.hub) [08:08] *** vOYtEC has joined #archiveteam [08:08] *** fie has joined #archiveteam [08:08] *** redlob has joined #archiveteam [08:08] *** K4k has joined #archiveteam [08:08] *** edsu has joined #archiveteam [08:08] *** wp494 has joined #archiveteam [08:08] *** west.us.hub sets mode: +oo edsu wp494 [08:08] *** swebb sets mode: +o edsu [08:15] *** MMovie1 has quit IRC (Read error: Connection reset by peer) [08:15] *** vitzli has joined #archiveteam [08:19] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [08:21] *** ariscop has quit IRC (Read error: Operation timed out) [08:26] *** BartoCH has joined #archiveteam [08:28] *** atomotic has joined #archiveteam [08:57] *** Coderjoe has quit IRC (Read error: Operation timed out) [09:12] *** Coderjoe has joined #archiveteam [09:21] *** vitzli has quit IRC (Quit: Leaving) [09:38] *** voltagex has joined #archiveteam [09:39] hey all, anyone awake? [09:39] indeed [09:39] just finished my lunch actually [09:39] what can I do for you voltagex [09:40] I've got $100+ of AWS credit expiring soon and I want to help. [09:40] I'm running 2x t2 mediums with TTT right now, but the setup has been tricky and manual so far [09:40] cool [09:41] I dont know how much $100+ aws credit is, in terms of what you get [09:41] but anyway [09:41] you got two options, depending on how involved you wish to be [09:41] you can either run one or more warriors [09:41] quite a lot when I can get hours and hours of server time for cents - if you bid on instances that can be turned off at any time [09:42] the pre-packaged archiving vm, container, whatever [09:42] or you can run the pipeline-script for projects directly [09:42] I've already raised a bug on https://github.com/ArchiveTeam/arto-grab/issues/2 [09:43] so what's the priority [09:43] arto seemed to be returning more 403s than anything else and it's definitely grabbing a lot of duplicate files [09:43] nice bug, with patch and everything. One of the better patches to that issue [09:44] then you perhaps know about http://tracker.archiveteam.org/ [09:44] there is listed all the active grabs we have going on now [09:44] urlteam is always going to need power [09:45] not all grabs are running, as you can probably tell [09:45] there are different reasons for this [09:47] ooh ooh wikiteam [09:47] I'd still like to get an automated build for URLTeam on EC2 but I won't be able to pay for AMI hosting long term [09:49] what I can't work out how much processing power / bandwidth the URLTeam grabber needs [09:58] *** ariscop has joined #archiveteam [10:06] it does not need much [10:30] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [10:33] *** DFJustin has joined #archiveteam [10:33] *** swebb sets mode: +o DFJustin [11:03] I got an Ansible playbook to sort of launch the Terror of Tiny Town grabber [11:03] it's still too fiddly [11:03] I'm now running something silly like 8 EC2 instances [11:05] *** BlueMaxim has quit IRC (Quit: Leaving) [11:08] 16 soon :D [11:11] better to run them on boxes that are idle anyway. [11:26] spot instances are idle EC2 capacity ;) [12:04] voltagex: hope you can be able to publish how you got it to play with ansible. I do not know if we got that noted down anywhere [12:09] *** atomotic has joined #archiveteam [12:17] yes, but someone will totally shoot themselves in the face/credit card with this (probably me) https://gist.github.com/voltagex/6511a43b75d0eca3224c07cf6d3ec53a [12:19] and you'll all bid up my spot instance prices [12:21] :P [12:22] a race for the top [12:22] * voltagex begins leaderboard climb [12:22] :) [12:22] welcome to the club [12:23] thanks [12:23] I may delete that gist, I'm really worried someone will run up a huge EC2 bill using it and blame me [12:23] voltagex, dont say delete to archiveteam [12:24] DONT [12:24] HAHAHA [12:24] good point [12:24] already archivebotted it [12:24] YOU CANT REMOVE IT NOW [12:25] *** kris33_ has joined #archiveteam [12:26] :P [12:26] *** kris33_ has quit IRC (Textual IRC Client: www.textualapp.com) [12:33] *** kris33 has quit IRC (Ping timeout: 506 seconds) [12:36] passed YOURNICKHERE [13:00] *** WinterFox has quit IRC (Remote host closed the connection) [14:00] *** Start has quit IRC (Quit: Disconnected.) [14:01] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [14:06] *** tomwsmf-a has joined #archiveteam [14:09] *** khaoohs_ has joined #archiveteam [14:19] *** MMovie has joined #archiveteam [14:27] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [14:52] *** atomotic has joined #archiveteam [14:56] *** Fletcher has quit IRC (Read error: Operation timed out) [15:03] *** Start has joined #archiveteam [15:11] *** Fletcher has joined #archiveteam [15:14] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:28] *** tomwsmf-a has joined #archiveteam [15:35] *** JesseW has joined #archiveteam [15:44] *** will has joined #archiveteam [15:47] *** JesseW has quit IRC (Read error: Operation timed out) [15:54] *** Smiley has joined #archiveteam [15:56] *** SmileyG has quit IRC (Read error: Operation timed out) [16:07] *** Start has quit IRC (Quit: Disconnected.) [16:13] *** brayden_ has joined #archiveteam [16:13] *** swebb sets mode: +o brayden_ [16:18] *** vitzli has joined #archiveteam [16:18] voltagex: you are working on URLteam? Please do join the #urlteam channel, too. [16:19] Also, if you notice the tracker not giving you enough jobs — we can always use help investigating new shorteners (see the URLTeam wiki page) [16:23] *** brayden has quit IRC (Ping timeout: 633 seconds) [16:31] *** Start has joined #archiveteam [16:58] *** philpem has joined #archiveteam [17:13] *** Start has quit IRC (Read error: Connection reset by peer) [17:13] *** Start has joined #archiveteam [17:29] *** VADemon has joined #archiveteam [17:30] *** Start has quit IRC (Quit: Disconnected.) [17:35] *** Rotab has quit IRC (Ping timeout: 194 seconds) [18:03] *** atomotic has joined #archiveteam [18:10] *** tomwsmf-a has quit IRC (Ping timeout: 258 seconds) [18:19] *** VADemon has quit IRC (Quit: left4dead) [18:24] *** BartoCH_ has joined #archiveteam [18:25] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [18:26] *** Start has joined #archiveteam [18:33] *** JW_work has quit IRC (Quit: Leaving.) [18:35] *** JW_work has joined #archiveteam [18:37] *** JW_work has quit IRC (Client Quit) [18:38] *** JW_work has joined #archiveteam [18:39] *** JW_work has quit IRC (Client Quit) [18:42] *** JW_work has joined #archiveteam [18:47] *** JW_work has quit IRC (Read error: Operation timed out) [18:49] *** vitzli has quit IRC (Quit: Leaving) [18:59] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [19:04] *** Start has quit IRC (Quit: Disconnected.) [19:14] *** atomotic has joined #archiveteam [19:15] *** atomotic has quit IRC (Client Quit) [19:18] *** Start has joined #archiveteam [19:18] *** ndiddy has joined #archiveteam [19:45] *** Start has quit IRC (Quit: Disconnected.) [19:47] *** JW_work has joined #archiveteam [20:04] *** tomwsmf-a has joined #archiveteam [20:32] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [20:44] *** ariscop has quit IRC (Ping timeout: 506 seconds) [20:54] *** tomwsmf-a has joined #archiveteam [20:59] *** schbirid has quit IRC (Quit: Leaving) [21:11] Seems the fotolog dump finished uploading to IA. Should start Gamefront, but that will have to wait for a few days. [21:28] *** BartoCH_ is now known as BartoCH [21:45] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [21:45] *** tomwsmf-a has joined #archiveteam [22:00] *** ariscop has joined #archiveteam [22:03] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [22:15] zino: awesome! Thank you so much :D [22:56] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [23:12] *** Start has joined #archiveteam [23:57] *** ndiddy has quit IRC (Read error: Operation timed out)