[00:21] *** Start has quit IRC (Read error: Connection reset by peer) [00:23] *** brayden has joined #archiveteam [00:36] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [01:12] *** philpem has quit IRC (Ping timeout: 252 seconds) [01:16] *** wvdp___ has quit IRC (Read error: Operation timed out) [01:21] *** JesseW has joined #archiveteam [01:23] *** primus104 has quit IRC (Leaving.) [01:24] *** nertzy has joined #archiveteam [01:39] *** Start has joined #archiveteam [01:55] *** schbirid has quit IRC (Read error: Operation timed out) [02:09] *** schbirid has joined #archiveteam [02:11] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [02:59] *** dcmorton_ has quit IRC (Read error: Operation timed out) [03:00] *** logan has quit IRC (Ping timeout: 362 seconds) [03:38] *** logan has joined #archiveteam [03:57] *** dashcloud has quit IRC (Read error: Connection reset by peer) [03:58] *** dashcloud has joined #archiveteam [04:10] *** aaaaaaaaa has quit IRC (Leaving) [04:37] <_desu_> Hey so im trying to install ArchiveBot and after spending days trying to compile CouchDB on debian I just gave up and tried it on Ubuntu, I get to the part where I need to import the databases and I keep getting this error from couchdb even though these databases are empty: {"error":"conflict","reason":"Document update conflict."} . Any insight? [04:45] you can try taking a look at the travis yaml file. maybe you missed a step [04:46] or the install file is missing something [04:47] install instructions file * [04:47] <_desu_> thanks it was the grep _rev stuff that was missing from the install file [04:56] <_desu_> New issue, trying to run firehose-client and im getting "Unable to load this gem. The libzmq library (or DLL) could not be found.” even after “bundle install” and “gem install libzmq" [05:16] *** i0npulse has quit IRC (Remote host closed the connection) [05:17] *** Silvan has joined #archiveteam [05:17] *** Sk1d has quit IRC (Remote host closed the connection) [05:17] *** xk_id_ has joined #archiveteam [05:21] *** xk_id has quit IRC (Ping timeout: 606 seconds) [05:21] *** SilSte has quit IRC (Ping timeout: 606 seconds) [05:28] *** khaoohs_ has joined #archiveteam [05:30] *** khaoohs has quit IRC (Read error: Operation timed out) [05:38] <_desu_> Also, install.backend, line 114: is db-name archivebot? [05:49] it's whatever you set it to be [05:50] _desu_: also if you just want to run a pipeline, you don't need to setup the whole backend [05:50] there is also https://github.com/ludios/grab-site which is based on archivebot code but has all the service bookkeeping stuff removed [06:04] *** Sk1d has joined #archiveteam [06:06] *** anomie has joined #archiveteam [06:06] *** robink has quit IRC (Remote host closed the connection) [06:06] *** robink has joined #archiveteam [06:23] *** JesseW has quit IRC (Read error: Operation timed out) [06:26] *** Sk1d has quit IRC (Read error: Operation timed out) [06:33] *** Sk1d has joined #archiveteam [06:35] *** i0npulse has joined #archiveteam [06:44] *** Sk1d has quit IRC (Remote host closed the connection) [06:45] *** RichardG has quit IRC (Remote host closed the connection) [07:12] *** PurpleSym has joined #archiveteam [07:18] *** Sk1d has joined #archiveteam [08:10] *** garyrh has quit IRC (Ping timeout: 600 seconds) [08:22] *** khaoohs__ has joined #archiveteam [08:24] *** khaoohs_ has quit IRC (Read error: Operation timed out) [08:33] *** philpem has joined #archiveteam [09:04] *** dashcloud has quit IRC (Read error: Connection reset by peer) [09:05] *** dashcloud has joined #archiveteam [09:08] *** chfoo has quit IRC (Read error: Operation timed out) [09:18] *** chfoo has joined #archiveteam [09:37] *** chfoo has quit IRC (Ping timeout: 258 seconds) [09:46] *** primus104 has joined #archiveteam [09:46] *** schbirid has quit IRC (Leaving) [09:51] *** chfoo has joined #archiveteam [10:08] *** primus104 has quit IRC (Leaving.) [10:20] *** wvdp___ has joined #archiveteam [11:07] *** bentpins has joined #archiveteam [11:27] *** zenguy_pc has quit IRC (Ping timeout: 306 seconds) [11:27] *** caber has quit IRC (Quit: Kids: talk with your parents about ad-blockers, and, at some point; social media. But fundamentals first!) [11:28] *** zenguy_pc has joined #archiveteam [11:47] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [11:53] *** zenguy_pc has joined #archiveteam [12:02] So we need to make some decisions on the blingee project [12:04] For each blingee 4 sizes exist: http://blingee.com/blingee/get_codes/44473783-The-2-Babies [12:04] *** schbirid has joined #archiveteam [12:05] And each of those 4 images has some code to embed the pictures: http://blingee.com/blingee/get_code/44473783?image=41032720 [12:05] The problem is that the large sized image is around 100kb. [12:06] with 130+ million blingees that leaves us with already 13 TB [12:07] And then we have a difference in links between the image shown on blingee and the image that can be embedded [12:08] Image shown in above url on the page: http://image.blingee.com/images15/content/output/000/000/000/2a6/41032720_2077204.gif?4 [12:08] Image shown in above url that can be embedded: http://image.blingee.com/images15/content/output/000/000/000/2a6/41032720_2077204.gif [12:10] And then we also have two links for all the other three sizes of the image [12:11] That, plus the html, etc., leaved us with a total size of around 130 TB. Which is not realistic due to the other big projects that's running now and the projects that are coming up [12:12] On top of that 130 TB we would also have around 20 more TB for groups, profiles, stamps, etc. [12:55] *** primus104 has joined #archiveteam [13:01] *** garyrh has joined #archiveteam [13:25] So we'll need a delay of at least 5 days of the shutdown of blip to be able to get everything [13:35] 130 tb of animated shitty gifs, that's a waste [13:36] nothing's a waste. it's just not possible currently due to the price of storage and the other big (upcoming) projects [13:39] *** wvdp___ has quit IRC (Read error: Connection reset by peer) [13:46] the cost versus potential benefit ratio here is insane though [13:46] on the animated gif thingy? [13:46] or on blip [13:47] animated gif thiingy [13:47] we saved this :) https://web.archive.org/web/20150806180219/http://blip.tv/cbr/batman-the-brave-and-the-bold-emperor-joker-clip-2-4269601 [13:48] thank god [13:49] blip videos are not playable in the wayback machine yet [13:49] don't expect them to become playable very soon either [13:50] the way the wayback machine is currently written, it's not possible to add special rules for rewriting some urls [13:50] but everything that would be needed to playback the videos in the wayback machine is saved, so some day they'll work [13:57] so the original video from https://web.archive.org/web/20150806180219/http://blip.tv/cbr/batman-the-brave-and-the-bold-emperor-joker-clip-2-4269601 can be downloaded by going to https://web.archive.org/web/20150806180216/http://blip.tv/rss/flash/4269601 [13:57] and then downloading the original file: https://web.archive.org/web/20150806180246/http://blip.tv/file/get/Cbr-BatmanTheBraveAndTheBoldEmperorJokerClip2438.mov [13:58] after which you'll be redirected to the actual original video [13:58] looks like wayback is having some problems atm [14:16] join #archiveteam-bs [14:16] whoops [14:29] chfoo: can you please add blingee to the projects.json? [14:29] *** xmc sets mode: +o swebb [14:29] *** swebb sets mode: +o brayden [14:40] *** SN4T14 has quit IRC (Read error: Connection reset by peer) [14:41] *** SN4T14 has joined #archiveteam [15:24] *** RichardG has joined #archiveteam [15:34] *** BlueMaxim has quit IRC (Quit: Leaving) [15:44] arkiver: ok, added [15:44] thanks! [15:44] chfoo: can you please also add a FOS rsync? [15:44] we'll start with the things that are the most important [15:44] we'll decide later then what more to save and what not [15:47] ok [15:56] Wheweeeee [15:57] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [15:57] SketchCow: we need you, important decisions need to be made! [15:57] *** zenguy_pc has joined #archiveteam [16:32] *** JesseW has joined #archiveteam [16:35] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [16:35] *** zenguy_pc has joined #archiveteam [16:46] arkiver: I haven't dug too deep into Blingee's setup, but I gathered that the .gifs were assembled out of a number of stamps, a base image, and some text. If so, would it be more feasible to save the components and rules to put them together? Or is that not available from Blingee? [16:51] *** yakfish has quit IRC (Read error: Operation timed out) [17:01] arkiver: is there an irc channel for blingee? [17:01] no idea [17:01] ok [17:02] nope, not ye [17:02] yet* [17:05] SketchCow: how is the manual saving project going? [17:07] *** yakfish has joined #archiveteam [17:13] *** schbirid has quit IRC (Leaving) [17:16] *** primus104 has quit IRC (Leaving.) [17:29] *** xk_id_ has quit IRC (Ping timeout: 252 seconds) [17:29] *** Protab is now known as Rotab [17:29] Remarkably inspiring that it got the fire. [17:29] *** xk_id has joined #archiveteam [17:29] Thousands of dollars sent in, possibly as many as 20 people coming [17:30] I drive down to Baltimore again tonight, get a room, and plan to be there for about 8am-8:30am. [17:30] And then people start showing up in scads [17:36] *** bentpins has quit IRC (Ping timeout: 483 seconds) [17:37] *** wvdp___ has joined #archiveteam [17:37] *** dan- has quit IRC (Read error: Connection reset by peer) [17:40] `sounds like a party [17:44] *** dan- has joined #archiveteam [17:55] *** aaaaaaaaa has joined #archiveteam [17:55] *** swebb sets mode: +o aaaaaaaaa [17:57] *** RichardG has quit IRC (Ping timeout: 362 seconds) [18:00] *** scyther has joined #archiveteam [18:00] *** Start has quit IRC (Quit: Disconnected.) [18:04] *** primus104 has joined #archiveteam [18:13] *** wvdp_ has joined #archiveteam [18:19] *** wvdp___ has quit IRC (Read error: Operation timed out) [18:48] SketchCow: we're going to start the grab of blingee in a bit. How much space does FOS currently have? [18:49] *** habi has joined #archiveteam [18:50] *** habi has left [19:08] What's the deal with blingee? [19:08] Nevermind. Google is my friend. [19:19] *** xk_id has quit IRC (Read error: Connection reset by peer) [19:29] glad to hear you got a lot more people to show up at the location [19:45] *** RichardG has joined #archiveteam [20:07] FOS has 3tb free. [20:07] I need to kick off some of this other garbage we've been downloading. [20:08] *** achip has quit IRC (Buhbye) [20:11] *** achip has joined #archiveteam [20:12] SketchCow: ok, we'll pause at 2.5 TB [20:13] Grab the largest images. [20:13] I'll see about blowing things into the archive. [20:13] Problem of the largest images is that there are two different URLs of exactly the same large image. [20:13] Use one. [20:14] so one url is shown on blingee.com, the other is used for the embeds in site, blogs, forums, etc. [20:14] so for now I think only grab the URL used on blingee.com [20:14] Right. [20:15] Always grab the largest, avoid grabbing two of the same large thing, unless it's tiny. [20:15] And document.... the FUCK out of. [20:16] Will do that! [20:17] I'll try to go through our projects and write some text for each of them on what is needed for them to playback good [20:18] as in what special rules the wayback machine should have for them to playback nicely (playing videos, etc.) [20:19] Yes. [20:19] I think that's really the only choice we need to have in the future. [20:19] Is for this, and for other stuff! To work with Kenji and other Waybackers to improve playback. [20:19] And make those notable. We're converting the whole wayback backend, so that's part of what's going on. [20:20] Ha ha, OK, so apparently the super-secret project to dupe that site is taking 3.5tb of remaining space. [20:20] So as soon as I start injecting that. [20:28] So wait, who DID do this thing, the area51 download. [20:28] arkiver: Was that you? [20:41] OK, that's cleared up. That will give us 3.5tb back on the FOS [20:43] *** PurpleSym has quit IRC (Remote host closed the connection) [20:54] Uh.... not immediately. [20:54] Obviously now it has to go through the hellscape [20:59] *** expr_ has joined #archiveteam [21:05] *** JesseW has quit IRC (Leaving.) [21:13] OK, off to NYC, then to Maryland, then, you know, lifting manuals all day. [21:14] *** brayden has quit IRC (Quit: Leaving) [21:16] Here's to saving fucking Blingee [21:22] *** habi has joined #archiveteam [21:23] *** habi has left [21:28] *** Start has joined #archiveteam [21:30] *** scyther has quit IRC (Read error: Connection reset by peer) [21:31] *** Start has quit IRC (Read error: Connection reset by peer) [21:34] *** Start has joined #archiveteam [22:13] *** nertzy has joined #archiveteam [22:31] *** wvdp_ has quit IRC (Read error: Operation timed out) [22:50] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [22:51] *** Start has quit IRC (Ping timeout: 362 seconds) [22:53] *** JesseW has joined #archiveteam [23:04] *** Muad-Dib has quit IRC (Ping timeout: 252 seconds) [23:15] *** Muad-Dib has joined #archiveteam [23:25] *** anomie has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in) [23:30] *** anomie has joined #archiveteam [23:39] *** Start has joined #archiveteam [23:41] *** mr-b has quit IRC (Quit: ZNC - http://znc.in) [23:49] *** Start has quit IRC (Ping timeout: 362 seconds)