#archiveteam 2015-08-16,Sun

↑back Search

Time Nickname Message
00:21 πŸ”— Start has quit IRC (Read error: Connection reset by peer)
00:23 πŸ”— brayden has joined #archiveteam
00:36 πŸ”— nertzy has quit IRC (Quit: This computer has gone to sleep)
01:12 πŸ”— philpem has quit IRC (Ping timeout: 252 seconds)
01:16 πŸ”— wvdp___ has quit IRC (Read error: Operation timed out)
01:21 πŸ”— JesseW has joined #archiveteam
01:23 πŸ”— primus104 has quit IRC (Leaving.)
01:24 πŸ”— nertzy has joined #archiveteam
01:39 πŸ”— Start has joined #archiveteam
01:55 πŸ”— schbirid has quit IRC (Read error: Operation timed out)
02:09 πŸ”— schbirid has joined #archiveteam
02:11 πŸ”— nertzy has quit IRC (Quit: This computer has gone to sleep)
02:59 πŸ”— dcmorton_ has quit IRC (Read error: Operation timed out)
03:00 πŸ”— logan has quit IRC (Ping timeout: 362 seconds)
03:38 πŸ”— logan has joined #archiveteam
03:57 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
03:58 πŸ”— dashcloud has joined #archiveteam
04:10 πŸ”— aaaaaaaaa has quit IRC (Leaving)
04:37 πŸ”— _desu_ Hey so im trying to install ArchiveBot and after spending days trying to compile CouchDB on debian I just gave up and tried it on Ubuntu, I get to the part where I need to import the databases and I keep getting this error from couchdb even though these databases are empty: {"error":"conflict","reason":"Document update conflict."} . Any insight?
04:45 πŸ”— chfoo you can try taking a look at the travis yaml file. maybe you missed a step
04:46 πŸ”— chfoo or the install file is missing something
04:47 πŸ”— chfoo install instructions file *
04:47 πŸ”— _desu_ thanks it was the grep _rev stuff that was missing from the install file
04:56 πŸ”— _desu_ New issue, trying to run firehose-client and im getting "Unable to load this gem. The libzmq library (or DLL) could not be found.” even after β€œbundle install” and β€œgem install libzmq"
05:16 πŸ”— i0npulse has quit IRC (Remote host closed the connection)
05:17 πŸ”— Silvan has joined #archiveteam
05:17 πŸ”— Sk1d has quit IRC (Remote host closed the connection)
05:17 πŸ”— xk_id_ has joined #archiveteam
05:21 πŸ”— xk_id has quit IRC (Ping timeout: 606 seconds)
05:21 πŸ”— SilSte has quit IRC (Ping timeout: 606 seconds)
05:28 πŸ”— khaoohs_ has joined #archiveteam
05:30 πŸ”— khaoohs has quit IRC (Read error: Operation timed out)
05:38 πŸ”— _desu_ Also, install.backend, line 114: is db-name archivebot?
05:49 πŸ”— yipdw it's whatever you set it to be
05:50 πŸ”— yipdw _desu_: also if you just want to run a pipeline, you don't need to setup the whole backend
05:50 πŸ”— yipdw there is also https://github.com/ludios/grab-site which is based on archivebot code but has all the service bookkeeping stuff removed
06:04 πŸ”— Sk1d has joined #archiveteam
06:06 πŸ”— anomie has joined #archiveteam
06:06 πŸ”— robink has quit IRC (Remote host closed the connection)
06:06 πŸ”— robink has joined #archiveteam
06:23 πŸ”— JesseW has quit IRC (Read error: Operation timed out)
06:26 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
06:33 πŸ”— Sk1d has joined #archiveteam
06:35 πŸ”— i0npulse has joined #archiveteam
06:44 πŸ”— Sk1d has quit IRC (Remote host closed the connection)
06:45 πŸ”— RichardG has quit IRC (Remote host closed the connection)
07:12 πŸ”— PurpleSym has joined #archiveteam
07:18 πŸ”— Sk1d has joined #archiveteam
08:10 πŸ”— garyrh has quit IRC (Ping timeout: 600 seconds)
08:22 πŸ”— khaoohs__ has joined #archiveteam
08:24 πŸ”— khaoohs_ has quit IRC (Read error: Operation timed out)
08:33 πŸ”— philpem has joined #archiveteam
09:04 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
09:05 πŸ”— dashcloud has joined #archiveteam
09:08 πŸ”— chfoo has quit IRC (Read error: Operation timed out)
09:18 πŸ”— chfoo has joined #archiveteam
09:37 πŸ”— chfoo has quit IRC (Ping timeout: 258 seconds)
09:46 πŸ”— primus104 has joined #archiveteam
09:46 πŸ”— schbirid has quit IRC (Leaving)
09:51 πŸ”— chfoo has joined #archiveteam
10:08 πŸ”— primus104 has quit IRC (Leaving.)
10:20 πŸ”— wvdp___ has joined #archiveteam
11:07 πŸ”— bentpins has joined #archiveteam
11:27 πŸ”— zenguy_pc has quit IRC (Ping timeout: 306 seconds)
11:27 πŸ”— caber has quit IRC (Quit: Kids: talk with your parents about ad-blockers, and, at some point; social media. But fundamentals first!)
11:28 πŸ”— zenguy_pc has joined #archiveteam
11:47 πŸ”— zenguy_pc has quit IRC (Read error: Connection reset by peer)
11:53 πŸ”— zenguy_pc has joined #archiveteam
12:02 πŸ”— arkiver So we need to make some decisions on the blingee project
12:04 πŸ”— arkiver For each blingee 4 sizes exist: http://blingee.com/blingee/get_codes/44473783-The-2-Babies
12:04 πŸ”— schbirid has joined #archiveteam
12:05 πŸ”— arkiver And each of those 4 images has some code to embed the pictures: http://blingee.com/blingee/get_code/44473783?image=41032720
12:05 πŸ”— arkiver The problem is that the large sized image is around 100kb.
12:06 πŸ”— arkiver with 130+ million blingees that leaves us with already 13 TB
12:07 πŸ”— arkiver And then we have a difference in links between the image shown on blingee and the image that can be embedded
12:08 πŸ”— arkiver Image shown in above url on the page: http://image.blingee.com/images15/content/output/000/000/000/2a6/41032720_2077204.gif?4
12:08 πŸ”— arkiver Image shown in above url that can be embedded: http://image.blingee.com/images15/content/output/000/000/000/2a6/41032720_2077204.gif
12:10 πŸ”— arkiver And then we also have two links for all the other three sizes of the image
12:11 πŸ”— arkiver That, plus the html, etc., leaved us with a total size of around 130 TB. Which is not realistic due to the other big projects that's running now and the projects that are coming up
12:12 πŸ”— arkiver On top of that 130 TB we would also have around 20 more TB for groups, profiles, stamps, etc.
12:55 πŸ”— primus104 has joined #archiveteam
13:01 πŸ”— garyrh has joined #archiveteam
13:25 πŸ”— arkiver So we'll need a delay of at least 5 days of the shutdown of blip to be able to get everything
13:35 πŸ”— schbirid 130 tb of animated shitty gifs, that's a waste
13:36 πŸ”— arkiver nothing's a waste. it's just not possible currently due to the price of storage and the other big (upcoming) projects
13:39 πŸ”— wvdp___ has quit IRC (Read error: Connection reset by peer)
13:46 πŸ”— schbirid the cost versus potential benefit ratio here is insane though
13:46 πŸ”— xmc on the animated gif thingy?
13:46 πŸ”— xmc or on blip
13:47 πŸ”— arkiver animated gif thiingy
13:47 πŸ”— arkiver we saved this :) https://web.archive.org/web/20150806180219/http://blip.tv/cbr/batman-the-brave-and-the-bold-emperor-joker-clip-2-4269601
13:48 πŸ”— xmc thank god
13:49 πŸ”— arkiver blip videos are not playable in the wayback machine yet
13:49 πŸ”— arkiver don't expect them to become playable very soon either
13:50 πŸ”— arkiver the way the wayback machine is currently written, it's not possible to add special rules for rewriting some urls
13:50 πŸ”— arkiver but everything that would be needed to playback the videos in the wayback machine is saved, so some day they'll work
13:57 πŸ”— arkiver so the original video from https://web.archive.org/web/20150806180219/http://blip.tv/cbr/batman-the-brave-and-the-bold-emperor-joker-clip-2-4269601 can be downloaded by going to https://web.archive.org/web/20150806180216/http://blip.tv/rss/flash/4269601
13:57 πŸ”— arkiver and then downloading the original file: https://web.archive.org/web/20150806180246/http://blip.tv/file/get/Cbr-BatmanTheBraveAndTheBoldEmperorJokerClip2438.mov
13:58 πŸ”— arkiver after which you'll be redirected to the actual original video
13:58 πŸ”— arkiver looks like wayback is having some problems atm
14:16 πŸ”— bentpins join #archiveteam-bs
14:16 πŸ”— bentpins whoops
14:29 πŸ”— arkiver chfoo: can you please add blingee to the projects.json?
14:29 πŸ”— xmc sets mode: +o swebb
14:29 πŸ”— swebb sets mode: +o brayden
14:40 πŸ”— SN4T14 has quit IRC (Read error: Connection reset by peer)
14:41 πŸ”— SN4T14 has joined #archiveteam
15:24 πŸ”— RichardG has joined #archiveteam
15:34 πŸ”— BlueMaxim has quit IRC (Quit: Leaving)
15:44 πŸ”— chfoo arkiver: ok, added
15:44 πŸ”— arkiver thanks!
15:44 πŸ”— arkiver chfoo: can you please also add a FOS rsync?
15:44 πŸ”— arkiver we'll start with the things that are the most important
15:44 πŸ”— arkiver we'll decide later then what more to save and what not
15:47 πŸ”— chfoo ok
15:56 πŸ”— SketchCow Wheweeeee
15:57 πŸ”— zenguy_pc has quit IRC (Read error: Connection reset by peer)
15:57 πŸ”— arkiver SketchCow: we need you, important decisions need to be made!
15:57 πŸ”— zenguy_pc has joined #archiveteam
16:32 πŸ”— JesseW has joined #archiveteam
16:35 πŸ”— zenguy_pc has quit IRC (Read error: Connection reset by peer)
16:35 πŸ”— zenguy_pc has joined #archiveteam
16:46 πŸ”— aschmitz arkiver: I haven't dug too deep into Blingee's setup, but I gathered that the .gifs were assembled out of a number of stamps, a base image, and some text. If so, would it be more feasible to save the components and rules to put them together? Or is that not available from Blingee?
16:51 πŸ”— yakfish has quit IRC (Read error: Operation timed out)
17:01 πŸ”— Start arkiver: is there an irc channel for blingee?
17:01 πŸ”— arkiver no idea
17:01 πŸ”— Start ok
17:02 πŸ”— arkiver nope, not ye
17:02 πŸ”— arkiver yet*
17:05 πŸ”— dashcloud SketchCow: how is the manual saving project going?
17:07 πŸ”— yakfish has joined #archiveteam
17:13 πŸ”— schbirid has quit IRC (Leaving)
17:16 πŸ”— primus104 has quit IRC (Leaving.)
17:29 πŸ”— xk_id_ has quit IRC (Ping timeout: 252 seconds)
17:29 πŸ”— Protab is now known as Rotab
17:29 πŸ”— SketchCow Remarkably inspiring that it got the fire.
17:29 πŸ”— xk_id has joined #archiveteam
17:29 πŸ”— SketchCow Thousands of dollars sent in, possibly as many as 20 people coming
17:30 πŸ”— SketchCow I drive down to Baltimore again tonight, get a room, and plan to be there for about 8am-8:30am.
17:30 πŸ”— SketchCow And then people start showing up in scads
17:36 πŸ”— bentpins has quit IRC (Ping timeout: 483 seconds)
17:37 πŸ”— wvdp___ has joined #archiveteam
17:37 πŸ”— dan- has quit IRC (Read error: Connection reset by peer)
17:40 πŸ”— db48x `sounds like a party
17:44 πŸ”— dan- has joined #archiveteam
17:55 πŸ”— aaaaaaaaa has joined #archiveteam
17:55 πŸ”— swebb sets mode: +o aaaaaaaaa
17:57 πŸ”— RichardG has quit IRC (Ping timeout: 362 seconds)
18:00 πŸ”— scyther has joined #archiveteam
18:00 πŸ”— Start has quit IRC (Quit: Disconnected.)
18:04 πŸ”— primus104 has joined #archiveteam
18:13 πŸ”— wvdp_ has joined #archiveteam
18:19 πŸ”— wvdp___ has quit IRC (Read error: Operation timed out)
18:48 πŸ”— arkiver SketchCow: we're going to start the grab of blingee in a bit. How much space does FOS currently have?
18:49 πŸ”— habi has joined #archiveteam
18:50 πŸ”— habi has left
19:08 πŸ”— anomie What's the deal with blingee?
19:08 πŸ”— anomie Nevermind. Google is my friend.
19:19 πŸ”— xk_id has quit IRC (Read error: Connection reset by peer)
19:29 πŸ”— dashcloud glad to hear you got a lot more people to show up at the location
19:45 πŸ”— RichardG has joined #archiveteam
20:07 πŸ”— SketchCow FOS has 3tb free.
20:07 πŸ”— SketchCow I need to kick off some of this other garbage we've been downloading.
20:08 πŸ”— achip has quit IRC (Buhbye)
20:11 πŸ”— achip has joined #archiveteam
20:12 πŸ”— arkiver SketchCow: ok, we'll pause at 2.5 TB
20:13 πŸ”— SketchCow Grab the largest images.
20:13 πŸ”— SketchCow I'll see about blowing things into the archive.
20:13 πŸ”— arkiver Problem of the largest images is that there are two different URLs of exactly the same large image.
20:13 πŸ”— SketchCow Use one.
20:14 πŸ”— arkiver so one url is shown on blingee.com, the other is used for the embeds in site, blogs, forums, etc.
20:14 πŸ”— arkiver so for now I think only grab the URL used on blingee.com
20:14 πŸ”— SketchCow Right.
20:15 πŸ”— SketchCow Always grab the largest, avoid grabbing two of the same large thing, unless it's tiny.
20:15 πŸ”— SketchCow And document.... the FUCK out of.
20:16 πŸ”— arkiver Will do that!
20:17 πŸ”— arkiver I'll try to go through our projects and write some text for each of them on what is needed for them to playback good
20:18 πŸ”— arkiver as in what special rules the wayback machine should have for them to playback nicely (playing videos, etc.)
20:19 πŸ”— SketchCow Yes.
20:19 πŸ”— SketchCow I think that's really the only choice we need to have in the future.
20:19 πŸ”— SketchCow Is for this, and for other stuff! To work with Kenji and other Waybackers to improve playback.
20:19 πŸ”— SketchCow And make those notable. We're converting the whole wayback backend, so that's part of what's going on.
20:20 πŸ”— SketchCow Ha ha, OK, so apparently the super-secret project to dupe that site is taking 3.5tb of remaining space.
20:20 πŸ”— SketchCow So as soon as I start injecting that.
20:28 πŸ”— SketchCow So wait, who DID do this thing, the area51 download.
20:28 πŸ”— SketchCow arkiver: Was that you?
20:41 πŸ”— SketchCow OK, that's cleared up. That will give us 3.5tb back on the FOS
20:43 πŸ”— PurpleSym has quit IRC (Remote host closed the connection)
20:54 πŸ”— SketchCow Uh.... not immediately.
20:54 πŸ”— SketchCow Obviously now it has to go through the hellscape
20:59 πŸ”— expr_ has joined #archiveteam
21:05 πŸ”— JesseW has quit IRC (Leaving.)
21:13 πŸ”— SketchCow OK, off to NYC, then to Maryland, then, you know, lifting manuals all day.
21:14 πŸ”— brayden has quit IRC (Quit: Leaving)
21:16 πŸ”— SketchCow Here's to saving fucking Blingee
21:22 πŸ”— habi has joined #archiveteam
21:23 πŸ”— habi has left
21:28 πŸ”— Start has joined #archiveteam
21:30 πŸ”— scyther has quit IRC (Read error: Connection reset by peer)
21:31 πŸ”— Start has quit IRC (Read error: Connection reset by peer)
21:34 πŸ”— Start has joined #archiveteam
22:13 πŸ”— nertzy has joined #archiveteam
22:31 πŸ”— wvdp_ has quit IRC (Read error: Operation timed out)
22:50 πŸ”— nertzy has quit IRC (Quit: This computer has gone to sleep)
22:51 πŸ”— Start has quit IRC (Ping timeout: 362 seconds)
22:53 πŸ”— JesseW has joined #archiveteam
23:04 πŸ”— Muad-Dib has quit IRC (Ping timeout: 252 seconds)
23:15 πŸ”— Muad-Dib has joined #archiveteam
23:25 πŸ”— anomie has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in)
23:30 πŸ”— anomie has joined #archiveteam
23:39 πŸ”— Start has joined #archiveteam
23:41 πŸ”— mr-b has quit IRC (Quit: ZNC - http://znc.in)
23:49 πŸ”— Start has quit IRC (Ping timeout: 362 seconds)

irclogger-viewer