| Time |
Nickname |
Message |
|
00:21
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
|
00:23
🔗
|
|
brayden has joined #archiveteam |
|
00:36
🔗
|
|
nertzy has quit IRC (Quit: This computer has gone to sleep) |
|
01:12
🔗
|
|
philpem has quit IRC (Ping timeout: 252 seconds) |
|
01:16
🔗
|
|
wvdp___ has quit IRC (Read error: Operation timed out) |
|
01:21
🔗
|
|
JesseW has joined #archiveteam |
|
01:23
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
01:24
🔗
|
|
nertzy has joined #archiveteam |
|
01:39
🔗
|
|
Start has joined #archiveteam |
|
01:55
🔗
|
|
schbirid has quit IRC (Read error: Operation timed out) |
|
02:09
🔗
|
|
schbirid has joined #archiveteam |
|
02:11
🔗
|
|
nertzy has quit IRC (Quit: This computer has gone to sleep) |
|
02:59
🔗
|
|
dcmorton_ has quit IRC (Read error: Operation timed out) |
|
03:00
🔗
|
|
logan has quit IRC (Ping timeout: 362 seconds) |
|
03:38
🔗
|
|
logan has joined #archiveteam |
|
03:57
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
03:58
🔗
|
|
dashcloud has joined #archiveteam |
|
04:10
🔗
|
|
aaaaaaaaa has quit IRC (Leaving) |
|
04:37
🔗
|
_desu_ |
Hey so im trying to install ArchiveBot and after spending days trying to compile CouchDB on debian I just gave up and tried it on Ubuntu, I get to the part where I need to import the databases and I keep getting this error from couchdb even though these databases are empty: {"error":"conflict","reason":"Document update conflict."} . Any insight? |
|
04:45
🔗
|
chfoo |
you can try taking a look at the travis yaml file. maybe you missed a step |
|
04:46
🔗
|
chfoo |
or the install file is missing something |
|
04:47
🔗
|
chfoo |
install instructions file * |
|
04:47
🔗
|
_desu_ |
thanks it was the grep _rev stuff that was missing from the install file |
|
04:56
🔗
|
_desu_ |
New issue, trying to run firehose-client and im getting "Unable to load this gem. The libzmq library (or DLL) could not be found.” even after “bundle install” and “gem install libzmq" |
|
05:16
🔗
|
|
i0npulse has quit IRC (Remote host closed the connection) |
|
05:17
🔗
|
|
Silvan has joined #archiveteam |
|
05:17
🔗
|
|
Sk1d has quit IRC (Remote host closed the connection) |
|
05:17
🔗
|
|
xk_id_ has joined #archiveteam |
|
05:21
🔗
|
|
xk_id has quit IRC (Ping timeout: 606 seconds) |
|
05:21
🔗
|
|
SilSte has quit IRC (Ping timeout: 606 seconds) |
|
05:28
🔗
|
|
khaoohs_ has joined #archiveteam |
|
05:30
🔗
|
|
khaoohs has quit IRC (Read error: Operation timed out) |
|
05:38
🔗
|
_desu_ |
Also, install.backend, line 114: is db-name archivebot? |
|
05:49
🔗
|
yipdw |
it's whatever you set it to be |
|
05:50
🔗
|
yipdw |
_desu_: also if you just want to run a pipeline, you don't need to setup the whole backend |
|
05:50
🔗
|
yipdw |
there is also https://github.com/ludios/grab-site which is based on archivebot code but has all the service bookkeeping stuff removed |
|
06:04
🔗
|
|
Sk1d has joined #archiveteam |
|
06:06
🔗
|
|
anomie has joined #archiveteam |
|
06:06
🔗
|
|
robink has quit IRC (Remote host closed the connection) |
|
06:06
🔗
|
|
robink has joined #archiveteam |
|
06:23
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
06:26
🔗
|
|
Sk1d has quit IRC (Read error: Operation timed out) |
|
06:33
🔗
|
|
Sk1d has joined #archiveteam |
|
06:35
🔗
|
|
i0npulse has joined #archiveteam |
|
06:44
🔗
|
|
Sk1d has quit IRC (Remote host closed the connection) |
|
06:45
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
07:12
🔗
|
|
PurpleSym has joined #archiveteam |
|
07:18
🔗
|
|
Sk1d has joined #archiveteam |
|
08:10
🔗
|
|
garyrh has quit IRC (Ping timeout: 600 seconds) |
|
08:22
🔗
|
|
khaoohs__ has joined #archiveteam |
|
08:24
🔗
|
|
khaoohs_ has quit IRC (Read error: Operation timed out) |
|
08:33
🔗
|
|
philpem has joined #archiveteam |
|
09:04
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
09:05
🔗
|
|
dashcloud has joined #archiveteam |
|
09:08
🔗
|
|
chfoo has quit IRC (Read error: Operation timed out) |
|
09:18
🔗
|
|
chfoo has joined #archiveteam |
|
09:37
🔗
|
|
chfoo has quit IRC (Ping timeout: 258 seconds) |
|
09:46
🔗
|
|
primus104 has joined #archiveteam |
|
09:46
🔗
|
|
schbirid has quit IRC (Leaving) |
|
09:51
🔗
|
|
chfoo has joined #archiveteam |
|
10:08
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
10:20
🔗
|
|
wvdp___ has joined #archiveteam |
|
11:07
🔗
|
|
bentpins has joined #archiveteam |
|
11:27
🔗
|
|
zenguy_pc has quit IRC (Ping timeout: 306 seconds) |
|
11:27
🔗
|
|
caber has quit IRC (Quit: Kids: talk with your parents about ad-blockers, and, at some point; social media. But fundamentals first!) |
|
11:28
🔗
|
|
zenguy_pc has joined #archiveteam |
|
11:47
🔗
|
|
zenguy_pc has quit IRC (Read error: Connection reset by peer) |
|
11:53
🔗
|
|
zenguy_pc has joined #archiveteam |
|
12:02
🔗
|
arkiver |
So we need to make some decisions on the blingee project |
|
12:04
🔗
|
arkiver |
For each blingee 4 sizes exist: http://blingee.com/blingee/get_codes/44473783-The-2-Babies |
|
12:04
🔗
|
|
schbirid has joined #archiveteam |
|
12:05
🔗
|
arkiver |
And each of those 4 images has some code to embed the pictures: http://blingee.com/blingee/get_code/44473783?image=41032720 |
|
12:05
🔗
|
arkiver |
The problem is that the large sized image is around 100kb. |
|
12:06
🔗
|
arkiver |
with 130+ million blingees that leaves us with already 13 TB |
|
12:07
🔗
|
arkiver |
And then we have a difference in links between the image shown on blingee and the image that can be embedded |
|
12:08
🔗
|
arkiver |
Image shown in above url on the page: http://image.blingee.com/images15/content/output/000/000/000/2a6/41032720_2077204.gif?4 |
|
12:08
🔗
|
arkiver |
Image shown in above url that can be embedded: http://image.blingee.com/images15/content/output/000/000/000/2a6/41032720_2077204.gif |
|
12:10
🔗
|
arkiver |
And then we also have two links for all the other three sizes of the image |
|
12:11
🔗
|
arkiver |
That, plus the html, etc., leaved us with a total size of around 130 TB. Which is not realistic due to the other big projects that's running now and the projects that are coming up |
|
12:12
🔗
|
arkiver |
On top of that 130 TB we would also have around 20 more TB for groups, profiles, stamps, etc. |
|
12:55
🔗
|
|
primus104 has joined #archiveteam |
|
13:01
🔗
|
|
garyrh has joined #archiveteam |
|
13:25
🔗
|
arkiver |
So we'll need a delay of at least 5 days of the shutdown of blip to be able to get everything |
|
13:35
🔗
|
schbirid |
130 tb of animated shitty gifs, that's a waste |
|
13:36
🔗
|
arkiver |
nothing's a waste. it's just not possible currently due to the price of storage and the other big (upcoming) projects |
|
13:39
🔗
|
|
wvdp___ has quit IRC (Read error: Connection reset by peer) |
|
13:46
🔗
|
schbirid |
the cost versus potential benefit ratio here is insane though |
|
13:46
🔗
|
xmc |
on the animated gif thingy? |
|
13:46
🔗
|
xmc |
or on blip |
|
13:47
🔗
|
arkiver |
animated gif thiingy |
|
13:47
🔗
|
arkiver |
we saved this :) https://web.archive.org/web/20150806180219/http://blip.tv/cbr/batman-the-brave-and-the-bold-emperor-joker-clip-2-4269601 |
|
13:48
🔗
|
xmc |
thank god |
|
13:49
🔗
|
arkiver |
blip videos are not playable in the wayback machine yet |
|
13:49
🔗
|
arkiver |
don't expect them to become playable very soon either |
|
13:50
🔗
|
arkiver |
the way the wayback machine is currently written, it's not possible to add special rules for rewriting some urls |
|
13:50
🔗
|
arkiver |
but everything that would be needed to playback the videos in the wayback machine is saved, so some day they'll work |
|
13:57
🔗
|
arkiver |
so the original video from https://web.archive.org/web/20150806180219/http://blip.tv/cbr/batman-the-brave-and-the-bold-emperor-joker-clip-2-4269601 can be downloaded by going to https://web.archive.org/web/20150806180216/http://blip.tv/rss/flash/4269601 |
|
13:57
🔗
|
arkiver |
and then downloading the original file: https://web.archive.org/web/20150806180246/http://blip.tv/file/get/Cbr-BatmanTheBraveAndTheBoldEmperorJokerClip2438.mov |
|
13:58
🔗
|
arkiver |
after which you'll be redirected to the actual original video |
|
13:58
🔗
|
arkiver |
looks like wayback is having some problems atm |
|
14:16
🔗
|
bentpins |
join #archiveteam-bs |
|
14:16
🔗
|
bentpins |
whoops |
|
14:29
🔗
|
arkiver |
chfoo: can you please add blingee to the projects.json? |
|
14:29
🔗
|
|
xmc sets mode: +o swebb |
|
14:29
🔗
|
|
swebb sets mode: +o brayden |
|
14:40
🔗
|
|
SN4T14 has quit IRC (Read error: Connection reset by peer) |
|
14:41
🔗
|
|
SN4T14 has joined #archiveteam |
|
15:24
🔗
|
|
RichardG has joined #archiveteam |
|
15:34
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
|
15:44
🔗
|
chfoo |
arkiver: ok, added |
|
15:44
🔗
|
arkiver |
thanks! |
|
15:44
🔗
|
arkiver |
chfoo: can you please also add a FOS rsync? |
|
15:44
🔗
|
arkiver |
we'll start with the things that are the most important |
|
15:44
🔗
|
arkiver |
we'll decide later then what more to save and what not |
|
15:47
🔗
|
chfoo |
ok |
|
15:56
🔗
|
SketchCow |
Wheweeeee |
|
15:57
🔗
|
|
zenguy_pc has quit IRC (Read error: Connection reset by peer) |
|
15:57
🔗
|
arkiver |
SketchCow: we need you, important decisions need to be made! |
|
15:57
🔗
|
|
zenguy_pc has joined #archiveteam |
|
16:32
🔗
|
|
JesseW has joined #archiveteam |
|
16:35
🔗
|
|
zenguy_pc has quit IRC (Read error: Connection reset by peer) |
|
16:35
🔗
|
|
zenguy_pc has joined #archiveteam |
|
16:46
🔗
|
aschmitz |
arkiver: I haven't dug too deep into Blingee's setup, but I gathered that the .gifs were assembled out of a number of stamps, a base image, and some text. If so, would it be more feasible to save the components and rules to put them together? Or is that not available from Blingee? |
|
16:51
🔗
|
|
yakfish has quit IRC (Read error: Operation timed out) |
|
17:01
🔗
|
Start |
arkiver: is there an irc channel for blingee? |
|
17:01
🔗
|
arkiver |
no idea |
|
17:01
🔗
|
Start |
ok |
|
17:02
🔗
|
arkiver |
nope, not ye |
|
17:02
🔗
|
arkiver |
yet* |
|
17:05
🔗
|
dashcloud |
SketchCow: how is the manual saving project going? |
|
17:07
🔗
|
|
yakfish has joined #archiveteam |
|
17:13
🔗
|
|
schbirid has quit IRC (Leaving) |
|
17:16
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
17:29
🔗
|
|
xk_id_ has quit IRC (Ping timeout: 252 seconds) |
|
17:29
🔗
|
|
Protab is now known as Rotab |
|
17:29
🔗
|
SketchCow |
Remarkably inspiring that it got the fire. |
|
17:29
🔗
|
|
xk_id has joined #archiveteam |
|
17:29
🔗
|
SketchCow |
Thousands of dollars sent in, possibly as many as 20 people coming |
|
17:30
🔗
|
SketchCow |
I drive down to Baltimore again tonight, get a room, and plan to be there for about 8am-8:30am. |
|
17:30
🔗
|
SketchCow |
And then people start showing up in scads |
|
17:36
🔗
|
|
bentpins has quit IRC (Ping timeout: 483 seconds) |
|
17:37
🔗
|
|
wvdp___ has joined #archiveteam |
|
17:37
🔗
|
|
dan- has quit IRC (Read error: Connection reset by peer) |
|
17:40
🔗
|
db48x |
`sounds like a party |
|
17:44
🔗
|
|
dan- has joined #archiveteam |
|
17:55
🔗
|
|
aaaaaaaaa has joined #archiveteam |
|
17:55
🔗
|
|
swebb sets mode: +o aaaaaaaaa |
|
17:57
🔗
|
|
RichardG has quit IRC (Ping timeout: 362 seconds) |
|
18:00
🔗
|
|
scyther has joined #archiveteam |
|
18:00
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
18:04
🔗
|
|
primus104 has joined #archiveteam |
|
18:13
🔗
|
|
wvdp_ has joined #archiveteam |
|
18:19
🔗
|
|
wvdp___ has quit IRC (Read error: Operation timed out) |
|
18:48
🔗
|
arkiver |
SketchCow: we're going to start the grab of blingee in a bit. How much space does FOS currently have? |
|
18:49
🔗
|
|
habi has joined #archiveteam |
|
18:50
🔗
|
|
habi has left |
|
19:08
🔗
|
anomie |
What's the deal with blingee? |
|
19:08
🔗
|
anomie |
Nevermind. Google is my friend. |
|
19:19
🔗
|
|
xk_id has quit IRC (Read error: Connection reset by peer) |
|
19:29
🔗
|
dashcloud |
glad to hear you got a lot more people to show up at the location |
|
19:45
🔗
|
|
RichardG has joined #archiveteam |
|
20:07
🔗
|
SketchCow |
FOS has 3tb free. |
|
20:07
🔗
|
SketchCow |
I need to kick off some of this other garbage we've been downloading. |
|
20:08
🔗
|
|
achip has quit IRC (Buhbye) |
|
20:11
🔗
|
|
achip has joined #archiveteam |
|
20:12
🔗
|
arkiver |
SketchCow: ok, we'll pause at 2.5 TB |
|
20:13
🔗
|
SketchCow |
Grab the largest images. |
|
20:13
🔗
|
SketchCow |
I'll see about blowing things into the archive. |
|
20:13
🔗
|
arkiver |
Problem of the largest images is that there are two different URLs of exactly the same large image. |
|
20:13
🔗
|
SketchCow |
Use one. |
|
20:14
🔗
|
arkiver |
so one url is shown on blingee.com, the other is used for the embeds in site, blogs, forums, etc. |
|
20:14
🔗
|
arkiver |
so for now I think only grab the URL used on blingee.com |
|
20:14
🔗
|
SketchCow |
Right. |
|
20:15
🔗
|
SketchCow |
Always grab the largest, avoid grabbing two of the same large thing, unless it's tiny. |
|
20:15
🔗
|
SketchCow |
And document.... the FUCK out of. |
|
20:16
🔗
|
arkiver |
Will do that! |
|
20:17
🔗
|
arkiver |
I'll try to go through our projects and write some text for each of them on what is needed for them to playback good |
|
20:18
🔗
|
arkiver |
as in what special rules the wayback machine should have for them to playback nicely (playing videos, etc.) |
|
20:19
🔗
|
SketchCow |
Yes. |
|
20:19
🔗
|
SketchCow |
I think that's really the only choice we need to have in the future. |
|
20:19
🔗
|
SketchCow |
Is for this, and for other stuff! To work with Kenji and other Waybackers to improve playback. |
|
20:19
🔗
|
SketchCow |
And make those notable. We're converting the whole wayback backend, so that's part of what's going on. |
|
20:20
🔗
|
SketchCow |
Ha ha, OK, so apparently the super-secret project to dupe that site is taking 3.5tb of remaining space. |
|
20:20
🔗
|
SketchCow |
So as soon as I start injecting that. |
|
20:28
🔗
|
SketchCow |
So wait, who DID do this thing, the area51 download. |
|
20:28
🔗
|
SketchCow |
arkiver: Was that you? |
|
20:41
🔗
|
SketchCow |
OK, that's cleared up. That will give us 3.5tb back on the FOS |
|
20:43
🔗
|
|
PurpleSym has quit IRC (Remote host closed the connection) |
|
20:54
🔗
|
SketchCow |
Uh.... not immediately. |
|
20:54
🔗
|
SketchCow |
Obviously now it has to go through the hellscape |
|
20:59
🔗
|
|
expr_ has joined #archiveteam |
|
21:05
🔗
|
|
JesseW has quit IRC (Leaving.) |
|
21:13
🔗
|
SketchCow |
OK, off to NYC, then to Maryland, then, you know, lifting manuals all day. |
|
21:14
🔗
|
|
brayden has quit IRC (Quit: Leaving) |
|
21:16
🔗
|
SketchCow |
Here's to saving fucking Blingee |
|
21:22
🔗
|
|
habi has joined #archiveteam |
|
21:23
🔗
|
|
habi has left |
|
21:28
🔗
|
|
Start has joined #archiveteam |
|
21:30
🔗
|
|
scyther has quit IRC (Read error: Connection reset by peer) |
|
21:31
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
|
21:34
🔗
|
|
Start has joined #archiveteam |
|
22:13
🔗
|
|
nertzy has joined #archiveteam |
|
22:31
🔗
|
|
wvdp_ has quit IRC (Read error: Operation timed out) |
|
22:50
🔗
|
|
nertzy has quit IRC (Quit: This computer has gone to sleep) |
|
22:51
🔗
|
|
Start has quit IRC (Ping timeout: 362 seconds) |
|
22:53
🔗
|
|
JesseW has joined #archiveteam |
|
23:04
🔗
|
|
Muad-Dib has quit IRC (Ping timeout: 252 seconds) |
|
23:15
🔗
|
|
Muad-Dib has joined #archiveteam |
|
23:25
🔗
|
|
anomie has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in) |
|
23:30
🔗
|
|
anomie has joined #archiveteam |
|
23:39
🔗
|
|
Start has joined #archiveteam |
|
23:41
🔗
|
|
mr-b has quit IRC (Quit: ZNC - http://znc.in) |
|
23:49
🔗
|
|
Start has quit IRC (Ping timeout: 362 seconds) |