Time |
Nickname |
Message |
00:25
🔗
|
|
LS64 has joined #archiveteam |
00:32
🔗
|
FalconK |
so we moved a bunch of these WARCs I uploaded to the web collection, but they aren't showing up in the wayback machine. What else is wrong? |
00:32
🔗
|
FalconK |
reference https://archive.org/details/archiveteam_archivebot_go_falconk_vez_mrsk_me_20160320 which is someone's crawl of http://vez.mrsk.me |
00:34
🔗
|
FalconK |
here is an example of a working one. don't see much different. https://archive.org/details/archiveteam_archivebot_go_20150504140003 |
00:35
🔗
|
FalconK |
maybe a derive needs doing? I wonder how I can do that |
00:37
🔗
|
SketchCow |
The difference is I did one, and you did another. |
00:37
🔗
|
SketchCow |
If you don't upload it as type web, it confuses the thing. |
00:38
🔗
|
SketchCow |
I just forced a rederive |
00:38
🔗
|
FalconK |
it was uploaded as type web. |
00:38
🔗
|
FalconK |
is there an API way for me to rederive, or any way at all? |
00:39
🔗
|
FalconK |
because I think all of the ones I've uploaded are broken. |
00:42
🔗
|
SketchCow |
I'm not sure. |
00:45
🔗
|
FalconK |
xmc: any ideas? |
00:46
🔗
|
xmc |
hm no |
00:47
🔗
|
xmc |
there is a way to rederive, at least for me |
00:47
🔗
|
FalconK |
ok, I found a way to rederive them (via the item manager) for the ones I uploaded |
00:48
🔗
|
LS64 |
Hello, I'm sorry to be a bother, but I was wondering if anyone could answer a question about the Fanfiction.net archival? |
00:48
🔗
|
FalconK |
hopefully there is an API, since there are quite a lot |
00:50
🔗
|
|
logchfoo2 starts logging #archiveteam at Fri Apr 01 00:50:34 2016 |
00:50
🔗
|
|
logchfoo2 has joined #archiveteam |
00:50
🔗
|
FalconK |
would you like it to be? what's up with it? |
00:50
🔗
|
xmc |
yes it was |
00:50
🔗
|
|
midas1 has joined #archiveteam |
00:51
🔗
|
|
midas has quit IRC (Read error: Operation timed out) |
00:51
🔗
|
FalconK |
oh, wait, I'm just bad at searching |
00:51
🔗
|
|
lbft_ has joined #archiveteam |
00:51
🔗
|
LS64 |
No, I just thought it was as it has a page, I've just been struggling to attempt to use the "getter" from github, unless it isn't supposed to work yet |
00:52
🔗
|
|
patrickod has quit IRC (Ping timeout: 258 seconds) |
00:52
🔗
|
|
achip has quit IRC (Ping timeout: 258 seconds) |
00:52
🔗
|
|
lbft has quit IRC (Read error: Operation timed out) |
00:52
🔗
|
|
Baljem_ has joined #archiveteam |
00:52
🔗
|
|
VADemon has quit IRC (hub.efnet.us irc.Prison.NET) |
00:52
🔗
|
|
db48x` has quit IRC (hub.efnet.us irc.Prison.NET) |
00:52
🔗
|
|
Baljem has quit IRC (hub.efnet.us irc.Prison.NET) |
00:52
🔗
|
FalconK |
lol, their robots.txt specifically disallows ia_archiver |
00:53
🔗
|
LS64 |
I assume you're talking about FF, which I hate them for |
00:54
🔗
|
LS64 |
such a childish decision it seems |
00:54
🔗
|
FalconK |
meh |
00:55
🔗
|
FalconK |
SketchCow: the rederive completed but didn't change anything |
00:59
🔗
|
LS64 |
Well, if no one can really help, my apologies for bothering, I'll continue trying to download the 2012 mirror from IA |
01:00
🔗
|
FalconK |
idk, does anyone have any strong feelings about crawling it? |
01:01
🔗
|
LS64 |
Oh, I'm not begging for a crawl at this moment, as if I could not do it myself it'd be rude of me to ask, I was just trying to access what has already been gathered |
01:01
🔗
|
bsmith093 |
LS64: i could help, whats your question? |
01:01
🔗
|
LS64 |
I'm attempting to make use of the ffnet-grab program you point to on the page for it |
01:02
🔗
|
bsmith093 |
LS64: story id 1-10 milliton is in archive.org/details/fanfictiondotnet_repack |
01:02
🔗
|
bsmith093 |
LS64: fanficfare? |
01:02
🔗
|
bsmith093 |
pip install fanficfare. much faster and easier |
01:02
🔗
|
LS64 |
oh okay! |
01:03
🔗
|
LS64 |
now, I downloaded the index text file from that repack prieviously |
01:03
🔗
|
bsmith093 |
the configuration is a little tricky |
01:03
🔗
|
LS64 |
I'd assume that lists every fanfiction available on there? |
01:05
🔗
|
bsmith093 |
https://archive.org/details/fanfictiondotnet_repack has a sql file, with the metadata. 6.7 million files |
01:06
🔗
|
LS64 |
Oh I didn't see the sqlite file, I'd only seen the inventory.zip |
01:06
🔗
|
bsmith093 |
it's a recent addition. |
01:07
🔗
|
LS64 |
ah, so is the inventory.zip out of date? |
01:07
🔗
|
bsmith093 |
zip files are faster to extract a single file from than one giant tar.gz |
01:08
🔗
|
bsmith093 |
slightly, yes, the Fanfiction_misc.zip fole contains every file that happens to start with an underscore |
01:08
🔗
|
|
Mayeau is now known as Mayonaise |
01:10
🔗
|
LS64 |
Oh okay, well, it seems like I might be outta luck, as I already searched the inventory file and didn't find what I was looking for listed, alas |
01:11
🔗
|
bsmith093 |
LS64: what are you looking for |
01:11
🔗
|
|
philpem has quit IRC (Ping timeout: 260 seconds) |
01:12
🔗
|
LS64 |
I'm just looking for a fanfiction that I know is gone from the main site, I know it's exact name because I have a bookmark from probably 3 years ago, but it possibly was yanked before this archive was done |
01:12
🔗
|
bsmith093 |
title? |
01:12
🔗
|
bsmith093 |
and category? |
01:14
🔗
|
|
BlueMaxim has quit IRC (Read error: Operation timed out) |
01:15
🔗
|
LS64 |
Title... "The Secrets Behind The Rose" |
01:15
🔗
|
|
BlueMaxim has joined #archiveteam |
01:15
🔗
|
LS64 |
and sorry, as I don't quite know what you mean by "category" |
01:15
🔗
|
bsmith093 |
LS64: what fandom? |
01:15
🔗
|
LS64 |
Oh, Sonic, game fandom |
01:16
🔗
|
LS64 |
I was organizing bookmarks when I found these, and once I get stuck on something, I find it hard to let go lol |
01:17
🔗
|
bsmith093 |
fanficfare has a calibre plugin |
01:17
🔗
|
bsmith093 |
great thing |
01:17
🔗
|
LS64 |
yeah, I'm reading up on fanfifare now |
01:18
🔗
|
bsmith093 |
anyway before we get the off-topic siren, join #archiveteam-bs |
01:18
🔗
|
|
ariscop has quit IRC (Quit: Leaving) |
01:19
🔗
|
|
ariscop has joined #archiveteam |
01:19
🔗
|
|
JetBalsa has joined #archiveteam |
01:20
🔗
|
|
useretail has joined #archiveteam |
01:20
🔗
|
|
patrickod has joined #archiveteam |
01:20
🔗
|
|
achip has joined #archiveteam |
01:38
🔗
|
|
Stiletto has joined #archiveteam |
01:42
🔗
|
|
VADemon_ has quit IRC (Quit: left4dead) |
02:13
🔗
|
|
JesseW has joined #archiveteam |
02:21
🔗
|
|
tomwsmf-a has joined #archiveteam |
02:25
🔗
|
|
nwf has joined #archiveteam |
02:29
🔗
|
|
godane has quit IRC (Quit: Leaving.) |
02:41
🔗
|
|
ariscop has quit IRC (Quit: Leaving) |
02:42
🔗
|
|
ariscop has joined #archiveteam |
03:22
🔗
|
|
tomwsmf-a has quit IRC (Read error: Operation timed out) |
03:54
🔗
|
|
metalcamp has joined #archiveteam |
03:59
🔗
|
|
JetBalsa has quit IRC (Read error: Connection reset by peer) |
04:02
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
04:10
🔗
|
|
dashcloud has joined #archiveteam |
04:16
🔗
|
|
Sk1d has quit IRC (Ping timeout: 250 seconds) |
04:23
🔗
|
|
Sk1d has joined #archiveteam |
04:32
🔗
|
|
db48x` has joined #archiveteam |
05:19
🔗
|
|
wp494 has quit IRC (Quit: LOUD UNNECESSARY QUIT MESSAGES) |
05:28
🔗
|
|
wp494 has joined #archiveteam |
05:51
🔗
|
|
WinterFox has joined #archiveteam |
05:56
🔗
|
|
Start has quit IRC (Ping timeout: 260 seconds) |
06:09
🔗
|
|
nwf has quit IRC (Read error: Operation timed out) |
06:13
🔗
|
|
metalcamp has quit IRC (Ping timeout: 244 seconds) |
06:17
🔗
|
|
LS64 has quit IRC (Ping timeout: 268 seconds) |
06:24
🔗
|
|
Start has joined #archiveteam |
06:25
🔗
|
|
JesseW has left |
06:35
🔗
|
|
Deewiant_ is now known as Deewiant |
06:42
🔗
|
|
Honno has joined #archiveteam |
06:48
🔗
|
|
ariscop has quit IRC (Quit: Leaving) |
07:29
🔗
|
|
schbirid has joined #archiveteam |
07:36
🔗
|
|
vitzli has joined #archiveteam |
07:36
🔗
|
|
Stiletto is now known as Stilett0 |
07:40
🔗
|
|
RichardG_ has joined #archiveteam |
07:41
🔗
|
|
RichardG has quit IRC (Ping timeout: 250 seconds) |
07:45
🔗
|
|
RichardG has joined #archiveteam |
07:47
🔗
|
|
RichardG_ has quit IRC (Ping timeout: 260 seconds) |
07:50
🔗
|
|
DFJustin has quit IRC (Read error: Connection reset by peer) |
07:50
🔗
|
|
DFJustin has joined #archiveteam |
07:50
🔗
|
|
swebb sets mode: +o DFJustin |
07:51
🔗
|
|
robink has quit IRC (Remote host closed the connection) |
07:59
🔗
|
|
robink has joined #archiveteam |
07:59
🔗
|
|
robink has quit IRC (Remote host closed the connection) |
08:02
🔗
|
|
robink has joined #archiveteam |
08:15
🔗
|
db48x` |
http://www.digitalstudies.org/ojs/index.php/digital_studies/article/view/325/412 |
08:16
🔗
|
|
ariscop has joined #archiveteam |
08:47
🔗
|
|
acridAxid has quit IRC (Read error: Operation timed out) |
08:48
🔗
|
|
bwn_ has quit IRC (Read error: Operation timed out) |
08:49
🔗
|
|
yakfish has quit IRC (Read error: Operation timed out) |
08:49
🔗
|
|
yakfish has joined #archiveteam |
08:49
🔗
|
|
acridAxid has joined #archiveteam |
09:01
🔗
|
|
bwn has joined #archiveteam |
09:11
🔗
|
|
Tomcat_ has joined #archiveteam |
09:13
🔗
|
|
atomotic has joined #archiveteam |
09:16
🔗
|
|
VADemon has joined #archiveteam |
09:39
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
10:03
🔗
|
|
vitzli has quit IRC (Leaving) |
10:38
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
10:57
🔗
|
|
Ymgve has quit IRC (Ping timeout: 506 seconds) |
11:20
🔗
|
|
Ymgve has joined #archiveteam |
11:32
🔗
|
|
wyatt8740 has quit IRC (Read error: Operation timed out) |
11:33
🔗
|
|
atomotic has joined #archiveteam |
11:34
🔗
|
|
wvdp has joined #archiveteam |
11:34
🔗
|
|
wyatt8740 has joined #archiveteam |
11:40
🔗
|
|
wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) |
11:40
🔗
|
|
ohhdemgir has quit IRC (Read error: Connection reset by peer) |
11:48
🔗
|
|
ohhdemgir has joined #archiveteam |
11:55
🔗
|
|
Honno has quit IRC (Read error: Operation timed out) |
12:01
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
12:16
🔗
|
|
Fletcher has quit IRC (Ping timeout: 244 seconds) |
12:16
🔗
|
|
BnA-Rob1n has quit IRC (Ping timeout: 244 seconds) |
12:16
🔗
|
|
lukeman_ has quit IRC (Ping timeout: 244 seconds) |
12:17
🔗
|
|
Froggypwn has quit IRC (Ping timeout: 244 seconds) |
12:17
🔗
|
|
Fletcher has joined #archiveteam |
12:17
🔗
|
|
Froggypwn has joined #archiveteam |
12:18
🔗
|
|
lukeman has joined #archiveteam |
12:18
🔗
|
|
Fletcher_ sets mode: +o Fletcher |
12:18
🔗
|
|
BnA-Rob1n has joined #archiveteam |
12:27
🔗
|
|
bentpins has joined #archiveteam |
12:33
🔗
|
|
MMovie has quit IRC (Read error: Operation timed out) |
12:34
🔗
|
|
aMunster has quit IRC (Read error: Operation timed out) |
12:37
🔗
|
|
beardicus has quit IRC (Read error: Operation timed out) |
12:40
🔗
|
|
beardicus has joined #archiveteam |
12:43
🔗
|
|
lbft_ has quit IRC (Read error: Operation timed out) |
12:45
🔗
|
|
Honno has joined #archiveteam |
12:45
🔗
|
|
lbft has joined #archiveteam |
12:47
🔗
|
|
mhazinsk has quit IRC (Ping timeout: 634 seconds) |
12:47
🔗
|
|
Mayonaise has quit IRC (Read error: Operation timed out) |
12:49
🔗
|
|
Mayonaise has joined #archiveteam |
12:52
🔗
|
|
aMunster has joined #archiveteam |
13:06
🔗
|
|
mhazinsk has joined #archiveteam |
13:07
🔗
|
tephra |
There's a call for help in preserving whatwg (web standards group) irc logs for historian here: https://lists.w3.org/Archives/Public/public-webhistory/2016Apr/0000.html |
13:09
🔗
|
|
godane has joined #archiveteam |
13:09
🔗
|
|
MMovie has joined #archiveteam |
13:09
🔗
|
arkiver |
tephra: http://krijnhoetmer.nl/irc-logs being archived |
13:16
🔗
|
|
atomotic has joined #archiveteam |
13:19
🔗
|
|
Emcy has joined #archiveteam |
13:21
🔗
|
tephra |
arkiver: great :) thank |
13:21
🔗
|
tephra |
s |
13:23
🔗
|
bentpins |
So I have about $50 in DO money due to expire in a monthish. I haven't been following things with archiveteam in a while, but is any more grunt needed? Or should I spin up some TOR middle nodes |
13:26
🔗
|
HCross |
yuku |
13:31
🔗
|
|
maseck_ has joined #archiveteam |
13:31
🔗
|
bentpins |
ah cheers. I'll look at spinning up ~9 instances |
13:39
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
13:40
🔗
|
|
davidar has quit IRC (Quit: Connection closed for inactivity) |
13:43
🔗
|
|
maseck has quit IRC (Ping timeout: 1208 seconds) |
13:44
🔗
|
phuzion |
bentpins: If you use ansible, check out the deployment script HCross and I were working on a while ago: https://github.com/HarryC145/archiveteam-deploy |
13:45
🔗
|
|
Stilett0 is now known as Stiletto |
13:46
🔗
|
HCross |
just change the config |
13:46
🔗
|
HCross |
so the vars |
13:47
🔗
|
bentpins |
Ansible plus script sounds ideal, but I'm not on ansible and I gather it's a paid service? |
13:47
🔗
|
HCross |
no it isnt |
13:50
🔗
|
bentpins |
Ah great, redhat's site doesn't give that impression with "pricing" and "trials" and such. The github is more telling. I'll have a go, I'm keen to learn |
13:50
🔗
|
bentpins |
cheers |
13:53
🔗
|
|
WinterFox has quit IRC (Remote host closed the connection) |
13:58
🔗
|
|
philpem has joined #archiveteam |
14:14
🔗
|
|
wvdp has joined #archiveteam |
14:15
🔗
|
|
davidar has joined #archiveteam |
14:35
🔗
|
|
nwf has joined #archiveteam |
14:42
🔗
|
|
atomotic has joined #archiveteam |
14:58
🔗
|
|
Tomcat_ has quit IRC (Remote host closed the connection) |
15:17
🔗
|
|
redlob has quit IRC (Read error: Operation timed out) |
15:18
🔗
|
|
redlob has joined #archiveteam |
15:19
🔗
|
|
scyther has joined #archiveteam |
15:29
🔗
|
|
phuz has joined #archiveteam |
15:30
🔗
|
|
phuzion has quit IRC (Read error: Connection reset by peer) |
15:32
🔗
|
|
dserodio has quit IRC (Excess Flood) |
15:32
🔗
|
|
RichardG has quit IRC (Ping timeout: 499 seconds) |
15:34
🔗
|
|
dserodio has joined #archiveteam |
16:04
🔗
|
|
RichardG has joined #archiveteam |
16:18
🔗
|
|
JesseW has joined #archiveteam |
16:29
🔗
|
|
RichardG has quit IRC (Ping timeout: 250 seconds) |
16:30
🔗
|
|
phuz is now known as phuzion |
16:33
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
16:48
🔗
|
|
JesseW has quit IRC (Ping timeout: 370 seconds) |
16:51
🔗
|
|
Tomcat_ has joined #archiveteam |
16:54
🔗
|
|
wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) |
16:54
🔗
|
|
wvdp has joined #archiveteam |
17:04
🔗
|
|
JetBalsa has joined #archiveteam |
17:18
🔗
|
|
scyther has quit IRC (Quit: Leaving) |
18:25
🔗
|
|
VADemon has quit IRC (Quit: left4dead) |
18:26
🔗
|
|
VADemon has joined #archiveteam |
18:28
🔗
|
|
RichardG has joined #archiveteam |
18:29
🔗
|
|
scyther has joined #archiveteam |
18:29
🔗
|
|
scyther has quit IRC (Connection closed) |
18:36
🔗
|
|
Stiletto has quit IRC () |
18:36
🔗
|
|
Stiletto has joined #archiveteam |
18:37
🔗
|
|
RichardG has quit IRC (Ping timeout: 499 seconds) |
18:43
🔗
|
|
RichardG has joined #archiveteam |
18:47
🔗
|
|
bsmith093 has quit IRC (Ping timeout: 260 seconds) |
19:12
🔗
|
|
Emcy has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
Deewiant has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
espes__ has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
ats has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
chfoo has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
ersi has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
goekesmi has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
koon has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
Fletcher_ has quit IRC (hub.se efnet.portlane.se) |
19:12
🔗
|
|
espes___ has joined #archiveteam |
19:13
🔗
|
|
ring has quit IRC (Read error: Operation timed out) |
19:13
🔗
|
|
tomwsmf-a has joined #archiveteam |
19:15
🔗
|
|
ring has joined #archiveteam |
19:16
🔗
|
|
chfoo0 has joined #archiveteam |
19:22
🔗
|
|
Fletcher_ has joined #archiveteam |
19:22
🔗
|
|
ats has joined #archiveteam |
19:22
🔗
|
|
koon has joined #archiveteam |
19:22
🔗
|
|
Deewiant has joined #archiveteam |
19:24
🔗
|
|
bsmith093 has joined #archiveteam |
19:26
🔗
|
|
goekesmi_ has joined #archiveteam |
19:39
🔗
|
|
ersi has joined #archiveteam |
19:39
🔗
|
|
swebb sets mode: +o ersi |
19:40
🔗
|
|
bwn has quit IRC (Read error: Operation timed out) |
19:51
🔗
|
|
VADemon has quit IRC (Quit: left4dead) |
19:58
🔗
|
|
bwn has joined #archiveteam |
20:14
🔗
|
|
bwn_ has joined #archiveteam |
20:23
🔗
|
|
bwn has quit IRC (Read error: Operation timed out) |
20:44
🔗
|
|
Honno has quit IRC (Ping timeout: 492 seconds) |
20:56
🔗
|
|
Tomcat_ has quit IRC (Remote host closed the connection) |
20:59
🔗
|
|
ats has quit IRC (Ping timeout: 250 seconds) |
21:00
🔗
|
|
BlueMaxim has joined #archiveteam |
21:00
🔗
|
|
ats has joined #archiveteam |
21:00
🔗
|
|
ersi has quit IRC (Ping timeout: 250 seconds) |
21:01
🔗
|
|
Deewiant_ has joined #archiveteam |
21:08
🔗
|
|
Fletcher_ has quit IRC (hub.se efnet.portlane.se) |
21:08
🔗
|
|
koon has quit IRC (hub.se efnet.portlane.se) |
21:08
🔗
|
|
Deewiant has quit IRC (hub.se efnet.portlane.se) |
21:15
🔗
|
|
vtyl has quit IRC (Ping timeout: 260 seconds) |
21:17
🔗
|
|
BlueMaxim has quit IRC (Read error: Operation timed out) |
21:18
🔗
|
|
vtyl has joined #archiveteam |
21:19
🔗
|
|
bwn_ is now known as bwn |
21:24
🔗
|
|
chfoo0 is now known as chfoo |
21:33
🔗
|
|
bwn has quit IRC (Read error: Operation timed out) |
21:34
🔗
|
|
bwn has joined #archiveteam |
21:46
🔗
|
|
WinterFox has joined #archiveteam |
21:49
🔗
|
|
Deewiant_ has quit IRC (hub.se irc.du.se) |
21:49
🔗
|
|
Sk1d has quit IRC (hub.se irc.du.se) |
21:56
🔗
|
|
bwn has quit IRC (Leaving) |
21:57
🔗
|
|
bwn has joined #archiveteam |
22:18
🔗
|
|
db48x` is now known as db48x |
22:24
🔗
|
|
schbirid has quit IRC (Quit: Leaving) |
22:24
🔗
|
|
Boppen has joined #archiveteam |
22:29
🔗
|
|
wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) |
22:37
🔗
|
|
aliz has quit IRC (Remote host closed the connection) |
23:09
🔗
|
|
midas1 is now known as midas |
23:24
🔗
|
|
tubenerd1 has joined #archiveteam |
23:24
🔗
|
|
Microguru has quit IRC (Read error: Connection reset by peer) |
23:25
🔗
|
tubenerd1 |
archive team, you available? |
23:25
🔗
|
|
tubenerd1 has quit IRC (Client Quit) |
23:26
🔗
|
|
nillary16 has joined #archiveteam |
23:27
🔗
|
|
nillary16 has quit IRC (Client Quit) |
23:36
🔗
|
|
tomwsmf-a has quit IRC (Read error: Operation timed out) |
23:52
🔗
|
dashcloud |
sure- what's up? |
23:55
🔗
|
|
JesseW has joined #archiveteam |
23:59
🔗
|
Frogging |
I think he left |