#archiveteam 2016-04-01,Fri

↑back Search

Time Nickname Message
00:25 🔗 LS64 has joined #archiveteam
00:32 🔗 FalconK so we moved a bunch of these WARCs I uploaded to the web collection, but they aren't showing up in the wayback machine. What else is wrong?
00:32 🔗 FalconK reference https://archive.org/details/archiveteam_archivebot_go_falconk_vez_mrsk_me_20160320 which is someone's crawl of http://vez.mrsk.me
00:34 🔗 FalconK here is an example of a working one. don't see much different. https://archive.org/details/archiveteam_archivebot_go_20150504140003
00:35 🔗 FalconK maybe a derive needs doing? I wonder how I can do that
00:37 🔗 SketchCow The difference is I did one, and you did another.
00:37 🔗 SketchCow If you don't upload it as type web, it confuses the thing.
00:38 🔗 SketchCow I just forced a rederive
00:38 🔗 FalconK it was uploaded as type web.
00:38 🔗 FalconK is there an API way for me to rederive, or any way at all?
00:39 🔗 FalconK because I think all of the ones I've uploaded are broken.
00:42 🔗 SketchCow I'm not sure.
00:45 🔗 FalconK xmc: any ideas?
00:46 🔗 xmc hm no
00:47 🔗 xmc there is a way to rederive, at least for me
00:47 🔗 FalconK ok, I found a way to rederive them (via the item manager) for the ones I uploaded
00:48 🔗 LS64 Hello, I'm sorry to be a bother, but I was wondering if anyone could answer a question about the Fanfiction.net archival?
00:48 🔗 FalconK hopefully there is an API, since there are quite a lot
00:50 🔗 logchfoo2 starts logging #archiveteam at Fri Apr 01 00:50:34 2016
00:50 🔗 logchfoo2 has joined #archiveteam
00:50 🔗 FalconK would you like it to be? what's up with it?
00:50 🔗 xmc yes it was
00:50 🔗 midas1 has joined #archiveteam
00:51 🔗 midas has quit IRC (Read error: Operation timed out)
00:51 🔗 FalconK oh, wait, I'm just bad at searching
00:51 🔗 lbft_ has joined #archiveteam
00:51 🔗 LS64 No, I just thought it was as it has a page, I've just been struggling to attempt to use the "getter" from github, unless it isn't supposed to work yet
00:52 🔗 patrickod has quit IRC (Ping timeout: 258 seconds)
00:52 🔗 achip has quit IRC (Ping timeout: 258 seconds)
00:52 🔗 lbft has quit IRC (Read error: Operation timed out)
00:52 🔗 Baljem_ has joined #archiveteam
00:52 🔗 VADemon has quit IRC (hub.efnet.us irc.Prison.NET)
00:52 🔗 db48x` has quit IRC (hub.efnet.us irc.Prison.NET)
00:52 🔗 Baljem has quit IRC (hub.efnet.us irc.Prison.NET)
00:52 🔗 FalconK lol, their robots.txt specifically disallows ia_archiver
00:53 🔗 LS64 I assume you're talking about FF, which I hate them for
00:54 🔗 LS64 such a childish decision it seems
00:54 🔗 FalconK meh
00:55 🔗 FalconK SketchCow: the rederive completed but didn't change anything
00:59 🔗 LS64 Well, if no one can really help, my apologies for bothering, I'll continue trying to download the 2012 mirror from IA
01:00 🔗 FalconK idk, does anyone have any strong feelings about crawling it?
01:01 🔗 LS64 Oh, I'm not begging for a crawl at this moment, as if I could not do it myself it'd be rude of me to ask, I was just trying to access what has already been gathered
01:01 🔗 bsmith093 LS64: i could help, whats your question?
01:01 🔗 LS64 I'm attempting to make use of the ffnet-grab program you point to on the page for it
01:02 🔗 bsmith093 LS64: story id 1-10 milliton is in archive.org/details/fanfictiondotnet_repack
01:02 🔗 bsmith093 LS64: fanficfare?
01:02 🔗 bsmith093 pip install fanficfare. much faster and easier
01:02 🔗 LS64 oh okay!
01:03 🔗 LS64 now, I downloaded the index text file from that repack prieviously
01:03 🔗 bsmith093 the configuration is a little tricky
01:03 🔗 LS64 I'd assume that lists every fanfiction available on there?
01:05 🔗 bsmith093 https://archive.org/details/fanfictiondotnet_repack has a sql file, with the metadata. 6.7 million files
01:06 🔗 LS64 Oh I didn't see the sqlite file, I'd only seen the inventory.zip
01:06 🔗 bsmith093 it's a recent addition.
01:07 🔗 LS64 ah, so is the inventory.zip out of date?
01:07 🔗 bsmith093 zip files are faster to extract a single file from than one giant tar.gz
01:08 🔗 bsmith093 slightly, yes, the Fanfiction_misc.zip fole contains every file that happens to start with an underscore
01:08 🔗 Mayeau is now known as Mayonaise
01:10 🔗 LS64 Oh okay, well, it seems like I might be outta luck, as I already searched the inventory file and didn't find what I was looking for listed, alas
01:11 🔗 bsmith093 LS64: what are you looking for
01:11 🔗 philpem has quit IRC (Ping timeout: 260 seconds)
01:12 🔗 LS64 I'm just looking for a fanfiction that I know is gone from the main site, I know it's exact name because I have a bookmark from probably 3 years ago, but it possibly was yanked before this archive was done
01:12 🔗 bsmith093 title?
01:12 🔗 bsmith093 and category?
01:14 🔗 BlueMaxim has quit IRC (Read error: Operation timed out)
01:15 🔗 LS64 Title... "The Secrets Behind The Rose"
01:15 🔗 BlueMaxim has joined #archiveteam
01:15 🔗 LS64 and sorry, as I don't quite know what you mean by "category"
01:15 🔗 bsmith093 LS64: what fandom?
01:15 🔗 LS64 Oh, Sonic, game fandom
01:16 🔗 LS64 I was organizing bookmarks when I found these, and once I get stuck on something, I find it hard to let go lol
01:17 🔗 bsmith093 fanficfare has a calibre plugin
01:17 🔗 bsmith093 great thing
01:17 🔗 LS64 yeah, I'm reading up on fanfifare now
01:18 🔗 bsmith093 anyway before we get the off-topic siren, join #archiveteam-bs
01:18 🔗 ariscop has quit IRC (Quit: Leaving)
01:19 🔗 ariscop has joined #archiveteam
01:19 🔗 JetBalsa has joined #archiveteam
01:20 🔗 useretail has joined #archiveteam
01:20 🔗 patrickod has joined #archiveteam
01:20 🔗 achip has joined #archiveteam
01:38 🔗 Stiletto has joined #archiveteam
01:42 🔗 VADemon_ has quit IRC (Quit: left4dead)
02:13 🔗 JesseW has joined #archiveteam
02:21 🔗 tomwsmf-a has joined #archiveteam
02:25 🔗 nwf has joined #archiveteam
02:29 🔗 godane has quit IRC (Quit: Leaving.)
02:41 🔗 ariscop has quit IRC (Quit: Leaving)
02:42 🔗 ariscop has joined #archiveteam
03:22 🔗 tomwsmf-a has quit IRC (Read error: Operation timed out)
03:54 🔗 metalcamp has joined #archiveteam
03:59 🔗 JetBalsa has quit IRC (Read error: Connection reset by peer)
04:02 🔗 dashcloud has quit IRC (Read error: Operation timed out)
04:10 🔗 dashcloud has joined #archiveteam
04:16 🔗 Sk1d has quit IRC (Ping timeout: 250 seconds)
04:23 🔗 Sk1d has joined #archiveteam
04:32 🔗 db48x` has joined #archiveteam
05:19 🔗 wp494 has quit IRC (Quit: LOUD UNNECESSARY QUIT MESSAGES)
05:28 🔗 wp494 has joined #archiveteam
05:51 🔗 WinterFox has joined #archiveteam
05:56 🔗 Start has quit IRC (Ping timeout: 260 seconds)
06:09 🔗 nwf has quit IRC (Read error: Operation timed out)
06:13 🔗 metalcamp has quit IRC (Ping timeout: 244 seconds)
06:17 🔗 LS64 has quit IRC (Ping timeout: 268 seconds)
06:24 🔗 Start has joined #archiveteam
06:25 🔗 JesseW has left
06:35 🔗 Deewiant_ is now known as Deewiant
06:42 🔗 Honno has joined #archiveteam
06:48 🔗 ariscop has quit IRC (Quit: Leaving)
07:29 🔗 schbirid has joined #archiveteam
07:36 🔗 vitzli has joined #archiveteam
07:36 🔗 Stiletto is now known as Stilett0
07:40 🔗 RichardG_ has joined #archiveteam
07:41 🔗 RichardG has quit IRC (Ping timeout: 250 seconds)
07:45 🔗 RichardG has joined #archiveteam
07:47 🔗 RichardG_ has quit IRC (Ping timeout: 260 seconds)
07:50 🔗 DFJustin has quit IRC (Read error: Connection reset by peer)
07:50 🔗 DFJustin has joined #archiveteam
07:50 🔗 swebb sets mode: +o DFJustin
07:51 🔗 robink has quit IRC (Remote host closed the connection)
07:59 🔗 robink has joined #archiveteam
07:59 🔗 robink has quit IRC (Remote host closed the connection)
08:02 🔗 robink has joined #archiveteam
08:15 🔗 db48x` http://www.digitalstudies.org/ojs/index.php/digital_studies/article/view/325/412
08:16 🔗 ariscop has joined #archiveteam
08:47 🔗 acridAxid has quit IRC (Read error: Operation timed out)
08:48 🔗 bwn_ has quit IRC (Read error: Operation timed out)
08:49 🔗 yakfish has quit IRC (Read error: Operation timed out)
08:49 🔗 yakfish has joined #archiveteam
08:49 🔗 acridAxid has joined #archiveteam
09:01 🔗 bwn has joined #archiveteam
09:11 🔗 Tomcat_ has joined #archiveteam
09:13 🔗 atomotic has joined #archiveteam
09:16 🔗 VADemon has joined #archiveteam
09:39 🔗 BlueMaxim has quit IRC (Quit: Leaving)
10:03 🔗 vitzli has quit IRC (Leaving)
10:38 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
10:57 🔗 Ymgve has quit IRC (Ping timeout: 506 seconds)
11:20 🔗 Ymgve has joined #archiveteam
11:32 🔗 wyatt8740 has quit IRC (Read error: Operation timed out)
11:33 🔗 atomotic has joined #archiveteam
11:34 🔗 wvdp has joined #archiveteam
11:34 🔗 wyatt8740 has joined #archiveteam
11:40 🔗 wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client)
11:40 🔗 ohhdemgir has quit IRC (Read error: Connection reset by peer)
11:48 🔗 ohhdemgir has joined #archiveteam
11:55 🔗 Honno has quit IRC (Read error: Operation timed out)
12:01 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
12:16 🔗 Fletcher has quit IRC (Ping timeout: 244 seconds)
12:16 🔗 BnA-Rob1n has quit IRC (Ping timeout: 244 seconds)
12:16 🔗 lukeman_ has quit IRC (Ping timeout: 244 seconds)
12:17 🔗 Froggypwn has quit IRC (Ping timeout: 244 seconds)
12:17 🔗 Fletcher has joined #archiveteam
12:17 🔗 Froggypwn has joined #archiveteam
12:18 🔗 lukeman has joined #archiveteam
12:18 🔗 Fletcher_ sets mode: +o Fletcher
12:18 🔗 BnA-Rob1n has joined #archiveteam
12:27 🔗 bentpins has joined #archiveteam
12:33 🔗 MMovie has quit IRC (Read error: Operation timed out)
12:34 🔗 aMunster has quit IRC (Read error: Operation timed out)
12:37 🔗 beardicus has quit IRC (Read error: Operation timed out)
12:40 🔗 beardicus has joined #archiveteam
12:43 🔗 lbft_ has quit IRC (Read error: Operation timed out)
12:45 🔗 Honno has joined #archiveteam
12:45 🔗 lbft has joined #archiveteam
12:47 🔗 mhazinsk has quit IRC (Ping timeout: 634 seconds)
12:47 🔗 Mayonaise has quit IRC (Read error: Operation timed out)
12:49 🔗 Mayonaise has joined #archiveteam
12:52 🔗 aMunster has joined #archiveteam
13:06 🔗 mhazinsk has joined #archiveteam
13:07 🔗 tephra There's a call for help in preserving whatwg (web standards group) irc logs for historian here: https://lists.w3.org/Archives/Public/public-webhistory/2016Apr/0000.html
13:09 🔗 godane has joined #archiveteam
13:09 🔗 MMovie has joined #archiveteam
13:09 🔗 arkiver tephra: http://krijnhoetmer.nl/irc-logs being archived
13:16 🔗 atomotic has joined #archiveteam
13:19 🔗 Emcy has joined #archiveteam
13:21 🔗 tephra arkiver: great :) thank
13:21 🔗 tephra s
13:23 🔗 bentpins So I have about $50 in DO money due to expire in a monthish. I haven't been following things with archiveteam in a while, but is any more grunt needed? Or should I spin up some TOR middle nodes
13:26 🔗 HCross yuku
13:31 🔗 maseck_ has joined #archiveteam
13:31 🔗 bentpins ah cheers. I'll look at spinning up ~9 instances
13:39 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
13:40 🔗 davidar has quit IRC (Quit: Connection closed for inactivity)
13:43 🔗 maseck has quit IRC (Ping timeout: 1208 seconds)
13:44 🔗 phuzion bentpins: If you use ansible, check out the deployment script HCross and I were working on a while ago: https://github.com/HarryC145/archiveteam-deploy
13:45 🔗 Stilett0 is now known as Stiletto
13:46 🔗 HCross just change the config
13:46 🔗 HCross so the vars
13:47 🔗 bentpins Ansible plus script sounds ideal, but I'm not on ansible and I gather it's a paid service?
13:47 🔗 HCross no it isnt
13:50 🔗 bentpins Ah great, redhat's site doesn't give that impression with "pricing" and "trials" and such. The github is more telling. I'll have a go, I'm keen to learn
13:50 🔗 bentpins cheers
13:53 🔗 WinterFox has quit IRC (Remote host closed the connection)
13:58 🔗 philpem has joined #archiveteam
14:14 🔗 wvdp has joined #archiveteam
14:15 🔗 davidar has joined #archiveteam
14:35 🔗 nwf has joined #archiveteam
14:42 🔗 atomotic has joined #archiveteam
14:58 🔗 Tomcat_ has quit IRC (Remote host closed the connection)
15:17 🔗 redlob has quit IRC (Read error: Operation timed out)
15:18 🔗 redlob has joined #archiveteam
15:19 🔗 scyther has joined #archiveteam
15:29 🔗 phuz has joined #archiveteam
15:30 🔗 phuzion has quit IRC (Read error: Connection reset by peer)
15:32 🔗 dserodio has quit IRC (Excess Flood)
15:32 🔗 RichardG has quit IRC (Ping timeout: 499 seconds)
15:34 🔗 dserodio has joined #archiveteam
16:04 🔗 RichardG has joined #archiveteam
16:18 🔗 JesseW has joined #archiveteam
16:29 🔗 RichardG has quit IRC (Ping timeout: 250 seconds)
16:30 🔗 phuz is now known as phuzion
16:33 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
16:48 🔗 JesseW has quit IRC (Ping timeout: 370 seconds)
16:51 🔗 Tomcat_ has joined #archiveteam
16:54 🔗 wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client)
16:54 🔗 wvdp has joined #archiveteam
17:04 🔗 JetBalsa has joined #archiveteam
17:18 🔗 scyther has quit IRC (Quit: Leaving)
18:25 🔗 VADemon has quit IRC (Quit: left4dead)
18:26 🔗 VADemon has joined #archiveteam
18:28 🔗 RichardG has joined #archiveteam
18:29 🔗 scyther has joined #archiveteam
18:29 🔗 scyther has quit IRC (Connection closed)
18:36 🔗 Stiletto has quit IRC ()
18:36 🔗 Stiletto has joined #archiveteam
18:37 🔗 RichardG has quit IRC (Ping timeout: 499 seconds)
18:43 🔗 RichardG has joined #archiveteam
18:47 🔗 bsmith093 has quit IRC (Ping timeout: 260 seconds)
19:12 🔗 Emcy has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 Deewiant has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 espes__ has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 ats has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 chfoo has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 ersi has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 goekesmi has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 koon has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 Fletcher_ has quit IRC (hub.se efnet.portlane.se)
19:12 🔗 espes___ has joined #archiveteam
19:13 🔗 ring has quit IRC (Read error: Operation timed out)
19:13 🔗 tomwsmf-a has joined #archiveteam
19:15 🔗 ring has joined #archiveteam
19:16 🔗 chfoo0 has joined #archiveteam
19:22 🔗 Fletcher_ has joined #archiveteam
19:22 🔗 ats has joined #archiveteam
19:22 🔗 koon has joined #archiveteam
19:22 🔗 Deewiant has joined #archiveteam
19:24 🔗 bsmith093 has joined #archiveteam
19:26 🔗 goekesmi_ has joined #archiveteam
19:39 🔗 ersi has joined #archiveteam
19:39 🔗 swebb sets mode: +o ersi
19:40 🔗 bwn has quit IRC (Read error: Operation timed out)
19:51 🔗 VADemon has quit IRC (Quit: left4dead)
19:58 🔗 bwn has joined #archiveteam
20:14 🔗 bwn_ has joined #archiveteam
20:23 🔗 bwn has quit IRC (Read error: Operation timed out)
20:44 🔗 Honno has quit IRC (Ping timeout: 492 seconds)
20:56 🔗 Tomcat_ has quit IRC (Remote host closed the connection)
20:59 🔗 ats has quit IRC (Ping timeout: 250 seconds)
21:00 🔗 BlueMaxim has joined #archiveteam
21:00 🔗 ats has joined #archiveteam
21:00 🔗 ersi has quit IRC (Ping timeout: 250 seconds)
21:01 🔗 Deewiant_ has joined #archiveteam
21:08 🔗 Fletcher_ has quit IRC (hub.se efnet.portlane.se)
21:08 🔗 koon has quit IRC (hub.se efnet.portlane.se)
21:08 🔗 Deewiant has quit IRC (hub.se efnet.portlane.se)
21:15 🔗 vtyl has quit IRC (Ping timeout: 260 seconds)
21:17 🔗 BlueMaxim has quit IRC (Read error: Operation timed out)
21:18 🔗 vtyl has joined #archiveteam
21:19 🔗 bwn_ is now known as bwn
21:24 🔗 chfoo0 is now known as chfoo
21:33 🔗 bwn has quit IRC (Read error: Operation timed out)
21:34 🔗 bwn has joined #archiveteam
21:46 🔗 WinterFox has joined #archiveteam
21:49 🔗 Deewiant_ has quit IRC (hub.se irc.du.se)
21:49 🔗 Sk1d has quit IRC (hub.se irc.du.se)
21:56 🔗 bwn has quit IRC (Leaving)
21:57 🔗 bwn has joined #archiveteam
22:18 🔗 db48x` is now known as db48x
22:24 🔗 schbirid has quit IRC (Quit: Leaving)
22:24 🔗 Boppen has joined #archiveteam
22:29 🔗 wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client)
22:37 🔗 aliz has quit IRC (Remote host closed the connection)
23:09 🔗 midas1 is now known as midas
23:24 🔗 tubenerd1 has joined #archiveteam
23:24 🔗 Microguru has quit IRC (Read error: Connection reset by peer)
23:25 🔗 tubenerd1 archive team, you available?
23:25 🔗 tubenerd1 has quit IRC (Client Quit)
23:26 🔗 nillary16 has joined #archiveteam
23:27 🔗 nillary16 has quit IRC (Client Quit)
23:36 🔗 tomwsmf-a has quit IRC (Read error: Operation timed out)
23:52 🔗 dashcloud sure- what's up?
23:55 🔗 JesseW has joined #archiveteam
23:59 🔗 Frogging I think he left

irclogger-viewer