#archiveteam 2016-12-15,Thu

↑back Search

Time Nickname Message
00:22 🔗 creature_ has joined #archiveteam
00:22 🔗 creature has quit IRC (Read error: Operation timed out)
00:31 🔗 russkick has joined #archiveteam
00:32 🔗 maxogden has joined #archiveteam
00:34 🔗 maxogden hey all, im starting to work through https://docs.google.com/spreadsheets/d/12-__RqTqQxuxHNOln3H5ciVztsDMJcZ2SVs1BrfqYCc/edit#gid=0 and try to produce manifests of files in the linked datasets, eventual goal being to download them but wanted to start by producing metadata
00:35 🔗 MrRadar maxogden: Welcome. We've got a dedicated channel for archiving federal data at #cheetoflee. Join us there
01:15 🔗 brayden_ has joined #archiveteam
01:15 🔗 swebb sets mode: +o brayden_
01:20 🔗 brayden has quit IRC (Ping timeout: 633 seconds)
01:35 🔗 PepsiMax has quit IRC (Remote host closed the connection)
01:36 🔗 coretx has quit IRC (Remote host closed the connection)
01:37 🔗 Somebody has joined #archiveteam
01:39 🔗 coretx has joined #archiveteam
01:42 🔗 K4k has quit IRC (Quit: WeeChat 1.6)
01:42 🔗 K4k has joined #archiveteam
01:44 🔗 K4k_ has joined #archiveteam
01:44 🔗 K4k_ has quit IRC (Client Quit)
01:50 🔗 DiscantX has joined #archiveteam
01:59 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
02:04 🔗 K4k has quit IRC (Quit: WeeChat 1.6)
02:08 🔗 i336 has quit IRC (Remote host closed the connection)
02:13 🔗 DiscantX has quit IRC (Read error: Operation timed out)
02:14 🔗 DFJustin has quit IRC (Remote host closed the connection)
02:15 🔗 Start has quit IRC (Remote host closed the connection)
02:16 🔗 K4k has joined #archiveteam
02:18 🔗 K4k has quit IRC (Client Quit)
02:19 🔗 Start has joined #archiveteam
02:19 🔗 DFJustin has joined #archiveteam
02:19 🔗 Start has quit IRC (Client Quit)
02:20 🔗 DFJustin has quit IRC (Remote host closed the connection)
02:22 🔗 K4k has joined #archiveteam
02:25 🔗 DFJustin has joined #archiveteam
02:32 🔗 Start has joined #archiveteam
02:59 🔗 ravetcofx has quit IRC (Read error: Operation timed out)
03:09 🔗 Sk1d has quit IRC (Ping timeout: 250 seconds)
03:15 🔗 ravetcofx has joined #archiveteam
03:16 🔗 Sk1d has joined #archiveteam
03:26 🔗 vOYtEC has quit IRC (Ping timeout: 245 seconds)
03:49 🔗 nwf has quit IRC (Read error: Operation timed out)
03:55 🔗 russkick has quit IRC ()
04:16 🔗 lukeman has quit IRC (Ping timeout: 260 seconds)
04:16 🔗 lukeman_ has joined #archiveteam
04:40 🔗 ravetcofx has quit IRC (Read error: Operation timed out)
04:49 🔗 ravetcofx has joined #archiveteam
04:56 🔗 Verifirs has joined #archiveteam
04:56 🔗 Verifirs A couple concerns and questions
04:57 🔗 Verifirs What are your thoughts on mirroring libgen?
04:57 🔗 Verifirs How large is libgen, at least in estimation?
05:01 🔗 Somebody Verifirs: unofficial rumor is that archive.org already has (but is not distributing) a copy of libgen.
05:02 🔗 Somebody ArchiveTeam, as an entity, doesn't generally host stuff itself (although various people associated with it may, although not generally stuff with such legal attention on it).
05:02 🔗 Somebody (and I don't speak for ArchiveTeam, of course)
05:03 🔗 Verifirs I'm guessing it's not that large
05:03 🔗 Verifirs Maybe 25 terabytes
05:05 🔗 Somebody IDK.
05:06 🔗 maxogden scihub is 160tb currently
05:07 🔗 Verifirs Hmm
05:07 🔗 Verifirs https://sites.google.com/site/themetalibrary/library-genesis <--- that source says 100 tb
05:16 🔗 Frogging has quit IRC (El Psy Kongroo!)
05:24 🔗 Frogging has joined #archiveteam
05:32 🔗 Sk1d has quit IRC (Ping timeout: 194 seconds)
05:38 🔗 Sk1d has joined #archiveteam
05:47 🔗 SketchCow This is quite a discussion.
05:49 🔗 WinterFox has joined #archiveteam
05:50 🔗 godane someone should grab this folder: ftp://nlmpubs.nlm.nih.gov/nlmdata/pir/
05:50 🔗 godane i think i can grab everything else
05:50 🔗 godane my problem with that folder is its over 9gb
05:53 🔗 godane anyways if someone else wants to do ftp://nimpubs.nlm.nih.gov go for it
05:58 🔗 maelstrom has quit IRC (Quit: Leaving)
06:06 🔗 ScariLD has quit IRC (Ping timeout: 268 seconds)
06:11 🔗 BlueMaxim has quit IRC (Read error: Operation timed out)
06:26 🔗 Verifirs has quit IRC (Ping timeout: 268 seconds)
06:35 🔗 Somebody godane: it's just 2 files, right? dc.zip and dr.zip
06:35 🔗 Somebody I'm getting dc.zip now
06:35 🔗 Somebody I'll upload them both to FOS
06:38 🔗 Somebody I think this is what they are: https://pir.nlm.nih.gov/pilot/instructions.html
06:40 🔗 godane thanks
06:40 🔗 Somebody happy to do it
06:42 🔗 i336__ has quit IRC (Read error: Operation timed out)
06:42 🔗 Somebody interesting -- it looks like we made a copy of this back ftp server back in 2014: https://ia802506.us.archive.org/8/items/nlmpubs.nlm.nih.gov
06:43 🔗 godane thats good
06:48 🔗 Somebody it's going very very slowly :-(
06:50 🔗 ravetcofx has quit IRC (Read error: Operation timed out)
06:58 🔗 maxogden i put [A-Z]* into http://oceandata.sci.gsfc.nasa.gov/search/file_search.cgi and it 500'd after 5 minutes :(
06:59 🔗 maxogden now it thinks im running a query already and wont let me do another
07:02 🔗 Frogging web applications are hard
07:06 🔗 ravetcofx has joined #archiveteam
07:08 🔗 maxogden now on https://ofmpub.epa.gov/storpubl/dw_pages.querycriteria, "Number of Results Returned: 219,578,132. The number of Results that match your search criteria has exceeded the allowable Result Report size limit of 13,000,000. " :D
07:15 🔗 brayden_ is now known as brayden
07:21 🔗 dashcloud has quit IRC (Ping timeout: 250 seconds)
07:22 🔗 dashcloud has joined #archiveteam
07:24 🔗 BlueMaxim has joined #archiveteam
07:45 🔗 Honno has joined #archiveteam
08:20 🔗 ndiddy has quit IRC (Read error: Connection reset by peer)
09:05 🔗 kristian_ has joined #archiveteam
09:21 🔗 DiscantX has joined #archiveteam
09:34 🔗 Smiley has joined #archiveteam
09:35 🔗 RichardG_ has joined #archiveteam
09:39 🔗 kanzure_ has joined #archiveteam
09:39 🔗 Igloo_ has joined #archiveteam
09:39 🔗 K4k has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 RichardG has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 Yoshimura has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 godane has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 SmileyG has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 achip has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 superkuh has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 Igloo has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 kanzure has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 tpw_rules has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 aschmitz has quit IRC (hub.efnet.us irc.Prison.NET)
09:39 🔗 patrickod has quit IRC (hub.efnet.us irc.Prison.NET)
09:41 🔗 patricko- has joined #archiveteam
09:51 🔗 K4k_ has joined #archiveteam
09:52 🔗 Honno has quit IRC (Read error: Operation timed out)
09:56 🔗 godane has joined #archiveteam
10:50 🔗 DiscantX has quit IRC (Ping timeout: 492 seconds)
11:14 🔗 superkuh has joined #archiveteam
11:15 🔗 Igloo_ is now known as Igloo
11:39 🔗 WinterFox has quit IRC (Remote host closed the connection)
12:08 🔗 BlueMaxim has quit IRC (Read error: Operation timed out)
12:47 🔗 Honno has joined #archiveteam
13:58 🔗 RichardG_ is now known as RichardG
14:25 🔗 wp494 has quit IRC (Ping timeout: 506 seconds)
14:27 🔗 wp494 has joined #archiveteam
14:45 🔗 i336_ has joined #archiveteam
15:22 🔗 REiN^ has quit IRC (Max SendQ exceeded)
15:22 🔗 REiN^ has joined #archiveteam
15:31 🔗 Start has quit IRC (Quit: Disconnected.)
15:32 🔗 REiN^ has quit IRC (Max SendQ exceeded)
15:32 🔗 REiN^ has joined #archiveteam
15:32 🔗 i336_ has quit IRC (Read error: Operation timed out)
16:18 🔗 atomotic has joined #archiveteam
16:27 🔗 K4k_ has quit IRC (WeeChat 1.6)
16:28 🔗 K4k has joined #archiveteam
16:28 🔗 db48x has joined #archiveteam
16:41 🔗 godane has quit IRC (Quit: Leaving.)
16:41 🔗 godane has joined #archiveteam
17:05 🔗 Somebody has quit IRC (Ping timeout: 370 seconds)
17:07 🔗 kristian_ has quit IRC (Quit: Leaving)
17:08 🔗 Deewiant has quit IRC (Ping timeout: 194 seconds)
17:09 🔗 Fletcher has quit IRC (Ping timeout: 244 seconds)
17:16 🔗 Fletcher has joined #archiveteam
17:25 🔗 Deewiant has joined #archiveteam
17:51 🔗 Stiletto has quit IRC ()
17:59 🔗 nwf has joined #archiveteam
18:00 🔗 Start has joined #archiveteam
18:12 🔗 Start has quit IRC (Quit: Disconnected.)
18:21 🔗 nwf has quit IRC (Read error: Operation timed out)
18:24 🔗 nwf has joined #archiveteam
18:50 🔗 bRick5772 has joined #archiveteam
19:22 🔗 TJ has joined #archiveteam
19:30 🔗 TJ On archivesteam.org I haven't seen any mention/projects involving https://gamesrepublic.com/ website yet, which just announced in my email 2 days ago they are shutting down. So I'm here to just pop a quick heads-up about it incase nobody else has brought it up yet. It was primarily a website where you could sorta buy games from, I put it in the same category of websites that gog.com and humblebundle.com try to be. Here is a copy of the
19:30 🔗 xmc TJ: cut off at "here is a copy of the"
19:31 🔗 TJ Here is a copy of the email shutdown message I received: http://pastebin.com/piH0GSkN, bye!
19:32 🔗 jrwr has quit IRC (Remote host closed the connection)
19:39 🔗 TJ has left
19:44 🔗 Start has joined #archiveteam
20:24 🔗 bwn has quit IRC (Ping timeout: 244 seconds)
20:25 🔗 Stiletto has joined #archiveteam
20:35 🔗 bwn has joined #archiveteam
20:37 🔗 Stiletto has quit IRC (Ping timeout: 362 seconds)
20:38 🔗 SketchCow (I've gone ahead and set off a raft of Gamesrepublic.com archivebot jobs - all should be handled. Also grabbed the website of the main site/studio that owned it.)
20:38 🔗 Stiletto has joined #archiveteam
20:51 🔗 maelstrom has joined #archiveteam
20:53 🔗 tpw_rules has joined #archiveteam
20:55 🔗 jfranusic has joined #archiveteam
20:56 🔗 jfranusic has quit IRC (Client Quit)
20:58 🔗 Start has quit IRC (Quit: Disconnected.)
21:04 🔗 Start has joined #archiveteam
21:12 🔗 Stilett0 has joined #archiveteam
21:14 🔗 Stiletto has quit IRC (Ping timeout: 246 seconds)
21:20 🔗 antomati_ is now known as antomatic
21:24 🔗 Stilett0 has quit IRC (Read error: Connection reset by peer)
21:30 🔗 DiscantX has joined #archiveteam
21:39 🔗 bRick5772 has quit IRC (Quit: Leaving.)
21:49 🔗 Stiletto has joined #archiveteam
21:51 🔗 i336_ has joined #archiveteam
21:54 🔗 DiscantX has quit IRC (Ping timeout: 492 seconds)
22:05 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
22:13 🔗 nox_ has quit IRC (Ping timeout: 260 seconds)
22:15 🔗 Famicoma1 has quit IRC (Ping timeout: 260 seconds)
22:18 🔗 PepsiMax has joined #archiveteam
22:18 🔗 Aqua has joined #archiveteam
22:20 🔗 Aqua is now known as nox_
22:22 🔗 Stiletto has quit IRC (Remote host closed the connection)
22:22 🔗 Stiletto has joined #archiveteam
22:22 🔗 Vix has joined #archiveteam
22:24 🔗 nox_ has quit IRC (Read error: Connection reset by peer)
22:25 🔗 nox_ has joined #archiveteam
22:26 🔗 Vix How should I go about finding archived Twitch vods from a specific user, which are not listed under http://archive.fart.website/archiveteam/twitchtv-index/html/ ?
22:27 🔗 Vix I already tried searching the wayback archive for his channel name and videos, but i only end up with VOD ID, not the direct link to the FLV
22:27 🔗 Stiletto has quit IRC (Ping timeout: 260 seconds)
22:27 🔗 Vix I need the FLV, and can't think of any other way of finding it
22:28 🔗 Stiletto has joined #archiveteam
22:28 🔗 Vix the vods I'm looking for are from may and june 2012
22:28 🔗 Vix I'll idle, maybe somebody will know how to help me..
22:28 🔗 Start has quit IRC (Quit: Disconnected.)
22:35 🔗 sep332 has quit IRC (Quit: konversation out)
22:37 🔗 vantec Was the user in the items list? A lot was not saved due to the massive size of VODs with little or no views.
22:38 🔗 Vix It's Lirik
22:38 🔗 Vix he's very popular
22:39 🔗 Vix After twitch went from FLV to HLS some of his vods are missing or are corrupted
22:39 🔗 Vix I'm trying to retrieve them
22:40 🔗 Vix some vods are completely black, like this one: https://www.twitch.tv/lirik/v/38461920
22:41 🔗 Vix other vods (which were muted because of copyrighted music) are blacked out for 30 minutes in the part where song was playing
22:41 🔗 Vix and a lot of vods from 2012 is just missing
22:47 🔗 vantec I believe most channels did not have a complete grab done on them, only selected VODs with 100+ views. It was up to the members of those channels (and channel owner) to retrieve their content.
22:50 🔗 RichardG has quit IRC (Quit: Keyboard not found, press F1 to continue)
22:51 🔗 RichardG has joined #archiveteam
22:53 🔗 Vix hmm
22:54 🔗 Vix Is there a way to scan web archive for "*.justin.tv/archives/*" and happen to find those files anyways?
22:54 🔗 Vix it doesn't let me use multiple asterisks though :P
22:57 🔗 kristian_ has joined #archiveteam
23:11 🔗 BlueMaxim has joined #archiveteam
23:23 🔗 Smiley has quit IRC (Ping timeout: 250 seconds)
23:24 🔗 Famicoma1 has joined #archiveteam
23:27 🔗 Smiley has joined #archiveteam
23:31 🔗 wp494_ has joined #archiveteam
23:32 🔗 wp494 has quit IRC (Ping timeout: 245 seconds)
23:33 🔗 Vix I'm guessing there's no way unless I contact Twitch storage team
23:33 🔗 wp494_ is now known as wp494
23:51 🔗 Stiletto has quit IRC (Read error: Operation timed out)
23:52 🔗 Stiletto has joined #archiveteam

irclogger-viewer