#archiveteam-bs 2018-04-12,Thu

↑back Search

Time Nickname Message
00:33 🔗 godane has quit IRC (Quit: Leaving.)
00:33 🔗 godane has joined #archiveteam-bs
00:43 🔗 Fusl has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 nyaomi has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 underscor has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 SmileyG has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 MrRadar2 has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 tsr has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 BnAboyZ has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 Sue has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 w0rp has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 Spydar007 has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 BnARobin has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 bsmith093 has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 Jonimus has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 hook54321 has quit IRC (hub.efnet.us irc.efnet.nl)
00:43 🔗 Fusl has joined #archiveteam-bs
00:43 🔗 nyaomi has joined #archiveteam-bs
00:43 🔗 underscor has joined #archiveteam-bs
00:43 🔗 SmileyG has joined #archiveteam-bs
00:43 🔗 MrRadar2 has joined #archiveteam-bs
00:43 🔗 tsr has joined #archiveteam-bs
00:43 🔗 BnAboyZ has joined #archiveteam-bs
00:43 🔗 Sue has joined #archiveteam-bs
00:43 🔗 w0rp has joined #archiveteam-bs
00:43 🔗 Spydar007 has joined #archiveteam-bs
00:43 🔗 BnARobin has joined #archiveteam-bs
00:43 🔗 bsmith093 has joined #archiveteam-bs
00:43 🔗 Jonimus has joined #archiveteam-bs
00:43 🔗 hook54321 has joined #archiveteam-bs
00:43 🔗 irc.efnet.nl sets mode: +oo MrRadar2 hook54321
00:56 🔗 dashcloud has quit IRC (Read error: Operation timed out)
00:59 🔗 Valentine has quit IRC (Read error: Operation timed out)
01:00 🔗 m007a83_ is now known as m007a83
01:02 🔗 Valentine has joined #archiveteam-bs
01:22 🔗 MrRadar_ is now known as MrRadar
01:25 🔗 dashcloud has joined #archiveteam-bs
01:27 🔗 MrRadar has quit IRC (Read error: Operation timed out)
01:29 🔗 godane has quit IRC (Read error: Operation timed out)
01:30 🔗 godane has joined #archiveteam-bs
01:48 🔗 dashcloud has quit IRC (Read error: Operation timed out)
01:49 🔗 Mateon1 has quit IRC (Read error: Operation timed out)
01:49 🔗 Mateon1 has joined #archiveteam-bs
01:54 🔗 dashcloud has joined #archiveteam-bs
01:56 🔗 MrRadar has joined #archiveteam-bs
01:58 🔗 MrRadar2 sets mode: +o MrRadar
02:00 🔗 dashcloud has quit IRC (Read error: Operation timed out)
02:02 🔗 dashcloud has joined #archiveteam-bs
02:04 🔗 atrocity has quit IRC (Read error: Connection reset by peer)
02:04 🔗 atrocity has joined #archiveteam-bs
03:49 🔗 qw3rty116 has joined #archiveteam-bs
03:55 🔗 qw3rty115 has quit IRC (Read error: Operation timed out)
04:04 🔗 dashcloud has quit IRC (Read error: Operation timed out)
04:07 🔗 dashcloud has joined #archiveteam-bs
04:08 🔗 odemg has quit IRC (Read error: Operation timed out)
04:09 🔗 beardicus has quit IRC (Read error: Operation timed out)
04:14 🔗 odemg has joined #archiveteam-bs
04:25 🔗 dashcloud has quit IRC (Read error: Operation timed out)
04:30 🔗 dashcloud has joined #archiveteam-bs
05:12 🔗 beardicus has joined #archiveteam-bs
05:39 🔗 dashcloud has quit IRC (Read error: Connection reset by peer)
05:40 🔗 dashcloud has joined #archiveteam-bs
05:51 🔗 Pixi has quit IRC (Quit: Pixi)
05:51 🔗 Pixi has joined #archiveteam-bs
06:30 🔗 wp494_ has joined #archiveteam-bs
06:35 🔗 wp494 has quit IRC (Read error: Operation timed out)
06:45 🔗 MrRadar_ has joined #archiveteam-bs
06:46 🔗 dashcloud has quit IRC (Read error: Operation timed out)
06:46 🔗 MrRadar has quit IRC (Read error: Operation timed out)
06:51 🔗 dashcloud has joined #archiveteam-bs
07:00 🔗 dashcloud has quit IRC (Read error: Operation timed out)
07:06 🔗 dashcloud has joined #archiveteam-bs
07:06 🔗 schbirid has joined #archiveteam-bs
07:36 🔗 dashcloud has quit IRC (Read error: Operation timed out)
07:40 🔗 schbirid has quit IRC (Quit: Leaving)
07:40 🔗 dashcloud has joined #archiveteam-bs
08:24 🔗 JAA My GIGA grab is still going strong. 582k thread IDs in a bit over 13 hours. There were some connection errors for a while, but no problems otherwise.
08:46 🔗 BlueMax has quit IRC (Leaving)
09:02 🔗 Mayonaise has quit IRC (Read error: Operation timed out)
09:02 🔗 m007a83_ has joined #archiveteam-bs
09:03 🔗 REiN^ has quit IRC (Read error: Operation timed out)
09:03 🔗 Mayonaise has joined #archiveteam-bs
09:03 🔗 rektide has quit IRC (Read error: Operation timed out)
09:03 🔗 twigfoot has quit IRC (Write error: Broken pipe)
09:03 🔗 squires has quit IRC (Write error: Broken pipe)
09:03 🔗 bmcginty has quit IRC (Read error: Operation timed out)
09:03 🔗 beardicus has quit IRC (Read error: Operation timed out)
09:03 🔗 Odd0002 has quit IRC (Read error: Operation timed out)
09:03 🔗 pikhq has quit IRC (Read error: Operation timed out)
09:03 🔗 FireFly has quit IRC (Read error: Operation timed out)
09:04 🔗 bwn has quit IRC (Read error: Operation timed out)
09:04 🔗 pikhq has joined #archiveteam-bs
09:04 🔗 bmcginty has joined #archiveteam-bs
09:04 🔗 twigfoot has joined #archiveteam-bs
09:04 🔗 rektide has joined #archiveteam-bs
09:04 🔗 Mateon1 has quit IRC (Read error: Operation timed out)
09:04 🔗 Mateon1 has joined #archiveteam-bs
09:04 🔗 C4K3 has quit IRC (Read error: Operation timed out)
09:05 🔗 sep332 has quit IRC (Read error: Operation timed out)
09:05 🔗 PotcFdk has quit IRC (Read error: Operation timed out)
09:06 🔗 unlobito has quit IRC (Read error: Operation timed out)
09:06 🔗 unlobito has joined #archiveteam-bs
09:06 🔗 Odd0002 has joined #archiveteam-bs
09:06 🔗 m007a83 has quit IRC (Read error: Operation timed out)
09:06 🔗 Dimtree has quit IRC (Read error: Operation timed out)
09:07 🔗 qw3rty116 has quit IRC (Read error: Operation timed out)
09:10 🔗 MrRadar has joined #archiveteam-bs
09:11 🔗 MrRadar_ has quit IRC (Read error: Operation timed out)
09:36 🔗 bwn has joined #archiveteam-bs
10:02 🔗 Dimtree has joined #archiveteam-bs
10:39 🔗 bad_faith has joined #archiveteam-bs
10:41 🔗 bad_faith Hi I'd like to BS about WARC 1.1 and Wayback 2.0 if/when anyone who has feelings about these things is available
10:43 🔗 bad_faith since archive.org generally does not make their full (w)arcs publicly accessible, discovery and access are (for now) limited to the CDX/availability APIs and various played-back "Mementos" derived from these warcs
10:45 🔗 bad_faith Still, enough information about the original capture event can be inferred from CDX records and headers provided by the playback/wayback server. I'd like to be able to analyze these records with the same tools being developed for "first-hand" warc files e.g. Archives Unleshed/Archivespark.
10:46 🔗 Jon one of these days I'm going to have to think about backing up ninlive.com, and that's a daunting prospect;
10:46 🔗 bad_faith It's also plausible to me that down the road - maybe way way down the road - the original warc files in archive.org's crawl collections will become public
10:48 🔗 bad_faith "big data" analysis of web archives is where it seems like this whole niche is heading. is anyone trying to do that without any kind of privileged access like I am?
10:57 🔗 bad_faith I find the technical aspects of this more frustrating than interesting and my real hope is I can just conform to whatever solution someone else has come up with to this problem because I would like to stop thinking about it and forget all of it.
11:50 🔗 BlueMax has joined #archiveteam-bs
12:02 🔗 qw3rty116 has joined #archiveteam-bs
12:02 🔗 beardicus has joined #archiveteam-bs
12:03 🔗 REiN^ has joined #archiveteam-bs
12:06 🔗 C4K3 has joined #archiveteam-bs
12:08 🔗 sep332 has joined #archiveteam-bs
12:16 🔗 FireFly has joined #archiveteam-bs
12:19 🔗 PotcFdk has joined #archiveteam-bs
12:47 🔗 zhongfu has quit IRC (Read error: Connection reset by peer)
12:47 🔗 zhongfu has joined #archiveteam-bs
12:50 🔗 zino has quit IRC (Remote host closed the connection)
13:07 🔗 BlueMax has quit IRC (Quit: Leaving)
15:15 🔗 Atom has quit IRC (Read error: Operation timed out)
16:02 🔗 lindalap has quit IRC (Quit: Restarting for IRC ident)
16:02 🔗 lindalap has joined #archiveteam-bs
16:17 🔗 schbirid has joined #archiveteam-bs
17:21 🔗 Darkstar has quit IRC (Ping timeout: 246 seconds)
17:23 🔗 Darkstar has joined #archiveteam-bs
17:29 🔗 Atom has joined #archiveteam-bs
17:31 🔗 keith20 has joined #archiveteam-bs
17:53 🔗 JAA Almost 1 million thread IDs scanned on GIGA in less than 24 hours. :-) So it should finish with the initial URLs sometime this night and start with the other pages of the existing threads, avatars, etc.
17:53 🔗 JAA And only 14.3 GiB of WARCs so far.
18:04 🔗 schbirid <3
18:49 🔗 HCross Who was interested in PureVolume?
19:17 🔗 VerifiedJ has joined #archiveteam-bs
19:39 🔗 arkiver HCross: meeeee
19:40 🔗 HCross arkiver: got 120GB of it so far, in 1.3mil requests, another 3.1mil requests to go
19:40 🔗 arkiver WARCs?
19:40 🔗 HCross yep
19:40 🔗 arkiver awesome!
19:41 🔗 arkiver so I've been working on this new decentralized crawler
19:41 🔗 arkiver I'd like to test it soon
19:41 🔗 arkiver who would like to help out with putting it on some servers and trying it out :)
19:41 🔗 arkiver ?
19:42 🔗 Sanqui arkiver: do you intend for it to replace archivebot?
19:43 🔗 arkiver not at the moment, but who knows what might happen
19:43 🔗 arkiver I'd like to start using it for large websites that we cannot really split up into parts
19:44 🔗 HCross arkiver: its not the quickest crawl at the moment, because im running a couple of others that are in the millions of URLs
19:44 🔗 arkiver and for possibly others project later as it becomes more stable and the network larger
19:45 🔗 noirscape has joined #archiveteam-bs
19:47 🔗 Kaz arkiver: happy to throw some power at it
19:47 🔗 arkiver awesome, thanks Kaz
19:48 🔗 arkiver Still working on some parts of it, but it should be ready for some testing soon on a larger scale than I have locally been testing it
20:19 🔗 lindalap_ has joined #archiveteam-bs
20:19 🔗 lindalap has quit IRC (Write error: Connection reset by peer)
20:20 🔗 lindalap_ is now known as WubTheCap
20:20 🔗 WubTheCap is now known as lindalap
20:26 🔗 keith20 has quit IRC (byeee)
20:46 🔗 jschwart has joined #archiveteam-bs
20:52 🔗 godane SketchCow: any news from Mank?
21:08 🔗 noirscape has quit IRC (Ping timeout: 268 seconds)
21:14 🔗 noirscape has joined #archiveteam-bs
21:14 🔗 dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
21:16 🔗 dashcloud has joined #archiveteam-bs
21:23 🔗 jschwart has quit IRC (Quit: Konversation terminated!)
21:50 🔗 schbirid has quit IRC (Quit: Leaving)
22:38 🔗 BlueMax has joined #archiveteam-bs
22:43 🔗 wp494_ is now known as wp494
22:51 🔗 VerifiedJ has left
23:52 🔗 Mateon1 has quit IRC (Read error: Operation timed out)
23:53 🔗 Mateon1 has joined #archiveteam-bs

irclogger-viewer