#archiveteam 2017-12-24,Sun

↑back Search

Time Nickname Message
00:02 🔗 svchost03 has joined #archiveteam
00:03 🔗 svchost01 sets mode: +o svchost03
00:04 🔗 svchost02 sets mode: +o svchost03
00:04 🔗 svchost01 sets mode: +o svchost03
00:06 🔗 svchost01 sets mode: +o svchost03
00:07 🔗 svchost01 sets mode: +o svchost03
00:08 🔗 svchost01 sets mode: +o svchost03
00:10 🔗 svchost01 sets mode: +o svchost03
00:11 🔗 svchost01 sets mode: +o svchost03
00:12 🔗 svchost01 sets mode: +o svchost03
00:14 🔗 svchost01 sets mode: +o svchost03
00:15 🔗 svchost01 sets mode: +o svchost03
00:17 🔗 svchost01 sets mode: +o svchost03
00:18 🔗 svchost01 sets mode: +o svchost03
00:19 🔗 svchost01 sets mode: +o svchost03
00:21 🔗 svchost01 sets mode: +o svchost03
00:22 🔗 svchost01 sets mode: +o svchost03
00:23 🔗 svchost01 sets mode: +o svchost03
00:24 🔗 svchost01 sets mode: +o svchost03
00:25 🔗 svchost01 sets mode: +o svchost03
00:26 🔗 svchost01 sets mode: +o svchost03
00:27 🔗 svchost01 sets mode: +o svchost03
00:28 🔗 svchost01 sets mode: +o svchost03
00:29 🔗 svchost01 sets mode: +o svchost03
00:30 🔗 svchost01 sets mode: +o svchost03
00:31 🔗 svchost01 sets mode: +o svchost03
00:32 🔗 svchost01 sets mode: +o svchost03
00:33 🔗 svchost01 sets mode: +o svchost03
00:34 🔗 svchost01 sets mode: +o svchost03
00:35 🔗 svchost01 sets mode: +o svchost03
00:36 🔗 svchost01 sets mode: +o svchost03
00:38 🔗 svchost01 sets mode: +o svchost03
00:38 🔗 JAA sets mode: -o svchost01
00:41 🔗 chirlu has quit IRC (Read error: Operation timed out)
00:47 🔗 EdTheNerd has joined #archiveteam
00:50 🔗 chirlu has joined #archiveteam
00:53 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
00:55 🔗 kristian_ has quit IRC (Quit: Leaving)
01:48 🔗 EdTheNerd has joined #archiveteam
01:52 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
02:01 🔗 Svekla has joined #archiveteam
02:01 🔗 Burak has quit IRC (Read error: Connection reset by peer)
02:20 🔗 icedice has quit IRC (Ping timeout: 260 seconds)
02:39 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
02:42 🔗 svchost01 has quit IRC (Read error: Operation timed out)
02:43 🔗 svchost01 has joined #archiveteam
02:47 🔗 EdTheNerd has joined #archiveteam
02:56 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
03:08 🔗 robink has quit IRC (Ping timeout: 246 seconds)
03:20 🔗 Mateon1 has quit IRC (Read error: Operation timed out)
03:20 🔗 Mateon1 has joined #archiveteam
03:22 🔗 ZexaronS has quit IRC (Read error: Connection reset by peer)
03:22 🔗 ZexaronS has joined #archiveteam
03:40 🔗 JAA sets mode: +o svchost01
03:47 🔗 EdTheNerd has joined #archiveteam
03:52 🔗 robink has joined #archiveteam
04:00 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
04:20 🔗 sivoais_ is now known as sivoais
04:30 🔗 jacketcha has joined #archiveteam
04:47 🔗 EdTheNerd has joined #archiveteam
04:52 🔗 qw3rty119 has joined #archiveteam
04:56 🔗 qw3rty118 has quit IRC (Read error: Operation timed out)
04:58 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
05:38 🔗 zino_ has quit IRC (Ping timeout: 1212 seconds)
05:44 🔗 zino_ has joined #archiveteam
05:47 🔗 EdTheNerd has joined #archiveteam
05:56 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
06:05 🔗 BlueMaxim has quit IRC (Read error: Connection reset by peer)
06:15 🔗 db48x has joined #archiveteam
06:27 🔗 Vito` hm okay
06:27 🔗 Vito` so five days into the lytro cdn assets mirror, wget ran out of memory and was killed
06:28 🔗 Vito` will this always happen? should I prune the URL list down into chunks?
06:29 🔗 Vito` and when I restart this, how do I restart it so it skips all the files it downloaded already?
06:29 🔗 Vito` and, is there any risk the last WARC in use is corrupt in any way?
06:29 🔗 Vito` my launch command was: wget -x --warc-file=lfes-not-in-ia-4 --warc-cdx --warc-max-size=1G --wait=1 --random-wait -i ~/lfes-not-in-ia-4.txt
06:46 🔗 EdTheNerd has joined #archiveteam
06:52 🔗 Vito` well, warcat at least thinks the final WARC is fine
06:56 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
06:58 🔗 Vito` damnit
06:58 🔗 Vito` "WARC output does not work with --no-clobber, --no-clobber will be disabled."
06:58 🔗 Vito` it just blew away the first gig WARC
06:59 🔗 Vito` so I guess I need to just prune the URL list
07:10 🔗 kyounko has quit IRC (Read error: Operation timed out)
07:13 🔗 bwn has quit IRC (Ping timeout: 260 seconds)
07:22 🔗 bwn has joined #archiveteam
07:25 🔗 kimmer1 has joined #archiveteam
07:25 🔗 chirlu has quit IRC (Quit: Bye)
07:35 🔗 BlueMaxim has joined #archiveteam
07:44 🔗 jspiros has quit IRC (brb)
07:46 🔗 EdTheNerd has joined #archiveteam
07:50 🔗 jspiros has joined #archiveteam
07:54 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
08:47 🔗 EdTheNerd has joined #archiveteam
08:47 🔗 schbirid has joined #archiveteam
08:58 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
09:11 🔗 odemg_ has joined #archiveteam
09:13 🔗 odemg has quit IRC (Read error: Operation timed out)
09:14 🔗 kimmer12 has joined #archiveteam
09:20 🔗 kimmer1 has quit IRC (Read error: Operation timed out)
09:47 🔗 EdTheNerd has joined #archiveteam
10:02 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
10:30 🔗 odemg_ has quit IRC (Quit: Leaving)
10:36 🔗 kimmer1 has joined #archiveteam
10:37 🔗 schbirid has quit IRC (Quit: Leaving)
10:44 🔗 kimmer12 has quit IRC (Ping timeout: 633 seconds)
10:47 🔗 EdTheNerd has joined #archiveteam
10:59 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
11:08 🔗 kristian_ has joined #archiveteam
11:46 🔗 EdTheNerd has joined #archiveteam
11:52 🔗 BlueMaxim has quit IRC (Quit: Leaving)
11:54 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
11:55 🔗 Dimtree has quit IRC (Peace)
12:06 🔗 schbirid has joined #archiveteam
12:11 🔗 kristian_ has quit IRC (Quit: Leaving)
12:32 🔗 pizzaiolo has joined #archiveteam
12:47 🔗 EdTheNerd has joined #archiveteam
13:01 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
13:38 🔗 MMovie2 has joined #archiveteam
13:39 🔗 MMovie has quit IRC (Read error: Operation timed out)
13:47 🔗 EdTheNerd has joined #archiveteam
13:48 🔗 undedu has joined #archiveteam
13:55 🔗 undedu I see that less than 7 days remain before the website at http://und.edu/misc/ gets deleted, and I've only been able to download around 5 effective Gigabytes with WinHTTrack, because it downloads any related websites.
13:55 🔗 undedu There's more information here: http://und.edu/web-support/server-decommission.cfm
13:56 🔗 undedu But I find that all subdirectories pointed there might contain the exact same data.
13:59 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
14:00 🔗 JAA undedu: We grabbed that with ArchiveBot a month ago. Can you check whether we missed anything? This is our grab: https://web.archive.org/web/20171124030911/http://und.edu/misc/
14:00 🔗 zgrant has joined #archiveteam
14:01 🔗 JAA undedu: These are the pages we grabbed (recursively, with offsite links): https://pastebin.com/raw/R37AHGyV
14:07 🔗 JAA Well, we're missing some images at least: http://und.edu/misc/ald/ vs. https://web.archive.org/web/20171124031643/http://und.edu/misc/ald/ (Probably due to the website being semi-broken HTML; it uses a backslash in the image URL.)
14:12 🔗 JAA Some links require manual changes, e.g. /misc links to https://web.archive.org/web/20171124030911/http://und.edu/misc/screamingeagles and the links on that page don't work unless you add a trailing slash. But it seems that it captured almost everything.
14:16 🔗 undedu For example this: https://web.archive.org/web/20171124031715/http://und.edu/misc/zhang
14:16 🔗 undedu http://www.und.nodak.edu/instruct/zhang/
14:16 🔗 undedu I have around 5 Megabytes.
14:17 🔗 JAA Add a slash and it works: https://web.archive.org/web/20171124031715/http://und.edu/misc/zhang/
14:19 🔗 JAA Clicking the links won't work because it's JavaScript.
14:20 🔗 undedu I've tried to upload the remaining files of that directory here:
14:20 🔗 undedu https://archive.org/details/und_edu_0000
14:21 🔗 undedu I parsed manually some XML files looking for file name properties for images.
14:21 🔗 undedu Downloaded more than 140 images and a video.
14:22 🔗 undedu I should upload what I get in the end to complete the mirrors.
14:23 🔗 JAA Please do. I don't think it will be available on the Wayback Machine though.
14:24 🔗 undedu But I think that it would probably be better to go to the physical place of the disks and ask for a copy of the data. This is how critical information gathering to edit technical/programming books was done, like the Encyclopedia of Graphics File Formats, where authors went in person to each entity that created each file format to ask for official information.
14:25 🔗 JAA Such a copy would definitely be nice, yes. If they're willing to give you one, absolutely do upload that to IA.
14:26 🔗 JAA This is getting a bit lengthy. Let's move to #archiveteam-bs .
14:26 🔗 undedu Somebody who is near should go.
14:26 🔗 undedu Or someone who can actually go.
14:27 🔗 JAA changes topic to: Archive Team: We're not archive.org | https://archiveteam.org/ | Lengthy discussions in #archiveteam-bs | Offtopic in #archiveteam-ot | We know about Storify
14:29 🔗 JAA Well, you don't need to actually be at their place to acquire a copy thanks to the internet. They could even just directly upload a copy to IA if they wanted to.
14:30 🔗 JAA But it might be a bit late to ask them for it now due to the holidays.
14:30 🔗 JAA Anyway, #archiveteam-bs please.
14:31 🔗 JAA (Type "/join #archiveteam-bs" to go there.)
14:43 🔗 kimmer12 has joined #archiveteam
14:44 🔗 kimmer1 has quit IRC (Read error: Connection reset by peer)
14:45 🔗 kimmer1 has joined #archiveteam
14:45 🔗 kimmer12 has quit IRC (Read error: Connection reset by peer)
14:46 🔗 EdTheNerd has joined #archiveteam
14:47 🔗 kimmer12 has joined #archiveteam
14:48 🔗 kimmer1 has quit IRC (Read error: Operation timed out)
14:50 🔗 cloudfnl has quit IRC (Read error: Operation timed out)
14:51 🔗 MrDignity has quit IRC (Ping timeout: 250 seconds)
14:53 🔗 MrDignity has joined #archiveteam
14:53 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
14:54 🔗 MrDignity has quit IRC (Remote host closed the connection)
14:57 🔗 MrDignity has joined #archiveteam
14:59 🔗 kimmer1 has joined #archiveteam
15:02 🔗 will has quit IRC (Quit: Goodbye)
15:05 🔗 will has joined #archiveteam
15:06 🔗 Odd0002 has quit IRC (Ping timeout: 248 seconds)
15:07 🔗 cloudfnl has joined #archiveteam
15:09 🔗 kimmer12 has quit IRC (Read error: Operation timed out)
15:22 🔗 odemg has joined #archiveteam
15:24 🔗 MMovie2 has quit IRC (Read error: Operation timed out)
15:26 🔗 MMovie has joined #archiveteam
15:29 🔗 kimmer12 has joined #archiveteam
15:38 🔗 kimmer1 has quit IRC (Ping timeout: 633 seconds)
15:41 🔗 kimmer1 has joined #archiveteam
15:43 🔗 kimmer13 has joined #archiveteam
15:46 🔗 kimmer12 has quit IRC (Ping timeout: 633 seconds)
15:48 🔗 EdTheNerd has joined #archiveteam
15:50 🔗 kimmer1 has quit IRC (Read error: Operation timed out)
15:54 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
16:47 🔗 EdTheNerd has joined #archiveteam
16:57 🔗 EdTheNerd has quit IRC (Ping timeout: 506 seconds)
17:41 🔗 undedu has quit IRC (Ping timeout: 260 seconds)
17:47 🔗 EdTheNerd has joined #archiveteam
17:53 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
18:19 🔗 kimmer13 has quit IRC (Ping timeout: 633 seconds)
18:43 🔗 odemg_ has joined #archiveteam
18:43 🔗 odemg has quit IRC (Read error: Operation timed out)
18:48 🔗 EdTheNerd has joined #archiveteam
18:58 🔗 Odd0002 has joined #archiveteam
19:01 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
19:46 🔗 EdTheNerd has joined #archiveteam
19:54 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
20:13 🔗 MMovie has quit IRC (Read error: Connection reset by peer)
20:41 🔗 kimmer1 has joined #archiveteam
20:46 🔗 EdTheNerd has joined #archiveteam
20:59 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
21:04 🔗 kimmer12 has joined #archiveteam
21:08 🔗 kimmer1 has quit IRC (Ping timeout: 633 seconds)
21:41 🔗 kimmer1 has joined #archiveteam
21:47 🔗 EdTheNerd has joined #archiveteam
21:48 🔗 kimmer12 has quit IRC (Read error: Operation timed out)
21:52 🔗 EdTheNerd has quit IRC (Ping timeout: 260 seconds)
21:59 🔗 kimmer12 has joined #archiveteam
22:00 🔗 sep332 has joined #archiveteam
22:06 🔗 kimmer1 has quit IRC (Read error: Operation timed out)
22:08 🔗 sep332 has quit IRC (Read error: Operation timed out)
22:12 🔗 godane has quit IRC (Read error: Operation timed out)
22:20 🔗 godane has joined #archiveteam
22:21 🔗 svchost03 sets mode: +o godane
22:28 🔗 godane has quit IRC (Quit: Leaving.)
22:28 🔗 godane has joined #archiveteam
22:29 🔗 svchost03 sets mode: +o godane
22:46 🔗 EdTheNerd has joined #archiveteam
23:03 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)
23:05 🔗 BlueMaxim has joined #archiveteam
23:14 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
23:16 🔗 pizzaiolo has joined #archiveteam
23:19 🔗 jschwart has quit IRC (Quit: Konversation terminated!)
23:22 🔗 kimmer1 has joined #archiveteam
23:28 🔗 kimmer12 has quit IRC (Ping timeout: 633 seconds)
23:47 🔗 EdTheNerd has joined #archiveteam
23:54 🔗 MorbusIff has quit IRC (Read error: Operation timed out)
23:55 🔗 Morbus has joined #archiveteam
23:55 🔗 EdTheNerd has quit IRC (Read error: Operation timed out)

irclogger-viewer