| Time |
Nickname |
Message |
|
00:01
🔗
|
balrog |
nico_: I fed those all to archivebot already |
|
00:01
🔗
|
balrog |
(pretty much right when the news broke) |
|
00:01
🔗
|
balrog |
if you want to do it anyway feel free |
|
00:02
🔗
|
odie5533_ |
What news? |
|
00:02
🔗
|
nico_ |
his dead |
|
00:03
🔗
|
odie5533_ |
oh |
|
00:03
🔗
|
nico_ |
(french) https://linuxfr.org/news/deces-de-cedric-blancher-chercheur-en-securite-informatique |
|
00:04
🔗
|
nico_ |
(english) http://www.theregister.co.uk/2013/11/12/cdric_sid_blancher_dead_at_37/ |
|
00:04
🔗
|
nico_ |
if you want more, we need to go to -bs |
|
05:46
🔗
|
godane |
SketchCow: i found webstock 2013 videos was released |
|
05:47
🔗
|
godane |
going to make a collection for them |
|
07:48
🔗
|
Nemo_bis |
13.15 <@Nemo_bis> where is odie5533 when one needs him :) https://code.google.com/p/wikiteam/issues/detail?id=78 |
|
07:49
🔗
|
Nemo_bis |
uh he's not here, stupid page up :) |
|
07:52
🔗
|
Turnip |
Page-up has been broken on my laptop for a while, irssi has been stressful |
|
08:12
🔗
|
odie5533_ |
Nemo_bis: hmm? |
|
08:13
🔗
|
odie5533_ |
I didn't write wikiteam scripts. |
|
08:14
🔗
|
odie5533_ |
Nemo_bis: I think the wikiteam script works by using urllib to get a list of urls, then wget for the actual download. If I were writing it, I'd write the url grabbing part as a Scrapy project that outputs urls. |
|
08:15
🔗
|
odie5533_ |
Nemo_bis: well, it's hitting a redirect loop. Can you give me a url that this occurs on? |
|
08:16
🔗
|
odie5533_ |
And in any case, the grabber should catch the HTTPError and just continue grabbing the other urls. |
|
08:18
🔗
|
Nemo_bis |
odie5533_: I know. Some days ago you asked me to provide URLs where the problems happen, here you go. :) |
|
08:18
🔗
|
Nemo_bis |
They are in the bug report, just add some dots to the domain names in dirs |
|
08:18
🔗
|
odie5533_ |
eh... can you give me a link? |
|
08:19
🔗
|
odie5533_ |
I'm really not sure what you mean since that won't give me a url to one of the images which seem to be having the problem |
|
08:20
🔗
|
odie5533_ |
None of the domains seem to exist. e.g. http://amzwikkii.com/ |
|
08:21
🔗
|
odie5533_ |
Nemo_bis: let's talk in #wikiteam |
|
08:44
🔗
|
godane |
Question about vimeo |
|
08:45
🔗
|
godane |
i'm looking at vmeo webstock archive and you can't download the original video file even though there is a link |
|
08:47
🔗
|
godane |
whats funny is if your over the link you get this message: "This source file has been lovingly stored in our archive." |
|
08:47
🔗
|
godane |
but you can freaking download it |
|
08:47
🔗
|
godane |
*can't |
|
08:57
🔗
|
godane |
i also found out that more videos from webstock 2011 was release more recently |
|
08:57
🔗
|
Nemo_bis |
maybe they moved them all to tapes in order to save money :D |
|
09:02
🔗
|
godane |
maybe but its very weird |
|
09:03
🔗
|
godane |
cause some videos in that area still have the original links working just fine |
|
09:04
🔗
|
godane |
anyways d-addicts.com wiki dump is done downloading |
|
09:04
🔗
|
godane |
making a 7z file of it |
|
09:12
🔗
|
Nemo_bis |
still uploading as 50 KB/s average to s3... |
|
09:16
🔗
|
odie5533_ |
That seems really low |
|
10:36
🔗
|
SketchCow |
I'm really pounding s3. |
|
11:45
🔗
|
Nemo_bis |
SketchCow: and derivers too now? :) |
|
11:46
🔗
|
Nemo_bis |
2,391,921,500 KB so far, I hope we'll have some slice of s3 available for us too soon :P |
|
13:09
🔗
|
joepie92 |
AAAAAAND WE'RE OFF: http://tracker.archiveteam.org/hyves/ |
|
13:49
🔗
|
SketchCow |
Is FOS getting this fun? |
|
14:16
🔗
|
SmileyG |
SketchCow: I think so |
|
14:16
🔗
|
SmileyG |
S[h]O[r]T has offered space too |
|
15:02
🔗
|
joepie92 |
SketchCow: I'm not sure if FOS has been added as a target yet |
|
15:02
🔗
|
joepie92 |
the initial target was icefortress, awaiting FOS space to be freed |
|
15:12
🔗
|
nico_ |
GLaDOS: VERSION = "20131116.02_" + subprocess.check_output(["git","rev-parse", "HEAD"]).strip()[:7] |
|
15:12
🔗
|
GLaDOS |
nico_: I saw it (i'm also twrist) |
|
15:12
🔗
|
nico_ |
(need a import subprocess before) |
|
18:14
🔗
|
w0rp |
http://bpaste.net/show/gWTMl6R6j3bdSgAuFHZj/ I used this for ripping a table from Wikipedia as CSV. Maybe someone else here might find this useful. |
|
18:15
🔗
|
w0rp |
It doesn't cope with there being two tables matching the selector on the table, but a minor set of modifications could make it do that. |
|
18:16
🔗
|
w0rp |
*matching the selector on the page |
|
18:55
🔗
|
godane |
uploaded: https://archive.org/details/wikid_addictscom_static-20131115-wikidump |
|
22:10
🔗
|
odie5533_ |
hmpf, people disabling the ability to submit Issues to stuff on github... |
|
22:23
🔗
|
odie5533_ |
Does anyone know what a CDX warcinfo/request entry is supposed to look like when the filename has a space in it? |
|
22:27
🔗
|
odie5533 |
The Python program CDX-Writer formats the massaged url as 'warcinfo:/output file.warc.gz/version'. Should there really be a space in the middle of an otherwise space-separated file? None of the test cases for CDX-Writer have spaces in the file names, so perhaps it was overlooked. |
|
22:31
🔗
|
odie5533 |
And the author disabled the Issues section on Github and doesn't list an email, so I can't even contact them. |
|
22:37
🔗
|
xmc |
you can usually get an email by cloning the repo and looking at their commits |
|
22:43
🔗
|
SketchCow |
Archive Team Bot GO! The Third: https://archive.org/details/archiveteam_archivebot_go_003\ |
|
22:46
🔗
|
ivan` |
SketchCow: did the plugins.jetbrains.com warc get nuked or are you specially handling it? |
|
22:49
🔗
|
ivan` |
it's from around Oct 23 |
|
22:51
🔗
|
ivan` |
it ran into the 40GB limit so maybe something went wrong with the rsync |