Time |
Nickname |
Message |
00:00
π
|
|
dashcloud has quit IRC (Ping timeout: 265 seconds) |
00:06
π
|
|
Nertsy has quit IRC (Read error: Connection reset by peer) |
00:06
π
|
|
dashcloud has joined #archiveteam |
00:11
π
|
|
Nertsy has joined #archiveteam |
00:28
π
|
|
schbirid has quit IRC (Leaving) |
01:11
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
01:12
π
|
|
Start has joined #archiveteam |
01:15
π
|
|
dashcloud has joined #archiveteam |
01:39
π
|
|
primus104 has quit IRC (Leaving.) |
01:45
π
|
|
BiggieJo1 has joined #archiveteam |
01:48
π
|
|
BiggieJon has quit IRC (Read error: Operation timed out) |
01:49
π
|
|
Start has quit IRC (Ping timeout: 492 seconds) |
02:11
π
|
|
ruukasu has quit IRC (Quit: WeeChat 1.0.1) |
02:11
π
|
|
ruukasu has joined #archiveteam |
02:42
π
|
|
signius has quit IRC (Ping timeout: 480 seconds) |
02:51
π
|
|
signius has joined #archiveteam |
02:56
π
|
|
brayden has quit IRC (Ping timeout: 606 seconds) |
02:59
π
|
|
brayden has joined #archiveteam |
03:03
π
|
|
mistym has quit IRC (Remote host closed the connection) |
04:17
π
|
|
Nertsy has quit IRC (Read error: Connection reset by peer) |
04:20
π
|
|
Nertsy has joined #archiveteam |
05:01
π
|
|
aaaaaaaaa has quit IRC (Leaving) |
05:06
π
|
|
Swizzle has quit IRC (Quit: HydraIRC -> http://www.hydrairc.com <- Would you like to know more?) |
05:52
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
05:56
π
|
|
dashcloud has joined #archiveteam |
06:00
π
|
|
aschmitz has joined #archiveteam |
06:31
π
|
|
Nertsy has quit IRC (Remote host closed the connection) |
06:31
π
|
|
Nertsy has joined #archiveteam |
06:52
π
|
|
mistym has joined #archiveteam |
07:10
π
|
|
rejon has quit IRC (Read error: Operation timed out) |
07:28
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
07:32
π
|
|
dashcloud has joined #archiveteam |
07:37
π
|
|
primus104 has joined #archiveteam |
08:17
π
|
|
primus104 has quit IRC (Leaving.) |
08:32
π
|
|
Ymgve has joined #archiveteam |
08:51
π
|
|
indigo_ has quit IRC (Remote host closed the connection) |
08:55
π
|
|
brayden has quit IRC (Quit: Leaving) |
09:14
π
|
|
brayden has joined #archiveteam |
09:46
π
|
|
schbirid has joined #archiveteam |
10:24
π
|
|
mistym has quit IRC (Remote host closed the connection) |
10:24
π
|
|
Ymgve__ has joined #archiveteam |
10:32
π
|
|
Ymgve has quit IRC (Ping timeout: 512 seconds) |
10:35
π
|
|
primus104 has joined #archiveteam |
10:47
π
|
arkiver |
So I have my two week holiday! |
10:47
π
|
arkiver |
Will be working on the upcoming projects for the warrior |
10:48
π
|
arkiver |
SketchCow: can the halo project start again? |
10:49
π
|
|
primus104 has quit IRC (Leaving.) |
10:57
π
|
|
APerti has quit IRC (Ping timeout: 370 seconds) |
11:44
π
|
|
primus104 has joined #archiveteam |
12:10
π
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
13:54
π
|
|
Nertsy` has joined #archiveteam |
13:54
π
|
|
Nertsy has quit IRC (Read error: Connection reset by peer) |
14:02
π
|
|
Nemo_bis has quit IRC (Remote host closed the connection) |
14:08
π
|
|
hive-mind has quit IRC (Ping timeout: 272 seconds) |
14:15
π
|
|
hive-mind has joined #archiveteam |
14:19
π
|
|
Daloader_ has joined #archiveteam |
14:21
π
|
|
T31M has joined #archiveteam |
14:24
π
|
|
Nertsy` has quit IRC (Remote host closed the connection) |
14:25
π
|
|
Nertsy has joined #archiveteam |
14:25
π
|
|
chazchaz has quit IRC (Read error: Connection reset by peer) |
14:25
π
|
|
chazchaz has joined #archiveteam |
14:25
π
|
|
Laverne has quit IRC (Ping timeout: 369 seconds) |
14:26
π
|
|
Laverne has joined #archiveteam |
14:26
π
|
|
T31m_ has quit IRC (Read error: Operation timed out) |
14:31
π
|
|
chazchaz has quit IRC (Remote host closed the connection) |
14:32
π
|
|
T31M has quit IRC (Read error: Operation timed out) |
14:32
π
|
|
Daloader_ has quit IRC (Read error: Operation timed out) |
14:36
π
|
|
chazchaz has joined #archiveteam |
14:40
π
|
|
smither has joined #archiveteam |
14:40
π
|
smither |
hi there |
14:40
π
|
smither |
Iβve been trying to do a grab of cbc.ca/Q |
14:40
π
|
smither |
but something prevents wget from retriving more than one page |
14:41
π
|
smither |
Iβm using user-agent=βnot Googleβ but that ainβt tricking the machine |
14:41
π
|
|
Daloader_ has joined #archiveteam |
14:42
π
|
smither |
(for background, CBC is deleting some of its archives because one of their anchor turned out to be a rapist. But itβs problematic because theyβre erasing a lot of info for journalists) |
14:42
π
|
smither |
http://www.huffingtonpost.ca/2014/12/17/ghomeshi-q-archives_n_6340882.html |
14:46
π
|
|
Jonimus has quit IRC (Excess Flood) |
14:47
π
|
|
Jonimus has joined #archiveteam |
14:48
π
|
|
BiggieJo1 is now known as BiggieJ |
14:49
π
|
|
Jonimus has quit IRC (Excess Flood) |
14:50
π
|
|
Jonimus has joined #archiveteam |
14:51
π
|
|
eprillios has quit IRC (Ping timeout: 369 seconds) |
14:51
π
|
|
eprillios has joined #archiveteam |
14:52
π
|
godane |
smither: looks like the podcast rss archived very often at least |
14:52
π
|
godane |
https://web.archive.org/web/*/http://www.cbc.ca/podcasting/includes/qpodcast.xml |
14:52
π
|
smither |
so it should be fine ? |
14:53
π
|
godane |
no |
14:53
π
|
godane |
i'm grabbing the mp3s right now |
14:54
π
|
godane |
also from what i can tell 2011 mp3 urls don't work anymore |
14:58
π
|
smither |
any idea why my wget didnβt work ? |
14:58
π
|
smither |
I used wget -mc --no-parent --no-clobber --adjust-extension --user-agent="not Google" --convert-links --page-requisites cbc.ca/q |
15:02
π
|
|
T31m_ has joined #archiveteam |
15:04
π
|
|
ohhdemgir has quit IRC (Read error: Operation timed out) |
15:04
π
|
|
brayden has quit IRC (Read error: Operation timed out) |
15:05
π
|
|
Nertsy has quit IRC (Read error: Connection reset by peer) |
15:05
π
|
godane |
i tryed my own way and it will not mirror either |
15:05
π
|
godane |
wget --mirror cbc.ca/q -U "firefox" -e robots=off --warc-file=cbc-q --warc-cdx -E -o wget.log |
15:06
π
|
|
ohhdemgir has joined #archiveteam |
15:07
π
|
|
Nertsy has joined #archiveteam |
15:11
π
|
|
zenguy_pc has quit IRC (Excess Flood) |
15:12
π
|
smither |
so itβs not the robot? |
15:12
π
|
|
zenguy_pc has joined #archiveteam |
15:14
π
|
|
Daloader_ has quit IRC (Read error: Operation timed out) |
15:15
π
|
|
brayden has joined #archiveteam |
15:23
π
|
|
Nemo_bis has joined #archiveteam |
15:27
π
|
|
T31M has joined #archiveteam |
15:29
π
|
|
primus104 has quit IRC (Leaving.) |
15:31
π
|
|
Nertsy has quit IRC (Remote host closed the connection) |
15:31
π
|
|
Nertsy has joined #archiveteam |
15:34
π
|
|
T31m_ has quit IRC (Read error: Operation timed out) |
15:39
π
|
Fusl |
am i currently the only one mirroring wallbase? |
15:39
π
|
|
smither has quit IRC (smither) |
15:40
π
|
|
Nertsy has quit IRC (Remote host closed the connection) |
15:41
π
|
Fusl |
i expect the server to go down beginning 2015, can someone please be so kind and mirror it and put the mirror in the wallbase mirror list? |
15:41
π
|
|
Nertsy has joined #archiveteam |
15:48
π
|
|
Nertsy` has joined #archiveteam |
15:48
π
|
|
Nertsy has quit IRC (Read error: Connection reset by peer) |
15:56
π
|
|
goekesmi has quit IRC (Ping timeout: 369 seconds) |
16:04
π
|
|
goekesmi has joined #archiveteam |
16:05
π
|
Kenshin |
Fusl: has the site been taken down? officially propose to push to IA? |
16:05
π
|
|
aaaaaaaaa has joined #archiveteam |
16:07
π
|
|
Daloader_ has joined #archiveteam |
16:07
π
|
arkiver |
Fusl: yeah, sure |
16:08
π
|
Fusl |
the site has not been taken down |
16:08
π
|
Fusl |
but the host node where i'm hosting that entire thing on (it's about 1.3TB huge) will be cancelled |
16:08
π
|
Fusl |
because of the lack of money |
16:08
π
|
arkiver |
Fusl: you're talking about this right? http://archive_wallbase.cc.mirror.fuslvz.ws/ |
16:08
π
|
Fusl |
yes, but there is rsync on this mirror |
16:09
π
|
Fusl |
rsync://mirror.fuslvz.ws/archive_wallbase.cc/ |
16:10
π
|
|
goekesmi has quit IRC (Read error: Connection reset by peer) |
16:10
π
|
|
indigo_ has joined #archiveteam |
16:13
π
|
|
T31M has quit IRC (Read error: Operation timed out) |
16:14
π
|
|
goekesmi has joined #archiveteam |
16:27
π
|
Kenshin |
iomart? |
16:28
π
|
Kenshin |
arkiver: do you have anywhere i can dump the halo stuff i was holding while FOS was down? |
16:28
π
|
|
T31M has joined #archiveteam |
16:28
π
|
Kenshin |
i have qwiki with me too |
16:28
π
|
arkiver |
halo: https://archive.org/details/archiveteam_halo |
16:28
π
|
Kenshin |
if u can get those off me i'll have space for Fusl |
16:28
π
|
arkiver |
But SketchCow needs to give you acces to upload to there |
16:28
π
|
Kenshin |
arkiver: rsync target |
16:28
π
|
Kenshin |
i'm holding rsync data only |
16:29
π
|
arkiver |
ah ok |
16:29
π
|
arkiver |
we might be able to use FOS's rsync, but I'm not sure if we can already start uploading to that one |
16:29
π
|
Kenshin |
will need to ask SketchCow i guess |
16:29
π
|
Kenshin |
what about qwiki? |
16:29
π
|
arkiver |
If SketchCow thinks FOS is fine again, we can move your stuff to FOS |
16:29
π
|
Kenshin |
it's not a lot, about 700MB |
16:30
π
|
arkiver |
qwiki the same |
16:30
π
|
arkiver |
We'll have to wait for SketchCow, what he says |
16:30
π
|
Kenshin |
k. if he gives the go ahead then i'll have 1.5T for Fusl's stuff |
16:30
π
|
arkiver |
yes |
16:30
π
|
Kenshin |
alternatively, fusl push straight to IA if it's get approved |
16:31
π
|
arkiver |
how much of halo do you have? |
16:31
π
|
Kenshin |
since the website is pretty much dead |
16:31
π
|
Fusl |
Kenshin: if you explain how, i can do that :) |
16:31
π
|
Kenshin |
arkiver: 1.7T |
16:31
π
|
arkiver |
Kenshin: ok, that'd be fine |
16:31
π
|
Kenshin |
Fusl: no idea. i've always left uploading to yip |
16:32
π
|
Fusl |
hm |
16:32
π
|
Kenshin |
arkiver: what would be the best way to push 1.3t of website data to IA? |
16:34
π
|
arkiver |
Kenshin: megawarc it and upload it to the collection, with https://pypi.python.org/pypi/internetarchive or https://github.com/kngenie/ias3upload |
16:34
π
|
arkiver |
Fusl: I'd suggest using one of the above tools ^ to upload all the stuff to IA |
16:38
π
|
|
primus104 has joined #archiveteam |
16:38
π
|
|
T31m_ has joined #archiveteam |
16:39
π
|
Fusl |
the lastter one, what .csv files do i need exactly? |
16:39
π
|
Fusl |
or don't i need them? |
16:41
π
|
|
Daloader_ has quit IRC (Read error: Operation timed out) |
16:42
π
|
|
Daloader_ has joined #archiveteam |
16:44
π
|
|
w0rp has quit IRC (Ping timeout: 1221 seconds) |
16:45
π
|
|
w0rp has joined #archiveteam |
16:45
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
16:45
π
|
|
ruukasu has quit IRC (Quit: WeeChat 1.0.1) |
16:45
π
|
|
ruukasu has joined #archiveteam |
16:46
π
|
|
T31M has quit IRC (Read error: Operation timed out) |
16:48
π
|
arkiver |
Fusl: in the csv file you'll write the information for your item, like the identifier, title, description, tags, etc. https://github.com/kngenie/ias3upload/blob/master/metadata.csv |
16:48
π
|
|
dashcloud has joined #archiveteam |
16:50
π
|
|
T31m_ has quit IRC (Read error: Operation timed out) |
16:50
π
|
Fusl |
unfortunately too much work at the moment |
16:50
π
|
Fusl |
i have to move other stuff :/ |
16:54
π
|
|
primus104 has quit IRC (Leaving.) |
16:57
π
|
arkiver |
Fusl: I'm mirroring parts of it now |
16:57
π
|
arkiver |
will get folders with images up in IA |
16:57
π
|
Fusl |
arkiver: if you want, i can throw your ssh key in the server so you can put it up on IA directly from there...? |
16:58
π
|
arkiver |
nah, it's going fine this way |
16:58
π
|
Fusl |
k |
16:59
π
|
arkiver |
I'll put them in seperate items for each folder. So the images from http://archive_wallbase.cc.mirror.fuslvz.ws/siterip/images/0000000/ will have the collection wallbase.cc-rip-0000000 |
17:11
π
|
arkiver |
Fusl: test item: https://archive.org/details/test_wallbase.cc-rip-0000000 |
17:11
π
|
arkiver |
looks good? (still uploading, not derived yet) |
17:12
π
|
Fusl |
neat |
17:14
π
|
arkiver |
Fusl: what's the full size of everything in this directory? http://archive_wallbase.cc.mirror.fuslvz.ws/siterip/images/ |
17:14
π
|
arkiver |
if not too big I'll put everything in one item |
17:15
π
|
Fusl |
calculating ... |
17:15
π
|
Fusl |
1.2T |
17:50
π
|
|
rejon has joined #archiveteam |
18:06
π
|
|
mistym has joined #archiveteam |
18:13
π
|
|
bsmith093 has quit IRC (Read error: Operation timed out) |
18:15
π
|
|
db48x has quit IRC (Ping timeout: 258 seconds) |
18:27
π
|
|
bsmith093 has joined #archiveteam |
18:43
π
|
|
APerti has joined #archiveteam |
19:49
π
|
|
okeuday has joined #archiveteam |
20:03
π
|
SketchCow |
What |
20:03
π
|
SketchCow |
Kenshin: FOS, go ahead. |
20:03
π
|
SketchCow |
Sorry for lack of response, maniacs |
20:04
π
|
SketchCow |
/join #aside |
20:05
π
|
|
rejon has quit IRC (Read error: Operation timed out) |
20:17
π
|
SketchCow |
How did I miss #roon |
20:19
π
|
|
thechip_ has quit IRC (Read error: Operation timed out) |
20:22
π
|
SketchCow |
Anyway, I guess we're doing roon. I'm doing the groupings now. |
20:24
π
|
|
primus104 has joined #archiveteam |
20:31
π
|
|
fluff is now known as fluff_ |
20:34
π
|
godane |
I'm in #roon on irc.efnet.net and is no one there |
20:35
π
|
chfoo |
#rooined |
20:41
π
|
Kenshin |
SketchCow: do you have the rsync urls for qwiki and halo? |
20:47
π
|
SketchCow |
chfoo does |
20:48
π
|
|
ohhdemgir has quit IRC (Read error: Operation timed out) |
20:49
π
|
|
ohhdemgir has joined #archiveteam |
20:54
π
|
chfoo |
Kenshin: for fos i assume: rsync://fos.textfiles.com/chfoo/warrior/qwiki/:downloader/ & rsync://fos.textfiles.com/chfoo/warrior/halo/:downloader/ |
20:56
π
|
Kenshin |
cool thanks, much appreciated |
20:56
π
|
chfoo |
replace :downloader with a nickname |
20:56
π
|
|
mistym has quit IRC (Remote host closed the connection) |
21:11
π
|
|
BlueMaxim has joined #archiveteam |
21:29
π
|
|
wp494 has quit IRC () |
21:31
π
|
|
wp494 has joined #archiveteam |
21:37
π
|
|
bzc6p has joined #archiveteam |
21:44
π
|
bzc6p |
chfoo, arkiver: In a template I made on the wiki, I included "some additional information" I miss from the script documentations on GitHub. Could you please include those pieces of information when you next time create projects on GitHub? |
21:45
π
|
bzc6p |
(I mean the missing ones, about the concurrency, stopping the script, and what to do when outdated β even rephrased if necessary.) I think these are important for newcomers, but if they were on github, I could remove them from the wiki. Thank you. |
21:46
π
|
chfoo |
the template we're using is located at https://github.com/ArchiveTeam/standalone-readme-template |
21:48
π
|
bzc6p |
Could you please expand that then? I don't want to create a github account just for that. |
21:49
π
|
bzc6p |
(Of course if you too find it a good idea.) |
21:50
π
|
ersi |
bzc6p: Where's your template on the wiki? |
21:51
π
|
bzc6p |
ersi: http://archiveteam.org/index.php?title=Template:Howcanihelp |
21:51
π
|
bzc6p |
Sorry, I indeed forgot to name it... |
21:51
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
21:52
π
|
bzc6p |
the additional info is put into a collapsible box |
21:54
π
|
ersi |
I'll take a look at merging it. I do, have a GitHub account. :) |
21:56
π
|
|
dashcloud has joined #archiveteam |
22:11
π
|
|
schbirid has quit IRC (Leaving) |
22:50
π
|
|
wp494 has quit IRC () |
22:52
π
|
|
wp494 has joined #archiveteam |
23:04
π
|
|
Start has joined #archiveteam |
23:16
π
|
|
bzc6p has left |
23:18
π
|
|
Start has quit IRC (Ping timeout: 606 seconds) |
23:23
π
|
|
primus has joined #archiveteam |