| Time |
Nickname |
Message |
|
00:09
🔗
|
joepie91 |
log-bookmark: why we do it (ignore that, it's for me to grep back to this point in the logs) |
|
00:10
🔗
|
joepie91 |
(for the next time "why would you bother saving that" comes up) |
|
00:25
🔗
|
|
philpem has quit IRC (Ping timeout: 252 seconds) |
|
00:42
🔗
|
|
Fusl has quit IRC (Max SendQ exceeded) |
|
00:43
🔗
|
|
Fusl has joined #archiveteam |
|
00:46
🔗
|
|
wutno` has quit IRC (Ping timeout: 306 seconds) |
|
00:58
🔗
|
|
Emcy_ has joined #archiveteam |
|
01:00
🔗
|
|
lytv has quit IRC (Read error: Operation timed out) |
|
01:01
🔗
|
|
JesseW has joined #archiveteam |
|
01:04
🔗
|
|
lytv has joined #archiveteam |
|
01:05
🔗
|
|
Emcy has quit IRC (Read error: Operation timed out) |
|
01:12
🔗
|
|
vitzli has joined #archiveteam |
|
01:13
🔗
|
|
brayden__ has joined #archiveteam |
|
01:13
🔗
|
|
swebb sets mode: +o brayden__ |
|
01:19
🔗
|
|
brayden_ has quit IRC (Read error: Operation timed out) |
|
01:36
🔗
|
arkiver |
SketchCow: can you ask him if http://blip.tv/lessig was his channel? |
|
01:37
🔗
|
arkiver |
when the archives are uploaded to IA I should be able toretrieve a list of his videos and get him the link to the original videos for download |
|
01:37
🔗
|
arkiver |
Blip also seems to be totally gone now |
|
01:40
🔗
|
aaaaaaaaa |
does it not resolve for you either? |
|
01:41
🔗
|
arkiver |
no, it doesn't resolve for me either |
|
01:41
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
01:42
🔗
|
SketchCow |
arkiver: I asked |
|
01:43
🔗
|
SketchCow |
No answer yet. |
|
01:43
🔗
|
arkiver |
thanks! |
|
01:43
🔗
|
arkiver |
SketchCow: a bit more then 100 videos haven't been saved from blip |
|
01:43
🔗
|
arkiver |
All the other videos are saved! |
|
01:44
🔗
|
MrRadar |
100 out of over 7 million isn't a bad miss rate |
|
01:45
🔗
|
aaaaaaaaa |
Depending on what they are, they may end up somewhere else anyway |
|
01:45
🔗
|
aaaaaaaaa |
e.g. channel awesome stuff or sfdebris |
|
01:46
🔗
|
SketchCow |
Got it. |
|
02:03
🔗
|
vitzli |
Guys, could you add links from archiveteam.org to digitize and fileformats wikis on a sidebar? |
|
02:05
🔗
|
vitzli |
fileformats wiki looks like a secret trove and like a proper secret trove is hidden :( |
|
02:07
🔗
|
SketchCow |
1. Shut up |
|
02:08
🔗
|
SketchCow |
2. Learn how to phrase suggestions |
|
02:08
🔗
|
SketchCow |
3. See how well phrased #1 is |
|
02:08
🔗
|
vitzli |
:( |
|
02:10
🔗
|
|
BlueMaxim has joined #archiveteam |
|
02:11
🔗
|
SketchCow |
Digitize is still in the oven. |
|
02:17
🔗
|
SketchCow |
I am not 100% happy with my flatbed scanning skills. Definitely need to get better at straightening. |
|
02:19
🔗
|
SketchCow |
Then again, I'm mostly digitizing these manuals as insurance against them not having made it into the big piles, which must be not true. |
|
02:22
🔗
|
SketchCow |
Like, I'm being over-reactive as it is. |
|
02:44
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
02:44
🔗
|
Ravenloft |
better safe than sorry |
|
02:49
🔗
|
|
xk_id has quit IRC (Remote host closed the connection) |
|
02:53
🔗
|
|
JesseW has joined #archiveteam |
|
03:08
🔗
|
|
yipdw changes topic to: Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | < BotoX_> dafuq is WARC | <SketchCow> 1. Shut up |
|
03:09
🔗
|
Zandro |
he'll take his buddhist stick to you for vocalizing |
|
03:11
🔗
|
|
yipdw changes topic to: Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | < BotoX_> dafuq is WARC |
|
03:14
🔗
|
|
SketchCow changes topic to: Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | <SketchCow> 1. Shut up |
|
03:25
🔗
|
SketchCow |
https://archive.org/details/CompactPotentiometers7231072212InstructionManual723102J |
|
03:31
🔗
|
|
VADemon has quit IRC (left4dead) |
|
03:44
🔗
|
|
Elegance has joined #archiveteam |
|
03:45
🔗
|
|
Elegance has quit IRC (Client Quit) |
|
03:54
🔗
|
|
aaaaaaaaa has quit IRC (Leaving) |
|
04:41
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
04:43
🔗
|
* |
JesseW is reading over the manual SketchCow linked above |
|
04:43
🔗
|
* |
JesseW is amused by the note on page 19 |
|
04:44
🔗
|
JesseW |
and apparently repeated on page 21 |
|
04:46
🔗
|
JesseW |
https://archive.org/search.php?query=creator%3A%22James+G.+Biddle+Company%22 |
|
04:46
🔗
|
JesseW |
apparently 5 other works by the same company were uploaded back in 2011. |
|
04:47
🔗
|
|
Ungstein has joined #archiveteam |
|
04:58
🔗
|
|
vitzli has joined #archiveteam |
|
05:00
🔗
|
|
dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.) |
|
05:01
🔗
|
|
dashcloud has joined #archiveteam |
|
05:11
🔗
|
|
bsmith096 has joined #archiveteam |
|
05:13
🔗
|
bsmith096 |
im just finishing up the last of my ffnet grab. it will be done by noon- ish est. does anyone have any suggestions for compressing these files on the fly? i dont have the room. almost 500GB, in roughly 9 million files |
|
05:15
🔗
|
bsmith096 |
if i just send them rsync as-is, it will thrash fos's disks, so im trying to minimize the inconvenience |
|
05:16
🔗
|
bsmith096 |
swebb SketchCow Coderjoe DFJustin closure ping |
|
05:18
🔗
|
DFJustin |
just loose html files? |
|
05:22
🔗
|
bsmith096 |
DFJustin: text files of the stories |
|
05:22
🔗
|
bsmith096 |
it took me about 2 years to grab almost all of them, up to id # 11 million |
|
05:26
🔗
|
DFJustin |
how much room do you have |
|
05:27
🔗
|
bsmith096 |
about 30gb |
|
05:28
🔗
|
bsmith096 |
DFJustin: the files are 400gb+ |
|
05:29
🔗
|
DFJustin |
so you need something that can put together a chunk of say 10gb at a time and keep track of where it left off |
|
05:30
🔗
|
DFJustin |
don't know a tool that can do that offhand other than writing a custom script but there might be something |
|
05:30
🔗
|
bsmith096 |
ive heard i can also compress on the fly with pipes, but the syntax is wierd |
|
05:31
🔗
|
bsmith096 |
ssh cat and gzip |
|
05:32
🔗
|
DFJustin |
yeah you can probably rig together something like that but the odds of getting all the way through 9 million files without some kind of error in the middle seems low |
|
05:38
🔗
|
Zandro |
if you compress by last integer, 0-9, the 10 archives will be predictably consistent in size. move each off disk before compressing the next. you can recombine them later |
|
05:40
🔗
|
Zandro |
bzip2 is great for text compression btw |
|
05:41
🔗
|
Zandro |
ppdm via 7zip is also a good choice |
|
05:41
🔗
|
Zandro |
ppmd* |
|
05:43
🔗
|
|
Ungstein has quit IRC (Quit: Leaving.) |
|
05:44
🔗
|
bsmith096 |
the problem is they're very small individually, but there's a crapload of them, hence compression, but i dont really have the room to store both the compressed archives and the fully expanded files |
|
05:46
🔗
|
|
Ungstein has joined #archiveteam |
|
05:46
🔗
|
garyrh |
gnu tar has a --remove-files option |
|
05:47
🔗
|
Zandro |
dangerous, in that he has the only copy |
|
05:50
🔗
|
bsmith096 |
SketchCow is just gonna compress them anyway, i thought i'd save time, bandwidth, and my disk r/w heads and compress in transit |
|
05:57
🔗
|
|
Ungstein has quit IRC (Read error: Connection reset by peer) |
|
05:57
🔗
|
bsmith096 |
what id like to do is have gzip (or whatever) make the folder into multi gb chunks, copied to fos as they're made then delete the chunks |
|
05:58
🔗
|
bsmith096 |
i could rsync the gzip chunks |
|
05:58
🔗
|
bsmith096 |
but i fully admit i have no idea how to do that |
|
05:59
🔗
|
DFJustin |
I think a lot of the folks here are asleep at this hour so you may have better luck in 8 hours or so |
|
06:00
🔗
|
|
Ungstein has joined #archiveteam |
|
06:01
🔗
|
bsmith096 |
k, the grab will probably be done by then. 2000 files to go! |
|
06:01
🔗
|
bsmith096 |
(ffnet throttles, or i would have been done a year and a half age) |
|
06:01
🔗
|
bsmith096 |
*ago |
|
06:12
🔗
|
fenn |
`scp -C` or `rsync -z` will compress during transfer (though it will be uncompressed on the other end) |
|
06:17
🔗
|
fenn |
just make tar chunks in your free space, rsync them, and delete them |
|
06:18
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
06:30
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
06:46
🔗
|
|
Ungstein has quit IRC (Quit: Leaving.) |
|
06:57
🔗
|
|
Ungstein has joined #archiveteam |
|
07:04
🔗
|
|
primus104 has joined #archiveteam |
|
07:33
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
07:35
🔗
|
|
philpem has joined #archiveteam |
|
07:36
🔗
|
|
Ravenloft has quit IRC (Read error: Connection reset by peer) |
|
08:02
🔗
|
|
primus104 has joined #archiveteam |
|
08:05
🔗
|
|
OregonRos has joined #archiveteam |
|
08:06
🔗
|
OregonRos |
Is there any place on the Internet where I could possibly find a cached view of my old Myspace or Facebook profile |
|
08:07
🔗
|
|
OregonRos has quit IRC (Client Quit) |
|
08:24
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
09:50
🔗
|
|
nmnn_ has joined #archiveteam |
|
09:51
🔗
|
|
habi has joined #archiveteam |
|
09:55
🔗
|
|
philpem has quit IRC (Ping timeout: 252 seconds) |
|
10:00
🔗
|
|
habi has quit IRC (Quit: Leaving.) |
|
10:12
🔗
|
|
vitzli has joined #archiveteam |
|
10:17
🔗
|
|
nmnn_ has quit IRC (Ping timeout: 483 seconds) |
|
10:17
🔗
|
|
bsmith096 has quit IRC (Ping timeout: 240 seconds) |
|
10:26
🔗
|
|
Dennisjr1 has quit IRC (Ping timeout: 240 seconds) |
|
10:34
🔗
|
|
BlueMaxim has quit IRC (Read error: Connection reset by peer) |
|
11:13
🔗
|
|
wutno has joined #archiveteam |
|
11:59
🔗
|
|
philpem has joined #archiveteam |
|
12:02
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
12:54
🔗
|
|
primus104 has joined #archiveteam |
|
13:04
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
13:05
🔗
|
|
zenguy_pc has quit IRC (Read error: Connection reset by peer) |
|
13:06
🔗
|
|
zenguy_pc has joined #archiveteam |
|
13:10
🔗
|
|
vitzli has joined #archiveteam |
|
13:28
🔗
|
|
garyrh has quit IRC (Remote host closed the connection) |
|
14:00
🔗
|
|
VADemon has joined #archiveteam |
|
14:04
🔗
|
|
nmnn_ has joined #archiveteam |
|
14:14
🔗
|
|
garyrh has joined #archiveteam |
|
14:17
🔗
|
SketchCow |
Hi what |
|
14:17
🔗
|
|
nmnn_ has quit IRC (Ping timeout: 483 seconds) |
|
14:18
🔗
|
SketchCow |
What a lot of technical discussion for on the fly compression |
|
14:21
🔗
|
|
nmnn_ has joined #archiveteam |
|
14:24
🔗
|
SketchCow |
Regarding blip.tv request for lessig, yes, the username is lessig. |
|
14:24
🔗
|
SketchCow |
So yes, to arkiver |
|
14:29
🔗
|
|
nmnn_ has quit IRC (Ping timeout: 483 seconds) |
|
14:37
🔗
|
|
nmnn_ has joined #archiveteam |
|
14:38
🔗
|
|
nertzy has joined #archiveteam |
|
14:47
🔗
|
|
nmnn_ has quit IRC (Ping timeout: 483 seconds) |
|
15:17
🔗
|
|
PurpleSym has joined #archiveteam |
|
15:19
🔗
|
SilSte |
SketchCow: They should give some money to the IA ;) . And us some beers :D |
|
15:27
🔗
|
|
nertzy has quit IRC (Quit: This computer has gone to sleep) |
|
15:28
🔗
|
ersi |
Uh, ok. |
|
15:29
🔗
|
|
nmnn_ has joined #archiveteam |
|
15:31
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
15:42
🔗
|
SilSte |
OVH is raising prices... check your mails if you have one ;) |
|
15:54
🔗
|
Smiley |
well theres an amusing article on why archiving matters... |
|
15:54
🔗
|
Smiley |
http://www.gamasutra.com/view/feature/167392/sad_but_true_we_cant_prove_when_.php |
|
15:56
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
15:58
🔗
|
|
JesseW has joined #archiveteam |
|
16:05
🔗
|
|
Ungstein has quit IRC (Quit: Leaving.) |
|
16:13
🔗
|
|
primus104 has joined #archiveteam |
|
16:15
🔗
|
|
hive-mind has quit IRC (Remote host closed the connection) |
|
16:15
🔗
|
|
primus105 has joined #archiveteam |
|
16:22
🔗
|
|
primus104 has quit IRC (Read error: Operation timed out) |
|
16:35
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
16:44
🔗
|
|
xk_id has joined #archiveteam |
|
16:54
🔗
|
|
primus105 has quit IRC (Leaving.) |
|
17:03
🔗
|
|
schbirid has joined #archiveteam |
|
17:26
🔗
|
joepie91 |
hmmm. we have gamespy archived, yes? |
|
17:26
🔗
|
joepie91 |
SilSte: er. wat? |
|
17:27
🔗
|
schbirid |
partially at least, its hard to tell |
|
17:27
🔗
|
joepie91 |
schbirid: well, it's still online, so... :P |
|
17:27
🔗
|
schbirid |
parts of it are D: |
|
17:28
🔗
|
joepie91 |
ah |
|
17:28
🔗
|
schbirid |
oh i forgot to get that vm back online. gamespy-archives.quaddicted.com usually has the old forums and the planet sites |
|
17:29
🔗
|
joepie91 |
SilSte: not seeing any changes for kimsufi anywy |
|
17:29
🔗
|
joepie91 |
anyway * |
|
17:39
🔗
|
|
RichardG has joined #archiveteam |
|
17:48
🔗
|
SilSte |
anyone in that grsecurity stuff? Shall it be saved? |
|
17:58
🔗
|
|
xk_id has quit IRC (Remote host closed the connection) |
|
18:09
🔗
|
|
bsmith096 has joined #archiveteam |
|
18:15
🔗
|
ersi |
SilSte: What specificly are you referring to? The announce.php? |
|
18:15
🔗
|
ersi |
That's saved. Or all stable releases? or wut |
|
18:15
🔗
|
SilSte |
i would say the stable releases |
|
18:16
🔗
|
SilSte |
everything what is going to be hidden |
|
18:18
🔗
|
|
primus104 has joined #archiveteam |
|
18:18
🔗
|
espes__ |
woot. another amazon pilot episode found on a random vkontakte page |
|
18:23
🔗
|
espes__ |
(http://piratepad.net/e8FaqvIGRx) |
|
18:24
🔗
|
MrRadar |
Is Amazon removing those? |
|
18:24
🔗
|
espes__ |
amazon only has rights to distribute them for a couple months |
|
18:25
🔗
|
MrRadar |
Ah |
|
18:26
🔗
|
espes__ |
and yeah, turns out nobody makes copies of free stuff on amazon |
|
18:26
🔗
|
espes__ |
specially kids shows |
|
18:29
🔗
|
|
xk_id has joined #archiveteam |
|
18:31
🔗
|
espes__ |
so a bunch are lost |
|
18:31
🔗
|
bsmith096 |
SketchCow: im done, or atleast ready to send you all the files, how do i not thrash the discs this time? |
|
18:37
🔗
|
|
aaaaaaaaa has joined #archiveteam |
|
18:37
🔗
|
|
swebb sets mode: +o aaaaaaaaa |
|
18:39
🔗
|
bsmith096 |
SketchCow: this is my current idea, tar -zc "/home/ben/Desktop/Ben's Stuff/Fanfiction" | pv | ssh wacko@fos.textfiles.com "cat > ~/bsmith/Fanfiction.tar.gz" |
|
18:46
🔗
|
xmc |
that's probably ok |
|
18:47
🔗
|
xmc |
i'd suggest instead making a list of directories and then giving it to tar, so if it breaks off you can trim the list up to a bit before that point and continue |
|
19:01
🔗
|
|
schbirid has quit IRC (Leaving) |
|
19:06
🔗
|
|
khaoohs has quit IRC (Read error: Connection reset by peer) |
|
19:20
🔗
|
|
xk_id has quit IRC (Remote host closed the connection) |
|
19:21
🔗
|
Nemo_bis |
bsmith096: the tar "file" can also be a remote one |
|
19:21
🔗
|
Nemo_bis |
at least in some OS |
|
19:23
🔗
|
|
khaoohs has joined #archiveteam |
|
19:24
🔗
|
bsmith096 |
Nemo_bis: DFJustin i'm going with my current idea, i passed 500G to pv as the expected size, b/c thats how big the drive is, and im using most of it. well over 470GB |
|
19:27
🔗
|
bsmith096 |
ETA 230 hrs , but thats an overestimate, the compression is probably adding to that, also im only doing this so i'll write one huge file instead of 10 million tiny ones, also, this doesnt leave temp files, right? b/c i seriously dont have the room |
|
19:32
🔗
|
MrRadar |
The command you posted should not make any temp files on your local machine |
|
19:33
🔗
|
bsmith096 |
MrRadar: great!, its still running, it will probably take a few days |
|
19:34
🔗
|
|
scyther has joined #archiveteam |
|
20:03
🔗
|
|
lytv has quit IRC (Quit: Leaving) |
|
20:04
🔗
|
|
db48x` has joined #archiveteam |
|
20:10
🔗
|
|
aliz has quit IRC (Ping timeout: 252 seconds) |
|
20:16
🔗
|
|
xk_id has joined #archiveteam |
|
20:17
🔗
|
|
aliz has joined #archiveteam |
|
20:27
🔗
|
|
lytv has joined #archiveteam |
|
20:33
🔗
|
|
scyther has quit IRC (Leaving) |
|
20:37
🔗
|
|
aliz has quit IRC (Ping timeout: 252 seconds) |
|
20:44
🔗
|
|
aliz has joined #archiveteam |
|
20:52
🔗
|
|
PurpleSym has quit IRC (Remote host closed the connection) |
|
21:02
🔗
|
|
nmnn_ has quit IRC (Quit: Ex-Chat) |
|
21:27
🔗
|
|
aaaaaaaa_ has joined #archiveteam |
|
21:27
🔗
|
|
aaaaaaaaa has quit IRC (Read error: Connection reset by peer) |
|
21:27
🔗
|
|
swebb sets mode: +o aaaaaaaa_ |
|
21:37
🔗
|
|
aaaaaaaa_ is now known as aaaaaaaaa |
|
21:41
🔗
|
|
goekesmi has quit IRC (Remote host closed the connection) |
|
22:08
🔗
|
|
RedType has quit IRC (Quit: leaving) |
|
22:08
🔗
|
|
RedType has joined #archiveteam |
|
22:19
🔗
|
|
goekesmi has joined #archiveteam |
|
22:24
🔗
|
|
db48x` has quit IRC (Read error: Operation timed out) |
|
22:27
🔗
|
|
bsmith096 has quit IRC (Ping timeout: 240 seconds) |
|
23:36
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
23:36
🔗
|
|
dashcloud has joined #archiveteam |
|
23:46
🔗
|
|
JesseW has joined #archiveteam |