| Time |
Nickname |
Message |
|
00:02
🔗
|
|
icedice has quit IRC (Ping timeout: 245 seconds) |
|
00:06
🔗
|
|
wp494 has joined #archiveteam-bs |
|
00:20
🔗
|
|
Stiletto has quit IRC (Read error: Operation timed out) |
|
00:30
🔗
|
|
Stilett0 has joined #archiveteam-bs |
|
00:34
🔗
|
|
Stilett0 is now known as Stiletto |
|
01:10
🔗
|
CoolCanuk |
sears tried really hard. RIP |
|
01:22
🔗
|
CoolCanuk |
"BobTheBuilder_Zoo-WT.exe" is this someone's personal computer? I thought CIA and FBI files were supposed to be highly secured... :/ |
|
01:28
🔗
|
CoolCanuk |
oh i'm an idiot. nvm |
|
01:42
🔗
|
|
kristian_ has joined #archiveteam-bs |
|
02:05
🔗
|
|
username1 has joined #archiveteam-bs |
|
02:08
🔗
|
|
Famicoman has quit IRC (Ping timeout: 260 seconds) |
|
02:09
🔗
|
|
schbirid2 has quit IRC (Read error: Operation timed out) |
|
02:23
🔗
|
|
wp494 has quit IRC (LOUD UNNECESSARY QUIT MESSAGES) |
|
02:26
🔗
|
|
wp494 has joined #archiveteam-bs |
|
02:29
🔗
|
|
kristian_ has quit IRC (Quit: Leaving) |
|
02:45
🔗
|
|
pizzaiolo has quit IRC (Remote host closed the connection) |
|
02:56
🔗
|
Somebody2 |
arkiver: please spread the ops in #internetarchive -- also, there's a spammer there who should probably be banned. |
|
02:57
🔗
|
Somebody2 |
SketchCow: I am interested in your incrediblly boring thing; I'll see which ones I get to. |
|
03:15
🔗
|
Somebody2 |
SketchCow: https://archive.org/details/archiveteam_wiki is confusing; it's megawarcs of various wikis, from 2015 through 2017 |
|
03:15
🔗
|
Somebody2 |
but I can't figure out where it was dicussed, or where the list of sites to grab came from. |
|
04:13
🔗
|
SketchCow |
Ha |
|
04:13
🔗
|
SketchCow |
It's the Archiveteam Wiki Project., |
|
04:18
🔗
|
Somebody2 |
I know, but the wiki page for that doesn't mention an effort from 2015 through 2017 to grab WARCs, at least not where I could find it... |
|
04:18
🔗
|
|
ranavalon has quit IRC (Read error: Connection reset by peer) |
|
04:18
🔗
|
SketchCow |
http://archiveteam.org/index.php?title=WikiTeam is pretty clear |
|
04:21
🔗
|
Somebody2 |
hm, that page has no mention of the words WARC or Wrapup, which is what made me confused. |
|
04:22
🔗
|
Somebody2 |
And the contents of https://archive.org/details/wikiteam (which is linked from there) is all using the wikiteam tools, not WARCs. |
|
04:23
🔗
|
Somebody2 |
But yeah, it certainly seems *related* |
|
04:30
🔗
|
godane |
SketchCow: the John Hoy 2000-06-22 tape 3 is really bad |
|
04:31
🔗
|
|
qw3rty113 has joined #archiveteam-bs |
|
04:31
🔗
|
godane |
its like very bright light so can't really see the picture |
|
04:35
🔗
|
|
qw3rty112 has quit IRC (Read error: Operation timed out) |
|
04:51
🔗
|
godane |
so i'm on my 3rd tape for today |
|
04:58
🔗
|
CoolCanuk |
tape? |
|
05:02
🔗
|
CoolCanuk |
https://i.imgur.com/kLetCe3.png |
|
05:03
🔗
|
Somebody2 |
SketchCow: https://archive.org/details/archiveteam_20170123051332 should be in https://archive.org/details/archiveteam_flickr |
|
05:03
🔗
|
Somebody2 |
it's in https://archive.org/details/archiveteam_torrents by mistake. |
|
05:04
🔗
|
Somebody2 |
ok, I've incremented the required library version and switched the two bad items to new people |
|
05:04
🔗
|
Somebody2 |
we'll see if that fixes it |
|
05:35
🔗
|
Somebody2 |
SketchCow: OK, here are descriptions for some of your items that lack them: https://hastebin.com/omiwasowop.sql |
|
05:37
🔗
|
|
Yurume has quit IRC (Read error: Operation timed out) |
|
05:42
🔗
|
|
Yurume has joined #archiveteam-bs |
|
05:47
🔗
|
godane |
i'm now on my 4th tape |
|
06:02
🔗
|
godane |
its one of the Larry Sanders Show tapes i got |
|
06:02
🔗
|
godane |
btw the label sticker didn't stick to the tape anymore |
|
06:03
🔗
|
godane |
anyways this tape has episode S06E02 S06E03 and S06E05 |
|
06:15
🔗
|
CoolCanuk |
anyone know what timezone "noon" means? http://www.dpconline.org/our-work/bit-list |
|
06:17
🔗
|
CoolCanuk |
lol RIP. I didnt realize it was a new day. I thought that was yesterday |
|
06:20
🔗
|
Somebody2 |
CoolCanuk: UTC, as the organization is based in the UK. |
|
06:20
🔗
|
|
ZexaronS has quit IRC (Quit: Leaving) |
|
06:20
🔗
|
Somebody2 |
so, another six hours. |
|
06:26
🔗
|
CoolCanuk |
just heard an explosion or maybe a house collapse, or car hit a hydro post or something. Weirdest sound ever |
|
06:27
🔗
|
CoolCanuk |
(I live in a fairly rural area. Also, it's 1:26 AM). Oh well. |
|
06:34
🔗
|
Somebody2 |
please don't die. |
|
06:49
🔗
|
CoolCanuk |
im fine |
|
06:49
🔗
|
Somebody2 |
godane: I'm looking at your grab of NPR's All Things Considered -- were you not able to get the recordings from after 2001? |
|
06:53
🔗
|
godane |
i only have upload up to 2001-12-31 |
|
06:53
🔗
|
godane |
i can get more of them |
|
06:53
🔗
|
Somebody2 |
godane: Yes please! |
|
06:54
🔗
|
Somebody2 |
Considering the current political climate, having offsite storage of as much of NPR as possible seems like a very good idea. |
|
06:54
🔗
|
godane |
good news is i have up to dec 2006 on my drive |
|
06:55
🔗
|
Somebody2 |
godane: nice |
|
06:55
🔗
|
Somebody2 |
I'm adding a description to the collection. |
|
06:55
🔗
|
Somebody2 |
which is how I came across it |
|
06:59
🔗
|
SketchCow |
Somebody2: I've made the additions. |
|
07:01
🔗
|
Somebody2 |
SketchCow: thanks; I'm making up another batch now. |
|
07:01
🔗
|
Somebody2 |
https://archive.org/details/archiveteam_dutchnews&tab=about <- somehow ended up in a weird format. |
|
07:03
🔗
|
Somebody2 |
and I screwed up the links to the archiveteam wiki. |
|
07:03
🔗
|
Somebody2 |
on e.g. https://archive.org/details/archiveteam_fanfiction |
|
07:13
🔗
|
Somebody2 |
godane: did you extract the UN Radio files from https://www.unmultimedia.org/radio/english/ or somewhere else? |
|
07:14
🔗
|
Somebody2 |
and do you have any from 2017 to upload? |
|
07:14
🔗
|
godane |
i only did up to 2016 |
|
07:14
🔗
|
Somebody2 |
OK. |
|
07:17
🔗
|
godane |
SketchCow: is there anyway to make the podcast-mirror be downloadable |
|
07:17
🔗
|
Somebody2 |
SketchCow: here's six more: https://hastebin.com/babetusaho.php |
|
07:18
🔗
|
Somebody2 |
and I'm going to sleep. |
|
07:18
🔗
|
Coderjo |
i hate cloudflare? what? |
|
07:18
🔗
|
CoolCanuk |
xD |
|
07:19
🔗
|
godane |
someone emailed me about Martin Yan shows and not being able to download them |
|
07:20
🔗
|
godane |
https://archive.org/details/Martin_Yan_Shows |
|
07:21
🔗
|
godane |
some try to download this: https://archive.org/download/Martin_Yan_Quick_and_Easy_S01E18/Martin_Yan_Quick_and_Easy_S01E18.avi |
|
07:21
🔗
|
godane |
i can do it in firefox login |
|
07:21
🔗
|
CoolCanuk |
takedown? |
|
07:21
🔗
|
CoolCanuk |
"The item is not available due to issues with the item's content." |
|
07:22
🔗
|
godane |
but wget gives me 403 forbidden item |
|
07:24
🔗
|
CoolCanuk |
I can't get https://ia902303.us.archive.org/29/items/Martin_Yan_Quick_and_Easy_S01E10/Martin_Yan_Quick_and_Easy_S01E10.mp4 to work either hmmmm |
|
07:25
🔗
|
CoolCanuk |
a bunch of files dont let me download :/ |
|
07:31
🔗
|
godane |
anyways i'm up to 1107k items now |
|
07:32
🔗
|
godane |
1,107,509 items |
|
07:53
🔗
|
|
Aerochrom has joined #archiveteam-bs |
|
07:56
🔗
|
CoolCanuk |
:o |
|
08:56
🔗
|
|
Stiletto has quit IRC (Ping timeout: 250 seconds) |
|
09:14
🔗
|
|
Mateon1 has quit IRC (Read error: Operation timed out) |
|
09:16
🔗
|
|
Mateon1 has joined #archiveteam-bs |
|
09:23
🔗
|
|
jschwart has joined #archiveteam-bs |
|
09:37
🔗
|
|
alfie has quit IRC (Ping timeout: 248 seconds) |
|
09:37
🔗
|
|
alfie has joined #archiveteam-bs |
|
09:38
🔗
|
|
jschwart has quit IRC (Read error: Operation timed out) |
|
09:39
🔗
|
|
medowar has joined #archiveteam-bs |
|
09:39
🔗
|
|
tuluu has quit IRC (Ping timeout: 248 seconds) |
|
09:39
🔗
|
|
midas3 has quit IRC (Ping timeout: 248 seconds) |
|
09:40
🔗
|
|
midas3 has joined #archiveteam-bs |
|
09:40
🔗
|
|
tuluu has joined #archiveteam-bs |
|
09:53
🔗
|
godane |
i'm doing the secrets of isis volume 1 tape |
|
10:06
🔗
|
|
jtn2_ has joined #archiveteam-bs |
|
10:06
🔗
|
|
Mateon1 has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
Ceryn has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
jtn2 has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
yuitimoth has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
SmileyG has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
ez has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
w0rp has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
MrRadar2 has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
second has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
Fusl has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
will has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
Tenebrae has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
Xibalba has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
hook54321 has quit IRC (se.hub irc.efnet.nl) |
|
10:07
🔗
|
|
pizzaiolo has joined #archiveteam-bs |
|
10:08
🔗
|
|
i0npulse has quit IRC (Ping timeout: 248 seconds) |
|
10:10
🔗
|
|
Mateon1 has joined #archiveteam-bs |
|
10:10
🔗
|
|
yuitimoth has joined #archiveteam-bs |
|
10:10
🔗
|
|
SmileyG has joined #archiveteam-bs |
|
10:10
🔗
|
|
ez has joined #archiveteam-bs |
|
10:10
🔗
|
|
w0rp has joined #archiveteam-bs |
|
10:10
🔗
|
|
MrRadar2 has joined #archiveteam-bs |
|
10:10
🔗
|
|
second has joined #archiveteam-bs |
|
10:10
🔗
|
|
Fusl has joined #archiveteam-bs |
|
10:10
🔗
|
|
will has joined #archiveteam-bs |
|
10:10
🔗
|
|
Tenebrae has joined #archiveteam-bs |
|
10:10
🔗
|
|
Xibalba has joined #archiveteam-bs |
|
10:10
🔗
|
|
hook54321 has joined #archiveteam-bs |
|
10:10
🔗
|
|
irc.efnet.nl sets mode: +o hook54321 |
|
10:10
🔗
|
|
i0npulse has joined #archiveteam-bs |
|
10:17
🔗
|
|
jschwart has joined #archiveteam-bs |
|
10:18
🔗
|
|
CoolCanuk has quit IRC (Quit: Connection closed for inactivity) |
|
10:21
🔗
|
|
jschwart has quit IRC (Remote host closed the connection) |
|
10:25
🔗
|
|
jschwart has joined #archiveteam-bs |
|
10:42
🔗
|
Igloo |
The way back machine appears to be broken :< |
|
10:55
🔗
|
godane |
so this is odd |
|
10:55
🔗
|
godane |
the isis volume 1 tape has sort of a star trek blooper reel |
|
10:56
🔗
|
godane |
at the end of it |
|
10:57
🔗
|
|
Ceryn has joined #archiveteam-bs |
|
11:14
🔗
|
SketchCow |
Somebody2: These are somewhat simplistic descriptions, but they're good placeholders, thank you. |
|
11:21
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
11:24
🔗
|
SketchCow |
godane: When you wake up - I MIGHT have screwed up the Larry Sanders Show tape due to a bad regex. I'd re-upload for safety. |
|
11:26
🔗
|
godane |
don't upload it |
|
11:26
🔗
|
godane |
i'm uploading it right now |
|
11:26
🔗
|
SketchCow |
you misunderstand |
|
11:26
🔗
|
SketchCow |
By mistake, my uploader tried to upload it. |
|
11:26
🔗
|
SketchCow |
There's a chance it's truncated. |
|
11:26
🔗
|
godane |
ok |
|
11:26
🔗
|
SketchCow |
so I'd say re-upload from scratch, it'll overwrite the thing |
|
11:27
🔗
|
SketchCow |
So when you get a chance, re-upload that file, so next time I overwrite the other |
|
11:27
🔗
|
godane |
i'm uploading it RIGHT Now |
|
11:27
🔗
|
SketchCow |
I know |
|
11:29
🔗
|
godane |
i reconnected and renamed the file |
|
11:29
🔗
|
Igloo |
godane: upload that and put it on youtube :) |
|
11:29
🔗
|
Igloo |
(Or link it here for the blooper reel!) |
|
11:31
🔗
|
godane |
i will see about splitting the blooper reel from the isis volume 1 tape file |
|
11:32
🔗
|
godane |
just know this tape is like 5 gens removed from masters it looks like |
|
11:50
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
|
12:12
🔗
|
godane |
i'm uploading King of Iron Chefs Tournament Grand Finale Part 2 Food Network Tape |
|
12:12
🔗
|
godane |
larry sanders show file is done |
|
12:21
🔗
|
godane |
so i'm at 8501 items this month |
|
12:44
🔗
|
|
marvinw has quit IRC (Leaving) |
|
12:45
🔗
|
|
marvinw has joined #archiveteam-bs |
|
12:47
🔗
|
|
dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.) |
|
12:49
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
12:50
🔗
|
|
Stilett0 has joined #archiveteam-bs |
|
13:27
🔗
|
jrwr |
VHS Ripping, man |
|
13:28
🔗
|
SketchCow |
It never ends |
|
13:39
🔗
|
godane |
I'm uploading Buffy S07E01 Rough Cut And Hanunted S01E01 Revised Pilot Tape |
|
14:06
🔗
|
jrwr |
godane: did you raid like a horde of VHS Tapes from film studio dumpster? |
|
14:11
🔗
|
godane |
no |
|
14:11
🔗
|
|
Darkstar has quit IRC (No Ping reply in 180 seconds.) |
|
14:11
🔗
|
godane |
this all from Jason Scott's boxes he mailed me |
|
14:12
🔗
|
SketchCow |
Yeah |
|
14:13
🔗
|
SketchCow |
My fiancee worked for Entertainment Weekly in the early 2000s and got a lot of screener tapes. |
|
14:19
🔗
|
godane |
and whats funny is you got a Entertainment Weekly preview 1999 tape |
|
14:19
🔗
|
godane |
which even though it was not touch it was unplayable |
|
14:20
🔗
|
godane |
the tracking was not working at all on that |
|
14:20
🔗
|
|
Darkstar has joined #archiveteam-bs |
|
14:21
🔗
|
godane |
anyways i resort some of the tape boxes so the one box i'm putting tapes has room |
|
14:55
🔗
|
jrwr |
Nice, its great that you are finding all kinds of good shit, If you need help with anything, I'm game |
|
14:56
🔗
|
jrwr |
I'm in Rochester NY now, so I'm not too far |
|
15:05
🔗
|
SketchCow |
We should just make you work for us at the Strong Museum |
|
15:05
🔗
|
SketchCow |
I have many things stored there, that need ripping/scanning |
|
15:48
🔗
|
jrwr |
Where is it at Sketch? |
|
15:49
🔗
|
jrwr |
NVM |
|
15:49
🔗
|
jrwr |
its here local |
|
15:49
🔗
|
jrwr |
dude I'm game |
|
15:49
🔗
|
jrwr |
I'm bored as shit on the weekends |
|
15:52
🔗
|
|
Aerochrom has quit IRC (Ping timeout: 248 seconds) |
|
15:52
🔗
|
|
Aerochrom has joined #archiveteam-bs |
|
15:53
🔗
|
godane |
i'm uploading Degrassi Junior High S01E01 to S01E12 The N Woc 2005-10 |
|
15:54
🔗
|
godane |
that one is going to take awhile cause its 6 hours |
|
16:01
🔗
|
godane |
i still have about 125gb of digitize tapes to upload |
|
16:02
🔗
|
godane |
i got maybe 16 tapes i can digitize |
|
16:03
🔗
|
godane |
there is maybe another 5 tapes i can't do cause of damage or mold |
|
16:07
🔗
|
godane |
i also have a 250gb of my own rips to upload |
|
16:07
🔗
|
jrwr |
Ya SketchCow hit me up if you need a man out in Rochester, I live in town and have lots of free time. My workplace used to do DR for a bank and has a TON of empty space just going to waste |
|
16:08
🔗
|
godane |
about 73gb is my family christmas tv recordings |
|
16:08
🔗
|
godane |
one of them goes to week of 1988-11-28 to 1988-12-05 |
|
16:09
🔗
|
|
Stilett0 has quit IRC (Read error: Operation timed out) |
|
16:11
🔗
|
|
fie has quit IRC (Read error: Operation timed out) |
|
16:23
🔗
|
|
RichardG_ has joined #archiveteam-bs |
|
16:24
🔗
|
|
RichardG has quit IRC (Ping timeout: 250 seconds) |
|
16:25
🔗
|
|
fie has joined #archiveteam-bs |
|
16:28
🔗
|
|
Stilett0 has joined #archiveteam-bs |
|
17:00
🔗
|
|
CoolCanuk has joined #archiveteam-bs |
|
17:01
🔗
|
CoolCanuk |
If there's anything I can do to help with the 508 error, let me know |
|
17:05
🔗
|
jrwr |
Its just on shared host, it does that |
|
17:08
🔗
|
CoolCanuk |
I could provide hosting... assuming the user/authentication database is separate from the wiki. |
|
17:09
🔗
|
CoolCanuk |
probably has been posted, but http://www.dpconline.org/our-work/bit-list is now available |
|
17:10
🔗
|
jrwr |
SketchCow has been hosting it for years, hes not moving any time soon |
|
17:10
🔗
|
CoolCanuk |
:) just thought i'd offer |
|
17:10
🔗
|
jrwr |
Ya |
|
17:24
🔗
|
|
schbirid2 has joined #archiveteam-bs |
|
17:30
🔗
|
|
username1 has quit IRC (Read error: Operation timed out) |
|
17:53
🔗
|
|
ranavalon has quit IRC (Read error: Connection reset by peer) |
|
17:54
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
18:22
🔗
|
|
phuzion has quit IRC (Remote host closed the connection) |
|
18:53
🔗
|
|
jschwart has quit IRC (Quit: Konversation terminated!) |
|
19:05
🔗
|
|
jschwart has joined #archiveteam-bs |
|
19:30
🔗
|
|
icedice has joined #archiveteam-bs |
|
19:36
🔗
|
|
icedice has quit IRC (Ping timeout: 245 seconds) |
|
19:41
🔗
|
|
icedice has joined #archiveteam-bs |
|
19:54
🔗
|
schbirid2 |
https://imgur.com/a/lOgWd "90s Stock Photography is the Bomb" |
|
19:57
🔗
|
CoolCanuk |
schbirid2: reminds me of http://www.annualreports.com/HostedData/AnnualReportArchive/s/TSX_SEARF_1999.pdf ... their annual report looks like a highschool textbook mixed with windows 95 |
|
19:57
🔗
|
schbirid2 |
wow |
|
20:01
🔗
|
CoolCanuk |
we should put this on our 508 page xD https://i.imgur.com/jzaikp2.jpg |
|
20:17
🔗
|
jrwr |
holy shit |
|
20:17
🔗
|
jrwr |
I love it |
|
20:28
🔗
|
CoolCanuk |
does archivebot use ia useragent? |
|
20:29
🔗
|
CoolCanuk |
or robots.txt at all |
|
20:44
🔗
|
jrwr |
it does not use the IA useragent |
|
20:44
🔗
|
jrwr |
or robots.txt |
|
20:44
🔗
|
CoolCanuk |
perfect |
|
20:45
🔗
|
CoolCanuk |
!a https://partners.sears.ca/PAMP/Pages/Login/UserLogin.aspx |
|
20:45
🔗
|
CoolCanuk |
oops. |
|
20:45
🔗
|
CoolCanuk |
sorry |
|
20:47
🔗
|
|
TheLovina has joined #archiveteam-bs |
|
20:50
🔗
|
bithippo |
Is it possible to make an API call to the IA to check for items with a specific tag set to a specific value? |
|
20:50
🔗
|
bithippo |
(ie check if item exists with "Originalurl" set to "$URL") |
|
20:51
🔗
|
|
acridAxid has quit IRC (marauder) |
|
20:52
🔗
|
|
acridAxid has joined #archiveteam-bs |
|
20:53
🔗
|
jrwr |
bithippo: mabybe |
|
20:54
🔗
|
jrwr |
https://archive.readme.io/docs |
|
20:54
🔗
|
bithippo |
Testing https://archive.org/advancedsearch.php#raw out now |
|
20:56
🔗
|
bithippo |
Extended tubeup to check for an item first before fetching from the source. |
|
20:56
🔗
|
bithippo |
Extending* |
|
20:59
🔗
|
|
ola_norsk has joined #archiveteam-bs |
|
21:00
🔗
|
jrwr |
cool |
|
21:02
🔗
|
ola_norsk |
Is there a way to determine if two items contain the same file (regardless of filename) without having to download all the files from each item? Been looking at the .sqlite db for one, in hope to find some kind of checksum |
|
21:03
🔗
|
ola_norsk |
is the IA sqlite db format documented somewhere? |
|
21:04
🔗
|
bithippo |
Each item has a torrent file associated with it, could get the file hash from there. |
|
21:04
🔗
|
ola_norsk |
that contains the hash of each containing file? |
|
21:05
🔗
|
bithippo |
Standby, testing. |
|
21:07
🔗
|
ola_norsk |
when looking at the sqlite, i note there's e.g "ETag: "3d6c17432d2dd1a48ee0664ca32caeff"" , though i don't know if thats a hash |
|
21:07
🔗
|
ola_norsk |
of the file, i mean |
|
21:09
🔗
|
bithippo |
I'm wrong, I thought you could get a hash of the file from the torrent file, but you need to retrieve the file first. I would _assume_ that ETag is a hash of some sort. |
|
21:11
🔗
|
ola_norsk |
yeah, it does look to be unique for each file in an item as far i can tell so far. I found it in the s3api_per_key_metadata > headers |
|
21:11
🔗
|
ola_norsk |
not "yeah, you are wrong", but that |
|
21:11
🔗
|
ola_norsk |
took me a second to write that line, sorry |
|
21:12
🔗
|
ola_norsk |
ill try downloading a file and see if its md5 |
|
21:12
🔗
|
ola_norsk |
wish the "headers" column wasnt so messy though lol |
|
21:13
🔗
|
bithippo |
Agree! And its okay, I don't mind being wrong :D |
|
21:15
🔗
|
ola_norsk |
that's the best kind of right there is :D |
|
21:15
🔗
|
bithippo |
lol |
|
21:16
🔗
|
bithippo |
Something cool, each file's crc32, md5, and sha1 hashes are available per item in the *_files.xml file within the item. |
|
21:16
🔗
|
bithippo |
Example: https://ia801609.us.archive.org/30/items/youtube-ARrNYyJEnFI/youtube-ARrNYyJEnFI_files.xml |
|
21:16
🔗
|
bithippo |
(xml format) |
|
21:16
🔗
|
ola_norsk |
thank you! |
|
21:18
🔗
|
bithippo |
Helpful if you know the item/item URL, but doesn't help if you want to search by hash, sorry about that! |
|
21:19
🔗
|
ola_norsk |
im just trying to fix one item, where i accidentally uploaded some stuff from a previous item :] |
|
21:20
🔗
|
ola_norsk |
though im unfortunatly sure if if the files are named the same, or how many of the first item ended up in the second item |
|
21:20
🔗
|
|
ranavalon has quit IRC (Read error: Connection reset by peer) |
|
21:20
🔗
|
ola_norsk |
not* |
|
21:21
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
21:22
🔗
|
ola_norsk |
the xml is gold stuff! so thanks again! :D |
|
21:22
🔗
|
bithippo |
no trouble at all! |
|
21:30
🔗
|
|
ranavalon has quit IRC (Remote host closed the connection) |
|
21:30
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
21:32
🔗
|
ola_norsk |
if i mv an (original) file from one item to another, the derivative files would get moved, deleted, regenerated as well? e.g would thumbnail files risk getting left behind? |
|
21:34
🔗
|
ola_norsk |
would it perhaps be wiser to just do a "ia copy" and then "ia delete", instead of "ia mv" ? |
|
21:35
🔗
|
|
ranavalon has quit IRC (Read error: Connection reset by peer) |
|
21:35
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
21:37
🔗
|
ola_norsk |
i saw in Internet Archive Help faq pages something a button with "Re-Derive", but i've yet to see it on an item |
|
21:37
🔗
|
|
jschwart has quit IRC (Konversation terminated!) |
|
21:39
🔗
|
|
BlueMaxim has joined #archiveteam-bs |
|
21:40
🔗
|
ola_norsk |
150.000 vidoes ... https://www.youtube.com/watch?v=Ih2CIdnHWms&ab_channel=MellisaHoneybeeZaccaria |
|
21:41
🔗
|
ola_norsk |
i guess child exploitation was sorely rampant... |
|
21:43
🔗
|
ola_norsk |
150K vidoes deleted. Though i have my doubts they were all "youtube kids videos targeted by child predators" |
|
21:43
🔗
|
ola_norsk |
oh well |
|
21:50
🔗
|
|
ranavalon has quit IRC (Read error: Connection reset by peer) |
|
21:50
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
22:02
🔗
|
|
ranavalon has quit IRC (Read error: Connection reset by peer) |
|
22:02
🔗
|
|
ranavalon has joined #archiveteam-bs |
|
22:08
🔗
|
ola_norsk |
is it just me, or does the statistics of 270 youtube (video?) accounts terminated, and 150K videoes deleted somewhat mismatch ? |
|
22:08
🔗
|
astrid |
lots of videos per account, seems reasonable to me |
|
22:10
🔗
|
ola_norsk |
in my experience it would be 250 videos per of these accounts. Going by the first one and foremost reknowned ones |
|
22:10
🔗
|
ola_norsk |
~250 to 300 |
|
22:11
🔗
|
ola_norsk |
but they did certainly kept popping back up, that is undeniable |
|
22:20
🔗
|
|
bithippo has quit IRC (Max SendQ exceeded) |
|
22:21
🔗
|
ola_norsk |
hopefully no "out baby lost his first tooth" family memrobilia got mingled into the debacle |
|
22:21
🔗
|
ola_norsk |
our* |
|
22:21
🔗
|
CoolCanuk |
oooh that would be bad! |
|
22:22
🔗
|
CoolCanuk |
first steps, etc |
|
22:22
🔗
|
ola_norsk |
aye |
|
22:26
🔗
|
|
RichardG_ is now known as RichardG |
|
22:30
🔗
|
|
Pixi has quit IRC (Quit: Pixi) |
|
22:30
🔗
|
|
Pixi has joined #archiveteam-bs |
|
22:44
🔗
|
|
icedice2 has joined #archiveteam-bs |
|
22:47
🔗
|
|
icedice has quit IRC (Ping timeout: 245 seconds) |
|
22:51
🔗
|
odemg |
Can we please get on this because it's going to be a shit show if we don't do it properly.... https://www.reddit.com/r/DataHoarder/comments/7gce4n/psndl_being_shut_down_if_not_enough_donations/ |
|
22:51
🔗
|
odemg |
arkiver schbirid2 chfoo balrog SketchCow ^^^ |
|
22:56
🔗
|
CoolCanuk |
so they use that site instead of paying $13 a month? |
|
22:57
🔗
|
CoolCanuk |
i'd contact this person and just ask if he'd let us download the files... https://github.com/jamesst20 |
|
22:58
🔗
|
CoolCanuk |
i dunno. Can you explain what PSNDL does? |
|
22:59
🔗
|
odemg |
I have no idea, I'm not in the know about ps3 or whatever, as far as I can tell it points to the official cdn and isn't serving the package files himself, the site just acts like a portal, stick it on archivebot just to grab the site itself, if we must we can concern ourselves over the packages later on (hosted by sony, not going anywhere anytime soon) |
|
23:01
🔗
|
dxrt |
Looks like JAA already grabbed it with AB. |
|
23:21
🔗
|
|
Stilett0 has quit IRC (Ping timeout: 250 seconds) |
|
23:33
🔗
|
|
Polylith has quit IRC (Quit: ZNC - http://znc.in) |
|
23:35
🔗
|
|
icedice2 has quit IRC (Quit: Leaving) |
|
23:36
🔗
|
ola_norsk |
may i ask what AB is? |
|
23:38
🔗
|
dxrt |
archivebot |
|
23:38
🔗
|
ola_norsk |
ty |
|
23:38
🔗
|
* |
ola_norsk is archive team n00b :) |
|
23:39
🔗
|
dxrt |
:) |
|
23:40
🔗
|
ola_norsk |
this reminds me, is there any way to circumenvent waybackmachine captures that's blocked by robots.txt? |
|
23:42
🔗
|
ola_norsk |
i actually think it was a news paper article that i tried to archive in relation to an item, that was blocked from being archived by their servers robots.txt |
|
23:42
🔗
|
zino |
Depends on how the data was grabbed. If archiveteam did it the raw warcs is probably around in a collection somewhere. |
|
23:42
🔗
|
zino |
Archivebot does not care about robots.txt |
|
23:43
🔗
|
ola_norsk |
how do i use the !ia ? |
|
23:43
🔗
|
ola_norsk |
i remember i was told how a while back |
|
23:43
🔗
|
ola_norsk |
!ia help |
|
23:44
🔗
|
zino |
You mean !a or !ao? Those are archivebot commands. See #archivebot. |
|
23:45
🔗
|
zino |
http://archivebot.readthedocs.io/en/latest/commands.html |
|
23:55
🔗
|
CoolCanuk |
how can we archive twitter/fb? |
|
23:55
🔗
|
CoolCanuk |
pages/profiles |
|
23:58
🔗
|
zino |
!a{o} https://twitter.com/somethingorother --igset twitter --phantomjs |
|
23:58
🔗
|
ola_norsk |
i've been running curl requests to web.archive.org/save/<url> to get the hashtags surrounding #netneutrality |
|
23:58
🔗
|
zino |
And then pray that phantomjs works on the pipeline that ends up on. |
|
23:58
🔗
|
ola_norsk |
would it do it twitter's own archives? |
|
23:59
🔗
|
zino |
Unsure what that means, and I'm going to go sleep now. |