Time |
Nickname |
Message |
00:16
π
|
godane |
SketchCow: maybe we fork this project: https://github.com/ikreymer/webarchiveplayer |
00:17
π
|
godane |
we have to make search a folder cause right now you need to point to one warc.gz file |
00:18
π
|
godane |
ok nevermind |
00:18
π
|
godane |
looks like i can point to a folder and it will work |
00:19
π
|
godane |
ok there is a bug |
00:19
π
|
godane |
it needs full path of web archive |
00:20
π
|
godane |
i copy a dump of my breitbart.com news index into a tmp does give me a index of the warc.gz |
00:20
π
|
godane |
but not the links works |
00:20
π
|
godane |
2018-01-09 19:19:16,579: [INFO]: www.breitbart.com-news-index-20160314.warc.gz: Archive File Not Found |
00:33
π
|
|
bwn has quit IRC (Read error: Operation timed out) |
00:47
π
|
|
Valentin- has quit IRC (Read error: Operation timed out) |
00:48
π
|
|
bwn has joined #archiveteam-bs |
00:55
π
|
|
Valentine has joined #archiveteam-bs |
01:07
π
|
|
zyphlar has joined #archiveteam-bs |
01:09
π
|
|
zyphlar has left |
01:12
π
|
|
RichardG has quit IRC (Ping timeout: 250 seconds) |
01:16
π
|
|
RichardG has joined #archiveteam-bs |
01:21
π
|
|
Valentine has quit IRC (Read error: Operation timed out) |
01:32
π
|
|
Valentine has joined #archiveteam-bs |
02:20
π
|
|
schbirid2 has joined #archiveteam-bs |
02:24
π
|
|
schbirid has quit IRC (Read error: Operation timed out) |
02:42
π
|
|
Ing3b0rg has quit IRC (Ping timeout: 260 seconds) |
02:42
π
|
|
robink has quit IRC (Read error: Connection reset by peer) |
02:42
π
|
|
robink has joined #archiveteam-bs |
02:47
π
|
|
Ing3b0rg has joined #archiveteam-bs |
03:21
π
|
|
k_o_ has quit IRC (Ping timeout: 260 seconds) |
03:33
π
|
|
MatrixBri has joined #archiveteam-bs |
03:34
π
|
|
zyphlar has joined #archiveteam-bs |
03:37
π
|
zyphlar |
hmm |
03:43
π
|
|
MatrixBri has quit IRC (Remote host closed the connection) |
03:44
π
|
|
MatrixBri has joined #archiveteam-bs |
03:44
π
|
|
root[m] has joined #archiveteam-bs |
03:48
π
|
zyphlar |
~~~~ bs ~~~~ |
03:56
π
|
|
root[m] has quit IRC (Remote host closed the connection) |
03:56
π
|
|
MatrixBri has quit IRC (Remote host closed the connection) |
03:56
π
|
|
MatrixBri has joined #archiveteam-bs |
03:56
π
|
|
root[m] has joined #archiveteam-bs |
03:57
π
|
root[m] |
?? |
04:01
π
|
|
WillBradl has joined #archiveteam-bs |
04:02
π
|
|
k_o has joined #archiveteam-bs |
04:02
π
|
|
root[m] has left User left |
04:08
π
|
|
WillBradl has quit IRC (Remote host closed the connection) |
04:08
π
|
|
MatrixBri has quit IRC (Remote host closed the connection) |
04:08
π
|
|
jacketcha has joined #archiveteam-bs |
04:09
π
|
|
MatrixBri has joined #archiveteam-bs |
04:09
π
|
|
WillBradl has joined #archiveteam-bs |
04:09
π
|
|
WillBradl has quit IRC (Remote host closed the connection) |
04:09
π
|
|
MatrixBri has quit IRC (Remote host closed the connection) |
04:11
π
|
|
MatrixBri has joined #archiveteam-bs |
04:12
π
|
|
WillBradl has joined #archiveteam-bs |
04:12
π
|
|
WillBradl has quit IRC (Remote host closed the connection) |
04:12
π
|
|
MatrixBri has quit IRC (Remote host closed the connection) |
04:23
π
|
|
WillBradl has joined #archiveteam-bs |
04:23
π
|
|
WillBradl has left |
04:24
π
|
|
WillBradl has joined #archiveteam-bs |
04:26
π
|
zyphlar |
okay |
04:28
π
|
|
WillBradl is now known as M-WillBra |
04:35
π
|
M-WillBra |
i think the BS is done |
04:35
π
|
|
zyphlar has left |
04:46
π
|
jacketcha |
? |
04:47
π
|
M-WillBra |
i set up a matrix.org bridge and this was my BS channel (also the one i care about joining <3 ) |
04:48
π
|
jacketcha |
cool |
04:49
π
|
|
qw3rty15 has joined #archiveteam-bs |
04:52
π
|
|
qw3rty14 has quit IRC (Read error: Operation timed out) |
05:12
π
|
|
Asparagir has joined #archiveteam-bs |
05:14
π
|
|
RichardG has quit IRC (Ping timeout: 245 seconds) |
05:32
π
|
|
godane has quit IRC (Read error: Operation timed out) |
06:01
π
|
|
Asparag-1 has joined #archiveteam-bs |
06:03
π
|
jacketcha |
hey does anybody know the password for the archivebot logs at archive.fart.websiite? |
06:03
π
|
|
Asparagir has quit IRC (Read error: Operation timed out) |
06:33
π
|
|
k_o has quit IRC (Quit: Page closed) |
06:45
π
|
|
zyphlar has joined #archiveteam-bs |
06:49
π
|
|
zyphlar has left |
07:13
π
|
|
godane has joined #archiveteam-bs |
07:25
π
|
godane |
so i'm on my new comcast modem now |
07:46
π
|
jacketcha |
how is it? |
08:22
π
|
|
Asparagir has joined #archiveteam-bs |
08:26
π
|
|
Asparag-1 has quit IRC (Ping timeout: 600 seconds) |
08:32
π
|
PurpleSym |
SketchCow: Iβll update my scripts so they upload to collection:archiveteam_yahoogroups directly. |
08:33
π
|
SketchCow |
Great |
08:33
π
|
SketchCow |
Give them a logo if you can |
08:38
π
|
|
schbirid2 has quit IRC (Quit: Leaving) |
08:50
π
|
|
SilSte has quit IRC (Read error: Operation timed out) |
09:08
π
|
|
Valentine has quit IRC (Ping timeout: 506 seconds) |
09:25
π
|
|
Valentine has joined #archiveteam-bs |
09:42
π
|
|
Valentine has quit IRC (Read error: Operation timed out) |
09:46
π
|
|
RichardG has joined #archiveteam-bs |
10:01
π
|
|
BlueMaxim has quit IRC (Leaving) |
10:26
π
|
|
Asparagir has quit IRC (Ping timeout: 600 seconds) |
10:29
π
|
|
Asparagir has joined #archiveteam-bs |
10:46
π
|
|
Ing3b0rg has quit IRC (hub.dk irc.underworld.no) |
10:46
π
|
|
Rai-chan has quit IRC (hub.dk irc.underworld.no) |
10:46
π
|
|
i0npulse has quit IRC (hub.dk irc.underworld.no) |
10:46
π
|
|
purplebot has quit IRC (hub.dk irc.underworld.no) |
10:48
π
|
|
robink has quit IRC (Ping timeout: 246 seconds) |
10:51
π
|
|
robink has joined #archiveteam-bs |
10:53
π
|
|
Valentine has joined #archiveteam-bs |
10:55
π
|
|
LeG0ax has joined #archiveteam-bs |
11:02
π
|
|
LeG0ax is now known as Ing3b0rg |
11:16
π
|
|
Valentine has quit IRC (Ping timeout: 506 seconds) |
12:09
π
|
|
purplebot has joined #archiveteam-bs |
12:11
π
|
|
Rai-chan has joined #archiveteam-bs |
12:21
π
|
|
Asparag-1 has joined #archiveteam-bs |
12:24
π
|
|
Asparagir has quit IRC (Read error: Operation timed out) |
12:26
π
|
|
HCross2 has joined #archiveteam-bs |
12:27
π
|
|
svchfoo3 sets mode: +o HCross2 |
12:33
π
|
|
medowar has joined #archiveteam-bs |
13:14
π
|
|
Valentine has joined #archiveteam-bs |
13:19
π
|
|
Mateon1 has quit IRC (Ping timeout: 255 seconds) |
13:19
π
|
|
Mateon1 has joined #archiveteam-bs |
13:57
π
|
PurpleSym |
SketchCow: Done. You might need to bulk-move a few items that were created before the change. |
14:12
π
|
|
Lord_Nigh has quit IRC (Read error: Operation timed out) |
14:12
π
|
|
Lord_Nigh has joined #archiveteam-bs |
14:23
π
|
|
Asparagir has joined #archiveteam-bs |
14:24
π
|
|
Asparag-1 has quit IRC (Read error: Operation timed out) |
15:12
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
15:23
π
|
|
Valentin- has joined #archiveteam-bs |
15:30
π
|
|
Valentin- has quit IRC (Read error: Connection reset by peer) |
15:32
π
|
|
Valentine has joined #archiveteam-bs |
15:33
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
15:37
π
|
|
Valentin- has joined #archiveteam-bs |
15:41
π
|
|
Valentin- has quit IRC (Read error: Connection reset by peer) |
15:48
π
|
|
Valentine has joined #archiveteam-bs |
15:50
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
15:53
π
|
|
Asparagir has quit IRC (Asparagir) |
15:54
π
|
|
Valentine has joined #archiveteam-bs |
15:54
π
|
|
atrocity has joined #archiveteam-bs |
15:57
π
|
|
tomaspark has joined #archiveteam-bs |
16:06
π
|
|
Valentine has quit IRC (Read error: Operation timed out) |
16:07
π
|
|
Valentine has joined #archiveteam-bs |
16:11
π
|
SketchCow |
That's fine, I'll find them. |
16:13
π
|
SketchCow |
So, Software just became more interesting, CD-ROM wise. |
16:19
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
16:23
π
|
|
Valentine has joined #archiveteam-bs |
16:43
π
|
|
Kimmer has quit IRC (Read error: Connection reset by peer) |
17:49
π
|
|
schbirid has joined #archiveteam-bs |
17:59
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:01
π
|
|
Valentine has joined #archiveteam-bs |
18:11
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:20
π
|
|
Valentine has joined #archiveteam-bs |
18:35
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:35
π
|
|
Valentine has joined #archiveteam-bs |
18:40
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:40
π
|
Darkstar |
there is still no way for an uploader to change the collection an item has been put into, is there? |
18:41
π
|
|
Valentine has joined #archiveteam-bs |
18:42
π
|
astrid |
i think you can if you have access to the target collection |
18:42
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:42
π
|
|
Valentine has joined #archiveteam-bs |
18:44
π
|
Darkstar |
for me the whole "collection" input box is non-editable |
18:44
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:46
π
|
astrid |
snag the "undisable" bookmarklet from here and then you can do it ;) https://www.squarefree.com/bookmarklets/forms.html |
18:46
π
|
Darkstar |
hm, I don't really think that will work :) |
18:46
π
|
astrid |
works for me! |
18:47
π
|
Darkstar |
yes you can edit it then, but I don't think clicking "update" will apply this. It would be a security leak if it did ... |
18:48
π
|
|
second has quit IRC (Read error: Connection reset by peer) |
18:48
π
|
astrid |
idk it worked for me last time i did it |
18:49
π
|
Darkstar |
ok let me try |
18:50
π
|
|
second has joined #archiveteam-bs |
18:51
π
|
SketchCow |
We just had the Reckoning of All Reckonings about uploading WARCs to the Archive |
18:51
π
|
SketchCow |
Impromptu 8 person meeting on the slack, it was glorious |
18:51
π
|
|
Valentine has joined #archiveteam-bs |
18:52
π
|
SketchCow |
So, the default is that just anyone can't add WARCs to archive and it will automatically go into Wayback. That'll be only if I clear off an account's uploads. |
18:52
π
|
|
Valentine has quit IRC (Read error: Connection reset by peer) |
18:52
π
|
SketchCow |
So I need to find out who a couple people are. |
18:52
π
|
SketchCow |
I don't want to drop their e-mails anywhere |
18:52
π
|
Darkstar |
heh, seems that it indeed works, at least for putting it into open_source_software ... I still have no access to put it where it belongs though ;-) |
18:52
π
|
|
Valentine has joined #archiveteam-bs |
18:53
π
|
Darkstar |
but at least it's not in "media" anymore... |
18:53
π
|
astrid |
Darkstar: i think SketchCow can help find the right collection maybe |
18:54
π
|
Darkstar |
the right collection would be "cdromsoftware" or "cd-roms" (don't know the difference between those two), and "cdinstall" for some others, but I have not yet been able to find out how to get access |
18:55
π
|
Darkstar |
I usually just wait, and after some days/weeks the items magically move to one of these collections (I think someone moves them, but they forget the occasional item from time to time ;-) |
19:32
π
|
|
qw3rty15 has quit IRC (Nettalk6 - www.ntalk.de) |
19:32
π
|
|
AeonG_ has quit IRC (Ping timeout: 600 seconds) |
20:02
π
|
|
AeonG_ has joined #archiveteam-bs |
20:20
π
|
Harzilein |
ola_norsk: i don't know about the api, but maybe they never intended to "drop" a file type? kinda like "unknown" is "nil" so it'd be removing a value instead of changing it |
20:25
π
|
|
AeonG_ has quit IRC (Read error: Operation timed out) |
20:27
π
|
|
ola_norsk has joined #archiveteam-bs |
20:27
π
|
ola_norsk |
Harzilein: btw, would you happen to know what happens to Items flagged as "Broken" ? |
20:29
π
|
|
ZexaronS has joined #archiveteam-bs |
20:30
π
|
Harzilein |
ola_norsk: as i said, i know nothing at all |
20:31
π
|
ola_norsk |
Harzilein: then we are two :D |
20:31
π
|
Harzilein |
my completion was broken and i thought you were here already. i wrote: "ola_norsk: i don't know about the api, but maybe they never intended to "drop" a file type? kinda like "unknown" is "nil" so it'd be removing a value instead of changing it" |
20:31
π
|
|
AeonG_ has joined #archiveteam-bs |
20:34
π
|
ola_norsk |
ah sry. But what i notice that the item have become more messed up/broken, even since i posted that issue on github. Now suddenly there's not only a few .d64 roms wrongly detected as "DV Audio" and "DV Video". there were no zip files at that time. |
20:35
π
|
ola_norsk |
Internet Archive is haunted! :[ |
20:35
π
|
ola_norsk |
lol |
20:35
π
|
Harzilein |
the playlist is funny :D |
20:35
π
|
Harzilein |
wonder what it's about these files that make it detect it as dv |
20:36
π
|
ola_norsk |
aye |
20:36
π
|
Harzilein |
ah, okay, the playlist is for "aac files" |
20:36
π
|
godane |
ola_norsk: i have a squashfs file i uploaded that thinks its a wav file |
20:37
π
|
Harzilein |
oh, no, it's for the oggs actually |
20:37
π
|
godane |
https://archive.org/details/slackwarearm-14.2-20170906-kiwix |
20:38
π
|
ola_norsk |
what could cause it though? |
20:38
π
|
Harzilein |
for the simpsons and steve keene private spy ones, i can actually get a glitchy sound by playing and then seeking in them :D |
20:38
π
|
Harzilein |
(with the web player) |
20:39
π
|
Harzilein |
the zak one does not seem to have even valid frames ;) |
20:40
π
|
ola_norsk |
it must be some kind of fileheader/data detection thingy that does it? |
20:40
π
|
Harzilein |
ola_norsk: so only the uploader can change the types through the web interface? |
20:41
π
|
ola_norsk |
Harzilein: it's possible to do it by using _ia_ tool, but like the issue shows there seems to be a bug |
20:42
π
|
Harzilein |
ola_norsk: so i think what happened is that 106 files got mis-detected as aac, then of those 3 happened to contain at least one good aac frame in them |
20:43
π
|
Harzilein |
<ola_norsk> Harzilein: sadly the issue is that choosing format trough the web gui for 2000+ files is something i gave up on trying to do. |
20:43
π
|
ola_norsk |
there were no where near 106 a while back :D |
20:43
π
|
Harzilein |
it's just that i don't see that option anywhere in the web gui |
20:44
π
|
ola_norsk |
it's a drop down menu at every file |
20:44
π
|
Harzilein |
the good thing is that the name did not get changed. so the system saw an opportunity to provide unencumbered oggs instead |
20:45
π
|
Harzilein |
when i click on "show all" i only get the download view |
20:47
π
|
ola_norsk |
what seems to have gone wrong is that "Format" was wrongly detected/set on some during upload. |
20:48
π
|
ola_norsk |
even if name extension was the same. My hope was to set all to same "Unknown" format metadata by using |
20:48
π
|
ola_norsk |
ia metadata 2813_d64_C64_roms_wwwC64com --target="Metal_Warrior_3.d64" --modify="format:Unknown" |
20:48
π
|
Harzilein |
yeah. but you mentioned that isn't possible due to an api bug. |
20:49
π
|
Harzilein |
but you also mentioned that it'd be cumbersome but possible to change the type in the web gui |
20:49
π
|
ola_norsk |
it is |
20:49
π
|
Harzilein |
but i can't find that gui element and i assume that's because i'm not the uploader |
20:49
π
|
ola_norsk |
yeah, that is done in "Edit Item/Edit files" |
20:50
π
|
|
AeonG_ has quit IRC (Read error: Operation timed out) |
20:51
π
|
ola_norsk |
if you check one of your own uploaded items, by cliking "Edit" > "Edit metadata", it will be on the bottom of that page. |
20:54
π
|
|
Asparagir has joined #archiveteam-bs |
20:54
π
|
ola_norsk |
to be fair though, Internet Archive does have a warning against items containing >1000 files i think :D |
20:55
π
|
astrid |
yeah it hard limits you somewhere around 1000 i think |
20:55
π
|
Harzilein |
i have a slightly different interface (i kind of get the "non-expert" wording: "Edit" > "I want to change the information (metadata) about my item. |
20:55
π
|
Harzilein |
For example, I want to change my [title] or [description]." |
20:55
π
|
Harzilein |
) |
20:55
π
|
ola_norsk |
astrid: i didn't hard limit my item :/ |
20:55
π
|
ola_norsk |
astrid: it* |
20:55
π
|
astrid |
how many files did you put into it? |
20:56
π
|
ola_norsk |
2813 |
20:56
π
|
astrid |
ok i suspect there's a limit somewhere but idk where it is |
20:56
π
|
ola_norsk |
most seems to be ok |
20:57
π
|
ola_norsk |
it seemed to me as it's just a caution notice/warning |
20:57
π
|
ola_norsk |
one sec |
20:58
π
|
|
Asparagir has quit IRC (Client Quit) |
20:58
π
|
ola_norsk |
"Because items can "break" we typically recommend that you not exceed 1,000 files and/or 50GB per item page." |
20:58
π
|
ola_norsk |
so i think i should've heeded that :D |
20:59
π
|
ola_norsk |
that, or not have done them all in one upload session |
21:00
π
|
Harzilein |
ola_norsk: i think you should try checking "block archive.php from queueing a derive (needed only in special circumstances; if youβre unsure, leave blank)" next time you initially upload d64 images |
21:00
π
|
ola_norsk |
i figured since the files were so miniscule, it would be ok |
21:01
π
|
ola_norsk |
Harzilein: doesn't that just apply to audio (and or video) files? |
21:01
π
|
Harzilein |
ola_norsk: i'm pretty sure the files uploaded fine, but whatever tries to make oggs for the web player might have too aggressively checked the file types |
21:02
π
|
Harzilein |
ola_norsk: for all the system knows they _are_ audio files w/ a weird extension ;) |
21:02
π
|
|
AeonG_ has joined #archiveteam-bs |
21:02
π
|
ola_norsk |
ill try that then |
21:03
π
|
Harzilein |
ola_norsk: i'm kind of curious what caused it, let me download the simpsons, zak mc cracken and steve keene private spy items |
21:04
π
|
ola_norsk |
it must have been some heuristic detection thingy that did it |
21:04
π
|
Harzilein |
perhaps there's such a thing as "raw aac frames"? |
21:05
π
|
Harzilein |
then every file that roughly fits some constraint could be a glitchy aac |
21:05
π
|
ola_norsk |
some of the d64 did not even pass virus checking at upload |
21:05
π
|
ola_norsk |
that's why there's not 2813 of them |
21:06
π
|
|
Asparagir has joined #archiveteam-bs |
21:06
π
|
Harzilein |
"In addition to the MP4, 3GP and other container formats based on ISO base media file format for file storage, AAC audio data was first packaged in a file for the MPEG-2 standard using Audio Data Interchange Format (ADIF),[43] consisting of a single header followed by the raw AAC audio data blocks." |
21:10
π
|
ola_norsk |
maybe if running a diff comparance on a couple of files, including godane's slackware squashfs file..one might find where similarity comparing failed? |
21:11
π
|
Harzilein |
well, the decoding fails really early, hence the very small derived files |
21:11
π
|
|
BlueMaxim has joined #archiveteam-bs |
21:12
π
|
Harzilein |
and i'd expect the values in the derived frames to be severely clipped too |
21:14
π
|
Harzilein |
Zak_Mckracken_and_the_Alien_Mindbenders_[Boot].d64 file info: |
21:14
π
|
Harzilein |
RAW |
21:14
π
|
Harzilein |
Error: Bitstream value not allowed by specification |
21:14
π
|
ola_norsk |
how does Internet Archive try to detect filetypes though? |
21:15
π
|
ola_norsk |
what software used, i mean |
21:17
π
|
ola_norsk |
it would be lovely if it was as easy to just run queries on the item's sqlite database hehe |
21:18
π
|
|
AeonG_ has quit IRC (Ping timeout: 633 seconds) |
21:28
π
|
|
second has quit IRC (Read error: Connection reset by peer) |
21:30
π
|
ola_norsk |
in hinsight, i think it might've been smarter of me to use the torrent upload ability of IA on that item |
21:30
π
|
|
second has joined #archiveteam-bs |
21:33
π
|
ola_norsk |
damn, even the item's torrent is broken :D |
21:41
π
|
ola_norsk |
i'll just wait and see what happens now that's it's flagged as Broken item |
22:16
π
|
|
schbirid has quit IRC (Quit: Leaving) |
22:31
π
|
|
RichardG has quit IRC (Ping timeout: 506 seconds) |
22:36
π
|
|
godane has quit IRC (Read error: Operation timed out) |
22:45
π
|
|
dashcloud has joined #archiveteam-bs |
22:47
π
|
|
godane has joined #archiveteam-bs |
23:16
π
|
|
qw3rty15 has joined #archiveteam-bs |
23:17
π
|
|
BartoCH has quit IRC (Ping timeout: 260 seconds) |
23:27
π
|
|
AeonG_ has joined #archiveteam-bs |
23:30
π
|
|
AeonG__ has joined #archiveteam-bs |
23:38
π
|
|
AeonG_ has quit IRC (Read error: Operation timed out) |
23:46
π
|
|
RichardG has joined #archiveteam-bs |