#archiveteam-bs 2018-01-10,Wed

↑back Search

Time Nickname Message
00:16 πŸ”— godane SketchCow: maybe we fork this project: https://github.com/ikreymer/webarchiveplayer
00:17 πŸ”— godane we have to make search a folder cause right now you need to point to one warc.gz file
00:18 πŸ”— godane ok nevermind
00:18 πŸ”— godane looks like i can point to a folder and it will work
00:19 πŸ”— godane ok there is a bug
00:19 πŸ”— godane it needs full path of web archive
00:20 πŸ”— godane i copy a dump of my breitbart.com news index into a tmp does give me a index of the warc.gz
00:20 πŸ”— godane but not the links works
00:20 πŸ”— godane 2018-01-09 19:19:16,579: [INFO]: www.breitbart.com-news-index-20160314.warc.gz: Archive File Not Found
00:33 πŸ”— bwn has quit IRC (Read error: Operation timed out)
00:47 πŸ”— Valentin- has quit IRC (Read error: Operation timed out)
00:48 πŸ”— bwn has joined #archiveteam-bs
00:55 πŸ”— Valentine has joined #archiveteam-bs
01:07 πŸ”— zyphlar has joined #archiveteam-bs
01:09 πŸ”— zyphlar has left
01:12 πŸ”— RichardG has quit IRC (Ping timeout: 250 seconds)
01:16 πŸ”— RichardG has joined #archiveteam-bs
01:21 πŸ”— Valentine has quit IRC (Read error: Operation timed out)
01:32 πŸ”— Valentine has joined #archiveteam-bs
02:20 πŸ”— schbirid2 has joined #archiveteam-bs
02:24 πŸ”— schbirid has quit IRC (Read error: Operation timed out)
02:42 πŸ”— Ing3b0rg has quit IRC (Ping timeout: 260 seconds)
02:42 πŸ”— robink has quit IRC (Read error: Connection reset by peer)
02:42 πŸ”— robink has joined #archiveteam-bs
02:47 πŸ”— Ing3b0rg has joined #archiveteam-bs
03:21 πŸ”— k_o_ has quit IRC (Ping timeout: 260 seconds)
03:33 πŸ”— MatrixBri has joined #archiveteam-bs
03:34 πŸ”— zyphlar has joined #archiveteam-bs
03:37 πŸ”— zyphlar hmm
03:43 πŸ”— MatrixBri has quit IRC (Remote host closed the connection)
03:44 πŸ”— MatrixBri has joined #archiveteam-bs
03:44 πŸ”— root[m] has joined #archiveteam-bs
03:48 πŸ”— zyphlar ~~~~ bs ~~~~
03:56 πŸ”— root[m] has quit IRC (Remote host closed the connection)
03:56 πŸ”— MatrixBri has quit IRC (Remote host closed the connection)
03:56 πŸ”— MatrixBri has joined #archiveteam-bs
03:56 πŸ”— root[m] has joined #archiveteam-bs
03:57 πŸ”— root[m] ??
04:01 πŸ”— WillBradl has joined #archiveteam-bs
04:02 πŸ”— k_o has joined #archiveteam-bs
04:02 πŸ”— root[m] has left User left
04:08 πŸ”— WillBradl has quit IRC (Remote host closed the connection)
04:08 πŸ”— MatrixBri has quit IRC (Remote host closed the connection)
04:08 πŸ”— jacketcha has joined #archiveteam-bs
04:09 πŸ”— MatrixBri has joined #archiveteam-bs
04:09 πŸ”— WillBradl has joined #archiveteam-bs
04:09 πŸ”— WillBradl has quit IRC (Remote host closed the connection)
04:09 πŸ”— MatrixBri has quit IRC (Remote host closed the connection)
04:11 πŸ”— MatrixBri has joined #archiveteam-bs
04:12 πŸ”— WillBradl has joined #archiveteam-bs
04:12 πŸ”— WillBradl has quit IRC (Remote host closed the connection)
04:12 πŸ”— MatrixBri has quit IRC (Remote host closed the connection)
04:23 πŸ”— WillBradl has joined #archiveteam-bs
04:23 πŸ”— WillBradl has left
04:24 πŸ”— WillBradl has joined #archiveteam-bs
04:26 πŸ”— zyphlar okay
04:28 πŸ”— WillBradl is now known as M-WillBra
04:35 πŸ”— M-WillBra i think the BS is done
04:35 πŸ”— zyphlar has left
04:46 πŸ”— jacketcha ?
04:47 πŸ”— M-WillBra i set up a matrix.org bridge and this was my BS channel (also the one i care about joining <3 )
04:48 πŸ”— jacketcha cool
04:49 πŸ”— qw3rty15 has joined #archiveteam-bs
04:52 πŸ”— qw3rty14 has quit IRC (Read error: Operation timed out)
05:12 πŸ”— Asparagir has joined #archiveteam-bs
05:14 πŸ”— RichardG has quit IRC (Ping timeout: 245 seconds)
05:32 πŸ”— godane has quit IRC (Read error: Operation timed out)
06:01 πŸ”— Asparag-1 has joined #archiveteam-bs
06:03 πŸ”— jacketcha hey does anybody know the password for the archivebot logs at archive.fart.websiite?
06:03 πŸ”— Asparagir has quit IRC (Read error: Operation timed out)
06:33 πŸ”— k_o has quit IRC (Quit: Page closed)
06:45 πŸ”— zyphlar has joined #archiveteam-bs
06:49 πŸ”— zyphlar has left
07:13 πŸ”— godane has joined #archiveteam-bs
07:25 πŸ”— godane so i'm on my new comcast modem now
07:46 πŸ”— jacketcha how is it?
08:22 πŸ”— Asparagir has joined #archiveteam-bs
08:26 πŸ”— Asparag-1 has quit IRC (Ping timeout: 600 seconds)
08:32 πŸ”— PurpleSym SketchCow: I’ll update my scripts so they upload to collection:archiveteam_yahoogroups directly.
08:33 πŸ”— SketchCow Great
08:33 πŸ”— SketchCow Give them a logo if you can
08:38 πŸ”— schbirid2 has quit IRC (Quit: Leaving)
08:50 πŸ”— SilSte has quit IRC (Read error: Operation timed out)
09:08 πŸ”— Valentine has quit IRC (Ping timeout: 506 seconds)
09:25 πŸ”— Valentine has joined #archiveteam-bs
09:42 πŸ”— Valentine has quit IRC (Read error: Operation timed out)
09:46 πŸ”— RichardG has joined #archiveteam-bs
10:01 πŸ”— BlueMaxim has quit IRC (Leaving)
10:26 πŸ”— Asparagir has quit IRC (Ping timeout: 600 seconds)
10:29 πŸ”— Asparagir has joined #archiveteam-bs
10:46 πŸ”— Ing3b0rg has quit IRC (hub.dk irc.underworld.no)
10:46 πŸ”— Rai-chan has quit IRC (hub.dk irc.underworld.no)
10:46 πŸ”— i0npulse has quit IRC (hub.dk irc.underworld.no)
10:46 πŸ”— purplebot has quit IRC (hub.dk irc.underworld.no)
10:48 πŸ”— robink has quit IRC (Ping timeout: 246 seconds)
10:51 πŸ”— robink has joined #archiveteam-bs
10:53 πŸ”— Valentine has joined #archiveteam-bs
10:55 πŸ”— LeG0ax has joined #archiveteam-bs
11:02 πŸ”— LeG0ax is now known as Ing3b0rg
11:16 πŸ”— Valentine has quit IRC (Ping timeout: 506 seconds)
12:09 πŸ”— purplebot has joined #archiveteam-bs
12:11 πŸ”— Rai-chan has joined #archiveteam-bs
12:21 πŸ”— Asparag-1 has joined #archiveteam-bs
12:24 πŸ”— Asparagir has quit IRC (Read error: Operation timed out)
12:26 πŸ”— HCross2 has joined #archiveteam-bs
12:27 πŸ”— svchfoo3 sets mode: +o HCross2
12:33 πŸ”— medowar has joined #archiveteam-bs
13:14 πŸ”— Valentine has joined #archiveteam-bs
13:19 πŸ”— Mateon1 has quit IRC (Ping timeout: 255 seconds)
13:19 πŸ”— Mateon1 has joined #archiveteam-bs
13:57 πŸ”— PurpleSym SketchCow: Done. You might need to bulk-move a few items that were created before the change.
14:12 πŸ”— Lord_Nigh has quit IRC (Read error: Operation timed out)
14:12 πŸ”— Lord_Nigh has joined #archiveteam-bs
14:23 πŸ”— Asparagir has joined #archiveteam-bs
14:24 πŸ”— Asparag-1 has quit IRC (Read error: Operation timed out)
15:12 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
15:23 πŸ”— Valentin- has joined #archiveteam-bs
15:30 πŸ”— Valentin- has quit IRC (Read error: Connection reset by peer)
15:32 πŸ”— Valentine has joined #archiveteam-bs
15:33 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
15:37 πŸ”— Valentin- has joined #archiveteam-bs
15:41 πŸ”— Valentin- has quit IRC (Read error: Connection reset by peer)
15:48 πŸ”— Valentine has joined #archiveteam-bs
15:50 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
15:53 πŸ”— Asparagir has quit IRC (Asparagir)
15:54 πŸ”— Valentine has joined #archiveteam-bs
15:54 πŸ”— atrocity has joined #archiveteam-bs
15:57 πŸ”— tomaspark has joined #archiveteam-bs
16:06 πŸ”— Valentine has quit IRC (Read error: Operation timed out)
16:07 πŸ”— Valentine has joined #archiveteam-bs
16:11 πŸ”— SketchCow That's fine, I'll find them.
16:13 πŸ”— SketchCow So, Software just became more interesting, CD-ROM wise.
16:19 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
16:23 πŸ”— Valentine has joined #archiveteam-bs
16:43 πŸ”— Kimmer has quit IRC (Read error: Connection reset by peer)
17:49 πŸ”— schbirid has joined #archiveteam-bs
17:59 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:01 πŸ”— Valentine has joined #archiveteam-bs
18:11 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:20 πŸ”— Valentine has joined #archiveteam-bs
18:35 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:35 πŸ”— Valentine has joined #archiveteam-bs
18:40 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:40 πŸ”— Darkstar there is still no way for an uploader to change the collection an item has been put into, is there?
18:41 πŸ”— Valentine has joined #archiveteam-bs
18:42 πŸ”— astrid i think you can if you have access to the target collection
18:42 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:42 πŸ”— Valentine has joined #archiveteam-bs
18:44 πŸ”— Darkstar for me the whole "collection" input box is non-editable
18:44 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:46 πŸ”— astrid snag the "undisable" bookmarklet from here and then you can do it ;) https://www.squarefree.com/bookmarklets/forms.html
18:46 πŸ”— Darkstar hm, I don't really think that will work :)
18:46 πŸ”— astrid works for me!
18:47 πŸ”— Darkstar yes you can edit it then, but I don't think clicking "update" will apply this. It would be a security leak if it did ...
18:48 πŸ”— second has quit IRC (Read error: Connection reset by peer)
18:48 πŸ”— astrid idk it worked for me last time i did it
18:49 πŸ”— Darkstar ok let me try
18:50 πŸ”— second has joined #archiveteam-bs
18:51 πŸ”— SketchCow We just had the Reckoning of All Reckonings about uploading WARCs to the Archive
18:51 πŸ”— SketchCow Impromptu 8 person meeting on the slack, it was glorious
18:51 πŸ”— Valentine has joined #archiveteam-bs
18:52 πŸ”— SketchCow So, the default is that just anyone can't add WARCs to archive and it will automatically go into Wayback. That'll be only if I clear off an account's uploads.
18:52 πŸ”— Valentine has quit IRC (Read error: Connection reset by peer)
18:52 πŸ”— SketchCow So I need to find out who a couple people are.
18:52 πŸ”— SketchCow I don't want to drop their e-mails anywhere
18:52 πŸ”— Darkstar heh, seems that it indeed works, at least for putting it into open_source_software ... I still have no access to put it where it belongs though ;-)
18:52 πŸ”— Valentine has joined #archiveteam-bs
18:53 πŸ”— Darkstar but at least it's not in "media" anymore...
18:53 πŸ”— astrid Darkstar: i think SketchCow can help find the right collection maybe
18:54 πŸ”— Darkstar the right collection would be "cdromsoftware" or "cd-roms" (don't know the difference between those two), and "cdinstall" for some others, but I have not yet been able to find out how to get access
18:55 πŸ”— Darkstar I usually just wait, and after some days/weeks the items magically move to one of these collections (I think someone moves them, but they forget the occasional item from time to time ;-)
19:32 πŸ”— qw3rty15 has quit IRC (Nettalk6 - www.ntalk.de)
19:32 πŸ”— AeonG_ has quit IRC (Ping timeout: 600 seconds)
20:02 πŸ”— AeonG_ has joined #archiveteam-bs
20:20 πŸ”— Harzilein ola_norsk: i don't know about the api, but maybe they never intended to "drop" a file type? kinda like "unknown" is "nil" so it'd be removing a value instead of changing it
20:25 πŸ”— AeonG_ has quit IRC (Read error: Operation timed out)
20:27 πŸ”— ola_norsk has joined #archiveteam-bs
20:27 πŸ”— ola_norsk Harzilein: btw, would you happen to know what happens to Items flagged as "Broken" ?
20:29 πŸ”— ZexaronS has joined #archiveteam-bs
20:30 πŸ”— Harzilein ola_norsk: as i said, i know nothing at all
20:31 πŸ”— ola_norsk Harzilein: then we are two :D
20:31 πŸ”— Harzilein my completion was broken and i thought you were here already. i wrote: "ola_norsk: i don't know about the api, but maybe they never intended to "drop" a file type? kinda like "unknown" is "nil" so it'd be removing a value instead of changing it"
20:31 πŸ”— AeonG_ has joined #archiveteam-bs
20:34 πŸ”— ola_norsk ah sry. But what i notice that the item have become more messed up/broken, even since i posted that issue on github. Now suddenly there's not only a few .d64 roms wrongly detected as "DV Audio" and "DV Video". there were no zip files at that time.
20:35 πŸ”— ola_norsk Internet Archive is haunted! :[
20:35 πŸ”— ola_norsk lol
20:35 πŸ”— Harzilein the playlist is funny :D
20:35 πŸ”— Harzilein wonder what it's about these files that make it detect it as dv
20:36 πŸ”— ola_norsk aye
20:36 πŸ”— Harzilein ah, okay, the playlist is for "aac files"
20:36 πŸ”— godane ola_norsk: i have a squashfs file i uploaded that thinks its a wav file
20:37 πŸ”— Harzilein oh, no, it's for the oggs actually
20:37 πŸ”— godane https://archive.org/details/slackwarearm-14.2-20170906-kiwix
20:38 πŸ”— ola_norsk what could cause it though?
20:38 πŸ”— Harzilein for the simpsons and steve keene private spy ones, i can actually get a glitchy sound by playing and then seeking in them :D
20:38 πŸ”— Harzilein (with the web player)
20:39 πŸ”— Harzilein the zak one does not seem to have even valid frames ;)
20:40 πŸ”— ola_norsk it must be some kind of fileheader/data detection thingy that does it?
20:40 πŸ”— Harzilein ola_norsk: so only the uploader can change the types through the web interface?
20:41 πŸ”— ola_norsk Harzilein: it's possible to do it by using _ia_ tool, but like the issue shows there seems to be a bug
20:42 πŸ”— Harzilein ola_norsk: so i think what happened is that 106 files got mis-detected as aac, then of those 3 happened to contain at least one good aac frame in them
20:43 πŸ”— Harzilein <ola_norsk> Harzilein: sadly the issue is that choosing format trough the web gui for 2000+ files is something i gave up on trying to do.
20:43 πŸ”— ola_norsk there were no where near 106 a while back :D
20:43 πŸ”— Harzilein it's just that i don't see that option anywhere in the web gui
20:44 πŸ”— ola_norsk it's a drop down menu at every file
20:44 πŸ”— Harzilein the good thing is that the name did not get changed. so the system saw an opportunity to provide unencumbered oggs instead
20:45 πŸ”— Harzilein when i click on "show all" i only get the download view
20:47 πŸ”— ola_norsk what seems to have gone wrong is that "Format" was wrongly detected/set on some during upload.
20:48 πŸ”— ola_norsk even if name extension was the same. My hope was to set all to same "Unknown" format metadata by using
20:48 πŸ”— ola_norsk ia metadata 2813_d64_C64_roms_wwwC64com --target="Metal_Warrior_3.d64" --modify="format:Unknown"
20:48 πŸ”— Harzilein yeah. but you mentioned that isn't possible due to an api bug.
20:49 πŸ”— Harzilein but you also mentioned that it'd be cumbersome but possible to change the type in the web gui
20:49 πŸ”— ola_norsk it is
20:49 πŸ”— Harzilein but i can't find that gui element and i assume that's because i'm not the uploader
20:49 πŸ”— ola_norsk yeah, that is done in "Edit Item/Edit files"
20:50 πŸ”— AeonG_ has quit IRC (Read error: Operation timed out)
20:51 πŸ”— ola_norsk if you check one of your own uploaded items, by cliking "Edit" > "Edit metadata", it will be on the bottom of that page.
20:54 πŸ”— Asparagir has joined #archiveteam-bs
20:54 πŸ”— ola_norsk to be fair though, Internet Archive does have a warning against items containing >1000 files i think :D
20:55 πŸ”— astrid yeah it hard limits you somewhere around 1000 i think
20:55 πŸ”— Harzilein i have a slightly different interface (i kind of get the "non-expert" wording: "Edit" > "I want to change the information (metadata) about my item.
20:55 πŸ”— Harzilein For example, I want to change my [title] or [description]."
20:55 πŸ”— Harzilein )
20:55 πŸ”— ola_norsk astrid: i didn't hard limit my item :/
20:55 πŸ”— ola_norsk astrid: it*
20:55 πŸ”— astrid how many files did you put into it?
20:56 πŸ”— ola_norsk 2813
20:56 πŸ”— astrid ok i suspect there's a limit somewhere but idk where it is
20:56 πŸ”— ola_norsk most seems to be ok
20:57 πŸ”— ola_norsk it seemed to me as it's just a caution notice/warning
20:57 πŸ”— ola_norsk one sec
20:58 πŸ”— Asparagir has quit IRC (Client Quit)
20:58 πŸ”— ola_norsk "Because items can "break" we typically recommend that you not exceed 1,000 files and/or 50GB per item page."
20:58 πŸ”— ola_norsk so i think i should've heeded that :D
20:59 πŸ”— ola_norsk that, or not have done them all in one upload session
21:00 πŸ”— Harzilein ola_norsk: i think you should try checking "block archive.php from queueing a derive (needed only in special circumstances; if you’re unsure, leave blank)" next time you initially upload d64 images
21:00 πŸ”— ola_norsk i figured since the files were so miniscule, it would be ok
21:01 πŸ”— ola_norsk Harzilein: doesn't that just apply to audio (and or video) files?
21:01 πŸ”— Harzilein ola_norsk: i'm pretty sure the files uploaded fine, but whatever tries to make oggs for the web player might have too aggressively checked the file types
21:02 πŸ”— Harzilein ola_norsk: for all the system knows they _are_ audio files w/ a weird extension ;)
21:02 πŸ”— AeonG_ has joined #archiveteam-bs
21:02 πŸ”— ola_norsk ill try that then
21:03 πŸ”— Harzilein ola_norsk: i'm kind of curious what caused it, let me download the simpsons, zak mc cracken and steve keene private spy items
21:04 πŸ”— ola_norsk it must have been some heuristic detection thingy that did it
21:04 πŸ”— Harzilein perhaps there's such a thing as "raw aac frames"?
21:05 πŸ”— Harzilein then every file that roughly fits some constraint could be a glitchy aac
21:05 πŸ”— ola_norsk some of the d64 did not even pass virus checking at upload
21:05 πŸ”— ola_norsk that's why there's not 2813 of them
21:06 πŸ”— Asparagir has joined #archiveteam-bs
21:06 πŸ”— Harzilein "In addition to the MP4, 3GP and other container formats based on ISO base media file format for file storage, AAC audio data was first packaged in a file for the MPEG-2 standard using Audio Data Interchange Format (ADIF),[43] consisting of a single header followed by the raw AAC audio data blocks."
21:10 πŸ”— ola_norsk maybe if running a diff comparance on a couple of files, including godane's slackware squashfs file..one might find where similarity comparing failed?
21:11 πŸ”— Harzilein well, the decoding fails really early, hence the very small derived files
21:11 πŸ”— BlueMaxim has joined #archiveteam-bs
21:12 πŸ”— Harzilein and i'd expect the values in the derived frames to be severely clipped too
21:14 πŸ”— Harzilein Zak_Mckracken_and_the_Alien_Mindbenders_[Boot].d64 file info:
21:14 πŸ”— Harzilein RAW
21:14 πŸ”— Harzilein Error: Bitstream value not allowed by specification
21:14 πŸ”— ola_norsk how does Internet Archive try to detect filetypes though?
21:15 πŸ”— ola_norsk what software used, i mean
21:17 πŸ”— ola_norsk it would be lovely if it was as easy to just run queries on the item's sqlite database hehe
21:18 πŸ”— AeonG_ has quit IRC (Ping timeout: 633 seconds)
21:28 πŸ”— second has quit IRC (Read error: Connection reset by peer)
21:30 πŸ”— ola_norsk in hinsight, i think it might've been smarter of me to use the torrent upload ability of IA on that item
21:30 πŸ”— second has joined #archiveteam-bs
21:33 πŸ”— ola_norsk damn, even the item's torrent is broken :D
21:41 πŸ”— ola_norsk i'll just wait and see what happens now that's it's flagged as Broken item
22:16 πŸ”— schbirid has quit IRC (Quit: Leaving)
22:31 πŸ”— RichardG has quit IRC (Ping timeout: 506 seconds)
22:36 πŸ”— godane has quit IRC (Read error: Operation timed out)
22:45 πŸ”— dashcloud has joined #archiveteam-bs
22:47 πŸ”— godane has joined #archiveteam-bs
23:16 πŸ”— qw3rty15 has joined #archiveteam-bs
23:17 πŸ”— BartoCH has quit IRC (Ping timeout: 260 seconds)
23:27 πŸ”— AeonG_ has joined #archiveteam-bs
23:30 πŸ”— AeonG__ has joined #archiveteam-bs
23:38 πŸ”— AeonG_ has quit IRC (Read error: Operation timed out)
23:46 πŸ”— RichardG has joined #archiveteam-bs

irclogger-viewer