[00:00] actually the 25th is the only day that account has anything uploaded [00:00] Yeah, in tests and other things the script has been reliable. [00:01] so it hasn't uploaded anything today. did you doublecheck your config files, access key, etc [00:01] I regen'd my key a day or so ago, but I'll try again. [00:02] if you regenerated your key but forgot to tell the script, that'd do it :) [00:02] tip: add a setting to the script to upload items into test_collection [00:02] No, I'm pretty sure the key is right but just incase it bugged. [00:04] *** BlueMax has joined #archiveteam-bs [00:04] Nope, new keys, definitely in the file, 503. [00:04] Wait a minute. [00:06] So that makes total sense. [00:07] Skipped the item it was attempting to create, the one after worked fine. [00:08] Which makes me think a metadata problem of some kind because a bytestream shouldn't be a problem. [00:11] https://www.youtube.com/watch?v=pKJ4atlC2fA is the one that was failing. The only thing that would trip something up I guess is the description (non alphanumeric), but that doesn't seem all that different to what I just uploaded. [00:12] I definitely encode the headers as well so why would that be a problem? [00:12] *** ta9le has quit IRC (Quit: Connection closed for inactivity) [00:12] make sure you urlencoded the description correctly [00:12] One moment, I'll grab the header. [00:14] x-archive-meta-description:uri(Post%20this%20in%20the%20thread%20whenever%20asks%20for%20character%20creation%20tips.%0A%0ATry%20this%20at%20home%3A%20http%3A%2F%2Fwww.loverslab.com%2Ftopic%2F21438-illustrated-tips-guidelines-making-beautiful-female-characters%2F%3Fp%3D520411%0A%0AMusic%0ALesnik%20-%20Hot%20Dogs%3A%20https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DJVqk5APBkQs%0AEdzes%20-%20Fish%20and%20Chips%3A%20https%3A%2F%2Fwww.youtube.com% [00:14] 2Fwatch%3Fv%3DoAA8C2nQqUA%0ACrome%20-%20Nova%20Superb%3A%20https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DD8ATvKpdoTw) [00:25] *** TC01 has joined #archiveteam-bs [00:28] "first path segment in URL cannot contain colon" [00:28] i... don't see a colon? [00:28] Well shit, where does it say that? [00:28] www.youtube.com%5:14 PM 2Fwa [00:29] oh that's from the irc client [00:29] Haha. [00:30] decoded fine [00:30] I wonder if the decoder on IA's end could be bugged, but then how would this not have already been found? [00:32] nah it's fine [00:32] something else's worng [00:33] A sequence of certain characters not being escaped properly maybe? This is strange. [00:34] i mean that all looked right so probably not the urlencoding [00:35] Sorry, I mean in whatever part of the S3 server handles metadata. [00:35] is it just that one failing? [00:36] They are supposed to be uploaded in order of their original upload on YT, because the Date Archived sorter on the site doesn't take the date field into account. [00:37] I'm guessing they probably would though, the 14 before that one were fine. [00:38] would work though* [00:51] Does S3 do anything special with URLs? Like send them to Wayback or something? [00:58] *** DragonMon has joined #archiveteam-bs [00:59] riking, it's definitely the description, I just blanked it and it works. [00:59] i do think the description is supposed to be html [00:59] How do you mean? [01:00] like
tags and stuff [01:00] actually disregard that [01:00] anyways you don't actually have any <> in there [01:00] Even then they would be encoded you would think. [01:01] yeah but you'd need to encode them as < -> %26lt%3B [01:03] If the header was encoded properly then what could S3 be having a problem with in that string. [01:03] If S3 had a decoding problem then surely someone would have had the issue. [01:08] Huh, I just tried using the web interface to put the description back and its just sitting there, usually only takes a second or so. [01:09] Okay, that's weird. It's returned a page that's like the archive.org homepage, but with just a grey bar and white background. [01:11] Well shit, another item just 503'd and it also has a description with a load of links. [01:12] Looks like archive.org has a problem. [01:12] *** BlueMax has quit IRC (Leaving) [01:13] How in the world has this not been encountered before? So many videos have been mirrored statistically someone should've tripped this. Totally nuts. [01:17] No weird/failed tasks in the item log either. [01:21] *** Fusl has quit IRC (Ping timeout: 480 seconds) [01:23] SketchCow: i'm going after cbsnews youtube channel [01:23] it has over 65k videos [01:23] going back to 2007 [01:24] godane, damn, hope you have a good connection. [01:24] *** Fusl has joined #archiveteam-bs [01:25] most of these videos are 133+140 from 2007 and before [01:25] Ha, good point. [01:25] i will run a script to make them into daily dumps [01:26] Interesting to see how much will have changed in those 11 years. [01:30] i found something: https://commerce.wazeedigital.com [01:33] Some ad revenue/licensing thing? [01:36] *** vegmitemo has quit IRC (Quit: Leaving) [03:17] *** DragonMon has quit IRC (Ping timeout: 252 seconds) [03:32] *** archodg_ has joined #archiveteam-bs [03:35] *** odemg has quit IRC (Ping timeout: 268 seconds) [03:35] *** BlueMax has joined #archiveteam-bs [03:36] *** archodg__ has quit IRC (Read error: Operation timed out) [03:47] *** odemg has joined #archiveteam-bs [04:47] *** Ctrl-S___ is now known as Crtl-S [04:47] *** Crtl-S is now known as Ctrl-S [06:15] *** DragonMon has joined #archiveteam-bs [06:42] *** Mateon1 has quit IRC (west.us.hub irc.Prison.NET) [06:42] *** RichardG has quit IRC (west.us.hub irc.Prison.NET) [06:42] *** wacky has quit IRC (west.us.hub irc.Prison.NET) [06:42] *** achip has quit IRC (west.us.hub irc.Prison.NET) [06:43] *** wacky_ has joined #archiveteam-bs [06:51] *** RichardG_ has joined #archiveteam-bs [07:12] *** achip has joined #archiveteam-bs [07:12] *** Mateon1 has joined #archiveteam-bs [08:29] *** dxrt has quit IRC (Quit: ZNC - http://znc.sourceforge.net) [08:51] *** Laverne has joined #archiveteam-bs [09:03] *** dxrt has joined #archiveteam-bs [09:48] *** wp494 has quit IRC (Ping timeout: 260 seconds) [09:49] *** wp494 has joined #archiveteam-bs [10:35] *** x[x] has joined #archiveteam-bs [11:20] *** x[x] has quit IRC (Quit: Going offline, see ya! (www.adiirc.com)) [11:30] *** Darkstar has quit IRC (Ping timeout: 1212 seconds) [11:46] *** Darkstar has joined #archiveteam-bs [12:04] *** Mateon1 has quit IRC (Ping timeout: 255 seconds) [12:05] *** Mateon1 has joined #archiveteam-bs [12:46] *** vegmitemo has joined #archiveteam-bs [12:49] riking, I've left a message on IA's help forum, hopefully they can find the bug and fix it. Thanks for the help. [12:50] *** vegmitemo has quit IRC (Client Quit) [13:21] Regarding AMO: I've received confirmation from Mozilla that the legacy addons are not being removed from AMO currently. The only information I got about when the removal will happen is "not yet". (Arctic, atluxity, hook54321, eientei95) [13:24] ty [14:36] *** Soni has quit IRC (Read error: Operation timed out) [14:42] *** BlueMax has quit IRC (Read error: Connection reset by peer) [14:44] *** ta9le has joined #archiveteam-bs [15:07] isn't the ESR releases still use them right? [15:07] for the firefox stuff [15:07] * jrwr still misses the mozilla suite from 2005 [15:08] Yeah, see #archiveteam from yesterday. Firefox 52 ESR is still supported until early September, so the legacy addons should remain on AMO until then. [15:37] *** schbirid has joined #archiveteam-bs [15:52] *** zyphlar has quit IRC (Ping timeout: 246 seconds) [15:56] *** zyphlar has joined #archiveteam-bs [16:24] *** x[x] has joined #archiveteam-bs [16:40] riking: I've encountered encoding bugs in archive.org similar to what you are describing, but they resulted in other problems than 503s. Maybe related: https://github.com/jjjake/internetarchive/issues/235 https://archive.org/post/1091492/ https://archive.org/post/1092054/ [17:27] *** jschwart has joined #archiveteam-bs [17:35] *** RichardG_ is now known as RichardG [17:48] *** jschwart has quit IRC (Quit: Konversation terminated!) [18:19] *** odemg has quit IRC (Quit: Leaving) [18:44] *** SimpBrain has quit IRC (Read error: Operation timed out) [19:02] *** archodg__ has joined #archiveteam-bs [19:06] *** archodg_ has quit IRC (Read error: Operation timed out) [19:11] *** archodg_ has joined #archiveteam-bs [19:15] *** archodg__ has quit IRC (Read error: Operation timed out) [19:21] Update on legacy addons on AMO: " removal of legacy add-ons is planned for Q4 this year, though no concrete dates are set yet" [19:37] *** SimpBrain has joined #archiveteam-bs [19:51] JAA: Ok cool [19:51] Any word on why that Legacy Theme Changer got removed? [19:51] ah, nvm, saw what you said in #archivebot [20:25] *** vitzli has joined #archiveteam-bs [20:27] *** Sk1d has quit IRC (Read error: Operation timed out) [20:28] *** Sk1d has joined #archiveteam-bs [21:10] *** schbirid has quit IRC (Quit: Leaving) [21:30] *** x[x] has quit IRC (Going offline, see ya! (www.adiirc.com)) [21:51] *** vitzli has quit IRC (Leaving) [21:57] *** tuluu has quit IRC (Remote host closed the connection) [21:58] *** tuluu has joined #archiveteam-bs [22:32] *** archodg_ has quit IRC (Quit: Leaving) [23:36] *** BlueMax has joined #archiveteam-bs [23:40] *** Soni has joined #archiveteam-bs [23:41] *** w0rmhole has joined #archiveteam-bs [23:48] *** ta9le has quit IRC (Quit: Connection closed for inactivity)