[00:25] The DeveloperWorks site seems to consist of a number of "communities" (groups). Each community has several of what I will call "features" (overview, recent updates, etc.). [00:26] Some "features" are what are clled "apps". I assume that the underlying reason some are apps and some are not is that apps were written/maintained by different groups of people, but how this manifests itself to the user is that all apps (except for the "Activities" app, which is completely restricted to registered users, as far as I can tell) have a life of their own, and can have items in them not attached to a c [00:26] as the rest of the "features" must be tied to a community. [00:29] Archiving it looks like like it would be fairly complicated - if you are in a community, some apps are accessed purely by AJAX, to the point that you nominally stay on some other URL, nearly everything has several GET parameters, & a lot of other caveats [00:30] *** LowLevelM has quit IRC (Read error: Operation timed out) [00:35] Anyone here want to help out archiving stuff related to the UK 2019 General Election? [00:35] I have a lists of: [00:35] twitter accounts (2740) [00:35] facebook accounts (2701) [00:35] websites (3573) [00:35] instagram accounts (202) [00:35] youtube accounts (55) [00:36] if no one else wants to do this, I can probably do it at some point, but I really want to focus on Yahoo Groups for the next two days [00:53] *** LowLevelM has joined #archiveteam-bs [01:14] betamax, maybe JAA or I could~ Elections are JAA's thing~ o.o; [01:14] betamax: Sure, I was going to look at UK stuff anyway. [01:15] Or Ryz [01:15] Want to make an Elections page? https://www.archiveteam.org/index.php?title=Elections [01:15] I'm bogged down with Yahoo Groups right now, so would only be able to do such things on Sunday [01:16] but it would be great if you could make a start [01:16] ...Ugh, creating ArchiveBot pages of it was rough~ <#>; [01:16] scraping all the twitters, facebooks and websites in particular will take an age, so getting started early would be a benifit [01:17] betamax: You could just dump all the links in a page like this, unformatted, and we can start sorting through them: https://www.archiveteam.org/index.php?title=Elections/2020_United_States_presidential_election [01:17] With that many social media accounts, it might be better to do them in one big snscrape file than try to feed them through socialbot one at a time [01:18] jodizzle, keep in mind this is the UK, United Kingdom~ [01:18] Right [01:19] I meant just a page like that, under 'Elections/' [01:19] You'd have to make a new one [01:19] Basically just somewhere to put the list [01:20] betamax, do you have the list for me or others to process? [01:22] yup, gimmie a set [01:22] *sec [01:24] http://s000.tinyupload.com/index.php?file_id=26959359146283624475 [01:24] that's a zip file with a text file for the twitters, a text file for the facebooks, etc.... [01:25] (sorry for using sucj a terrible file storage site, by the way!) [01:25] Gonna probably go for Instagram pages first~ [01:25] Ryz: I'm going to dump these on an 'Election/' page, so that we can have a public point of reference [01:26] From there, I will label the jobs with "UK 2019 General Election" [01:27] huge thanks for doing this [01:27] I could also run snscrape on e.g., the twitters in parallel on one of my machines to make it go faster. Though maybe using socialbot is still better, since it pings chromebot [01:28] Not sure [01:42] Ryz: https://www.archiveteam.org/index.php?title=Elections/2019_United_Kingdom_general_election [01:42] For reference [01:42] I might do some cleanup there, there's a decent amount of duplication [02:48] *** Tholos has quit IRC (Read error: Operation timed out) [02:51] *** Myself has joined #archiveteam-bs [03:03] *** cerca has quit IRC (Remote host closed the connection) [03:14] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [03:52] SketchCow: you going to be getting some japanese radio program that had archives going back to 2000 [03:53] https://www.stv.jp/radio/podcast/ainugo/index.html [04:14] Flashfire the worker uses the browser’s fetch api to get the results from the straw poll api then pushes them in blocks of 1000 to my server. [04:15] super cool [04:15] nice job [04:16] *** Craigle_ has joined #archiveteam-bs [04:18] *** odemgi has joined #archiveteam-bs [04:23] *** odemgi_ has quit IRC (Read error: Operation timed out) [04:29] So Wingy does that mean I should just leave it open? [04:34] *** tech234a has joined #archiveteam-bs [04:39] *** DogsRNice has quit IRC (Read error: Connection reset by peer) [04:39] Flashfire: As long as you want to be fetching straw poll, yes [04:41] *** Craigle_ has quit IRC (Quit: The Lounge - https://thelounge.chat) [04:44] *** Craigle has quit IRC () [04:45] *** Craigle has joined #archiveteam-bs [04:45] *** oxguy3 has quit IRC (My MacBook has gone to sleep. ZZZzzz…) [04:54] Flashfire if you want to you can :) [04:55] if the progress bars stop resetting something has gone wrong. Ping me if they stop resetting. [04:56] *** qw3rty has joined #archiveteam-bs [05:06] *** qw3rty2 has quit IRC (Ping timeout: 745 seconds) [05:16] wingy if you write any other archivers like this let me know. Just letting my browser do the work is easiest [05:17] *** oxguy3 has joined #archiveteam-bs [05:26] *** kiska has quit IRC (Remote host closed the connection) [05:26] *** Flashfire has quit IRC (Remote host closed the connection) [05:27] *** Flashfire has joined #archiveteam-bs [05:27] *** kiska has joined #archiveteam-bs [05:28] *** svchfoo3 sets mode: +o kiska [05:28] *** svchfoo1 sets mode: +o kiska [06:06] https://www.wikipediasucks.co/forum/viewtopic.php?f=19&t=1541 [06:06] https://poal.co/s/ModAbuse/120094 [06:06] I think we can all agree this assertion is accurate [06:06] Naturally, anyone who disagrees with be kicked to death [06:08] You've been cancel cultured. You can't hold an opinion now as you've been invalidated. [06:13] lol, an opinion from 2004 no less. [06:14] can someone add this to archivebot? it's 113MB of documents related to major league baseball's international broadcasts http://mlbigsp.majorleaguebaseball.com/filestore/ [06:21] *** Dallas has quit IRC (Quit: The Lounge - https://thelounge.chat) [06:22] wbm is going to town on it. http://web.archive.org/web/20191213062050/http://mlbigsp.majorleaguebaseball.com/filestore/ [06:22] grazi grazi [06:23] oh wait, sorry -- i tried to archive it using archive.org's archiving tool, but it only grabbed a random handful of the files instead of all of them [06:24] i did it and it seemed to work [06:24] used 'save outlinks' [06:24] didn't verify every file as it's still working [06:24] yeah i saved outlinks too, and it grabbed a big list of outlinks, but it was only a subset of the whole directory [06:25] try poking at them in an hour to see if a random set are all there [06:27] *** coderobe has quit IRC (Read error: Connection reset by peer) [06:41] *** tech234a has quit IRC (Quit: Connection closed for inactivity) [07:18] *** coderobe has joined #archiveteam-bs [07:23] *** oxguy3 has quit IRC (Read error: Operation timed out) [07:41] *** whytho has joined #archiveteam-bs [07:41] * whytho slaps eientei95 around a bit with a large fishbot [07:43] *** whytho has quit IRC (Client Quit) [07:53] *** mtntmnky has quit IRC (Remote host closed the connection) [07:54] *** mtntmnky has joined #archiveteam-bs [07:58] *** benjins has quit IRC (Read error: Connection reset by peer) [08:04] *** Raccoon has quit IRC (Remote host closed the connection) [08:15] *** tech234a has joined #archiveteam-bs [09:06] *** bluefoo has quit IRC (Ping timeout: 255 seconds) [09:49] *** Raccoon has joined #archiveteam-bs [10:14] *** deevious has quit IRC (Quit: deevious) [10:21] *** tech234a has quit IRC (Quit: Connection closed for inactivity) [10:22] *** zhongfu has quit IRC (Ping timeout: 745 seconds) [10:26] *** deevious has joined #archiveteam-bs [10:50] *** VerifiedJ has joined #archiveteam-bs [12:26] I just got here so I don't know if this was previously mentioned: https://www.reddit.com/r/DataHoarder/comments/ea0wx2/urgent_request_federal_databases_disappearing/ more info https://envirodatagov.org/goodbye-to-toxmap-and-our-environmental-right-to-know/ [12:40] so good news is i'm grabbing there technical bulletin [12:40] goes back to 1969 [12:45] folks in the reddit thread suggested it's already being hit so hard as to impact the servers, though =/ them folks need coordination! [12:57] *** killsushi has quit IRC (Quit: Leaving) [13:01] *** BlueMax has quit IRC (Read error: Connection reset by peer) [13:02] *** balrog has quit IRC (Bye) [13:10] *** cerca has joined #archiveteam-bs [13:37] *** deevious has quit IRC (Quit: deevious) [13:47] Flashfire Will do :) [13:56] Looking at tracker. [14:00] *** tech234a has joined #archiveteam-bs [14:05] kiska: Does Irista have anything public? [14:07] SketchCow: i noticed my folder was gone on FOS [14:07] i recreated that folder and i'm finally uploading the cnn sports tape [14:23] https://old.reddit.com/r/mobileweb/comments/e7yivg/join_reddit_to_keep_reading_an_account_is_now/ [14:27] *** Sora_Uta has quit IRC (Read error: Connection reset by peer) [14:27] *** jc86035 has joined #archiveteam-bs [14:29] my warrior instances are not loading any projects, even after restarting Docker. is this a known issue / is there a known resolution? [14:29] > Phooey… No warrior projects are available for participation yet! [14:29] > seesaw.warrior - DEBUG - Warrior ID ''. [14:32] jc86035 tracker is down, assume it's related unless it persists [14:32] noted [14:53] *** deevious has joined #archiveteam-bs [15:29] *** zhongfu has joined #archiveteam-bs [15:59] *** kiskabak has quit IRC (Read error: Operation timed out) [16:03] *** Flashfire has quit IRC (Remote host closed the connection) [16:03] *** kiska has quit IRC (Remote host closed the connection) [16:04] *** Flashfire has joined #archiveteam-bs [16:04] *** kiska has joined #archiveteam-bs [16:04] *** svchfoo1 sets mode: +o kiska [16:04] *** svchfoo3 sets mode: +o kiska [16:15] *** DogsRNice has joined #archiveteam-bs [16:21] *** VerifiedJ has quit IRC (Remote host closed the connection) [16:22] *** VerifiedJ has joined #archiveteam-bs [16:23] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [16:24] *** VerifiedJ has joined #archiveteam-bs [16:25] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [16:26] *** VerifiedJ has joined #archiveteam-bs [16:27] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [16:28] *** VerifiedJ has joined #archiveteam-bs [16:35] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [16:36] *** VerifiedJ has joined #archiveteam-bs [16:44] The FTP site isn't an empty locker room that gets reservations. When people upload, they can create folders, everything does that for it. [16:50] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [16:51] *** VerifiedJ has joined #archiveteam-bs [16:51] *** VerifiedJ has quit IRC (Remote host closed the connection) [16:53] *** VerifiedJ has joined #archiveteam-bs [16:53] *** VerifiedJ has quit IRC (Remote host closed the connection) [16:55] *** VerifiedJ has joined #archiveteam-bs [16:56] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [16:56] *** VerifiedJ has joined #archiveteam-bs [16:58] *** DigiDigi has quit IRC (Quit: Leaving) [17:05] *** DigiDigi has joined #archiveteam-bs [17:06] *** mls_ has quit IRC (Ping timeout: 258 seconds) [17:07] *** mls has joined #archiveteam-bs [17:09] *** m007a83 has quit IRC (Read error: Operation timed out) [17:26] *** markedL has quit IRC (Read error: Operation timed out) [17:27] *** asdf0101 has quit IRC (Read error: Operation timed out) [17:27] *** jc86035 has quit IRC (Quit: Connection closed for inactivity) [17:28] *** bluefoo has joined #archiveteam-bs [17:52] *** balrog has joined #archiveteam-bs [17:53] *** asdf0101 has joined #archiveteam-bs [17:53] *** markedL has joined #archiveteam-bs [18:05] *** markedL has quit IRC (Read error: Operation timed out) [18:06] *** asdf0101 has quit IRC (Read error: Operation timed out) [18:13] *** asdf0101 has joined #archiveteam-bs [18:13] *** markedL has joined #archiveteam-bs [18:19] *** balrog has quit IRC (Read error: Operation timed out) [18:23] *** balrog has joined #archiveteam-bs [18:23] *** icedice has quit IRC (Quit: Leaving) [18:26] *** balrog has quit IRC (Read error: Operation timed out) [18:30] *** tech234a has quit IRC (Quit: Connection closed for inactivity) [18:32] *** balrog has joined #archiveteam-bs [19:38] *** kiska has quit IRC (Remote host closed the connection) [19:38] *** Flashfire has quit IRC (Remote host closed the connection) [19:38] *** kiska has joined #archiveteam-bs [19:39] *** Flashfire has joined #archiveteam-bs [19:39] *** svchfoo3 sets mode: +o kiska [19:39] *** svchfoo1 sets mode: +o kiska [19:48] *** X-Scale` has joined #archiveteam-bs [19:49] *** erkinalp has joined #archiveteam-bs [19:54] *** X-Scale has quit IRC (Ping timeout: 610 seconds) [19:54] *** X-Scale` is now known as X-Scale [20:01] https://news.ycombinator.com/item?id=21780092 [20:01] is a rush needed to write a warrior script [20:01] for this reddit change [20:02] *** TC01 has quit IRC (Read error: Operation timed out) [20:03] *** satoshi has joined #archiveteam-bs [20:03] https://old.reddit.com/r/mobileweb/comments/e7yivg/join_reddit_to_keep_reading_an_account_is_now/ [20:04] as stated by asWhydoineedanemail69 [20:05] i think maybe they will get rid of old.reddit.com first [20:06] *** TC01 has joined #archiveteam-bs [20:07] "I was enjoying the new mobile Reddit experience..." my guess is the desktop version won't follow anytime soon [20:11] *** godane has quit IRC (Read error: Connection reset by peer) [20:13] then it's middle priority [20:13] I have only observed the behaviour on mobile, and only inconsistently. But I use https://addons.mozilla.org/en-CA/firefox/addon/old-reddit-redirect/ on my desktop so I wouldn't know how the desktop new site behaves [20:13] Interesting stats for old/new/apps/mobile from a top 100 sub here: https://news.ycombinator.com/item?id=21782685 [20:13] *** godane has joined #archiveteam-bs [20:14] i'm using the new desktop site but always as a logged in user [20:14] so cannot tell anything [20:15] SketchCow: be careful to no upload my tape rips for a while [20:15] my wifi acted up [20:28] *** X-Scale` has joined #archiveteam-bs [20:32] *** X-Scale has quit IRC (Read error: Operation timed out) [20:32] *** X-Scale` is now known as X-Scale [20:34] *** tech234a has joined #archiveteam-bs [20:49] *** Raccoon has quit IRC (Remote host closed the connection) [20:50] *** Raccoon has joined #archiveteam-bs [21:11] *** erkinalp has quit IRC (Quit: Page closed) [21:17] *** d5f4a3622 has quit IRC (Read error: Connection reset by peer) [21:27] *** bsmith093 has quit IRC (Quit: Leaving.) [21:30] *** d5f4a3622 has joined #archiveteam-bs [21:33] *** bsmith093 has joined #archiveteam-bs [21:35] we've got a channel for reddit called #shreddit [21:35] *** jc86035 has joined #archiveteam-bs [21:53] "I was enjoying the new mobile Reddit experience..." - Sentences that have never been said by anyone. [22:00] https://www.google.com/search?q=Sorry+about+formatting+on+mobile+site%3Areddit.com -- 158,000 results [22:30] is down-the-tube (YouTube liked lists) still active? [22:32] *** BlueMax has joined #archiveteam-bs [22:32] It more or less was suspended when some human at YT made a change to ban us (or so I am told) [22:33] And it looks like they've gone private, now [22:34] great [22:40] *** AlsoJAA has quit IRC (Quit: leaving) [22:44] *** AlsoJAA has joined #archiveteam-bs [22:44] *** JAA sets mode: +o AlsoJAA [22:46] we need to do thumbnails next [22:50] *** Jopik has quit IRC (Read error: Connection reset by peer) [22:51] YahooSucks deadlines tonight. [22:59] I always feel so bad when we miss [22:59] but we also almost always miss. [23:13] Didn't they push the deadline back? [23:14] *** Wingy has quit IRC (Read error: Operation timed out) [23:14] not for the web, they reset a deadline for GetMyData [23:15] Ah [23:16] It's tomorrow, no? "11:59 PM PT on Dec 14, 2019" [23:18] *** kiiwii has joined #archiveteam-bs [23:28] The website shutdown, that is. [23:35] *** SoraUta has joined #archiveteam-bs [23:38] we're not sure. Before they extended the deadline, GMD requests needed to be made bt 11.59PM PST [23:38] seems logical that this would also be the wesbite shutdown... [23:38] but there's been nothing logical about this so far [23:41] That's 2019-12-15 07:59 UTC for those who aren't on the US west coast. [23:43] Quote is from https://twitter.com/YahooCare/status/1204312076379926528 [23:43] (Also, fuck people who use "PT" instead of PDT or PST.) [23:44] nobody knows how to use PST vs PDT here, so you're screwed no matter [23:45] I have no idea what they even stand for, thanks for UTC-ifying it for ease of use [23:45] Winter = PST, summer = PDT. This isn't hard. [23:45] (PST is -8, PDT is -7) [23:46] people will say PST when PDT is in effect out of habit [23:46] Yes, there are many idiots out there. [23:46] PT is idiot proof [23:46] Yes, and also idiotic. [23:47] It bugs me that P_S_T isn't in _s_ummer. That's what always screws me up. [23:47] If I care, I give a UTC offset. [23:47] How about how UTC stands for Coordinated Universal Time? [23:48] ¯\_(ツ)_/¯ At least UTC is unambiguous. [23:48] What the abbreviation means exactly doesn't even matter. [23:49] With "PT", it's always a guessing game in spring and autumn whether the US already switched to/from DST because *of course* the switch is not on the same date everywhere. [23:49] Anyway, [23:50] In most other languages, the noun comes first, so it'd be TCU or TUC. In English, we put the noun at the end, so CUT. They deliberately chose UTC as to be neutral and favor neither. [23:50] Correct, and more precisely, it's a compromise between English and French. [23:51] Yep [23:51] PT isn't really ambiguous either, it's just not fixed to UTC, but it is fixed with regard to other US zones so it's just as convenient for people there [23:51] I don't care, was just asking if it bugged Myself too :) [23:52] klg_: That's exactly my point, it's convenient only for people inside the US, but it's used in contexts relevant to people outside of the US. [23:52] And yes, "PT" can be ambiguous during one hour in autumn. [23:53] s/can be/is/ [23:53] Swatch Internet Time will save us all, one day. [23:53] where? [23:53] afaik, Swatch is no longer making beat time watches [23:54] That just makes the time more valuable [23:54] If Swatch had based it in UTC instead of Switzerland time (UTC+1), I'd still be championing it. [23:55] half the point was it was arbitrary, that everyone should just deal with it [23:56] klg_: You mean, when? On the switch from DST to standard time, the hour between 02:00 and 03:00 local time occurs twice. So "2019-10-27 02:30 PT" is ambiguous. [23:56] oh that, okay [23:56] And that shit is why nobody in their right mind would ever use a local TZ for storing dates. [23:57] UTC has leapseconds too. TAI is the only sensible thing. [23:57] except when they need to schedule events in their local time zone which might unpredictably change before that event happens [23:57] Leap seconds are still unambiguous, just a bit annoying to handle since your seconds counter can become 60. [23:58] That's why you regularly update your time zone data. [23:58] Anyway, this definitely no longer belongs in this channel. [23:58] Oh hey, there's -ot over there