[00:48] *** JH88 has joined #archiveteam-bs [00:57] *** Exairnous has quit IRC (Read error: Operation timed out) [01:00] *** Exairnous has joined #archiveteam-bs [01:19] *** m007a83 has quit IRC (Read error: Connection reset by peer) [01:22] *** m007a83 has joined #archiveteam-bs [01:51] *** kiska1 has quit IRC (Read error: Operation timed out) [01:51] *** betamax_ has quit IRC (Read error: Operation timed out) [01:51] *** branden has quit IRC (Read error: Operation timed out) [01:51] *** underscor has quit IRC (Read error: Operation timed out) [01:51] *** underscor has joined #archiveteam-bs [01:52] *** HashbangI has quit IRC (Write error: Broken pipe) [01:52] *** ivan has quit IRC (Write error: Broken pipe) [01:52] *** sep332 has quit IRC (Write error: Broken pipe) [01:52] *** betamax has joined #archiveteam-bs [01:52] *** ivan has joined #archiveteam-bs [01:53] *** GLaDOS has quit IRC (Read error: Operation timed out) [01:53] *** benjinsmi has joined #archiveteam-bs [01:53] *** TigerbotH has quit IRC (Read error: Operation timed out) [01:54] *** Odd0002 has quit IRC (Read error: Operation timed out) [01:54] *** Smiley has joined #archiveteam-bs [01:54] *** Mayonaise has quit IRC (Read error: Operation timed out) [01:54] *** Odd0002 has joined #archiveteam-bs [01:54] *** benjins has quit IRC (Read error: Operation timed out) [01:55] *** PhrackD has quit IRC (Read error: Operation timed out) [01:55] *** qw3rty116 has joined #archiveteam-bs [01:55] *** PotcFdk has quit IRC (Read error: Operation timed out) [01:56] *** Mayonaise has joined #archiveteam-bs [01:56] *** qw3rty115 has quit IRC (Read error: Operation timed out) [01:56] *** JH88 has quit IRC (Read error: Operation timed out) [01:57] *** JH88 has joined #archiveteam-bs [01:57] *** kiskabak has joined #archiveteam-bs [01:59] *** godane has quit IRC (Ping timeout: 600 seconds) [02:01] *** step has quit IRC (Ping timeout: 600 seconds) [02:01] *** paul2520 has quit IRC (Ping timeout: 600 seconds) [02:02] *** branden has joined #archiveteam-bs [02:03] *** qw3rty116 has quit IRC (Read error: Operation timed out) [02:04] *** branden has quit IRC (Read error: Operation timed out) [02:05] *** SmileyG has quit IRC (Read error: Operation timed out) [02:05] *** qw3rty116 has joined #archiveteam-bs [02:11] *** qw3rty116 has quit IRC (Read error: Operation timed out) [02:20] *** qw3rty116 has joined #archiveteam-bs [02:23] *** kiska1 has joined #archiveteam-bs [02:26] *** paul2520 has joined #archiveteam-bs [02:27] *** GLaDOS has joined #archiveteam-bs [02:27] *** TigerbotH has joined #archiveteam-bs [02:29] *** sep332 has joined #archiveteam-bs [02:32] *** HashbangI has joined #archiveteam-bs [02:32] *** PhrackD has joined #archiveteam-bs [02:32] *** step has joined #archiveteam-bs [02:37] *** icedice has quit IRC (Quit: Leaving) [02:38] *** PotcFdk has joined #archiveteam-bs [03:03] *** paul2520 has quit IRC (Read error: Operation timed out) [03:03] *** sep332 has quit IRC (Read error: Operation timed out) [03:04] *** Mateon1 has quit IRC (Ping timeout: 265 seconds) [03:04] *** paul2520 has joined #archiveteam-bs [03:05] *** step has quit IRC (Read error: Operation timed out) [03:07] *** PhrackD has quit IRC (Read error: Operation timed out) [03:08] *** kiska1 has quit IRC (Read error: Operation timed out) [03:09] *** Sk1d has quit IRC (Read error: Operation timed out) [03:11] *** TigerbotH has quit IRC (Ping timeout: 600 seconds) [03:11] *** HashbangI has quit IRC (Read error: Operation timed out) [03:11] *** qw3rty117 has joined #archiveteam-bs [03:11] *** GLaDOS has quit IRC (Ping timeout: 600 seconds) [03:13] *** PotcFdk has quit IRC (Ping timeout: 600 seconds) [03:14] *** Sk1d has joined #archiveteam-bs [03:14] *** qw3rty116 has quit IRC (Ping timeout: 600 seconds) [03:15] *** paul2520 has quit IRC (Ping timeout: 600 seconds) [03:18] *** qw3rty117 has quit IRC (Read error: Operation timed out) [03:22] *** ppsym has joined #archiveteam-bs [03:22] *** kisspunch has quit IRC (Ping timeout: 492 seconds) [03:23] *** mgrytbak has joined #archiveteam-bs [03:25] *** PurpleSym has quit IRC (Ping timeout: 492 seconds) [03:25] *** mgrytbak^ has quit IRC (Ping timeout: 492 seconds) [03:25] *** ppsym is now known as PurpleSym [03:29] *** qw3rty117 has joined #archiveteam-bs [03:33] *** qw3rty118 has joined #archiveteam-bs [03:33] *** kiska1 has joined #archiveteam-bs [03:33] *** kisspunch has joined #archiveteam-bs [03:35] *** Sk1d has quit IRC (Read error: Operation timed out) [03:37] *** PhrackD has joined #archiveteam-bs [03:37] *** TigerbotH has joined #archiveteam-bs [03:38] *** GLaDOS has joined #archiveteam-bs [03:39] *** qw3rty117 has quit IRC (Ping timeout: 600 seconds) [03:40] *** Sk1d has joined #archiveteam-bs [03:42] *** HashbangI has joined #archiveteam-bs [03:42] *** paul2520 has joined #archiveteam-bs [03:42] *** step has joined #archiveteam-bs [03:47] *** Sk1d has quit IRC (Read error: Operation timed out) [03:49] *** Sk1d has joined #archiveteam-bs [03:50] *** sep332 has joined #archiveteam-bs [03:52] *** PotcFdk has joined #archiveteam-bs [03:55] *** Sk1d has quit IRC (Read error: Operation timed out) [03:57] *** Sk1d has joined #archiveteam-bs [04:16] *** odemgi_ has joined #archiveteam-bs [04:17] *** godane has joined #archiveteam-bs [04:19] *** odemgi has quit IRC (Read error: Operation timed out) [04:56] *** qw3rty119 has joined #archiveteam-bs [05:00] *** qw3rty118 has quit IRC (Read error: Operation timed out) [05:27] *** Sk1d has quit IRC (Read error: Operation timed out) [05:30] *** Sk1d has joined #archiveteam-bs [05:41] *** Mateon1 has joined #archiveteam-bs [06:08] *** turnkit_ has joined #archiveteam-bs [06:08] *** turnkit has quit IRC (Read error: Connection reset by peer) [06:42] *** deevious has joined #archiveteam-bs [07:00] *** Raccoon` is now known as Raccoon [07:21] *** odemgi has joined #archiveteam-bs [07:24] *** odemgi_ has quit IRC (Read error: Operation timed out) [07:26] *** Sk1d has quit IRC (Read error: Operation timed out) [07:28] *** Sk1d has joined #archiveteam-bs [07:47] *** Oddly has joined #archiveteam-bs [07:48] *** dxrt_ has joined #archiveteam-bs [07:48] *** dxrt sets mode: +o dxrt_ [08:00] *** Sk1d has quit IRC (Read error: Operation timed out) [08:03] *** svchfoo1 sets mode: +o PurpleSym [08:03] *** Sk1d has joined #archiveteam-bs [08:10] *** Sk1d has quit IRC (Read error: Operation timed out) [08:13] *** Sk1d has joined #archiveteam-bs [08:14] mr_archiv: there's also a youtube-comments.py script in the repo for https://www.archiveteam.org/index.php?title=UC_Berkeley_Course_Captures [08:14] *** Exairnous has quit IRC (Ping timeout: 615 seconds) [08:15] It uses the API and needs an API key or OAuth2 creds. It expects to be able to download all the comments in one go, so I suspect it will fail on *really* big comment threads (that exhaust an API quota) [08:17] It also doesn't download avatar images. [08:18] *** wp494 has quit IRC (Ping timeout: 615 seconds) [08:20] *** wp494 has joined #archiveteam-bs [08:22] *** kode54 has quit IRC (Quit: ZNC 1.7.2 - https://znc.in) [08:30] *** kode54 has joined #archiveteam-bs [08:36] *** Sk1d has quit IRC (Read error: Operation timed out) [08:39] *** Sk1d has joined #archiveteam-bs [09:33] *** Despatche has quit IRC (Quit: Connection reset by deer) [09:35] *** Sk1d has quit IRC (Read error: Operation timed out) [09:37] *** Sk1d has joined #archiveteam-bs [10:13] *** Sk1d has quit IRC (Read error: Operation timed out) [10:16] *** Sk1d has joined #archiveteam-bs [11:00] *** wyatt8740 has joined #archiveteam-bs [11:35] *** Oddly has quit IRC (Ping timeout: 255 seconds) [11:45] *** BlueMax has quit IRC (Quit: Leaving) [11:53] *** Sk1d has quit IRC (Read error: Operation timed out) [11:57] *** Sk1d has joined #archiveteam-bs [12:22] *** Raccoon has quit IRC (Read error: Connection reset by peer) [13:39] *** Stiletto has quit IRC (Read error: Operation timed out) [14:51] *** Sk1d has quit IRC (Read error: Operation timed out) [14:54] *** Sk1d has joined #archiveteam-bs [15:12] jodizzle: that happens when .json is missing https://archive.fart.website/archivebot/viewer/job/euhc0 [15:13] jodizzle: you are saving more twitter accounts? i have some suggestions [15:15] The lost one has returned. :-) [15:39] JAA: hehe [15:39] jodizzle: here are some https://www.archiveteam.org/index.php?title=Talk:ArchiveBot/Venezuela_politics [16:44] VoynichCr: Thanks, some of these may already have been included in my bigger transfer.sh dumps. I'll take a look later. [16:57] *** Stiletto has joined #archiveteam-bs [17:02] VoynichCr: Also, why are some of the .json files missing? [17:11] jodizzle: Various things can cause that. In this particular case, the pipeline crashed, and the remaining data was never uploaded it seems. [17:11] So that job was definitely incomplete. [17:12] *** Sk1d has quit IRC (Read error: Operation timed out) [17:14] *** wp494 has quit IRC (Read error: Operation timed out) [17:14] *** schbirid has joined #archiveteam-bs [17:15] *** Sk1d has joined #archiveteam-bs [17:15] *** wp494 has joined #archiveteam-bs [17:23] *** Dimtree has quit IRC (Peace) [17:24] JAA: Ah okay, thanks. [17:25] Though it does seem like there are some other cases where there is JSON present, but the table doesn't mark the row as "Saved!" at all. [17:26] Like this one: https://archive.fart.website/archivebot/viewer/?q=https://resistenciav58.wordpress.com/ [17:26] I suppose it's possible the bot just hasn't updated that one yet, since it was done only a few days ago? [17:29] *** Dimtree has joined #archiveteam-bs [17:43] JAA, what's the g+ channel? [17:43] odemgi: #googleminus [18:37] *** Oddly has joined #archiveteam-bs [18:57] *** Exairnous has joined #archiveteam-bs [19:02] *** bitBaron has joined #archiveteam-bs [19:02] *** wyatt8740 has quit IRC (Ping timeout: 246 seconds) [19:17] *** Sk1d has quit IRC (Read error: Operation timed out) [19:19] *** wyatt8740 has joined #archiveteam-bs [19:20] *** Sk1d has joined #archiveteam-bs [19:26] *** Sk1d has quit IRC (Read error: Operation timed out) [19:28] *** Sk1d has joined #archiveteam-bs [19:38] *** wyatt8740 has quit IRC (Ping timeout: 360 seconds) [19:52] *** fuzy802 has joined #archiveteam-bs [19:54] *** fuzzy8021 has quit IRC (Read error: Operation timed out) [20:02] *** fuzy802 is now known as fuzzy8021 [20:14] *** Dimtree has quit IRC (Peace) [20:17] *** Dimtree has joined #archiveteam-bs [20:30] *** Sk1d has quit IRC (Read error: Operation timed out) [20:33] *** Sk1d has joined #archiveteam-bs [20:34] *** sknebel_ has quit IRC (Read error: Operation timed out) [20:45] *** Sk1d has quit IRC (Read error: Operation timed out) [20:45] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [20:48] *** Sk1d has joined #archiveteam-bs [20:53] *** c4rc4s has quit IRC (Adios) [20:53] *** c4rc4s has joined #archiveteam-bs [20:57] *** Oddly has quit IRC (Ping timeout: 256 seconds) [21:03] *** bitBaron has joined #archiveteam-bs [21:06] *** sknebel has joined #archiveteam-bs [21:07] *** wyatt8740 has joined #archiveteam-bs [21:14] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [21:33] *** bitBaron has joined #archiveteam-bs [21:33] *** Despatche has joined #archiveteam-bs [21:35] Someone on Reddit pointed out that LineageOS 14.1 is now deprecated, and those builds (well, the ones that weren't superseded by 15.x builds already) will get deleted in a few weeks. This made me realise that apparently LineageOS builds are not preserved at all in general (not just on major version changes), and I now wonder if that needs to change. There's generally one build per day and phone model; [21:36] one build is roughly 300 MB, and there are some 242 models listed on the download directory https://mirrorbits.lineageos.org/full/ (note that this actually redirects to another server, e.g. https://lineageos.mirrorhub.io/full/ ; there appear to be several servers), though some directories are empty. So that would be up to about 70 GiB per day or ~25 TB per year. [21:38] SketchCow (but also anyone else really): Any thoughts on this? [21:39] (If the dataset's too large, this could of course also be reduced by grabbing only e.g. one build per week or per month.) [21:52] *** Albardin has joined #archiveteam-bs [22:08] *** kisspunch has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** Hani has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** closure has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** Frogging has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** Fusl has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** noirscape has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** argus has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** micro_ has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** jodizzle has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** N4Y has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** VoynichCr has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** MrRadar2 has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** Tenebrae has quit IRC (hub.efnet.us irc.efnet.nl) [22:08] *** BnAboyZ has quit IRC (hub.efnet.us irc.efnet.nl) [22:09] *** Darkstar has quit IRC (Read error: Operation timed out) [22:09] *** k0de54 has joined #archiveteam-bs [22:09] *** slyphic has quit IRC (Read error: Operation timed out) [22:10] *** slyphic has joined #archiveteam-bs [22:10] *** kisspunch has joined #archiveteam-bs [22:10] *** Hani has joined #archiveteam-bs [22:10] *** closure has joined #archiveteam-bs [22:10] *** Frogging has joined #archiveteam-bs [22:10] *** Fusl has joined #archiveteam-bs [22:10] *** micro_ has joined #archiveteam-bs [22:10] *** noirscape has joined #archiveteam-bs [22:10] *** argus has joined #archiveteam-bs [22:10] *** jodizzle has joined #archiveteam-bs [22:10] *** N4Y has joined #archiveteam-bs [22:10] *** VoynichCr has joined #archiveteam-bs [22:10] *** MrRadar2 has joined #archiveteam-bs [22:10] *** Tenebrae has joined #archiveteam-bs [22:10] *** BnAboyZ has joined #archiveteam-bs [22:10] *** irc.efnet.nl sets mode: +o MrRadar2 [22:10] *** kode54 has quit IRC (Remote host closed the connection) [22:10] *** k0de54 is now known as kode54 [22:12] *** Darkstar has joined #archiveteam-bs [22:13] *** robogoat has quit IRC (Read error: Operation timed out) [22:27] *** robogoat has joined #archiveteam-bs [22:29] *** BlueMax has joined #archiveteam-bs [22:46] *** lenary has joined #archiveteam-bs [22:57] *** Albardin has quit IRC (Read error: Operation timed out) [22:58] *** HashbangI has quit IRC (Write error: Broken pipe) [22:58] *** kode54 has quit IRC (Read error: Operation timed out) [22:58] *** kode54 has joined #archiveteam-bs [22:59] *** paul2520 has quit IRC (Read error: Operation timed out) [22:59] *** GLaDOS has quit IRC (Read error: Operation timed out) [23:00] *** step has quit IRC (Read error: Operation timed out) [23:00] *** dxrt_ has quit IRC (Read error: Operation timed out) [23:01] *** sep332 has quit IRC (Read error: Operation timed out) [23:01] *** qw3rty119 has quit IRC (Read error: Operation timed out) [23:01] *** qw3rty119 has joined #archiveteam-bs [23:02] *** PhrackD has quit IRC (Read error: Operation timed out) [23:05] *** kiska1 has quit IRC (Read error: Operation timed out) [23:05] *** godane has quit IRC (Ping timeout: 600 seconds) [23:06] *** TigerbotH has quit IRC (Ping timeout: 600 seconds) [23:06] *** godane has joined #archiveteam-bs [23:08] *** paul2520 has joined #archiveteam-bs [23:09] *** qw3rty119 has quit IRC (Read error: Operation timed out) [23:10] *** paul2520 has quit IRC (Read error: Operation timed out) [23:10] *** schbirid has quit IRC (Remote host closed the connection) [23:12] *** qw3rty119 has joined #archiveteam-bs [23:12] *** kiska1 has joined #archiveteam-bs [23:13] *** Albardin has joined #archiveteam-bs [23:13] *** paul2520 has joined #archiveteam-bs [23:15] *** TigerbotH has joined #archiveteam-bs [23:16] *** sep332 has joined #archiveteam-bs [23:16] *** HashbangI has joined #archiveteam-bs [23:17] *** step has joined #archiveteam-bs [23:18] *** dxrt_ has joined #archiveteam-bs [23:18] *** dxrt sets mode: +o dxrt_ [23:20] *** PhrackD has joined #archiveteam-bs [23:20] *** GLaDOS has joined #archiveteam-bs [23:34] *** chimyatta has quit IRC (Quit: quitting) [23:46] *** Sk1d has quit IRC (Read error: Operation timed out) [23:49] *** Darkstar has quit IRC (Read error: Operation timed out) [23:50] *** Sk1d has joined #archiveteam-bs [23:52] *** Dark_Star has joined #archiveteam-bs