[00:02] *** Sk1d has quit IRC (Read error: Operation timed out) [00:06] *** Sk1d has joined #archiveteam-bs [00:12] *** Sk1d has quit IRC (Read error: Operation timed out) [00:15] *** Sk1d has joined #archiveteam-bs [00:19] znak the script I made does not require an API key, however it also has the issue you mention where on huge videos it does not finish. [00:19] Also mine does avatar images [00:21] Thank you for mentioning this, for the API we could consider using it assuming it is free [00:21] The only thing is when I tried creating a Google account it tried to make me provide a phone number and I don't like that. [00:21] So I just declined. [00:22] Also from what I remember the API has a limit that never resets until you purchase something. [00:23] I get a 400 error on large videos. The problem is as you keep getting more comments the page token becomes longer and longer. [00:24] I wonder if the issue is within the request library. [00:24] *** wyatt8740 has quit IRC (Read error: Operation timed out) [00:24] 400 usually means you went something invalid to the server. [00:24] * you sent. [00:45] *** Sk1d has quit IRC (Read error: Operation timed out) [00:48] *** Sk1d has joined #archiveteam-bs [00:54] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [00:55] i got new capture usb stick [00:58] comes with a usb cable so i'm not plugin directly anymore [00:58] this cause a problem over time to cause the usb stick to bend [00:59] due to weight of cables and stuck [01:01] VoynichCr: Did you see my question a while ago about adding section handling to your bot? Also, any instructions for creating new lists? I'd like to track my jobs for the stuff around votes and elections in Switzerland. The reason I'm asking about sections is because that would allow grouping all votes of the same voting day onto a single page without mixing the individual votes' websites together and [01:01] creating a huge mess. I imagine that it would also be useful for other pages, e.g. the Venezuelan politics one (current government sites, politicians' social media, and opposition could be good sections there). [01:01] *** Sk1d has quit IRC (Read error: Operation timed out) [01:02] i may start capturing at 30fps exact cause the video looks weird like its not smooth video [01:05] *** Sk1d has joined #archiveteam-bs [01:12] looks to be the same with 30fps so i'm going back to 29.97 [01:30] *** Sk1d has quit IRC (Read error: Operation timed out) [01:33] *** Sk1d has joined #archiveteam-bs [01:48] *** VADemon_ has quit IRC (Quit: left4dead) [01:56] *** bitBaron has joined #archiveteam-bs [02:08] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [02:09] *** slyphic has quit IRC (Read error: Operation timed out) [02:09] *** cascode has quit IRC (Read error: Operation timed out) [02:09] *** cascode has joined #archiveteam-bs [02:10] *** brayden_ has quit IRC (Read error: Operation timed out) [02:11] *** slyphic has joined #archiveteam-bs [02:15] *** wp494 has quit IRC (Read error: Operation timed out) [02:15] *** wp494 has joined #archiveteam-bs [02:16] *** brayden has joined #archiveteam-bs [02:36] *** Pixi` has joined #archiveteam-bs [02:40] *** Pixi has quit IRC (Read error: Operation timed out) [02:41] *** Pixi` is now known as Pixi [02:42] *** Despatche has quit IRC (Read error: Connection reset by peer) [02:43] *** Despatche has joined #archiveteam-bs [02:52] *** bitBaron has joined #archiveteam-bs [03:04] *** Sk1d has quit IRC (Read error: Operation timed out) [03:07] *** Sk1d has joined #archiveteam-bs [03:35] *** godane has quit IRC (Ping timeout: 612 seconds) [03:44] *** Mateon1 has quit IRC (Ping timeout: 265 seconds) [03:44] *** Mateon1 has joined #archiveteam-bs [03:50] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [04:15] *** odemgi_ has joined #archiveteam-bs [04:18] *** odemgi has quit IRC (Read error: Operation timed out) [04:38] *** m007a83 has quit IRC (Read error: Connection reset by peer) [04:43] *** m007a83 has joined #archiveteam-bs [04:56] *** qw3rty111 has joined #archiveteam-bs [05:00] *** qw3rty119 has quit IRC (Read error: Operation timed out) [05:08] *** BlueMax has quit IRC (Quit: Leaving) [05:47] *** BlueMax has joined #archiveteam-bs [06:29] *** marked has quit IRC (Read error: Operation timed out) [06:53] *** wp494 has quit IRC (Read error: Connection reset by peer) [06:54] *** wp494 has joined #archiveteam-bs [06:54] *** marked has joined #archiveteam-bs [06:58] *** m007a83 has quit IRC (Ping timeout: 252 seconds) [07:00] *** m007a83 has joined #archiveteam-bs [07:05] *** Sk1d has quit IRC (Read error: Operation timed out) [07:07] *** Sk1d has joined #archiveteam-bs [07:08] *** DopefishJ is now known as DFJustin [07:23] ok, i'm not sure if this is supposed to be blacked out or not, but this may be the last fully intact copy of beebe's plot79 fortran package on the internet, presumably left exposed on an ftp by accident: ftp://ftp.math.utah.edu/outgoing/plot79.tar.gz-ae97bdf8bba4692b6556c8c64707ebc9 [07:23] beebe does NOT have this on his website, and i do not know why [07:24] its supposedly a famous package first released in 1979, and was used by or distributed by SIGGRAPH [07:26] http://www.math.utah.edu/~beebe/software/ is beebe's site [07:38] *** Exairnous has quit IRC (Read error: Operation timed out) [08:49] *** Oddly has joined #archiveteam-bs [08:54] *** wyatt8740 has joined #archiveteam-bs [09:07] *** BlueMax has quit IRC (Quit: Leaving) [10:05] *** m007a83_ has joined #archiveteam-bs [10:08] *** m007a83 has quit IRC (Ping timeout: 252 seconds) [10:10] *** achip has quit IRC (Ping timeout: 255 seconds) [10:11] *** wp494 has quit IRC (Ping timeout: 255 seconds) [10:12] *** marked has quit IRC (Ping timeout: 255 seconds) [10:12] *** wp494 has joined #archiveteam-bs [10:12] *** marked has joined #archiveteam-bs [10:15] *** achip has joined #archiveteam-bs [10:17] *** Oddly has quit IRC (Read error: Operation timed out) [10:23] *** dxrt_ has quit IRC (Read error: Operation timed out) [10:23] *** cascode has quit IRC (Read error: Operation timed out) [10:24] *** paul2520 has quit IRC (Read error: Operation timed out) [10:24] *** HashbangI has quit IRC (Write error: Broken pipe) [10:24] *** kiska1 has quit IRC (Read error: Operation timed out) [10:24] *** cascode has joined #archiveteam-bs [10:25] *** sep332 has quit IRC (Read error: Operation timed out) [10:25] *** GLaDOS has quit IRC (Read error: Operation timed out) [10:26] *** Albardin has quit IRC (Read error: Operation timed out) [10:26] *** PotcFdk has quit IRC (Read error: Operation timed out) [10:27] *** PhrackD has quit IRC (Read error: Operation timed out) [10:28] *** step has quit IRC (Read error: Operation timed out) [10:28] *** Albardin has joined #archiveteam-bs [10:28] *** kiska1 has joined #archiveteam-bs [10:29] *** qw3rty111 has quit IRC (Read error: Operation timed out) [10:31] *** TigerbotH has quit IRC (Ping timeout: 600 seconds) [10:32] *** qw3rty111 has joined #archiveteam-bs [10:32] *** paul2520 has joined #archiveteam-bs [10:35] *** sep332 has joined #archiveteam-bs [10:36] *** GLaDOS has joined #archiveteam-bs [10:37] *** TigerbotH has joined #archiveteam-bs [10:39] *** dxrt_ has joined #archiveteam-bs [10:39] *** dxrt sets mode: +o dxrt_ [10:40] *** PhrackD has joined #archiveteam-bs [10:41] *** step has joined #archiveteam-bs [10:58] *** PotcFdk has joined #archiveteam-bs [11:14] *** wp494 has quit IRC (Read error: Operation timed out) [11:14] *** wp494 has joined #archiveteam-bs [11:56] *** bitBaron has joined #archiveteam-bs [12:31] Lord_Nigh: http://web.archive.org/web/20190226123111/http://ftp.math.utah.edu/outgoing/plot79.tar.gz-ae97bdf8bba4692b6556c8c64707ebc9 and is now in IA [13:00] *** superkuh has quit IRC (Read error: Operation timed out) [13:20] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [13:27] *** eLbot has quit IRC (Ping timeout: 615 seconds) [13:36] *** bitBaron has joined #archiveteam-bs [13:37] *** JH88 has quit IRC (Quit: JH88) [14:01] *** HashbangI has joined #archiveteam-bs [14:02] *** Oddly has joined #archiveteam-bs [14:14] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [14:20] *** bitBaron has joined #archiveteam-bs [14:31] *** godane has joined #archiveteam-bs [14:31] starting to upload old blazetv highlights i have: https://archive.org/details/gbtv_25542163_1200K [14:32] full set on drive that i have is over 400gb [14:33] i stop archiving for awhile then theblaze changed there video website in 2016 or 2017 [14:38] eientei95: excellent! I didn't see that it was also exposed as http [14:40] *** Soni has quit IRC (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.) [14:42] *** Soni has joined #archiveteam-bs [15:48] *** Oddly has quit IRC (Ping timeout: 255 seconds) [16:27] *** RichardG has quit IRC (Ping timeout: 615 seconds) [16:36] *** RichardG has joined #archiveteam-bs [16:36] *** deevious has quit IRC (Remote host closed the connection) [16:50] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [17:07] *** bitBaron has joined #archiveteam-bs [17:15] *** kisspunch has quit IRC (se.hub irc.efnet.nl) [17:15] *** Hani has quit IRC (se.hub irc.efnet.nl) [17:15] *** closure has quit IRC (se.hub irc.efnet.nl) [17:15] *** Frogging has quit IRC (se.hub irc.efnet.nl) [17:15] *** Fusl has quit IRC (se.hub irc.efnet.nl) [17:15] *** noirscape has quit IRC (se.hub irc.efnet.nl) [17:15] *** argus has quit IRC (se.hub irc.efnet.nl) [17:15] *** micro_ has quit IRC (se.hub irc.efnet.nl) [17:15] *** jodizzle has quit IRC (se.hub irc.efnet.nl) [17:15] *** N4Y has quit IRC (se.hub irc.efnet.nl) [17:15] *** VoynichCr has quit IRC (se.hub irc.efnet.nl) [17:15] *** MrRadar2 has quit IRC (se.hub irc.efnet.nl) [17:15] *** Tenebrae has quit IRC (se.hub irc.efnet.nl) [17:15] *** BnAboyZ has quit IRC (se.hub irc.efnet.nl) [17:17] *** kisspunch has joined #archiveteam-bs [17:17] *** Hani has joined #archiveteam-bs [17:17] *** closure has joined #archiveteam-bs [17:17] *** Frogging has joined #archiveteam-bs [17:17] *** Fusl has joined #archiveteam-bs [17:17] *** micro_ has joined #archiveteam-bs [17:17] *** noirscape has joined #archiveteam-bs [17:17] *** argus has joined #archiveteam-bs [17:17] *** jodizzle has joined #archiveteam-bs [17:17] *** N4Y has joined #archiveteam-bs [17:17] *** VoynichCr has joined #archiveteam-bs [17:17] *** MrRadar2 has joined #archiveteam-bs [17:17] *** Tenebrae has joined #archiveteam-bs [17:17] *** BnAboyZ has joined #archiveteam-bs [17:17] *** irc.efnet.nl sets mode: +o MrRadar2 [17:28] *** Sk1d has quit IRC (Read error: Operation timed out) [17:31] *** Sk1d has joined #archiveteam-bs [17:39] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [17:46] *** Sk1d has quit IRC (Read error: Operation timed out) [17:49] *** Sk1d has joined #archiveteam-bs [17:57] *** schbirid has joined #archiveteam-bs [18:06] *** bitBaron has joined #archiveteam-bs [18:12] JAA: INFO FINISHED. [18:12] INFO Duration: 4:51:02. Speed: 13.1 MiB/s. [18:12] INFO Downloaded: 1460 files, 245.7 GiB. [18:12] (wpull --concurrent 8 --recursive -x https://lineageos.mirrorhub.io/full/) [18:37] Fusl: Nice. Can you check how many actual builds, i.e. .zip files, were grabbed? I think "1460 files" includes the directory listings. [18:37] around 700 i think, ill check in a few [18:39] but yeah, we can probably throw that into archivebot to get them archived directly [18:39] I did, but I ignored the 15.1 images. [18:39] Job cbdshbtfawd9ztif2f8ohl3ur [18:43] It seems that usually the five most recent builds are kept per target. That would give around 50 GiB of data per day. A bit less than my previous estimate, as expected. [18:55] 601 .zip files [18:55] 603* [19:07] https://twitter.com/86reality/status/1100285735855374336 [19:08] *** Oddly has joined #archiveteam-bs [19:26] *** Exairnous has joined #archiveteam-bs [19:33] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [19:34] *** m007a83_ is now known as m007a83 [19:36] *** bitBaron has joined #archiveteam-bs [19:40] *** Sk1d has quit IRC (Read error: Operation timed out) [19:43] *** Sk1d has joined #archiveteam-bs [19:50] *** mib_3nu2h has joined #archiveteam-bs [19:51] what is the key to registering? [19:51] Ask for it by typing WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD [19:52] yes, what do you want to *do* on the wiki? [19:53] see the album [19:54] the album? [19:57] if they told me that I could find interesting things and I have been investigating for days [19:57] sorry my english is not very good [20:08] You don't need an account to view the wiki [20:08] only to edit it [20:13] *** wp494 has quit IRC (Read error: Operation timed out) [20:14] *** wp494 has joined #archiveteam-bs [20:15] *** Oddly has quit IRC (Ping timeout: 255 seconds) [20:24] *** kisspunch has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** Hani has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** closure has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** Frogging has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** Fusl has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** noirscape has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** argus has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** micro_ has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** jodizzle has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** N4Y has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** VoynichCr has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** MrRadar2 has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** Tenebrae has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** BnAboyZ has quit IRC (hub.efnet.us irc.efnet.nl) [20:24] *** purplebot has quit IRC (Remote host closed the connection) [20:25] *** purplebot has joined #archiveteam-bs [20:25] *** cascode has quit IRC (Read error: Operation timed out) [20:25] *** cascode has joined #archiveteam-bs [20:29] *** kisspunch has joined #archiveteam-bs [20:29] *** Hani has joined #archiveteam-bs [20:29] *** closure has joined #archiveteam-bs [20:29] *** Frogging has joined #archiveteam-bs [20:29] *** Fusl has joined #archiveteam-bs [20:29] *** micro_ has joined #archiveteam-bs [20:29] *** noirscape has joined #archiveteam-bs [20:29] *** argus has joined #archiveteam-bs [20:29] *** jodizzle has joined #archiveteam-bs [20:29] *** N4Y has joined #archiveteam-bs [20:29] *** VoynichCr has joined #archiveteam-bs [20:29] *** MrRadar2 has joined #archiveteam-bs [20:29] *** Tenebrae has joined #archiveteam-bs [20:29] *** BnAboyZ has joined #archiveteam-bs [20:29] *** irc.efnet.nl sets mode: +o MrRadar2 [20:41] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [20:43] *** Sk1d has quit IRC (Read error: Operation timed out) [20:48] *** Sk1d has joined #archiveteam-bs [20:57] Fusl: Thanks. So according to https://github.com/LineageOS/hudson/blob/master/lineage-build-targets there are currently 89 build targets active. But that list can't be right; it lists everything as "W", i.e. weekly, although those are being built nightly. Quite a mess... [20:58] Do you have a log file from wpull? [20:59] *** mib_3nu2h has quit IRC (Quit: http://www.mibbit.com ajax IRC Client) [20:59] JAA: stdout? [20:59] *** bitBaron has joined #archiveteam-bs [21:00] Yeah, that's fine as well. [21:02] I think the log file wpull writes is just the stderr output with timestamps anyway, basically. [21:03] yeah i dont have that but here is the docker logs output http://xor.meo.ws/QZqi3Z-MNP3APwa6phtcnn1EQAPa1I-w/c91b344fda7d.txt [21:04] That's fine, don't need timestamps for this. Thanks! [21:05] Fusl: Hmm, that file seems to be incomplete. [21:05] it's not. i'm currently giving it another go [21:05] Oh, I see. [21:05] so it's still doing its thing [21:06] so far 876 files so a bunch more [21:06] Right. Should be done real soon then? [21:06] i am absolutely not sure why it didnt download them the first time i ran wpull [21:08] ZIPs from 164 models were retrieved according to that file. There are 242 listed on /full/, but some are empty. [21:25] Can you give me the log again when it finishes? [21:42] *** Pixi has quit IRC (Quit: Pixi) [21:51] *** Pixi has joined #archiveteam-bs [21:57] *** ndiddy has joined #archiveteam-bs [22:13] *** schbirid has quit IRC (Remote host closed the connection) [22:25] *** BlueMax has joined #archiveteam-bs [22:33] *** Albardin has quit IRC (Read error: Operation timed out) [22:34] *** GLaDOS has quit IRC (Read error: Operation timed out) [22:35] *** paul2520 has quit IRC (Read error: Operation timed out) [22:35] *** kiska1 has quit IRC (Read error: Operation timed out) [22:35] *** cascode has quit IRC (Read error: Operation timed out) [22:35] *** dxrt_ has quit IRC (Write error: Broken pipe) [22:35] *** Albardin has joined #archiveteam-bs [22:35] *** cascode has joined #archiveteam-bs [22:35] *** sep332 has quit IRC (Read error: Operation timed out) [22:36] *** TigerbotH has quit IRC (Read error: Operation timed out) [22:37] *** paul2520 has joined #archiveteam-bs [22:37] *** kiska1 has joined #archiveteam-bs [22:38] *** step_ has joined #archiveteam-bs [22:38] *** TigerbotH has joined #archiveteam-bs [22:38] *** dxrt_ has joined #archiveteam-bs [22:38] *** dxrt sets mode: +o dxrt_ [22:38] *** GLaDOS has joined #archiveteam-bs [22:39] *** step has quit IRC (Read error: Operation timed out) [22:40] *** step_ is now known as step [22:41] *** Albardin has quit IRC (Read error: Operation timed out) [22:42] *** HashbangI has quit IRC (Read error: Operation timed out) [22:42] *** paul2520 has quit IRC (Read error: Operation timed out) [22:44] *** TigerbotH has quit IRC (Read error: Operation timed out) [22:47] *** step has quit IRC (Read error: Operation timed out) [22:47] *** kiska1 has quit IRC (Read error: Operation timed out) [22:47] *** PotcFdk has quit IRC (Ping timeout: 600 seconds) [22:48] *** Sk1d has quit IRC (Read error: Operation timed out) [22:49] *** dxrt_ has quit IRC (Write error: Broken pipe) [22:49] *** GLaDOS has quit IRC (Read error: Connection reset by peer) [22:49] *** casc0de has joined #archiveteam-bs [22:50] *** cascode has quit IRC (Read error: Connection reset by peer) [22:51] *** human has joined #archiveteam-bs [22:51] *** Sk1d has joined #archiveteam-bs [22:53] *** qw3rty111 has quit IRC (Ping timeout: 600 seconds) [22:55] *** Albardin has joined #archiveteam-bs [22:55] *** kiska1 has joined #archiveteam-bs [22:55] *** qw3rty112 has joined #archiveteam-bs [22:57] *** paul2520 has joined #archiveteam-bs [22:59] *** HashbangI has joined #archiveteam-bs [22:59] *** TigerbotH has joined #archiveteam-bs [22:59] *** sep332 has joined #archiveteam-bs [22:59] *** GLaDOS has joined #archiveteam-bs [23:00] *** PotcFdk has joined #archiveteam-bs [23:02] *** step has joined #archiveteam-bs [23:03] *** dxrt_ has joined #archiveteam-bs [23:03] *** dxrt sets mode: +o dxrt_ [23:07] *** Sk1d has quit IRC (Read error: Operation timed out) [23:10] *** Sk1d has joined #archiveteam-bs [23:43] JAA: 1200 zips [23:43] and still going [23:59] JAA: i now know why it appeared that a lot of files weren't downloaded [23:59] i actually redownloaded all of them into a subdirectory of where i downloaded them already, urgh