[00:07] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [00:11] *** marvinw is now known as ivan [00:12] JAA: want to write another web crawler using https://github.com/GoogleChrome/puppeteer? ;) [00:15] ivan: PurpleSym has been working on something like that. (Not with Node though, fortunately.) [00:16] https://github.com/PromyLOPh/crocoite [00:17] cool [00:19] I'm still considering building a minimal tool with aiohttp and warcio though, primarily for relatively simple stuff like APIs where the heavy machinery of a browser isn't needed. [00:20] But for now I'm busy rescuing the crashed CompuServe ArchiveBot job. [00:22] why anothe web crawler/ [00:26] The browser-based one because wpull doesn't handle heavily scripted websites well. It also means we can support HTTP/2, which may improve performance as a side-effect. Furthermore, the traffic will look much more real, so it might help getting around (some) bans. [00:26] The other one because wpull has a ton of ugly bugs that make it quite annoying to work with at times. [00:26] :/ [00:26] *** bithippo has joined #archiveteam-bs [00:27] And I think aiohttp+warcio would also be more lightweight than wpull. It would be a quite simplistic and specialist tool for certain use cases only, in particular API archiving (I've done a few of those with wpull). [00:27] I have no intention of rebuilding wpull. [00:27] That time would be better spent in debugging wpull instead. It's a great tool, it just suffers from a number of issues that make it barely usable really. [00:45] so i did 2 tapes [00:45] one is Loch Ness Discovered tape [00:45] another is called Titanic [00:45] that aired on A&E [00:46] both are from 1994 [00:49] *** pizzaiolo has joined #archiveteam-bs [00:54] *** ZexaronS- has joined #archiveteam-bs [00:55] *** ZexaronS has quit IRC (Read error: Operation timed out) [01:00] I'm impressed. My IP is still banned at Wine.Woot (since three weeks now). [01:33] *** bithippo has quit IRC (My MacBook Air has gone to sleep. ZZZzzz…) [01:37] *** zalgo has joined #archiveteam-bs [01:41] *** zalgo has quit IRC (Remote host closed the connection) [01:45] *** zalgo has joined #archiveteam-bs [02:03] *** pizzaiolo has quit IRC (Remote host closed the connection) [02:06] *** zalgo has quit IRC (Remote host closed the connection) [02:22] I use grab-site and wpull 1.2.3 all the time and while it's useful only like 90% of the time I wouldn't call it barely usable [02:23] sure it would be nice to archive everything but there's plenty of stuff it can archive [02:25] without it maybe you'd be dealing with heritrix or httrack :-) [02:27] Oh yeah, 1.2.3 is pretty good. The plugin interface of 2.0 is much better though. [02:27] And yes, it's definitely the best tool we have. [02:28] (I'm using 1.2.3 for most of my manual grabs as well.) [02:54] *** ZexaronS- has quit IRC (Leaving) [03:22] *** CoolCanuk has quit IRC (Quit: Connection closed for inactivity) [03:36] *** BlueMaxim has joined #archiveteam-bs [04:09] *** qw3rty116 has joined #archiveteam-bs [04:15] *** qw3rty115 has quit IRC (Read error: Operation timed out) [04:35] *** ranavalon has quit IRC (Remote host closed the connection) [04:36] *** ranavalon has joined #archiveteam-bs [05:43] *** Dimtree has quit IRC (Read error: Operation timed out) [06:01] *** Dimtree has joined #archiveteam-bs [06:02] *** Dimtree has quit IRC (Client Quit) [06:03] *** Dimtree has joined #archiveteam-bs [06:37] *** Dimtree has quit IRC (Read error: Operation timed out) [06:38] *** Pixi has quit IRC (Ping timeout: 255 seconds) [06:41] *** Pixi has joined #archiveteam-bs [06:49] *** Mateon1 has quit IRC (Remote host closed the connection) [06:50] *** kimmer2 has quit IRC (Ping timeout: 633 seconds) [06:50] *** Mateon1 has joined #archiveteam-bs [07:00] *** kimmer2 has joined #archiveteam-bs [07:10] *** omglolbah has quit IRC (Ping timeout: 250 seconds) [07:10] *** tuluu has quit IRC (Read error: Operation timed out) [07:11] *** tuluu has joined #archiveteam-bs [07:12] *** Mateon1 has quit IRC (Read error: Operation timed out) [07:13] *** Mateon1 has joined #archiveteam-bs [07:15] *** Dimtree has joined #archiveteam-bs [07:27] *** Asparagir has quit IRC (Asparagir) [08:02] *** Mateon1 has quit IRC (Remote host closed the connection) [08:04] *** Mateon1 has joined #archiveteam-bs [08:14] JAA: Distributed archiving using celery and an IRC bot are functional, but not checked in yet. I’m setting up a testing environment currently. [08:33] *** du_ has quit IRC (Ping timeout: 260 seconds) [08:42] *** Mateon1 has quit IRC (Remote host closed the connection) [08:43] *** Mateon1 has joined #archiveteam-bs [10:14] *** godane has quit IRC (Read error: Operation timed out) [10:25] *** godane has joined #archiveteam-bs [10:57] *** pizzaiolo has joined #archiveteam-bs [11:40] *** BlueMaxim has quit IRC (Quit: Leaving) [11:56] *** refeed has joined #archiveteam-bs [11:59] *** refeed has quit IRC (Client Quit) [11:59] *** refeed has joined #archiveteam-bs [12:06] *** _refeed_ has joined #archiveteam-bs [12:06] *** refeed has quit IRC (Read error: Connection reset by peer) [12:08] *** Mateon1 has quit IRC (Remote host closed the connection) [12:09] *** Mateon1 has joined #archiveteam-bs [12:20] *** __refeed_ has joined #archiveteam-bs [12:20] *** _refeed_ has quit IRC (Read error: Connection reset by peer) [12:29] *** _refeed_ has joined #archiveteam-bs [12:29] *** __refeed_ has quit IRC (Read error: Connection reset by peer) [12:33] *** refeed has joined #archiveteam-bs [12:33] *** refeed has quit IRC (Connection closed) [12:34] *** _refeed_ has quit IRC (Read error: Connection reset by peer) [12:34] *** refeed has joined #archiveteam-bs [12:35] *** refeed has quit IRC (Client Quit) [12:48] *** Mateon1 has quit IRC (Remote host closed the connection) [12:49] *** Mateon1 has joined #archiveteam-bs [13:10] *** Nugamus has joined #archiveteam-bs [13:18] *** godane has quit IRC (Quit: Leaving.) [13:23] *** TheLovina has quit IRC (Read error: Operation timed out) [14:05] *** du_ has joined #archiveteam-bs [14:21] *** K4k has quit IRC (Ping timeout: 260 seconds) [14:29] *** K4k has joined #archiveteam-bs [14:32] *** Stilett0 has joined #archiveteam-bs [14:35] *** K4k has quit IRC (Ping timeout: 250 seconds) [14:35] SketchCow: Another random "a" file in item archiveteam_archivebot_go_20171210190001 (see my messages in here about three days ago). This time, it's a WARC again. [14:35] See also #archivebot [14:40] *** K4k has joined #archiveteam-bs [14:48] Figured it out, it was a typo on one of the ArchiveBot pipelines. I hope it didn't overwrite any files. [14:49] I'll send you a list of what each of those files should be named via email. [15:05] *** Mateon1 has quit IRC (Remote host closed the connection) [15:06] *** Stilett0 is now known as Stiletto [15:32] Okay, my first pass of the parsed Miiverse Database finished up over the weekend. I have to redownload some WARCs from IA that were corrupted and validate one more time to make sure I got everything, but it's a pretty good representation of what we got last month. Total size so far of the database is 191 GB. [15:33] We saved 2,117,420 deleted posts (by marking their non-existence), 128,727,868 posts, and 206,473,819 replies. [15:33] We saved 69,955,548 drawings, of which 27,086,692 were posted in replies, which Nintendo didn't send to users in their "archive". [15:33] Likewise, we saved 72,985,588 screenshots, with 14,089,107 posted in replies. [15:34] Here's a spreadsheet of the amount of posts and replies per day. https://usercontent.irccloud-cdn.com/file/UJeNuOnR/TotalTimestamp.xlsx [15:37] Once I validate the database, I'll throw it on IA. [15:44] *** refeed has joined #archiveteam-bs [15:44] *** refeed has quit IRC (Connection closed) [15:45] *** refeed has joined #archiveteam-bs [15:56] *** _refeed_ has joined #archiveteam-bs [16:00] *** refeed has quit IRC (Read error: Connection reset by peer) [16:31] *** __refeed_ has joined #archiveteam-bs [16:34] *** refeed has joined #archiveteam-bs [16:35] *** _refeed_ has quit IRC (Read error: Connection reset by peer) [16:38] *** __refeed_ has quit IRC (Read error: Connection reset by peer) [16:38] JAA: HI [16:38] Yeah, I'm all for fixing these after the fact [16:38] FOS is dying under this Manga Plus Everything Else [16:39] *** _refeed_ has joined #archiveteam-bs [16:39] So I'm going to just upload and then we'll issue fixes and re-checks [16:39] SketchCow: i've raised the first pull to fix the 2.0.3 issue [16:39] I'm writing the changes to uploader now [16:42] *** refeed has quit IRC (Read error: Connection reset by peer) [16:45] Great [16:46] But yeah, we are due lots of audit of already uploaded shiznat, frankly [16:46] I do it when I can [16:46] Right now, though, FOS is definitely heaving and I'm trying to fix that [16:46] Some of it is just because it's the new FOS and automatic upload processes weren't running [16:46] Other is MANGA [16:46] MANGA baby [16:47] Sure, we can fix that down the line as well. [16:48] Can we prevent files being overwritten though? [16:48] We probably lost about a third of Cracked due to that, for example. [16:48] Is cracked deleted? [16:48] I mean, we can re-do it [16:49] No, it's still online. [16:49] But everything I see is that it is on the rsync command side, not the server side. [16:49] So some --ignore-existing or something [16:49] Hmm. [16:49] So it's the pipeline, really [16:49] Yeah, it should be possible to do something there. [16:49] I don't THINK there's an rsyncd setting (or rsyncd.conf setting) to stop overwriting, but I'll look [16:50] https://download.samba.org/pub/rsync/rsyncd.conf.html [16:50] If someone sees something, I'll toss it in [16:51] I mean, I looked and it REALLY looks like it's on the rsync command's side to not be the asshole [16:54] Maybe something could be done with "filter". [16:54] "Files excluded by the daemon filter chain (daemon-excluded files) are treated as non-existent if the client tries to pull them, are skipped with an error message if the client tries to push them (triggering exit code 23), and are never deleted from the module." [16:56] That filter would have to be updated with the current file list after every upload, I think. [16:56] This is too much work. [16:56] Client just needs a new setting. [16:56] *** __refeed_ has joined #archiveteam-bs [16:57] https://storify.com/faq-eol [16:59] ha ha wow [17:00] *** _refeed_ has quit IRC (Read error: Connection reset by peer) [17:02] SketchCow: I can come up with a number of ugly workarounds. For example, periodically moving the files to a different directory with mv -n. The rsync target directory would just be an "inbox". If that move command is running every few minutes, the probability of anything being overwritten should be very small. [17:02] *** CoolCanuk has joined #archiveteam-bs [17:02] Or chattr +i every file. (Needs to be undone to delete it after upload to the IA systems though, obviously.) [17:03] I heard JAA's IP is banned from wine.woot [17:03] In any case, yes, we should add a flag to the client side, but I'm not a fan of relying on that. [17:03] I think that if we don't rely on the clients being functional, we're sunk anyway [17:04] Well yeah, but the clients should only be able to do what they're supposed to do, i.e. push new data onto FOS. [17:05] how about just using `incoming chmod` to set `ugo-w` for each transfer, preventing further writes to that file? [17:05] not sure if that'll work [17:05] *** K4k has quit IRC (Read error: Operation timed out) [17:06] it doesn't seem to have an incoming chown option or I'd have suggested changing its owner to a non-rsync owner [17:06] I'd be surprised if it did, to be honest. [17:06] worth a shot? [17:06] It's the write permission of the directory that dictates whether you're able to overwrite a file. [17:06] I... don't think so? [17:07] touch file && chmod -w file && rm -f file [17:07] I don't think rm will restore the write permission before it deletes the file. [17:07] Deleting a file simply means removing the directory entry, so... [17:07] *** sep332 has joined #archiveteam-bs [17:07] And you *can* overwrite read-only files with rsync, so... [17:07] hm [17:10] JAA: i'll update the uploader.py too [17:10] Thanks [17:11] *** bwn has quit IRC (Read error: Operation timed out) [17:22] *** Cyn_ has joined #archiveteam-bs [17:24] *** BnAboyZ has quit IRC (Quit: The Lounge - https://thelounge.github.io) [17:48] *** icedice has joined #archiveteam-bs [17:49] *** schbirid has joined #archiveteam-bs [17:56] *** _refeed_ has joined #archiveteam-bs [18:01] *** __refeed_ has quit IRC (Read error: Connection reset by peer) [18:07] *** K4k has joined #archiveteam-bs [18:25] *** icedice has quit IRC (Read error: Operation timed out) [18:29] *** icedice has joined #archiveteam-bs [18:31] *** icedice2 has joined #archiveteam-bs [18:35] *** icedice has quit IRC (Ping timeout: 260 seconds) [18:54] *** bwn has joined #archiveteam-bs [18:56] *** __refeed_ has joined #archiveteam-bs [19:01] *** _refeed_ has quit IRC (Read error: Connection reset by peer) [19:02] Does anyone know if myspleen invites are open? [19:15] *** icedice has joined #archiveteam-bs [19:16] *** icedice has quit IRC (Client Quit) [19:20] *** icedice2 has quit IRC (Read error: Operation timed out) [19:30] *** omglolbah has joined #archiveteam-bs [19:37] *** jschwart has joined #archiveteam-bs [19:37] *** ola_norsk has joined #archiveteam-bs [19:38] does anyone know if "/history/file/*~1~" serve a special purpose on internet archive? [19:38] example? [19:39] the 'history' folder does not seem to show up in item editor [19:39] pls link to an example, i have not seen this before [19:39] 1 sec [19:43] in the item sowhat-vidme_archive, "ia ls " returns several files listed with such a directory (history/files/), e.g the file "history/files/20171111_YouTubers explain YT Censorship_Iv3WZ.description.~1~" [19:44] and the ".~1~" seems to have been added to original filename [19:44] filenames* [19:45] looks like it retains changes in a directory /history/files/ https://archive.org/download/sowhat-vidme_archive/history/files/ [19:45] but other items that i uploaded and modified don't have the directory [19:46] first time i've seen it [19:46] i know .~1~ is used as a version suffix for backup files that emacs and some other tools create [19:47] is it possible it's just a case of wrong detection of file format? [19:47] or do you mean e.g -re-uploaed files? [19:47] thinking the second one [19:48] On that job a lot of work was re-done [19:48] So likely [19:48] the guy apparently has 1080+ videos on his vidme [19:48] so 'a lot of work' might also make sense :D [19:49] We've done somewhere around 70TB of data from vid.me [19:49] It's around 1.2PB in size [19:50] i simply focused on some channels with videos that i know are not on youtube.. E.g "AfterPrisonJoe" from "AfterPrisonShow" on youtube [19:51] wasn't it 600TB ? [19:51] That was our estimate. We got hold of someone who worked there, It's a lot more... [19:51] doh [19:54] maybe 'job scripts' should simply be made the moment a new service pops up :D [19:55] though there's no doubt a majority of that 1.2PB is most likely already existing on youtube [19:56] *** _refeed_ has joined #archiveteam-bs [20:00] *** __refeed_ has quit IRC (Read error: Connection reset by peer) [20:01] a bit is the users fault though, as i see it. There's e.g no reason for a having a news commentary video in 1080p, max bitrate, and at 60fps.. [20:02] example, the 'reading scary creepypasta stories' [20:03] most often composed of someone narrating over what's basically slideshows of still images [20:04] That should compress extremely well though. [20:04] yeah, but if there's smoke effects over it [20:06] *** __refeed_ has joined #archiveteam-bs [20:08] i don't know how compression on vidme worked. But most of the files in the item i mentioned at join are basically the person browsing german newspapers while delivering english commentary. Some are 1 hour + [20:09] and i think i saw one of those 1+h videoes being 1+ GB [20:09] i'm not saying i pirate movies, but 1.3GB seems to be well for a 1080p movie [20:10] *** _refeed_ has quit IRC (Read error: Connection reset by peer) [20:12] 1.3GB for a full length movie at 1080p is bitrate starved [20:14] 'starved' ? [20:20] it looks like shit [20:20] yeah, as in, not enough bitrate to maintain a high level of quality at that resolution, at least for a live action movie. if a 1.3GB video is 2 hours long, the average bitrate is about 1.4Mbps. it's subjective and does depend on the codec, but to me it seems low for 1080p. [20:21] "eye of the beholder then" :) [20:21] It measurably bad in the psychovisual model. [20:22] 'measurably worse'...'bad' is relative [20:22] but anyway, i'm not codec expert :D [20:23] Who doesn't like nice blocks and banding? /s [20:24] JAA: VP9 pisses me off with how much banding it creates in dark scenes [20:24] even at relatively high nitrates [20:25] bit [20:29] Don't have much experience with VP9. [20:29] feel like it's a step backwards from H.264 at least in that record [20:29] regard* (autocorrect) [20:30] Uhm, did IA just change the /save page? [20:30] :/ [20:30] web.archive.org/save ? [20:31] I'm getting a page with a textarea "List of archived elements in reverse order" instead of the usual save page. [20:31] Yeah [20:31] i hope not [20:31] It also doesn't actually archive anything for me. [20:31] As far as I can tell anyway. [20:31] Shows same as normal to me [20:32] as long as it's just the looks [20:33] *** Coderjo has quit IRC (Read error: Operation timed out) [20:33] *** yipdw has quit IRC (Read error: Operation timed out) [20:33] how else could i be capturing rather repetitiv tweets about #netneutrality :/ [20:33] #archivebot [20:33] ;-) [20:33] *** me has joined #archiveteam-bs [20:33] wouldn't that mean i had to download it first? [20:34] https://share.riseup.net/#tuQ-OKPcwjop11jI_XzhzA [20:34] Nope, Just feed it URLs [20:34] It's just sitting there forever. [20:34] And then it does the rest [20:34] For Twitter, make sure you use --phantomjs with archivebot [20:34] Since Twitter uses JS for pretty much everything [20:34] it would autoscroll? [20:34] Indeed [20:34] No, because it's broken. [20:35] Use the right pipeline and it works ;-) [20:35] Well, yeah. [20:35] Does it work on yours? [20:35] On CA it does [20:35] I haven't tested the rest [20:36] Let's move this to over there. [20:36] *** atomicthu has quit IRC (Ping timeout: 260 seconds) [20:36] Nope, can't save anything into the Wayback Machine right now. Ew. [20:38] *** atomicthu has joined #archiveteam-bs [20:41] *** kimmer1 has joined #archiveteam-bs [20:42] *** Coderjo has joined #archiveteam-bs [20:46] *** kimmer12 has quit IRC (Read error: Operation timed out) [20:46] :O [20:46] did WE break it?!? [20:47] Lol no [20:48] /save/ appears broken for me too [20:52] same [20:56] *** _refeed_ has joined #archiveteam-bs [20:57] Good to see that I'm not the only one. I've sent an email to info@ just in case they don't know about it already. [21:00] *** __refeed_ has quit IRC (Read error: Connection reset by peer) [21:11] *** _refeed_ has quit IRC (Leaving) [21:20] Somebody2: I noticed in #archivebot logs a few months ago you tagged me and a few other people and said that the owner of autistics.org wanted to pass the site onto someone else, and that we could redirect stuff to pages on the wayback machine. If that's still open I'd be interested in that. [21:36] the problem with 'tweep' is that it seems to be oriented to specified user account [21:37] oooops', no it doesn't "python tweep.py -s pineapple - Collect every Tweet containing pineapple from everyone's Tweets." [21:40] Yep, and it looks like you can just search for the hashtag there. [21:41] (For those out of the loop, we're talking about https://github.com/haccer/tweep ) [21:44] what if it just bounce around on twitter domain, checking every tweet? :D that could take some time :D [21:45] Have fun with that. [21:45] skål! [22:17] Got a reply from Mark, they're "testing an update to page of our Save Page Now service". He asked me to try again, and my saves went through this time. [22:23] *** Cyn_ has quit IRC (Ping timeout: 262 seconds) [22:25] tweep appears to be broken :/ [22:25] Nyaaaa! YouTube won't let accounts upload more than 100 videos per day. [22:26] Progress on the news archive is gonna be slow. :) [22:27] antomatic: at least that's not a sign it's all going to hell :) [22:30] *** schbirid has quit IRC (Quit: Leaving) [22:31] *** PoorHomes has joined #archiveteam-bs [22:32] *** PoorHomie has quit IRC (Quit: No) [22:33] it's about high time users adopt a pessim-op-tic view of "the internet" https://youtu.be/1VD_pJOFnZ0 [22:33] If I upload 100 items each and every day, then I think I can get everything up to the end of 2017 uploaded... by the end of 2018. :) [22:33] *** PoorHomes is now known as PoorHomie [22:34] antomatic: if i had to guess, the cause might be users of several nations trying to reupload their catalog that they moved to vidme [22:35] Heh. I think it's just a generic dont-spam-us-too-hard measure. :) [22:36] antomatic: that, or youtube's wallet is getting a bit scrawny for Google to be happy with.. :/ [22:36] This is a waste of time anyway with the 3 strikes rule, because /someone/ is bound to object hard enough to it and the whole lot comes down [22:37] still, gives the opportunity to trim the recordings nicely, collate some metadata and run out the caption transcripts. [22:37] antomatic: Can you upload them to archive.org? [22:38] the entire internet needs a trimming, there's too much monopoly going on [22:38] I don't know if archive.org necessarily wants 30tb+ of news bulletins though. :) [22:39] did you email info@ like we discussed [22:39] did we discussed? sorry, must have missed that [22:40] Don't they already record tv news though? [22:40] US/International yes, I think, but I don't know if they record UK domestic [22:41] yeah we spoke about this a few days ago [22:41] *** pizzaiolo has quit IRC (Ping timeout: 246 seconds) [22:41] look in your logs around 5th december [22:41] you and i [22:42] I asked Jason if IA records Catalan TV once and he said "ish" [22:43] 'so and so' recording? [22:43] astird: ah, yes, I did see that. [22:43] thx [22:44] wanted to check whether I had a decent process for transcoding & transcribing it all first [22:44] where can i find which channels IA records? most of NRK (norwegian state television) is nation locked [22:45] oops, sorry, astrid. thanks. typo. :) [22:45] IA will transcode it [22:45] transcription is something else [22:46] youtube caption autogenerated is horrible :( [22:46] Mm, some of the source files are a little proprietary, but I can bounce them out to .TS which I believe IA accepts [22:46] I can extract the live captioning to SRT, which should be /something/ [22:47] oh that's good [22:47] although the accuracy is a little ropey. seems to have improved some over the last few years [22:47] YouTube's autosubs are actually *amazingly* good - in their own way - compared to the broadcast subs [22:48] not only is it vtt, but their translatiation is shit. Even at rather clean english (i think?) https://youtu.be/fpZD_3D8_WQ [22:48] But they totally fall to bits when background noise reaches a certain level [22:48] ah... [22:48] They've improved *enormously* over the last year or so [22:48] that video got that :D [22:48] still a long way from perfect though [22:49] *** Pixi has quit IRC (Ping timeout: 255 seconds) [22:49] hook54321: CCMA? [22:50] *** Pixi has joined #archiveteam-bs [22:51] antomatic: and yet, there's Google Deepmind AI able to master chess after just 4 hours.. [22:51] * ola_norsk is not impressed [22:54] if i was more Alex Jonesy, i'd be likely to say that when these supermachines beat human champs in Chess and Go, it's got to do with money paid to the looser... [22:54] Did you see that video of Alex Jones 'interviewing' an Amazon Echo? :) [22:54] lol no [22:55] I like Alex 'coloidal silver niacent' Jones :D [22:55] He's all like 'Alexa, who is Jeff Bezos?', 'Alexa, you are lying to me', etc. Nuts. :) [22:56] Alex's got 'Brain Power' to prevent being beaten by any disinformation or misleading [22:57] Hehe. :) [22:57] Do you have a link to that "interview", sounds hilarious [22:57] aye, givvus link [22:58] https://www.youtube.com/watch?v=u5kNP7tyhk8 [22:59] thx [22:59] I'll be watching that in a private browsing window so as to not pollute my Youtube history [22:59] *** JAA sets mode: +b *!*@185.143.40.157 [23:00] (Just a preventive ban for a spammer that showed up in #vidmeh and #miiworse) [23:00] lol, i love how he's stern when saying "Alexa! ; .." ..like talking to a little girl child :D [23:00] "Alexa, I have mainstream news articles that Amazon is owned by the CIA" XD [23:01] "Aleeexaaaa; did you steal out of fridge?" [23:01] *** BlueMaxim has joined #archiveteam-bs [23:02] LOL, Alex Jones fails the Turing test [23:02] That *look* on his face, like he's seriously thinking everything through and knows that if he asks the right question he'll blow the whole thing open at any moment. :) [23:03] lol [23:03] "Alexaaaaa! Have you finished your homework?" [23:04] "Alexaaa! Did you take out the trash?" [23:04] Wait, did he just spin that whole crazy video out of that deal that Amazon made to have a dedicatd AWS region for the federal government??? [23:06] well, i guess it's settled, it's the "ultimate control system" [23:06] Mm, there was a viral [jukin] the day before where someone asked their Alexa about the CIA and it turned off, apparently. [23:07] (well, just didn't answer, really) [23:07] for it to detect even "Alexa!", it DOES mean that it's listening all the time though.. [23:08] mm, it's listening locally (not streaming to the cloud) until it hears 'Alexa', then it sends your voice to the cloud to be recognised/answered. [23:08] supposedly. [23:09] but of course, it's an internet-connected mic and how it works is just down to the software. [23:11] *** CoolCanuk has quit IRC (Quit: Connection closed for inactivity) [23:11] But TBF, the mute button on the top is a hardwired switch [23:11] ah, nice [23:12] Which shows at least a little bit about security [23:12] *** CoolCanuk has joined #archiveteam-bs [23:12] Which shows at least Amazon cares a little bit about security * [23:12] important for when Alex is discussing international affairs. :) [23:12] Whoa. Alex. Alexa. How deep does the conspiracy go? :) [23:13] someone needs to check out Alexa with wireshark! See when she says something to the internet and when she doesn't! [23:13] lol [23:13] it's been checked to death [23:13] aye [23:13] "Alexa. Order ladies' panties." [23:13] it only streams when you say Alexa [23:13] period [23:14] But of course, whoever controls the update keys can change that at any second, without your knowledge [23:14] *** JAA sets mode: +b botnickna!*@* [23:14] *** JAA sets mode: +b *!*@185.143.40.* [23:14] do not adjust your set. we control the signing, we control the update. [23:14] they should pick starwars names like R2D2 and shit like that..There's people named Siri and Alexa :D [23:15] You can change the trigger word to "Amazon" [23:15] Also 'Echo' and now 'Computer' apparently, which is especially excellent. [23:15] but in my household that would be even more annoying [23:16] we use the shit out of prime so every 4th word is amazon lol [23:16] PoorHomie: are you here to archive or to talk about irrelevant shit [23:19] is there anyway to figure out at what point 'tweep' (python) fails ? [23:19] without having to rewrite the entire damn thing :/ [23:20] *** JAA sets mode: +b *!*@37.237.65.* [23:20] *** JAA sets mode: +b Ya_ALLAH_!*@* [23:20] a python debugger of sorts?.. :/ [23:21] @astrid, excuse me? [23:21] ola_norsk: How does it fail? [23:21] JAA: quietly :/ [23:22] JAA: no output whatsoever, no file, no text output [23:25] Hmm, that's odd. [23:25] aye [23:26] the 'pip install image' command in requisties section on their page is apparently outdated though [23:27] so that failed, and apparently that is reference to PIL, witch is currently 'Pillow" :/ [23:28] when running 'pip install image' it even looked for Django..so something is odd [23:28] could be my machine is messed up though [23:29] You could just comment out the from PIL import Image line in the script if you don't use --pics. [23:31] i've only tried the 2 non-user specific commands "python tweep.py -s pineapple" , "python tweep.py -s "Donald Trump" --verified --users" [23:32] no --picture argument added to them [23:32] both fail without any output [23:32] Hmm [23:33] if there was output, i'd have something to go by [23:33] JAA: what python version you use? [23:34] *** Atom has quit IRC (Read error: Connection reset by peer) [23:34] i tried using it on 2.7.14 [23:34] ola_norsk: I can reproduce that with 2.7.something. [23:34] *** BnAboyZ has joined #archiveteam-bs [23:35] *** MrDignity has joined #archiveteam-bs [23:36] PoorHomie: let's rolle the pinapple https://youtu.be/zwF_mGA-YQg [23:36] PoorHomie: ;) [23:38] PoorHomie: just tell astrid to mute you at discretion, and you'd be fine :) [23:38] ola_norsk: pip install lxml [23:38] Undocumented requirement of the script, apparently. [23:38] ola_norsk: pip install django==1.11.8 [23:38] Then it works when commenting out the PIL line. [23:39] ty [23:39] django2.0+ is for Python 3 [23:40] And that's excellent. Python 2 needs to die already. [23:41] *** Atom has joined #archiveteam-bs [23:42] might as well used c++ the strict way python 3 is going :D [23:45] Collecting lmxl [23:45] Could not find a version that satisfies the requirement lmxl (from versions: ) [23:45] No matching distribution found for lmxl [23:45] JAA: what distro are you using? [23:45] Debian [23:46] ill try that before i mess up even more [23:46] What are you running? [23:46] lubuntu [23:46] Hmm [23:47] lxml should be in the repos, I think. [23:47] lubuntu Ubuntu 17.10 \n \l [23:47] You could try installing the python-lxml package. [23:47] Otherwise, you might need a bunch of -dev packages of various libraries to install from source. [23:48] ty [23:48] This will install it system-wide, obviously. Just so you're aware of that. [23:48] yeah but it doesn't matter [23:48] I'm using pyenv to handle numerous independent installations of Python etc. [23:49] that worked though [23:49] i'm not sure what's scrolling by at the momment, but somethings working [23:50] It just prints tweet ID, date, username, and message. [23:50] aye, "python tweep.py -s "Donald Trump" --verified --users" seems to be usernames [23:51] it's scrolling as fuck though :D [23:51] ty! [23:53] if e.g "940731016880312321" is the id of a tweet, urls can be constructed from that [23:53] Yep, together with the username. [23:53] 940731016880312321 2017-12-12 23:52:07 CET u.s. congress after they vote to have net neutrality repealed then realize they gotta pay the package deals just like the rest of us. (go save net neutrality y’all: http://battleforthenet.com )pic.twitter.com/WdM [23:54] maybe is user? [23:54] 940731013398904832 2017-12-12 23:52:06 CET Keaton Jones ™ is a distraction put by the US Government to distract you from Net Neutrality [23:54] Yes, that's the username. [23:54] this is badass [23:55] waybackmachine is going to feel this :D [23:55] So the URL for the first one would be https://twitter.com/drkbri/status/940731016880312321 [23:56] wget get requisites getting, i think this is a winner [23:56] with* [23:56] ola_norsk: python tweep.py -s pineapple | head | awk '{print "https://twitter.com/" substr($5, 2, length($5) - 2) "/status/" $1}' [23:59] anyone good at math to calculate what a file containing link to one tweet, back to ~2006 might require of storage? [23:59] I don't understand the question.