[00:05] *** Sanqui has joined #archiveteam-bs [00:05] *** Kaz has joined #archiveteam-bs [00:05] *** abstract has joined #archiveteam-bs [00:05] *** svchfoo1 sets mode: +o Kaz [00:05] *** PotcFdk has joined #archiveteam-bs [00:06] *** apache2 has joined #archiveteam-bs [00:06] *** acridAxid has joined #archiveteam-bs [00:08] *** atg has joined #archiveteam-bs [00:09] *** riking_ has joined #archiveteam-bs [00:09] *** lenary has joined #archiveteam-bs [00:09] *** lenary has quit IRC (Read error: Connection reset by peer) [00:09] *** bashNinja has joined #archiveteam-bs [00:10] *** diggan has joined #archiveteam-bs [00:10] *** diggan has quit IRC (Read error: Connection reset by peer) [00:10] *** riking_ has quit IRC (Read error: Connection reset by peer) [00:11] *** kpcyrd has joined #archiveteam-bs [00:11] *** revi has joined #archiveteam-bs [00:12] *** justcool4 has joined #archiveteam-bs [00:13] *** riking_ has joined #archiveteam-bs [00:15] *** diggan has joined #archiveteam-bs [00:17] *** lenary has joined #archiveteam-bs [00:19] *** mattl has joined #archiveteam-bs [00:25] *** justcool4 is now known as justcool3 [00:30] *** abartov__ has joined #archiveteam-bs [00:31] *** picklefac has joined #archiveteam-bs [00:34] *** DrasticAc has joined #archiveteam-bs [00:35] *** pnJay has joined #archiveteam-bs [00:38] *** amelia386 has joined #archiveteam-bs [00:39] *** tchaypo_ has joined #archiveteam-bs [00:39] *** starlord has joined #archiveteam-bs [00:40] *** jesse-s has joined #archiveteam-bs [00:41] *** alex73 has joined #archiveteam-bs [00:41] *** Vito` has joined #archiveteam-bs [00:42] *** JSharp___ has joined #archiveteam-bs [00:43] *** ThisAsYou has joined #archiveteam-bs [00:45] *** xit has joined #archiveteam-bs [00:46] *** fallenoak has joined #archiveteam-bs [00:47] *** tech234a has joined #archiveteam-bs [00:47] *** hook54321 has joined #archiveteam-bs [00:47] *** deathy__ has joined #archiveteam-bs [00:47] *** Iglooop1 sets mode: +o hook54321 [00:47] *** Ctrl-S___ has joined #archiveteam-bs [00:50] *** Ivy has joined #archiveteam-bs [00:50] *** c0mpass has joined #archiveteam-bs [00:51] *** HCross has joined #archiveteam-bs [00:51] *** horkermon has joined #archiveteam-bs [01:28] *** pew has quit IRC (Ping timeout: 276 seconds) [01:37] *** coderobe has joined #archiveteam-bs [01:39] *** pew has joined #archiveteam-bs [02:38] *** ShellyRol has quit IRC (Ping timeout: 492 seconds) [02:42] *** ShellyRol has joined #archiveteam-bs [03:17] *** qw3rty__ has joined #archiveteam-bs [03:21] *** qw3rty_ has quit IRC (Ping timeout: 276 seconds) [03:22] *** wp494 has joined #archiveteam-bs [03:30] *** Fionera has joined #archiveteam-bs [03:31] *** MillerBOS has quit IRC (Read error: Operation timed out) [03:31] *** Fionera_ has quit IRC (Read error: Operation timed out) [03:31] *** halt_ has quit IRC (Read error: Operation timed out) [03:31] *** MillerBO- has joined #archiveteam-bs [03:32] *** MillerBO- is now known as MillerBOS [03:32] *** wyatt8740 has quit IRC (Read error: Operation timed out) [03:32] *** wyatt8740 has joined #archiveteam-bs [03:33] *** halt_ has joined #archiveteam-bs [04:43] *** vitzli has joined #archiveteam-bs [04:46] *** ShellyRol has quit IRC (Read error: Operation timed out) [04:48] *** lennier1 has quit IRC (Read error: Operation timed out) [04:49] *** lennier1 has joined #archiveteam-bs [04:56] *** lennier2 has joined #archiveteam-bs [05:01] *** ShellyRol has joined #archiveteam-bs [05:01] *** lennier1 has quit IRC (Ping timeout: 496 seconds) [05:02] *** lennier2 is now known as lennier1 [08:45] *** Ryz has quit IRC (Read error: Connection reset by peer) [08:45] *** kiska1825 has quit IRC (Read error: Connection reset by peer) [08:45] *** Ryz8 has joined #archiveteam-bs [08:45] *** kiska1825 has joined #archiveteam-bs [08:57] *** vitzli has quit IRC (Leaving) [09:34] *** BlueMax has joined #archiveteam-bs [09:43] *** BlueMaxim has joined #archiveteam-bs [09:49] *** lennier2 has joined #archiveteam-bs [09:49] *** Muad-Dib has joined #archiveteam-bs [09:51] *** godane has quit IRC (Read error: Connection reset by peer) [09:53] *** BlueMax has quit IRC (Ping timeout: 745 seconds) [09:55] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [09:57] *** lennier1 has quit IRC (Ping timeout: 745 seconds) [10:06] *** lennier2_ has joined #archiveteam-bs [10:06] *** lennier2_ is now known as lennier1 [10:09] *** lennier2_ has joined #archiveteam-bs [10:10] *** lennier2 has quit IRC (Ping timeout: 745 seconds) [10:14] *** lennier1 has quit IRC (Read error: Operation timed out) [10:14] *** lennier2_ is now known as lennier1 [10:21] *** ivan has joined #archiveteam-bs [10:23] *** mgrytbak has joined #archiveteam-bs [10:29] *** Jens has joined #archiveteam-bs [10:30] *** Jens has quit IRC (Client Quit) [10:32] *** Jens has joined #archiveteam-bs [10:57] *** Jens has quit IRC (Quit: Jens) [10:58] *** Jens has joined #archiveteam-bs [11:02] *** ShellyRol has quit IRC (Read error: Operation timed out) [11:03] *** ShellyRol has joined #archiveteam-bs [11:43] *** Meroje has joined #archiveteam-bs [12:06] *** tomaspark has joined #archiveteam-bs [12:53] *** Raccoon has quit IRC (Remote host closed the connection) [12:59] *** Raccoon has joined #archiveteam-bs [13:02] *** Raccoon has quit IRC (Remote host closed the connection) [13:31] *** balrog has quit IRC (Ping timeout: 492 seconds) [14:00] *** balrog has joined #archiveteam-bs [14:48] *** godane has joined #archiveteam-bs [15:33] *** tempnicl has joined #archiveteam-bs [15:34] Hi. I need to archive the past few days of a twitter account, including its responses to other tweets. Can you fine people give me some advise on best practices? [15:34] @JAA recommended the use of snscrape + grab-site. [15:35] This will give me a WARC. How can I export the tweets in a text-format as well, that can be easily published to a e.g. a gist? [15:35] Ah, if you just want the contents directly, not a proper archive, then just snscrape. [15:36] Specifically, with --format, you can produce any output you like. [15:37] It's not really documented currently, but here's what you can use: https://github.com/JustAnotherArchivist/snscrape/blob/b6cc3180d97f1f9e9004f52e832333678d8c46f7/snscrape/modules/twitter.py#L15-L24 [15:37] Ah, interesting. I thought snscrape only returns urls to the tweets which still have to be processed by some other tool [15:37] That's what it does by default, yes. [15:37] *** ShellyRol has quit IRC (Read error: Operation timed out) [15:38] Example usage: [15:38] > snscrape -n 1 --format '{content!r}' twitter-user textfiles [15:38] *** ShellyRol has joined #archiveteam-bs [15:38] '@0x29 @polm23 I have a flux reader if they want to send it along.' [15:38] The --format value is a Python formatting string, and the variables are what I linked above. [15:39] (Beware of linebreaks etc.) [15:42] @JAA, I really don't grok snstools help output. How can I get the actual tweets by a user? I used `snstool twitter-user someuser > urls`. But, what now? Can you help? [15:42] I.e. each tweets content [15:42] Example above. [15:43] I don't know what information you want exactly and in what format. [15:43] Ah, overlooked that. Thx [15:44] If you need it even more customisable (e.g. JSON output), you'd have to use snscrape as a package through Python directly. That's completely undocumented though. [15:48] @JAA, best output would be .csv. How can I add the url to `--format '{content!r}'` ? I tried with ` --format '{content,url!r}'` which seems to be wrong... [15:49] What does the `!r` do? [15:49] '{url},{content!r}' or similar [15:50] !r is the same as repr(). It transforms the value into something that could be used again directly as a string in Python, in this case. [15:50] So linebreaks get translated into \n for example. [15:51] Ah, thx! [15:51] I'm not sure you can produce completely valid CSV like this, by the way. If a tweet's text contains a comma, for example, that might be problematic. [15:52] Ah true [16:21] *** lennier2 has joined #archiveteam-bs [16:27] *** lennier1 has quit IRC (Read error: Operation timed out) [16:27] *** lennier2 is now known as lennier1 [16:37] *** Pixi` has quit IRC (Quit: Leaving) [16:37] *** Pixi has joined #archiveteam-bs [17:40] *** Maylay has quit IRC (Read error: Operation timed out) [18:04] *** Maylay has joined #archiveteam-bs [18:04] *** tempnicl has quit IRC (Quit: Page closed) [18:11] *** HP_Archiv has quit IRC (Ping timeout: 276 seconds) [18:12] *** HP_Archiv has joined #archiveteam-bs [18:21] *** Ajay1 has joined #archiveteam-bs [19:13] *** Mateon1 has joined #archiveteam-bs [19:24] *** Mateon1 has quit IRC (Ping timeout: 622 seconds) [19:27] *** Mateon1 has joined #archiveteam-bs [19:46] *** lennier2 has joined #archiveteam-bs [19:48] *** lennier1 has quit IRC (Read error: Operation timed out) [19:48] *** lennier2 is now known as lennier1 [20:26] *** godane has quit IRC (Read error: Operation timed out) [20:50] *** systwi_ has joined #archiveteam-bs [20:57] *** systwi has quit IRC (Read error: Operation timed out) [21:59] *** robogoat has quit IRC (Read error: Operation timed out) [22:35] *** Ryz8 is now known as Ryz [23:05] *** ndiddy_ is now known as ndiddy [23:53] *** BlueMax has joined #archiveteam-bs [23:59] *** HP_Archiv has quit IRC (Quit: Leaving)