#archiveteam-bs 2019-03-19,Tue

↑back Search

Time Nickname Message
00:10 πŸ”— Stilett0 has joined #archiveteam-bs
00:12 πŸ”— julientm has joined #archiveteam-bs
00:13 πŸ”— Stiletto has quit IRC (Read error: Operation timed out)
00:14 πŸ”— Stilett0 is now known as Stiletto
00:37 πŸ”— second has quit IRC (Remote host closed the connection)
00:50 πŸ”— julientm has quit IRC (Ping timeout: 252 seconds)
00:55 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
01:01 πŸ”— bitBaron has joined #archiveteam-bs
01:13 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
01:18 πŸ”— Odd0002_ has joined #archiveteam-bs
01:19 πŸ”— bitBaron has joined #archiveteam-bs
01:23 πŸ”— Odd0002 has quit IRC (Ping timeout: 615 seconds)
01:23 πŸ”— Odd0002_ is now known as Odd0002
01:31 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
01:36 πŸ”— julientm has joined #archiveteam-bs
01:39 πŸ”— Odd0002_ has joined #archiveteam-bs
01:42 πŸ”— Odd0002 has quit IRC (Ping timeout: 600 seconds)
01:42 πŸ”— Odd0002_ is now known as Odd0002
01:48 πŸ”— bitBaron has joined #archiveteam-bs
01:48 πŸ”— Odd0002_ has joined #archiveteam-bs
01:53 πŸ”— Odd0002 has quit IRC (Ping timeout: 600 seconds)
01:53 πŸ”— Odd0002_ is now known as Odd0002
02:06 πŸ”— SimpBrain has quit IRC (Read error: Operation timed out)
02:13 πŸ”— SimpBrain has joined #archiveteam-bs
02:17 πŸ”— second has joined #archiveteam-bs
02:31 πŸ”— julientm has quit IRC (Leaving)
02:35 πŸ”— ndiddy has quit IRC ()
02:41 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
02:59 πŸ”— julientm has joined #archiveteam-bs
03:05 πŸ”— icedice has quit IRC (Read error: Operation timed out)
03:08 πŸ”— Stiletto has quit IRC (Read error: Connection reset by peer)
03:08 πŸ”— Stiletto has joined #archiveteam-bs
03:13 πŸ”— marked has quit IRC (west.us.hub irc.Prison.NET)
03:13 πŸ”— achip has quit IRC (west.us.hub irc.Prison.NET)
03:13 πŸ”— SynMonger has quit IRC (west.us.hub irc.Prison.NET)
03:16 πŸ”— synm0nger has joined #archiveteam-bs
03:27 πŸ”— Dj-Wawa has quit IRC (Quit: Connection closed for inactivity)
03:32 πŸ”— achip has joined #archiveteam-bs
03:32 πŸ”— marked has joined #archiveteam-bs
03:36 πŸ”— znak has quit IRC (Read error: Operation timed out)
03:36 πŸ”— wabu has quit IRC (Read error: Operation timed out)
03:37 πŸ”— Polylith_ has quit IRC (Read error: Operation timed out)
03:37 πŸ”— simon816 has quit IRC (Ping timeout: 246 seconds)
03:37 πŸ”— c4rc4s has quit IRC (Read error: Operation timed out)
03:38 πŸ”— ivan has quit IRC (Ping timeout: 246 seconds)
03:38 πŸ”— swebb_ has joined #archiveteam-bs
03:38 πŸ”— swebb has quit IRC (Ping timeout: 246 seconds)
03:38 πŸ”— JAA has quit IRC (Ping timeout: 246 seconds)
03:38 πŸ”— K4k__ has quit IRC (Ping timeout: 246 seconds)
03:38 πŸ”— svchfoo1 has quit IRC (Ping timeout: 246 seconds)
03:39 πŸ”— swebb_ is now known as swebb
03:39 πŸ”— colona has quit IRC (Ping timeout: 246 seconds)
03:39 πŸ”— betamax has quit IRC (Ping timeout: 246 seconds)
03:39 πŸ”— sknebel has quit IRC (Ping timeout: 246 seconds)
03:39 πŸ”— joepie91 has quit IRC (Ping timeout: 246 seconds)
03:39 πŸ”— TC01 has quit IRC (Ping timeout: 246 seconds)
03:39 πŸ”— sknebel has joined #archiveteam-bs
03:39 πŸ”— TC01 has joined #archiveteam-bs
03:40 πŸ”— ivan has joined #archiveteam-bs
03:41 πŸ”— betamax has joined #archiveteam-bs
03:41 πŸ”— colona has joined #archiveteam-bs
03:46 πŸ”— Polylith has joined #archiveteam-bs
03:48 πŸ”— joepie91 has joined #archiveteam-bs
03:50 πŸ”— Despatche has quit IRC (Quit: Connection reset by deer)
03:50 πŸ”— wyatt8740 has joined #archiveteam-bs
03:52 πŸ”— K4k__ has joined #archiveteam-bs
03:58 πŸ”— znak has joined #archiveteam-bs
04:20 πŸ”— julientm has quit IRC (Remote host closed the connection)
04:28 πŸ”— Binzhou5 has joined #archiveteam-bs
04:31 πŸ”— qw3rty113 has joined #archiveteam-bs
04:35 πŸ”— SimpBrain has quit IRC (Read error: Connection reset by peer)
04:35 πŸ”— SimpBrain has joined #archiveteam-bs
04:36 πŸ”— c4rc4s has joined #archiveteam-bs
04:36 πŸ”— simon816 has joined #archiveteam-bs
04:37 πŸ”— svchfoo1 has joined #archiveteam-bs
04:37 πŸ”— qw3rty112 has quit IRC (Ping timeout: 600 seconds)
04:38 πŸ”— JAA has joined #archiveteam-bs
04:38 πŸ”— bakJAA sets mode: +o JAA
04:41 πŸ”— wabu has joined #archiveteam-bs
04:48 πŸ”— odemgi has joined #archiveteam-bs
04:50 πŸ”— odemgi_ has quit IRC (Ping timeout: 252 seconds)
04:52 πŸ”— powerKitt has joined #archiveteam-bs
04:52 πŸ”— powerKitt How would I use Wikiteam to dump a wiki with $wgEnableAPI=false; set?
04:53 πŸ”— eientei95 powerKitt: By using Special:Export?
04:54 πŸ”— powerKitt no Special:Export
04:56 πŸ”— powerKitt https://ggwiki.deepfreeze.it/index.php?title=Special:Export Trying to dump the GamerGate wiki but it's really locked down for some reason
04:56 πŸ”— odemg has quit IRC (Ping timeout: 615 seconds)
04:57 πŸ”— powerKitt (I don't agree with #GamerGate, but I think dumping it would be useful for future historians trying to understand what happened)
05:03 πŸ”— odemg has joined #archiveteam-bs
05:05 πŸ”— t3 We are featured on https://youtu.be/FeAMpG4KbEc.
05:05 πŸ”— t3 To back up Google+.
05:05 πŸ”— t3 It's the last part of the video.
05:09 πŸ”— powerKitt https://archive.org/details/youtube-FeAMpG4KbEc tubeup mirror
05:17 πŸ”— dhyan_nat has joined #archiveteam-bs
05:18 πŸ”— Binzhou5 has quit IRC (Quit: Page closed)
05:24 πŸ”— powerKitt has quit IRC (Quit: Page closed)
06:17 πŸ”— SimpBrain has quit IRC (Read error: Connection reset by peer)
06:20 πŸ”— SimpBrain has joined #archiveteam-bs
06:42 πŸ”— wp494 has quit IRC (Read error: Operation timed out)
06:43 πŸ”— wp494 has joined #archiveteam-bs
07:00 πŸ”— logchfoo4 starts logging #archiveteam-bs at Tue Mar 19 07:00:51 2019
07:00 πŸ”— logchfoo4 has joined #archiveteam-bs
07:01 πŸ”— atbk_ has joined #archiveteam-bs
07:01 πŸ”— LordNigh2 has joined #archiveteam-bs
07:02 πŸ”— atbk has quit IRC (Ping timeout: 615 seconds)
07:02 πŸ”— Laverne has joined #archiveteam-bs
07:02 πŸ”— kiskabak has joined #archiveteam-bs
07:02 πŸ”— xoxo has joined #archiveteam-bs
07:02 πŸ”— Kaz has joined #archiveteam-bs
07:02 πŸ”— efnet.portlane.se sets mode: +o Kaz
07:03 πŸ”— Gfy has joined #archiveteam-bs
07:09 πŸ”— underscor has joined #archiveteam-bs
07:12 πŸ”— C4K3_ has joined #archiveteam-bs
07:15 πŸ”— LordNigh2 is now known as Lord_Nigh
07:48 πŸ”— SimpBrain has quit IRC (Remote host closed the connection)
07:53 πŸ”— dhyan_nat has quit IRC (Read error: Operation timed out)
07:55 πŸ”— SimpBrain has joined #archiveteam-bs
08:39 πŸ”— svchfoo1 has quit IRC (Read error: Operation timed out)
08:40 πŸ”— logchfoo4 has quit IRC (Ping timeout: 246 seconds)
08:41 πŸ”— logchfoo0 starts logging #archiveteam-bs at Tue Mar 19 08:41:16 2019
08:41 πŸ”— logchfoo0 has joined #archiveteam-bs
08:54 πŸ”— SimpBrain has quit IRC (Read error: Connection reset by peer)
09:00 πŸ”— SimpBrain has joined #archiveteam-bs
09:22 πŸ”— dhyan_nat has joined #archiveteam-bs
09:38 πŸ”— S1mpbrain has joined #archiveteam-bs
09:38 πŸ”— wabu has joined #archiveteam-bs
09:38 πŸ”— c4rc4s has joined #archiveteam-bs
09:38 πŸ”— simon816 has joined #archiveteam-bs
09:39 πŸ”— SimpBrain has quit IRC (Read error: Connection reset by peer)
09:40 πŸ”— JAA has joined #archiveteam-bs
09:40 πŸ”— bakJAA sets mode: +o JAA
09:41 πŸ”— svchfoo1 has joined #archiveteam-bs
09:49 πŸ”— PurpleSym JAA: The tool tcp_closer ↑ works very well when gdb does not.
10:03 πŸ”— dhyan_nat has quit IRC (Read error: Operation timed out)
10:15 πŸ”— PurpleSym (With the -t parameter you could even run it as a cron job to auto-fix stuck connections.)
10:17 πŸ”— wyatt8740 has quit IRC (Ping timeout: 255 seconds)
10:26 πŸ”— dhyan_nat has joined #archiveteam-bs
10:43 πŸ”— dhyan_nat has quit IRC (Read error: Operation timed out)
10:45 πŸ”— dhyan_nat has joined #archiveteam-bs
10:49 πŸ”— BlueMax has quit IRC (Read error: Connection reset by peer)
10:56 πŸ”— Joseph__ has joined #archiveteam-bs
10:57 πŸ”— VerifiedJ has quit IRC (Ping timeout: 252 seconds)
11:19 πŸ”— dhyan_nat has quit IRC (Quit: Konversation terminated!)
11:19 πŸ”— dhyan_nat has joined #archiveteam-bs
11:33 πŸ”— robbierut has joined #archiveteam-bs
11:34 πŸ”— bitBaron has joined #archiveteam-bs
11:59 πŸ”— S1mpbrain has quit IRC (Read error: Connection reset by peer)
11:59 πŸ”— S1mpbrain has joined #archiveteam-bs
12:07 πŸ”— dhyan_nat has quit IRC (Ping timeout: 268 seconds)
12:35 πŸ”— S1mpbrain has quit IRC (Read error: Connection reset by peer)
12:35 πŸ”— SimpBrain has joined #archiveteam-bs
12:52 πŸ”— Flashfire has quit IRC (Ping timeout: 252 seconds)
12:52 πŸ”— kiska has quit IRC (Ping timeout: 252 seconds)
13:04 πŸ”— JAA PurpleSym: Good to know, thanks!
13:05 πŸ”— JAA Won't work on all pipelines due to the kernel version requirement (Ubuntu 16.04 LTS still has 4.4, for example), unfortunately.
13:06 πŸ”— JAA The -t option seems very nice indeed.
13:06 πŸ”— JAA I'll do some testing with that on jap-saola I think.
13:06 πŸ”— PurpleSym There’s always a catch. You don’t even need cron though. I’m running it as `tcp-closer -t 300000 -i 120 -d 443 -d 80` on my pipelines now.
13:07 πŸ”— PurpleSym That closes connections idle for 5 minutes every two minutes.
13:07 πŸ”— JAA Hmm, are there any valid connections which may be idle for 5 minutes?
13:08 πŸ”— JAA Seems a bit short to me.
13:09 πŸ”— PurpleSym I’m not sure, but I can’t think of any reason for idling 5 minutes right now.
13:12 πŸ”— PurpleSym But if you’re uncomfortable with that, we can always push it up to 30 or 60 minutes.
13:29 πŸ”— w0rmhole has joined #archiveteam-bs
13:29 πŸ”— kiska has joined #archiveteam-bs
13:30 πŸ”— Flashfire has joined #archiveteam-bs
13:58 πŸ”— bitBaron has quit IRC (My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
13:59 πŸ”— bitBaron has joined #archiveteam-bs
14:00 πŸ”— Tenebrae has quit IRC (Read error: Operation timed out)
14:01 πŸ”— Tenebrae has joined #archiveteam-bs
14:09 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
14:16 πŸ”— bitBaron has joined #archiveteam-bs
14:26 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
14:28 πŸ”— bitBaron has joined #archiveteam-bs
14:38 πŸ”— bitBaron has quit IRC (My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
14:49 πŸ”— SimpBrain has quit IRC (Read error: Operation timed out)
14:53 πŸ”— SimpBrain has joined #archiveteam-bs
15:12 πŸ”— minoa has joined #archiveteam-bs
15:13 πŸ”— dhyan_nat has joined #archiveteam-bs
15:14 πŸ”— JAA minoa: Are the SVGs stored on the same host and in the same directory (or a subdirectory) as the main page you pass to wget?
15:15 πŸ”— minoa Yes, but right now I am trying with robots turned off.
15:16 πŸ”— minoa Ah, I got it now: wget -ckm -e robots=off --user-agent="(Agent)" "https://minoa.li/" --warc-file="minoa". I never intended to block archive sites, and I thought ia_archiver did the trick … at the time.
15:17 πŸ”— JAA Ah yeah, "User-agent: * Disallow: /". :-|
15:18 πŸ”— minoa I will whitelist ArchiveBot. Sorry for any inconvenience
15:18 πŸ”— JAA ArchiveBot ignores robots.txt anyway.
15:18 πŸ”— minoa I was after the SEO and referrer spammers, not archivsts.
15:19 πŸ”— JAA And the ia_archiver block should make it visible on the Wayback Machine, I think.
15:19 πŸ”— JAA (Except for those two directories, obviously.)
15:21 πŸ”— minoa They are for internal use only. Not private but intended for other sites.
15:21 πŸ”— Exairnous has quit IRC (Read error: Operation timed out)
15:22 πŸ”— JAA (By the way, if you haven't, take a look at our wiki page on robots.txt also.)
15:24 πŸ”— Exairnous has joined #archiveteam-bs
15:25 πŸ”— minoa I saw that, but be assured I am only after the referrer spammers, not you. In fact I only learned about you today.
15:32 πŸ”— minoa So now I created a WARC dump: should it be compressed, and if so, what format?
15:35 πŸ”— Kaz gzip if poss
15:40 πŸ”— JAA I think wget should already write gzipped WARCs. Note that this is not the same as writing an uncompressed WARC and then running it through gzip. Each WARC record is compressed individually to allow random access.
15:40 πŸ”— minoa I know the WARC is gzipped, but I am referring to packing the complete archive for submission, when ready
15:41 πŸ”— wp494 has quit IRC (Read error: Operation timed out)
15:42 πŸ”— wp494 has joined #archiveteam-bs
15:42 πŸ”— JAA Ah, no, nothing else is needed.
15:46 πŸ”— minoa I am also going to backup a MediaWiki wiki: https://nsindex.net - what considerations are needed for MediaWiki sites, because if I recall correctly there some pointless pages to skip (like the login page).
15:47 πŸ”— bitBaron has joined #archiveteam-bs
15:47 πŸ”— MrRadar minoa: If the MediaWiki API is available you can use a special tool to dump the entire wiki through the API
15:47 πŸ”— MrRadar https://www.archiveteam.org/index.php?title=WikiTeam#Tools_and_source_code
15:48 πŸ”— MrRadar Though it's probably a good idea to grab it as static WARC files as well
15:48 πŸ”— SimpBrain has quit IRC (Read error: Operation timed out)
15:49 πŸ”— MrRadar In addition the ArchiveBot project has a bunch of exclusion regexes that are useful for MediaWikis: https://github.com/ArchiveTeam/ArchiveBot/blob/master/db/ignore_patterns/mediawiki.json
15:50 πŸ”— minoa I know dumpBackup.php and dumpUploads.php, but I thought you may prefer as it would appear now.
15:51 πŸ”— MrRadar Both are useful in different contexts
15:51 πŸ”— minoa It's not like NSindex is disappearing on 20 March, I have backing up high in my priority list.
15:52 πŸ”— minoa But due to personal health issues I feel a checkpoint is in order.
15:55 πŸ”— SimpBrain has joined #archiveteam-bs
15:57 πŸ”— minoa I think I am a bit too new to archiving NSindex from the Archive Team side β€” maybe I will have to submit the site to the queue while I deal with the dump scripts sort of thing.
15:58 πŸ”— minoa I do not even know if a Warrior VM will let me archive my own sites.
15:59 πŸ”— jut Warrior is for big things
15:59 πŸ”— jut #archivebot for one off
16:00 πŸ”— jut https://github.com/ArchiveTeam/grab-site for personal archiving
16:01 πŸ”— minoa I probably have to come back here on 21 March if #archivebot does not have a scheduling system.
16:04 πŸ”— minoa BTW, what does it mean by β€œwe are not the Internet Archive”? I know about robots.txt being ignored (which is not always a bad thing), but is there anything that I have missed?
16:05 πŸ”— VADemon has quit IRC (Quit: left4dead)
16:05 πŸ”— svchfoo3 has joined #archiveteam-bs
16:05 πŸ”— MrRadar We upload our stuff to the Internet Archive but we have no official connection to them
16:05 πŸ”— PurpleSym sets mode: +oo svchfoo1 svchfoo3
16:06 πŸ”— JAA Well, we are not the Internet Archive. We're a group of crazy people who throw terabytes of archives into IA each day, but everything we do is completely separate from IA's infrastructure and organisation.
16:06 πŸ”— svchfoo1 sets mode: +o joepie91
16:07 πŸ”— Stiletto minoa: it's clear archive.org wants a little distance. First off, you aren't their employees...
16:07 πŸ”— svchfoo1 sets mode: +o kiska
16:07 πŸ”— minoa I wanted to make sure I get everything right before submitting NSindex to archivebot. I don't want to make a mistake that may upset you or something.
16:09 πŸ”— JAA ArchiveBot's quite busy currently, so unless nsindex.net is in danger of disappearing soon, I'd suggest we delay that until we have more free resources.
16:10 πŸ”— JAA But you can archive it yourself with grab-site if you want.
16:10 πŸ”— JAA (Downside: it won't become available in the Wayback Machine.)
16:13 πŸ”— minoa And if I can use grab-site, how do I submit the completed project?
16:14 πŸ”— JAA You can upload it to the Internet Archive directly.
16:15 πŸ”— fuzzy8021 has quit IRC (Read error: Connection reset by peer)
16:15 πŸ”— fuzzy8021 has joined #archiveteam-bs
16:16 πŸ”— minoa I used to upload monthly data dumps there until they removed it. It was at https://archive.org/details/nsindex
16:19 πŸ”— JAA Well, in that case, you should probably talk to IA before reuploading it (in whichever format).
16:24 πŸ”— minoa Sent the email off to them.
16:24 πŸ”— minoa Thanks for the help so far.
16:24 πŸ”— minoa has left
17:26 πŸ”— Hani has quit IRC (Ping timeout: 615 seconds)
17:27 πŸ”— Stiletto has quit IRC ()
17:43 πŸ”— sebras has joined #archiveteam-bs
17:46 πŸ”— Stiletto has joined #archiveteam-bs
18:25 πŸ”— icedice has joined #archiveteam-bs
18:29 πŸ”— Hani has joined #archiveteam-bs
18:34 πŸ”— icedice has quit IRC (Quit: Leaving)
18:42 πŸ”— omarroth has joined #archiveteam-bs
19:06 πŸ”— voltagex has quit IRC (Ping timeout: 264 seconds)
21:20 πŸ”— icedice has joined #archiveteam-bs
21:23 πŸ”— dhyan_nat has quit IRC (Read error: Operation timed out)
21:59 πŸ”— BlueMax has joined #archiveteam-bs
22:11 πŸ”— omarroth has quit IRC (Ping timeout: 506 seconds)
22:11 πŸ”— omarroth has joined #archiveteam-bs
22:12 πŸ”— icedice has quit IRC (Quit: Leaving)
22:25 πŸ”— tuluu has quit IRC (Ping timeout: 615 seconds)
22:26 πŸ”— tuluu has joined #archiveteam-bs
22:31 πŸ”— bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴πŸ˜ͺZZZzzz…)
22:43 πŸ”— Hani has quit IRC (Ping timeout: 255 seconds)
22:44 πŸ”— wyatt8740 has joined #archiveteam-bs
22:50 πŸ”— arbin has quit IRC (Quit: .)
22:51 πŸ”— Hani has joined #archiveteam-bs
22:51 πŸ”— arbin has joined #archiveteam-bs
22:55 πŸ”— tuluu_ has joined #archiveteam-bs
22:59 πŸ”— tuluu has quit IRC (Ping timeout: 265 seconds)
23:04 πŸ”— ttteessst has joined #archiveteam-bs
23:04 πŸ”— icedice has joined #archiveteam-bs
23:04 πŸ”— icedice has quit IRC (Remote host closed the connection)
23:05 πŸ”— bitBaron has joined #archiveteam-bs
23:17 πŸ”— Ryz has joined #archiveteam-bs
23:36 πŸ”— Ryz JAA: So, I installed Python 3.7.2 https://www.python.org/downloads/release/python-372/ (Windows x86-64 embeddable zip file) - I tried to install snscrape but not sure how, I tried downloading Windows help file and opened it, but it didn't seem to work...what
23:38 πŸ”— Ryz So yeah I'm stuck at the moment~
23:40 πŸ”— JAA Ryz: I probably won't be able to help you with that since I haven't used Windows in many years. Stack Overflow suggests that the installation comes with pip and you should therefore be able to run 'pip install snscrape', possibly explicitly specifying a path for pip or pip.exe. But I'm sure someone else in this channel has installed Python packages on Windows before and can help you better.
23:44 πŸ”— Ryz Oh, so use the installer instead that has pip
23:59 πŸ”— robbierut has quit IRC (Ping timeout: 262 seconds)

irclogger-viewer