#archiveteam-bs 2017-12-03,Sun

↑back Search

Time Nickname Message
00:06 πŸ”— ola_norsk JAA: does the mktemp write anything to disk?
00:06 πŸ”— ola_norsk JAA: https://github.com/xrgtn/nullfs
00:07 πŸ”— ola_norsk JAA: or is it simply gone whatever that's save to it?
00:08 πŸ”— JAA Well, depends on where $TMPDIR is located.
00:08 πŸ”— ola_norsk tmp?
00:08 πŸ”— JAA I don't think you can use a nullfs. It seems that wget has to save the file to disk and then read it again to extract the images etc.
00:08 πŸ”— JAA (For whatever reason...)
00:08 πŸ”— ola_norsk ok
00:08 πŸ”— JAA Yeah, /tmp is a ramdisk on my machine, but I don't know what it is on yours.
00:09 πŸ”— ola_norsk my /tmp is on SSD :/
00:09 πŸ”— CoolCanuk :o
00:09 πŸ”— ola_norsk lol
00:10 πŸ”— JAA I specifically moved /tmp to a ramfs when I migrated to SSDs to avoid the wear. :-D
00:10 πŸ”— ola_norsk aye
00:11 πŸ”— ola_norsk i have a 256mb ramdisk mounted, any way to make that vary in size as needed?
00:12 πŸ”— JAA I believe tmpfs does that.
00:12 πŸ”— JAA You specify the maximum size (default is some fraction of the physical RAM you have), and it adapts as necessary.
00:12 πŸ”— JAA It will always look like a FS having that size, but it won't occupy RAM if you don't use it.
00:13 πŸ”— ola_norsk but, with mktemp is the stuff written (and then removed) ?
00:13 πŸ”— ola_norsk to disk i mean
00:13 πŸ”— ola_norsk or does it _vanish_
00:14 πŸ”— JAA It writes to disk (or RAM or whatever file system you're using).
00:14 πŸ”— JAA You could do the same thing with mkdir and chmod, I believe.
00:15 πŸ”— ola_norsk i don't mind it writing to the ramdisk, as long as it's gone the moment after
00:16 πŸ”— ola_norsk i've been a fool not having tmp and swamp in ram
00:16 πŸ”— ola_norsk swap*
00:16 πŸ”— JAA You'll have to delete it yourself.
00:19 πŸ”— ola_norsk damnit. And here IA deletes 1300 files of CoolCanuk's stuff..That's not going to fly, there must be a better way
00:19 πŸ”— CoolCanuk LOL
00:20 πŸ”— JAA Haha
00:20 πŸ”— JAA CoolCanuk: Have you figured out yet why those files disappeared?
00:20 πŸ”— CoolCanuk they say I deleted them
00:20 πŸ”— CoolCanuk I told them there was no way I accidentally deleted 1300+ files
00:22 πŸ”— wp494_ has joined #archiveteam-bs
00:25 πŸ”— wp494 has quit IRC (Ping timeout: 492 seconds)
00:34 πŸ”— ola_norsk JAA: --delete-after
00:34 πŸ”— ola_norsk This option tells Wget to delete every single file it downloads, after having done so. It is useful for pre-fetching popular pages through a proxy, e.g.:
00:35 πŸ”— ola_norsk i think that is it
00:36 πŸ”— JAA Yeah, maybe. I didn't try that when I saw that it didn't work with -O /dev/null. But maybe it does the processing before deleting the file.
00:37 πŸ”— JAA Still use a tmpfs or whatever though to avoid the useless writes to disk.
00:37 πŸ”— ola_norsk aye
00:46 πŸ”— ola_norsk Lastet ned: 54 filer, 1,7M pΓ₯ 2,0s (875 KB/s) downloaded .. that was just on the web.archive.org/save/ request
00:47 πŸ”— ola_norsk so no doubt it's working somewhat, not a single file came from elsewhere
00:47 πŸ”— ola_norsk and the folder is 'clean as a whistle' after with --delete-after
00:49 πŸ”— JAA Sweet!
00:49 πŸ”— ola_norsk wget --delete-after --page-requisites -e robots=off 'https://web.archive.org/save/https://twitter.com/hashtag/bogus?f=tweets'
00:49 πŸ”— ola_norsk gold stuff, and thanks for the help :D
00:53 πŸ”— JAA Anytime
00:54 πŸ”— ola_norsk though...is this how waybackmachine eventually turns to captcha? :d
00:54 πŸ”— ola_norsk i hope not hehe
00:55 πŸ”— JAA They won't.
00:56 πŸ”— ola_norsk would it make a difference on their end if i limited the download rate?
00:56 πŸ”— ola_norsk or would it just cause shit to take longer
00:57 πŸ”— JAA That won't make any difference.
00:57 πŸ”— ola_norsk k
00:57 πŸ”— JAA (I think.)
00:59 πŸ”— ola_norsk i wouldn't mind having it run as multiple slow-as-hell threads on my end. I'm deleteing it, after all
01:02 πŸ”— ola_norsk the requests is what makes it save the stuff
01:03 πŸ”— JAA Amazon.fr is discontinuing unlimited cloud storage as well. I guess .co.uk is the last one offering it now? I now accept bets for how long it will take until they announce their changes...
01:04 πŸ”— ola_norsk the internet is over
01:04 πŸ”— ola_norsk (as we know it)
01:06 πŸ”— ola_norsk JAA: wasnt it you who posted this https://youtu.be/1VD_pJOFnZ0 :d
01:07 πŸ”— JAA Yeah, when you posted the vid.me link. I didn't actually watch it.
01:07 πŸ”— ola_norsk aye :d
01:08 πŸ”— ola_norsk the internet is d0000000med :d
01:09 πŸ”— ZexaronS has quit IRC (Read error: Connection reset by peer)
01:10 πŸ”— ola_norsk JAA: it's good stuff though ;)
01:11 πŸ”— JAA Yeah, I should watch it probably.
01:11 πŸ”— ola_norsk aye. "Technological Normalcy"...
01:12 πŸ”— ola_norsk JAA: i'm getting to sauced to be in chats. Have a good one, and thanks so much again for the help! SkΓ₯l!
01:13 πŸ”— ola_norsk has quit IRC (ΓΈl ΓΈl og meira ΓΈl)
01:14 πŸ”— ZexaronS has joined #archiveteam-bs
01:25 πŸ”— pizzaiolo has quit IRC (Remote host closed the connection)
01:34 πŸ”— wp494_ is now known as wp494
01:53 πŸ”— superkuh has quit IRC (Quit: the neuronal action potential is an electrical manipulation of reversible abrupt phase changes in the lipid bilaye)
02:05 πŸ”— ola_norsk has joined #archiveteam-bs
02:08 πŸ”— ola_norsk JAA: that wget is capturing images like a mofo!! https://web.archive.org/web/20171203012756/https:/twitter.com/hashtag/bogus?f=tweets
02:12 πŸ”— ola_norsk i feel sorry the one having to run pngquant on the shit :d
02:14 πŸ”— ola_norsk for*
02:18 πŸ”— ola_norsk has quit IRC (Leaving)
02:38 πŸ”— odemg JAA, did we archivebot http://forums.ncix.com/ after there bankruptcy announcement? It wont be up much longer.
02:45 πŸ”— ola_norsk has joined #archiveteam-bs
02:46 πŸ”— ola_norsk JAA: would it be naughty to set --read-timeout intentionally low you think? to prevent even downloading any stuff at the requests?
02:47 πŸ”— ola_norsk basically it would cause IA to go 'saved it! here you go!..and then 'nah, too late!' :d
02:50 πŸ”— ola_norsk the request could run quicker then, i THINK, but save outgoing bandwith for IA..
02:52 πŸ”— ola_norsk i'm hoping the latter is more beneficial
02:57 πŸ”— CoolCanuk a read timeout has the potential to not reach archive.org at all, which is risky. Your request might time out before the request is made.
02:58 πŸ”— ola_norsk hmm
02:58 πŸ”— ola_norsk that makes sense
03:00 πŸ”— ola_norsk i feel it's kind of rotten of me to first request shit, and the moment i get it it's deleted :d
03:00 πŸ”— ola_norsk thats all
03:02 πŸ”— ola_norsk i feel a little like i'm setting fire to stuff that's handed to me. Not sure how else to word it.
03:11 πŸ”— Mateon1 has quit IRC (Read error: Operation timed out)
03:11 πŸ”— Mateon1 has joined #archiveteam-bs
03:14 πŸ”— ola_norsk the reason i thought it might be an idea is it being an option in 'Downloading' https://www.gnu.org/software/wget/manual/wget.html#Download-Options
03:16 πŸ”— ola_norsk CoolCanuk: could it apply to 'connection timing' ?
03:19 πŸ”— ola_norsk JAA, CoolCanuk: damnit. I suck as this shit. But doesn't this wget stuff warrant a wiki page? :/
03:23 πŸ”— ola_norsk damn saturdays! It's the hardest day to be drunk at! Illuminati, canunks, ramdisks and damn GET REQUESTS... https://youtu.be/Fm-y9UsJ2L4
03:24 πŸ”— ola_norsk CoolCanuk: you know tragically hip right?
03:25 πŸ”— ola_norsk has quit IRC (sudo killall hexchat)
03:34 πŸ”— CoolCanuk yes it needs a wiki page
03:35 πŸ”— CoolCanuk if you can actually get it to work, pls start one
03:35 πŸ”— CoolCanuk yes i know tragically hip
03:58 πŸ”— CoolCanuk could someone make sure I'm doing this right? https://github.com/ArchiveTeam/NewsGrabber-Services/pulls newsgrabber IRC is more dead than ask.com
04:21 πŸ”— CoolCanuk hopefully it's right. Because im going HAM on it.. and want to do it right the first time
04:28 πŸ”— qw3rty116 has joined #archiveteam-bs
04:32 πŸ”— qw3rty115 has quit IRC (Read error: Operation timed out)
05:05 πŸ”— Zalgo has joined #archiveteam-bs
05:05 πŸ”— CoolCanuk welcome :)
05:08 πŸ”— Zalgo :p
05:10 πŸ”— CoolCanuk if anyone knows of news sites, and doesn't feel comfortable editing github, just pm me it and ill add it to a pull request for #newsgrabber I have a fairly automated process now.
05:12 πŸ”— Zalgo i believe i read something about dnainfo shutting down if we didnt archive that one already
05:13 πŸ”— CoolCanuk dnateam appears to be providing us with an archive, but I haven't heard much about it recently http://archiveteam.org/index.php?title=DNAinfo
05:14 πŸ”— Zalgo oki
05:15 πŸ”— CoolCanuk omg. dnainfo's 404 page is hilarious :P https://www.dnainfo.com/new-york/blah
05:16 πŸ”— Zalgo lol
05:27 πŸ”— Zalgo has quit IRC ()
05:31 πŸ”— Stilett0 has quit IRC (Ping timeout: 250 seconds)
05:33 πŸ”— CoolCanuk notice anything about this article? http://www.elliotlakestandard.ca/2017/12/02/elliot-lake-santa-claus-parade
05:34 πŸ”— godane i'm uploaing the GDC Keynote Nintendo 1999 Tape
05:34 πŸ”— godane *uploading
05:34 πŸ”— CoolCanuk OMG
05:34 πŸ”— CoolCanuk hurry
05:34 πŸ”— CoolCanuk jk jk
05:35 πŸ”— zalgo has joined #archiveteam-bs
05:35 πŸ”— CoolCanuk I mean, it's on YouTube, but your version is better
05:35 πŸ”— godane its also on youtube
05:36 πŸ”— zalgo i archived the whole of markipliers channel to test out a new 2TB drive
05:36 πŸ”— zalgo kek
05:36 πŸ”— CoolCanuk excellent
05:36 πŸ”— CoolCanuk what brand of drive? :o
05:36 πŸ”— zalgo seagate
05:36 πŸ”— godane the opening will make's it longer then that one
05:36 πŸ”— CoolCanuk nice
05:37 πŸ”— zalgo although i bet if the videos were just a little bit more compressed they could fit on a 1tb
05:37 πŸ”— zalgo but quality is more important ;/
05:37 πŸ”— CoolCanuk webm it ;p
05:37 πŸ”— CoolCanuk https://www.seagate.com/ca/en/consumer/backup/duet-amazon-drive/ is pretty cool. Too bad my upload speed sucks
05:39 πŸ”— zalgo id love to archive some twitch channels but that would take an absolutely absurd amount of storage, more than i have
05:46 πŸ”— Stilett0 has joined #archiveteam-bs
05:46 πŸ”— zalgo has quit IRC (Remote host closed the connection)
05:51 πŸ”— zalgo has joined #archiveteam-bs
05:58 πŸ”— godane http://www.dtic.mil is giving me 'connection has time out ' error
05:58 πŸ”— godane so i will have to wait to upload more dtic docs
05:59 πŸ”— CoolCanuk name resolution failed for me
05:59 πŸ”— CoolCanuk you broke it, dane..
06:00 πŸ”— CoolCanuk http://apacs.dtic.mil/ doesnt work either
06:02 πŸ”— CoolCanuk godane: do ANY mil sites work for you?
06:03 πŸ”— CoolCanuk army.mil , navy.mil don't work for me
06:03 πŸ”— CoolCanuk https://health.mil/ does
06:04 πŸ”— godane https://github.com/esonderegger/dotmil-domains/blob/master/dotmil-domains.csv
06:04 πŸ”— godane af.mil is not working either
06:04 πŸ”— CoolCanuk oh.. I had to use www.army.mil
06:05 πŸ”— CoolCanuk www.af.mil works
06:05 πŸ”— CoolCanuk what a terrible set up. they should redirect..
06:10 πŸ”— CoolCanuk you def broke it dane :P :P
06:15 πŸ”— zalgo ive reach a new level of stupidity, archiving /r/DataHoarder on reddit
06:15 πŸ”— zalgo kek
06:16 πŸ”— CoolCanuk godane: my friend is getting the default apache page. weird
06:19 πŸ”— CoolCanuk nameservers dont respond. no A record found either
06:28 πŸ”— CoolCanuk heh godane look what I found https://archive.fart.website/bin/irclogger_log/archiveteam-bs?date=2017-01-21,Sat&raw=on
06:32 πŸ”— zalgo im gonna be going to sleep soon, any ideas for an overnight backup or no
06:34 πŸ”— CoolCanuk hm
06:34 πŸ”— CoolCanuk dunno
06:40 πŸ”— zalgo alright, cya tomorrow, hopefully we can start on vidme then
06:41 πŸ”— CoolCanuk hopefully :D
08:54 πŸ”— schbirid has joined #archiveteam-bs
09:10 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
09:28 πŸ”— CoolCanuk has quit IRC (Quit: Connection closed for inactivity)
09:30 πŸ”— dashcloud has joined #archiveteam-bs
10:10 πŸ”— jschwart has joined #archiveteam-bs
10:55 πŸ”— pizzaiolo has joined #archiveteam-bs
11:08 πŸ”— Jusque_ has joined #archiveteam-bs
11:12 πŸ”— Jusque has quit IRC (Read error: Operation timed out)
11:12 πŸ”— Jusque_ is now known as Jusque
11:19 πŸ”— BlueMaxim has quit IRC (Read error: Connection reset by peer)
11:28 πŸ”— Jusque has quit IRC (Ping timeout: 260 seconds)
11:29 πŸ”— Jusque has joined #archiveteam-bs
11:42 πŸ”— JAA odemg: Doesn't look like we did.
11:52 πŸ”— zalgo has quit IRC (Read error: Operation timed out)
13:18 πŸ”— ZexaronS has quit IRC (Read error: Connection reset by peer)
13:19 πŸ”— ZexaronS has joined #archiveteam-bs
13:44 πŸ”— alfie has left Textual IRC Client: www.textualapp.com
14:43 πŸ”— superkuh has joined #archiveteam-bs
14:45 πŸ”— nottom has joined #archiveteam-bs
15:02 πŸ”— alex___ has joined #archiveteam-bs
15:32 πŸ”— jspiros_ well, shit
15:32 πŸ”— jspiros_ is now known as jspiros
15:33 πŸ”— jspiros I had forgotten about A Prairie Home Companion until a moment ago when I was reading some old post that referenced it, so I go to see about archiving old episodes of it
15:34 πŸ”— jspiros turns out just a few days ago there was yet another misconduct scandal that led to the distribution contract being severed
15:34 πŸ”— jspiros so almost the entire back history of the show is no longer available from the source
15:34 πŸ”— jspiros hopefully in a few months the rightsholders will find a new way to distribute them...
15:39 πŸ”— dashcloud has quit IRC (Remote host closed the connection)
15:52 πŸ”— SketchCow Yes
15:55 πŸ”— CoolCanuk has joined #archiveteam-bs
15:59 πŸ”— jspiros completely unrelated BS: I recently bought a legitimate copy of a TV show from the distributor, and received DVD-Rs (burned with the correct content) in the package
16:00 πŸ”— jspiros turns out the distributor felt it was wiser for them to bring disc "production" in-house
16:01 πŸ”— jspiros nice violation of the DVD-Video trademark (which was on the packaging) terms
16:01 πŸ”— * jspiros returned it and found an older properly-pressed copy on eBay
16:28 πŸ”— TheLovina has quit IRC (Read error: Connection reset by peer)
16:41 πŸ”— zalgo has joined #archiveteam-bs
16:51 πŸ”— dashcloud has joined #archiveteam-bs
16:54 πŸ”— CoolCanuk interesting
17:31 πŸ”— schbirid has quit IRC (Read error: Operation timed out)
17:35 πŸ”— CoolCanuk anyone alive :P ?
17:37 πŸ”— tklk has joined #archiveteam-bs
17:47 πŸ”— alex___ has left
18:10 πŸ”— Ing3b0rg has quit IRC (Ping timeout: 260 seconds)
18:29 πŸ”— ZexaronS has quit IRC (Read error: Connection reset by peer)
18:29 πŸ”— Ing3b0rg has joined #archiveteam-bs
18:29 πŸ”— ZexaronS has joined #archiveteam-bs
18:37 πŸ”— schbirid has joined #archiveteam-bs
18:57 πŸ”— zalgo has quit IRC (Remote host closed the connection)
18:58 πŸ”— ZexaronS has quit IRC (Read error: Connection reset by peer)
18:59 πŸ”— ZexaronS has joined #archiveteam-bs
19:13 πŸ”— ZexaronS has quit IRC (Read error: Connection reset by peer)
19:15 πŸ”— ZexaronS has joined #archiveteam-bs
19:15 πŸ”— Pixi has quit IRC (Quit: Pixi)
19:48 πŸ”— Pixi has joined #archiveteam-bs
20:02 πŸ”— zalgo has joined #archiveteam-bs
20:23 πŸ”— Lord_Nigh has quit IRC (Read error: Operation timed out)
20:23 πŸ”— Lord_Nigh has joined #archiveteam-bs
20:27 πŸ”— Jusque has quit IRC (ZNC - http://znc.in)
20:28 πŸ”— Jusque has joined #archiveteam-bs
20:48 πŸ”— nottom has quit IRC (Quit: Page closed)
20:50 πŸ”— dashcloud hi there
20:52 πŸ”— dashcloud CoolCanuk: I know you wanted to be able to do item uploads efficiently through the commandline- here's a good way to do that, complete with a way to pre-fill the metadata using a csv: https://github.com/kngenie/ias3upload
20:52 πŸ”— CoolCanuk omg thank you !! :O
20:52 πŸ”— CoolCanuk so much
20:53 πŸ”— CoolCanuk I am so happy now :)
20:53 πŸ”— zalgo heya
20:54 πŸ”— dashcloud actually- here's a more recent one, in case that doesn't work: https://github.com/vmbrasseur/iaupload with a metadata example here: https://github.com/vmbrasseur/iaupload/blob/master/md.yaml.example
21:02 πŸ”— zalgo has quit IRC (Read error: Operation timed out)
21:07 πŸ”— zalgo has joined #archiveteam-bs
21:37 πŸ”— schbirid has quit IRC (Quit: Leaving)
21:41 πŸ”— Odd0002 has quit IRC (Quit: ZNC - http://znc.in)
22:02 πŸ”— Odd0002 has joined #archiveteam-bs
22:14 πŸ”— BlueMaxim has joined #archiveteam-bs
22:18 πŸ”— CoolCanuk Breach of the day: PayPal Says 1.6 Million Customer Details Stolen in Breach at Canadian Subsidiary
22:43 πŸ”— fie has quit IRC (Ping timeout: 246 seconds)
22:51 πŸ”— Frogging that's nice
22:52 πŸ”— Frogging "A review of TIO’s network has identified a potential compromise of personally identifiable information for approximately 1.6 million customers. The PayPal platform is not impacted in any way, as the TIO systems are completely separate from the PayPal network, and PayPal’s customers’ data remains secure."
22:52 πŸ”— Frogging >a potential compromise of personally identifiable information for approximately 1.6 million customers
22:52 πŸ”— Frogging >PayPal’s customers’ data remains secure.
22:52 πŸ”— Frogging so whose customer data was compromised
22:53 πŸ”— SN4T14_ has joined #archiveteam-bs
22:53 πŸ”— Polylith_ has joined #archiveteam-bs
22:55 πŸ”— Frogging okay, it was TIO customers' data. I've never heard of that company. Hopefully it's not one of those shadow-behemoths that everyone is involved with and nobody knows about
22:56 πŸ”— fie has joined #archiveteam-bs
22:57 πŸ”— ppsym has joined #archiveteam-bs
22:59 πŸ”— i0npulse has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— purplebot has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— Polylith has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— dboard2 has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— medowar has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— SN4T14 has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— Rai-chan has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— PurpleSym has quit IRC (Ping timeout: 248 seconds)
22:59 πŸ”— ppsym is now known as PurpleSym
23:00 πŸ”— medowar has joined #archiveteam-bs
23:00 πŸ”— purplebot has joined #archiveteam-bs
23:00 πŸ”— i0npulse has joined #archiveteam-bs
23:02 πŸ”— Rai-chan has joined #archiveteam-bs
23:02 πŸ”— ndiddy has quit IRC (Read error: Operation timed out)
23:06 πŸ”— dboard2 has joined #archiveteam-bs
23:14 πŸ”— ZexaronS has quit IRC (Read error: Connection reset by peer)
23:15 πŸ”— ZexaronS has joined #archiveteam-bs
23:35 πŸ”— joepie91 lol: https://investor.paypal-corp.com/releasedetail.cfm?releaseid=1048334
23:35 πŸ”— joepie91 "PayPal Holdings, Inc. (Nasdaq: PYPL) announced that TIO Networks (TIO), a publicly traded company PayPal acquired in July 2017, has suspended operations to protect TIO's customers. This suspension of services is a result of PayPal's discovery of security vulnerabilities on the TIO platform and issues with TIO's data security program that do not adhere to PayPal's information security standards. TIO is not integrated into PayPal's platform. "
23:41 πŸ”— joepie91 TIO is a leading multi-channel bill payment processor in North America and processed more than $7 billion USD in consumer bill payments in fiscal 2016. TIO serves 16 million consumer bill pay accounts* and offers convenient solutions for expedited bill payment services to financially underserved consumers.
23:42 πŸ”— joepie91 sounds like one of those companies that handles pay-your-bills-at-the-supermarket type bills, Frogging
23:42 πŸ”— joepie91 "TIO integrates with the back office of billing systems to accept, validate, and collect payments via self-service kiosk, retail walk-in, mobile, and web solutions."
23:42 πŸ”— joepie91 yep
23:43 πŸ”— joepie91 separate company, acquired by paypal in july of this year, kept operating independently, vulnerabilities found, a month later a likely compromise was found - going to guess that it predates paypal's acquisition and somebody didn't do their due diligence during acquisition talks
23:49 πŸ”— Stilett0 has quit IRC ()
23:58 πŸ”— CoolCanuk fml

irclogger-viewer