#archiveteam-bs 2017-06-22,Thu

↑back Search

Time Nickname Message
00:02 🔗 joepie91 https://twitter.com/joepie91/status/877676538019409926 -- "Gathering weak npm credentials -- how poor passwords and auth mechanisms put 52% of the Node.js ecosystem at risk https://github.com/ChALkeR/notes/blob/master/Gathering-weak-npm-credentials.md"
00:02 🔗 joepie91 (since we don't have a titlebot here)
00:04 🔗 Frogging amazing
00:04 🔗 Frogging :p
00:04 🔗 MrRadar From the community that brought you left-pad...
00:05 🔗 joepie91 MrRadar: nope, from the developer community
00:05 🔗 Frogging yes, them
00:05 🔗 Frogging :p
00:05 🔗 joepie91 MrRadar: I'm almost certain that you could run these numbers for basically every package registry that doesn't either outsource auth or centralize packaging, and end up with the same result
00:05 🔗 joepie91 because the other registries *also* don't seem to have any measures against this
00:06 🔗 joepie91 if it weren't for github being reasonably okay at finding leaked credentials and preventing bruteforcing etc., you could probably do this on github too
00:06 🔗 joepie91 but nobody seems to have run the numbers for any of the other registries yet :)
00:06 🔗 MrRadar Yeah, I was just taking a cheap shot :P
00:06 🔗 joepie91 actually, I think somebody might have done this for CPAN at some point
00:06 🔗 joepie91 but I'm not sure
00:10 🔗 BlueMaxim has joined #archiveteam-bs
00:14 🔗 antomatic has joined #archiveteam-bs
00:14 🔗 swebb sets mode: +o antomatic
00:24 🔗 joepie91 MrRadar: anyway, be wary with the cheap shots like that, because it can very easily be interpreted seriously by people, and lead to people going "oh it's just Node being dumb" (ie. missing the point and leaving all the non-Node shit insecure)
00:24 🔗 joepie91 I regularly speak to people who would interpret your statement totally seriously :)
00:26 🔗 MrRadar Yeah, understood
00:28 🔗 lucysun thank you - i was poking around there
00:32 🔗 dan- has quit IRC (Ping timeout: 260 seconds)
00:37 🔗 crusher2 arkiver: i paused all the other warriors, and the imzy ones are just happily exchanging http requests
00:37 🔗 crusher2 i can't see why the script isn't doing anything
00:53 🔗 crusher2 ah ha...
00:54 🔗 crusher2 any idea why i'd be getting: "I give up... process wgetdownload returned exit code -6
00:54 🔗 luckcolor has joined #archiveteam-bs
01:15 🔗 joepie91 FYI: imgur has started serving up HTML + JS pages for direct image links
01:15 🔗 joepie91 which for some reason requires JS and loads React, to show a fucking image
01:16 🔗 joepie91 this means that any kind of referer-less archiving is very likely to fail if imgur does not recognize your client as a headless client
01:16 🔗 joepie91 (also, what the fuck imgur?)
01:27 🔗 alembic it's my opinion that theres an emerging evil-pattern to purposely have sites crap the bed w/o javascript to force you to enable it so they can get their analytics
01:27 🔗 dan- has joined #archiveteam-bs
01:28 🔗 alembic w/o javascript, shopify will literally load the content of the website (as in, you can read the website in view-sources://) but the page renders as a blank white page
01:28 🔗 alembic someone should write a plugin to decrapify this bullshit
01:31 🔗 ndiddy joepie91: : direct links work for me
01:31 🔗 ndiddy ex http://i.imgur.com/Nj0znc8.png
01:32 🔗 MrRadar Same here, though I was seeing some very strange redirects when I was doing a bunch of imgur archiving last eek
01:32 🔗 MrRadar It was redirecting from i.imgur.com/whatever.png to URLs along the lines of i.imgur.com/original/w/h/a/whatever ... which redirected to themselves
01:34 🔗 ndiddy are you sure they were direct image links
01:34 🔗 ndiddy i just downloaded that image using wget and it worked fine
01:34 🔗 ndiddy http://i.imgur.com/aR6mdR1.png
01:34 🔗 MrRadar Yes, or at least they should have been
01:34 🔗 MrRadar And it wasn't consistent for me
01:34 🔗 MrRadar Sometimes the direct links would work as expected, sometimes it would redirect
01:35 🔗 MrRadar It only affected < 1 % of the grab, so I just re-ran the ones that got the redirect
01:38 🔗 ndiddy so do you know a better free imagehost
01:39 🔗 ndiddy most of the other ones i know of delete images after so long
01:45 🔗 alembic honestly, they all turn to crap in time...
01:46 🔗 alembic seems like the best thing atm is to use S3/B2 or spin up a VPS
01:46 🔗 alembic and I get why those are not perfect replacements
01:50 🔗 bitBaron has quit IRC (Quit: My computer has gone to sleep. ZZZzzz…)
01:51 🔗 joepie91 ndiddy: some people are reporting HTML pages, others are reporting direct links still work
01:51 🔗 joepie91 ndiddy: but imgur is detecting headless clients and referers, so you can't reproduce with wget
01:51 🔗 DFJustin yeah I started noticing it last week I think
01:51 🔗 joepie91 they're definitely throwing up HTML+JS for *some* clients though
01:51 🔗 DFJustin even right click, save as on an image from firefox produces an html file sometimes
01:51 🔗 ndiddy joepie91: just set your referer to some random site
01:52 🔗 joepie91 ndiddy: overriding referer globally will break a bunch of stuff :)
01:52 🔗 joepie91 and that doesn't change that creators of archival tools need to be aware of this
02:03 🔗 Frogging there is no good free imagehost. it's a hopeless venture and they all fall eventually
02:03 🔗 Frogging I've long since switched to self-hosting images that I share
02:04 🔗 Frogging all free image hosts to date have either shut down, become unusable, or are in the process of doing either of those
02:10 🔗 crusher2 https://i.imgflip.com/1rasb4.jpg
02:26 🔗 joepie91 ha
02:26 🔗 joepie91 where's that from? :P
02:33 🔗 Crusher_ I was slightly bored and had a few minutes
02:36 🔗 joepie91 nice
02:37 🔗 thuban2 has joined #archiveteam-bs
02:55 🔗 Crusher has joined #archiveteam-bs
02:55 🔗 Crusher_ has quit IRC (Read error: Connection reset by peer)
02:55 🔗 Somebody2 Has anyone else noticed that twitter recently blocked access by non-logged-in users to the /with_replies pane; e.g. https://twitter.com/textfiles/with_replies
02:56 🔗 Somebody2 That's quite a big change, and I was surprised to not be able to find *any* mention of it anywhere.
02:56 🔗 Crusher It does?
02:57 🔗 Crusher I clicked on it an... Oh wait, I'm logged in...
02:58 🔗 Somebody2 kisspunch: regarding #internetarchive.bak -- probably good to discuss your idea of a Windows client in there, too.
03:09 🔗 dashcloud Somebody2: I thought I saw that today, but I wasn't sure, so I checked, and it's definitely happening now
03:11 🔗 crusher2 i've just realized it's faster for me to install ubuntu using the network installer than by disk...
03:12 🔗 crusher2 is now known as Crusher2
03:34 🔗 th1x has quit IRC (Ping timeout: 633 seconds)
03:36 🔗 th1x has joined #archiveteam-bs
04:24 🔗 Crusher2 has quit IRC (Ping timeout: 268 seconds)
04:43 🔗 Sk1d has quit IRC (Ping timeout: 250 seconds)
04:50 🔗 Sk1d has joined #archiveteam-bs
07:04 🔗 schbirid has joined #archiveteam-bs
07:27 🔗 Jonison has joined #archiveteam-bs
07:59 🔗 SHODAN_UI has joined #archiveteam-bs
09:08 🔗 j08nY has joined #archiveteam-bs
11:41 🔗 lucysun has quit IRC (Quit: Page closed)
12:03 🔗 icedice has joined #archiveteam-bs
12:05 🔗 BlueMaxim has quit IRC (Read error: Operation timed out)
12:07 🔗 BlueMaxim has joined #archiveteam-bs
12:11 🔗 BlueMaxim has quit IRC (Client Quit)
12:38 🔗 Crusher has quit IRC (Ping timeout: 492 seconds)
12:42 🔗 timmc arkiver: weffey had to kill API access because the DB was getting overloaded
12:47 🔗 user0815 has joined #archiveteam-bs
12:57 🔗 qw3rty has joined #archiveteam-bs
13:09 🔗 Hiccup has joined #archiveteam-bs
13:15 🔗 pizzaiolo has joined #archiveteam-bs
13:31 🔗 Hiccup Hi. How would I go about adding downloads to a WARC file? I used https://webrecorder.io to record the non-download stuff, but there doesn't seem to be a way to add downloads.
13:35 🔗 SilSte has quit IRC (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.)
13:40 🔗 SilSte has joined #archiveteam-bs
13:44 🔗 timmc arkiver: Maybe try halving the rate on Imyz?
13:46 🔗 timmc The current rate is hundreds of concurrent sessions and the DB connection is getting overloaded.
14:03 🔗 Odd0002 has joined #archiveteam-bs
14:05 🔗 Hiccup has left
14:06 🔗 icedice has quit IRC (Read error: Connection reset by peer)
14:08 🔗 MrRadar Kaz: ^^^
14:08 🔗 MrRadar (In case you're around)
14:22 🔗 thuban3 has joined #archiveteam-bs
14:25 🔗 thuban2 has quit IRC (Read error: Operation timed out)
14:51 🔗 qw3rty has left
14:55 🔗 Kaz Hello
14:56 🔗 Kaz tracker limit 200 -> 50
15:22 🔗 RichardG has quit IRC (Read error: Connection reset by peer)
15:22 🔗 qw3rty has joined #archiveteam-bs
15:23 🔗 RichardG has joined #archiveteam-bs
15:28 🔗 JAA Ten days and 1.6 million URLs later, my Tilt grab is finally done. :-)
15:32 🔗 JAA The website grab, that is. I'll set up the API grab later.
15:42 🔗 Crusher has joined #archiveteam-bs
16:05 🔗 Odd0002 has quit IRC (Remote host closed the connection)
16:17 🔗 kristian_ has joined #archiveteam-bs
16:23 🔗 SketchCo1 is now known as SketchCow
16:24 🔗 Crusher Is that who I think it is?
16:25 🔗 xmc depends on who you think who they are
16:25 🔗 Crusher *Jason Scott's name echoes through the irc*
16:30 🔗 Crusher Ahem.. silliness aside, that is him if I'm not mistaken, right?
16:31 🔗 MrRadar Yes
16:32 🔗 Frogging mhmm
16:33 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
16:36 🔗 SketchCow ?
16:40 🔗 Crusher I must thank you for your excellent talks, I found them online and that's what inspired me to help in whatever way I can
16:40 🔗 Crusher So thank you.
16:45 🔗 jrwr SketchCow: is my hero
16:45 🔗 jrwr I hope to get to see one of his talks, or even better meet him at a vendor booth or something :)
16:46 🔗 nyany has quit IRC (Ping timeout: 506 seconds)
16:47 🔗 jrwr I love talking to bombastic people, its great fun. I am working on my old BBS Archives, most of them are Japanese
16:47 🔗 joepie91 lol: https://twitter.com/mike_hasarms/status/877779717935333377
16:49 🔗 greycat has joined #archiveteam-bs
16:51 🔗 greycat Hi there! I wanted to ask if anyone from Archive Team knows of Kaitai Struct project and if it's possible to discuss potential collaboration between our projects?
16:51 🔗 greycat We've launched http://formats.kaitai.io/ recently, which is a site for formal specifications of various file formats
16:52 🔗 greycat I guess it should be of interest to archival teams, as it provides a machine-readable, non-ambiguous specs which can be visualized as a diagram,
16:53 🔗 greycat or interpreted with a given file to highlight file structure as a tree,
16:53 🔗 kristian_ has quit IRC (Read error: Operation timed out)
16:53 🔗 greycat or compiled into a ready-made parser for that file format in many programming languages
16:54 🔗 xmc hm, neato
16:55 🔗 greycat I thought of whether we can interlink our projects i.e. http://fileformats.archiveteam.org/wiki/ICO <=> http://formats.kaitai.io/ico/
16:57 🔗 xmc i'm 100% in favor of this
16:58 🔗 pizzaiolo has joined #archiveteam-bs
16:58 🔗 greycat I thought of adding `meta/xref` key into Kaitai Struct format
16:59 🔗 greycat that could be used to add links to other projects into any .ksy file
17:00 🔗 greycat i.e. ico.ksy would include something like
17:00 🔗 greycat meta:
17:00 🔗 greycat xref:
17:00 🔗 greycat archiveteam: http://fileformats.archiveteam.org/wiki/ICO
17:00 🔗 greycat would "archiveteam" be OK name to reference archiveteam wiki with, or would you prefer something else?
17:02 🔗 DFJustin it's not really the archive team wiki it's just hosted on the same domain
17:02 🔗 brayden has quit IRC (Read error: Connection reset by peer)
17:02 🔗 brayden has joined #archiveteam-bs
17:02 🔗 swebb sets mode: +o brayden
17:04 🔗 xmc yeah, it's a separate wiki
17:04 🔗 greycat DFJustin: Ok, then how should I address it and who can I contact about it?
17:04 🔗 SHODAN_UI has quit IRC (Remote host closed the connection)
17:04 🔗 xmc using the url is probably the best way to talk about it
17:06 🔗 bitBaron has joined #archiveteam-bs
17:06 🔗 greycat for example, I wonder if it would be acceptable to modify FormatInfo template there to add a link to .ksy spec at formats.kaitai.io, if it exists?
17:06 🔗 greycat It already includes other interlinks, such as `pronom={{PRONOM|fmt/978}}`
17:06 🔗 xmc i approve
17:07 🔗 greycat Shall we wait for some quorum / more opinions, or could I try to add it right away?
17:08 🔗 xmc i wouldn't expect to see dissent about this. go ahead and add it if you feel comfortable :)
17:11 🔗 kristian_ has joined #archiveteam-bs
17:18 🔗 joepie91 greycat: ohai, somebody pointed me at kaitai a while ago, quite an interesting project :)
17:18 🔗 joepie91 greycat: one remark that's not really archiveteam-related: the format documentation is not very googleable
17:19 🔗 greycat Ok, here we go: http://fileformats.archiveteam.org/wiki/ICO
17:19 🔗 joepie91 googling for eg. datatypes (u1 and whatnot) yielded no useful results
17:19 🔗 greycat What do you think about such a link?
17:19 🔗 greycat joepie91: What could be done to improve that?
17:20 🔗 greycat joepie91: Techincally, user guide fully describes that - http://doc.kaitai.io/user_guide.html#_basic_data_types
17:20 🔗 joepie91 greycat: hmm. I think I'd call it "Kaitai Spec" instead of "Formal Spec" -- a lot of people will probably interpret "Formal" as "Official" (even if that's not what it means)
17:20 🔗 joepie91 greycat: well, I'm not sure why it's not googleable in the first place
17:20 🔗 joepie91 haven't looked into it too deeply
17:20 🔗 joepie91 just registered it in the back of my mind :)
17:21 🔗 joepie91 greycat: and yeah, I'm aware that the documentation exists, it just wasn't easy to find :p
17:22 🔗 greycat oh bummer...
17:22 🔗 greycat I wonder how many people encounter these problems...
17:22 🔗 joepie91 I really quite like the project though
17:22 🔗 joepie91 considering giving it a shot in terms of specifying the OpenTTD wire protocol
17:23 🔗 joepie91 I have a hand-written implementation in JS right now, but it'd be nicer to have a proper declarative spec
17:23 🔗 greycat about "Kaitai Spec" - technically, binary parsing project is called "Kaitai Struct", "Kaitai" is the name of team
17:23 🔗 joepie91 right, "Kaitai Struct Spec" then :p
17:23 🔗 greycat we have other projects beside "Kaitai Struct" ;)
17:24 🔗 greycat "Kaitai Struct Spec" would be probably too long and it will stretch the box... Would be it alright? May be "Declarative Spec" then?
17:24 🔗 joepie91 greycat: kaitai.io yields kaitai struct though :)
17:24 🔗 greycat Or it will be misleading as well?
17:24 🔗 joepie91 I wouldn't worry about stretching the box too much
17:24 🔗 joepie91 and you can leave out the .ksy if you call it 'kaitai struct'
17:25 🔗 joepie91 main goal is to have accurate information on the site, aesthetics comes second :P
17:26 🔗 greycat Ok, then I'll make it "Kaitai Struct Spec"
17:27 🔗 joepie91 greycat: also, fwiw, archiveteam is very much structured as "if you want to get something done, do it" so I wouldn't worry too much about doing something that somebody doesn't like - if they don't like it, they'll tell you and/or fix it, but there's no formal structure on who decides what
17:27 🔗 greycat http://fileformats.archiveteam.org/wiki/ICO <= ok, here we go
17:28 🔗 greycat well, at the very least, "Kaitai Struct" is a very new project compared to massive amount of data already gathered by archiveteam
17:28 🔗 greycat so I thought it would be polite to ask :)
17:28 🔗 joepie91 looks good :)
17:28 🔗 joepie91 and yeah, no worries
17:31 🔗 greycat ok, next step: link from formats.kaitai.io to Just Solve Format Problem wiki %)
17:31 🔗 greycat how shall I name it? I've already got it that just "archiveteam" is a bad idea...
17:32 🔗 arkiver I saw imzy was at 50 items/min
17:32 🔗 arkiver not sure if I set that
17:32 🔗 arkiver did someone set it to that here?
17:32 🔗 joepie91 greycat: "File Format wiki" is a fairly commonly used shorthand description, I think
17:32 🔗 joepie91 arkiver: thoughts on a short name for the fileformats wiki?
17:32 🔗 MrRadar arkiver: Kaz reduced it down from 200 earlier
17:32 🔗 arkiver why?
17:33 🔗 MrRadar They said we were overloading their DB server at 200
17:33 🔗 arkiver joepie91: ffwiki ? :P what do we need it for?
17:33 🔗 MrRadar 08:44 < timmc> arkiver: Maybe try halving the rate on Imyz?
17:33 🔗 MrRadar 08:46 < timmc> The current rate is hundreds of concurrent sessions and the DB connection is getting overloaded.
17:33 🔗 arkiver I can skip just a few more URLs, but not much more
17:33 🔗 arkiver for imzy
17:33 🔗 arkiver I'll do that
17:33 🔗 greycat joepie91: "File Format wiki" it's *very* ambiguous :(
17:34 🔗 Crusher arkiver : any idea why an imzy worker would spit a wget -6 error?
17:34 🔗 arkiver no
17:34 🔗 arkiver do you have a log?
17:34 🔗 xmc greycat: "just solve the problem" "justsolve" ?
17:34 🔗 greycat technically, it's called "Just Solve the File Format Problem"...
17:34 🔗 Crusher Not on me, what's the best way to get one from a worker VM?
17:35 🔗 greycat may be "jstffp"? or "justsolve" might be just fine and non-ambiguous...
17:35 🔗 joepie91 arkiver: a reference from another file format spec site to the wiki
17:35 🔗 Crusher wget -6 is an authentication problem btw
17:35 🔗 joepie91 arkiver: per format
17:38 🔗 joepie91 greycat: idk if it's ambiguous, are there any other file format wikis? :P
17:41 🔗 xmc i feel like there probably are ...
17:44 🔗 Lord_Nigh SketchCow: is there an "archival standard" for ripping vhs tapes?
17:52 🔗 bitBaron has quit IRC (Read error: Connection reset by peer)
17:52 🔗 Crusher arkiver : what's the best way to get a log out of the VM?
17:53 🔗 Crusher I remember what the error said
17:53 🔗 bitBaron has joined #archiveteam-bs
17:55 🔗 kristian_ has quit IRC (Read error: Operation timed out)
18:02 🔗 greycat joepie91: There is http://fileformats.wikia.com, http://wiki.osdev.org/, http://www.fileformat.info/format/, etc, etc
18:03 🔗 greycat joepie91: http://wiki.xentax.com/ is file format wiki mostly on game-related file formats
18:03 🔗 greycat There are many of them
18:08 🔗 SketchCow Archival Standard is either MPEG2, or a lossless AVI if you're an idiot
18:09 🔗 Crusher_ has joined #archiveteam-bs
18:09 🔗 Crusher has quit IRC (Bye)
18:29 🔗 pikhq There's better codecs, but MPEG-2 is "good enough" and you can be pretty sure it'll still be supported in future decades.
18:29 🔗 pikhq Makes sense.
18:33 🔗 Crusher_ Hmm. Something made my processor decide to clock down to 190 MHz, and I have no idea why...
18:33 🔗 Crusher_ >_>
18:34 🔗 Crusher_ Hold that thought, I just got a bluescreen printing line by line
18:35 🔗 JAA What kind of a machine do you have? I don't think I even *could* clock down my processor to 190 MHz if I tried.
18:35 🔗 MrRadar ♫ Tonight we're gonna party like it's 1999! ♫
18:36 🔗 pikhq Slightly old mobile chips can do that, IIRC.
18:37 🔗 Crusher_ It's a Pentium G3258
18:37 🔗 Crusher_ It *was* going happily at 4.2 GHz
18:38 🔗 JAA What's "slightly old", pikhq?
18:38 🔗 Crusher_ LOL I think the Intel fan just HCF'd
18:39 🔗 Crusher_ It posted, so I probably didn't bake the poor chip
18:39 🔗 MrRadar Chips these days are pretty good at thermal throttling before they release the magic smoke
18:40 🔗 MrRadar IIRC the last mainstream desktop CPU that didn't have thermal protection was the AthlonXP
18:40 🔗 MrRadar As seen in this classic video: https://www.youtube.com/watch?v=YYQSHXNFvUk
18:40 🔗 pikhq JAA: Haswell, apparently. :)
18:41 🔗 SketchCow I know everyone was hanging on the edge of their seats on this one, but
18:41 🔗 SketchCow https://archive.org/details/sega_sms_library?&sort=publicdate&page=1
18:42 🔗 Crusher_ Well, I just found out Haswell will do whatever it takes to not die.
18:42 🔗 SketchCow I have fixed all the screenshots so they're actually screenshots, and I'm now changing the right items to the Japanese model and re-rendering screenshots because not only does the SMS not boot, sometimes it WILL boot in the wrong region and show a default game.
18:42 🔗 Crusher_ Turns out a molex got tangled in the blade
18:43 🔗 pikhq Crusher_: Seems a good feature for a CPU to have.
18:43 🔗 SketchCow Ot
18:43 🔗 SketchCow It's only down to the Gs with the re-render, so there's a pile of the default game shots, now being redone.
18:44 🔗 Crusher_ pikhq : but then Intel doesn't get to charge me for a new one :P
18:45 🔗 SHODAN_UI has joined #archiveteam-bs
18:52 🔗 Crusher_ Well, it's back up to 4.2 GHz with no complaints, though I don't even want to think about what that frying electronics smell was
18:53 🔗 Crusher_ 42°C idle, Good I didn't bake the thermal paste out of existence
18:58 🔗 timmc that was the smell of the "inject random errors" fuse melting open
18:59 🔗 Crusher_ Lol
19:01 🔗 Crusher_ I'm surprised how long it operated without the fan
19:01 🔗 Crusher_ It was down to convection cooling, there's no case fan circulating where the CPU is
19:02 🔗 Crusher_ Such is the downside of squeezing a full ATX motherboard in a cardboard box
19:04 🔗 Crusher_ http://imgur.com/a/DWjPu
19:06 🔗 timmc this answers the question of the molex flopping around randomly near the fan
19:07 🔗 JAA :-|
19:07 🔗 timmc 1 star - shoebox contained GPU, not shoes
19:08 🔗 timmc This is like the grown-up version of my home server, which is a raspberry pi in a cardboard box.
19:08 🔗 Crusher_ Lol. My pi-webserver is actually sitting about a foot away
19:09 🔗 nyany has joined #archiveteam-bs
19:09 🔗 Crusher_ It was acting as a webserver for pushing files to people and for grade 11 compsci
19:10 🔗 Crusher_ It was like the 90s again, a whole class of people with no stylistic taste making really cringy HTML pages
19:10 🔗 timmc takes me back
19:11 🔗 JAA timmc: Still better than that shoebox from Reddit though. 2/5 from my side.
19:11 🔗 timmc Link?
19:11 🔗 th1x has quit IRC (Quit: Leaving)
19:11 🔗 JAA I'd rather not.
19:11 🔗 Kalroth haha
19:12 🔗 timmc oh
19:12 🔗 timmc *that* one
19:12 🔗 timmc no link thank you
19:12 🔗 JAA Yep, that one :-D
19:12 🔗 Crusher_ When it's closed, it has an additional fan taking in air to the 570 through that hole in the front
19:12 🔗 Kalroth yeah, no link needed, that image is burned in
19:12 🔗 Crusher_ It's not pretty, but it works*
19:12 🔗 Crusher_ *so long as the molex doesn't flop into the fan
19:13 🔗 timmc One of these days I'm going to take apart an old laptop and nail all the parts to a plywood board while keeping them connected. Maybe I can even make it look nice.
19:13 🔗 Crusher_ Lol
19:14 🔗 timmc vivisection, but for Acers
19:14 🔗 Crusher_ In case anyone was worried about stability, this is NOT the machine I'm using for scraping :P
19:14 🔗 Kalroth i had an bitcoin (well scrypt coin) mining setup with a motherboard and psu in the open and 4 GPUs hanging from a grill-like setup .. :)
19:14 🔗 timmc "no, this is the one that hosts example.com"
19:15 🔗 Kalroth "that one? yeah, thats just 8.8.8.8"
19:17 🔗 JAA "The cloud."
19:22 🔗 Lord_Nigh SketchCow: i meant more time base correction and what sort of vcr should be used to play/digitize it
19:30 🔗 Crusher has joined #archiveteam-bs
19:30 🔗 Crusher_ has quit IRC (Read error: Connection reset by peer)
19:32 🔗 Crusher_ has joined #archiveteam-bs
19:32 🔗 Crusher has quit IRC (Read error: Connection reset by peer)
19:32 🔗 Crusher has joined #archiveteam-bs
19:32 🔗 Crusher_ has quit IRC (Read error: Connection reset by peer)
19:40 🔗 JAA has quit IRC (leaving)
19:41 🔗 JAA has joined #archiveteam-bs
20:26 🔗 greenie hey sorry I havent been around, but wanted to remind folks that Imzy will be going entirely offline tonight, probably around midnight EST. Not sure if folks here got stuff they wanted to off the site.
20:28 🔗 Crusher We're only a third of the way
20:29 🔗 arkiver I'm updating the scripts to skip more URLs, hopefully we'll go faster then
20:29 🔗 Crusher Is there a non warrior script for imzy?
20:30 🔗 Crusher Seeing as the warrior hates me
20:30 🔗 MrRadar You just run the script directly
20:31 🔗 MrRadar There are instructions in the github repository's readme
20:31 🔗 Crusher It's on git? Ok.
20:31 🔗 MrRadar https://github.com/archiveteam/imzy-grab
20:38 🔗 arkiver greenie: for me posts are returning 404
20:38 🔗 arkiver or well, 301 and then redirected to the not found page
20:39 🔗 arkiver timmc ^
20:40 🔗 greenie hm yeah I'm not getting loads to happen on web (too lazy to open my console to see whats up). I'm no longer staff but will ping them and see whats up. There might not be much they can do at this point, I'm not sure.
20:41 🔗 Crusher I'm getting an error during get-wget-lua of Lua not found
20:41 🔗 Crusher I installed Lua 5.1 from repo but it's still getting stuck on it
20:45 🔗 JAA Crusher: liblua5.1-dev
20:46 🔗 Crusher Thanks, installing
20:48 🔗 kisspunch this is basically in the same boat as pop-unders (that is, load the real page in the popup, switch the main page to an add) in terms of filth, except that I'm less sure it's genuinely on purpose
20:48 🔗 kisspunch (re: JS needed to load)
20:58 🔗 Crusher #$&$ Now it won't even tell me why it can't compile
21:00 🔗 C4K3_ has quit IRC (Read error: Operation timed out)
21:00 🔗 C4K3 has joined #archiveteam-bs
21:05 🔗 Crusher Ah ha
21:05 🔗 Crusher It's failing on all
21:06 🔗 Crusher all-am*
21:07 🔗 Crusher Specifically: "cat: css.c: no such file or directory"
21:10 🔗 JAA Crusher: Based on my notes, here's how I installed it: packages needed: python3 gcc libssl-dev lua5.1 liblua5.1-dev make autoconf m4 flex
21:10 🔗 JAA python3 get-pip.py --user
21:10 🔗 JAA .local/bin/pip install --upgrade --user seesaw
21:10 🔗 JAA I edited get-wget-lua.sh to force it to use OpenSSL.
21:11 🔗 JAA This was on Debian Jessie.
21:12 🔗 JAA My get-wget-lua.sh edit was CONFIGURE_SSL_OPT="--with-ssl=openssl" above the WGET_DOWNLOAD_URL line.
21:12 🔗 Crusher Kk
21:16 🔗 Crusher YESSS! Got it. Thanks JAA
21:17 🔗 JAA :-)
21:18 🔗 thuban3 has quit IRC (Read error: Operation timed out)
21:19 🔗 Crusher Hmm. Any chance we could raise the rate limiter?
21:20 🔗 Crusher If I understood the message earlier correctly, it's now or never
21:20 🔗 timmc Crusher: Last night the DB got overloaded, with hundreds of concurrent sessions.
21:20 🔗 thuban3 has joined #archiveteam-bs
21:21 🔗 timmc So... running out of time, but there's only so much headroom in the app too.
21:21 🔗 Crusher Right now it looks like nobody is getting anything through...
21:22 🔗 Crusher The tracker is getting dusty at this point xD
21:23 🔗 timmc Wait, does that mean nobody is requesting stuff right now?
21:23 🔗 timmc or that people are, but are getting errors
21:23 🔗 Crusher I'm just getting placarded with rate limiting messages
21:24 🔗 Crusher And I haven't seen the tracker update in awhile now
21:25 🔗 Crusher Their site is still functioning (quickly) if anyone's Wondering
21:27 🔗 Crusher I don't have a single thread active
21:30 🔗 timmc Maybe therate limiter is at 0. :-)
21:30 🔗 JAA arkiver: ^
21:30 🔗 arkiver hi
21:30 🔗 arkiver yes
21:30 🔗 arkiver posts were dead last time I checked
21:30 🔗 arkiver all 404
21:30 🔗 arkiver ah
21:30 🔗 arkiver not dead anymore
21:31 🔗 Crusher I was about to say
21:31 🔗 arkiver restarted
21:31 🔗 arkiver I'll make an update to the scripts soon
21:31 🔗 Crusher I see a webpage full of cat posts xD
21:31 🔗 arkiver so be prepared to upgrade
21:31 🔗 Crusher Define upgrade?
21:31 🔗 SHODAN_UI has quit IRC (Remote host closed the connection)
21:31 🔗 Crusher No wait, it's working now
21:34 🔗 Crusher_ has joined #archiveteam-bs
21:35 🔗 JAA Yeah, but he's adding more ignore patterns, I think.
21:35 🔗 arkiver yeah
21:36 🔗 Crusher has quit IRC (Read error: Connection reset by peer)
21:36 🔗 timmc What's the error rate like at this point?
21:36 🔗 Crusher has joined #archiveteam-bs
21:36 🔗 Crusher_ has quit IRC (Read error: Connection reset by peer)
21:38 🔗 arkiver http://www.bbc.com/news/technology-40326544
21:38 🔗 arkiver :)
21:39 🔗 Crusher Woot woot!
21:47 🔗 Crusher_ has joined #archiveteam-bs
21:49 🔗 JAA Nice.
21:50 🔗 Crusher has quit IRC (Ping timeout: 492 seconds)
21:58 🔗 Crusher_ has quit IRC (Bye)
21:58 🔗 Crusher has joined #archiveteam-bs
22:16 🔗 kristian_ has joined #archiveteam-bs
22:23 🔗 Sanqui Does anybody know of https://anarchivism.org/ ?
22:30 🔗 Jonison has quit IRC (Read error: Connection reset by peer)
22:36 🔗 schbirid has quit IRC (Quit: Leaving)
22:37 🔗 timmc arkiver: Do you think imzy will finish in time at this rate?
22:37 🔗 arkiver no
22:38 🔗 arkiver but I'm trying to get the rate up
22:38 🔗 timmc What do you think the rate would need to be?
22:38 🔗 arkiver hard to say
22:38 🔗 timmc (assuming you don't filter out any more URLs)
22:38 🔗 arkiver a big problem is that many items seem to fail
22:38 🔗 arkiver so those need to be requeued again
22:38 🔗 arkiver and we still have to do communities
22:38 🔗 arkiver we're now only doing posts and users
22:39 🔗 arkiver but we're getting as much as possible
22:39 🔗 arkiver and might maybe just make it
22:39 🔗 * arkiver is afk for an hour
22:39 🔗 timmc ok
22:39 🔗 Stiletto has quit IRC (Ping timeout: 260 seconds)
22:41 🔗 timmc I wonder if just focusing on posts or on users would help, since otherwise every comment and post is covered twice.
23:01 🔗 Stilett0 has joined #archiveteam-bs
23:32 🔗 Crusher I don't suppose we can just ask for their hard drives :P
23:32 🔗 Crusher has quit IRC (Bye)
23:33 🔗 SketchCow HELLO I FEEL BETTER
23:34 🔗 jrwr HELLO SketchCow
23:34 🔗 jrwr What happened?
23:36 🔗 jrwr Also SketchCow, if IA complains about 163.172.128.219 pegging the crap out of their Wayback API, its just the newsgrabber project
23:36 🔗 Crusher has joined #archiveteam-bs
23:44 🔗 MrRadar arkiver: Imzy's servers appear overwhelmed again. Most of my jobs are failing due to connection timeouts of 5xx errors
23:50 🔗 bitBaron has quit IRC (Read error: Connection reset by peer)
23:50 🔗 MrRadar And now they seem better
23:52 🔗 bitBaron has joined #archiveteam-bs

irclogger-viewer