[00:15] *** toad1 has joined #archiveteam-bs [00:58] *** mistym has quit IRC (Remote host closed the connection) [02:00] *** mistym has joined #archiveteam-bs [02:29] *** dashcloud has quit IRC (Read error: Operation timed out) [02:36] *** dashcloud has joined #archiveteam-bs [03:02] *** schbirid2 has joined #archiveteam-bs [03:03] *** schbirid has quit IRC (Read error: Operation timed out) [04:10] *** aaaaaaaaa has quit IRC (Leaving) [04:30] *** mistym has quit IRC (Remote host closed the connection) [04:57] *** mistym has joined #archiveteam-bs [05:28] *** Jonimus has quit IRC (Ping timeout: 370 seconds) [05:42] *** Jonimus has joined #archiveteam-bs [05:52] *** Jonimus has quit IRC (Ping timeout: 370 seconds) [06:04] *** Jonimus has joined #archiveteam-bs [06:14] i'm getting Wentworth Millitary Academy Yearbooks [06:15] i found them when trying to find old pri the world podcast episodes in google [08:02] *** vitzli has joined #archiveteam-bs [08:03] *** vitzli has quit IRC (Client Quit) [08:33] *** dashcloud has quit IRC (Read error: Operation timed out) [08:39] *** dashcloud has joined #archiveteam-bs [08:46] i'm starting to upload Westworth Military Academy Yearbooks: https://archive.org/details/Westworth_Military_Academy_Yearbook_1881 [09:00] *** mistym has quit IRC (Remote host closed the connection) [09:18] also i'm grabbing ARG Netcasts that are in wayback machine [09:18] there site is down/gone [09:19] good news is most of episodes are in wayback [09:35] nice, wpull finished with 200kb left on the harddisk [09:39] what wore you downloading [09:43] you need a bigger HDD [09:44] some forums [09:49] *** dashcloud has quit IRC (Read error: Operation timed out) [09:55] *** dashcloud has joined #archiveteam-bs [10:29] *** johtso has joined #archiveteam-bs [10:31] schbirid2, yeah, parallel definitely wont understand the spaces in the paths [10:32] not sure how to expand a glob and deal with the spaces.. [10:32] maybe I should be piping into parallel instead [10:32] hm, i never had that problem [10:32] maybe my script is bad? [10:33] i use "parallel ffmpeg -i {} -acodec libvorbis -aq 0 -y -vn {}.ogg ::: *.mp3" all the time with spaces in filenames, no issues [10:34] oh lol [10:34] johtso: file=$1 [10:34] not the other way around [10:35] never trust me again ;) [10:35] also, of course "parallel bash script.sh {} ::: *.rar" or make it executable then "parallel ./script.sh {} ::: *.rar" [10:38] schbirid2, doh! [10:38] works beautifully now [10:38] :) [10:38] now just to wait for the next back of downloads to finish and take it for a spin :) [10:39] schbirid2, quick question, what kind of server would you use for doing quick archive ops, I'm currently doing this on a digitalocean instance [10:39] free data transfer being a nice plus [10:39] no idea [10:39] i have a cheap and huge online.net one [10:40] via oneprovider.com [10:40] very small hard drives being pretty annoying.. [10:40] 20€ for 2tb hdd [10:40] ah okay [10:41] I quite liked the idea of being able to only pay for the instance while it was needed [10:56] this data archiving lark could get rather addictive :) [11:33] *** dashcloud has quit IRC (Read error: Operation timed out) [11:44] *** dashcloud has joined #archiveteam-bs [12:18] *** BlueMaxim has quit IRC (Quit: Leaving) [12:55] *** schbirid2 has quit IRC (Read error: Operation timed out) [12:58] *** schbirid2 has joined #archiveteam-bs [13:09] *** dashcloud has quit IRC (Read error: Operation timed out) [13:16] *** dashcloud has joined #archiveteam-bs [13:25] *** primus104 has joined #archiveteam-bs [14:10] [11:39] free data transfer being a nice plus [14:10] that is offered by nearly every VPS provider :) [14:10] not AWS :P [14:10] hence 'nearly' [14:10] AWS is not a very good thing to compare to [14:10] it's overpriced, unreliable, high on vendor lock-in [14:10] yeah [14:11] and the fee structure is ridiculous [14:11] I'm seeing some very cheap VPS options out there.. [14:11] johtso: http://vps-list.cryto.net/index.php?action=list (somewhat outdated) [14:11] and there's vpsboard and lowendtalk [14:12] and a few other VPS comparison things but I can't recommend those [14:12] as they use affiliate links [14:13] lol aws [14:15] this seems pretty cheap http://backupsy.com/ [14:15] johtso: and don't forget that there's kimsufi ;) [14:16] johtso: http://www.kimsufi.com/en/ [14:17] now that's cheap.. [14:17] and leaseweb: https://www.leaseweb.com/bare-metal-server/all-servers [14:18] and online.net, but I don't like their network: http://www.online.net/en/dedicated-server/dedicated-server-overview-perso [14:18] and hetzner: http://www.serverbidding.com/ [14:18] 2 x 3TB for 30eur/mo right now, it seems [14:19] https://robot.your-server.de/order/market/398741/country/NL/culture/en_GB#398741 [14:21] or 10.5TB for 57 eur/mo: https://robot.your-server.de/order/market/page/4/sortcol/hd/sorttype/down/hdsize/1000/limit/100#412236 [14:21] er, might as well get 12TB for the same price then: https://robot.your-server.de/order/market/page/4/sortcol/hd/sorttype/down/hdsize/1000/limit/100#404248 [14:21] so yeah :P [14:25] thanks for the links! [14:25] there's a lot out there.. [14:26] these seem tempting too.. especially the ssd-cached options https://www.ramnode.com/vps.php [14:31] that's a lot of data waiting to be dealt with :P https://archive.org/catalog.php?identifier=MadrotterTreasureHuntBlogDownloads [14:37] *** primus104 has quit IRC (Leaving.) [15:16] *** DopefishJ has joined #archiveteam-bs [15:19] *** pir^2 has joined #archiveteam-bs [15:19] *** serapeum has quit IRC (Read error: Connection reset by peer) [15:20] *** deathy_ has joined #archiveteam-bs [15:20] *** DFJustin has quit IRC (Ping timeout: 417 seconds) [15:20] *** torvik has quit IRC (Ping timeout: 417 seconds) [15:20] *** LittUp has quit IRC (Ping timeout: 417 seconds) [15:20] *** deathy has quit IRC (Ping timeout: 417 seconds) [15:20] *** deathy_ is now known as deathy [15:20] *** Zebranky has joined #archiveteam-bs [15:20] *** Zebranky_ has quit IRC (Read error: Connection reset by peer) [15:25] *** pir^2 has quit IRC (bye) [15:27] *** deathy has quit IRC (Ping timeout: 393 seconds) [15:27] *** johtso has quit IRC (Ping timeout: 393 seconds) [15:27] *** balrog has quit IRC (Ping timeout: 393 seconds) [15:27] *** marvinw has quit IRC (Ping timeout: 393 seconds) [15:27] *** johtso has joined #archiveteam-bs [15:27] *** deathy has joined #archiveteam-bs [15:30] *** torvik has joined #archiveteam-bs [15:38] *** serapeum has joined #archiveteam-bs [15:38] *** deathy has quit IRC (Ping timeout: 498 seconds) [15:38] *** jk[SVP] has quit IRC (Ping timeout: 498 seconds) [15:38] *** deathy has joined #archiveteam-bs [15:39] *** LittUp has joined #archiveteam-bs [15:40] *** marvinw has joined #archiveteam-bs [15:46] *** jk[SVP] has joined #archiveteam-bs [15:48] *** LittUp has quit IRC (Ping timeout: 488 seconds) [15:48] *** deathy has quit IRC (Ping timeout: 488 seconds) [15:48] *** Kazzy has quit IRC (Ping timeout: 488 seconds) [15:48] *** LittUp has joined #archiveteam-bs [15:48] *** deathy has joined #archiveteam-bs [15:48] *** aaaaaaaaa has joined #archiveteam-bs [15:49] *** Kazzy has joined #archiveteam-bs [15:51] *** balrog has joined #archiveteam-bs [15:51] *** swebb sets mode: +o balrog [16:02] *** Nertsy has quit IRC (Quit: Nertsy) [16:08] *** Nertsy has joined #archiveteam-bs [16:10] johtso: oh, still there? [16:10] (yes, ramnode is very very good, but not quite as bang-for-the-buck storage-wise as some other hosts, though certainly better than most) [16:10] joepie91_, yep :) [16:10] johtso: another suggestion! [16:10] https://servercrate.com/storage-vps [16:11] 10gbit uplink, too [16:11] wow, nice! [16:12] not quite the disk perf of ramnode, but the ramnode disk perf is hard to beat anyway :) [16:12] but certainly more than enough [16:13] 1073741824 bytes (1.1 GB) copied, 3.03315 s, 354 MB/s [16:13] (sequential) [16:30] joepie91_, a big pipe isn't very useful when I'm only getting 20Mbps upload to IA :( [16:33] *** primus104 has joined #archiveteam-bs [17:08] johtso: hehe, you can parallelize [17:08] basically, parallelize away until you get a 522 [17:09] (iirc) [17:09] joepie91_, just run multiple instances of "ia upload ident --delete" on a single directory? [17:12] johtso: no [17:12] if ia offers a parallel option, use that [17:12] otherwise, one ia process per item [17:12] oh yeah, that'll fail when it gets to a file that's deleted [17:12] oh WHAT? nice!! [17:12] (per IA item, not per file) [17:12] johtso: you may want to look into GNU parallel [17:12] it's specifically for this kind of thing [17:13] takes a list of paths [17:13] and manages X processes concurrently [17:13] giving each process a chunk of the paths [17:13] yeah, I've been playing around with gnu parallel today for validating the rars and zips [17:14] *** vitzli has joined #archiveteam-bs [17:14] oh, "if ia offers a parallel option" [17:14] missed the if :) [17:14] :P [17:14] it does for metadata [17:14] not sure if also for uploads [17:21] joepie91_, so many options, struggling to find the right ones for concurrency and job size [17:22] okay -P for job slots [17:30] *** mistym has joined #archiveteam-bs [17:33] joepie91_, parallel -j 8 -N 10 --eta ia upload MadrotterTreasureHuntBlogDownloads {} --no-derive --delete --checksum ::: ./alpha/* [17:33] awesome [17:35] *** primus104 has quit IRC (Leaving.) [17:48] 503 Server Error: Slow Down :( [17:49] i get those sometimes [17:51] https://i.imgur.com/removed.png [17:52] does anybody know why could imgur remove an image in less than 24 hours?? [17:53] uploader can delete [17:54] wonder when IA will forgive me.. [17:54] take my damn data already :P [17:54] I don't think he did, but I'll ask him [17:57] so i found some video of linux 2002 korea conference [17:58] i was briefly in kbs korea culture news for 2002-10-25 [18:27] *** Jonimus has quit IRC (Ping timeout: 370 seconds) [18:32] *** vitzli has quit IRC (Quit: Leaving) [18:37] *** primus104 has joined #archiveteam-bs [18:58] *** DopefishJ is now known as DFJustin [19:06] *** Jonimus has joined #archiveteam-bs [19:23] johtso: 503s aren't on a per-client basis [19:23] they're system-wide [19:25] *** mistym_ has joined #archiveteam-bs [19:31] *** mistym has quit IRC (Read error: Operation timed out) [19:39] joepie91_, ah, so I've just overloaded the s3 server? [19:42] wow.. still getting a 503 2 hours later :( [19:47] *** SN4T14_ has joined #archiveteam-bs [19:51] maybe once it lets you upload again, you could tar the files together so there aren't as many S3 requests being made. I think that would pl [19:51] *would probably keep you from hitting the rate limiting. [19:55] *** SN4T14__ has quit IRC (Ping timeout: 512 seconds) [20:06] johtso: it just means the infra is a bit busy [20:06] kyan: iirc the explanation was that the 503 is based on the task queue [20:06] and it's largely derives that are responsible for the task queue [20:06] Aah [20:06] so it's likely just unusually busy right now [20:06] cool [20:07] I am using --no-derive too [20:07] johtso: yeah, it's prob not your jobs :) [20:07] I'd wait a day or so and retry [20:07] or half a day [20:07] I broke everything! :) [20:07] beat the timezones [20:07] :p [20:07] anyway [20:07] good news everyone! [20:07] .t http://arstechnica.com/business/2015/03/eu-announces-plans-to-banish-geo-blocking-modernize-copyright-law/ [20:07] er [20:07] .title http://arstechnica.com/business/2015/03/eu-announces-plans-to-banish-geo-blocking-modernize-copyright-law/ [20:07] joepie91_: EU announces plans to banish geo-blocking, modernize copyright law | Ars Technica [20:08] great news! [20:08] this is good [20:09] first step europe, next step the world [20:09] first step modernizing, next step abolishment ;) [20:09] true [20:10] one comment in the comments thread that I particularly like [20:10] "Either than MEP is bought and paid for, or they might actually be dumber than some of the cleverer rocks." [20:20] *** espes__ has joined #archiveteam-bs [20:20] If it is anything like the US, it will get introduced and the amended to the point where you'd wish they'd left it alone. [20:21] Of course, Europe doesn't seem quite as bought-and-sold as the US is; but that might just be an outsider not seeing the sausage made. [20:26] Presumably this will be as successful as the 1989 'Television Without Frontiers' directive, which supposedly promised free movement and reception across europe, etc [20:26] Industry said "no" and the European Parliament spent the next 20 years issuing reports saying how successful it was [20:34] *** Start has quit IRC (Read error: Connection reset by peer) [20:35] *** Start has joined #archiveteam-bs [20:42] *** DFJustin has quit IRC (Quit: IMHOSTFU) [20:42] *** DFJustin has joined #archiveteam-bs [20:42] *** swebb sets mode: +o DFJustin [20:57] *** BlueMaxim has joined #archiveteam-bs [21:49] man, wtf irc [21:49] "user" (ie. the equivalent of 'ident' in rfc2812) is 1 to unlimited characters and can contain tab, bell, and various other nonprintables [21:49] yes, the bit after the exclamation mark [22:17] joepie91_: i got CDs for you \o/ [22:19] haha, what [22:31] schbirid2: :D [22:31] for the IRC-curious [22:31] user = 1*( %x01-09 / %x0B-0C / %x0E-1F / %x21-3F / %x41-FF ) [22:31] grab an extended ASCII table [22:31] and see exactly how insane that is [22:32] schbirid2: still have my address btw? [22:32] i remember almost BEEP getting thrown out of class by our professor BEEP when i wrote a random ascii print loop that BEEPed a lot [22:32] should have, will confirm before i send them [22:32] dont expect them too soon [22:32] but they are now at my place [22:33] gonna check which ones i want to keep for sentimental value and i will ask the kultcds.com guy first if he needs specific ones [22:33] (fun fact: some xchat builds will STILL beep when you send them a bell char - appending it to random messages in a large IRC channel is a great source of amusement) [22:34] schbirid2: no hurry :P [22:34] also, doesn't sending it to kultcds mean no images [22:36] wait, might that be the cause of the mysterious random neeps i get every once a month? [22:36] i hate those [22:36] hehe [22:36] kultcds has a limit on how recent discs are i think [22:36] older ones are online [22:36] oh dear, good job irc specs [22:36] schbirid2: computer bild spiele shows 0 images [22:36] only PDFs [22:36] as far as I can tell [22:37] dan_: best part is that this is the newer one [22:37] that supersedes the old one which DID have a limit [22:37] lol [22:37] ooh, incubation full version [22:37] such a great game [22:37] (old RFC relied on ident RFC limitations - the entire notion of ident has been abandoned in 2812) [22:37] ha, nice [22:37] oooh biing, such memories [22:37] the joys of writing an IRCd implementation... [22:38] there's a reason lots of implementors basically ignore the newer rfcs as invalid [22:38] hehe [22:38] IRC is a mess [22:38] (but it's OUR mess) [22:38] actually helping write a python ircd right now, lots of fun to go through [22:39] I tried a Python IRCd [22:39] gave up [22:39] now writing one in Node.js [22:39] going much better [22:39] screenfun 2/03 [22:39] haha, fair enough [22:39] 1/00 [22:39] err cbs [22:40] dan_: https://github.com/joepie91/circd/blob/master/ircd.py [22:40] 1/02 [22:40] 12/99 [22:41] ah shiny, https://github.com/kaniini/mammon here [22:42] (part of that ircv3 crowd here, so we're using it to prototype new implementations and specs and such, lots of fun) [22:42] ha, kaniini writing a new one? [22:42] (say hi for me) [22:42] haha yep, will do [22:43] ok, this is mostly poop [22:43] but hey [22:43] early 2000s [22:43] to mid [22:43] schbirid2: archived poop is best poop [22:43] working with about three groups that absolutely hate each other [22:43] * schbirid2 hands joepie91_ a jar [22:43] dan_: sounds like IRC then [22:43] don't you love the irc community? :) [22:43] hehe [22:43] I'm surprised it's only three [22:43] I'd expected a different order of magnitude [22:44] say, 10 warring networks times 3 warring opers on each of them [22:44] haha, when I work with more groups I'm sure it'll grow [22:44] dan_: anyway, I've been wanting to implement ircv3 [22:44] but the documentation really confuses me [22:45] it's built off 14x RFC [22:45] but it's unclear to me how one writes an implementation that supports clients targeted at 14x, 28x AND ircv3 [22:46] presumably it is possible, but I have no idea how [22:46] kind of a barrier to adoption :) [22:47] dan_: so yeah, I'd really recommend writing a short document for implementors that basically comes down to "ignore these things from these RFCs, implement these things from these RFCs, then implement these IRCv3 RFCs in this order, and then your IRCd should work with all common clients" [22:47] fair enough, the ircv3 stuff is designed to be hidden behind the CAP command, so that they're only implemented when the client requests them [22:47] yes, but the 14x root causes issues [22:47] but that does make sense, especially with the 2x rfcs [22:48] the ancestry is not linear [22:48] which makes it hard to determine what supersedes what [22:48] and it seems like a step back to me [22:48] since everything's optional with CAP, that means you're effectively forced to implement a 14x IRCd [22:48] with optional IRCv3 [22:49] and no apparent ability to do stuff according to 28x *at all* [22:49] and the lack of docs on this makes it hard for me to determine whether that impression is accurate or not [22:49] (hence just writing an 28x implementation for now and worrying about SSLv3 later, which is probably not the desirable option, but the only realistic option for an implementor right now who isn't intimately familiar with the edge cases of IRC) [22:50] I get what you mean, irc standards are a bloody mess [22:50] yeah, for a production ircd that makes sense [22:51] but I'd really like to implement IRCv3 stuff, so please write docs like I suggested above ;) [22:51] there are a few guys working on something along those lines, but I'll have a chat and see what sorta stuff in that direction we could do, since it'd be really useful [22:51] even if it's just "for experimental implementations" [22:51] alright, great :) [22:52] (they don't have to be hand-holding docs btw, as long as they give enough information to figure out my way) [22:53] (and of course I'm speaking from my personal POV here, but I suspect this would be similar for other implementors) [22:55] yeah, the guys I'm talking about were looking at basically taking RFC1x, and then updating it to actually match what production networks use these days, making a new 'descriptive' rfc that actually has everything people need to implement, etc [22:55] but I'll see what they're doing with it, try to make sure they get that perspective in there :) [22:55] right [22:56] that'd be a great thing [22:56] 28x seems like it was an effort at something similar [22:56] ... but then they cheaped out and said "fuck it, unlimited length idents" and more such fun... [22:57] and I am still amazed at how apparently nobody went "hmm, maybe bell characters in unique user identifiers are not the best of ideas" [22:57] while writing that spec [22:57] especially since bells are explicitly excluded elsewhere... [22:57] 28x was apparently "only actually implemented on IRCNet, and nowhere else" [22:57] lol [22:57] see also: safe channels, and almost everything but # channels [22:58] dan_: right now, would you recommend I implement 14x or 28x? [22:59] I'm not a big implementer, basically been a bot/services coder most of the time [22:59] from what I see, 14x seems more common as a baseline [23:00] and then going through, maybe seeing what other implementations take from 28x and implementing those [23:00] as well as ISUPPORT/etc that have come out since and aren't in any RFCs [23:01] ISUPPORT>\ [23:01] ? * [23:03] ah: https://github.com/grawity/irc-docs/blob/master/client/RPL_ISUPPORT/draft-hardy-irc-isupport-00.txt [23:03] basically takes the 005 numeric, and you get it on connect on almost every network out there [23:04] gives the client lots of useful info on channel prefixes you support, nick lengths, casemapping [23:04] ahhhhhhhh [23:04] yep [23:04] definitely using that [23:05] aha yep [23:06] there's lots of stuff you can implement, but that should give you a decent baseline ircd right now [23:06] open to further suggestions ;) [23:06] (fwiw, the IRCd I'm implementing is meant to be highly modular) [23:06] (as in, there is almost no non-plugin code) [23:07] best thing I've found is just looking at another implementation or two, I always have the chary/hybrid source trees open while I'm coding, see how they handle things if I run into troubles [23:07] mmm [23:07] those are both C, no? [23:08] yeah, based off the same codebase (hybrid), but chary is Atheme's 'newer' ircd, implements some things differently [23:08] my C/C++ code reading skills are not great :) [23:08] but any implementations that are decently stable should do fine, just picked those because they're popular and I work with them a lot :) [23:09] there's not much non-C IRCd code, unfortunately [23:10] and so far the only C/C++ codebase I've been able to read through without issues, has been the Node.js codebase [23:10] (not v8) [23:11] ah yeah, that's fair enough [23:27] ha! victory [23:28] dan_: https://gist.github.com/joepie91/ee2b8b3d19fd26b13658 [23:28] auto-generating error code \o/ [23:28] (by parsing the RFC) [23:29] ooh, nice [23:30] now I just need to give them fancy names [23:30] haha [23:31] * joepie91_ Achievement Unlocked: Generating code from RFC [23:33] what are you doing guys? [23:33] something coding ircds? [23:33] raylee: writing an IRCd! [23:33] :P [23:37] lots of fun [23:37] in what language? [23:37] raylee: coffeescript + Node.js [23:37] raylee: python3 here [23:37] heh [23:37] if i could be bothered i'd think about writing an ircd in dlang [23:39] raylee: dlang is memory-safe? [23:40] you can write it that way [23:40] or you can write it interfacing with c/c++, using pointers, etc [23:40] raylee: is it the default? :p [23:40] dlang being memory-safe is the default [23:41] okay, approved :D [23:41] well [23:42] *** c_b has joined #archiveteam-bs [23:42] there's a compiler switch at least, but if you're not using pointers then.. [23:42] * joepie91_ raises eyebrow [23:42] raylee: if a clueless newbie were to start dlang [23:42] would his code be memory-safe? [23:43] *** antomatic has quit IRC (Read error: Operation timed out) [23:43] *** Sk1d has quit IRC (Read error: Operation timed out) [23:43] *** antomatic has joined #archiveteam-bs [23:43] *** primus has quit IRC (Read error: Operation timed out) [23:44] *** Sk1d has joined #archiveteam-bs [23:44] *** fenn has quit IRC (Read error: Operation timed out) [23:44] *** primus has joined #archiveteam-bs [23:44] *** fenn has joined #archiveteam-bs [23:46] *** w0rp has quit IRC (Read error: Operation timed out) [23:46] *** w0rp has joined #archiveteam-bs [23:48] *** brayden has quit IRC (Read error: Operation timed out) [23:49] dan_: what is the difference between ERR_UNKNOWNMODE and ERR_UMODEUNKNOWNFLAG [23:49] in 1459 [23:52] ah, ERR_UMODEUNKNOWNFLAG looks like it's used when there's a nickname parameter with the MODE message [23:52] example case? [23:52] whereas ERR_UNKNOWNMODE just says it's an unknown mode character, so I guess otherwise that's used [23:53] *** serapeum has quit IRC (Ping timeout: 606 seconds) [23:55] ah, I see [23:56] when someone is trying to set modes on a user and the mode character is unknown, you use ERR_UMODEUNKNOWNFLAG [23:56] when setting modes on a channel and the character is unknown, ERR_UNKNOWNMODE [23:58] dan_: aha, so UnknownUserMode vs UnknownChannelMode would be sensible error names? [23:59] should be yeah, don't think you can set modes on anything but users/channels but I haven't scanned the rfc for that yet