[00:10] *** icedice has quit IRC (Quit: Leaving) [00:19] *** GE has quit IRC (Remote host closed the connection) [00:29] https://www.wired.com/2017/02/diehard-coders-just-saved-nasas-earth-science-data/ [00:55] *** VADemon has quit IRC (Quit: left4dead) [01:13] *** Oddy has quit IRC (Read error: Operation timed out) [01:58] arkiver (or anyone with access to the tracker): Please queue more IMDB items [02:00] Also, update on my troubles with Scaleway: I've done enough testing to determine that either Scaleway's network is corrupting outgoing packets or Cogent is. Note that all my servers are in their Amsterdam data center, I haven't tested Paris [02:00] have a look at the tracker [02:00] Ooops, need to upgrade my scripts [02:01] (I just glanced at it and though it was still the "no more items" message) [02:03] If you want to see the amazing data corruption for yourself, download http://scaleway1.2by2.info:8000/test.dat (32 MB of random garbage) and hash it with sha1sum. The correct checksum is 7a205b814a593d6d96bf5a209c2ce4db1aa41810 but I've never gotten a non-corrupted version over plain HTTP at home, work, or on my phone (over cellular data) [02:03] (^^^ regarding Scaleway/Cogent) [02:07] *** grafchokr has joined #archiveteam-bs [02:09] *** User404 has quit IRC (Read error: Operation timed out) [02:09] *** odemg has quit IRC (Remote host closed the connection) [02:10] MrRadar, a bit off topic but I banned whole IP ranges of scaleway and ovh from accessing my websites - so much wordpress trackback spam/ddos attempts [02:10] have not such a good opinion about them [02:10] *** odemg has joined #archiveteam-bs [02:10] Yeah, I only went with them because they were dirt cheap and warriors don't need much beyond reliable network... and they can't even provide that [02:11] If their support doesn't get back to me with good news I'll just cut my losses and find another VPS provider. I should only be out about $10 [02:12] support might help yeah.. anyways, sleep timebb [02:12] *** grafchoke has quit IRC (Read error: Operation timed out) [02:14] *** odemg has quit IRC (Remote host closed the connection) [02:15] *** odemg has joined #archiveteam-bs [02:18] *** odemg has quit IRC (Remote host closed the connection) [02:19] *** odemg has joined #archiveteam-bs [02:24] *** spiko has quit IRC (Leaving) [02:26] *** ndiddy has quit IRC (Read error: Connection reset by peer) [02:27] *** ndiddy has joined #archiveteam-bs [02:31] *** odemg has quit IRC (Remote host closed the connection) [02:32] *** odemg has joined #archiveteam-bs [02:35] JensRex, sucks that there's no mention there of our 150TB of stuff from ftp-gov [02:58] *** pizzaiolo has quit IRC (Remote host closed the connection) [03:02] someone should grab this: https://online.wr.usgs.gov/calendar/archives.html [03:28] godane, this doesn't look *that* big [03:29] Or am I missing something? [03:34] It's 165 lectures, they seem to be about 500MB on average, that's only 80GB or so [03:55] *** User404 has joined #archiveteam-bs [04:05] *** GE has joined #archiveteam-bs [04:18] *** odemg has quit IRC (Remote host closed the connection) [04:34] *** odemg has joined #archiveteam-bs [04:48] *** GE has quit IRC (Remote host closed the connection) [04:56] *** odemg has quit IRC (Remote host closed the connection) [04:57] *** odemg has joined #archiveteam-bs [05:22] *** Sk1d has joined #archiveteam-bs [05:45] *** Aoede has quit IRC (Ping timeout: 260 seconds) [06:28] *** Aoede has joined #archiveteam-bs [06:29] *** odemg has quit IRC (Remote host closed the connection) [06:30] I think my DNS died... Added google dns to resolv.conf and now stuff works again [06:38] *** kniffy has quit IRC (Ping timeout: 240 seconds) [06:45] *** kniffy has joined #archiveteam-bs [06:50] no wait it's just my broken ipv6 [06:59] *** Honno has joined #archiveteam-bs [07:10] *** superkuh has quit IRC (Read error: Operation timed out) [07:33] *** superkuh has joined #archiveteam-bs [07:39] *** schbirid has joined #archiveteam-bs [07:41] *** schbirid has quit IRC (Client Quit) [07:46] *** schbirid has joined #archiveteam-bs [07:50] *** Aoede has quit IRC (Ping timeout: 260 seconds) [07:51] *** bwn has quit IRC (Ping timeout: 244 seconds) [07:56] *** bwn has joined #archiveteam-bs [07:59] *** Aoede has joined #archiveteam-bs [08:07] *** icedice has joined #archiveteam-bs [08:17] *** HP has joined #archiveteam-bs [08:49] *** ravetcofx has quit IRC (Read error: Operation timed out) [09:00] *** Oddy has joined #archiveteam-bs [09:13] *** GE has joined #archiveteam-bs [09:15] *** superkuh has quit IRC (Read error: Operation timed out) [09:19] *** superkuh has joined #archiveteam-bs [09:21] *** Oddy has quit IRC (Read error: Operation timed out) [09:48] *** Aoede has quit IRC (Read error: Connection reset by peer) [09:51] *** icedice has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) [09:59] *** Aoede has joined #archiveteam-bs [10:02] Well I fixed it... No idea what I did though [10:11] *** icedice has joined #archiveteam-bs [10:18] *** GE has quit IRC (Remote host closed the connection) [10:19] *** altlabel has joined #archiveteam-bs [11:22] *** BlueMaxim has quit IRC (Quit: Leaving) [11:35] *** User405 has joined #archiveteam-bs [11:37] *** User404 has quit IRC (Read error: Connection reset by peer) [11:41] *** Honno_ has joined #archiveteam-bs [11:42] *** User405 has quit IRC (Quit: Leaving) [11:45] *** Honno has quit IRC (Ping timeout: 370 seconds) [11:45] *** GE has joined #archiveteam-bs [11:58] Someone with Wiki access, please add a note to the imdb page about limiting concurrency to 1, to avoid debilitating rate limiting or even IP bans. [12:17] *** RichardG has joined #archiveteam-bs [12:36] *** VADemon has joined #archiveteam-bs [12:38] *** Boppen has quit IRC (Ping timeout: 194 seconds) [12:38] *** Boppen has joined #archiveteam-bs [13:10] You can sign up for the wiki yourself if you want [13:10] The signup password should yahoosucks [13:10] ping JensRex [13:11] *should be [13:11] Alright. I didn't know how stingy AT is with handing out Wiki access to internet strangers :) [13:12] I think the password was changed, after someone spammed the wiki [13:13] I have created an unholy abomination in VirtualBox. [13:14] My warrior with 8GB RAM is out of memory running ftp-gov. [13:15] So I created a ram disk in Windows. Mounted it as a shared folder. Created an ext4 filesystem in a file. Mounted it as a loopback device. Created a swapfile there and enabled that. [13:15] I don't know if this will work :/ [13:26] It's not going well. [13:29] Wpull or some under the hood processes has insane RAM requirements [13:29] Indeed. ftp-gov pulled a 9,1GB file. [13:29] Something is wonky with vbox shared folders. [13:30] cat /proc/swaps [13:30] Filename Type Size Used Priority [13:30] /mnt/ffs/dongs file 9215996 2850440 -1 [13:30] Still, grind grind grind. [13:32] 45 MB/s on the mounted "RAM" drive. [13:33] Btw, what makes you prefer wpull over wget? [13:33] Just curious [13:33] (Noob here) [13:35] *** pizzaiolo has joined #archiveteam-bs [14:37] what' the channel for the Yahoo Answers grab? [14:39] #noanswers / whois [15:05] thanks VADemon [15:06] ;) [15:31] *** icedice has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) [15:34] can i archiveteam pizzahut [15:34] whois VADemon [15:34] #whois VADemon [15:34] @whois VADemon [15:35] ~whois VADemon [15:47] raiden_0x: which projects? [15:59] PDFY [15:59] PDFY2 [16:00] Whichever [16:00] Something that is ephemeral document storage to the public [16:00] Go there upload file send short link sort of thing [16:01] Scribd be damnged [16:01] *damnge [16:01] **damnged [16:01] ****Damned [16:21] raiden_0x: not sure I'd consider 'ephemeral' a feature there [16:23] More of fan letters from ACDC get off my back ) [16:24] * :) [16:44] *** ravetcofx has joined #archiveteam-bs [17:49] *** Honno_ has quit IRC (Ping timeout: 245 seconds) [18:23] *** Aranje has joined #archiveteam-bs [18:37] *** Ravenloft has quit IRC (Read error: Operation timed out) [18:55] *** odemg has joined #archiveteam-bs [18:59] JensRex: ftp-gov needs up to 20GB of RAM with current default settings (2*item_size) [19:18] *** kristian_ has joined #archiveteam-bs [19:40] Madness. [19:57] *** tapedrive has quit IRC (Ping timeout: 244 seconds) [19:57] *** whydomain has quit IRC (Ping timeout: 244 seconds) [19:59] Why again do we use wpull and not wget for ftp? [20:01] JensRex: VADemon: Usually it's ExtractRecordsInfo that causes everything to stall and eventually die if my kernel deems it necessary. [20:02] I've been wondering if it does something different compared to issuing --warc-cdx to the wpull process [20:02] Though I'm low on space so I haven't been very eager to test it [20:14] *** btfo has quit IRC (Read error: Operation timed out) [20:32] *** btfo has joined #archiveteam-bs [20:40] *** icedice has joined #archiveteam-bs [20:42] *** mls has quit IRC (Ping timeout: 250 seconds) [20:44] *** mls has joined #archiveteam-bs [21:01] *** grafchoke has joined #archiveteam-bs [21:01] *** grafchokr has quit IRC (Read error: Connection reset by peer) [21:35] *** BlueMaxim has joined #archiveteam-bs [21:55] *** schbirid has quit IRC (Remote host closed the connection) [22:02] *** kristian_ has quit IRC (Quit: Leaving) [22:09] *** grafchoke has quit IRC (Read error: Operation timed out) [22:14] *** odemg has quit IRC (Remote host closed the connection) [22:15] *** odemg has joined #archiveteam-bs [22:53] *** pizzaiolo has quit IRC (Read error: Connection reset by peer) [22:55] *** pizzaiolo has joined #archiveteam-bs [23:57] *** ravetcofx has quit IRC (Read error: Operation timed out) [23:57] *** GE has quit IRC (Remote host closed the connection)