[02:41] *** warmwaffl has quit IRC (Remote host closed the connection) [03:22] *** bithippo has joined #warrior [03:43] *** bithippo has quit IRC (Textual IRC Client: www.textualapp.com) [05:52] *** Atom__ has quit IRC (Read error: Connection reset by peer) [05:56] *** Atom__ has joined #warrior [13:10] *** minkafigh has joined #warrior [13:16] hello, anybody here? I have a vServer on Digitalocean and am currently archiving google+. I only download with 30-60MBit/s though, why is that? My vServer can easily do 500mbit/s down and 700mbit/s up. [13:16] How many concurrent are you running? [13:17] 6 [13:17] Each thread downloads 1 page at a time and much of that time is taken up by connecting to Google's servers [13:17] In other words it's normal. [13:18] is there a way to speed up that connecting? [13:18] are there like, datacenters that are faster than others? [13:18] in terms of connecting to google [13:18] Yes, but I'm not sure which ones are best [13:18] Additionally, with Google+ the main bottleneck right now is actually our upload capacity, not download [13:19] Why is that? [13:19] If you want to increase your throughput the best way to do it is to run with a higher concurrent [13:19] Our upload staging servers are completely full and the Internet Archive's upload service is pretty much fully maxed out [13:19] If you run higher concurrent i face a error that says that i have too many connections reached [13:20] Is that an rsync error? If so, that's normal [13:20] That's due to the aformentioned upload bottleneck [13:22] How much storage will google+ need? [13:23] We're currently at 800 TB [13:23] It's looking like it will be over 1 petabyte total [13:23] You can check the stats here: http://tracker.archiveteam.org/googleplus/ [13:24] How can Fusl upload that fast when the upload servers are full? [13:24] does he just have very many servers? [13:25] Yes, exactly. She's running tons of servers [13:25] Including several of the upload servers [13:25] *** Atom-- has joined #warrior [13:26] Do I just send my stuff to the upload servers and then those servers are uploading it to archive.org or what are they doing? [13:26] correct [13:26] Yes, that's exactly what's happening [13:26] Why have those "middle mans"? [13:27] The servers you are uploading to combine all of the smaller files you send into 50 GB "megawarcs" that are easier for the IA's infrastructure to handle [13:27] damn :O [13:28] How much time do we have till google+ is closed? [13:28] And can we reach the goal to archive everything? [13:28] with out current speed [13:28] The channel for Google+ is #googleminus . [13:28] *** Atom__ has quit IRC (Ping timeout: 252 seconds) [15:24] *** minkafigh has quit IRC (Quit: Page closed) [15:24] *** minkafigh has joined #warrior [15:24] *** minkafigh has quit IRC (Client Quit) [15:29] *** minkafigh has joined #warrior [17:00] interestingly after the last seasaw update i haven't had the virtualbox stop processing on me in its entirety...but 2 of the workers(?) within one virtualbox instance have stalled [17:03] What is the last output from those workers? [17:05] 1885=200 https://lh3.googleusercontent.com/-kMl27Cqwj6U/VxkQ-QgQL5I/AAAAAAAAOig/1-yZcxtUOwMbAc7hTSpObLvBR8RzN-XhA/w530-h636-n/2016%2B-%2B1 [17:05] and 141=200 https://plus.google.com/113183347162063862096/posts/aPn7KwAgSf8 [17:05] both of which are real posts that exist but are not that large [17:25] not a seesaw issue I don't think [17:25] I know I had wget-lua get stuck before. This was two years ago though. [17:31] Yeah, it definitely happens, but I'd expect to see more of it than just this one event. Would blame local issues first (seriously, maybe it's just slow) rather than assume seesaw/wget-lua issues [17:31] Yeah [17:34] 1 of my vm's totally froze, couldnt enter the webinterface (was running headless vm) and for some reason ate all my disk io. Now I stopped it. [17:35] Another vm keeps restarting every time, after I shut it down via the soft close web button [17:53] Kaz: it's not stuck for 2 days slow. [17:53] ctrl-c and see what happens [17:54] or in fact if it's warrior you can't even do that [17:54] Kaz: also, up until these 2 workers froze my vm freezes about once every day or two in its entirety up until yesterday - this is the first time only 2 (and not all 6) workers stopped processing [17:55] i'll try ctrl-c if the rest stop [17:56] i mean could ctrl-c on the vm main screen [20:10] *** minkafigh has quit IRC (Quit: Page closed) [20:37] *** warmwaffl has joined #warrior [21:49] *** _54674341 has joined #warrior [22:26] *** netsound has quit IRC (Read error: Connection reset by peer) [22:27] *** netsound has joined #warrior [23:25] *** VADemon has joined #warrior