[01:00] ersi: these are the official warrior VM, converted to vmware esxi [02:29] What's tracker rate limiting? I'm doing the Formspring project and it's constantly cycling that message [02:35] +chronomex for BlueMax's question [08:51] BlueMaxim: Basically limiting how many items we hand out a minute, to prevent overloading their servers. [08:52] ah, I must have been one of the unlucky ones [08:53] from what I saw my Warrior process I only got like 1/2 items [08:53] although I could be wrong [08:54] 7 items [08:54] lol [08:56] It's at 50 a minute [09:13] How is tracker rate limiting not a descriptive enough message to understand? [09:13] >_> [09:14] well when it was constantly repeating to the point where I had my tracker running for a few hours and still only getting a handful of items [09:14] made me think it might be something other than the obvious [09:15] You mean, your warrior running, right? [09:16] You don't have a tracker up [09:17] and I'm not just asking because you asked about it, a lot of people do. Maybe change it to "ArchiveTeam Project Tracker rate limiting"? Would that be clearer? [09:57] What about adding a little bit of descriptive flair to the projects that have it enabled? Like right below the Formspring project: "This project is currently limited in order to avoid overloading their servers. Do not be alarmed if your Warrior takes some time to begin downloading files." [09:57] oslt. [10:01] Where to? [10:01] The projects page [10:01] Below the description of each project that's limited [10:02] Who would check there, if it's updated after you've selected a project? [10:02] don't know how much a project can handle, before we start adding pressure to it [10:03] Maybe the Warrior could use some sort of news system for projects [10:03] to inform about updates and such [10:06] maybe as an RSS feed so that you can keep updated about Warrior shit even when you're not near your PC [10:24] you mean other than twitter? [10:25] integrating the Twitter feed into the Warrior could work [10:27] @at_warrior [10:28] For "detailed" updates about the Warrior (project halt, resume, etc.) [10:28] Yeah [10:42] Well, it'll most likely end in my murder, but here goes. [10:53] I dunno, I want to do something about this because people keep asking and thinking the wrong thing with the current error message - but neither of those options sounds good [10:54] it's worth a try ersi [10:55] what is worth a try? [10:56] the Twitter feed [11:24] just caught one of my VMs trying to run out of RAM. I just added a new swap file on the /data partition which seems to be working fine. [11:35] sep332: Formspring? How many threads are you running and how much memory per wget-lua process ish? :o [11:37] posterous, it's set to 6 but only 4 are running [11:37] *5 [11:37] one of them has been trying to update the project code for at least a day :p [11:38] ò_ò [11:38] resident memory: 99M, 80M, 41M, 38M [11:38] i can't even see the 5th at the top of the list though. [11:39] nvm there are only 4. d'oh [11:40] Mem: 400844k total, 394828k used [11:40] swap: 676848k total, 471512k used [11:42] i just added 100MB of swap because i wasn't sure it would work. will add more when i get to work [12:08] that's way too much. What else is taking memory? The Mem is counted for, what's swapping? [13:19] well that was just resident memory which now that i think of it is not so informative. [13:19] virtual: 162M, 133M, 258M, 195M [13:19] how do i tell what's swapping? [13:23] basically, the wget processes are accounted for and they fit into the actual memory used - what more than that takes memory? [13:23] ps aux > ps.txt and post it somewhere :) [13:23] well i just found out how to make top sort by swap used, and the warriors are at the top. [13:26] "mail: command not found" [13:27] hm, how to get files off... [13:30] take a look at http://sprunge.us/ [13:33] warriors? wget-lua you mean? [13:33] Cameron_D: sweet :) [13:33] ersi: yeah [13:35] Okay, but it's kinda weird - since they seem to fit well in memory.. But your memory in the warrior is full. [13:40] true, they're only taking ~60% of my ram but i have less than 4MB free right now! [13:42] probably cached [13:43] what numbers does `free -m` return [13:44] shouldn't swap/kill wget-lua processes though [13:45] free -m : total 391, used 386, free 5, shared, 0, buffers 0, cached 7. [13:46] swap: total 660, used 499, free 161 [13:47] ok it took forever but sprunge worked! http://sprunge.us/FdXU [14:35] so yeah, the wget-lua-warrior proc's are using >700MB of virtual memory. they have a total of 500k URLs. using over 1 kb/URL seems unreasonably large [18:45] oh man, it was formspring this whole time. i think my brain was off this morning [20:23] hi folks - got a daft question if I may - having been running the Warrior VM for a while, is it possible to remove the data volume and create a smaller one in its place or will trying that cause problems? [20:24] unfortunately I need to free up a few gig on the host I provisioned the VM on due to a work emergency :-/ [20:26] Baljem: Yes (but not while it's running, obviously). [20:26] The warrior will create a partition and format it when it boots. [20:26] brilliant, thank you! [20:26] that's a relief, I would have felt bad to have abandoned the effort ;) [20:27] Well, thanks!