#warrior 2013-03-27,Wed

↑back Search

Time Nickname Message
01:00 🔗 trs80 ersi: these are the official warrior VM, converted to vmware esxi
02:29 🔗 BlueMax What's tracker rate limiting? I'm doing the Formspring project and it's constantly cycling that message
02:35 🔗 pilgrim +chronomex for BlueMax's question
08:51 🔗 GLaDOS BlueMaxim: Basically limiting how many items we hand out a minute, to prevent overloading their servers.
08:52 🔗 BlueMaxim ah, I must have been one of the unlucky ones
08:53 🔗 BlueMaxim from what I saw my Warrior process I only got like 1/2 items
08:53 🔗 BlueMaxim although I could be wrong
08:54 🔗 BlueMaxim 7 items
08:54 🔗 BlueMaxim lol
08:56 🔗 GLaDOS It's at 50 a minute
09:13 🔗 ersi How is tracker rate limiting not a descriptive enough message to understand?
09:13 🔗 ersi >_>
09:14 🔗 BlueMaxim well when it was constantly repeating to the point where I had my tracker running for a few hours and still only getting a handful of items
09:14 🔗 BlueMaxim made me think it might be something other than the obvious
09:15 🔗 GLaDOS You mean, your warrior running, right?
09:16 🔗 ersi You don't have a tracker up
09:17 🔗 ersi and I'm not just asking because you asked about it, a lot of people do. Maybe change it to "ArchiveTeam Project Tracker rate limiting"? Would that be clearer?
09:57 🔗 BlueMaxim What about adding a little bit of descriptive flair to the projects that have it enabled? Like right below the Formspring project: "This project is currently limited in order to avoid overloading their servers. Do not be alarmed if your Warrior takes some time to begin downloading files."
09:57 🔗 BlueMaxim oslt.
10:01 🔗 ersi Where to?
10:01 🔗 BlueMaxim The projects page
10:01 🔗 BlueMaxim Below the description of each project that's limited
10:02 🔗 ersi Who would check there, if it's updated after you've selected a project?
10:02 🔗 ersi don't know how much a project can handle, before we start adding pressure to it
10:03 🔗 BlueMaxim Maybe the Warrior could use some sort of news system for projects
10:03 🔗 BlueMaxim to inform about updates and such
10:06 🔗 BlueMaxim maybe as an RSS feed so that you can keep updated about Warrior shit even when you're not near your PC
10:24 🔗 Smiley you mean other than twitter?
10:25 🔗 BlueMaxim integrating the Twitter feed into the Warrior could work
10:27 🔗 GLaDOS @at_warrior
10:28 🔗 GLaDOS For "detailed" updates about the Warrior (project halt, resume, etc.)
10:28 🔗 BlueMaxim Yeah
10:42 🔗 GLaDOS Well, it'll most likely end in my murder, but here goes.
10:53 🔗 ersi I dunno, I want to do something about this because people keep asking and thinking the wrong thing with the current error message - but neither of those options sounds good
10:54 🔗 BlueMaxim it's worth a try ersi
10:55 🔗 ersi what is worth a try?
10:56 🔗 BlueMaxim the Twitter feed
11:24 🔗 sep332 just caught one of my VMs trying to run out of RAM. I just added a new swap file on the /data partition which seems to be working fine.
11:35 🔗 ersi sep332: Formspring? How many threads are you running and how much memory per wget-lua process ish? :o
11:37 🔗 sep332 posterous, it's set to 6 but only 4 are running
11:37 🔗 sep332 *5
11:37 🔗 sep332 one of them has been trying to update the project code for at least a day :p
11:38 🔗 ersi ò_ò
11:38 🔗 sep332 resident memory: 99M, 80M, 41M, 38M
11:38 🔗 sep332 i can't even see the 5th at the top of the list though.
11:39 🔗 sep332 nvm there are only 4. d'oh
11:40 🔗 sep332 Mem: 400844k total, 394828k used
11:40 🔗 sep332 swap: 676848k total, 471512k used
11:42 🔗 sep332 i just added 100MB of swap because i wasn't sure it would work. will add more when i get to work
12:08 🔗 ersi that's way too much. What else is taking memory? The Mem is counted for, what's swapping?
13:19 🔗 sep332 well that was just resident memory which now that i think of it is not so informative.
13:19 🔗 sep332 virtual: 162M, 133M, 258M, 195M
13:19 🔗 sep332 how do i tell what's swapping?
13:23 🔗 ersi basically, the wget processes are accounted for and they fit into the actual memory used - what more than that takes memory?
13:23 🔗 ersi ps aux > ps.txt and post it somewhere :)
13:23 🔗 sep332 well i just found out how to make top sort by swap used, and the warriors are at the top.
13:26 🔗 sep332 "mail: command not found"
13:27 🔗 sep332 hm, how to get files off...
13:30 🔗 Cameron_D take a look at http://sprunge.us/
13:33 🔗 ersi warriors? wget-lua you mean?
13:33 🔗 sep332 Cameron_D: sweet :)
13:33 🔗 sep332 ersi: yeah
13:35 🔗 ersi Okay, but it's kinda weird - since they seem to fit well in memory.. But your memory in the warrior is full.
13:40 🔗 sep332 true, they're only taking ~60% of my ram but i have less than 4MB free right now!
13:42 🔗 Cameron_D probably cached
13:43 🔗 Cameron_D what numbers does `free -m` return
13:44 🔗 ersi shouldn't swap/kill wget-lua processes though
13:45 🔗 sep332 free -m : total 391, used 386, free 5, shared, 0, buffers 0, cached 7.
13:46 🔗 sep332 swap: total 660, used 499, free 161
13:47 🔗 sep332 ok it took forever but sprunge worked! http://sprunge.us/FdXU
14:35 🔗 sep332 so yeah, the wget-lua-warrior proc's are using >700MB of virtual memory. they have a total of 500k URLs. using over 1 kb/URL seems unreasonably large
18:45 🔗 sep332 oh man, it was formspring this whole time. i think my brain was off this morning
20:23 🔗 Baljem hi folks - got a daft question if I may - having been running the Warrior VM for a while, is it possible to remove the data volume and create a smaller one in its place or will trying that cause problems?
20:24 🔗 Baljem unfortunately I need to free up a few gig on the host I provisioned the VM on due to a work emergency :-/
20:26 🔗 alard Baljem: Yes (but not while it's running, obviously).
20:26 🔗 alard The warrior will create a partition and format it when it boots.
20:26 🔗 Baljem brilliant, thank you!
20:26 🔗 Baljem that's a relief, I would have felt bad to have abandoned the effort ;)
20:27 🔗 alard Well, thanks!

irclogger-viewer