| Time |
Nickname |
Message |
|
05:28
🔗
|
trs80 |
man, formspring is so much faster than posterour |
|
05:37
🔗
|
GLaDOS |
soultcer: how's the Amazon AMI coming along? |
|
08:50
🔗
|
Smiley |
oi ops |
|
08:50
🔗
|
Smiley |
and fix topic |
|
08:50
🔗
|
Smiley |
put ami in it |
|
08:51
🔗
|
Smiley |
1c35ad75 |
|
08:51
🔗
|
GLaDOS |
Whoops, missed the kick button. |
|
08:52
🔗
|
Smiley |
:P |
|
08:52
🔗
|
Smiley |
http://archive.org/download/archiveteam-warrior/archiveteam-warrior-v2-20121008.ova | AMI for EC2: 1c35ad75 |
|
08:52
🔗
|
Smiley |
hmmm |
|
08:52
🔗
|
soultcer |
That's not going to be very useful until they set the userdata |
|
08:52
🔗
|
Smiley |
soultcer: indeed. |
|
08:53
🔗
|
Smiley |
I have instructions on my user page on wiki |
|
08:53
🔗
|
Smiley |
need to do proper page tho |
|
09:36
🔗
|
Smiley |
GLaDOS: as your doing the whole millions of instances thing |
|
09:36
🔗
|
Smiley |
you gonna do digital ocean too? |
|
09:37
🔗
|
GLaDOS |
Nah |
|
09:37
🔗
|
GLaDOS |
Can't be bothered setting it up |
|
09:37
🔗
|
Smiley |
:D |
|
09:37
🔗
|
Smiley |
apt-get update && apt-get -y install git make python-pip libgnutls-dev liblua5.1-dev && pip install seesaw && git clone https://github.com/ArchiveTeam/yahoomessages-grab.git && cd yahoomessages-grab/ && ./get-wget-lua.sh && run-pipeline pipeline.py --disable-web-server Smiley |
|
09:37
🔗
|
Smiley |
Now, are you going to do it? |
|
09:37
🔗
|
GLaDOS |
eh, cbf |
|
09:37
🔗
|
Smiley |
:D |
|
09:37
🔗
|
Smiley |
fair enough |
|
09:37
🔗
|
* |
GLaDOS launches 50 more AMIs |
|
09:38
🔗
|
Smiley |
:D |
|
09:38
🔗
|
Smiley |
dude, too much $$$ you have. |
|
09:38
🔗
|
soultcer |
Does DIgital ocean support custom images? |
|
09:38
🔗
|
soultcer |
Joyent doesn't and it's kind of annoying |
|
09:38
🔗
|
GLaDOS |
I have about 100AUD to spend |
|
09:38
🔗
|
GLaDOS |
DO supports personal images. |
|
09:38
🔗
|
GLaDOS |
Doesn't have a database of global ones though. |
|
09:39
🔗
|
GLaDOS |
I think |
|
09:56
🔗
|
Smiley |
http://archiveteam.org/index.php?title=ArchiveTeam_Warrior#Warrior_FAQ |
|
09:56
🔗
|
Smiley |
Anymore FAQ's? |
|
09:56
🔗
|
Smiley |
I'd added the ones which appeared this weekend. |
|
13:14
🔗
|
codec |
hi |
|
13:24
🔗
|
codec |
hmm.. i think my VMs kinda crash on the formspring project.. at least the websocket stuff breaks. how can i verify it still works on tasks? |
|
13:32
🔗
|
Cameron_D |
I've noticed the same issue |
|
13:33
🔗
|
Cameron_D |
I'm running the formspring project standalone and after an hour the web interface dies so I'd guess that it is specific to that project |
|
13:34
🔗
|
sep332 |
my warrior process got OOM-killed over the weekend |
|
13:34
🔗
|
sep332 |
strange |
|
13:36
🔗
|
sep332 |
pretty sure it was running on yahoo messages. what does it even keep in RAM? |
|
13:43
🔗
|
ersi |
You're most likely not running Y! M anymore, if you were running ArchiveTeams choice - you've been moved over to Formspring project |
|
13:44
🔗
|
ersi |
Maybe the Formspring items are bigger, taking more memory per wget and OOMing. I havn't looked into it though. |
|
14:04
🔗
|
sep332 |
i have no problem bumping up the ram on the vm, but didn't expect it to need a lot |
|
14:06
🔗
|
ersi |
Don't take my word for it - I'd advise you to log on to the Warrior and check top |
|
14:06
🔗
|
ersi |
You could also run lesser threads (If it's lack of memory) |
|
14:08
🔗
|
sep332 |
ok, i'll keep an eye on them |
|
14:12
🔗
|
codec |
hmm, i'll add some ram as well |
|
15:14
🔗
|
codec |
ersi: sep332: works fine with 4g ram, btw |
|
15:15
🔗
|
Smiley |
II have like 6 gb ram on mine ¬_¬ |
|
15:15
🔗
|
Smiley |
not ideal for testing ;D |
|
15:15
🔗
|
ersi |
Default is 476MB or something like that. Just check if any wget process is taking hundreds of MB's in RAM |
|
17:50
🔗
|
tcv |
Hope someone can help... |
|
17:51
🔗
|
tcv |
I've been running Warrior all weekend. First time running it. (VirtualBox / Win7x64). Twice now I've found the web interface to be unresponsive after many hours of running the warrior. Problem is in all web browsers. Restarting the VM allows me back into the web UI, but? that's not ideal. Is there a grace way to recover and resolve the problem? |
|
18:51
🔗
|
ersi |
tcv: You're running "The ArchiveTeam's choice"-project, right? |
|
18:51
🔗
|
tcv |
Yes. |
|
18:52
🔗
|
tcv |
BTW: I've since looked at the IRC logs and found that some others were reporting this problem and they suggested increasing the VM's RAM, which I've done. |
|
18:52
🔗
|
tcv |
Using top, I found two wget processes each consuming over 100MB each. |
|
18:52
🔗
|
ersi |
that's not necessarily a solution, that's merely a theory |
|
18:52
🔗
|
ersi |
Hm |
|
18:53
🔗
|
tcv |
@esri I hear you. |
|
18:54
🔗
|
ersi |
I'll fire up an warrior instance and let it run (I mostly run the project code stand-alone) and look into the matter |
|
18:54
🔗
|
tcv |
Thanks. |
|
18:55
🔗
|
tcv |
Upping the RAM also meant I had to restart the VM. So, I've been running ? about two hours now? I'll let it run through the day and night. (Very happy to be helping to archive Item "HelloKittyPorn.") |
|
18:57
🔗
|
ersi |
haha, zing |
|
18:57
🔗
|
ersi |
Thanks for helping out and testing it out :) It's appreciated |
|
18:58
🔗
|
tcv |
I am seeing an occasional error that states?? |
|
18:59
🔗
|
tcv |
"lua runtime error: formspring.lua:311: bad argument #1 to 'ipairs' (table expected, got nil). |
|
19:00
🔗
|
tcv |
Seems rare. Perhaps tied to a particular a formspring username? Dunno. Just guessing. |
|
19:01
🔗
|
ersi |
hm~ I'll keep an eye out for it :) |
|
19:01
🔗
|
tcv |
What's the underlying web server software? I was wondering if I could restart that if I get into that UI freeze again...? |
|
19:06
🔗
|
ersi |
tcv: It's integrated with the project code - easiest way is probably just restart the warrior (though you can kill the software and start it again if you'd like) |
|
19:07
🔗
|
tcv |
ersi: Would killing the software remove all of the downloaded stuff to-date? I'm hoping for a way to recover without jettisoning all the work done so-far. |
|
19:13
🔗
|
ersi |
Yes and no, it would still be there - but not worked on more, so effectively yes :P |
|
19:14
🔗
|
ersi |
(Old data gets removed on warrior boot, if any orphan data is there)' |
|
19:20
🔗
|
tcv |
ersi: So maybe just best to recycle the VM. |
|
19:25
🔗
|
ersi |
just a reboot works fine if it's stuck :) |
|
19:25
🔗
|
ersi |
could always tail/less the wget.log and see if it's still working though |
|
19:41
🔗
|
zenpho |
howdy doo! can anyone offer advice on how to configure a warrior to connect thru a proxy? I tried (briefly) today to run the warrior whilst connected to my university internet connection - but the rsync uploading phase seemed to fail. Unfortunately I forget the exact error text but I definitely remember a connection timeout. |
|
19:44
🔗
|
zenpho |
The university proxy behaviour seems unusual to me. It rejects unless you have given your username/password in a webbrowser. I beleive requests are apparently redirected |
|
19:46
🔗
|
ersi |
We got that here in Uppsala University, Sweden |
|
19:46
🔗
|
zenpho |
...I believe HTTP requests are simply responded to with the authentication page, and other requests are rejected until you have successfully authenticated |
|
19:46
🔗
|
ersi |
but the warrior will use your regular connection. so if you authenticate with your laptop - that should work for the warrior |
|
19:46
🔗
|
ersi |
the network is 'bridged' by default |
|
19:47
🔗
|
zenpho |
Today I was trying to set the env var RSYNC_PROXY using the /etc/security/pam_env.conf |
|
19:47
🔗
|
zenpho |
yes, that's what I was thinking - perhaps I could set "bridged" mode instead of "NAT" mode - and authenticate inside the guest os |
|
19:48
🔗
|
ersi |
oh, oops. I meant NATed |
|
19:48
🔗
|
ersi |
NATed > bridged in this case |
|
19:48
🔗
|
zenpho |
or do you mean the opposite? use my host os browser to authenticate and let the guest os piggy back onto the connection? |
|
19:49
🔗
|
ersi |
yeah |
|
19:49
🔗
|
ersi |
Or did that not work? |
|
19:49
🔗
|
zenpho |
I believe HTTP connections were being made by the guest os when I tried using my host os browser to authenticate, but rsync upload was definitely failing |
|
19:51
🔗
|
ersi |
rsync's using ssh I think, maybe your university got that blocked :o |
|
19:51
🔗
|
zenpho |
I can try again tomorrow and give more details about the error messages etc when I have a bit more time... was almost hoping this was a problem someone here had seen before and there might be a (relatively) easy solution |
|
19:52
🔗
|
ersi |
I know someone has been running the warrior through TOR |
|
19:52
🔗
|
ersi |
so it's definately possible, just no write up that I know of |
|
19:52
🔗
|
zenpho |
It would not surprise me if the university proxy blocked everything but port 80 |
|
19:53
🔗
|
ersi |
me neither |
|
19:53
🔗
|
zenpho |
it's interesting you mention rsync and ssh |
|
19:55
🔗
|
zenpho |
I know I can connect to my home machine via ssh - but only if I use an unusual port number (not 22) |
|
19:56
🔗
|
ersi |
probably port blocked then :) Simplest way to block stuff |
|
19:58
🔗
|
zenpho |
its a crazy (silly) hack and I don't blame you if you say "nope - not gonna happen" but maybe a workaround for those behind paranoid-university proxies like mine would be for someone at the archive team to run an rsync server on a different port |
|
19:59
🔗
|
ersi |
sounds like having some 'proxy settings page' would be far the easiest |
|
19:59
🔗
|
zenpho |
of course, that might not be necessary - i might have just done something silly today - like I say, i'll try again tomorrow. |
|
19:59
🔗
|
ersi |
it's certainly good to know that people have these kinds of problems though |
|
20:00
🔗
|
zenpho |
thanks for your thoughts ersi, i'm sure i'll bug you or someone else here again tomorrow |
|
20:01
🔗
|
zenpho |
I'd really like to use that phat university pipe to help out! ;o) |
|
20:01
🔗
|
ersi |
no problem |
|
20:01
🔗
|
ersi |
and be gentle with shared resources like uni lines :) |
|
20:02
🔗
|
zenpho |
yes I will, I will tweak the downlaod speed down during office hours |
|
20:03
🔗
|
zenpho |
I mean both download and upload speed |
|
20:03
🔗
|
ersi |
goodie :) |
|
20:18
🔗
|
tcv |
ersi: Just another observation. I've noticed that the hang-up in the web UI happens well after 12 hours. I should be able to give you an up-time if it happens again after the RAM increase. |