| Time |
Nickname |
Message |
|
00:30
๐
|
hneio |
@ERROR: max connections (200) reached -- try again later |
|
00:30
๐
|
hneio |
Process RsyncUpload returned exit code 5 for Item postcrackxi.posterous.com |
|
00:30
๐
|
hneio |
rsync error: error starting client-server protocol (code 5) at main.c(1524) [sender=3.0.7] |
|
00:30
๐
|
hneio |
Oh noes |
|
03:22
๐
|
GLaDOS |
So the warrior is contained in https://github.com/ArchiveTeam/seesaw-kit |
|
03:23
๐
|
GLaDOS |
You can run |
|
03:23
๐
|
GLaDOS |
sudo pip install -e "git+https://github.com/ArchiveTeam/seesaw-kit.git@master#egg=seesaw" |
|
03:23
๐
|
GLaDOS |
to install the commands run-warrior and run-pipeline |
|
03:24
๐
|
GLaDOS |
Make the directories ~/data and ~/projects |
|
03:24
๐
|
GLaDOS |
Then run run-warrior --projects-dir ~/projects --data-dir ~/data --warrior-hq http://warriorhq.archiveteam.org --port 8001 |
|
03:24
๐
|
GLaDOS |
(note: best to use a screen session) |
|
03:25
๐
|
GLaDOS |
Note: in some cases, you will have to go into a project directory, and execute ./get-wget-lua.sh |
|
03:27
๐
|
PovAddict |
yea after a quick look at the repos in github I got a vague idea of how it works: each project has its own scripts so warrior is not exactly a self-contained program :) |
|
03:29
๐
|
Cameron_D |
also if you look in the pipeline.py of any project is lists the paths that it looks at to find wget-lua |
|
15:53
๐
|
tcv |
Hmmmmรขยยฆ After about 24 hours I am still attempting to upload a slug of formspring data. I think I get a little further along each time (due to rsync, natch), but รขยยฆ I never progress beyond 0% for each try. The errors I am seeing: "io timeout after 300 seconds -- exiting" and "rsync error: timeout in data send/receive (code 30) at io.cรขยยฆ" Also, it looks like max connections was reduced to 100? |
|
15:53
๐
|
tcv |
Should I just it be or do something else? |
|
15:55
๐
|
Smiley |
tcv: let it be really |
|
15:56
๐
|
Smiley |
the server is slowly recovering |
|
15:56
๐
|
tcv |
Smiley: Thanks. No worries unless you guys tell me I need to have worries. |
|
15:56
๐
|
Smiley |
it will honestly sort it's self eventually |
|
15:56
๐
|
Smiley |
Just unfortunate that the resync takes so long. It should hopefully be faster than before, once it's finished syncing. |
|
16:00
๐
|
tcv |
you're referring to the resync associated with the hard drive failure? |
|
16:02
๐
|
SketchCow |
http://teamarchive-1.us.archive.org:8088/mrtg/ |
|
16:02
๐
|
SketchCow |
The drive sync is over. The machine is hammered. |
|
16:02
๐
|
SketchCow |
I may turn rsync off for 24 hours just to let it catch its breath |
|
16:02
๐
|
tcv |
Woooooow. |
|
16:03
๐
|
Smiley |
wow. |
|
16:04
๐
|
tcv |
My thought is: If none of the projects are so time-sensitive right now, it might be a good idea to decline rsync for awhile. Warriors can still down URLs. |
|
16:04
๐
|
tcv |
*download |
|
16:07
๐
|
SketchCow |
Posterous is time-sensitive |
|
16:10
๐
|
SketchCow |
I've knocked it to 10 |
|
16:10
๐
|
tcv |
I guess technically, they're ALL time-sensitive. So, it would be precious time lost. But if the machine is literally not capable of accepting much as it is, the question becomes: Would disabling rsync for a certain number of hours allow for more collection once it comes back up vs. letting the server limp along? |
|
16:11
๐
|
tcv |
"knocked it to 10." What's that mean? Max conn.? |
|
16:13
๐
|
omf_ |
I was going to ask alard to setup another tracker for a project.. maybe do it using a different server |
|
17:11
๐
|
ersi |
tcv: I assume it's a reference to Spinal Tap's mockumentary where someone says that one of the amps "goes to eleven" (instead of the normal 0-10). "It's one louder". http://en.wikipedia.org/wiki/Up_to_eleven |
|
17:12
๐
|
tcv |
ersi: Oh. Here I am over-thinking it. :-P |
|
18:01
๐
|
alard |
omf_: The tracker server is fine (if you still need it). |
|
18:02
๐
|
omf_ |
yes please |
|
18:02
๐
|
alard |
Maybe we can do that later? I'll be back soon. |
|
18:03
๐
|
omf_ |
sure |
|
18:03
๐
|
omf_ |
Here is the domain list: http://paste.archivingyoursh.it/yahifivide.avrasm |
|
18:06
๐
|
soultcer |
Do we have a backup server for FOS? |
|
18:10
๐
|
omf_ |
Yeah it is the hammer SketchCow uses to beat it back into being up. :) |
|
18:10
๐
|
omf_ |
and/or any people around him at the time |
|
18:15
๐
|
ersi |
omf_: that's not an answer :P |
|
18:15
๐
|
ersi |
soultcer: not that i know of, excluding GLaDOS(?) xTB (8?) system |
|
18:18
๐
|
omf_ |
ersi, On a serious note. FOS is part of some virtual environment which was how the bad drive was able to be rebuilt |
|
18:24
๐
|
ersi |
Of course, it's a VM ontop of a petabox (Uh.. it could be rebuildable anyways, there's such things as RAID and backups) - but you answered "what's FOS?" rather than "do we have a spare server to upload to?" |
|
18:47
๐
|
alard |
I did a little experiment: Ubuntu packages for Wget+Lua, https://launchpad.net/~archiveteam/+archive/wget-lua |
|
18:50
๐
|
ersi |
alard: neat! |
|
18:52
๐
|
alard |
ersi: Want access to the archiveteam team on Launchpad? (It's probably useful to share that a bit.) |
|
18:59
๐
|
ersi |
alard: Sure |
|
19:17
๐
|
GLaDOS |
01[13seesaw-kit01] 15alard tagged 060.0.14 at 06development: 02 https://github.com/ArchiveTeam/seesaw-kit/commits/0.0.14 |