| Time |
Nickname |
Message |
|
22:22
🔗
|
the-gibso |
Hey guys, anyone around? |
|
22:23
🔗
|
the-gibso |
I have a couple of warrior agents running but it seems like none of them are uploading anything |
|
22:24
🔗
|
soultcer |
What makes you think that? Is there any error message? Or any other log messages? |
|
22:24
🔗
|
the-gibso |
they have all downloaded a bunch of things, but they havnt uploaded things |
|
22:24
🔗
|
omf_ |
which site are you doing the-gibso ? |
|
22:24
🔗
|
the-gibso |
well for example, one of the agents downloaded 1.5GB |
|
22:24
🔗
|
the-gibso |
but only uploaded 100 mb |
|
22:25
🔗
|
the-gibso |
this is for posterous |
|
22:25
🔗
|
the-gibso |
its just sitting there at around idle |
|
22:26
🔗
|
soultcer |
Is your warrior username also "the-gibso"? |
|
22:26
🔗
|
the-gibso |
nope madmaze |
|
22:27
🔗
|
soultcer |
Well, uploading seems to work since you have already uploaded 300 items. |
|
22:27
🔗
|
soultcer |
Can you tell me the last line in the log entry for that stuck item? |
|
22:28
🔗
|
the-gibso |
you know, I think posterour may have blocked my IP |
|
22:29
🔗
|
the-gibso |
i had 2 agents running behind 1 ip |
|
22:30
🔗
|
soultcer |
That might be the case. Unfortunately there is no message in the log if your IP has been banned |
|
22:30
🔗
|
the-gibso |
hmm bummer |
|
22:30
🔗
|
the-gibso |
ill just farm it out to other machines, thanks |
|
22:31
🔗
|
the-gibso |
also is there doc on getting the warrior source directly, without the VM? |
|
22:32
🔗
|
soultcer |
Kind of |
|
22:32
🔗
|
soultcer |
There is a git repo, https://github.com/ArchiveTeam/seesaw-kit which has the code for running the generic project pipeline |
|
22:32
🔗
|
soultcer |
And the second repo, https://github.com/ArchiveTeam/posterous-grab which has the posterous-specific grab |
|
22:33
🔗
|
the-gibso |
thanks |
|
22:33
🔗
|
soultcer |
Clone both of them, make sure you have rsync curl, python, tornado, tornadio2 installed |
|
22:33
🔗
|
soultcer |
Then go to posterous-grab, run get-wget-lua |
|
22:33
🔗
|
soultcer |
Once you have compiled wget-lua, you can use run-pipeline from seesaw-kit with the pipeline.py from posterous-grab. run-pipeline --help displays a list of options |
|
22:35
🔗
|
the-gibso |
k thanks |