| Time |
Nickname |
Message |
|
21:46
🔗
|
joepie91 |
how would I detect the HTTP status code that wget received? |
|
21:47
🔗
|
joepie91 |
that is, primarily, distinguish between 404 and 503 |
|
21:48
🔗
|
joepie91 |
cc yipdw omf_ etc. |
|
21:48
🔗
|
yipdw |
joepie91: as an exit code, you can't |
|
21:48
🔗
|
yipdw |
well |
|
21:48
🔗
|
yipdw |
you can differentiate between category |
|
21:48
🔗
|
yipdw |
but if you want the status code per-response, you'll want to hook in via Lua |
|
21:51
🔗
|
joepie91 |
'category'? |
|
21:52
🔗
|
yipdw |
server error, client error, network error, etc |
|
22:36
🔗
|
joepie91 |
I think I'm doing stuff right now that isn't possible... >. |
|
22:36
🔗
|
joepie91 |
>.> * |
|
22:36
🔗
|
joepie91 |
guess all will become clear when I test |
|
22:38
🔗
|
* |
joepie91 implemented RangeInterpolation |
|
23:07
🔗
|
joepie91 |
whoa. |
|
23:07
🔗
|
joepie91 |
I think my custom downloader actually works. |
|
23:07
🔗
|
joepie91 |
can't figure out how to stop wget from outputting to terminal |
|
23:07
🔗
|
joepie91 |
but still |
|
23:07
🔗
|
joepie91 |
it works! |
|
23:08
🔗
|
joepie91 |
:P |
|
23:08
🔗
|
joepie91 |
nvm, my fault |
|
23:13
🔗
|
joepie91 |
AW YEAH |
|
23:13
🔗
|
joepie91 |
http://pastie.org/private/uhpi44ls3mzd2ih0civgig |
|
23:15
🔗
|
joepie91 |
haha |
|
23:15
🔗
|
joepie91 |
and then the exceptions happened! |
|
23:25
🔗
|
joepie91 |
omf_, yipdw, if either of you is awake |
|
23:25
🔗
|
joepie91 |
I'm having trouble |
|
23:25
🔗
|
joepie91 |
(or anyone else who speaks seesaw) |
|
23:25
🔗
|
joepie91 |
this is my current pipeline: https://github.com/joepie91/isohunt-grab/blob/develop/pipeline.py |
|
23:25
🔗
|
joepie91 |
downloading works fine |
|
23:25
🔗
|
joepie91 |
as does 404 handling |
|
23:26
🔗
|
joepie91 |
but when the item finishes, this is what happens: |
|
23:26
🔗
|
joepie91 |
http://pastie.org/private/ikiogbva7kyu5vsfp74azg |
|
23:26
🔗
|
joepie91 |
*many* of these errors in rapid succession |
|
23:26
🔗
|
joepie91 |
and the stacktraces in seesaw are very very confusing so I can't figure out what's causing it :P |
|
23:45
🔗
|
yipdw |
joepie91: why do you need that async popen fix |
|
23:45
🔗
|
yipdw |
just wondering |
|
23:45
🔗
|
yipdw |
I'm actually not sure what's going on in that stacktrace |
|
23:48
🔗
|
joepie91 |
yipdw: neither am I :( |
|
23:48
🔗
|
joepie91 |
and all the AsyncPopen stuff is copypasted from puush basically |
|
23:48
🔗
|
joepie91 |
I mean, it *works* |
|
23:48
🔗
|
joepie91 |
just after it's done downloading all the torrents it errors out |
|
23:48
🔗
|
yipdw |
hm |
|
23:48
🔗
|
joepie91 |
but I suspect I might've found at least one of the issues |
|
23:48
🔗
|
joepie91 |
so I'm running now and waiting to see what happens |
|
23:49
🔗
|
joepie91 |
also, we have 5 more days or so for isohunt |
|
23:49
🔗
|
joepie91 |
at most |
|
23:50
🔗
|
joepie91 |
I see what's going wrong, I think... |
|
23:52
🔗
|
joepie91 |
yeah I think I found it |
|
23:53
🔗
|
joepie91 |
I do have to say that seesaw makes debugging *really* hard :P |
|
23:53
🔗
|
joepie91 |
it seems to completely fuck the stacktraces |
|
23:53
🔗
|
yipdw |
a little bit |
|
23:53
🔗
|
yipdw |
the main reason is that that the pipeline is evaled |
|
23:53
🔗
|
yipdw |
but you can extract the line from the stacktrace, at least |
|
23:53
🔗
|
joepie91 |
heh |
|
23:53
🔗
|
joepie91 |
yeah, I realized that when it told me the issue was in <string> |
|
23:53
🔗
|
yipdw |
sticking to small tasks help a lot |
|
23:54
🔗
|
yipdw |
er, helps |
|
23:54
🔗
|
joepie91 |
I'm running it again, the problem -should- be solved now... |
|
23:55
🔗
|
joepie91 |
File "/usr/lib/python2.7/site-packages/seesaw/task.py", line 56, in enqueue |
|
23:55
🔗
|
joepie91 |
Traceback (most recent call last): |
|
23:55
🔗
|
joepie91 |
File "/usr/lib/python2.7/site-packages/seesaw/tracker.py", line 118, in process |
|
23:55
🔗
|
joepie91 |
TypeError: 'RangeInterpolation' object is not iterable |
|
23:55
🔗
|
joepie91 |
self.process(item) |
|
23:55
🔗
|
joepie91 |
total_bytes[group] = sum([ os.path.getsize(realize(f, item)) for f in files]) |
|
23:55
🔗
|
joepie91 |
well hey |
|
23:55
🔗
|
joepie91 |
it was a nice try! |
|
23:55
🔗
|
joepie91 |
:P |
|
23:56
🔗
|
joepie91 |
at least download succeeds now! |
|
23:57
🔗
|
yipdw |
heh |
|
23:57
🔗
|
yipdw |
one suggestion: eliminate all copypasta, start from a very simple pipeline |
|
23:57
🔗
|
yipdw |
seesaw doesn't have much documentation but if you think of it as a stream processor it's pretty easy from there |
|
23:58
🔗
|
yipdw |
or an event loop |
|
23:58
🔗
|
yipdw |
equivalent model |