#warrior 2013-10-18,Fri

↑back Search

Time Nickname Message
21:46 🔗 joepie91 how would I detect the HTTP status code that wget received?
21:47 🔗 joepie91 that is, primarily, distinguish between 404 and 503
21:48 🔗 joepie91 cc yipdw omf_ etc.
21:48 🔗 yipdw joepie91: as an exit code, you can't
21:48 🔗 yipdw well
21:48 🔗 yipdw you can differentiate between category
21:48 🔗 yipdw but if you want the status code per-response, you'll want to hook in via Lua
21:51 🔗 joepie91 'category'?
21:52 🔗 yipdw server error, client error, network error, etc
22:36 🔗 joepie91 I think I'm doing stuff right now that isn't possible... >.
22:36 🔗 joepie91 >.> *
22:36 🔗 joepie91 guess all will become clear when I test
22:38 🔗 * joepie91 implemented RangeInterpolation
23:07 🔗 joepie91 whoa.
23:07 🔗 joepie91 I think my custom downloader actually works.
23:07 🔗 joepie91 can't figure out how to stop wget from outputting to terminal
23:07 🔗 joepie91 but still
23:07 🔗 joepie91 it works!
23:08 🔗 joepie91 :P
23:08 🔗 joepie91 nvm, my fault
23:13 🔗 joepie91 AW YEAH
23:13 🔗 joepie91 http://pastie.org/private/uhpi44ls3mzd2ih0civgig
23:15 🔗 joepie91 haha
23:15 🔗 joepie91 and then the exceptions happened!
23:25 🔗 joepie91 omf_, yipdw, if either of you is awake
23:25 🔗 joepie91 I'm having trouble
23:25 🔗 joepie91 (or anyone else who speaks seesaw)
23:25 🔗 joepie91 this is my current pipeline: https://github.com/joepie91/isohunt-grab/blob/develop/pipeline.py
23:25 🔗 joepie91 downloading works fine
23:25 🔗 joepie91 as does 404 handling
23:26 🔗 joepie91 but when the item finishes, this is what happens:
23:26 🔗 joepie91 http://pastie.org/private/ikiogbva7kyu5vsfp74azg
23:26 🔗 joepie91 *many* of these errors in rapid succession
23:26 🔗 joepie91 and the stacktraces in seesaw are very very confusing so I can't figure out what's causing it :P
23:45 🔗 yipdw joepie91: why do you need that async popen fix
23:45 🔗 yipdw just wondering
23:45 🔗 yipdw I'm actually not sure what's going on in that stacktrace
23:48 🔗 joepie91 yipdw: neither am I :(
23:48 🔗 joepie91 and all the AsyncPopen stuff is copypasted from puush basically
23:48 🔗 joepie91 I mean, it *works*
23:48 🔗 joepie91 just after it's done downloading all the torrents it errors out
23:48 🔗 yipdw hm
23:48 🔗 joepie91 but I suspect I might've found at least one of the issues
23:48 🔗 joepie91 so I'm running now and waiting to see what happens
23:49 🔗 joepie91 also, we have 5 more days or so for isohunt
23:49 🔗 joepie91 at most
23:50 🔗 joepie91 I see what's going wrong, I think...
23:52 🔗 joepie91 yeah I think I found it
23:53 🔗 joepie91 I do have to say that seesaw makes debugging *really* hard :P
23:53 🔗 joepie91 it seems to completely fuck the stacktraces
23:53 🔗 yipdw a little bit
23:53 🔗 yipdw the main reason is that that the pipeline is evaled
23:53 🔗 yipdw but you can extract the line from the stacktrace, at least
23:53 🔗 joepie91 heh
23:53 🔗 joepie91 yeah, I realized that when it told me the issue was in <string>
23:53 🔗 yipdw sticking to small tasks help a lot
23:54 🔗 yipdw er, helps
23:54 🔗 joepie91 I'm running it again, the problem -should- be solved now...
23:55 🔗 joepie91 File "/usr/lib/python2.7/site-packages/seesaw/task.py", line 56, in enqueue
23:55 🔗 joepie91 Traceback (most recent call last):
23:55 🔗 joepie91 File "/usr/lib/python2.7/site-packages/seesaw/tracker.py", line 118, in process
23:55 🔗 joepie91 TypeError: 'RangeInterpolation' object is not iterable
23:55 🔗 joepie91 self.process(item)
23:55 🔗 joepie91 total_bytes[group] = sum([ os.path.getsize(realize(f, item)) for f in files])
23:55 🔗 joepie91 well hey
23:55 🔗 joepie91 it was a nice try!
23:55 🔗 joepie91 :P
23:56 🔗 joepie91 at least download succeeds now!
23:57 🔗 yipdw heh
23:57 🔗 yipdw one suggestion: eliminate all copypasta, start from a very simple pipeline
23:57 🔗 yipdw seesaw doesn't have much documentation but if you think of it as a stream processor it's pretty easy from there
23:58 🔗 yipdw or an event loop
23:58 🔗 yipdw equivalent model

irclogger-viewer