#archiveteam 2013-08-01,Thu

↑back Search

Time Nickname Message
03:30 🔗 chfoo would it be possible for me to have commit access on the puush-grab repo? or should i keep using pull requests?
03:34 🔗 yipdw chfoo: done
03:36 🔗 chfoo yipdw: thanks
05:58 🔗 Guest___ is help needed with xanga?
05:58 🔗 Guest___ i totally missed the memo on this
06:11 🔗 SmileyG kennethre: yes
06:11 🔗 SmileyG tho it's a pita
06:11 🔗 SmileyG think posterous but worse.
06:12 🔗 kennethre haha, wonderful
06:12 🔗 kennethre how so?
06:17 🔗 yipdw kennethre: it seems like we get heavily throttled and banned
06:18 🔗 kennethre spinning things up now
06:18 🔗 yipdw kennethre: also, in my experience, I've not been able to contact any xanga subdomain in the EC2 US-East region
06:18 🔗 yipdw er, from that region
06:18 🔗 yipdw Northern California works, a bi
06:18 🔗 yipdw t
06:18 🔗 kennethre ugh, that sucks
06:18 🔗 kennethre i could run some in europe, but not many
06:18 🔗 yipdw so I'm not sure if they blocked all traffic from there or what
06:19 🔗 yipdw you might have better luck; US-East was acting a bit odd when I tried it
06:20 🔗 kennethre let's see...
06:24 🔗 kennethre yipdw: appears to maybe be working
06:25 🔗 yipdw kennethre: hmm, interesting
06:25 🔗 kennethre oh nvm
06:25 🔗 kennethre response code 599 wtf
06:25 🔗 yipdw kennethre: so, another thing that seems to slow down the grabbing is that the xanga-grab script, in some cases, seems to route traffic through a proxy (for cases like that)
06:25 🔗 yipdw I haven't fully read the Lua script to see precisely what it does
06:25 🔗 yipdw 599, though, pretty much means "you are banned gtfo"
06:25 🔗 kennethre i'll try EU
06:27 🔗 kennethre OSError: out of pty devices
06:27 🔗 yipdw oh, HTTP 599 in Xangaland actually means something a bit different
06:28 🔗 yipdw https://github.com/ArchiveTeam/xanga-grab/blob/master/xanga.lua#L238
06:29 🔗 kennethre ./wget-lua-local: [Errno 2] No such file or directory
06:29 🔗 kennethre looks like this wasn't tested on the warrior buildpack maybe
06:30 🔗 yipdw hmm, which buildpack?
06:30 🔗 yipdw there's two in the ArchiveTeam organization
06:30 🔗 kennethre https://github.com/ArchiveTeam/heroku-buildpack-archiveteam-warrior.git
06:30 🔗 yipdw oh
06:30 🔗 yipdw interesting
06:31 🔗 kennethre very odd, i did have a few complete successfully
06:31 🔗 kennethre might be something on my end
06:31 🔗 kennethre prob runtime differences in eu and us
06:32 🔗 kennethre yeah i keep getting OSError: out of pty devices
06:32 🔗 yipdw huh, I've never seen that before
06:32 🔗 kennethre oh well
06:32 🔗 kennethre maybe i can setup some digitalocean boxes later
06:33 🔗 yipdw ooh, good point
06:33 🔗 yipdw I forgot they existed
06:33 🔗 kennethre :D
06:33 🔗 yipdw maybe they're not banned
06:34 🔗 yipdw " You agree that you will NOT use DigitalOcean's services to: violate any applicable state and federal law and regulation, including, but not limited to, any copyright, trademark, patent, anti-piracy, or other intellectual property law or regulation, or encourage or enable others to violate any such law or regulation. "
06:34 🔗 yipdw heh
06:35 🔗 yipdw might as well write "You agree that you will NOT use DigitalOcean's services"
06:40 🔗 yipdw hmm
06:52 🔗 yipdw kennethre: is there any way to get the DigitalOcean dashboard to show your droplet's host SSH key fingerprints?
12:15 🔗 SmileyG mkpty?
13:11 🔗 Cameron_D Oh dear, my scrape of The H died
13:11 🔗 Cameron_D our ot memory
13:12 🔗 Cameron_D *out
13:13 🔗 Cameron_D running another one that skips all of the /priceinsight/ directory, that was the killer (lots of filtering options, etc.)
15:55 🔗 balrog ouch. http://www.theverge.com/2013/7/30/4570610/instagram-deleting-photos-from-instance-windows-phone-app

irclogger-viewer