#urlteam 2017-11-30,Thu

↑back Search

Time Nickname Message
02:58 🔗 Somebody2 Jusque: we have error messages -- let me print them out for you:
03:30 🔗 Somebody2 Jusque: hm, tinyurl seems to be working again. IDK
03:31 🔗 Somebody2 if it starts breaking, I'll be sure and paste the error messages here
03:35 🔗 Somebody2 Jusque: OK, got them:
03:35 🔗 Somebody2 Traceback (most recent call last):
03:35 🔗 Somebody2 File "scraper.py", line 42, in main
03:35 🔗 Somebody2 result = scraper_client.run()
03:35 🔗 Somebody2 File "/home/ts/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/client/scraper.py", line 61, in run
03:35 🔗 Somebody2 result = self.service.scrape_one(item)
03:35 🔗 Somebody2 File "/home/ts/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/services/base.py", line 56, in scrape_one
03:35 🔗 Somebody2 url_status, result_url, encoding = self.process_response(response)
03:35 🔗 Somebody2 File "/home/ts/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/services/base.py", line 96, in process_response
03:35 🔗 Somebody2 return self.process_redirect(response)
03:35 🔗 Somebody2 File "/home/ts/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/services/tinyurl.py", line 30, in process_redirect
03:35 🔗 Somebody2 self.current_shortcode, response.headers['Location']
03:35 🔗 Somebody2 File "/home/ts/terroroftinytown-client-grab/terroroftinytown/terroroftinytown/services/tinyurl.py", line 124, in _preview
03:35 🔗 Somebody2 raise UnexpectedNoResult("Unexpected HTTP status %i on preview page %s" % (response.status_code, response.url))
03:35 🔗 Somebody2 UnexpectedNoResult: Unexpected HTTP status 301 on preview page http://tinyurl.com/preview.php?num=pzpi9az
03:35 🔗 Somebody2
03:36 🔗 Somebody2 Also this one: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe5 in position 86: invalid continuation byte
03:37 🔗 Somebody2 full traceback here: https://hastebin.com/ozojajimig.sql
03:59 🔗 * arkiver has some bad experience with 'invalid' continuation bytes and python
04:06 🔗 Somebody2 Haven't we all... :-(
04:07 🔗 Somebody2 The errors are absolutely intermittent across different URLs, but do seem consistent with particular URLs.
04:12 🔗 Somebody2 So there are two URLs that are unexpectedly returning 301 -- that's easy enough to work around, at least...
04:12 🔗 Somebody2 http://tinyurl.com/preview.php?num=pzpi9az
04:12 🔗 Somebody2 http://tinyurl.com/preview.php?num=kk5rtj7
04:28 🔗 Somebody2 And the other two failing items are the ones generating the Unicode error:
04:28 🔗 Somebody2 which *DOES* seem to be intermittent at the same item, so we don't need to worry about it for now.
04:32 🔗 Somebody2 OK, it's an easy enough fix. The tinyurl code includes a hardcoded http: url that needs to be changed to https:
04:32 🔗 Somebody2 I *think* I have sufficient rights to commit the fix, but not to deploy it. chfoo?
04:38 🔗 chfoo yeah, i can do that right now
04:39 🔗 Somebody2 Cool, thanks.
04:40 🔗 Somebody2 The tracker is complaining about https errors itself, now...
04:40 🔗 Somebody2 I presume because you are re-deploying?
04:41 🔗 Somebody2 I'll let you hit the Merge button.
04:42 🔗 chfoo i'm trying to log in right now
04:42 🔗 Somebody2 seems to be working again for me
04:43 🔗 chfoo oh, i think it was redis dumping the database file. it hasn't gotten maintenance in a long time i suppose
04:43 🔗 Somebody2 Ha.
04:46 🔗 GitHub127 has joined #urlteam
04:46 🔗 GitHub127 has left
04:48 🔗 chfoo it might take a few more minutes. it's not responding for periods of time
04:58 🔗 chfoo should be ok now
05:02 🔗 Somebody2 Will try.
05:12 🔗 Somebody2 Cool, tinyurl is fixed!
05:12 🔗 Somebody2 and turned back on
07:53 🔗 Aerochrom has joined #urlteam
09:39 🔗 medowar has joined #urlteam
09:53 🔗 Jusque Yay. Happy to help. What else can I ask questions about? :P
10:07 🔗 yuitimoth has quit IRC (se.hub irc.efnet.nl)
10:07 🔗 SmileyG has quit IRC (se.hub irc.efnet.nl)
10:07 🔗 Fusl has quit IRC (se.hub irc.efnet.nl)
10:09 🔗 yuitimoth has joined #urlteam
10:09 🔗 SmileyG has joined #urlteam
10:09 🔗 Fusl has joined #urlteam
12:44 🔗 marvinw has quit IRC (Leaving)
12:45 🔗 marvinw has joined #urlteam
12:47 🔗 dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
12:49 🔗 dashcloud has joined #urlteam
15:52 🔗 Aerochrom has quit IRC (Ping timeout: 248 seconds)
15:52 🔗 Aerochrom has joined #urlteam
18:17 🔗 CoolCanuk has joined #urlteam
18:22 🔗 phuzion has quit IRC (Remote host closed the connection)
21:40 🔗 figpucker has quit IRC (Read error: Connection reset by peer)

irclogger-viewer