[22:50] <chfoo> i want to scrape a url shortener. should i do this myself or would it be best to get it working with the warrior infrastructure?
[23:02] <ersi> Preferebly extending the current urlteam tinyback project
[23:02] <ersi> but you could go and scrape it yourself if it's a small one I guess. We'd love to get the data do. :)
[23:02] <ersi> preferebly in the same format as we currently utilise. shortcode|link like Dg4fsd|http://archiveteam.org
[23:11] <chfoo> i'll try extending tinyback
[23:13] <ersi> Feel free to check in on some of my commits (I've mainly only added shortener sites ("services")
[23:14] <ersi> and ask if there's anything
[23:21] <chfoo> how should i deal with double redirects? it redirects to a personal share url like username.shortener.com/share/xxxxxx and then to the final url?
[23:22] <chfoo> oh wait, hold on
[23:23] <ersi> currently we just iterate through all the short codes that we've generated a pattern for (like sequential or if it's some other way)'
[23:23] <ersi> I've never seen a user.shortener.tld/share/cake
[23:23] <ersi> what are you digging into? delicious or something?
[23:23] <chfoo> its sharedby.co formerly visibli.com
[23:24] <chfoo> i realized the shortcode is the same during the double redirect
[23:25] <ersi> indeed