[11:48] <soultcer> swebb: My bad, I misunderstood the output of your program
[11:49] <soultcer> e.g for bit.ly/glyyh, which redirects to http://friends.myspace.com/index.cfm?fuseaction=profile.friendmoods&friendId=401903151&dateTime=308909160, which redirects to another URL and finally to http://www.myspace.com/error?ETOID=0&EC=404
[11:50] <soultcer> I expected the line in the output file to be: bit.ly/glyyh|http://friends.myspace.com/index.cfm?fuseaction=profile.friendmoods&friendId=401903151&dateTime=308909160
[11:50] <soultcer> But it actually is bit.ly/glyyh|http://www.myspace.com/error?ETOID=0&EC=404
[11:53] <soultcer> This unfortunately means that I can't directly import your file into the tinyarchive database
[11:53] <soultcer> But I will add it to the release as a separate file because it is still very useful
[11:53] <soultcer> Apropos release. i am currently running the database dump/release process on a server at home
[12:33] <soultcer> Oh snap, I broke the tracker
[12:36] <ersi> oops :)
[16:00] <swebb> soultcer: Hmm.  I see that you're right, but it shouldn't be that way.  I'll have to check things out.
[16:01] <swebb> The example that I gave was not transparently ignoring all intermediate redirects.  It should be logging all 301's.
[16:12] <swebb> Ok, I think that I found my bug.  I can re-run everything and re-crawl/re-unwind everything, but it might take a (long) while since I've got like 15 months of data to re-crawl.
[16:13] <ersi> Ouch
[16:14] <swebb> Yup.  Not a pleasant thing to do, but doable.
[16:14] <ersi> yeah