#archiveteam 2012-01-02,Mon

↑back Search

Time Nickname Message
00:07 🔗 Wyatt|Wor I'd like to free up some space on my VPS. At what point will it be "safe" to remove some splinder stuff and which bits exactly?
00:07 🔗 dnova I deleted all my splinder stuff a while ago
00:07 🔗 Wyatt|Wor (I ran out of quota last night when I started a dld-streamer to redo the things that had errors.)
00:08 🔗 Wyatt|Wor Also, what about rsyncing the accounts with special characters that uploading choked on? Do we have a procedure for that yet?
00:08 🔗 dnova you can fix them manually
00:08 🔗 dnova that's what I did
00:09 🔗 Coderjoe i also fixed it, by taking out the backslashes that got added
00:10 🔗 Coderjoe the problem for rsync isn't the special characters. it is that the step between finding profiles and feeding them to rsync loses the backslashes
00:10 🔗 Coderjoe so the dir has backslashes and rsync is looking for a directory without backslashes
00:14 🔗 Wyatt|Wor Aaah, I see.
00:14 🔗 Wyatt|Wor find -print0| xargs -0?
00:20 🔗 Coderjoe the script is doing a find to get the profile dirs, then checking for .incomplete and some other things and then echos the acceptable profiles to rsync, which is getting a list of files from stdin
00:20 🔗 Coderjoe and the backslashes are getting lost in that last echo
00:20 🔗 chronomex leaning toothpick syndrome
00:22 🔗 Coderjoe the backslashes are not part of the actual profile name, but were added by the downloader script.
00:34 🔗 bsmith093 whats xargs -0, infinite?
00:38 🔗 Wyatt|Wor bsmith093: null termination.
00:39 🔗 Wyatt|Wor find -print0 outputs null-terminated; xargs -0 accepts null-terminated input.
02:38 🔗 bsmith093 yipdw: whats the command to automate the storyinator script, i have your list of id numbers, and my own method has been running for 2 dtys now and only has 25k stories.
04:01 🔗 Wyatt|Wor Oh, what's this? We have something for defeating Fanfiction.net?
04:04 🔗 dnova defeating?
04:07 🔗 Wyatt|Wor I would consider us doing our thing on a site with a draconian robots.txt and millions of users and their content is a "win".
04:07 🔗 Wyatt|Wor So they will then have "lost" as they have been "defeated". Or something like that.
04:07 🔗 dnova ehhh
04:15 🔗 Wyatt|Wor It's all bombast and metaphor because that's fun. At the end of the day, as long as we manage to rescue fanfiction.net from itself, we can consider it a job well-done, IMO.
04:17 🔗 dnova fanfiction.net is not making any shutdown threats
04:17 🔗 dnova nor have they (as far as I know) been hostile to archival efforts
04:19 🔗 Wyatt|Wor User-agent: ia_archiver
04:19 🔗 Wyatt|Wor Disallow: /
04:19 🔗 Wyatt|Wor That's good enough for me.
04:19 🔗 * dnova sighs
07:06 🔗 SketchCow Huzzzzah
08:01 🔗 SketchCow Back at the ranch for a few days, before heading to MD
14:57 🔗 Schbirid http://www.youtube.com/watch?v=S8e7g8kJIlo
15:32 🔗 yipdw bsmith093: what is storyinator
20:31 🔗 Schbirid oh fantastic, i found another source for .plan/finger files http://web.archive.org/web/200102032203/http://www.webdog.org/tracker/newstrack.shtml
20:33 🔗 Schbirid hm, prolly no new ones

irclogger-viewer