[15:00] soultcer: How's the unshortening going? Dataset growing a bunch, these days? :-) [15:00] It's going great [15:00] How many IP's are you using? I see you're absolutely grabbin' it :) [15:00] In fact, when the default warrior project is set to urlteam I have trouble keeping up with importing the tasks into the main database [15:01] :D [15:01] I am running from 9 hosts at the moment [15:01] Though that is mostly a background task, the hosts are obviously used for something else most of the time [15:01] That explains the stats :-) [15:02] Yeah, of course [15:02] Disk isn't starting to be a problem on the tinyarchive tracker, though? [15:04] Well for the tracker both disk and mostly the software is the problem [15:04] I should switch from sqlite to some better system [15:04] mysql? [15:04] * GLaDOS is hilarious [15:05] Was thinking postgres or some nosql db [15:05] MySQL or Postgres would probably be just fine [15:05] The tracker runs completely separate from the main database though [15:05] Yarr [15:05] Every couple of days I export all finished tasks to a tar archive and then import those files on the main db [15:06] The main db is an old WD Green 1.5 TB that sits in an external drive usb connector thingie [15:06] But I plan to upgrade soon [15:06] How large is the set, these days? :o [15:07] 512G /media/tinyarchive/database/data/ [15:07] david@kat:~$ du -hs /media/tinyarchive/database/data/ [15:07] Yowza :-) [15:07] That's a lot of short URLS [15:07] That's a lot of long URLS [15:09] Also, I guarantee that you're going to hate me for this next sentence. [15:09] I have managed to secure at.fi [15:10] Well have fun spending a couple hours each day just on blocking spam links [15:17] Oh, and of course be prepared to be attacked by the tinyback scrapers :D [15:17] ;-) [15:21] soultcer: how often does the tracker stats update? [15:26] GLaDOS: Do you mean the dashboard? [15:26] Yeah [15:26] Immediately. [15:27] huh [15:27] For what I've seen in the source - it does queries directly to the db [15:35] It updates every 60 seconds, the script sends an expires header and nginx does the caching [15:40] ah