| Time |
Nickname |
Message |
|
15:00
🔗
|
ersi |
soultcer: How's the unshortening going? Dataset growing a bunch, these days? :-) |
|
15:00
🔗
|
soultcer |
It's going great |
|
15:00
🔗
|
ersi |
How many IP's are you using? I see you're absolutely grabbin' it :) |
|
15:00
🔗
|
soultcer |
In fact, when the default warrior project is set to urlteam I have trouble keeping up with importing the tasks into the main database |
|
15:01
🔗
|
ersi |
:D |
|
15:01
🔗
|
soultcer |
I am running from 9 hosts at the moment |
|
15:01
🔗
|
soultcer |
Though that is mostly a background task, the hosts are obviously used for something else most of the time |
|
15:01
🔗
|
ersi |
That explains the stats :-) |
|
15:02
🔗
|
ersi |
Yeah, of course |
|
15:02
🔗
|
ersi |
Disk isn't starting to be a problem on the tinyarchive tracker, though? |
|
15:04
🔗
|
soultcer |
Well for the tracker both disk and mostly the software is the problem |
|
15:04
🔗
|
soultcer |
I should switch from sqlite to some better system |
|
15:04
🔗
|
GLaDOS |
mysql? |
|
15:04
🔗
|
* |
GLaDOS is hilarious |
|
15:05
🔗
|
soultcer |
Was thinking postgres or some nosql db |
|
15:05
🔗
|
ersi |
MySQL or Postgres would probably be just fine |
|
15:05
🔗
|
soultcer |
The tracker runs completely separate from the main database though |
|
15:05
🔗
|
ersi |
Yarr |
|
15:05
🔗
|
soultcer |
Every couple of days I export all finished tasks to a tar archive and then import those files on the main db |
|
15:06
🔗
|
soultcer |
The main db is an old WD Green 1.5 TB that sits in an external drive usb connector thingie |
|
15:06
🔗
|
soultcer |
But I plan to upgrade soon |
|
15:06
🔗
|
ersi |
How large is the set, these days? :o |
|
15:07
🔗
|
soultcer |
512G /media/tinyarchive/database/data/ |
|
15:07
🔗
|
soultcer |
david@kat:~$ du -hs /media/tinyarchive/database/data/ |
|
15:07
🔗
|
ersi |
Yowza :-) |
|
15:07
🔗
|
GLaDOS |
That's a lot of short URLS |
|
15:07
🔗
|
ersi |
That's a lot of long URLS |
|
15:09
🔗
|
GLaDOS |
Also, I guarantee that you're going to hate me for this next sentence. |
|
15:09
🔗
|
GLaDOS |
I have managed to secure at.fi |
|
15:10
🔗
|
soultcer |
Well have fun spending a couple hours each day just on blocking spam links |
|
15:17
🔗
|
soultcer |
Oh, and of course be prepared to be attacked by the tinyback scrapers :D |
|
15:17
🔗
|
ersi |
;-) |
|
15:21
🔗
|
GLaDOS |
soultcer: how often does the tracker stats update? |
|
15:26
🔗
|
ersi |
GLaDOS: Do you mean the dashboard? |
|
15:26
🔗
|
GLaDOS |
Yeah |
|
15:26
🔗
|
ersi |
Immediately. |
|
15:27
🔗
|
GLaDOS |
huh |
|
15:27
🔗
|
ersi |
For what I've seen in the source - it does queries directly to the db |
|
15:35
🔗
|
soultcer |
It updates every 60 seconds, the script sends an expires header and nginx does the caching |
|
15:40
🔗
|
ersi |
ah |