[04:04] chfoo: does the auto expire claim thing work? i couldn't test it in sqlite as it can't do datetime comparison reliably [04:08] willwill: it seems to work, i haven't looked at the code deeply yet [04:10] from what i test in sqlite it will always mark item as expired thus repeated /api/get will always have same result until that job is done [04:11] i'm still working out some thing but i noticed that we should make that a user cannot claim more than 1 item per service per ip address [04:11] and that the minimum script version check could probably be a global setting [04:17] otherwise, it is working nicely. no crashes so far [04:17] i was thinking that /api/get should offer work that is compatible user's version [04:18] a global one could work too [04:18] if we can do both, that would be good [04:19] i needed a global one because during setting up the tracker, someone had very old code that would return a bad api response [04:31] do you have any other issues or ideas with the tracker? tomorrow, i plan to fix the client so it will sleep and show the error message from the tracker on http errors [04:45] how do other projects compare their versions? [04:48] each project has their own minimum version. the previous urlteam tracker had a global list of acceptable versions. [04:48] i'm thinking of adding an int field similar to android's version code and comparing them [04:48] but i saw that the client code had a date as the version code so i'm not sure that i should just comparing date [04:49] i only used date because that is the convention the warrior tracker uses, but an int could be used as well [04:51] ok, i think i will add that and maybe project stats [04:52] we actually might not need a global version check because the warrior pipeline upgrades the submodule automatically. [04:54] if we can minimize the amount of code in scraper.py in terroroftinytown-client-grab, it would mean less manual script upgrading [04:54] for custom codes, why can't just the client check for the project name/url and load up the proper class to load data? [04:55] i can think of only use case is that the tracker explicitly ask admin for class name and sent that to clients [04:55] but could be a security issue if not properly secured [04:56] yes, i didn't want the client to download code from the tracker for security reasons. but i would like to make use of the automatic git submodule updating [04:57] i mean, what's the motivation behind the "use custom code" checkbox in the admin page [04:57] if it require one, it can be checked by minimum version already [04:59] oh, i understand now. the checkbox is probably redundant [04:59] the client code can just check the project name and do extra things if needed [05:05] ok, i have to go sleep now. let me know what else you'll be working on. [05:05] ok, gotta get lunch too [15:26] 13terroroftinytown/06master 1467aa5e0 15Manatsawin Hanmongkolchai: Only allow 1 claim/project/ip [15:26] 13terroroftinytown/06master 14ab62f2a 15Manatsawin Hanmongkolchai: Minimum client version checking [15:26] [13terroroftinytown] 15whs pushed 2 new commits to 06master: 02https://github.com/ArchiveTeam/terroroftinytown/compare/327a89d5c333...ab62f2aa6f6f [15:32] 13terroroftinytown-client-grab/06master 14fedfc1a 15Manatsawin Hanmongkolchai: Version now an integer. Also bumped client version [15:32] [13terroroftinytown-client-grab] 15whs pushed 1 new commit to 06master: 02https://github.com/ArchiveTeam/terroroftinytown-client-grab/commit/fedfc1a1f82949baf102cc1569966d514bc46fae [15:33] added a check to prevent claiming more than 1 item/ip/project. if there is multiple project you will get work from another project instead until you have pending claim from all projects [15:33] also added a minimum version check. the version number is integer and there are 2 for the submodule, and the -client-grab one. minimum version can be set seperately [15:35] the tracker will hand out works from projects that has its minimum requirement compatible with users. that means outdated user might still get jobs from some projects [15:37] one of these commits has schema change for projects table [15:40] going to bed, i hope GLaDOS will relay stuff when i'm back [15:58] 13terroroftinytown-client-grab/06master 14ae4aa4a 15Christopher Foo: Use --remote to track master submodule if possible. [15:58] [13terroroftinytown-client-grab] 15chfoo pushed 2 new commits to 06master: 02https://github.com/ArchiveTeam/terroroftinytown-client-grab/compare/fedfc1a1f829...33f899e01872 [15:58] 13terroroftinytown-client-grab/06master 1433f899e 15Christopher Foo: Merge branch 'master' of github.com:ArchiveTeam/terroroftinytown-client-grab [16:37] [13terroroftinytown] 15chfoo pushed 2 new commits to 06master: 02https://github.com/ArchiveTeam/terroroftinytown/compare/ab62f2aa6f6f...b15a822946e1 [16:37] 13terroroftinytown/06master 143300af5 15Christopher Foo: tracker:Use reason keyword in order to pass custom HTTP reasons. [16:37] 13terroroftinytown/06master 14b15a822 15Christopher Foo: client: Reraise RequestError as TrackerErrors [16:45] 13terroroftinytown/06master 148575b70 15Christopher Foo: fixup! client: Reraise RequestError as TrackerErrors [16:45] 13terroroftinytown/06master 14b184200 15Christopher Foo: tracker.model: Check for case where no projects are configured. [16:45] [13terroroftinytown] 15chfoo pushed 2 new commits to 06master: 02https://github.com/ArchiveTeam/terroroftinytown/compare/b15a822946e1...b18420099a9f [16:47] 13terroroftinytown-client-grab/06master 140056b9c 15Christopher Foo: Bump submodule. [16:47] [13terroroftinytown-client-grab] 15chfoo pushed 1 new commit to 06master: 02https://github.com/ArchiveTeam/terroroftinytown-client-grab/commit/0056b9c967c526ed24b393fae00becb146ef78d8 [17:44] 13terroroftinytown/06master 147146282 15Christopher Foo: tracker.test: Fix unit test so each item is 20 URLs, not 21. [17:44] [13terroroftinytown] 15chfoo pushed 1 new commit to 06master: 02https://github.com/ArchiveTeam/terroroftinytown/commit/7146282292065301cc890ddb9762cdb301526a2f [18:09] [13terroroftinytown] 15chfoo pushed 1 new commit to 06master: 02https://github.com/ArchiveTeam/terroroftinytown/commit/b28132360ca4707c614cb114ff1bde45166812fc [18:09] 13terroroftinytown/06master 14b281323 15Christopher Foo: tracker: Add GzipRotatingFileHandler