| Time |
Nickname |
Message |
|
07:44
🔗
|
GLaDOS |
Torrent is at 5% |
|
07:44
🔗
|
GLaDOS |
soultcer: oh, ok then. |
|
08:35
🔗
|
GLaDOS |
Importing tasks.. |
|
08:37
🔗
|
ersi |
\o/ |
|
08:46
🔗
|
GLaDOS |
http://urlteam.terrywri.st/ hello world |
|
08:48
🔗
|
GLaDOS |
Would've preferred tracker.urlte.am as the URL, but oh well |
|
08:50
🔗
|
ersi |
Poke SketchCow and he'll fix the DNS for you. |
|
08:51
🔗
|
GLaDOS |
SketchCow: tracker.urlte.am to 37.59.60.160 |
|
08:51
🔗
|
GLaDOS |
much love |
|
09:08
🔗
|
soultcer |
Great! |
|
09:09
🔗
|
soultcer |
Make sure to run the cleanup.py script every 15 minutes or so to free assigned tasks |
|
09:10
🔗
|
GLaDOS |
I'm sure cron wants some attention on anarchive |
|
09:26
🔗
|
soultcer |
GLaDOS: Create a directory calles files in the tracker directory and make it writable for the webserver |
|
09:27
🔗
|
GLaDOS |
done |
|
09:28
🔗
|
soultcer |
That's where the output files will go until you get them with the fetch_finished.py script |
|
09:30
🔗
|
GLaDOS |
Is it worth it running fetch_finished.py on cron? |
|
09:30
🔗
|
GLaDOS |
s/on/with/ |
|
09:31
🔗
|
soultcer |
not necessary but you could run it like once a day |
|
09:31
🔗
|
soultcer |
The db gets kind of slow when there are lots of tasks, so I usually ran it every couple of days, put the results into a tar file and then later reimported that tar file into the main db |
|
09:33
🔗
|
GLaDOS |
So for importing tasks into the main db, what format do you suggest them to be in? (tar, in a file, etc.) |
|
09:34
🔗
|
* |
omf_ scribbles furious notes :D |
|
09:36
🔗
|
soultcer |
The same format that fetch_finished spits out |
|
09:36
🔗
|
soultcer |
I just tarred them up for transport because the tracker db and main db were on different hosts |
|
09:37
🔗
|
GLaDOS |
Ah |
|
09:37
🔗
|
GLaDOS |
It'd be so simple to make a cron script to do this. |
|
09:37
🔗
|
GLaDOS |
If I had the time. |
|
09:38
🔗
|
GLaDOS |
Which I technically do. |
|
12:34
🔗
|
GLaDOS |
soultcer: it's still adding tasks |
|
12:34
🔗
|
GLaDOS |
how long does it usually take? |
|
12:34
🔗
|
GLaDOS |
(i forgot to run it in a screen session) |
|
12:37
🔗
|
GLaDOS |
(chain_multiple(tracker, "bitly", "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ", 6, 600, 10) and sequence_from_to(tracker, "tinyurl", "0123456789abcdefghijklmnopqrstuvwxyz", "azzzzz", "bzzzzz", 600), btw) |
|
12:41
🔗
|
soultcer |
Well from azzzzz to bzzzzz with 36 characters is 60466176 codes, divided by 600 codes per task is 100777 tasks, of which you have already created 86000 |
|
12:41
🔗
|
soultcer |
So another 14000 remaining |
|
12:42
🔗
|
ersi |
So, soon done ;) |
|
12:42
🔗
|
GLaDOS |
01[13tinyback01] 15twrist pushed 1 new commit to 06master: 02 https://github.com/ArchiveTeam/tinyback/commit/12d8610b6cbfb49ba4ce060834a14755e255ffcf |
|
12:42
🔗
|
GLaDOS |
13tinyback/06master 1412d8610 15Terry Wrist: Change tracker to http://tracker.terrywri.st/ |
|
12:43
🔗
|
GLaDOS |
IGNORE URL IN COMMIT MESSAGE, IT'S CORRECT IN CODE |
|
12:44
🔗
|
omf_ |
I see graphs! |
|
12:45
🔗
|
GLaDOS |
So other shorteners to add are ur1.ca, ow.ly, snipurl.com, vbly.us, arseh.at, and is.gd |
|
12:50
🔗
|
omf_ |
let me fire up my warrior and pull down a few tasks to check it is all good |
|
12:52
🔗
|
omf_ |
GLaDOS, you need to update http://warriorhq.archiveteam.org/projects.json to point to the new urlteam leaderboard |
|
12:53
🔗
|
GLaDOS |
I don't have access |
|
12:53
🔗
|
GLaDOS |
Smiley: ^ |
|
12:53
🔗
|
ersi |
Whos machine is that anyhow? |
|
12:53
🔗
|
omf_ |
SketchCow, I believe |
|
12:53
🔗
|
GLaDOS |
warriorhq? chronomex |
|
12:54
🔗
|
ersi |
xmc: ^* |
|
12:54
🔗
|
GLaDOS |
Smiley also has access |
|
12:54
🔗
|
ersi |
We'll see who's the fastest |
|
12:55
🔗
|
GLaDOS |
Bonus points to whoever gives us access |
|
12:55
🔗
|
omf_ |
I will add an entry to http://archiveteam.org/index.php?title=Tracker |
|
12:58
🔗
|
GLaDOS |
01[13tinyarchive01] 15twrist pushed 1 new commit to 06master: 02 https://github.com/ArchiveTeam/tinyarchive/commit/f5515b833b1f6a1331736b25352b377b01ef68fd |
|
12:58
🔗
|
GLaDOS |
13tinyarchive/06master 14f5515b8 15Terry Wrist: That's http://urlteam.terrywri.st/ of course.... |
|
12:59
🔗
|
GLaDOS |
sequence_from_to(tracker, "isgd", "0123456789abcdefghijklmnopqrstuvwxyz", "kZZZZ", "zZZZZ", 600) |
|
12:59
🔗
|
GLaDOS |
Will that work? |
|
13:10
🔗
|
omf_ |
I updated the People section on the tracker page |
|
13:18
🔗
|
omf_ |
soultcer, pass that ops around |
|
13:23
🔗
|
omf_ |
We need to setup another admin for the projects file so 3 people have access |
|
13:23
🔗
|
omf_ |
As we talked about way back it would be best if all services had 3-4 people as admins to better the odds of an admin being around in irc |
|
13:24
🔗
|
GLaDOS |
I should set up access to pad/paste then |
|
13:24
🔗
|
GLaDOS |
The URLTeam tracker is just running under user urlteam on anarchive |
|
13:24
🔗
|
omf_ |
I saw and updated the wiki page with yourself, Smiley and myself as admins |
|
13:36
🔗
|
Smiley |
sup? |
|
13:37
🔗
|
omf_ |
Smiley, fix up the projects.json file to point to the new urlteam leaderboard tracker.terrywri.st |
|
13:37
🔗
|
omf_ |
fuck |
|
13:37
🔗
|
omf_ |
I mean urlteam.terrywri.st |
|
13:39
🔗
|
Smiley |
hmmm ok let me see |
|
13:39
🔗
|
Smiley |
"name": "URLTeam", |
|
13:39
🔗
|
Smiley |
"description": "Archiving various URL shorteners", |
|
13:39
🔗
|
Smiley |
"logo": "http://archiveteam.org/images/9/9d/Urlteam-logo-reasonable-size.png", |
|
13:39
🔗
|
Smiley |
"repository": "https://github.com/ArchiveTeam/tinyback.git", |
|
13:39
🔗
|
Smiley |
"title": "URLTeam", |
|
13:39
🔗
|
Smiley |
"marker_html": "<a href='http://tracker.tinyarchive.org/v1/'><img src='http://archiveteam.org/images/th$ |
|
13:40
🔗
|
Smiley |
"lat_lng": [50.4773, 12.3692], |
|
13:40
🔗
|
Smiley |
"leaderboard": "http://tracker.tinyarchive.org/v1/" |
|
13:40
🔗
|
Smiley |
So which bits? |
|
13:40
🔗
|
omf_ |
the leaderboard value |
|
13:40
🔗
|
omf_ |
just change it to "http://urlteam.terrywri.st/" |
|
13:41
🔗
|
omf_ |
GLaDOS, got the graphs up |
|
13:41
🔗
|
Smiley |
done, I think? |
|
13:41
🔗
|
GLaDOS |
And marker |
|
13:41
🔗
|
GLaDOS |
change the href |
|
13:41
🔗
|
Smiley |
ah k |
|
13:41
🔗
|
GLaDOS |
Also, forwardslash at end |
|
13:42
🔗
|
Smiley |
ah k |
|
13:42
🔗
|
Smiley |
of.... both? |
|
13:42
🔗
|
Smiley |
right, better? |
|
13:43
🔗
|
GLaDOS |
Better |
|
13:43
🔗
|
GLaDOS |
Many thansk go to Smiley |
|
13:43
🔗
|
Smiley |
:) |
|
13:43
🔗
|
Smiley |
commited to github too |
|
13:44
🔗
|
GLaDOS |
Can someone verify if sequence_from_to(tracker, "isgd", "0123456789abcdefghijklmnopqrstuvwxyz", "kZZZZ", "zZZZZ", 600) is correct? |
|
13:45
🔗
|
Cameron_D |
What do I need to do to run the project standalone again? |
|
13:45
🔗
|
GLaDOS |
I think you only need https://github.com/ArchiveTeam/tinyback |
|
13:56
🔗
|
omf_ |
GLaDOS, how much more testing you think this has got before it is ready for many users |
|
13:57
🔗
|
GLaDOS |
I honestly think we should just try for many users |
|
13:59
🔗
|
omf_ |
You should add it to the Current wiki page as a warrior project then |
|
14:02
🔗
|
GLaDOS |
Done |
|
14:04
🔗
|
omf_ |
checking |
|
14:05
🔗
|
omf_ |
make sure to put this in a tmux |
|
14:05
🔗
|
GLaDOS |
I will |
|
14:05
🔗
|
omf_ |
all the commas and quotes match up |
|
14:05
🔗
|
GLaDOS |
I feel as if not all of them should have 600 URLs per item |
|
14:06
🔗
|
soultcer |
Some pointers: |
|
14:06
🔗
|
soultcer |
- Snipurl's highest code is currently 27wmtgk, going highe won't help |
|
14:07
🔗
|
soultcer |
- Ur1ca's highest is frv3a, not fzzzz |
|
14:07
🔗
|
omf_ |
is this data from the wiki page? |
|
14:08
🔗
|
ersi |
no |
|
14:08
🔗
|
ersi |
he just checked. The 'highest' changes constantly |
|
14:08
🔗
|
ersi |
GLaDOS: Depends on how much work there is @ re 600 URLs per item |
|
14:09
🔗
|
GLaDOS |
I've checked arsehat |
|
14:09
🔗
|
soultcer |
arsehat as well, they are only up to 4hc5 |
|
14:10
🔗
|
omf_ |
We should have a note on the wiki that highest urls need to be checked and how to do that. |
|
14:10
🔗
|
soultcer |
And you seem to have confused visibli (vsb.li/shared.co) with vbly.us (formlery vb.ly) |
|
14:11
🔗
|
GLaDOS |
ah |
|
14:11
🔗
|
soultcer |
Oh and the new tr.im is also net yet up to 50zzzz |
|
14:12
🔗
|
soultcer |
Besides bit.ly, tinyurl.com maybe the bit smaller ones like is.gd and ow.ly, the other shorteners don't get many new URLs |
|
14:12
🔗
|
GLaDOS |
yeah, added an extra z |
|
14:13
🔗
|
GLaDOS |
http://paste.archivingyoursh.it/wageluxutu.md |
|
15:40
🔗
|
GLaDOS |
soultcer: what happens if I add a task that's already added? |
|
21:06
🔗
|
soultcer |
GLaDOS: The task will be run a second time, and then on import into the main db it will hopefully have no conflicts because none of the data on the url shortener is supposed to change |
|
21:06
🔗
|
soultcer |
But there is no "deduplication". Tasks get an uuid and are always refered to by that id instead of the contents |