| Time |
Nickname |
Message |
|
00:56
🔗
|
|
IceMaster has joined #archiveteam-bs |
|
00:59
🔗
|
hook54321 |
arkiver: That person that that made that tweet about ISP injection that you linked to in #archiveteam awhile ago messaged me on Twitter and said he saw another one, and I think he wants me to pass this link along to you. https://www.dropbox.com/s/uxmebnokwfq2i9k/OptimumOnline-HTMLInjection-Example.zip?dl=0 |
|
01:29
🔗
|
|
hdch has joined #archiveteam-bs |
|
01:59
🔗
|
|
godane has quit IRC (Remote host closed the connection) |
|
02:00
🔗
|
|
godane has joined #archiveteam-bs |
|
02:01
🔗
|
|
VerfiedJ has quit IRC (Quit: Leaving) |
|
02:22
🔗
|
|
DFJustin has quit IRC (Remote host closed the connection) |
|
03:16
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
03:16
🔗
|
|
swebb sets mode: +o DFJustin |
|
03:16
🔗
|
MR9K |
wrt XIVDB, try poking Vekien on Discord? |
|
03:16
🔗
|
MR9K |
"yo can we have a little more time to archive the site" or whatever |
|
03:17
🔗
|
|
DFJustin has quit IRC (Remote host closed the connection) |
|
03:18
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
03:18
🔗
|
|
swebb sets mode: +o DFJustin |
|
03:20
🔗
|
JAA |
MR9K: If you know where to find them, please try. I'm researching how the site works right now so I can start an ArchiveBot job for it. |
|
03:22
🔗
|
Flashfire |
MR9K What discord? |
|
03:22
🔗
|
|
espes__ has quit IRC (Ping timeout: 252 seconds) |
|
03:28
🔗
|
|
tuluu has quit IRC (Read error: Connection refused) |
|
03:29
🔗
|
|
tuluu has joined #archiveteam-bs |
|
03:38
🔗
|
|
K4k_ has joined #archiveteam-bs |
|
03:38
🔗
|
|
K4k_ has quit IRC (Client Quit) |
|
04:02
🔗
|
|
BlueMax has quit IRC (Quit: Leaving) |
|
04:02
🔗
|
|
BlueMax has joined #archiveteam-bs |
|
04:03
🔗
|
|
exoire has quit IRC (Read error: Operation timed out) |
|
04:12
🔗
|
|
espes__ has joined #archiveteam-bs |
|
04:24
🔗
|
Flashfire |
Im thinking of making a list for Gopher sites similar to the one for FTP sites |
|
04:25
🔗
|
MR9K |
Flashfire: read the end page |
|
04:25
🔗
|
MR9K |
it specifically mentions a Discord |
|
04:27
🔗
|
|
hdch has quit IRC (Remote host closed the connection) |
|
04:27
🔗
|
|
hdch has joined #archiveteam-bs |
|
04:34
🔗
|
|
qw3rty118 has joined #archiveteam-bs |
|
04:41
🔗
|
|
qw3rty117 has quit IRC (Ping timeout: 600 seconds) |
|
04:42
🔗
|
JAA |
I'm grabbing the XIVDB with wpull now. |
|
04:42
🔗
|
JAA |
I created a seed list for the pages I could find (items, places, etc.) except those that have a way too large numeric range to scan in time. |
|
04:43
🔗
|
JAA |
Let's see how far this gets. |
|
04:43
🔗
|
JAA |
The site will not be very browsable since much of it is JS-based. |
|
04:43
🔗
|
JAA |
There's also an API, and I'll look into what can be done with that. |
|
04:44
🔗
|
|
odemg has quit IRC (Ping timeout: 265 seconds) |
|
04:46
🔗
|
benjins |
JAA This search (with page incremented) seems to get all the items, etc. in their database https://api.xivdb.com/search?language=en&strict=off&page=1 |
|
04:46
🔗
|
benjins |
Basically, a search with no filters |
|
04:46
🔗
|
|
hdch has quit IRC (Read error: Operation timed out) |
|
04:53
🔗
|
JAA |
benjins: Thanks. Iterating over that now to see whether my seed list was good. |
|
04:54
🔗
|
Flashfire |
JAA since you dont use discord do you want me to get you an API key? |
|
04:56
🔗
|
|
odemg has joined #archiveteam-bs |
|
04:59
🔗
|
JAA |
Flashfire: Doesn't look like a key is needed. |
|
05:00
🔗
|
JAA |
"There is no sensitive data on the API and you can only retrieve game related data, nothing regarding members, comments or screenshots." :-/ |
|
05:01
🔗
|
Flashfire |
You get more requests per second with an api key |
|
05:02
🔗
|
Flashfire |
Also has endpoint restrictions |
|
05:03
🔗
|
JAA |
Huh, https://github.com/zamnetwork/api mentions nothing about that. |
|
05:03
🔗
|
JAA |
Is there more API documentation? |
|
05:03
🔗
|
Flashfire |
https://xivapi.com/docs |
|
05:04
🔗
|
JAA |
Uhm, that doesn't look related to xivdb.com. |
|
05:04
🔗
|
Flashfire |
It is mentioned in their closing page to go to a discord |
|
05:04
🔗
|
Flashfire |
thats what the discord gives me as info |
|
05:05
🔗
|
JAA |
Huh |
|
05:05
🔗
|
Flashfire |
Have questions or fancy making your own database? Get in touch on Discord: https://discord.gg/MFFVHWC. |
|
05:06
🔗
|
Flashfire |
Sorry for the slight flood in your DM |
|
05:06
🔗
|
JAA |
Can you ask them whether XIVAPI is shutting down as well? |
|
05:09
🔗
|
JAA |
xivapi.com is also linked on the right side on https://xivdb.com/end.html, by the way. |
|
05:09
🔗
|
JAA |
And described as "A free open source, community driven API for FFXIV". |
|
05:11
🔗
|
Flashfire |
XIVAPI is shiny and new they said |
|
05:11
🔗
|
Flashfire |
xivapi is replacing xivdb basically |
|
05:11
🔗
|
Flashfire |
but only the xivdb api part |
|
05:17
🔗
|
JAA |
Ah, makes sense. |
|
05:21
🔗
|
Flashfire |
xivdb = old site with navigation/search + api |
|
05:21
🔗
|
Flashfire |
xivapi = completely new site with only api |
|
05:21
🔗
|
Flashfire |
idk what archiveteam is or what you're trying to do but everything in xivdb is outdated and not particularly useful besides for maybe the user comments |
|
05:21
🔗
|
Flashfire |
there isnt really anything worth archiving because none of the data is gone |
|
05:21
🔗
|
Flashfire |
also, no static content so you can't exactly scrape the site |
|
05:21
🔗
|
Flashfire |
the api data in xivapi is the same as xibdb except better schema and actually up to date |
|
05:21
🔗
|
Flashfire |
Thats a comment from the devs |
|
05:22
🔗
|
Flashfire |
JAA |
|
05:23
🔗
|
JAA |
Right, I assumed as much. |
|
05:23
🔗
|
JAA |
Comments and screenshots appear to be unique though. |
|
06:04
🔗
|
|
hdch has joined #archiveteam-bs |
|
06:07
🔗
|
|
wp494 has quit IRC (Read error: Operation timed out) |
|
06:08
🔗
|
|
wp494 has joined #archiveteam-bs |
|
06:47
🔗
|
JAA |
arkiver: Any update on the UOL Forums error message fix? |
|
08:09
🔗
|
|
IceMaster has quit IRC (Leaving) |
|
08:25
🔗
|
VoynichCr |
disk fail? https://catalogd.archive.org/history/archiveteam_archivebot_go_20180920180002 |
|
09:04
🔗
|
|
Stilett0 has quit IRC (Read error: Connection reset by peer) |
|
09:04
🔗
|
|
Stiletto has joined #archiveteam-bs |
|
09:05
🔗
|
|
Iridium has quit IRC (Remote host closed the connection) |
|
09:07
🔗
|
|
Iridium has joined #archiveteam-bs |
|
09:28
🔗
|
|
hdch has quit IRC (Quit: oops) |
|
09:30
🔗
|
|
espes__ has quit IRC (Ping timeout: 268 seconds) |
|
09:44
🔗
|
|
BlueMax has quit IRC (Quit: Leaving) |
|
10:42
🔗
|
|
Ganonmast has joined #archiveteam-bs |
|
11:55
🔗
|
|
tomaspark has quit IRC (Read error: Operation timed out) |
|
11:57
🔗
|
|
tomaspark has joined #archiveteam-bs |
|
12:21
🔗
|
|
tomaspark has quit IRC (Ping timeout: 360 seconds) |
|
12:37
🔗
|
|
ubahn has joined #archiveteam-bs |
|
12:42
🔗
|
|
tomaspark has joined #archiveteam-bs |
|
13:39
🔗
|
|
macrosoft has quit IRC (Ping timeout: 633 seconds) |
|
13:54
🔗
|
|
ubahn has quit IRC (Quit: ubahn) |
|
14:09
🔗
|
|
Swiss-- has quit IRC (Read error: Connection reset by peer) |
|
14:11
🔗
|
Jens |
Oh shit: https://twitter.com/archiveis/status/1081276424781287427 |
|
14:11
🔗
|
Jens |
[ Please do not use http://archive.IS mirror for linking, use others mirrors [.TODAY .FO .LI .VN .MD .PH]. .IS might stop working soon. ] |
|
14:13
🔗
|
|
Swiss- has joined #archiveteam-bs |
|
14:13
🔗
|
|
ubahn has joined #archiveteam-bs |
|
14:18
🔗
|
eientei95 |
Jens: archive.is currently redirects to archive.vn so it appears that it is just an issue with the domain name and not the server (thank god) |
|
14:19
🔗
|
Jens |
Yea, but it'll cause a lot of link rot if they lose the domain. |
|
14:19
🔗
|
|
Swiss- has quit IRC (Remote host closed the connection) |
|
14:20
🔗
|
|
Swiss- has joined #archiveteam-bs |
|
15:16
🔗
|
|
wp494 has quit IRC (Read error: Operation timed out) |
|
15:16
🔗
|
|
wp494 has joined #archiveteam-bs |
|
15:38
🔗
|
|
ubahn has quit IRC (Quit: ubahn) |
|
15:43
🔗
|
|
ubahn has joined #archiveteam-bs |
|
16:06
🔗
|
|
ubahn has quit IRC (Ping timeout: 260 seconds) |
|
16:10
🔗
|
|
vitzli has joined #archiveteam-bs |
|
16:13
🔗
|
C4K3 |
cccccchvbcdtdrbjvfefufigflindueklvhheghvcluk |
|
16:13
🔗
|
C4K3 |
wrong chat |
|
16:14
🔗
|
|
Swiss- has quit IRC (Read error: Operation timed out) |
|
16:17
🔗
|
fireglow |
hello yubikey |
|
16:30
🔗
|
|
vitzli has quit IRC (Leaving) |
|
16:58
🔗
|
|
Mateon1 has quit IRC (Ping timeout: 268 seconds) |
|
16:58
🔗
|
|
Mateon1 has joined #archiveteam-bs |
|
17:02
🔗
|
|
chimyatta has joined #archiveteam-bs |
|
17:22
🔗
|
JAA |
Can anyone give me push access to the uolforums-* repos on GitHub please? Not much time left to restart this... Kaz, astrid, chfoo, DFJustin |
|
17:23
🔗
|
astrid |
let me try and figure this out |
|
17:23
🔗
|
JAA |
There's also data on FOS that we need to move out of the way or something. Who has access there besides SketchCow? |
|
17:24
🔗
|
astrid |
what's your github username JAA |
|
17:25
🔗
|
JAA |
astrid: JustAnotherArchivist |
|
17:25
🔗
|
astrid |
ok i made you an owner of the whole github org because i'm too dumb to figure it out |
|
17:25
🔗
|
astrid |
also you seem trustworthy and motivated |
|
17:26
🔗
|
astrid |
hope this doesn't backfire _fingers crossed_ |
|
17:28
🔗
|
|
exoire has joined #archiveteam-bs |
|
17:28
🔗
|
JAA |
Sweet, thanks astrid. :-) |
|
17:32
🔗
|
JAA |
So any idea about the previous data? |
|
18:03
🔗
|
|
macrosoft has joined #archiveteam-bs |
|
18:04
🔗
|
|
m007a83_ has joined #archiveteam-bs |
|
18:08
🔗
|
|
m007a83 has quit IRC (Ping timeout: 252 seconds) |
|
18:09
🔗
|
|
m007a83_ has quit IRC (Ping timeout: 252 seconds) |
|
18:16
🔗
|
|
macrosoft has quit IRC (Quit: Leaving.) |
|
18:44
🔗
|
JAA |
So the ArchiveBot job for the Jogos UOL Forums just finished, but it only grabbed 420k URLs, so I'm pretty sure it didn't grab the whole thing. |
|
18:50
🔗
|
JAA |
New UOL code is ready, just need to sort out the data situation now. |
|
19:16
🔗
|
|
hdch has joined #archiveteam-bs |
|
19:17
🔗
|
|
hdch has quit IRC (Client Quit) |
|
19:17
🔗
|
|
hdch has joined #archiveteam-bs |
|
19:22
🔗
|
|
chimyatta has quit IRC (Ping timeout: 252 seconds) |
|
19:23
🔗
|
|
chimyatta has joined #archiveteam-bs |
|
19:35
🔗
|
|
SmileyG has joined #archiveteam-bs |
|
19:37
🔗
|
|
Smiley has quit IRC (Read error: Operation timed out) |
|
19:55
🔗
|
|
BasDub has joined #archiveteam-bs |
|
19:55
🔗
|
|
DasBub has quit IRC (Read error: Connection reset by peer) |
|
20:17
🔗
|
|
chimyatta has quit IRC (Ping timeout: 252 seconds) |
|
20:18
🔗
|
|
chimyatta has joined #archiveteam-bs |
|
20:28
🔗
|
|
chimyatta has quit IRC (Read error: Connection reset by peer) |
|
20:59
🔗
|
|
madalynn has joined #archiveteam-bs |
|
21:00
🔗
|
|
madalynn has quit IRC (Client Quit) |
|
21:05
🔗
|
arkiver |
JAA: sorry, was absent for some time |
|
21:06
🔗
|
arkiver |
hmm, I thought I had given you access |
|
21:18
🔗
|
JAA |
arkiver: All good now. Do you have access to FOS to clean up the old data? |
|
21:18
🔗
|
arkiver |
I´ll create a new target an SketchCow can delete the old data later |
|
21:19
🔗
|
JAA |
Sounds good. |
|
21:19
🔗
|
arkiver |
unfortunately have to go again, back in a few hours |
|
21:19
🔗
|
arkiver |
feel free to update the -grab repo |
|
21:19
🔗
|
* |
arkiver is back soon |
|
21:20
🔗
|
JAA |
Yup, I can handle everything except the target. |
|
21:45
🔗
|
|
Oddly has joined #archiveteam-bs |
|
22:30
🔗
|
|
ndiddy has joined #archiveteam-bs |
|
22:37
🔗
|
|
ndiddy has quit IRC () |
|
22:51
🔗
|
|
BlueMax has joined #archiveteam-bs |
|
23:16
🔗
|
|
BlueMax has quit IRC (Quit: Leaving) |
|
23:25
🔗
|
|
phirephly has quit IRC (Quit: ZNC 1.6.3+deb1 - http://znc.in) |
|
23:27
🔗
|
|
phirephly has joined #archiveteam-bs |