| Time |
Nickname |
Message |
|
00:12
🔗
|
|
kristian_ has quit IRC (Leaving) |
|
00:30
🔗
|
|
GE has quit IRC (Remote host closed the connection) |
|
00:34
🔗
|
|
VADemon has quit IRC (Quit: left4dead) |
|
00:39
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
00:39
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
00:50
🔗
|
|
Stiletto has joined #archiveteam-bs |
|
01:05
🔗
|
|
BlueMaxim has joined #archiveteam-bs |
|
01:08
🔗
|
|
JesseW has joined #archiveteam-bs |
|
01:09
🔗
|
|
tomwsmf has quit IRC (Read error: Operation timed out) |
|
01:20
🔗
|
|
username1 has joined #archiveteam-bs |
|
01:23
🔗
|
|
schbirid2 has quit IRC (Read error: Operation timed out) |
|
01:26
🔗
|
Frogging |
http://arstechnica.com/gaming/2016/08/pokemon-go-sheds-more-than-10m-users/ |
|
01:26
🔗
|
Frogging |
"investors have been concerned that this new user experience has been detracting from time spent on other mobile focused apps," |
|
01:26
🔗
|
|
tomwsmf has joined #archiveteam-bs |
|
01:26
🔗
|
Frogging |
oh no, those poor investors |
|
01:26
🔗
|
Frogging |
the horror |
|
01:32
🔗
|
|
tomwsmf has quit IRC (Read error: Operation timed out) |
|
01:53
🔗
|
|
chrchr has joined #archiveteam-bs |
|
01:55
🔗
|
godane |
i'm up to 2015 with gawker.com |
|
01:55
🔗
|
godane |
i'm uploading 2008 daily daily dumps |
|
01:56
🔗
|
|
rchrch has quit IRC (Ping timeout: 492 seconds) |
|
02:01
🔗
|
|
Honno has joined #archiveteam-bs |
|
02:05
🔗
|
|
rchrch has joined #archiveteam-bs |
|
02:08
🔗
|
|
chrchr has quit IRC (Read error: Operation timed out) |
|
02:58
🔗
|
|
dxrt- sets mode: +o dxrt |
|
03:08
🔗
|
|
robink has quit IRC (Ping timeout: 246 seconds) |
|
03:15
🔗
|
|
bzc6p_ has joined #archiveteam-bs |
|
03:15
🔗
|
|
swebb sets mode: +o bzc6p_ |
|
03:18
🔗
|
|
tomwsmf has joined #archiveteam-bs |
|
03:22
🔗
|
|
bzc6p has quit IRC (Ping timeout: 633 seconds) |
|
03:24
🔗
|
|
tomwsmf has quit IRC (Read error: Operation timed out) |
|
03:36
🔗
|
|
JesseW has quit IRC (Ping timeout: 370 seconds) |
|
03:45
🔗
|
|
robink has joined #archiveteam-bs |
|
04:14
🔗
|
|
Sk1d has quit IRC (Ping timeout: 194 seconds) |
|
04:20
🔗
|
|
Sk1d has joined #archiveteam-bs |
|
04:48
🔗
|
|
JesseW has joined #archiveteam-bs |
|
05:06
🔗
|
|
GE has joined #archiveteam-bs |
|
05:06
🔗
|
godane |
so i have all of gawker.com on my drive |
|
05:17
🔗
|
|
Coderjoe_ has quit IRC (Read error: Connection reset by peer) |
|
05:18
🔗
|
|
HCross2 has joined #archiveteam-bs |
|
05:21
🔗
|
godane |
i'm starting to go after kotaku.com |
|
05:21
🔗
|
godane |
only for sitemaps from 2008 on |
|
05:21
🔗
|
godane |
sitemaps from 2005 to 2007 was very small so they should be all there with my year dumps |
|
05:24
🔗
|
|
trs80 has joined #archiveteam-bs |
|
05:29
🔗
|
|
Coderjoe has joined #archiveteam-bs |
|
05:31
🔗
|
GE |
http://gawker.com/how-things-work-1785604699 Rest in piss Gawker |
|
05:35
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
05:39
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
05:39
🔗
|
|
GE_ has joined #archiveteam-bs |
|
05:39
🔗
|
|
GE has quit IRC (Ping timeout: 255 seconds) |
|
05:39
🔗
|
|
GE_ is now known as GE |
|
05:41
🔗
|
|
BartoCH has quit IRC (Ping timeout: 260 seconds) |
|
05:55
🔗
|
yipdw_ |
running VirtualBox in Xen over X over ssh is a very interesting experience |
|
05:56
🔗
|
yipdw_ |
namely it's very slow and then suddenly you get amazing spooky action like |
|
05:56
🔗
|
yipdw_ |
$ dmesg => Segmentation fault |
|
05:56
🔗
|
joepie91 |
yipdw_: add mosh for bonus points |
|
05:56
🔗
|
yipdw_ |
joepie91: I can barely click a button as-is |
|
05:57
🔗
|
yipdw_ |
well I can but it takes about 11 seconds |
|
05:59
🔗
|
|
bzc6p_ has left |
|
06:06
🔗
|
|
bzc6p has joined #archiveteam-bs |
|
06:06
🔗
|
|
swebb sets mode: +o bzc6p |
|
06:06
🔗
|
bzc6p |
So the DNS History guys finally put up a real shutdown notice on their main page (although with small letters), including this sentence: |
|
06:07
🔗
|
bzc6p |
"Due to the ArchiveTeams self-righteous attitude CloudFlares DDoS protection has been enabled. They are quite open about their attitude of ignoring robots.txt files and work around blocks on their user agent - this as abuse." |
|
06:08
🔗
|
Atluxity |
have we tried to communicate with them? |
|
06:08
🔗
|
bzc6p |
Yes |
|
06:09
🔗
|
bzc6p |
I could have not been more polite in the beginning. |
|
06:09
🔗
|
Atluxity |
ah, well then |
|
06:09
🔗
|
Atluxity |
good job |
|
06:09
🔗
|
Atluxity |
that was not sarcasm |
|
06:09
🔗
|
bzc6p |
However, when it turned out there is no hope, I wrote them a mail that was less friendly |
|
06:16
🔗
|
bzc6p |
"Due to the ArchiveTeams self-righteous attitude CloudFlares DDoS protection has been enabled. They are quite open about their attitude of ignoring robots.txt files and work around blocks on their user agent - this as abuse." |
|
06:16
🔗
|
bzc6p |
no |
|
06:16
🔗
|
bzc6p |
"DNS History has now shutdown, any updates are for my personal requirements only - the site may go up/down as I need the resources it uses. The number of servers dedicated to the DNS History project is now ZERO." |
|
06:16
🔗
|
bzc6p |
i wanted to paste this |
|
06:16
🔗
|
bzc6p |
There are at least two things here I find strange |
|
06:16
🔗
|
bzc6p |
But i should stop minding these... guys. |
|
06:17
🔗
|
bzc6p |
*guy |
|
06:17
🔗
|
bzc6p |
who is a whole consultancy |
|
06:17
🔗
|
* |
bzc6p shuts up |
|
06:17
🔗
|
JesseW |
eh, my guess is that the site owner was already feeling unappareciated (and underpaid) for providing the free service they provided for many years, and had already reached a breaking point in being willing to put up with it (therefore deciding to shut it down) -- so other people coming in and wanting copies (but not offering to *pay* for them, or even to pay for the cost of transferring them), further pissed him off. |
|
06:18
🔗
|
joepie91 |
afaik the initial e-mails sent to them were perfectly polite yet ignored |
|
06:18
🔗
|
JesseW |
I can understand us not being in a position to offer to pay for the transfer costs, but I do suspect that might have prompted a better response |
|
06:18
🔗
|
joepie91 |
as in, no response at all |
|
06:18
🔗
|
JesseW |
sure they were polite, but politely asking for what? |
|
06:19
🔗
|
JesseW |
as I understand it, the initial emails were asking (quite reasonably for our point of view), for a free copy of the data |
|
06:19
🔗
|
bzc6p |
Well, we don't like such sentences, but if they really shut it down on July 10, there wouldn't have been all that hoo-haa. And keeping the site up for needing the resources sounds just bullshit to me, although I'm not a web engineer. |
|
06:20
🔗
|
joepie91 |
(it's bullshit) |
|
06:20
🔗
|
JesseW |
unfortunently, (I suspect) from the site maintainer's point of view, providing free access was exactly what they had run out of patience in doing |
|
06:20
🔗
|
bzc6p |
ALSO JesseW : please read my reasoning in my later mail: https://secure.8086.net/portal/viewticket.php?tid=NTM-405143&c=9JIYc46J |
|
06:20
🔗
|
joepie91 |
JesseW: my problem is that they _ignored_ it. |
|
06:20
🔗
|
JesseW |
looking now |
|
06:20
🔗
|
joepie91 |
they didn't say "nah don't feel like it" |
|
06:20
🔗
|
joepie91 |
they didn't ask "can you cover transfer costs" |
|
06:20
🔗
|
joepie91 |
they just ignored the entire message(s) |
|
06:21
🔗
|
bzc6p |
joepie91: that's what i mean, yes |
|
06:21
🔗
|
bzc6p |
I've phrase my thoughts about that |
|
06:21
🔗
|
bzc6p |
*d |
|
06:21
🔗
|
JesseW |
sure, they were *already* in "F***k all you people on the internet, I'm taking my ball and going home!" mode *before* we contacted them. |
|
06:22
🔗
|
joepie91 |
sure, but that is their problem |
|
06:22
🔗
|
JesseW |
certainly. I'm not excusing it, just (trying to) explain it |
|
06:22
🔗
|
joepie91 |
like, I'm perfectly willing to work together with somebody who wants to make things go smoothly, but if somebody doesn't given another person the light of day then fuck them |
|
06:23
🔗
|
joepie91 |
and at the point where somebody ignores communications, that leaves no options other than to just grab the goddamn thing |
|
06:23
🔗
|
JesseW |
right, my point is just that the site maintainer was *already* in a "fuck everyone" mood -- and our (perfectly polite) emails weren't able to redirect that |
|
06:23
🔗
|
joepie91 |
yes, I understand the problem |
|
06:24
🔗
|
joepie91 |
I'm not disagreeing there |
|
06:24
🔗
|
bzc6p |
In these terms, they were no better than us just standing on the other (more rightful) side. |
|
06:24
🔗
|
joepie91 |
I'm just saying that I consider them to be in the wrong for that |
|
06:25
🔗
|
bzc6p |
Who operates something should not be in a "fuck everyone" mode. Or they could just push the turnoff button. Isn't, killing a service, so easy? |
|
06:25
🔗
|
bzc6p |
I still can't understand his behaviour either he was an employee or a boss. |
|
06:26
🔗
|
bzc6p |
Or I can understand, because the world is full of... |
|
06:26
🔗
|
|
GE has quit IRC (Ping timeout: 255 seconds) |
|
06:26
🔗
|
bzc6p |
but can't accept |
|
06:27
🔗
|
bzc6p |
What do we expect from someone who communicates their shutdown as "it was a fun project thanks folks" |
|
06:27
🔗
|
bzc6p |
they don't see how GREAT it was |
|
06:28
🔗
|
JesseW |
as I see it, the site owner was (and still is) wavering between wanting to be of service, and not wanting to pay the (emotional, financial, etc) costs of doing so, and feeling like he was getting insufficient help with it. Our approaches reminded him that the service was appareciated, but didn't manage to communicate an offer of help in a way that made sense to him. |
|
06:30
🔗
|
JesseW |
but, I've said my piece, and should really aim in the direction of sleep now |
|
06:30
🔗
|
joepie91 |
JesseW: what's the practical conclusion of the above? |
|
06:30
🔗
|
bzc6p |
Well, if I decided to kill a service I'd do it. Not this keep-kill dilemma for weeks. I hate that. |
|
06:30
🔗
|
joepie91 |
because the only way forward I see at this point is breaking cloudflare's thing |
|
06:30
🔗
|
PurpleSym |
whydomain: Nothing fancy. Just a M-shaped table for the book and a cheap digital camera (Canon with CHDK). Works surprisingly well. |
|
06:31
🔗
|
bzc6p |
joepie91: if we did it, maybe they'd finally find the turnoff button, so... |
|
06:34
🔗
|
JesseW |
joepie91: it's not that practical, but if one of the silent masses reading this thread happens to have a bunch of money available, I think approaching the site owner with an explicit offer to *buy* a full copy of the data (and permission to post it freely) would be the best choice at this point. |
|
06:35
🔗
|
bzc6p |
I wonder whether they keep a database realizing its historical value, or just rm * |
|
06:35
🔗
|
JesseW |
figuring out how to break Cloudflare's thing is a good general idea, but I don't think this copy of this data is worth it |
|
06:35
🔗
|
joepie91 |
JesseW: "worth it" in what sense? |
|
06:35
🔗
|
bzc6p |
in the former case, JesseW's point is even more fair |
|
06:36
🔗
|
joepie91 |
chfoo: if I write a Python port of my cloudflare browser check breaking thing, can you add it to wpull? |
|
06:36
🔗
|
joepie91 |
JesseW: right. I'm not expecting that to happen, though... |
|
06:37
🔗
|
JesseW |
worth the cost (in terms of further responses from the site owner towards us, and maybe IA, and the energy cost of the servers) vs the fact that this isn't unique or creative data |
|
06:37
🔗
|
JesseW |
there are other people who have kept copies of DNS records, even going back as far |
|
06:37
🔗
|
JesseW |
but most of them already charge for access |
|
06:37
🔗
|
joepie91 |
JesseW: I'm not aware of any other comparably complete public sources |
|
06:37
🔗
|
joepie91 |
yeah, not counting the ones that charge |
|
06:37
🔗
|
JesseW |
which is probably one of the reasons the site owner felt put upon |
|
06:38
🔗
|
joepie91 |
I would definitely consider this unique. |
|
06:38
🔗
|
JesseW |
being the only one giving away something for which other people are charging can get pretty frustrating |
|
06:38
🔗
|
joepie91 |
yes, I am aware of that, but what I'm looking for right now is a practical solution, before the entire thing goes boom |
|
06:38
🔗
|
JesseW |
joepie91: say more about why you consider it unique? |
|
06:38
🔗
|
joepie91 |
[08:37] <joepie91> JesseW: I'm not aware of any other comparably complete public sources |
|
06:39
🔗
|
JesseW |
That doesn't make the *data* unique |
|
06:39
🔗
|
JesseW |
that just makes the (up till now) *access* unique |
|
06:39
🔗
|
joepie91 |
for all practical purposes, it does. |
|
06:39
🔗
|
joepie91 |
it is not viable to obtain the records through other means. |
|
06:40
🔗
|
joepie91 |
bzc6p: btw, the cloudflare bot check is easy to break. |
|
06:40
🔗
|
joepie91 |
JesseW: the problem is that by the very nature of this data, it is hard to obtain and the businesses that sell it exist purely because of that fact |
|
06:41
🔗
|
joepie91 |
meaning that none of the commercial sellers of this data are likely to ever contribute it to an archive |
|
06:41
🔗
|
JesseW |
given that continuing to access the site is a very obvious instance of "access without authorization" (cf CFAA), I don't think this is any more accessible than the commerical sellers |
|
06:41
🔗
|
joepie91 |
not a concern outside of the US, and the above is the case for almost anything archiveteam does |
|
06:41
🔗
|
bzc6p |
Let me remind that it's still an ASSUMPTION that they don't want to provide it for the money reason. |
|
06:41
🔗
|
joepie91 |
I don't see how this holds relevance |
|
06:41
🔗
|
JesseW |
one could, theoretically, break into the stores of the commercial sellers and release the data that way |
|
06:41
🔗
|
joepie91 |
I am talking about practical feasibility |
|
06:42
🔗
|
JesseW |
both are pratical |
|
06:42
🔗
|
JesseW |
and access without authorization is not US specific |
|
06:42
🔗
|
joepie91 |
JesseW: to be honest, I'm not finding this a constructive disxcussion |
|
06:42
🔗
|
joepie91 |
discussion* |
|
06:42
🔗
|
JesseW |
(although the CFAA is, of course) |
|
06:42
🔗
|
joepie91 |
in the sense of the CFAA? it absolutely is |
|
06:42
🔗
|
joepie91 |
this is not a thing that exists in such a form outside of the US and a few other countries |
|
06:43
🔗
|
joepie91 |
but again, _I don't really care_, what I care about is obtaining the data as quickly as possible and as completely as possible before it goes away |
|
06:43
🔗
|
bzc6p |
Well, it's true that we try to "break in" (~abuse) with more tricks than usual joepie91 |
|
06:43
🔗
|
joepie91 |
and I think we're getting stuck on the "is this worth saving" discussion which is _precisely_ how things get lost |
|
06:43
🔗
|
JesseW |
I think I agree with you about this not being a particularly constructive discussion at this point |
|
06:43
🔗
|
joepie91 |
bzc6p: that is no different from eg. isohunt or anything with a rate limiter |
|
06:44
🔗
|
JesseW |
it's clear that the site owner will continue to do whatever they can to interfere with attempts to get mass copies of the data |
|
06:44
🔗
|
bzc6p |
joepie91: there is such thing that "breaking into computer system" in all countries. |
|
06:44
🔗
|
joepie91 |
... |
|
06:45
🔗
|
joepie91 |
bzc6p: the problem with the CFAA is not that it criminalizes "breaking into computer systems" |
|
06:45
🔗
|
JesseW |
it's also pretty obvious that with sufficient determination, some additional fraction could be gotten, before the site is fully shut down |
|
06:45
🔗
|
joepie91 |
and that is not what is applicable here |
|
06:45
🔗
|
joepie91 |
the problem is that any kind of non-sanctioned form of access to an otherwise public resource can be considered in violation of the CFAA whereas this is *not* covered on the usual "breaching a computer system" laws in other countries |
|
06:46
🔗
|
JesseW |
but such attempts are likely to cause the site to be shut down sooner than it would be otherwise |
|
06:46
🔗
|
joepie91 |
JesseW: it's already on its deathbed |
|
06:46
🔗
|
bzc6p |
joepie91: if you do break the CF and we can go on with it, I personally don't mind. These idiots (sorry) could just turn the fuck down if they really wanted to. And without the resources bullshit. |
|
06:46
🔗
|
JesseW |
whether this is worth it or not... I agree isn't a particularly productive discussion. |
|
06:46
🔗
|
joepie91 |
right, so I have two practical questions |
|
06:47
🔗
|
joepie91 |
1) are there any other ideas for obtaining the data without further conflict? |
|
06:47
🔗
|
bzc6p |
(1) no |
|
06:47
🔗
|
joepie91 |
2) if I write an implementation of the CF breaking thing in Python, is there somebody who can integrate this into wpull? |
|
06:48
🔗
|
JesseW |
1) aside from buying it (either from the site owner or someone else), I don't think so |
|
06:48
🔗
|
JesseW |
2) Not me, IDK if someone else would |
|
06:48
🔗
|
joepie91 |
okay, is there anybody willing/able to spend the $ on it? |
|
06:48
🔗
|
* |
bzc6p thinks noboy else is reading us atm |
|
06:48
🔗
|
bzc6p |
*nobody |
|
06:49
🔗
|
JesseW |
heh, the channel is logged -- anyone (including the site owner) may read this sometime in the future |
|
06:50
🔗
|
joepie91 |
I mean, I can barely pay my rent, so I'm out |
|
06:50
🔗
|
joepie91 |
lol |
|
06:51
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
06:51
🔗
|
JesseW |
makes sense. as I said, it wasn't a very practical suggestion |
|
06:52
🔗
|
JesseW |
(although I wouldn't be astonished if Brewster of IA doesn't end up buying a copy (and permission) to post it from one of the other commercial sources eventually) |
|
06:53
🔗
|
joepie91 |
idk, I doubt it |
|
06:53
🔗
|
joepie91 |
bulk availability of their records would basically destroy their business model |
|
06:53
🔗
|
joepie91 |
which is based on exclusivity of data |
|
06:53
🔗
|
joepie91 |
it would likely be so expensive that it isn't worth it to anybody including IA |
|
06:54
🔗
|
joepie91 |
it's one thing to shoot somebody a few hundred bucks for a database dump |
|
06:54
🔗
|
joepie91 |
it's a whole other thing to essentially acquire a profitable business |
|
06:55
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
06:55
🔗
|
Medowar |
what is this about again? Still DNS History? Buying the site? |
|
06:55
🔗
|
joepie91 |
Medowar: dnshistory, yes, initially about offering to buy their data |
|
06:55
🔗
|
Medowar |
how much? |
|
06:55
🔗
|
joepie91 |
now the discussion moved on to buying data from other commercial providers later on |
|
06:56
🔗
|
joepie91 |
Medowar: no idea, it was a suggestion from JesseW with the rationale that perhaps the owner just doesn't want to deal with shit for free anymore |
|
06:56
🔗
|
joepie91 |
so, "whatever would sway the owner" basically |
|
06:57
🔗
|
JesseW |
it's clearly not based on exclusivity, as there are multiple providers (including this free one, until now) |
|
06:57
🔗
|
joepie91 |
Medowar: like, I'd ballpark it as "a couple hundred bucks" but that can be either massively overestimated or massively underestimated |
|
06:57
🔗
|
joepie91 |
or the owner may not even care about money at all |
|
06:57
🔗
|
joepie91 |
JesseW: not really true |
|
06:57
🔗
|
joepie91 |
dnshistory didn't offer (free) bulk access |
|
06:57
🔗
|
joepie91 |
so is not a competitor in that space |
|
06:57
🔗
|
joepie91 |
no API whatsoever either |
|
06:57
🔗
|
JesseW |
do any of the providers give bulk access? |
|
06:57
🔗
|
Medowar |
joepie91: If it would be in that range, Id be willig to buy it |
|
06:58
🔗
|
joepie91 |
JesseW: yes, that's what you pay for :P |
|
06:58
🔗
|
joepie91 |
pricing schemes vary but the usecase is largely "bulk historic DNS lookups" |
|
06:58
🔗
|
joepie91 |
like, the odd researcher buying 100 records at a time is a nice extra, but not where they make their $ |
|
06:58
🔗
|
JesseW |
joepie91: well, as I understood, the other providers also charge for individual, random access |
|
06:58
🔗
|
joepie91 |
sure, they do, but there's only so much profit you can make from that |
|
06:58
🔗
|
Medowar |
but I think the owner already hates achiveteam so much, we would not get an offer like that.. |
|
06:59
🔗
|
joepie91 |
"people interested in DNS history but not in bulk" is a pretty small target demographic |
|
06:59
🔗
|
JesseW |
Medowar: I'd suggest you could try contacting the owner *without* mentioning Archiveteam |
|
06:59
🔗
|
joepie91 |
mostly going to be hobbyist malware/corporation researchers, and random skids looking to cause trouble |
|
06:59
🔗
|
joepie91 |
and while the skid market is growing, it's still not very big :P |
|
07:00
🔗
|
joepie91 |
and yeah, I'd suggest the same |
|
07:00
🔗
|
joepie91 |
I would much prefer sorting it out this way |
|
07:00
🔗
|
joepie91 |
we still need to eventually break CF for archivebot, but de-escalation is preferable here |
|
07:00
🔗
|
JesseW |
agreed |
|
07:00
🔗
|
joepie91 |
(lots of people who use CF but have no intention of blocking archiveteam) |
|
07:00
🔗
|
* |
JesseW /msg'ed Medowar |
|
07:00
🔗
|
joepie91 |
(plus "I'm under attack" mode is sometimes enabled automatically anyway) |
|
07:02
🔗
|
JesseW |
to continue my hypothetical about IA/Brewser buying such bulk data sometime -- but some of these companies go into bankruptcy; at which time their assets could be sold for (relatively) cheap. |
|
07:02
🔗
|
Medowar |
should I remove all references on archiveteam from my website, since I will be writing from my domain? |
|
07:04
🔗
|
JesseW |
Medowar: I don't think that's necessary. If the site owner bothers to look hard enough, he'll figure out that we discussed it here, in public. The only reason not to mention it in the email is to make it clear that you are acting independently, and avoid prejudicing him to start with. |
|
07:05
🔗
|
joepie91 |
(in case the site owner *does* read here: to re-emphasize, the problem isn't shutting down the service or not wanting to deal with shit, but the lack of communication.) |
|
07:05
🔗
|
JesseW |
Medowar: The basic point is to make it clear that you noticed he stated "funding issues" as the reason for the closure, and you are interested in providing money to help with those. |
|
07:05
🔗
|
JesseW |
That seems (to me at least) the right way to open discussions. |
|
07:06
🔗
|
JesseW |
If/when he responds positively to that, you can talk more about the best way to preserve and continue to provide access to the data. |
|
07:07
🔗
|
JesseW |
and with that, I really *am* going to sleep |
|
07:08
🔗
|
joepie91 |
Medowar: JesseW: on which note, if he just wants to hand off the entire site to a third party for them to run it, I can handle that. |
|
07:08
🔗
|
joepie91 |
it's not a resource-intensive thing to run |
|
07:08
🔗
|
joepie91 |
(in and of itself) |
|
07:13
🔗
|
|
JesseW has quit IRC (Ping timeout: 370 seconds) |
|
07:17
🔗
|
joepie91 |
Medowar: anyhow, let me know what the response is, please :) |
|
07:35
🔗
|
|
bzc6p has left |
|
07:58
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
08:01
🔗
|
|
RichardG has quit IRC (Ping timeout: 370 seconds) |
|
08:01
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
08:19
🔗
|
|
kristian_ has joined #archiveteam-bs |
|
08:32
🔗
|
|
fie__ has quit IRC (Read error: Connection reset by peer) |
|
08:33
🔗
|
|
fie has joined #archiveteam-bs |
|
08:38
🔗
|
|
BartoCH has joined #archiveteam-bs |
|
08:44
🔗
|
|
DFJustin has quit IRC (Ping timeout: 260 seconds) |
|
08:45
🔗
|
|
RichardG has joined #archiveteam-bs |
|
08:54
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
08:54
🔗
|
|
swebb sets mode: +o DFJustin |
|
08:57
🔗
|
|
DopefishJ has joined #archiveteam-bs |
|
08:57
🔗
|
|
swebb sets mode: +o DopefishJ |
|
08:59
🔗
|
|
DFJustin has quit IRC (Ping timeout: 260 seconds) |
|
09:06
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
09:06
🔗
|
|
swebb sets mode: +o DFJustin |
|
09:07
🔗
|
|
DopefishJ has quit IRC (Ping timeout: 260 seconds) |
|
09:21
🔗
|
|
DopefishJ has joined #archiveteam-bs |
|
09:21
🔗
|
|
swebb sets mode: +o DopefishJ |
|
09:21
🔗
|
|
DFJustin has quit IRC (Read error: Operation timed out) |
|
09:39
🔗
|
|
DopefishJ has quit IRC (Read error: Operation timed out) |
|
09:39
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
09:39
🔗
|
|
swebb sets mode: +o DFJustin |
|
09:57
🔗
|
|
GE has joined #archiveteam-bs |
|
10:12
🔗
|
arkiver |
joepie91: where was that non-python script again of getting around cloudflare? |
|
10:12
🔗
|
arkiver |
(I haven't read the entire above conversation) |
|
10:14
🔗
|
arkiver |
We need to break cloudflare anyway for feature projects |
|
10:17
🔗
|
joepie91 |
arkiver: https://gist.github.com/joepie91/c5949279cd52ce5cb646d7bd03c3ea36 |
|
10:17
🔗
|
* |
arkiver is still reading the log |
|
10:17
🔗
|
joepie91 |
arkiver: the timeout in the original code is necessary btw and you need to put together a few other things |
|
10:17
🔗
|
joepie91 |
but that code breaks the weird expressions |
|
10:19
🔗
|
* |
arkiver finally finished reading the log |
|
10:19
🔗
|
arkiver |
thanks joepie91 :D |
|
10:20
🔗
|
joepie91 |
arkiver: it still works btw |
|
10:20
🔗
|
joepie91 |
just tested it |
|
10:20
🔗
|
joepie91 |
like a few hours ago |
|
10:23
🔗
|
arkiver |
awesome |
|
10:25
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
10:29
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
11:47
🔗
|
|
DFJustin has quit IRC (Read error: Operation timed out) |
|
11:55
🔗
|
|
GE has quit IRC (Quit: zzz) |
|
12:04
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
12:04
🔗
|
|
swebb sets mode: +o DFJustin |
|
12:27
🔗
|
|
kristian_ has quit IRC (Leaving) |
|
13:07
🔗
|
|
tuankiet has quit IRC (Ping timeout: 244 seconds) |
|
13:16
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
13:19
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
13:20
🔗
|
|
tuankiet has joined #archiveteam-bs |
|
13:50
🔗
|
|
BartoCH has quit IRC (Ping timeout: 260 seconds) |
|
13:51
🔗
|
|
BartoCH has joined #archiveteam-bs |
|
13:56
🔗
|
|
GE has joined #archiveteam-bs |
|
14:14
🔗
|
|
tuankiet has quit IRC (Ping timeout: 244 seconds) |
|
14:14
🔗
|
|
tuankiet has joined #archiveteam-bs |
|
14:36
🔗
|
sep332 |
If you put cloudflare in front of your site, your site is still at the same IP address, right? |
|
14:36
🔗
|
sep332 |
It's just the nameservers that point to the CF IP |
|
14:46
🔗
|
Atluxity |
no |
|
14:46
🔗
|
Atluxity |
well |
|
14:47
🔗
|
Atluxity |
the hostname will resolve to cloudfront |
|
14:47
🔗
|
Atluxity |
but if you connect to the old ip, and say "Host: X" in the header, then you will probably get the site |
|
14:48
🔗
|
Atluxity |
they can change ip and go into hiding |
|
15:00
🔗
|
sep332 |
Yeah I thought it would be worth a shot. And it would be harder for them to move to a different subnet, so scanning a /24 or so would be easy |
|
15:01
🔗
|
sep332 |
depends how motivated they are and if they know what they're doing :p |
|
15:06
🔗
|
joepie91 |
er |
|
15:06
🔗
|
joepie91 |
Atluxity: cloudfront != cloudflare |
|
15:06
🔗
|
joepie91 |
:p |
|
15:07
🔗
|
joepie91 |
sep332: anyhow, the site can reject connections from non-cloudflare |
|
15:07
🔗
|
joepie91 |
making it a bit harder to discover it |
|
15:10
🔗
|
|
kristian_ has joined #archiveteam-bs |
|
15:12
🔗
|
Atluxity |
ah :P |
|
15:12
🔗
|
Atluxity |
yes, it can... especially if it monitors its logs and gets lots of pesty traffic |
|
15:21
🔗
|
sep332 |
right - i knew that |
|
15:21
🔗
|
sep332 |
*goes to get more caffeine* |
|
15:23
🔗
|
Atluxity |
but it might not |
|
15:27
🔗
|
joepie91 |
... |
|
15:28
🔗
|
joepie91 |
I just thought of something |
|
15:28
🔗
|
joepie91 |
Atluxity: sep332: we are scraping a *DNS history* site, are we not |
|
15:28
🔗
|
joepie91 |
:| |
|
15:28
🔗
|
Atluxity |
:P |
|
15:28
🔗
|
sep332 |
oh you mean https://dnshistory.org/historical-dns-records/a/dnshistory.org |
|
15:28
🔗
|
Atluxity |
haha |
|
15:28
🔗
|
joepie91 |
yeah... |
|
15:29
🔗
|
Atluxity |
this channel is not publicly logged, eh? |
|
15:29
🔗
|
sep332 |
In fact you can pull up the old nameservers as well https://dnshistory.org/dns-records/dnshistory.org |
|
15:30
🔗
|
joepie91 |
sven@linux-etoq:~> http get 178.63.72.28 Host:dnshistory.org |
|
15:30
🔗
|
joepie91 |
[...] |
|
15:30
🔗
|
joepie91 |
<p>The document has moved <a href="https://dnshistory.org/">here</a>.</p> |
|
15:30
🔗
|
joepie91 |
Atluxity: no idea |
|
15:30
🔗
|
Medowar |
use https |
|
15:31
🔗
|
sep332 |
or IPv6 https://dnshistory.org/historical-dns-records/aaaa/dnshistory.org |
|
15:32
🔗
|
sep332 |
Medowar: good eye |
|
15:32
🔗
|
joepie91 |
well yes ofc, but it's surprisingly hard to do that |
|
15:32
🔗
|
joepie91 |
from a terminal client |
|
15:32
🔗
|
joepie91 |
:P |
|
15:32
🔗
|
joepie91 |
because SNI |
|
15:33
🔗
|
sep332 |
throw it in your /etc/hosts ? |
|
15:34
🔗
|
arkiver |
it is logged |
|
15:35
🔗
|
arkiver |
http://archive.fart.website/bin/irclogger_logs |
|
15:35
🔗
|
Medowar |
telnet 178.63.72.28 443 |
|
15:35
🔗
|
Medowar |
the server doesnt even support https |
|
15:35
🔗
|
Medowar |
that is also, how you move entirely to cloudflare |
|
15:36
🔗
|
joepie91 |
eh |
|
15:36
🔗
|
joepie91 |
the server definitely supports https |
|
15:37
🔗
|
joepie91 |
yep |
|
15:37
🔗
|
joepie91 |
works |
|
15:37
🔗
|
joepie91 |
added to hosts |
|
15:38
🔗
|
Medowar |
oh, yeah right |
|
15:38
🔗
|
Medowar |
my entire server is banned |
|
15:38
🔗
|
Medowar |
that is why it cant connect |
|
15:38
🔗
|
sep332 |
lol |
|
15:38
🔗
|
atrocity |
lol |
|
15:39
🔗
|
atrocity |
i love the loathing message they put on dnshistory to us, lol |
|
15:39
🔗
|
Medowar |
yeah, was crawling them heavy and they ip banned a few of our servers before moving to cloudflare |
|
15:39
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
|
15:40
🔗
|
Medowar |
I cant even ping them |
|
15:50
🔗
|
joepie91 |
I find there to be some delicious irony in the fact that a DNS history site that sells delisting services for "people who have set up DDoS mitigation" fails to delist their own IP after setting up bot mitigation |
|
15:55
🔗
|
Medowar |
professionals working. |
|
15:55
🔗
|
|
GE has quit IRC (Ping timeout: 255 seconds) |
|
15:55
🔗
|
Medowar |
80/86 would consult |
|
15:56
🔗
|
|
GE has joined #archiveteam-bs |
|
16:09
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
16:10
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
16:14
🔗
|
atrocity |
lol |
|
16:20
🔗
|
joepie91 |
(similarly, I also find irony in the fact that we didn't think to check the DNS history of the DNS history service...) |
|
16:22
🔗
|
Sanqui |
why are we using a public dns service instead of doing the job ourselves? do we know anybody there so we can trust them? |
|
16:22
🔗
|
joepie91 |
...? |
|
16:23
🔗
|
Sanqui |
sorry, i didn't read much of the scrollback |
|
16:23
🔗
|
Sanqui |
assumed from it that we are scraping dnshistory.org? |
|
16:23
🔗
|
Sanqui |
oh, dnshistory is shutting down |
|
16:23
🔗
|
Sanqui |
ok |
|
16:50
🔗
|
|
BartoCH has quit IRC (Ping timeout: 260 seconds) |
|
16:51
🔗
|
|
BartoCH has joined #archiveteam-bs |
|
17:01
🔗
|
|
GE has quit IRC (Ping timeout: 255 seconds) |
|
17:04
🔗
|
|
GE has joined #archiveteam-bs |
|
17:09
🔗
|
|
kristian_ has quit IRC (Leaving) |
|
17:11
🔗
|
|
BartoCH has quit IRC (Ping timeout: 260 seconds) |
|
17:39
🔗
|
|
RichardG_ has joined #archiveteam-bs |
|
17:44
🔗
|
|
RichardG has quit IRC (Ping timeout: 370 seconds) |
|
17:49
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
17:54
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
17:59
🔗
|
username1 |
dd claims another victim |
|
17:59
🔗
|
|
username1 is now known as schbirid |
|
17:59
🔗
|
schbirid |
luckily of a drive i am currently (and still) rsyncing |
|
17:59
🔗
|
schbirid |
only noticed when i tried to eject )( |
|
18:12
🔗
|
|
tomwsmf has joined #archiveteam-bs |
|
18:44
🔗
|
|
BartoCH has joined #archiveteam-bs |
|
18:45
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
18:53
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
19:02
🔗
|
hook54321 |
Sanqui: I'm assuming we can't archive it because it uses cloudflare, correct? |
|
19:25
🔗
|
|
DFJustin has quit IRC (Read error: Operation timed out) |
|
19:26
🔗
|
hook54321 |
I feel like who ever runs dns history is sitting in our irc channels to spy on us |
|
19:44
🔗
|
sep332 |
nice try hook54321, we all know you're the real spy |
|
19:45
🔗
|
hook54321 |
I've been here for months :P |
|
19:53
🔗
|
|
GE has quit IRC (Ping timeout: 255 seconds) |
|
19:54
🔗
|
SketchCow |
Just like a true spy |
|
19:55
🔗
|
|
GE has joined #archiveteam-bs |
|
20:12
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
20:12
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
20:13
🔗
|
* |
xmc nods quietly |
|
20:13
🔗
|
xmc |
i've been spying on you all for *years* |
|
20:13
🔗
|
xmc |
little did you know |
|
20:14
🔗
|
xmc |
i infiltrated some of your inmost circles |
|
20:14
🔗
|
xmc |
now i'm going to turn you in to the internet copyright police |
|
20:14
🔗
|
xmc |
muahahaha |
|
20:18
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
20:23
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
20:51
🔗
|
|
GE_ has joined #archiveteam-bs |
|
20:52
🔗
|
|
GE has quit IRC (Ping timeout: 255 seconds) |
|
20:52
🔗
|
|
GE_ is now known as GE |
|
21:12
🔗
|
godane |
i'm going to be running a check of my gawker.com recent grabs to make sure all files are go uploaded before deleting them |
|
21:13
🔗
|
godane |
just so i know that archive.org took all my files |
|
21:28
🔗
|
SketchCow |
I have secretly been planning to bake you all into brownies |
|
21:28
🔗
|
SketchCow |
Also, I am about to go out to my shipping container. If I don't respond in a week, something fell on me |
|
21:43
🔗
|
|
JW_work1 has quit IRC (Ping timeout: 633 seconds) |
|
21:55
🔗
|
|
atrocity has quit IRC (Read error: Connection reset by peer) |
|
22:39
🔗
|
|
RichardG_ has quit IRC (Ping timeout: 370 seconds) |
|
23:06
🔗
|
|
GE has quit IRC (Quit: zzz) |
|
23:08
🔗
|
|
JW_work has joined #archiveteam-bs |
|
23:32
🔗
|
|
kristian_ has joined #archiveteam-bs |
|
23:45
🔗
|
|
Honno has quit IRC (Read error: Operation timed out) |
|
23:59
🔗
|
hook54321 |
SketchCow: You might as well just give us a live stream with a gopro |