Lord_Nigh: Considering the IA staffer in question was Jason, I think a more likely case than legal is "Jason is known to oppose robots.txt entirely, so ask him not to talk about IA's policies on it"
hmm. could be.
he may be pissed off about the current situation but his hands are tied
who knows
If some *other* IA staffer were to say "I'm actually not allowed to comment on the robots.txt thing", I'd be a lot more likely to point to legal stuff.
true
for all i know the entire "sudden change" in the way robots.txt is a bug
but if so i have no idea who is in charge of fixing it
I have no doubt Jason would prefer to see the Wayback Machine ignore robots.txt, but I am also certain it isn't his decision.
And I know he pissed off various people when he talked about the proposed changes earlier, so it doesn't surprise me *at all* that he was asked not to talk about it further.
pissed off various people *OUTSIDE IA*
important clarification.
I have no idea what anyone actually connected with IA felt about it.
I'm also pretty amused/frustrated by MogMiner's comment: "given how open IA usually is, I'm just baffled why this is such a touchy subject with them"
It's really *obvious* to me at least why this is a touchy subject!
IA's fundamental way to deal with people upset about them distributing stuff is to say, "OK, sorry, we've stopped now. Bye!"
Adjusting robots.txt functionality in Wayback (either way) affects that -- *of course* they are touchy about it!
And the tactic (of saying, sorry-we've-stopped-now) works better the less attention is paid to it -- so minimizing what is said about things ...
... associated with the topic is basic sensible strategy.
OK, rant over.
tapedrive: http://database.savefanfiction.tk/ throws a 404 Sorry, the requested URL 'http://database.savefanfiction.tk/' caused an error:Not found: '/'
Somebody2: also don't prevent from saving anyway and just not displaying
gotta think of the long game :)
jrwr: wait, what?
I think I forgot the thread of this conversation.
Yarg! Evil Robots.txt
:)
Somebody2: Having Wayback ignore Robots.txt, it could still store and not display
I would suspect that is whats going on
Ah, I see.
I bet its just some temp bullshit going down
I agree that is likely, yes.
Somebody2: just got done setting up a Onion Enabled ArchiveBot
jrwr: I saw -- good work!
I feel like that is a ever growing gap
I know there can be some crazy shit, but its still our culture
another possibility is that they're trying different robots.txt behaviours to see how much people complain about them, perhaps
davidar: that's an ... amusing ... possibility
hahaha
I love it
I went on a watching spree of defcon and other con videos
man I think ive watched over 3-4 hours of SketchCow talking now
jrwr: only 4 hours? There's lots more...
I know, just haven't hunted them down "yet"
jrwr: http://ascii.textfiles.com/speaking
jrwr: it's a list, not a specific one
Im adding them all
nods
He is good fun
Right now im watching  3C33 on a 6502 found in a cable box
I don't know why, but i love the 6502, I've done a ton of arcade machine repair and love the layout it promotes
I need to publish my work on a clip on SRAM tester and 6502 Debugger
it uses a fuck ton of pogo pins, but allows me to sniff all the data off the 6502 as it comes in and out, and the rPi keeps up with it in a 6502 emulated CPU, this allows the rPi to in theory replace the 6502 on board
according to https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/#comment-359347 "ia_archiver" isn't even wayback's useragent (anymore?)
davidar: interesting
arkiver: I'm still seeing the infinite 206s for IMzy
It's not on every item, but I've got a big backlog of items on my VPSes with them
MrRadar: isn't that expected? 206 just means there's more to download
But fot literally 3000+ requests in a row?
3514=206 https://www.imzy.com/api/accounts/profiles/mcnulty?check=true
For the same URL
yeah, i got a lot of those too, i guess i assumed it was fetching some more data or something
the page is empty, maybe it's imzys way of saying come back later? dunno
I get different sized responses when refreshing in Chrome
MrRadar: i'm beginning to suspect that you're right, something is very strange with the imzy archiver
p/win 2
erji0jwq9r1r0rjaodsjfiq
dsaffdiqi023
rong.
Verizon closed on their acquisition of Yahoo: https://techcrunch.com/2017/06/13/verizon-closes-4-5b-acquisition-of-yahoo-marissa-mayer-resigns-memo/
Kaboom
I have a feeling its going to die now
its going to get renamed
Veriyoo
Interesting link from /r/DataHoarder: https://it.reddit.com/r/DataHoarder/comments/6gyspx/2_tib_of_old_scene_games_finally_released/
nice, thanks
Grab it, Catalog it, and store it
Someday, some one will want it
thats how I see it
even if they can't use it for the next 50 years
if we dont save it, no one else will, save it for the future when 100 years from now we can show them how all this went down
\