#archiveteam-bs 2017-11-22,Wed

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
ola_norsk"THAT IS THE CASE!"...it's just my shorthand english that blows chunks [00:00]
JAAAh, got it. [00:00]
ola_norsk"da e da som e saken" (nor: "Det er det som er saken")...en: That is the case
what about reddit?
frontpage here is showing me a VPN deal for "black friday"
[00:01]
JAA20 of the 25 posts on the frontpage are currently about FCC's net neutrality announcement. [00:02]
ola_norskone seriously must make an internet 2.0... [00:03]
JAA"Calm down about the Net Neutrality thing... Paying additional money to access certain sites will give you a sense of pride and accomplishment."
Ahahahaha
[00:04]
ola_norskwhich site would that be? :D
facebook? lol
fucking hell..Gopher and telnet protocol is coming back!
[00:04]
JAAThis is what it could look like: https://i.imgur.com/QTL3At5.jpg
That's a real screenshot from a Portuguese mobile provider.
This crap is happening already.
Here's the direct link to that page: https://www.meo.pt/internet/internet-movel/telemovel/pacotes-com-telemovel
[00:06]
ola_norskif it happens more, i think that would actually be beneficial to show people how shit it is
it's redicicouls, and against every intention of what internet was meant and planned to be
ola_norsk is drunk
pardon the typos
it's not that scary though i think. I would claim it's impossible to keep going like that in the long run. Eventually it would backlash and implode. Hell, even "dark web" is just shit that was searcable on good not that long ago.
or dank web or deep web, or whatever the fuck it's called these days..(eventhough it's technically "higher web")
[00:07]
***ola_norsk has quit IRC (d.r.u.n.k) [00:14]
JAA"Deep web" is the term you're looking for. "Dark web" is a subsection of that and refers to darknets, e.g. Tor or Freenet. [00:22]
yipdwactually I really prefer "dank web"
that sounds like a way better term
[00:22]
JAAIt does. [00:23]
***ola_norsk has joined #archiveteam-bs [00:26]
ola_norski'd be happy to pay taxes for IA mirror here: http://www.bbc.com/specialfeatures/horizonsbusiness/clips-library/?autoplay=true&vid=p01jssrc&tab=2
i'm freezing my feet of
make it happen!
change.org WOULD work..
[00:27]
***ola_norsk has quit IRC (Leaving) [00:28]
...................... (idle for 1h46mn)
odemgAudioBooks this way come: http://the-eye.eu/audiobookbay.mp4 [02:14]
***j08nY has quit IRC (Remote host closed the connection) [02:15]
..... (idle for 24mn)
godanei think internet 2.0 is going be like the piratebay van in diggnation [02:39]
***ranavalon has quit IRC (Read error: Connection reset by peer) [02:46]
godaneor maybe we take the book scanning van and add rpi+librarybox+kiwix project to it [02:51]
..... (idle for 21mn)
***Mateon1 has quit IRC (Read error: Operation timed out)
Mateon1 has joined #archiveteam-bs
[03:12]
Stiletto has quit IRC ()
qw3rty113 has quit IRC (Read error: Connection reset by peer)
qw3rty113 has joined #archiveteam-bs
[03:17]
pizzaiolo has quit IRC (Remote host closed the connection)
godane has quit IRC (Quit: Leaving.)
[03:26]
........ (idle for 36mn)
wp494_ has joined #archiveteam-bs
godane has joined #archiveteam-bs
[04:04]
godaneso i'm starting to think i need to run a list of packages from slackware into debian apt-get to get close to what i need
also found out my wifi tp-link stick doesn't light up on boot in my slax-debian
[04:06]
***wp494 has quit IRC (Read error: Operation timed out) [04:10]
..... (idle for 21mn)
DrasticAcThere a good place to host a Postgres database with somewhat flexible storage?
I’m parsing the Miiverse warcs I grabbed and are putting them into a database with a web front end, so it’ll be easier to find the stuff we grabbed
I was thinking of just getting a linode and hosting it there, but I don’t know what other options are out there. Usually I host on Azure, but Postgres is in preview.
[04:31]
***qw3rty114 has joined #archiveteam-bs
wp494_ is now known as wp494
qw3rty113 has quit IRC (Read error: Operation timed out)
[04:38]
............. (idle for 1h4mn)
godane has quit IRC (Leaving.) [05:48]
..... (idle for 22mn)
TheLovina has joined #archiveteam-bs [06:10]
........ (idle for 37mn)
godane has joined #archiveteam-bs [06:47]
godaneso i'm now on my debian based slax system [06:48]
.......... (idle for 47mn)
***robogoat has quit IRC (Read error: Operation timed out) [07:35]
robogoat has joined #archiveteam-bs [07:41]
..... (idle for 20mn)
wp494_ has joined #archiveteam-bs [08:01]
godaneso some good news
and bad news
looks like my telegraph.co.uk upload script didn't upload them all
good news is still have the files so i can upload them
its all the 2006 pages as daily dumps of there sitemap archives
[08:03]
***wp494 has quit IRC (Ping timeout: 492 seconds)
schbirid has joined #archiveteam-bs
[08:07]
godanecdx and warc.gz finally uploaded: https://archive.org/details/www.telegraph.co.uk-archive-2006-04-30-pages-20160707 [08:08]
***wp494_ has quit IRC (Ping timeout: 248 seconds)
wp494 has joined #archiveteam-bs
[08:10]
godanemost of may 2006 archives have to be uploaded [08:10]
........ (idle for 37mn)
***MrDignity has quit IRC (Remote host closed the connection)
MrDignity has joined #archiveteam-bs
[08:47]
............... (idle for 1h10mn)
pizzaiolo has joined #archiveteam-bs [09:57]
....... (idle for 33mn)
j08nY has joined #archiveteam-bs [10:30]
godanei'm starting to upload my collection of rush Limbaugh radio show
https://archive.org/details/rush-limbaugh-radio-show-2005-06-03
[10:37]
..... (idle for 22mn)
***BlueMaxim has quit IRC (Quit: Leaving) [10:59]
Ravenloft has quit IRC (Read error: Connection reset by peer) [11:13]
........... (idle for 50mn)
icedice has joined #archiveteam-bs [12:03]
...... (idle for 25mn)
dashcloud has quit IRC (Read error: Connection reset by peer)
dashcloud has joined #archiveteam-bs
[12:28]
ranavalon has joined #archiveteam-bs
ranavalon has quit IRC (Read error: Connection reset by peer)
ranavalon has joined #archiveteam-bs
RichardG has quit IRC (Read error: Connection reset by peer)
RichardG has joined #archiveteam-bs
[12:40]
..... (idle for 24mn)
Stilett0 has joined #archiveteam-bs [13:08]
refeed has joined #archiveteam-bs
refeed has quit IRC (Client Quit)
[13:22]
......... (idle for 43mn)
bithippo has quit IRC (Textual IRC Client: www.textualapp.com)
bithippo has joined #archiveteam-bs
[14:05]
......... (idle for 41mn)
jrwraww they closed web-beta.archive.org
well made it private
I used the fuck out of it
[14:48]
JAATime to send an email then. :-) [14:50]
***icedice has quit IRC (Quit: Leaving) [14:57]
....... (idle for 30mn)
ZexaronS has quit IRC (Ping timeout: 633 seconds) [15:27]
schbirid2 has joined #archiveteam-bs
schbirid has quit IRC (Read error: Operation timed out)
[15:34]
bithippoAnyone have recommendations on the "best" way to archive youtube videos in cold storage locally? [15:44]
JAAyoutube-dl, I guess. [15:44]
bithippoUsing youtube-dl now, but that doesn't store the metadata, headers, etc. [15:44]
JAAIt doesn't? I thought it should. I've only rarely used it myself though. [15:45]
bithippoIt'll grab the highest quality audio and video renditions, mux them together, and voila, file created.
Today's project I suppose!
[15:45]
JAAYeah, but I think it also writes a JSON (or XML?) file with metadata. [15:46]
bithippo:thinking: Good call, going to investigate more. Lost some videos out of my Favorites playlist today that were deleted or made private, never again!
@JAA: Maw gawd it does
--write-info-json Write video metadata to a .info.json file
Thank you!!
[15:46]
JAA:-) [15:49]
***MrDignity has quit IRC (Remote host closed the connection)
MrDignity has joined #archiveteam-bs
pizzaiolo has quit IRC (Read error: Operation timed out)
pizzaiolo has joined #archiveteam-bs
[15:57]
Kazbithippo: youtube-dl --title --continue --retries 4 --write-info-json --write-description --write-thumbnail --write-annotations --all-subs --ignore-errors -f bestvideo+bestaudio URL
From http://archiveteam.org/index.php?title=YouTube
[16:05]
***pizzaiolo has quit IRC (Client Quit)
pizzaiolo has joined #archiveteam-bs
[16:06]
Chorca has quit IRC (Quit: leaving) [16:19]
bithippoDoh. Thanks @Kaz [16:19]
***MrDignity has quit IRC (Read error: Connection reset by peer)
MrDignity has joined #archiveteam-bs
[16:22]
..... (idle for 21mn)
fie has quit IRC (Ping timeout: 248 seconds) [16:43]
.... (idle for 17mn)
fie has joined #archiveteam-bs [17:00]
............. (idle for 1h2mn)
jrwrMan, if AT ever has to save youtube [18:02]
..... (idle for 22mn)
ivanthere are channels on YouTube that delete a very high percentage of their content (e.g. Apple or nokia) or upload television streams (of interest, e.g. news) that are inevitably taken down by the copyright holder
if you want a project, scrape youtube channels/users i.e. UU* playlists every day, learn which channels remove content
archive those and you'll have a nice collection in no time
[18:24]
YouTube loses some non-trivial percentage of videos every year because there are so many parties that can get a video taken down (uploader, copyright holder, random guy with fake copyright claim, privacy complainant, YouTube for ToS violation)
after some number of unresolved copyright strikes the entire channel gets nuked
[18:39]
bithippoDoes YouTube provide a list of DMCA notices/removals in a consumable format? [18:44]
ivanbithippo: I don't think so, just the copyright holder notices on individual /watch pages [18:44]
jrwrYa, no chilling affects for Youtube [18:57]
bithippoDisappoint. [18:58]
jrwrwait
it does!
https://www.lumendatabase.org/
it reports to these guys
everything
[19:10]
***j08nY has quit IRC (Read error: Connection reset by peer) [19:12]
jrwrwell
not everything
[19:12]
bithippoStill something! Thanks for sharing! [19:13]
***ola_norsk has joined #archiveteam-bs [19:15]
ola_norskso someone who is not me, wrote a representative of Den Norske Dataforening yesterday. Regarding making a Norwegian IA mirror happen: https://imgur.com/1spd0ny [19:17]
jrwrWhat does it say? [19:18]
ola_norsktl;dr "I like the idea, but could you tell me more about it?"
give me a few secs
[19:18]
jrwrNice! thats a good sign
Running a IA mirror is no feat mind you, its huge!
[19:18]
ola_norskaye i know [19:20]
***j08nY has joined #archiveteam-bs [19:21]
ola_norskbut DND (The Norwegian Computer Society) is not exactly small
http://www.dataforeningen.no/in-english.128921.no.html
[19:21]
bithippo@jrwr Does every IA item provide a torrent to download it? [19:23]
jrwrJust about
there are a ton of non public collections
https://archive.org/stats/
they are storing a "metric fuckton" of data
[19:23]
bithippoI assume the non public collections can be gotten at for cold storage mirroring with someone's approval? [19:25]
jrwrYa [19:25]
bithippow00t [19:25]
jrwryou would need to hit up SketchCow I guess for all that nonsense since he works there [19:26]
ola_norskjrwr: here's a (semi-bad) google translate version of the response https://pastebin.com/v4Hjp7Ys [19:26]
jrwralso in 2014 they said they are hosting Total used storage: 50 PetaBytes [19:26]
bithippoCrazy amount of data [19:27]
ola_norskmy frustration is that the person seems to think waybackmachine is internetarchive [19:27]
jrwrthats SUPER common [19:27]
ola_norskaye [19:27]
jrwrI'll ask people have they ever seen Archive / Internet Archive
and they will nope out, then I fall back to the Wayback machine
[19:27]
ola_norsk:D [19:28]
jrwrI would send over samples of what is stored to this guy, like all the news casts from the states ever done and indexed [19:28]
ola_norskmy hope is that it will stirr up to something, and they look more into it
someone might do that...
[19:28]
bithippoWould be awesome for Brewster and Co to do a "What is the Internet Archive?" 2 min video for these sorts of thigns [19:29]
jrwrold movies, films, maybe have a more active part from that part of the world archiving its history to it
I would love a CGP Gray on this
[19:29]
ola_norskone point should be that Alexiandria is not only in a political "hotzone", but i think someone here said it's ~10 years outdated(?) [19:30]
jrwrYa
I know there is another one as well
its the old petaboxes, they are in a datacenter deep in the EU
doing /something/ that is unknown to me and the others I was speaking to
[19:30]
ola_norskbeing kept current? [19:31]
jrwrUnknown
it was asked if AT could take it over at one point
[19:31]
ola_norsk"deep in the EU" :D
lol
[19:32]
jrwr== I forgot where it was in the EU
this was a few months ago
[19:32]
ola_norskyeah, i just liked that expression :D made me instantly think of swiss alps :D [19:33]
jrwrhttps://archive.org/about/graphs.php [19:33]
ola_norskthing is, if it's not being kept current it's still good stuff..But then it's like "timebox", i don't know the proper term for it; When people bury a box with keep-sakes to dig up later. [19:38]
astridin english, "time capsule" [19:39]
ola_norskyes
it seems to me like Alexandria is not so much mirror as well, but a time capsule, if its 10 years behind current "main data"
bithippo: Yes, an official presentation would be helpful. I think it's called a "pitch stack"..like a small facts presentation that could be included in propositions
[19:39]
jrwrWe could always use more warriors :) [19:44]
ola_norskPitchDeck seems to be the word
my problem is i can't and don't want to be a any sort of representative or spokesperson. My hope is simply to contact the key person in my country that would.
"stirr it up" as he said :D https://youtu.be/SRyELKGLGag
beside NCS, which is independend community; There's Kulturdepartementet (Culture Department), Kunnskapsdepartementet (Knowledge/Education dep)..But i know Dataforerningen holds good sway in both
both of those state departments hold decisions that could make it happen
[19:45]
godanei put a post about the stuff i'm archiving before net neutrality is over: https://www.reddit.com/r/DataHoarder/comments/7etrwy/things_to_archive_before_net_neutrality_is_over/ [19:58]
ola_norskthings are that bad? :/ [19:59]
astridi dont think so [19:59]
godaneits just in case
better safe then sorry
[19:59]
ola_norskgood point
imagine, ISP beginning to sell "gaming packages"..where your game lags as f*ck if you don't subscribe to it :/
"Sign up for decent connection to EA servers, for only $10 extra a month!"
"For only $2 extra, you could play CS:GO with acceptable latency!"
:)
[20:00]
jrwrStart bundling game season passes on your cable bill [20:17]
ola_norsk_D
"It looks your playing the latest World of Warcraft addon, would you like to play it without lag? Subscribe today!" :D
[20:17]
***jschwart has joined #archiveteam-bs [20:21]
ola_norsk"For only £1, you could be accesing the best handpicked items that Internet Archive has to offer! Kind regards - ISP"
IMO though, just like DRM's and DNS blocking, it won't hold. But incentivice people to break and circumvent it.
[20:22]
astridi feel like this topic is vaguely offtopic, but also not [20:25]
ola_norskastrid: I doubt IA would be high priority connection..
but yeah, i don't think it will be that bad. It has the potential for it though, but i see it as impossible to happen.
[20:26]
astridare you here to work on archiveteam projects, or to chat [20:27]
ola_norsklet me check if my upload is done.. [20:27]
astridwhat're you uploading? [20:28]
ola_norskBig_Cartoon video archive [20:28]
astridah [20:28]
godanehttps://www.theverge.com/2017/11/22/16691794/net-neutrality-fcc-ajit-pai-comcast-block-bittorrent [20:28]
***Asparagir has joined #archiveteam-bs [20:29]
godaneso we need a way to switch protocols mid stream and random but still download files thur bittorrent
make it in impossible for them to filter it out without killing the hole net
[20:29]
ola_norskit's possible to leach trough Tor is it not?
(most likely slow as h*ll though)
[20:39]
***bithippo has quit IRC (My MacBook Air has gone to sleep. ZZZzzz…) [20:40]
schbirid2torrents are really bad to the tor network
if you want anonymous torrenting, use i2p, it is officially supported there and works quite well
[20:44]
ola_norskyeah, but not impossible are they? [20:45]
schbirid2it is possible but you would be a massive dick as it is very stressful traffic [20:45]
ola_norskaye [20:45]
godaneone of my thoughts it make all look like https data without domain to tell where its coming from
i have no idea if can be done though
[20:46]
ola_norski dont know how they detect it to be https, outside of traffic going to port 80 or 8080
maybe it could be obfuscated by some packets containing "genuine" http packets (which the torrent client/tracker would ignore?
enough so that it looks like http connection attempt for a "sniffer detector" i mean?
the trackers and peers on the other end would receive it as garbage though
it would be borderline ddosing perhaps? :/
[20:47]
***bithippo has joined #archiveteam-bs [20:55]
ola_norsk has quit IRC (Veit ikkje du så veit ikkje eg)
pizzaiolo has quit IRC (Read error: Operation timed out)
pizzaiolo has joined #archiveteam-bs
pizzaiolo has quit IRC (Client Quit)
pizzaiolo has joined #archiveteam-bs
ola_norsk has joined #archiveteam-bs
[21:06]
ola_norsk has quit IRC (Leaving) [21:18]
.... (idle for 16mn)
JensRexArh shit. My 1,5 GB scratch disk is making awful clicking noises. [21:34]
***Darkstar has quit IRC (Ping timeout: 260 seconds)
Darkstar has joined #archiveteam-bs
hook54321 sets mode: +o Asparagir
[21:34]
JAAYou have a 1.5 GB HDD? [21:51]
hook54321I would recommend not storing anything important on it. [21:51]
JensRexNothing terribly important on it. SMART has been complaining about it for years.
I'd just be slightly annoyed if my virtual machines were lost, but that's it.
[21:52]
hook54321I haven't gotten a reply back about my archive.org account yet, although thanksgiving is this week, so I guess some people might have multiple days off.
JensRex: You should move the virtual machines off of the 1.5 GB Hard Drive.
[21:53]
astrid1.5 gigabyte? still in active use? [21:54]
JensRexEh, TB. [21:54]
hook54321That's a bit different lol [21:55]
JAAThat makes more sense.
(Although, who ever buys hard disks that aren't a power of two in size??)
[21:55]
godanestill can get a 4tb from bestbuy this week for $80 [21:55]
***BlueMaxim has joined #archiveteam-bs [21:56]
godanethats a black friday special [21:56]
hook54321??
link me?
I might need it if my account doesn't get unlocked lol
[21:56]
JAAgodane, ola_norsk: That is possible. It's fairly easy to set up OpenVPN to look similar to HTTPS traffic. Doesn't mean it's undetectable though, of course. [21:56]
godanehttps://i.imgur.com/x0yTsiK.png [21:57]
JensRex4 TB disks are about 150 USD in Denmark.
That's 150 more than I have to spend on hardware.
[21:57]
JAAYeah, Europe never gets any of those sweet deals from the US... [21:57]
***ola_norsk has joined #archiveteam-bs [21:57]
hook54321That's 8 GB o_O [21:57]
JAA8 TB for $129, for example... [21:57]
godanehttps://www.reddit.com/r/DataHoarder/comments/7e5yc2/black_friday_wd_easystore_8tb_for_12999_valid/ [21:57]
hook54321*TB [21:58]
ola_norskis there a way to revert an issued (deleteion) task id? [21:58]
hook54321I'll be back in a bit [21:58]
***robink has quit IRC (Ping timeout: 506 seconds) [21:58]
ola_norskspecifically task_id=783333772 [21:58]
JensRexMy storage server is still using 2*2 TB disks :/ [21:58]
***RichardG has quit IRC (Read error: Connection reset by peer) [21:58]
JAAJensRex: Similar here, two 2 TB and two 1 TB disks. [21:59]
JensRex4 TB external for backups. [22:00]
ola_norski accentidentally mixed to items, issued an delete all on the latest; But seems like around 200+ of that channels videos are removed since yesterday. [22:00]
JAAI'll probably get some ST8000AS0002 (Seagate Archive 8 TB) drives soonish, but I might wait a bit more until the Exos 5E8 become available, hoping that the older model's price decreases a bit. [22:01]
ola_norskanyway, no worries, just wondering if it would be possible to recall the delete all on an item [22:01]
***robink has joined #archiveteam-bs
ola_norsk has quit IRC (Leaving)
[22:04]
..... (idle for 24mn)
bithippo@JensRex: I have a 1.5TB Seagate spinning disk you can have for freesies if you want. [22:29]
***jschwart has quit IRC (Quit: Konversation terminated!) [22:33]
Kazooh, I found 8TB seagate archive drives for £21/TB [22:34]
JAAYeah, that sounds about right. [22:34]
Kazor Ironwolf Pro's for £27.50/TB [22:35]
JAAThey have been around that price here for several months. [22:35]
KazI haven't been paying attention then [22:35]
***ranavalon has quit IRC (Read error: Connection reset by peer) [22:36]
JAAHere != UK, maybe that's a first for the UK, not sure. [22:37]
***ranavalon has joined #archiveteam-bs
ranavalon has quit IRC (Remote host closed the connection)
ranavalon has joined #archiveteam-bs
[22:38]
........ (idle for 35mn)
ranav has joined #archiveteam-bs
ranav has quit IRC (Remote host closed the connection)
ranav has joined #archiveteam-bs
[23:14]
ranavalon has quit IRC (Read error: Operation timed out) [23:21]
RichardG has joined #archiveteam-bs [23:35]
JensRexbithippo: Thanks, but I think I have something stored away in The Box Of Things. [23:42]
bithippoNo worries, slowly parting out my own Box Of Things. [23:43]
***Asparagir has quit IRC (Asparagir) [23:50]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)