#archiveteam-bs 2017-07-19,Wed

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***pizzaiolo has quit IRC (Ping timeout: 506 seconds)
pizzaiolo has joined #archiveteam-bs
pizzaiolo has quit IRC (Client Quit)
pizzaiolo has joined #archiveteam-bs
[00:00]
....... (idle for 33mn)
jrwrxmc: I have thought about making a botnet that is if you are an op in here you can control it, have it join channels and op you in ever channel you and the bots are in upon request
basic commands like !join #channel and !opme -- with a small whitelist of people who have static hostnames in case of failure to auto op
have 3 bots connected to server that is connected to a different hub
[00:36]
***pnJay has quit IRC (Leaving) [00:42]
BlueMaxim has joined #archiveteam-bs [00:55]
dxrt- has joined #archiveteam-bs [01:06]
NstkVdwn has quit IRC (Quit: Leaving)
th1x has joined #archiveteam-bs
[01:14]
.... (idle for 19mn)
JensRex has joined #archiveteam-bs
username1 has joined #archiveteam-bs
j08nY has quit IRC (Quit: Leaving)
schbirid2 has quit IRC (Read error: Operation timed out)
[01:36]
......... (idle for 40mn)
wm_ has quit IRC (Ping timeout: 260 seconds)
wm_ has joined #archiveteam-bs
Fusl has quit IRC (Ping timeout: 250 seconds)
[02:19]
Fusl has joined #archiveteam-bs [02:34]
........ (idle for 39mn)
qw3rty has joined #archiveteam-bs [03:13]
qw3rty2 has quit IRC (Read error: Operation timed out) [03:18]
pizzaiolo has quit IRC (Quit: pizzaiolo) [03:30]
.... (idle for 17mn)
robink has quit IRC (Ping timeout: 246 seconds)
robink has joined #archiveteam-bs
[03:47]
icedice has quit IRC (Ping timeout: 245 seconds) [03:59]
jspiros has quit IRC (Ping timeout: 492 seconds) [04:07]
.......... (idle for 47mn)
Sk1d has quit IRC (Ping timeout: 194 seconds) [04:54]
Sk1d has joined #archiveteam-bs [05:00]
jspiros has joined #archiveteam-bs
th1x has quit IRC (Read error: Operation timed out)
[05:14]
.... (idle for 17mn)
th1x has joined #archiveteam-bs [05:35]
.............. (idle for 1h6mn)
username1 has quit IRC (Quit: Leaving)
schbirid has joined #archiveteam-bs
Honno has joined #archiveteam-bs
[06:41]
kimmer has quit IRC (Read error: Connection reset by peer)
kimmer has joined #archiveteam-bs
[06:51]
............ (idle for 55mn)
kimmer has quit IRC (Read error: Connection reset by peer) [07:47]
kimmer22 has joined #archiveteam-bs [08:01]
kimmer2 has quit IRC (Ping timeout: 632 seconds) [08:09]
kimmer2 has joined #archiveteam-bs [08:15]
th1x has quit IRC (Read error: Operation timed out)
kimmer22 has quit IRC (Ping timeout: 633 seconds)
[08:20]
........... (idle for 53mn)
icedice has joined #archiveteam-bs [09:16]
schbirid2 has joined #archiveteam-bs
schbirid has quit IRC (Read error: Operation timed out)
[09:22]
.... (idle for 18mn)
Honno has quit IRC (Read error: Operation timed out) [09:44]
...... (idle for 26mn)
icedice has quit IRC (Quit: Leaving) [10:10]
kimmer22 has joined #archiveteam-bs [10:21]
kimmer2 has quit IRC (Read error: Operation timed out) [10:27]
kimmer2 has joined #archiveteam-bs
kimmer23 has joined #archiveteam-bs
icedice has joined #archiveteam-bs
kimmer24 has joined #archiveteam-bs
kimmer22 has quit IRC (Ping timeout: 633 seconds)
kimmer2 has quit IRC (Ping timeout: 633 seconds)
kimmer has joined #archiveteam-bs
[10:32]
username1 has joined #archiveteam-bs
kimmer2 has joined #archiveteam-bs
kimmer23 has quit IRC (Ping timeout: 633 seconds)
schbirid2 has quit IRC (Read error: Operation timed out)
kimmer24 has quit IRC (Ping timeout: 633 seconds)
icedice has quit IRC (Quit: Leaving)
[10:51]
kimmer22 has joined #archiveteam-bs [11:09]
kimmer2 has quit IRC (Read error: Operation timed out)
kimmer22 has quit IRC (Read error: Connection reset by peer)
kimmer2 has joined #archiveteam-bs
[11:18]
.... (idle for 17mn)
kimmer22 has joined #archiveteam-bs
j08nY has joined #archiveteam-bs
quantum has joined #archiveteam-bs
kimmer2 has quit IRC (Ping timeout: 633 seconds)
[11:37]
....... (idle for 31mn)
pizzaiolo has joined #archiveteam-bs [12:18]
pnJay has joined #archiveteam-bs
kimmer22 has quit IRC (Ping timeout: 633 seconds)
pizzaiolo has quit IRC (pizzaiolo)
pizzaiolo has joined #archiveteam-bs
quantum has quit IRC (Quit: Page closed)
[12:23]
dxrt sets mode: +o dxrt- [12:40]
th1x has joined #archiveteam-bs [12:46]
......... (idle for 42mn)
BlueMaxim has quit IRC (Quit: Leaving) [13:28]
schbirid2 has joined #archiveteam-bs
username1 has quit IRC (Read error: Operation timed out)
bwn has quit IRC (Ping timeout: 268 seconds)
[13:35]
NstkVdwn has joined #archiveteam-bs [13:49]
pizzaiolo has quit IRC (Quit: pizzaiolo)
pizzaiolo has joined #archiveteam-bs
pizzaiolo has quit IRC (Client Quit)
pizzaiolo has joined #archiveteam-bs
[13:57]
......... (idle for 40mn)
TC01 has quit IRC (Read error: Operation timed out)
TC01 has joined #archiveteam-bs
[14:41]
..... (idle for 20mn)
bwn has joined #archiveteam-bs [15:02]
...... (idle for 28mn)
bwn has quit IRC (Read error: Operation timed out)
bwn_ has joined #archiveteam-bs
bwn_ is now known as bwn
[15:30]
....... (idle for 34mn)
NstkVdwn has quit IRC (Ping timeout: 506 seconds) [16:04]
NstkVdwn has joined #archiveteam-bs [16:12]
............ (idle for 58mn)
tsuckow has joined #archiveteam-bs [17:10]
jrwrtsuckow: what you having issues with
arm is a PITA, but if you are just dedicating a machine to it
just run the commands inside the dockerfile by hand
[17:14]
tsuckowjrwr: At the moment building wget-lua. Nothing blocking, just keep finding packages I need to install which takes forever. [17:16]
jrwrya
the docker file has the apt-get line that covers most of it
[17:16]
tsuckowI switched to raspbian base image because it is armv6 but it apparently doesn't include some packages by default like the other one [17:17]
***fallenoak has joined #archiveteam-bs [17:17]
tsuckowand the prebuilt wget-lua isn't compatible
Some day I will finish making a backup utility for C.H.I.P. and I could start using those.
[17:17]
jrwrI just got done making a new warriorvm
the current ones we use are kind of old
like 2010 old
[17:21]
tsuckowIf it's not broke take it apart and find out why. [17:22]
jrwroh but it is
the SSL engine in that thing is so old
some modern websites just fail to work
[17:22]
tsuckowYa, I noticed python bitching [17:23]
jrwrit uses Alpine Linux and Docker
it just uses the docker version (so it can stay up to date) on boot
60MB
[17:24]
tsuckownearly a third smaller. [17:26]
jrwrit downloads a 300MB Docker image
so it comes out in the wash
[17:27]
tsuckowIt looked like you also upped the disk to 100GB. Do the projects ever really approach the 60GB? [17:27]
jrwrsome can
I've seen some come back before
but if its not used, its not used
https://archive.org/download/AT-Warrior100G/Warrior-100G.ova Your welcome to try it, its "Unsupported" but ill help where I can
[17:27]
tsuckowOne of these days I need to look at just running the docker image on windows 10
Though the point of running it on the pi is so I can turn the desktop off.
Vm seems to work fine
[17:32]
jrwrNice :) [17:35]
tsuckowIf you wanted to be as minimal as the old one you could disable usb and reduce video memory to 1MB
But i doubt it matters
[17:37]
jrwrSave all of 10MB of ram
on boot the base os uses 40MB
then once the warrior boots its 100MB
so
if you switch to TTY2 its root:warrior
htop is installed
[17:38]
tsuckowYou must know what yuo are doing if you put htop init [17:41]
jrwrlol [17:42]
xmcor, we don't know what it's doing :) [17:43]
jrwrlol
its a base alpine install with a boot.sh as TTY1 that just runs docker and checks if docker is running
I am going to write some docs on the edits I did (/etc/inittab) and apk add docker htop nano
[17:43]
tsuckowAnyone know why isc-dhcp-client is in the docker container? [17:51]
.... (idle for 15mn)
mundus201Is there a faster way to archive a site rather than just doing wget -m? [18:06]
schbirid2wpull -m with concurrency and all that shit [18:14]
mundus201oh didn't know wpull did concurrency [18:14]
***phuzion has joined #archiveteam-bs [18:19]
TheLovina has joined #archiveteam-bs [18:30]
Odd0002 has quit IRC (Remote host closed the connection) [18:37]
........ (idle for 36mn)
Asparagir has joined #archiveteam-bs [19:13]
NstkVdwn has quit IRC (Quit: Leaving)
Odd0002 has joined #archiveteam-bs
username1 has joined #archiveteam-bs
[19:21]
schbirid2 has quit IRC (Read error: Operation timed out) [19:34]
.... (idle for 16mn)
kimmer has quit IRC (Read error: Connection reset by peer)
kimmer has joined #archiveteam-bs
username1 has quit IRC (Quit: Leaving)
Odd0002 has quit IRC (Remote host closed the connection)
[19:50]
Honno has joined #archiveteam-bs [20:00]
.... (idle for 16mn)
ja0Hai has joined #archiveteam-bs [20:16]
th1x has quit IRC (Leaving)
th1x has joined #archiveteam-bs
[20:24]
schbirid has joined #archiveteam-bs [20:34]
..... (idle for 21mn)
schbirid2 has joined #archiveteam-bs
schbirid has quit IRC (Read error: Operation timed out)
schbirid2 has quit IRC (Remote host closed the connection)
[20:55]
.... (idle for 19mn)
Sk1d has quit IRC (Ping timeout: 250 seconds)
pnJay has quit IRC (Quit: Leaving)
Sk1d has joined #archiveteam-bs
Sk1d has quit IRC (Connection Closed)
Sk1d has joined #archiveteam-bs
[21:17]
hook54321Fyi, FamilySearch is discontinuing microfilm/fiche distribution on September 1, 2017. If you want something, send it to a Family History Center by August 31. [21:21]
AsparagirAsparagir waves to fellow genealogy nerd [21:21]
hook54321hook54321 waves back [21:21]
AsparagirAlso, if you know any genealogical/archival records that you want to see released to the public, send my org a heads up: www.ReclaimTheRecords.org [21:22]
***ndiddy has quit IRC (Read error: Operation timed out) [21:22]
AsparagirWe use Freedom of Information laws to force government archives, libraries, and agencies to hand over copies of data they have.
And if they don't comply, we sue. :-)
Lots of success in the past two years, millions of records uploaded to the Internet Archive. And we just filed for non-proft status with the IRS so my little project is no going big-time.
[21:22]
hook54321They're going to apparantly have it all digitized by 2020. But I'm not sure how or if copyright will affect what they have publicly available online. [21:23]
Asparagir*now [21:23]
hook54321What kind of records would FOI laws require them to hand over? [21:24]
AsparagirIt will, because some countries (Ukraine and Poland, for example) need to have their contracts renegotiated to allow the microfilm images to go online, not just be on film. Some of their books were filmed in the 1980's before online images existed.
Re: FOI -- In the past two years, I got the NYC marriage index 1908-1929 from the NYC Municipal Archives, then followed that up with getting the rest of teh record set 1930-1995 from the New York City Clerk's Office.
Neither had EVER been available online before, nor on FamilySearch microfilm.
Because NYC are jerks about not allowing access.
Speaking of which, NYC is another example where they have b/m/d certificates on microfilm, but they are refusing to allow FamilySearch to put the images online! So FS went ahead and did a text-transcription of the images. Which is not quite as good.
[21:24]
hook54321Unfortunately (at least for me) it costs $7.50 for shipping and handling to get an item delivered to a Family History Center. [21:27]
AsparagirYeah.
I also got the New York State (minus NYC) death index 1880-1956 through an FOI fight over the past two years and just finished uploading it all to the IA. No one had that! NY State!
It's amazing what FOI laws and the willingness to fight can do. :-)
Got lots of other stuff too.
And have a lawsuit pending in MIssouri for the first ever copy of their birth index (post-1910) and death index (post-1966). Very very basic index, not actual certificates, but they're being jerks and don't want to hand them over. Even though they sell that same data! Or maybe because they sell thaht same data, and don't want to lose the revenue stream.
[21:27]
hook54321hmm... If someone lives in Utah, could they just walk into the Family History Library and start making digital copies of the NYC d/m//d certificates? [21:28]
AsparagirYup! From the films.
But NYC is refusing to grant the right to put the images online.
[21:29]
hook54321Why doesn't NYC let them put it online?
I mean, there's only so much they can do to keep it offline...
[21:29]
AsparagirBecause they want to be the sole source of this data. They're data hoarders. They make money selling copies.
It's like Gollum with the ring MY PRECCCCIOUS
So I have no compunction about suing them (twice now!) to get data from them.
But I haven't gone after any actual certificates yet.
Getting an index under FOI laws seems much easier.
Getting a certificate will be a harder fight.
[21:29]
fallenoakAsparagir: Are you a lawyer?
Or is it easy enough to sue under FOIA that you don't need one?
[21:31]
hook54321Couldn't someone get digital copies at the Family History Library in Utah and then host it on a server in another country? [21:31]
AsparagirNot a lawyer, but my parents wish I were. :-)
I was pre-law in college.
[21:31]
fallenoakTo my lay mind, I'd have thought suing the government would be expensive and time consuming haha [21:31]
AsparagirI had/have attorneys for all three lawsuits. [21:32]
yipdwSuing State And Federal Government For Fun And Profit [21:32]
fallenoakAwesome [21:32]
hook54321What would happen if someone uploaded the NYC b/m/d certificates to libgen? [21:32]
Asparagir#1 was against the NYC Municipal Archives' parent agency DORIS. I won settlement and got all records, but did not win my attorneys fees. Luckily they were not bad at all, because I used a public interest law firm that likes to stick up for the litle guys.
Lawuist #2 was against the New York City Clerk's Office. I won a settlement again, won all the records but had to agree to take slightly redacted copies (which was okay). And that time I did win attorneys fees!
[21:32]
hook54321why did they redact some stuff? [21:34]
AsparagirThe thing is, for STATE FOI requests, most states allow you to potentailly win your attorneys fees, but only five state mandate that you will definitely get reimbursed if you win the records: NJ, California, and three more I don't remember right now. [21:34]
***pnJay has joined #archiveteam-bs [21:35]
hook54321Also, what are we going to do about this? : http://www.thedailybeast.com/cia-plans-to-destroy-some-of-its-old-leak-files [21:35]
AsparagirThey redacted the bride and groom's dates of birth. They claimed that part was too invasive. I asked if they could just leave in the year of birth and cut the month/day, but they said no. It's an unsettled part of NY FOIL (their FOI law) whether dates of birth need to be redacted or not. So I could have fought them in court over that issue. But I decided to just take the rest of the data instead.
They also cut the upper bound of the years off at 1995. That's because my FOIL request had asked them for a copy of the marriage index. But it turns out that starting in 1996, there isn't any separate index for NYC marriages, the data was "born digital" in database form right at the city clerk's office window.
So I need to file a new FOI request later this year asking for "a redcated section of the NYC marriage database" to get the rest of the years, 1996-2016. It's an index but I can't call it an index in my request, what a pain.
[21:35]
Froggingwhy do we want all this stuff online anyway? [21:39]
AsparagirBecause genealogists like having open records?
Because public records belong to the public and we're tired of getting gouged on $22 fees per record search.
[21:39]
hook54321I have a 503 MB text file titled "voters" [21:40]
Froggingidk why birth certificates are public records anyway [21:40]
AsparagirThe certificates usually are not. The INDEX to the sometimes is.
*them
You don't want to enable identity fraud.
[21:40]
hook54321Does anyone want this text document? [21:40]
AsparagirWhat's in it? [21:41]
Froggingexactly that's what I was thinking [21:41]
yipdwASCII art
of "voters" repeated 300 million times
[21:41]
hook54321Notepad is having issues opening [21:41]
Frogginga bunch of porn in base64 [21:41]
hook54321it
Probably because it's 503 MB
[21:41]
Frogginguse less
:p
[21:41]
hook54321It's something that was leaked like a year a ago. [21:42]
Froggingor head or tail or cat or anything that doesn't try and load the whole file into memory at once [21:42]
yipdwthen it's probably https://www.forbes.com/sites/thomasbrewster/2015/12/28/us-voter-database-leak/#491dd53c5b98
so, nothx
[21:43]
Froggingand display it in a win32 text control [21:43]
hook54321Top of the text document: "Registration Date","Original Registration Date","Party","Phone","Mailing Address","Mailing city, state zip","County ID","Precinct","House Number","House Number Suffix","Direction Prefix","Street","Direction Suffix","Street Type","Unit Type","Unit Number","City","Zip","DOB","Congressional","State House","State Senate","State Schoolboard","Local Schoolboard","County Council","City Council" [21:44]
AsparagirFYI, lots of states (like NY) allow you to get copies of their voter databases under their FOI laws! They just want you to swear you won't use the addresses to sign people up for junk mail. The laws vary a lot from state to state. [21:44]
hook54321this is also on the top line:
"11/6/1990","11/5/1991","11/3/1992","11/2/1993","11/8/1994","5/23/1995","9/12/1995","10/3/1995","11/7/1995","6/25/1996","8/6/1996","11/5/1996","2/4/1997","5/6/1997","8/1/1997","10/7/1997","11/4/1997","6/23/1998","11/3/1998","5/4/1999","8/3/1999","10/5/1999","11/2/1999","5/2/2000","6/27/2000","11/7/2000","2/6/2001","10/2/2001","11/6/2001","6/25/2002","11/5/2002","2/4/2003","8/5/2003","10/7/2003","11/4/2003","5/4/2004","
6/22/2004","8/3/2004","11/2/2004","10/4/2005","11/8/2005","6/27/2006","11/7/2006","6/26/2007","9/11/2007","11/6/2007
I'm pretty sure this is just people in Utah...
Yeah. Just people in Utah.
No idea what it all means though
oh. You can open it in LibreOffice Calc.
[21:45]
yipdwthe unexplored cyberpunk scenario
"I have all this data, I don't know what to do with it"
"I just download stuff people send me"
[21:47]
JAAyipdw: I think that's a different data set though. The one you linked was a ~300 GB MongoDB IIRC. [21:48]
hook54321LibreOffice now just has a white screen. [21:48]
yipdwJAA: oh ok, well in that case it's comforting to know that database will have all sorts of integrity problems [21:48]
hook54321Actually, apparantly I downloaded this in 2013... O_o [21:49]
xmclol [21:49]
yipdwNever Forget
To Rag On MongoDB
[21:49]
hook54321At least according to the Created and Accessed dates
https://raidforums.com/Thread-Utah-Voter-Database-Leaked-Download
The date on that is 2017 though
[21:50]
JAAHow many records does that file have, hook54321? [21:54]
hook54321I'll tell you if I can get LibreOffice to open it [21:55]
JAAwc -l ftw [21:55]
hook54321? [21:55]
JAAOr are you on Windows? [21:55]
***ndiddy has joined #archiveteam-bs [21:56]
hook54321yup. It's on a computer in a Family History Center. [21:58]
JAAI see. No clue then. [21:58]
hook54321It might have been from this site: http://utvoters.com/ [22:00]
JAAYep, that sounds right. 528252166 bytes, created on 2013-06-21 20:42. [22:06]
hook54321wait, how did you? [22:07]
JAAAs mentioned on the page you linked, this is still downloadable from https://www.indymedia.org.uk/en/2014/02/515559.html [22:07]
hook54321Yup. That's the one. Same exact size. [22:09]
JAA:-) [22:09]
hook54321Is there anything we can do about this? http://www.thedailybeast.com/cia-plans-to-destroy-some-of-its-old-leak-files [22:12]
dashcloudyou can try FOIAing them, but it will take a while and there's no guarantee [22:21]
hook54321Asparagir ^ ? [22:23]
AsparagirEmma Best (@NatSecGeek on Twitter) and Nate Jones (@NSANate on twitter) would be your best people to ask. [22:25]
hook54321Will contacting them get me put on some list? [22:25]
Asparagir*shrugs*
They're already trying to stop the destruction: https://twitter.com/NSANate/status/887300618217500672
Looks like NARA is taking comments from the public at request.schedule@nara.gov .
[22:26]
hook54321They would have tried to FOIAing them if they could have, right? [22:27]
***mhazinsk has quit IRC (Read error: Operation timed out)
mhazinsk has joined #archiveteam-bs
[22:28]
AsparagirI guess? Why not ask them? [22:29]
hook54321list [22:29]
AsparagirWhose list? [22:29]
hook54321**the** list [22:29]
JAAYou're already on that one. [22:29]
hook54321why? [22:29]
AsparagirWe're allll on someone's list. Some of us were born there! [22:29]
JAABecause you mentioned the list. [22:30]
hook54321:/ [22:30]
***kimmer2 has joined #archiveteam-bs [22:30]
AsparagirFub fact: Emma Best, under her previous name Mike Best, is one of the people who has uploaded the most files to the Internet Archive. She's our kind of person -- likes open records, saving history, and so on. Because of her work, CIA finally put their CREST database online earlier this year.
*Fun
[22:31]
hook54321CREST database? [22:31]
AsparagirIt's super-cool! One sec, let me dig up the link...
https://www.muckrock.com/news/archives/2017/jan/17/cias-declassified-database-now-online/
Direct link: https://www.cia.gov/library/readingroom/collection/crest-25-year-program-archive/
Emma has already started uploading/archiving/backing up the whole database to the Internet Archive.
I think it's probably still rendering/deriving.
Just checked the Catalog page and she's still at it. 1.2 million uploads and counting: https://archive.org/details/@the_mike_best
[22:32]
hook54321I just followed Emma Best on Twitter, right after I did that Twitter recommended that I follow Chelsea Manning 🤔 [22:35]
AsparagirHahaha. [22:35]
***omglolbah has joined #archiveteam-bs [22:38]
........ (idle for 36mn)
dashcloudAsparagir: that answers so much- I was wondering why I couldn't Mike Best around, because I was going to point out that effort [23:14]
***Atom has joined #archiveteam-bs [23:19]
hook54321dashcloud: "couldn't __?__ Mike Best around" [23:27]
dashcloudcouldn't find the name online [23:28]
hook54321oh
https://archive.org/details/@the_mike_best
[23:28]
dashcloudthanks [23:30]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)