[00:10] So this is for writing large amounts of data to optical disks? [00:12] most likely to grab tons of discs [00:22] Oh I see, it's mostly a feeder built around an all-purpose optical drive. [00:23] *** BlueMaxim has joined #archiveteam-bs [01:07] *** j08nY has quit IRC (Quit: Leaving) [01:14] *** Sk1d has quit IRC (Ping timeout: 250 seconds) [01:21] *** Sk1d has joined #archiveteam-bs [01:31] *** DFJustin has quit IRC (Remote host closed the connection) [01:37] *** DFJustin has joined #archiveteam-bs [01:37] *** DFJustin has quit IRC (Remote host closed the connection) [01:37] *** DFJustin has joined #archiveteam-bs [01:37] *** swebb sets mode: +o DFJustin [01:39] *** tuluu_ has quit IRC (Read error: Operation timed out) [01:46] *** braxxox_ has joined #archiveteam-bs [02:11] *** Petri152 has quit IRC (Read error: Operation timed out) [02:34] *** Petri152 has joined #archiveteam-bs [04:18] *** Sk1d has quit IRC (Ping timeout: 194 seconds) [04:24] *** Sk1d has joined #archiveteam-bs [04:30] Hey all, I'm curious if folks over here have played with using ipfs for archiving purposes. [04:30] Or what general interest level in something like that would be? [04:31] braxxox_: it has been brought up in the past [04:32] IPFS is not remote storage, just a way to access eomething on someone's machine [04:33] Any movement on it? @jrwr, yeah not remote storage but a great way to keep a lot of stuff safe in a lot of different places. And increase bandwidth. [04:33] This is true and has been discussed for internetarchive.bak, but the current system they have is what the community is going to stick with for now [04:34] Yeah of course :) [04:34] also IPFS has a bigger flaw that will prevent it from replacing the web for the most part [04:34] Oh yeah? What's that? [04:34] Its static, its very hard to have a webapp in IPFS [04:35] Very few users, haha [04:35] the internet is moving away from static data and more to a community [04:35] being limited to static content is both a curse *and* a blessing [04:35] if I could build a forum based community on IPFS I would like to see it being done [04:35] Because dynamic content is shit for archiving [04:35] It is and is not [04:36] IPFS + directory watchers I'm sure it could be done [04:36] A key limitation with IPFS is that, from what I understand, it requires someone to "pin" the data in order for it to actually stick around [04:36] Yep [04:36] If an organization like the IA provided public pins of archive-worthy data everyone would abuse that [04:36] true. There are just so many more moving parts with dynamic content. Plus likely nothing will run your setup in a few decades. [04:37] I would say javascript will still be around in 2050 [04:37] 2100, hrm, maybe [04:37] yes the language but not the web browser apis of 2017 [04:37] Well, we know JS sticks around through 2035 at least ;) https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript [04:38] @MrRadar if IA pinned content you think folks would only leech? [04:38] At least enough to ruin the bandwidth? [04:38] Overall, I don't see the web changing much in how its run in the next 50 years, Tech will change, sure but the World Wide Web will still be here [04:39] Its like IPv4, that shit should of died in 2000 [04:39] Yeah I don't realistically see HTTP going anywhere fast. Everything is built on it. [04:39] The internet is like running a social network [04:39] lol true. Nobody wants to read ipv6 addys out loud haha [04:39] trying to start a new internet has a chicken and egg issue [04:40] Look at the issues IPv6 is having to even get started [04:40] braxxox_: It sucks, but a /64 or /48 is not too bad [04:40] I'm not saying ipfs will replace the web. I definitately don't think it will. but I thnk it has some amazing affordances and the distributed nature and content addressing are both super plusses for archiving. [04:41] ] [04:43] I've been playing with ipfs over the past couple of days and I've got some motivation to work on setting up an archive of some sort and just seeing what happens. If nobody downloads then so be it, that would speak for itself. But any content that y'all might think would be of particular interest for such an endevor? [04:49] It works great as a kind of CDN [04:50] *** braxxox_ has quit IRC (Ping timeout: 268 seconds) [04:54] *** braxxox_ has joined #archiveteam-bs [04:55] *** zyphlar has joined #archiveteam-bs [04:56] oops just lost my internet connection. Not sure if anybody responded to that ^. If not carry on and sry 4 tha noise. [04:57] IPFS is really just a P2P CDN [04:58] Anyway, since with archiving the data is mostly about saving the data and not serving it [04:59] presenting the data is up to the future people who will want the data. [05:00] IPFS Use case is nice for like a bittorrent but for websites [05:01] but as with turrents [05:01] if the seeds dry up, the torrent is gone [05:03] *** RedType has quit IRC (Read error: Operation timed out) [05:05] *** RedType has joined #archiveteam-bs [05:07] yeah that is the real danger. Especially with something as new as ipfs. Who knows how long it will be maintained? [05:07] or actively developed. [05:08] That said I don't think it is a bad idea to try. Even if at a somewhat philosophical level. Or to go through the motions of exercising the data. [05:08] between IA and textfiles [05:09] Also the best way I can think of to store ia.bak is QR codes on paper [05:09] that would last forever [05:09] Because ultimately, if everyone the web worked more like ipfs we would not have so much of the linkrot that currently crushes the web. [05:09] Ish [05:09] between IA and textfiles? [05:09] if the underlying storage goes kaboom it and another source is trying to link to it [05:09] it stills rotten [05:10] Internet Archive and Text-Files are the two biggest stores of archival computer data that I know of [05:12] Of course. But if the content it distributed and the act of accessing the data serves to dissiminate it the long tail of machines that still have it a decade later could still be very long. [05:13] This is true [05:13] Much better than our current situation. Content is on 1+ machine and all it takes is for the sysadmin to keel over and its gone for good. [05:14] overall [05:14] nothing is stopping you from trying to get people on board to do backing up things to IPFS [05:15] Yeah I hope we see a shift towards more distributed content store in the future. I think we would all be better off for it :) [05:15] Ok yeah, I'm gunna try it. I'll ping back here if I land anywhere of interest. Thanks for the mussings about :) [05:15] braxxox_: check out #internetarchive.bak [05:15] will do, thx [05:15] they are the nerds that handle the biggest project that AT covers [05:16] sweet [05:16] oh! [05:16] http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK/ipfs_implementation [05:16] looks like the IPFS devs have something in the works [05:17] oh no way! [05:17] Oh no, last modified March 2015 :( [05:17] But they at least seem like at one point they were open to it. [05:18] I don't wan't to introduce noise is all. They are up to some awesome stuff and I am sure they have themselves pretty busy. [05:18] I will bring it up over there though. IPFS has matured a lot since 2015. [06:26] *** superkuh has quit IRC (Read error: Operation timed out) [06:27] *** superkuh has joined #archiveteam-bs [06:39] *** jrwr has quit IRC (Read error: Operation timed out) [06:40] *** superkuh has quit IRC (Read error: Operation timed out) [06:41] *** will has quit IRC (Read error: Operation timed out) [06:41] *** dashcloud has quit IRC (Read error: Connection reset by peer) [06:43] *** dashcloud has joined #archiveteam-bs [06:44] *** jrwr has joined #archiveteam-bs [06:45] *** will has joined #archiveteam-bs [06:47] greenie: Makes sense. [06:49] *** superkuh has joined #archiveteam-bs [06:57] *** jtn2 has quit IRC (Read error: Operation timed out) [06:58] *** jtn2 has joined #archiveteam-bs [07:04] *** zyphlar has quit IRC (Quit: Connection closed for inactivity) [07:30] *** jtn2 has quit IRC (Read error: Operation timed out) [07:35] *** Honno_ has joined #archiveteam-bs [07:37] *** jtn2 has joined #archiveteam-bs [07:38] *** Honno has quit IRC (Ping timeout: 370 seconds) [08:12] *** BlueMaxim has quit IRC (Read error: Operation timed out) [08:14] *** SHODAN_UI has joined #archiveteam-bs [08:15] *** Jonison has joined #archiveteam-bs [08:28] *** SHODAN_UI has quit IRC (Quit: zzz) [08:28] *** SHODAN_UI has joined #archiveteam-bs [09:12] *** braxxox_ has quit IRC (Ping timeout: 272 seconds) [09:36] *** SHODAN_UI has quit IRC (Remote host closed the connection) [09:45] *** Honno__ has joined #archiveteam-bs [09:51] *** Honno_ has quit IRC (Ping timeout: 370 seconds) [09:58] *** Jonison has quit IRC (Quit: Leaving) [10:05] *** j08nY has joined #archiveteam-bs [11:15] *** Fletcher has quit IRC (Read error: Operation timed out) [11:17] *** SHODAN_UI has joined #archiveteam-bs [11:55] SketchCow: you getting the Fox Network Premiere of Wolf [11:55] recorded back on 1997-10-28 [11:56] this a 125 minute movie crammed to be 2 hours with commercials [12:27] *** Kenshin has quit IRC (Read error: Operation timed out) [12:29] *** Kenshin has joined #archiveteam-bs [13:20] *** mgrytbak has joined #archiveteam-bs [13:44] *** robink has quit IRC (Read error: Connection reset by peer) [13:45] *** robink has joined #archiveteam-bs [14:12] *** Honno has joined #archiveteam-bs [14:13] *** Honno__ has quit IRC (Ping timeout: 370 seconds) [14:13] Steam Forums is starting to be uploaded. [14:33] obligatory note that IPFS is absolutely not designed for archival purposes [14:33] it's a distribution mechanism, not a storage mechanism [14:44] SketchCow: i have ~330gb of appnet sitting about that hasn't been uploaded yet. Can I push it over to FOS if your stuff is still running over there [14:46] Yes [14:47] FOS is in goodish shape at the moment. [14:47] will start sending it over now, cheers [14:58] The CD-ROM autoloader thing is working amazingly. [14:58] Except when the CD goes to shit but that'd be my life anyway [15:11] *** Frogging has quit IRC (Ping timeout: 268 seconds) [15:54] *** Frogging has joined #archiveteam-bs [15:56] *** dashcloud has quit IRC (Read error: Operation timed out) [16:21] what makes me really sad is [16:21] http://ipetcompanion.com/ [16:23] hours of groundbreaking virtual cat interactions unrecorded https://voatpic.me/image/6CB5BB35A2 [16:31] Lord_Nigh: I'd say re-send your email to info@ with a note about it having maybe gotten lost in the cracks. [16:32] *** godane has quit IRC (Ping timeout: 250 seconds) [16:32] *** pikhq has quit IRC (Ping timeout: 250 seconds) [16:33] *** pikhq has joined #archiveteam-bs [16:46] *** godane has joined #archiveteam-bs [16:49] *** schbirid has joined #archiveteam-bs [17:10] *** SHODAN_UI has quit IRC (Remote host closed the connection) [17:14] oops, ran into a non-javascript based quiz thingie on a website. wpull seems to have gotten all the combinations of all the quizzes quite well. 4 million files, 300GB D: [17:15] hahahah [17:16] wget would have OOMd early in [17:44] https://archive.org/details/tednelsonjunkmail [18:05] *** wp494 has quit IRC (Read error: Operation timed out) [18:06] *** wp494 has joined #archiveteam-bs [18:09] can i make wpull only write to a warc, not also store all the files? [18:10] Yes, use --delete-after [18:14] *** icedice has joined #archiveteam-bs [18:19] awesome [18:20] what does "after" refer to? immediately after downloading and parsing? [18:23] Yes, as far as I can tell [19:10] *** SHODAN_UI has joined #archiveteam-bs [19:53] *** powerArch has quit IRC (Read error: Operation timed out) [21:03] *** schbirid has quit IRC (Quit: Leaving) [21:06] *** powerArch has joined #archiveteam-bs [21:09] so i got the hbo airing of this: https://en.wikipedia.org/wiki/Garth:_Live_from_Central_Park [21:09] It was later edited and released on VHS, followed by its 2006 DVD release, retitled Live in Central Park, in Brooks' The Entertainer video collection. [21:10] since it said edit i think they may have change something [21:13] *** icedice has quit IRC (Quit: Leaving) [21:31] *** SHODAN_UI has quit IRC (Remote host closed the connection) [21:37] *** dashcloud has joined #archiveteam-bs [22:28] *** j08nY has quit IRC (Read error: Operation timed out) [22:33] *** j08nY has joined #archiveteam-bs [22:33] *** Stiletto has quit IRC (Read error: Operation timed out) [22:49] *** Ravenloft has quit IRC (Read error: Connection reset by peer) [22:51] *** Ravenloft has joined #archiveteam-bs [22:54] SketchCow: you going to Defcon this year, Always wanted to go a talk of yours [22:56] huh, didn't realise googlebots use phantomjs [22:56] I would think they would convert over to chrome [22:56] they do run chrome [22:57] just noticed while searching, maybe the pages haven't been updated in a while https://s.kurt.gg/64F8EYM.png [22:57] Interesting [22:59] hm. i read somewhere in one of their blogs years ago that they run headless chrome [23:00] the amount of CPU that must take [23:09] *** ZexaronS has joined #archiveteam-bs [23:45] *** BlueMaxim has joined #archiveteam-bs