[00:10] *** nwf has quit IRC (Ping timeout: 633 seconds) [00:10] *** nwf has joined #archiveteam [00:21] *** nwf has quit IRC (Ping timeout: 633 seconds) [00:22] *** nwf has joined #archiveteam [00:27] *** vtyl has quit IRC (Read error: Operation timed out) [00:28] *** vtyl has joined #archiveteam [00:31] *** Start has joined #archiveteam [00:49] *** nwf has quit IRC (Ping timeout: 633 seconds) [00:49] *** nwf has joined #archiveteam [01:04] *** JesseW has joined #archiveteam [01:11] *** nwf has quit IRC (Ping timeout: 633 seconds) [01:25] *** nwf has joined #archiveteam [01:50] *** nwf has quit IRC (Ping timeout: 633 seconds) [01:51] *** nwf has joined #archiveteam [01:59] *** RedType has joined #archiveteam [02:24] *** Ravenloft has quit IRC (Read error: Operation timed out) [02:29] *** nwf has quit IRC (Ping timeout: 633 seconds) [02:31] *** zhongfu has quit IRC (Ping timeout: 260 seconds) [02:36] *** beardicus has quit IRC (Read error: Operation timed out) [02:36] *** beardicus has joined #archiveteam [02:36] *** swebb sets mode: +o beardicus [02:38] *** aMunster_ has joined #archiveteam [02:39] *** aMunster_ has quit IRC (Client Quit) [02:40] *** nwf has joined #archiveteam [02:51] *** nwf has quit IRC (Ping timeout: 633 seconds) [03:11] *** nwf has joined #archiveteam [04:30] *** RedType_ has joined #archiveteam [04:32] *** RedType has quit IRC (Ping timeout: 260 seconds) [04:33] *** Sk1d has quit IRC (Ping timeout: 194 seconds) [04:40] *** WinterFox has joined #archiveteam [04:41] *** Sk1d has joined #archiveteam [04:49] *** nwf has quit IRC (Ping timeout: 633 seconds) [04:58] "There's nothing about startup culture that convinces me it's not just a bunch of sociopaths who have learned a new way to run a lemonade stand." [04:58] http://kernelmag.dailydot.com/issue-sections/features-issue-sections/16616/archive-team-saving-the-web/ [05:03] excellent quote [05:05] *** nwf has joined #archiveteam [05:09] they pivoted the lemonade stand and they just made the street sticky [05:15] *** vitzli has joined #archiveteam [05:24] *** BlueMaxim has joined #archiveteam [05:28] *** vitzli has quit IRC (Quit: Leaving) [05:31] *** nwf has quit IRC (Ping timeout: 633 seconds) [05:33] *** nwf has joined #archiveteam [05:38] *** vOYtEC has joined #archiveteam [05:39] I hope narf-archive wasn't anything valuable because it's gone now http://www.fastquake.com/images/screen-finalgradesall-20160504-133249.png [05:39] oh wait that isn't twitter [05:40] https://twitter.com/narfarchive/status/727441735555670016 [05:40] what is narf-archive? [05:41] I don't know, some people in another channel talk about it a fair bit though :p [05:42] I *think* it's 4chan related, so probably nothing important [05:42] Actually meant to post in -bs. I need sleep [05:48] ah [05:49] *** vOYtEC has quit IRC (Read error: Operation timed out) [05:53] *** nwf has quit IRC (Ping timeout: 633 seconds) [06:14] *** nwf has joined #archiveteam [06:28] *** nwf has quit IRC (Ping timeout: 633 seconds) [06:29] *** nwf has joined #archiveteam [06:38] *** metalcamp has joined #archiveteam [06:56] *** schbirid has joined #archiveteam [07:19] *** JesseW has quit IRC (Read error: Operation timed out) [07:35] *** nwf has quit IRC (Ping timeout: 633 seconds) [07:38] *** nwf has joined #archiveteam [07:52] *** atomotic has joined #archiveteam [07:57] *** nwf has quit IRC (Ping timeout: 633 seconds) [07:57] *** nwf has joined #archiveteam [07:58] *** Fake-Nam1 has quit IRC (Read error: Operation timed out) [08:00] *** Fake-Name has joined #archiveteam [08:06] *** Yoshimura has joined #archiveteam [08:07] *** ariscop has quit IRC (Quit: Leaving) [08:24] *** Fletcher_ has quit IRC (Ping timeout: 250 seconds) [08:25] *** nwf has quit IRC (Ping timeout: 633 seconds) [08:25] *** nwf has joined #archiveteam [08:26] *** SilSte has joined #archiveteam [08:26] *** borg has joined #archiveteam [08:30] *** borg has quit IRC (Client Quit) [08:45] *** nwf has quit IRC (Ping timeout: 633 seconds) [08:46] *** nwf has joined #archiveteam [09:01] *** nwf has quit IRC (Ping timeout: 633 seconds) [09:01] *** nwf has joined #archiveteam [09:13] *** ariscop has joined #archiveteam [09:21] *** zhongfu has joined #archiveteam [09:27] *** Fletcher_ has joined #archiveteam [09:53] *** nwf has quit IRC (Ping timeout: 633 seconds) [09:54] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [10:00] *** nwf has joined #archiveteam [10:13] *** nwf has quit IRC (Ping timeout: 633 seconds) [10:13] *** nwf has joined #archiveteam [10:24] *** nwf has quit IRC (Ping timeout: 633 seconds) [10:24] *** nwf has joined #archiveteam [10:26] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [10:33] *** BartoCH has joined #archiveteam [10:38] *** ralphdnak has quit IRC (Ping timeout: 499 seconds) [10:39] *** ralphdnak has joined #archiveteam [10:53] *** nwf has quit IRC (Ping timeout: 633 seconds) [10:54] *** nwf has joined #archiveteam [11:08] *** nwf has quit IRC (Ping timeout: 633 seconds) [11:14] *** nwf has joined #archiveteam [11:21] *** WinterFox has quit IRC (Remote host closed the connection) [11:22] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [11:22] *** WinterFox has joined #archiveteam [11:29] *** nwf has quit IRC (Ping timeout: 633 seconds) [11:30] *** nwf has joined #archiveteam [11:37] *** BlueMaxim has quit IRC (Quit: Leaving) [11:44] *** nwf has quit IRC (Ping timeout: 633 seconds) [11:44] *** nwf has joined #archiveteam [12:08] *** VADemon has joined #archiveteam [12:13] hey guys [12:13] i have a warc file [12:13] i would like to remove from it every reqest+data that matches a filter [12:13] do you know how to do it? [12:21] *** atomotic has joined #archiveteam [12:23] *** RedType_ has quit IRC (Ping timeout: 250 seconds) [12:24] *** RedType has joined #archiveteam [12:40] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [12:40] *** toad2 has joined #archiveteam [12:42] *** toad1 has quit IRC (Read error: Operation timed out) [13:06] *** nwf has quit IRC (Ping timeout: 633 seconds) [13:07] *** nwf has joined #archiveteam [13:09] *** WinterFox has quit IRC (Remote host closed the connection) [13:29] *** atomotic has joined #archiveteam [13:57] *** nwf has quit IRC (Ping timeout: 633 seconds) [13:58] *** nwf has joined #archiveteam [14:04] luckcolor: Likely you want a library that can work with warc format, and filter by regex. [14:04] luckcolor: What filter exactly? ulr? [14:07] Will be back later. Ping me, if noone gets to that, or notthing on the net, might make a tool for that [14:07] *** Yoshimura has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) [14:23] *** Honno has joined #archiveteam [14:28] *** nwf has quit IRC (Ping timeout: 633 seconds) [14:32] *** nwf has joined #archiveteam [14:43] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [14:57] *** metalcamp has joined #archiveteam [15:02] *** Start has quit IRC (Quit: Disconnected.) [15:23] Yoshimura: like grep i suppose so regural expression/url/string [15:28] #archiveteam-bs off topic [15:29] *** nwf has quit IRC (Ping timeout: 633 seconds) [15:29] *** nwf has joined #archiveteam [15:31] *** atomotic has joined #archiveteam [15:47] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:52] *** JesseW has joined #archiveteam [16:02] *** nwf has quit IRC (Ping timeout: 633 seconds) [16:03] *** RichardG_ has joined #archiveteam [16:03] *** nwf has joined #archiveteam [16:04] *** RichardG has quit IRC (Ping timeout: 258 seconds) [16:08] *** JesseW has quit IRC (Ping timeout: 370 seconds) [16:48] *** Start has joined #archiveteam [17:05] *** RichardG_ is now known as RichardG [17:18] *** Emcy has quit IRC (Read error: Connection reset by peer) [17:25] *** nwf has quit IRC (Ping timeout: 633 seconds) [17:25] *** nwf has joined #archiveteam [17:29] *** RedType has quit IRC (Quit: leaving) [17:36] *** RedType has joined #archiveteam [17:38] *** Start has quit IRC (Quit: Disconnected.) [17:49] *** nwf has quit IRC (Ping timeout: 633 seconds) [17:51] *** nwf has joined #archiveteam [18:02] Hey guys [18:03] i would like to propose a project [18:04] so basically http://www.illustris-project.org they did a uiverse simulation using a super computer [18:04] and they release the data [18:04] 252+ tb of data [18:05] it's not that the website is indanger as far as i know [18:05] but like if they ever decide to pull the plug all of that data would be just lost [18:05] for downloading an account is required [18:06] which can be freely made on the website [18:06] they have to manually approve it [18:11] the raw files are in chunks of 1.5 tb each [18:11] Also it's possible to use the api to have more accurate data slices [18:15] *** tomwsmf-a has joined #archiveteam [18:26] *** Start has joined #archiveteam [18:35] *** tomwsmf-a has quit IRC (Ping timeout: 258 seconds) [18:44] 252 tb is too big for something that's not in danger [18:44] can be reconstructed too [18:52] *** nwf has quit IRC (Ping timeout: 633 seconds) [18:57] *** Emcy has joined #archiveteam [18:59] i suppose [19:01] *** Yoshimura has joined #archiveteam [19:04] *** nwf has joined #archiveteam [19:30] *** nwf has quit IRC (Ping timeout: 633 seconds) [19:31] *** nwf has joined #archiveteam [19:37] *** Wuked has joined #archiveteam [19:43] *** nwf has quit IRC (Ping timeout: 633 seconds) [19:43] *** nwf has joined #archiveteam [19:44] *** Start has quit IRC (Quit: Disconnected.) [19:44] *** atomotic has joined #archiveteam [20:01] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [20:15] *** TC01 has quit IRC (Read error: Operation timed out) [20:18] *** TC01 has joined #archiveteam [20:22] *** nwf has quit IRC (Ping timeout: 633 seconds) [20:23] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [20:25] *** ariscop has quit IRC (Read error: Operation timed out) [20:26] It is. it's too big. [20:28] *** schbirid has quit IRC (Quit: Leaving) [20:28] *** Honno has quit IRC (Read error: Operation timed out) [20:28] *** nwf has joined #archiveteam [20:38] *** TC01 has quit IRC (Read error: Operation timed out) [20:48] *** nwf has quit IRC (Ping timeout: 633 seconds) [20:48] *** nwf has joined #archiveteam [20:51] *** dcmorton has quit IRC (Quit: ZNC - http://znc.in) [20:53] luckcolor: Find out where they describe how to rerun the simulation and make sure that and any initial conditions are mirrored. [20:54] *** dcmorton has joined #archiveteam [20:54] *** swebb sets mode: +o dcmorton [21:00] *** nwf has quit IRC (Ping timeout: 633 seconds) [21:01] *** nwf has joined #archiveteam [21:15] *** nwf has quit IRC (Ping timeout: 633 seconds) [21:16] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [21:17] ..then hopefully in 20 years your cell-phone will be able to reproduce the data in 5 mins. [21:21] *** ariscop has joined #archiveteam [21:22] *** BartoCH has joined #archiveteam [21:23] *** Ravenloft has joined #archiveteam [21:30] *** nwf has joined #archiveteam [21:47] *** Wuked has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [22:18] *** tomwsmf-a has joined #archiveteam [22:37] *** nwf has quit IRC (Ping timeout: 633 seconds) [22:37] *** nwf has joined #archiveteam [22:50] *** nwf has quit IRC (Ping timeout: 633 seconds) [22:51] *** nwf has joined #archiveteam [22:53] *** Start has joined #archiveteam [23:06] *** nwf has quit IRC (Ping timeout: 633 seconds) [23:06] *** nwf has joined #archiveteam [23:13] *** Ravenloft has quit IRC () [23:16] *** zino has quit IRC (Read error: Connection reset by peer) [23:17] *** nwf has quit IRC (Ping timeout: 633 seconds) [23:18] *** nwf has joined #archiveteam [23:43] *** nwf has quit IRC (Ping timeout: 633 seconds) [23:47] *** nwf has joined #archiveteam [23:59] *** nwf has quit IRC (Ping timeout: 633 seconds)