#archiveteam-bs 2020-08-13,Thu

↑back Search

Time Nickname Message
01:05 πŸ”— lennier1 has quit IRC (Quit: Going offline, see ya! (www.adiirc.com))
01:11 πŸ”— lennier1 has joined #archiveteam-bs
01:13 πŸ”— Soni has joined #archiveteam-bs
01:13 πŸ”— Soni has quit IRC (Remote host closed the connection)
01:59 πŸ”— Mayonaise has joined #archiveteam-bs
01:59 πŸ”— Mayonaise has quit IRC (Client Quit)
02:27 πŸ”— Mayonaise has joined #archiveteam-bs
03:35 πŸ”— qw3rty_ has quit IRC (Read error: Operation timed out)
03:48 πŸ”— Wingy has joined #archiveteam-bs
05:28 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
06:39 πŸ”— qw3rty has joined #archiveteam-bs
07:22 πŸ”— jshoard has joined #archiveteam-bs
07:50 πŸ”— sirvy has quit IRC (Ping timeout: 615 seconds)
08:26 πŸ”— sirvy has joined #archiveteam-bs
09:54 πŸ”— pie_ SketchCow: where do i learn more about this stuff so im not reinventing everything
10:10 πŸ”— BlueMax has quit IRC (Quit: Leaving)
10:53 πŸ”— VerifiedJ has joined #archiveteam-bs
13:34 πŸ”— ats has quit IRC (Remote host closed the connection)
13:34 πŸ”— ats has joined #archiveteam-bs
13:44 πŸ”— SketchCow What stuff
13:44 πŸ”— SketchCow I guess I'm trying to understand the intent
15:00 πŸ”— Arcorann_ has quit IRC (Read error: Connection reset by peer)
15:23 πŸ”— maxfan8 https://hub.docker.com/search?q=bioinformatics&type=image&page=12
15:23 πŸ”— maxfan8 (From #archiveteam)
15:23 πŸ”— maxfan8 A quick search reveals that there are only 298 images we'd need to backup (for bioinformatics)
15:23 πŸ”— maxfan8 I think this is doable
15:25 πŸ”— SketchCow Correct
15:28 πŸ”— maxfan8 The real question is how do we make sure the docker images are accessible to people trying to reproduce the papers
15:29 πŸ”— maxfan8 Maybe contacting Docker or an annoucement to the bioinformatics community?
15:29 πŸ”— maxfan8 My point is it's not that useful if we archive the docker images and nobody else knows that they're on IA
15:29 πŸ”— OrIdow6 Seeing how scant most of their metadata is, I'll guess there's more than the 298 that come up in that search
15:30 πŸ”— OrIdow6 But yeah, I don't expect there to be a million
15:31 πŸ”— OrIdow6 As I've said before, I'm not too familiar with Docker
15:31 πŸ”— maxfan8 Yeah, just it was just a quick feasibility check (I'm not claiming the # of images we're archiving is gonna be exactly 298)
15:31 πŸ”— maxfan8 Also other fields may have the same problem
15:32 πŸ”— maxfan8 (the audience on HN aren't super representative of all the sciences)
15:32 πŸ”— OrIdow6 And I only have a vauge idea of what "bioinformatics" is
15:33 πŸ”— OrIdow6 If there's a focus on sciences, maybe there's some way to look through links in citations or something like that?
15:34 πŸ”— SketchCow I am WAYYYYY less worried about "people don't know it's on IA" than "we properly get them on IA".
15:34 πŸ”— SketchCow How big is a typical image.
15:40 πŸ”— maxfan8 Pretty big actually (usually in the single-digit GB range)
17:17 πŸ”— JAA Soo Docker Hub... This won't be fun or easy.
17:17 πŸ”— JAA What we should *really* do is archive all images with all past versions.
17:18 πŸ”— JAA For the example of bioinformatics, a change in the base system can affect the outcome, so it's important to preserve the base image that was used at the time.
17:18 πŸ”— JAA The problem is: as far as I know, there is no way to find past versions.
17:20 πŸ”— JAA The website and I believe also the API only returns what each tag for an image currently points to. But you need to find the image IDs used in the past.
17:20 πŸ”— JAA For 'official' images, there is a repository on GitHub that has the necessary metadata, but that's not the case for any unofficial images.
17:20 πŸ”— JAA I haven't looked into it deeper, but this won't be easy to tackle.
17:22 πŸ”— JAA As for storage, in the ideal world, we'd have on IA item for each layer I guess and one for each image (which would basically just be metadata listing the layers). That's probably not feasible though.
18:25 πŸ”— DLoader_ has joined #archiveteam-bs
18:26 πŸ”— superkuh Inevitable consequence of centralization, re: docker problems now.
18:36 πŸ”— DLoader has quit IRC (Ping timeout: 745 seconds)
18:36 πŸ”— DLoader_ is now known as DLoader
18:39 πŸ”— Stilett0 has joined #archiveteam-bs
18:46 πŸ”— Stiletto has quit IRC (Read error: Operation timed out)
19:08 πŸ”— maxfan8 JAA: Good point
19:55 πŸ”— Smiley has quit IRC (Remote host closed the connection)
19:55 πŸ”— Dallas has quit IRC (Read error: Connection reset by peer)
19:55 πŸ”— Smiley has joined #archiveteam-bs
19:58 πŸ”— ats_ has joined #archiveteam-bs
19:58 πŸ”— Hoolootwo has joined #archiveteam-bs
20:03 πŸ”— SJon____ has joined #archiveteam-bs
20:04 πŸ”— SJon___ has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— ats has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— lunik1 has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— SJon____ is now known as SJon___
20:04 πŸ”— Hooloovoo has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— Tugboat has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— Jens has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— SJon___ has quit IRC (Killed (se.hub (Nick collision (new))))
20:04 πŸ”— pew has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— OrIdow6 has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— arkiver has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— robbi5 has quit IRC (Ping timeout: 265 seconds)
20:04 πŸ”— purplebot has quit IRC (Ping timeout: 265 seconds)
20:13 πŸ”— i0npulse has quit IRC (Ping timeout: 265 seconds)
20:18 πŸ”— OrIdow6 has joined #archiveteam-bs
20:18 πŸ”— Tugboat has joined #archiveteam-bs
20:19 πŸ”— i0npulse has joined #archiveteam-bs
20:20 πŸ”— Jens has joined #archiveteam-bs
20:25 πŸ”— Ajay1 has joined #archiveteam-bs
21:02 πŸ”— bleb has joined #archiveteam-bs
21:05 πŸ”— Ajay1 has quit IRC (se.hub irc.underworld.no)
21:05 πŸ”— i0npulse has quit IRC (se.hub irc.underworld.no)
21:05 πŸ”— OrIdow6 has quit IRC (se.hub irc.underworld.no)
21:05 πŸ”— Tugboat has quit IRC (se.hub irc.underworld.no)
21:05 πŸ”— Jens has quit IRC (se.hub irc.underworld.no)
21:05 πŸ”— cm has quit IRC (se.hub irc.underworld.no)
21:11 πŸ”— fredgido_ has joined #archiveteam-bs
21:15 πŸ”— fredgido has quit IRC (Read error: Operation timed out)
21:18 πŸ”— JAA Oh yeah, haven't seen it mentioned yet, but: 'As the world’s largest repository of container images, Docker Hub stores more than 15PB of data. After detailed analysis of the container images stored on Docker Hub, we found that 4.5PB of the data have not been pushed or pulled within 6 months or longer.' (from https://www.docker.com/pricing/retentionfaq)
21:19 πŸ”— JAA So 4.5 PB of data at immediate risk, 15+ PB of data in total
21:20 πŸ”— bleb is now known as cm
21:41 πŸ”— OrIdow6 has joined #archiveteam-bs
21:41 πŸ”— Tugboat has joined #archiveteam-bs
21:42 πŸ”— i0npulse has joined #archiveteam-bs
21:44 πŸ”— arkiver has joined #archiveteam-bs
21:44 πŸ”— Ajay19 has joined #archiveteam-bs
21:45 πŸ”— svchfoo3 sets mode: +o arkiver
21:48 πŸ”— Ajay19 is now known as Ajay1
21:48 πŸ”— Ajay18 has joined #archiveteam-bs
21:50 πŸ”— Ajay18 has quit IRC (Client Quit)
21:51 πŸ”— Ajay19 has joined #archiveteam-bs
21:53 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
21:53 πŸ”— Ajay19 is now known as Ajay1
21:53 πŸ”— Ajay14 has joined #archiveteam-bs
21:55 πŸ”— Ajay14 has quit IRC (Client Quit)
21:56 πŸ”— Ajay19 has joined #archiveteam-bs
21:57 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
21:57 πŸ”— Ajay19 is now known as Ajay1
21:58 πŸ”— Ajay19 has joined #archiveteam-bs
22:00 πŸ”— Ajay19 has quit IRC (Client Quit)
22:01 πŸ”— Ajay16 has joined #archiveteam-bs
22:02 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:02 πŸ”— Ajay16 is now known as Ajay1
22:03 πŸ”— Ajay10 has joined #archiveteam-bs
22:03 πŸ”— Jens has joined #archiveteam-bs
22:05 πŸ”— Ajay10 has quit IRC (Client Quit)
22:05 πŸ”— Ajay19 has joined #archiveteam-bs
22:07 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:07 πŸ”— Ajay19 is now known as Ajay1
22:08 πŸ”— Ajay19 has joined #archiveteam-bs
22:10 πŸ”— Ajay19 has quit IRC (Client Quit)
22:10 πŸ”— Ajay17 has joined #archiveteam-bs
22:12 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:12 πŸ”— Ajay17 is now known as Ajay1
22:12 πŸ”— Ajay15 has joined #archiveteam-bs
22:13 πŸ”— Kaz feels like a 'just tell us which 4.5PB to grab' type job
22:14 πŸ”— Ajay15 has quit IRC (Client Quit)
22:15 πŸ”— Ajay11 has joined #archiveteam-bs
22:17 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:17 πŸ”— Ajay11 is now known as Ajay1
22:17 πŸ”— Ajay13 has joined #archiveteam-bs
22:18 πŸ”— JAA Ajay1, Ajay13: Fix your connection please.
22:19 πŸ”— Ajay13 has quit IRC (Client Quit)
22:20 πŸ”— Ajay13 has joined #archiveteam-bs
22:21 πŸ”— JAA Kaz: Yeah, and I doubt they'll tell us, unfortunately. We'll basically have to find a way to discover old builds' image IDs, and I'm not sure if that's possible.
22:22 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:22 πŸ”— Ajay13 is now known as Ajay1
22:22 πŸ”— JAA Also, maybe we can just periodically download every image to reset their counts and prevent deletion altogether. :-P
22:22 πŸ”— JAA Since their criterion according to the FAQ is 'no pulls in the past 6 months'.
22:22 πŸ”— Ajay15 has joined #archiveteam-bs
22:24 πŸ”— Ajay15 has quit IRC (Client Quit)
22:25 πŸ”— Ajay13 has joined #archiveteam-bs
22:26 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:26 πŸ”— Ajay13 is now known as Ajay1
22:27 πŸ”— Ajay18 has joined #archiveteam-bs
22:29 πŸ”— Ajay18 has quit IRC (Client Quit)
22:30 πŸ”— Ajay14 has joined #archiveteam-bs
22:31 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:31 πŸ”— Ajay14 is now known as Ajay1
22:32 πŸ”— Ajay16 has joined #archiveteam-bs
22:36 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:36 πŸ”— Ajay16 is now known as Ajay1
22:37 πŸ”— Ajay19 has joined #archiveteam-bs
22:39 πŸ”— Ajay19 has quit IRC (Client Quit)
22:39 πŸ”— Ajay13 has joined #archiveteam-bs
22:41 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:41 πŸ”— Ajay13 is now known as Ajay1
22:41 πŸ”— Ajay12 has joined #archiveteam-bs
22:42 πŸ”— HP_Archiv has joined #archiveteam-bs
22:43 πŸ”— Ajay12 has quit IRC (Client Quit)
22:44 πŸ”— voltagex has joined #archiveteam-bs
22:44 πŸ”— Ajay15 has joined #archiveteam-bs
22:46 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:46 πŸ”— Ajay15 is now known as Ajay1
22:46 πŸ”— Ajay14 has joined #archiveteam-bs
22:48 πŸ”— Ajay14 has quit IRC (Client Quit)
22:49 πŸ”— Ajay15 has joined #archiveteam-bs
22:51 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:51 πŸ”— Ajay15 is now known as Ajay1
22:51 πŸ”— Ajay13 has joined #archiveteam-bs
22:53 πŸ”— Ajay13 has quit IRC (Client Quit)
22:54 πŸ”— Ajay18 has joined #archiveteam-bs
22:55 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
22:55 πŸ”— Ajay18 is now known as Ajay1
22:56 πŸ”— sirvy has quit IRC (Ping timeout: 615 seconds)
22:56 πŸ”— Ajay15 has joined #archiveteam-bs
22:58 πŸ”— Ajay15 has quit IRC (Client Quit)
22:58 πŸ”— Ajay17 has joined #archiveteam-bs
22:59 πŸ”— Mateon1 has quit IRC (Read error: Operation timed out)
23:00 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:00 πŸ”— Ajay17 is now known as Ajay1
23:01 πŸ”— Ajay14 has joined #archiveteam-bs
23:03 πŸ”— Ajay14 has quit IRC (Client Quit)
23:03 πŸ”— Ajay16 has joined #archiveteam-bs
23:05 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:05 πŸ”— Ajay16 is now known as Ajay1
23:06 πŸ”— Ajay16 has joined #archiveteam-bs
23:08 πŸ”— Ajay16 has quit IRC (Client Quit)
23:08 πŸ”— Ajay16 has joined #archiveteam-bs
23:10 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:10 πŸ”— Ajay16 is now known as Ajay1
23:10 πŸ”— Ajay11 has joined #archiveteam-bs
23:12 πŸ”— sirvy has joined #archiveteam-bs
23:12 πŸ”— Ajay11 has quit IRC (Client Quit)
23:13 πŸ”— Ajay12 has joined #archiveteam-bs
23:13 πŸ”— HP_Archiv has quit IRC (Quit: Leaving)
23:15 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:15 πŸ”— Ajay12 is now known as Ajay1
23:15 πŸ”— Ajay16 has joined #archiveteam-bs
23:17 πŸ”— Ajay16 has quit IRC (Client Quit)
23:18 πŸ”— Ajay11 has joined #archiveteam-bs
23:19 πŸ”— Mateon1 has joined #archiveteam-bs
23:20 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:20 πŸ”— Ajay11 is now known as Ajay1
23:20 πŸ”— Ajay17 has joined #archiveteam-bs
23:22 πŸ”— Ajay17 has quit IRC (Client Quit)
23:23 πŸ”— Ajay16 has joined #archiveteam-bs
23:24 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:24 πŸ”— Ajay16 is now known as Ajay1
23:25 πŸ”— Ajay18 has joined #archiveteam-bs
23:27 πŸ”— Ajay18 has quit IRC (Client Quit)
23:27 πŸ”— Ajay19 has joined #archiveteam-bs
23:29 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:29 πŸ”— Ajay19 is now known as Ajay1
23:30 πŸ”— Ajay12 has joined #archiveteam-bs
23:31 πŸ”— VerifiedJ has quit IRC (Quit: Leaving)
23:31 πŸ”— fredgido has joined #archiveteam-bs
23:32 πŸ”— Ajay12 has quit IRC (Client Quit)
23:32 πŸ”— Ajay16 has joined #archiveteam-bs
23:34 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:34 πŸ”— Ajay16 is now known as Ajay1
23:35 πŸ”— Ajay12 has joined #archiveteam-bs
23:35 πŸ”— fredgido_ has quit IRC (Read error: Operation timed out)
23:37 πŸ”— Ajay12 has quit IRC (Client Quit)
23:37 πŸ”— Ajay15 has joined #archiveteam-bs
23:38 πŸ”— lennier2 has joined #archiveteam-bs
23:39 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:39 πŸ”— Ajay15 is now known as Ajay1
23:39 πŸ”— Ajay18 has joined #archiveteam-bs
23:41 πŸ”— lennier1 has quit IRC (Ping timeout: 272 seconds)
23:41 πŸ”— lennier2 is now known as lennier1
23:41 πŸ”— Ajay18 has quit IRC (Client Quit)
23:42 πŸ”— Ajay19 has joined #archiveteam-bs
23:44 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:44 πŸ”— Ajay19 is now known as Ajay1
23:44 πŸ”— Ajay16 has joined #archiveteam-bs
23:46 πŸ”— Ajay16 has quit IRC (Client Quit)
23:47 πŸ”— Ajay13 has joined #archiveteam-bs
23:49 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:49 πŸ”— Ajay13 is now known as Ajay1
23:49 πŸ”— Ajay11 has joined #archiveteam-bs
23:51 πŸ”— Ajay11 has quit IRC (Client Quit)
23:52 πŸ”— Ajay11 has joined #archiveteam-bs
23:53 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:53 πŸ”— Ajay11 is now known as Ajay1
23:54 πŸ”— Arcorann_ has joined #archiveteam-bs
23:54 πŸ”— Ajay15 has joined #archiveteam-bs
23:56 πŸ”— Ajay15 has quit IRC (Client Quit)
23:56 πŸ”— Ajay18 has joined #archiveteam-bs
23:58 πŸ”— Ajay1 has quit IRC (Ping timeout: 265 seconds)
23:58 πŸ”— Ajay18 is now known as Ajay1
23:59 πŸ”— Ajay19 has joined #archiveteam-bs

irclogger-viewer