#archiveteam 2015-08-03,Mon

↑back Search

Time Nickname Message
00:00 πŸ”— Balrog-34 I wasn't sure what to expect, really
00:22 πŸ”— SimpBrain has quit IRC (Ping timeout: 258 seconds)
00:33 πŸ”— aschmitz has joined #archiveteam
00:36 πŸ”— philpem has quit IRC (Ping timeout: 252 seconds)
00:41 πŸ”— Balrog-34 has quit IRC (Quit: I'm offensive and I find this Jewish. ~ Decanter)
00:43 πŸ”— JesseW has joined #archiveteam
01:21 πŸ”— xk_id has joined #archiveteam
01:28 πŸ”— JesseW has quit IRC (Quit: Leaving.)
01:40 πŸ”— godane has quit IRC (Quit: Leaving.)
01:51 πŸ”— JesseW has joined #archiveteam
02:01 πŸ”— ruukasu has joined #archiveteam
02:02 πŸ”— ploopkazo has quit IRC (Ping timeout: 265 seconds)
02:07 πŸ”— xk_id has quit IRC (Remote host closed the connection)
02:38 πŸ”— xk_id has joined #archiveteam
02:44 πŸ”— Infreq has quit IRC (Quit: 始めましょう!)
02:44 πŸ”— Infreq has joined #archiveteam
02:49 πŸ”— xk_id has quit IRC (Remote host closed the connection)
02:49 πŸ”— xk_id has joined #archiveteam
03:00 πŸ”— vitzli has joined #archiveteam
03:05 πŸ”— xk_id_ has joined #archiveteam
03:05 πŸ”— xk_id has quit IRC (Read error: Connection reset by peer)
03:19 πŸ”— SketchCow Blorp
03:19 πŸ”— SketchCow I get a reading assignment and a pithy eaction requirement
03:19 πŸ”— SketchCow One moment
03:20 πŸ”— kyan has joined #archiveteam
03:22 πŸ”— SketchCow 19:32 < Balrog-34> I already checked and it's not illegal where Internet Archive is based or in large chunks of the uS
03:23 πŸ”— SketchCow We are not going to archive that shit hee.
03:23 πŸ”— SketchCow here.
03:23 πŸ”— xmc correct
03:23 πŸ”— SketchCow Also, wow, that guy went into -bs territory 5 pages in
03:25 πŸ”— godane has joined #archiveteam
03:25 πŸ”— SketchCow Also, LOL on Paheal is "old internet history". Registered in 2006. "Wild West Days of the Internet" were long, long gone.
03:26 πŸ”— SketchCow This is like wandering into a 1990s music discussion where people talk about the "wild days"
03:26 πŸ”— SketchCow xmc covered most of this nicely.
03:26 πŸ”— SketchCow I don't need you guys sitting in a cell next to Jared
03:26 πŸ”— SketchCow most will break like little girls
03:27 πŸ”— SketchCow Some will become little girls
03:39 πŸ”— SketchCow Some are already little girls
03:42 πŸ”— mistym has joined #archiveteam
03:42 πŸ”— JesseW has quit IRC (Quit: Leaving.)
03:42 πŸ”— SketchCow sets mode: +oooo arkiver BlueMaxim chfoo closure
03:42 πŸ”— SketchCow sets mode: +ooo Coderjoe db48x dcmorton
04:08 πŸ”— pikhq Lol, 2006 the "wild west" days? That's far removed from even the wild west days of the web.
04:11 πŸ”— SketchCow LOLd ahead of you
04:15 πŸ”— kyan has quit IRC (Read error: Operation timed out)
04:16 πŸ”— kyan has joined #archiveteam
04:17 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
04:19 πŸ”— dashcloud has joined #archiveteam
04:19 πŸ”— kyan_ has joined #archiveteam
04:21 πŸ”— kyan has quit IRC (Ping timeout: 258 seconds)
04:23 πŸ”— JesseW has joined #archiveteam
04:25 πŸ”— aaaaaaaaa has quit IRC (Leaving)
04:28 πŸ”— Sue_ has quit IRC (Read error: Operation timed out)
04:57 πŸ”— mistym_ has joined #archiveteam
05:03 πŸ”— mistym has quit IRC (Read error: Operation timed out)
05:06 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
05:10 πŸ”— useretail has quit IRC (Read error: Operation timed out)
05:11 πŸ”— dashcloud has joined #archiveteam
05:12 πŸ”— Dark_Star has quit IRC (Read error: Connection reset by peer)
05:12 πŸ”— garyrh has quit IRC (Read error: Operation timed out)
05:13 πŸ”— will has quit IRC (Read error: Operation timed out)
05:13 πŸ”— will has joined #archiveteam
05:13 πŸ”— Jonimus has quit IRC (Read error: Connection reset by peer)
05:14 πŸ”— Dark_Star has joined #archiveteam
05:15 πŸ”— maz_ has quit IRC (Read error: Operation timed out)
05:15 πŸ”— sep332 has quit IRC (Read error: Operation timed out)
05:15 πŸ”— aMunster_ has quit IRC (Read error: Operation timed out)
05:15 πŸ”— marvinw has quit IRC (Read error: Operation timed out)
05:18 πŸ”— vegbrasil has quit IRC (Ping timeout: 600 seconds)
05:18 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
05:18 πŸ”— dashcloud has joined #archiveteam
05:20 πŸ”— logchfoo_ starts logging #archiveteam at Mon Aug 03 05:20:22 2015
05:20 πŸ”— logchfoo_ has joined #archiveteam
05:21 πŸ”— sep332 has joined #archiveteam
05:21 πŸ”— aMunster has joined #archiveteam
05:23 πŸ”— maz_ has joined #archiveteam
05:36 πŸ”— SadDM has joined #archiveteam
06:00 πŸ”— vitzli has quit IRC (Quit: Leaving)
06:05 πŸ”— Dark_Star has quit IRC (Read error: No route to host)
06:05 πŸ”— Dark_Star has joined #archiveteam
06:08 πŸ”— Jonimus has joined #archiveteam
06:10 πŸ”— mistym has joined #archiveteam
06:13 πŸ”— signius blimey just woke up & read the scroll-back from last night
06:13 πŸ”— garyrh has joined #archiveteam
06:16 πŸ”— Dark_Star has quit IRC (Read error: Operation timed out)
06:16 πŸ”— mistym_ has quit IRC (Read error: Operation timed out)
06:21 πŸ”— JesseW has quit IRC (Quit: Leaving.)
06:36 πŸ”— Dark_Star has joined #archiveteam
06:37 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
06:37 πŸ”— dashcloud has joined #archiveteam
06:42 πŸ”— mistym has quit IRC (Remote host closed the connection)
06:42 πŸ”— mistym has joined #archiveteam
06:42 πŸ”— mistym has quit IRC (Remote host closed the connection)
06:53 πŸ”— SadDM has quit IRC (Ping timeout: 483 seconds)
07:10 πŸ”— SadDM has joined #archiveteam
07:27 πŸ”— schbirid has joined #archiveteam
07:43 πŸ”— mistym has joined #archiveteam
08:05 πŸ”— mistym has quit IRC (Read error: Operation timed out)
08:36 πŸ”— arkiver SketchCow: we are currently pushing 300 Mbit of xfire to IA
08:37 πŸ”— arkiver But we need it faster, to clear space for the blip.tv project
08:37 πŸ”— arkiver https://monitor.archive.org/weathermap/weathermap.html shows that both incoming connections are using less then 25% of their bandwidth
08:38 πŸ”— arkiver Do you think we can speed this up to 1 gbit or more?
08:46 πŸ”— xk_id_ has quit IRC (Remote host closed the connection)
08:52 πŸ”— Guest25 has joined #archiveteam
08:52 πŸ”— Guest25 is now known as expr_
09:03 πŸ”— dugo_ has quit IRC (Read error: Operation timed out)
09:03 πŸ”— dugo has joined #archiveteam
09:03 πŸ”— superkuh has quit IRC (Ping timeout: 258 seconds)
09:07 πŸ”— db48x has quit IRC (Ping timeout: 258 seconds)
09:07 πŸ”— sunnymilk has quit IRC (Ping timeout: 258 seconds)
09:08 πŸ”— kyan_ has quit IRC (hub.efnet.us irc.Prison.NET)
09:08 πŸ”— nertzy2 has quit IRC (hub.efnet.us irc.Prison.NET)
09:08 πŸ”— wyatt8740 has quit IRC (hub.efnet.us irc.Prison.NET)
09:08 πŸ”— Stiletto has quit IRC (hub.efnet.us irc.Prison.NET)
09:08 πŸ”— SketchCow has quit IRC (hub.efnet.us irc.Prison.NET)
09:11 πŸ”— Scabwinge has joined #archiveteam
09:11 πŸ”— superkuh_ has joined #archiveteam
09:11 πŸ”— Scabwinge Anyone want to archive this glorious mess? :D http://www.sarkeesianeffect.com
09:11 πŸ”— Scabwinge I would have asked @archivebot but don't want to risk gators piling on me.
09:14 πŸ”— wyatt8740 has joined #archiveteam
09:14 πŸ”— sunnymilk has joined #archiveteam
09:14 πŸ”— kyan has joined #archiveteam
09:14 πŸ”— nertzy2 has joined #archiveteam
09:14 πŸ”— Stiletto has joined #archiveteam
09:14 πŸ”— Scabwinge has left
09:15 πŸ”— nertzy2 has quit IRC (Ping timeout: 258 seconds)
09:18 πŸ”— nertzy has joined #archiveteam
09:33 πŸ”— SadDM has quit IRC (Ping timeout: 483 seconds)
09:49 πŸ”— SadDM has joined #archiveteam
10:00 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
10:00 πŸ”— khaoohs_ has joined #archiveteam
10:06 πŸ”— khaoohs has quit IRC (Ping timeout: 483 seconds)
10:07 πŸ”— dashcloud has joined #archiveteam
10:20 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
10:25 πŸ”— dashcloud has joined #archiveteam
10:43 πŸ”— mistym has joined #archiveteam
10:45 πŸ”— dan- has quit IRC (Ping timeout: 483 seconds)
10:49 πŸ”— dan- has joined #archiveteam
10:53 πŸ”— mistym has quit IRC (Read error: Operation timed out)
10:57 πŸ”— chfoo has quit IRC (Ping timeout: 306 seconds)
11:13 πŸ”— nmnn has joined #archiveteam
11:23 πŸ”— Ungstein has joined #archiveteam
11:49 πŸ”— trs80 has quit IRC (Ping timeout: 186 seconds)
11:50 πŸ”— Ungstein has quit IRC (Ping timeout: 265 seconds)
11:54 πŸ”— Ungstein has joined #archiveteam
11:55 πŸ”— arkiver Scabwinge: being archived.
12:05 πŸ”— vitzli has joined #archiveteam
12:22 πŸ”— khaoohs_ is now known as khaoohs
12:36 πŸ”— Stilett0 has joined #archiveteam
12:37 πŸ”— Stiletto has quit IRC (Ping timeout: 258 seconds)
12:40 πŸ”— expr_ has quit IRC (Read error: Connection reset by peer)
12:45 πŸ”— S[h]O[r]T has joined #archiveteam
12:49 πŸ”— Guest25 has joined #archiveteam
12:52 πŸ”— Guest25 has quit IRC (Client Quit)
12:52 πŸ”— brayden_ has quit IRC (Read error: Connection reset by peer)
12:53 πŸ”— brayden has joined #archiveteam
13:00 πŸ”— kyan has quit IRC (Quit: This computer has gone to sleep)
13:05 πŸ”— Jonimus has quit IRC (Read error: Operation timed out)
13:19 πŸ”— tomwsmf-a has joined #archiveteam
13:21 πŸ”— nmnn has quit IRC (Ping timeout: 483 seconds)
13:25 πŸ”— Jonimus has joined #archiveteam
13:29 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
13:32 πŸ”— mistym has joined #archiveteam
13:32 πŸ”— dashcloud has joined #archiveteam
13:40 πŸ”— SketchCow has joined #archiveteam
13:40 πŸ”— SketchCow arkiver: I can't affect bandwidth. Period.
13:40 πŸ”— mistym has quit IRC (Read error: Operation timed out)
13:41 πŸ”— arkiver SketchCow: what I mean is we can start more concurrent upload to IA, which means our speed will go up.
13:41 πŸ”— arkiver But can IA handle 1 gbit if we are able to reach that?
13:42 πŸ”— SketchCow Yes
13:46 πŸ”— trs80 has joined #archiveteam
13:46 πŸ”— arkiver ok!
13:47 πŸ”— arkiver yipdw: can you start more concurrent upload for xfire so we can reach 1 gbit?
13:51 πŸ”— BlueMaxim has quit IRC (Quit: Leaving)
13:52 πŸ”— brayden has quit IRC (Read error: Connection reset by peer)
13:56 πŸ”— lytv has quit IRC (Read error: Operation timed out)
14:00 πŸ”— lytv has joined #archiveteam
14:02 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
14:05 πŸ”— dashcloud has joined #archiveteam
14:20 πŸ”— phuzion has joined #archiveteam
14:24 πŸ”— arkiver We are now at 10T for blip!
14:27 πŸ”— ersi ding-ding!
14:37 πŸ”— SketchCow Is this being uploaded into archive.org?
14:38 πŸ”— SketchCow I'd really like to see what we're actually saving.
14:38 πŸ”— SketchCow When it's 10tb and before it's 110tb
14:38 πŸ”— arkiver it's first uploaded to Kenshin
14:39 πŸ”— arkiver I'll give you a more detailed report here of what we are exactly saving in a bit
14:39 πŸ”— SketchCow OK.
14:39 πŸ”— SketchCow Let's not let this get away from us
14:39 πŸ”— Kenshin SketchCow: blip seems to be another twitch purgatory attempt
14:39 πŸ”— Kenshin the amount is going to be insane
14:40 πŸ”— SketchCow Yes, I'm not 100% on board.
14:40 πŸ”— SketchCow For archive.org, anyway
14:40 πŸ”— SketchCow I mean, fine, save it
14:48 πŸ”— mistym has joined #archiveteam
14:50 πŸ”— vitzli has quit IRC (Ping timeout: 483 seconds)
14:54 πŸ”— SketchCow My number one concern is that Blip TV is mostly just mirroring a bunch of YouTube garbage.
15:05 πŸ”— mistym has quit IRC (Remote host closed the connection)
15:16 πŸ”— arkiver So what are we exactly downloading for blip.tv
15:16 πŸ”— SketchCow I don't know!
15:16 πŸ”— arkiver I was going to explain that in detail...
15:16 πŸ”— SketchCow I mean, I am REALLY SORRY to be the classic wander-in-on-last-inning arm-waving manager, by the way.
15:17 πŸ”— SketchCow I finally looked at Blip and didn't realize it was now a desparate also-ran youtube-like shitshow
15:17 πŸ”— SketchCow No wonder it's going down
15:18 πŸ”— arkiver They do have 500 million views/month
15:18 πŸ”— arkiver So for every video we are starting from the rss page: http://blip.tv/rss/flash/7205592
15:18 πŸ”— JesseW has joined #archiveteam
15:19 πŸ”— arkiver From all videos on that page we are only downloading the source video, in this case that is http://blip.tv/file/get/NostalgiaCritic-NCPlanetOfTheApes237.mp4 (see page source)
15:20 πŸ”— arkiver Then we are downloading the video page http://blip.tv/nostalgiacritic/nc-planet-of-the-apes-7205592
15:20 πŸ”— arkiver Which has some files needed for the playback of videos:
15:20 πŸ”— arkiver The video page leads us to http://blip.tv/players/standard?no_wrap=1&id=7205592&autoplay=true&onsite=true&&no_postroll=true&data_url=http://blip.tv/players/xplayer&utm_campaign=&swf_location=http://a.blip.tv/scripts/flash/stratos.swf&referrer=http://blip.tv&site_url=http://blip.tv
15:21 πŸ”— arkiver Which leads us to http://a.blip.tv/scripts/flash/stratos.swf?file=http://blip.tv/rss/flash/7205592&autostart=true&showinfo=false&onsite=true&nopostroll=true&noendcap=true&showsharebutton=false&removebrandlink=false&page=episode&skin=BlipClassic&frontcolor=0x999999&lightcolor=0xAAAAAA&basecolor=0x1E1E1E&backcolor=0x1E1E1E&floatcontrols=true&fixedcontrols=true&largeplaybutton=true&controlsalpha=.8&autohideidle=6000&utm_campaign=&adprovider=auditude
15:21 πŸ”— arkiver 7323&referrer=http%3A%2F%2Fblip.tv&destinationtag=blip_tv
15:21 πŸ”— arkiver And that swf page leads us to http://blip.tv/file/get/NostalgiaCritic-NCPlanetOfTheApes401.m4v?showplayer=2014093037100220150422135039&referrer=http://blip.tv&mask=11&skin=flashvars&view=url
15:22 πŸ”— arkiver where mask=11 can also be mask=0 or mask=7 or others
15:23 πŸ”— arkiver The /file/get/ url then has a link to the low quality video, which is also downloaded
15:23 πŸ”— arkiver In this case http%3A%2F%2Fj41.video2.blip.tv%2F5520014255207%2FNostalgiaCritic-NCPlanetOfTheApes401.m4v%3Fir%3D96428%26sr%3D2334
15:23 πŸ”— arkiver http://j41.video2.blip.tv/5520014255207/NostalgiaCritic-NCPlanetOfTheApes401.m4v?ir=96428&sr=2334
15:24 πŸ”— arkiver And then we are also downloading all images from the video and other files except the video files and the static files
15:24 πŸ”— arkiver So that is what we download
15:25 πŸ”— arkiver So in theory it should also be possible to actually playback the video from http://blip.tv/nostalgiacritic/nc-planet-of-the-apes-7205592 in the wayback machine
15:25 πŸ”— arkiver But that requires some changes in the wayback machine specific for blip.tv urls
15:28 πŸ”— SketchCow Document this.
15:28 πŸ”— SketchCow Document it HARD.
15:28 πŸ”— SketchCow Nerd 100% out on it.
15:31 πŸ”— lof has joined #archiveteam
15:34 πŸ”— arkiver Yes, I'll document it
15:35 πŸ”— arkiver I guess we'll then send that to wayback's coders and see if playback in the wayback machine is possible
15:35 πŸ”— arkiver That would be awesome
15:40 πŸ”— lof hello is there a way to stay up-to-date with the archiving efforts around gitorious.org?
15:43 πŸ”— JesseW has quit IRC (Quit: Leaving.)
15:43 πŸ”— SketchCow There is a chance we will end up having to pay for a bunch of hard drives and put this work onto those drives and set them aside.
15:44 πŸ”— SketchCow Anyway, carry on
15:45 πŸ”— SketchCow I'm working on a fundraiser of a general sort for December anyway
15:45 πŸ”— Guest25 has joined #archiveteam
15:45 πŸ”— SketchCow As much as AT people want to help with that, great, but I don't want it a requirement or to draft people
15:45 πŸ”— Guest25 is now known as expr_
15:46 πŸ”— arkiver Have you spoken with Brewster Kahle or the person who makes storage decisions at IA about blip and the upcoming big projects? (sourceforge, google code)
16:03 πŸ”— ruukasu has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in)
16:13 πŸ”— chfoo has joined #archiveteam
16:19 πŸ”— SketchCow Nooooooooooo
16:19 πŸ”— SketchCow No Noo nooooey no
16:19 πŸ”— SketchCow Let's not wake the beast
16:22 πŸ”— ploopkazo has joined #archiveteam
16:25 πŸ”— expr_ Are the plans to archive Frontback still active? Even though they're not shutting down, it's possible the data could end up inaccessible depending on what this "new team" decide to do
16:33 πŸ”— expr_ has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…)
16:33 πŸ”— yipdw arkiver: so, even if view counts aren't available, you can still send out a "what do you want us to get" request
16:34 πŸ”— yipdw there are people here who have reach into social networks
16:34 πŸ”— yipdw it's not perfect coverage but literally nothing we have done has that
16:35 πŸ”— yipdw this will help avoid clogging people's stuff with terabytes of video, most of which is now on YouTube or Vimeo
16:43 πŸ”— Guest25 has joined #archiveteam
16:45 πŸ”— godane has quit IRC (Quit: Leaving.)
16:46 πŸ”— lof has quit IRC (Quit: Page closed)
16:46 πŸ”— tomwsmf-a has quit IRC (Read error: Operation timed out)
16:52 πŸ”— philpem has joined #archiveteam
16:56 πŸ”— arkiver yipdw: we can do that. Though I'd rather get everything
16:57 πŸ”— arkiver but if that's really too much because sourceforge and google code are also coming up I understand that
16:57 πŸ”— yipdw we can do that when 110 TB is about as trivial as 110 GB
16:57 πŸ”— yipdw better get cracking on quantum-event-level storage
16:58 πŸ”— yipdw the cast-line-into-social-network approach is also not mutually exclusive with a continuing grab, it just gives you sorting criteria
16:59 πŸ”— yipdw SketchCow: can you leverage the twitters to start that sort of query?
16:59 πŸ”— arkiver SketchCow: should we actually do the social media thing or just grab everything?
17:00 πŸ”— nmnn has joined #archiveteam
17:02 πŸ”— yipdw it's not an either-or thing, you can do both and use that to inform the best use of the 45 TB (well, 35 TB now I guess) you have
17:03 πŸ”— yipdw and if the Drive Angel shows up with more space then no harm done
17:03 πŸ”— arkiver midas might have some more space, but that's not sure yet
17:04 πŸ”— SketchCow I'd REALLY prefer if we could find the OLDEST stuff.
17:05 πŸ”— Deewiant Would it be sensible/possible to do youtube searches and filter out close-to-exact matches?
17:05 πŸ”— arkiver SketchCow: we should start from the bottom then, we now started from 6000000
17:06 πŸ”— Deewiant Since significant parts are likely to be mirrored
17:06 πŸ”— arkiver Though a lot of older videos are removed
17:06 πŸ”— arkiver Deewiant: no
17:06 πŸ”— SketchCow I'd rather we do older stuff and then arbitrarily somewhere
17:06 πŸ”— SketchCow we stop
17:06 πŸ”— SketchCow Or, when I'm stopped.
17:06 πŸ”— PurpleSym has joined #archiveteam
17:07 πŸ”— arkiver yipdw: chfoo: are you able to get me list of items that are now in the to be done queue for blip?
17:07 πŸ”— Deewiant arkiver: Why not?
17:08 πŸ”— arkiver Deewiant: it's just not possible to find out without actually looking at them if they are the same
17:08 πŸ”— xmc Deewiant: sounds like a lot of work
17:08 πŸ”— Deewiant xmc: I was thinking that it could even be offloaded to the warriors but maybe the infra doesn't support such a thing well
17:08 πŸ”— Start are we doing requests for specific blip channels? i have a few i can think of
17:09 πŸ”— arkiver Start: for now we are first going to do the older videos
17:09 πŸ”— Deewiant arkiver: I'd guess that there are enough 95+% matches to save a few terabytes but maybe not
17:09 πŸ”— Start arkiver: ok
17:09 πŸ”— arkiver 95% is not enough
17:09 πŸ”— arkiver we need to be sure it exists already or not
17:09 πŸ”— Deewiant It could be a de-prioritization if you don't feel safe even at 100%
17:10 πŸ”— Start what ranges will we be doing? 0-2000000?
17:10 πŸ”— arkiver Start: we started at 6000000
17:10 πŸ”— arkiver When I have the list of items in the queue (which will be requeued later) I'll queue items 0-1000000, 1000000-2000000, etc.
17:11 πŸ”— Start ok
17:14 πŸ”— yipdw arkiver: yeah, chfoo may be faster at it; xmc can also get it
17:14 πŸ”— * yipdw is currently performing occult rituals with Core Motion
17:14 πŸ”— xmc i know nothing, nothing
17:15 πŸ”— yipdw by which I mean (__bridge_retained CMWhatTheFuckRef)
17:21 πŸ”— signius Once we do a massive grab of stuff & have it stored away from the closing down site, is there any de-duplication ran against the pulled data to reduce the storage size?
17:22 πŸ”— yipdw no automatic deduplication
17:23 πŸ”— signius With storage becoming an issue with some of the grabs & the amounts of data involved far from insignificant is de-duplication something work looking at
17:23 πŸ”— signius worth*
17:23 πŸ”— yipdw I don't think so, it complicates retrieval
17:24 πŸ”— yipdw there is a much easier solution and that is to Grab Less
17:24 πŸ”— xmc redundancy = reliability
17:25 πŸ”— yipdw while I'm back in here I guess I should get that list brb
17:33 πŸ”— yipdw it's going to take a while; the list is under active modification and I don't want to block the tracker with an SMEMBERS
17:34 πŸ”— achip has quit IRC (Read error: Operation timed out)
17:34 πŸ”— achip has joined #archiveteam
17:39 πŸ”— Ungstein1 has joined #archiveteam
17:40 πŸ”— Ungstein has quit IRC (Ping timeout: 265 seconds)
17:46 πŸ”— arkiver yipdw: ok, thank you.
17:46 πŸ”— SketchCow Let's be clear about "storage becoming an issue"
17:46 πŸ”— SketchCow Storage has always been an issue.
17:46 πŸ”— SketchCow When I started, 10gb flipped them out.
17:46 πŸ”— SketchCow Much less now.
17:46 πŸ”— SketchCow But more than that, we have two classes.
17:46 πŸ”— SketchCow Some site, which ranges from 1-30tb
17:47 πŸ”— SketchCow And then these bloated motherfucking shitdumps, endlessly uselessless, composed of low-bandwith simpsons rips and one-off shots of the side of a building, that hit into hundreds of TBs.
17:49 πŸ”— Asparagir has joined #archiveteam
17:50 πŸ”— Kenshin has quit IRC (Ping timeout: 252 seconds)
17:53 πŸ”— Kenshin has joined #archiveteam
17:54 πŸ”— SketchCow It is very hard, emotionally, to look at these shitdumps and feel sad we might only do core samples.
17:55 πŸ”— xmc yeah
17:55 πŸ”— xmc sets mode: +o SketchCow
17:55 πŸ”— xmc sets mode: +o swebb
17:55 πŸ”— swebb sets mode: +o DFJustin
17:55 πŸ”— swebb sets mode: +o SadDM
17:58 πŸ”— Smiley damnit we need the worlds largest dedupe :<
18:05 πŸ”— Ravenloft has quit IRC (Ping timeout: 606 seconds)
18:05 πŸ”— SketchCow I'm just saying signius is wrong.
18:05 πŸ”— SketchCow And the de-dupe let's-make-it-all-work-now-but-a-nightmare-later may or may not be worth it.
18:07 πŸ”— Smiley a million symlinks lol
18:07 πŸ”— Smiley a non-lossy dedup
18:08 πŸ”— arkiver We're not going to deduplicate
18:09 πŸ”— signius I understand the need for wanting to keep every bit of data as original, but i also not so convinced about keeping multiple hundred GB copies of exactly the same video...........but i was merely enquiring with my question i am not a deduplication zealot :)
18:13 πŸ”— ersi well, by now you kind of look like one regardless
18:20 πŸ”— habi has joined #archiveteam
18:22 πŸ”— habi has left
18:25 πŸ”— Smiley I am, I guess :/
18:25 πŸ”— Smiley Not that I believe theres an ideal way of achieving it, just that I feel it's a good idea.
18:27 πŸ”— Guest25 eventually, I'd like to colocate servers for AT over here as rsync targets, depending on how much it costs
18:28 πŸ”— Guest25 is now known as expr_
18:28 πŸ”— Smiley wouldn't we all? ;/
18:28 πŸ”— Smiley Anyway, discussion to #archiveteam-bs
18:28 πŸ”— expr_ heh, indeed
18:40 πŸ”— db48x has joined #archiveteam
18:50 πŸ”— estrabd has joined #archiveteam
19:03 πŸ”— aaaaaaaaa has joined #archiveteam
19:03 πŸ”— swebb sets mode: +o aaaaaaaaa
19:12 πŸ”— Jonimus has quit IRC (Ping timeout: 252 seconds)
19:20 πŸ”— expr_ has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…)
19:21 πŸ”— expr_ has joined #archiveteam
19:24 πŸ”— Jonimus has joined #archiveteam
19:26 πŸ”— godane has joined #archiveteam
19:26 πŸ”— nmnn has quit IRC (Ping timeout: 483 seconds)
19:30 πŸ”— nmnn has joined #archiveteam
19:31 πŸ”— godane has quit IRC (Ping timeout: 265 seconds)
19:52 πŸ”— godane has joined #archiveteam
20:01 πŸ”— yipdw arkiver: http://depot.ninjawedding.org/blip-done.gz <- accurate as of a couple hours ago
20:25 πŸ”— godane has quit IRC (Quit: Leaving.)
20:39 πŸ”— PurpleSym has quit IRC (Remote host closed the connection)
20:46 πŸ”— godane has joined #archiveteam
20:48 πŸ”— tomwsmf-a has joined #archiveteam
20:59 πŸ”— Ravenloft has joined #archiveteam
21:06 πŸ”— SadDM has quit IRC (Ping timeout: 483 seconds)
21:11 πŸ”— mistym has joined #archiveteam
21:17 πŸ”— aaaaaaaaa has quit IRC (Read error: Operation timed out)
21:19 πŸ”— nmnn has quit IRC (Quit: Ex-Chat)
21:23 πŸ”— SadDM has joined #archiveteam
21:23 πŸ”— swebb sets mode: +o SadDM
21:33 πŸ”— schbirid has quit IRC (Leaving)
21:43 πŸ”— brayden has joined #archiveteam
21:43 πŸ”— swebb sets mode: +o brayden
21:43 πŸ”— tomwsmf-a has quit IRC (Ping timeout: 258 seconds)
21:49 πŸ”— arkiver yipdw: thank you!
21:49 πŸ”— arkiver I'll pause the tracker, remove the current queue and queue items 1 to 1000000
21:54 πŸ”— Asparagir has quit IRC (Asparagir)
22:02 πŸ”— arkiver ok here we go
22:07 πŸ”— arkiver The first one million items have now been added
22:07 πŸ”— arkiver most will be dead though
22:07 πŸ”— brayden has quit IRC (Read error: Operation timed out)
22:20 πŸ”— mistym has quit IRC (Remote host closed the connection)
22:23 πŸ”— useretail i have access to a lot of scanned books, how should i upload them? one item for one book, or in a pack?
22:24 πŸ”— arkiver what kind of books?
22:25 πŸ”— useretail scanned ones, digital copies etc
22:25 πŸ”— arkiver I mean are they still copyrighted or too old/not copyrighted
22:26 πŸ”— arkiver ?
22:26 πŸ”— useretail yeah, they are still copyrighted
22:26 πŸ”— useretail some of them are not
22:26 πŸ”— useretail like books from ussr
22:30 πŸ”— mistym has joined #archiveteam
22:31 πŸ”— arkiver IA might be interested I think, but they won't make copyrighted books available for public
22:31 πŸ”— mistym_ has joined #archiveteam
22:33 πŸ”— useretail arkiver: but should i upload them in packs or one by one? what's your opinion? it's more than 100gb...
22:34 πŸ”— arkiver I think you should ask SketchCow about that
22:34 πŸ”— useretail ok, i will
22:41 πŸ”— mistym has quit IRC (Read error: Operation timed out)
22:43 πŸ”— Ravenloft has quit IRC (Ping timeout: 240 seconds)
22:43 πŸ”— aaaaaaaaa has joined #archiveteam
22:43 πŸ”— swebb sets mode: +o aaaaaaaaa
22:52 πŸ”— mistym has joined #archiveteam
23:00 πŸ”— mistym_ has quit IRC (Read error: Operation timed out)
23:01 πŸ”— mistym_ has joined #archiveteam
23:03 πŸ”— expr_ has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…)
23:06 πŸ”— mistym_ has quit IRC (Remote host closed the connection)
23:08 πŸ”— mistym has quit IRC (Read error: Operation timed out)
23:11 πŸ”— mistym has joined #archiveteam
23:11 πŸ”— mistym has quit IRC (Remote host closed the connection)
23:23 πŸ”— DFJustin useretail: one by one is ideal, if not sketchcow may be able to arrange something where he can accept the pack and upload it with scripts
23:29 πŸ”— Stiletto has joined #archiveteam
23:32 πŸ”— Stilett0 has quit IRC (Read error: Operation timed out)
23:39 πŸ”— Stilett0 has joined #archiveteam
23:41 πŸ”— Dark_Star has quit IRC (Read error: Operation timed out)
23:43 πŸ”— Stiletto has quit IRC (Read error: Operation timed out)
23:49 πŸ”— SadDM has quit IRC (Ping timeout: 483 seconds)
23:51 πŸ”— SadDM has joined #archiveteam
23:51 πŸ”— swebb sets mode: +o SadDM

irclogger-viewer