#archiveteam 2015-03-03,Tue

↑back Search

Time Nickname Message
00:04 πŸ”— signius has joined #archiveteam
00:17 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
00:17 πŸ”— dashcloud has joined #archiveteam
00:28 πŸ”— BlueMaxim has joined #archiveteam
00:56 πŸ”— Rotab has quit IRC (hub.se irc.du.se)
01:27 πŸ”— Boppen has joined #archiveteam
01:30 πŸ”— Boppen has quit IRC (hub.se irc.du.se)
01:48 πŸ”— xtr-201 has joined #archiveteam
02:02 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
02:02 πŸ”— dashcloud has joined #archiveteam
02:07 πŸ”— mistym has quit IRC (Remote host closed the connection)
02:13 πŸ”— primus104 has quit IRC (Leaving.)
02:21 πŸ”— mistym has joined #archiveteam
02:28 πŸ”— BlueMaxim has quit IRC (Read error: Operation timed out)
02:28 πŸ”— BlueMaxim has joined #archiveteam
03:18 πŸ”— BlueMaxim has quit IRC (Read error: Connection reset by peer)
03:19 πŸ”— garyrh has quit IRC (Remote host closed the connection)
03:42 πŸ”— garyrh has joined #archiveteam
04:11 πŸ”— BlueMaxim has joined #archiveteam
04:30 πŸ”— VonGuard_ is now known as VonGuard
05:09 πŸ”— antomatic has quit IRC (Read error: Connection reset by peer)
05:09 πŸ”— lytv has quit IRC (Read error: Connection reset by peer)
05:09 πŸ”— fresco___ has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— VonGuard has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— russss has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— deathy has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— danneh_ has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— LittUp has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— Muad-Dib has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— Rickster has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— lhobas has quit IRC (hub.dk efnet.port80.se)
05:09 πŸ”— nox has quit IRC (Read error: Operation timed out)
05:09 πŸ”— NovaKing_ has quit IRC (Read error: Operation timed out)
05:09 πŸ”— yipdw has quit IRC (hub.dk irc.homelien.no)
05:09 πŸ”— pikhq has quit IRC (hub.dk irc.homelien.no)
05:09 πŸ”— altlabel has quit IRC (hub.dk irc.homelien.no)
05:09 πŸ”— ionpulse has quit IRC (hub.dk irc.homelien.no)
05:09 πŸ”— antomati_ has joined #archiveteam
05:09 πŸ”— NovaKing_ has joined #archiveteam
05:09 πŸ”— nox has joined #archiveteam
05:11 πŸ”— lytv has joined #archiveteam
05:17 πŸ”— antomati_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Zebranky_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Fusl has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— ryan__ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— ruukasu has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Deewiant has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— edsu_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Kazzy has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— ex-parrot has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Gfy has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— SketchCow has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— w0rp has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Sellyme_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— jk[SVP] has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Kniffy has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Kenshin has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Nemo_bis has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— yan has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— nico_32 has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— raylee has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Atluxity has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— is- has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— nox has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— NovaKing_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— espes__ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— aNthraXx has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— cadbury_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— underscor has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Sue__ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— gibigian1 has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— kanzure_ has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— lukeman has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— warthurto has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Sk1d has quit IRC (hub.efnet.us hub.dk)
05:17 πŸ”— Void_ has quit IRC (hub.efnet.us hub.dk)
05:21 πŸ”— espes___ has joined #archiveteam
05:48 πŸ”— dashcloud has quit IRC (Quit: No Ping reply in 210 seconds.)
05:50 πŸ”— dashcloud has joined #archiveteam
06:26 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
06:27 πŸ”— dashcloud has joined #archiveteam
07:15 πŸ”— cadbury_ has joined #archiveteam
07:15 πŸ”— lhobas has joined #archiveteam
07:15 πŸ”— Muad-Dib has joined #archiveteam
07:15 πŸ”— Rickster has joined #archiveteam
07:15 πŸ”— danneh_ has joined #archiveteam
07:15 πŸ”— LittUp has joined #archiveteam
07:15 πŸ”— deathy has joined #archiveteam
07:15 πŸ”— russss has joined #archiveteam
07:15 πŸ”— VonGuard has joined #archiveteam
07:15 πŸ”— fresco___ has joined #archiveteam
07:15 πŸ”— warthurto has joined #archiveteam
07:15 πŸ”— lukeman has joined #archiveteam
07:15 πŸ”— Sue__ has joined #archiveteam
07:15 πŸ”— aNthraXx has joined #archiveteam
07:15 πŸ”— Void_ has joined #archiveteam
07:15 πŸ”— Rotab has joined #archiveteam
07:15 πŸ”— underscor has joined #archiveteam
07:15 πŸ”— ionpulse has joined #archiveteam
07:15 πŸ”— altlabel has joined #archiveteam
07:15 πŸ”— pikhq has joined #archiveteam
07:15 πŸ”— yipdw has joined #archiveteam
07:15 πŸ”— gibigiana has joined #archiveteam
07:15 πŸ”— Sk1d has joined #archiveteam
07:15 πŸ”— antomati_ has joined #archiveteam
07:15 πŸ”— Nemo_bis has joined #archiveteam
07:15 πŸ”— yan has joined #archiveteam
07:15 πŸ”— nico_32 has joined #archiveteam
07:15 πŸ”— Fusl has joined #archiveteam
07:15 πŸ”— Zebranky_ has joined #archiveteam
07:15 πŸ”— ryan__ has joined #archiveteam
07:15 πŸ”— is- has joined #archiveteam
07:15 πŸ”— ruukasu has joined #archiveteam
07:15 πŸ”— Deewiant has joined #archiveteam
07:15 πŸ”— raylee has joined #archiveteam
07:15 πŸ”— edsu_ has joined #archiveteam
07:15 πŸ”— Kazzy has joined #archiveteam
07:15 πŸ”— ex-parrot has joined #archiveteam
07:15 πŸ”— jk[SVP] has joined #archiveteam
07:15 πŸ”— Sellyme_ has joined #archiveteam
07:15 πŸ”— w0rp has joined #archiveteam
07:15 πŸ”— SketchCow has joined #archiveteam
07:15 πŸ”— Gfy has joined #archiveteam
07:15 πŸ”— Kenshin has joined #archiveteam
07:15 πŸ”— Kniffy has joined #archiveteam
07:15 πŸ”— Atluxity has joined #archiveteam
07:15 πŸ”— hub.se sets mode: +ooo raylee SketchCow Kenshin
07:15 πŸ”— swebb sets mode: +o underscor
07:15 πŸ”— swebb sets mode: +o SketchCow
07:17 πŸ”— kanzure has joined #archiveteam
07:59 πŸ”— Jonimus has quit IRC (Ping timeout: 370 seconds)
08:06 πŸ”— mistym has quit IRC (Remote host closed the connection)
08:10 πŸ”— Jonimus has joined #archiveteam
09:03 πŸ”— schbirid has joined #archiveteam
09:04 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
09:04 πŸ”— dashcloud has joined #archiveteam
09:17 πŸ”— Ymgve has joined #archiveteam
09:19 πŸ”— primus104 has joined #archiveteam
09:39 πŸ”— antomati_ is now known as antomatic
10:10 πŸ”— primus104 has quit IRC (Leaving.)
10:22 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
10:25 πŸ”— Sk1d has joined #archiveteam
10:45 πŸ”— Sk2d has joined #archiveteam
10:46 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
10:46 πŸ”— Sk2d is now known as Sk1d
11:25 πŸ”— dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
11:26 πŸ”— dashcloud has joined #archiveteam
11:31 πŸ”— Muad-Dib Alright, I want to grab a big-ass mirror of a niche art site that includes a lot of stuff that has been οΏ½deletedοΏ½ from the net earlier, itοΏ½s probably multiple TBs and seems to have limited bandwidth, shall I just put it in archivebot or do we grab this seperately? http://vj5pbopejlhcbz4n.onion.city/indexes
11:32 πŸ”— Muad-Dib οΏ½deletedοΏ½ from the site
11:32 πŸ”— Muad-Dib *
11:35 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
11:35 πŸ”— Ctrl-S I want a copy of this too
11:36 πŸ”— Muad-Dib there's a lot of furry porn in there, lol :D
11:36 πŸ”— Muad-Dib long live the internet
11:37 πŸ”— Muad-Dib freaky place I'd trade for no other
11:37 πŸ”— Muad-Dib Ctrl-S, you have terabytes available ATM?
11:37 πŸ”— Sk1d has joined #archiveteam
11:37 πŸ”— Muad-Dib I have a feeling this archive this might well pass the 10 TB
11:37 πŸ”— Muad-Dib mark
11:38 πŸ”— Ctrl-S I'm on a capped connection though :(
11:38 πŸ”— Ctrl-S could you host it for a year or so so i can afford to grab a copy?
11:39 πŸ”— Muad-Dib looks like there's no archivebot pipeline with enough storage for grabbing it all at once either :C http://dashboard.at.ninjawedding.org/pipelines
11:39 πŸ”— Muad-Dib lol
11:39 πŸ”— Muad-Dib Ctrl-S, you might as well hire dedi hosting, lol
11:39 πŸ”— Muad-Dib for one month, grab everything, and post it to IA
11:39 πŸ”— Ctrl-S i'm serious about wanting this mirrored
11:39 πŸ”— Muad-Dib me too
11:39 πŸ”— Ctrl-S what do i have to do to get it done?
11:40 πŸ”— Muad-Dib but its way too much for me to hold
11:40 πŸ”— Ctrl-S where do i send the drive money
11:40 πŸ”— Muad-Dib It's already kicking up controversy in the art site's community for hosting people's old and deleted stuff
11:40 πŸ”— Muad-Dib I don't expect it to be up for long
11:40 πŸ”— Ctrl-S because mailing HDDs is the only way i can get a copy of this
11:41 πŸ”— Muad-Dib don't expect it to be up for long on clearnet anywa
11:41 πŸ”— Muad-Dib y
11:41 πŸ”— Ctrl-S >controversy on furaffinity
11:41 πŸ”— Muad-Dib IKR
11:41 πŸ”— Muad-Dib "OH NO, I POSTED MY STUFF TO THE PUBLIC INTERNET AND I CANT GET RID OF IT ANYMORE"
11:42 πŸ”— Ctrl-S can we contact the admin?
11:42 πŸ”— Ctrl-S of this mirror i mean
11:42 πŸ”— Muad-Dib no one knows who's hosting this
11:42 πŸ”— Muad-Dib but it might be site staff, since it includes so many "deleted" files
11:44 πŸ”— Ctrl-S I would seriously pay the several hundred dollars for disk space for this
11:44 πŸ”— Ctrl-S because I KNOW it's endangered
11:45 πŸ”— Atluxity would be nice to not archive this via onion.city, but rather do it via tor? looks like a hidden service proxy to me
11:46 πŸ”— Muad-Dib maybe I should just throw it in archivebot and see how far it gets
11:46 πŸ”— arkiver 10TB is nothing for archivebot
11:46 πŸ”— Muad-Dib Atluxity: ideally, yes
11:46 πŸ”— arkiver if we want this we can create a warrior project
11:46 πŸ”— Muad-Dib arkiver: http://dashboard.at.ninjawedding.org/pipelines
11:46 πŸ”— Muad-Dib a warrior project that grabs shit from tor?
11:46 πŸ”— arkiver yeah, why not
11:47 πŸ”— Atluxity arkiver: warrior project getting archiving a tor hidden service? sounds...interesting
11:47 πŸ”— arkiver onion.city for now
11:47 πŸ”— Muad-Dib won't that require extra dependencies on the warrior VM's?
11:47 πŸ”— Ctrl-S We DO need to get around to backing up the tor hidden sites
11:47 πŸ”— Atluxity correct
11:47 πŸ”— arkiver I can create a project for this .onion.city site easily
11:47 πŸ”— arkiver but 10TB is a lot
11:47 πŸ”— Muad-Dib but IA might not be willing to host hidden services, with good reason
11:47 πŸ”— Ctrl-S they hold an especially high degree of cultural relevance due to their often illicit nature
11:47 πŸ”— arkiver not sure if IA is willing to take that all in
11:47 πŸ”— Muad-Dib talk to the onion.city admins first ;)
11:48 πŸ”— schbirid <Muad-Dib> but IA might not be willing to host hidden services, with good reason
11:48 πŸ”— schbirid also the opposite
11:48 πŸ”— schbirid they might be very willing, with good reason
11:48 πŸ”— Muad-Dib I know
11:48 πŸ”— Ctrl-S they don't have to provide open access, just hang onto the data
11:48 πŸ”— Muad-Dib I think they'd probably be a bit... conflicted about it
11:48 πŸ”— arkiver if SketchCow thinks IA is willing to take multiple TB's from http://vj5pbopejlhcbz4n.onion.city/indexes
11:49 πŸ”— arkiver if that ^ I'll have a project running soon
11:51 πŸ”— Ctrl-S I actually wrote a script to save things from FA, but i'm on a capped connection so i can't save everything
11:53 πŸ”— Muad-Dib <arkiver> 10TB is nothing for archivebot
11:53 πŸ”— Muad-Dib 3tb max free diskspace isn't agreeing with you, ark http://dashboard.at.ninjawedding.org/pipelines
11:53 πŸ”— arkiver what I meant is that a website of 10TB whould
11:53 πŸ”— arkiver shouldn't be archived with arcivebot
11:54 πŸ”— Muad-Dib oh
11:54 πŸ”— Muad-Dib okay
11:54 πŸ”— Muad-Dib misinterpretation :P
11:54 πŸ”— arkiver yep, I wasn't clear
11:54 πŸ”— Ctrl-S >That wonderous feel when you find a copy of something yo'd long thought deleted
11:55 πŸ”— Muad-Dib <3
11:56 πŸ”— Ctrl-S whatever the case, this needs backing up right now, and i will do anything in my power to help you do so
11:57 πŸ”— Ctrl-S I've seen too many artists go bezerk and delete everything to lose this
11:57 πŸ”— arkiver do you have 10TB of free diskspace?
11:57 πŸ”— arkiver if you do, we'll start
11:57 πŸ”— Ctrl-S maybe, but 1TB/month cap
11:57 πŸ”— Ctrl-S Being australian is suffering
12:21 πŸ”— Muad-Dib ;_;7
12:52 πŸ”— nox has joined #archiveteam
13:04 πŸ”— BlueMaxim has quit IRC (Read error: Connection reset by peer)
13:15 πŸ”— * ersi points and laughs
13:26 πŸ”— Rotab ;)
13:54 πŸ”— primus104 has joined #archiveteam
13:56 πŸ”— sankin has joined #archiveteam
14:32 πŸ”— Muad-Dib https://www.youtube.com/watch?v=EWCLpaynj4Y fuck my country and its people ;_;
14:32 πŸ”— Muad-Dib but at least we dont have bandwidth caps ;)
14:40 πŸ”— midas rofl Muad-Dib
14:41 πŸ”— Muad-Dib white trash, white trash everywhere ;_;
14:41 πŸ”— midas aye
14:41 πŸ”— Muad-Dib glorious YUROP
14:42 πŸ”— aNthraXx has quit IRC (Read error: Operation timed out)
14:43 πŸ”— aNthraXx has joined #archiveteam
15:19 πŸ”— Start has quit IRC (Disconnected.)
15:25 πŸ”— Sk2d has joined #archiveteam
15:28 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
15:29 πŸ”— Sk1d has joined #archiveteam
15:30 πŸ”— Sk2d has quit IRC (Ping timeout: 265 seconds)
15:33 πŸ”— Froggypwn has quit IRC (Read error: Operation timed out)
15:34 πŸ”— Froggypwn has joined #archiveteam
15:35 πŸ”— arkiver midas: are you able to get the list of ftps back online?
15:36 πŸ”— midas is it offline?
15:37 πŸ”— arkiver yeah, 503
15:37 πŸ”— arkiver 502*
15:39 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
15:40 πŸ”— midas stupid pad crashed
15:43 πŸ”— Sk1d has joined #archiveteam
15:51 πŸ”— dashcloud has quit IRC (Read error: Connection reset by peer)
15:51 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
15:53 πŸ”— Sk1d has joined #archiveteam
15:56 πŸ”— dashcloud has joined #archiveteam
15:58 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
16:00 πŸ”— mistym has joined #archiveteam
16:01 πŸ”— mistym has quit IRC (Remote host closed the connection)
16:04 πŸ”— Start has joined #archiveteam
16:05 πŸ”— dashcloud has joined #archiveteam
16:06 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
16:06 πŸ”— Sk2d has joined #archiveteam
16:06 πŸ”— Sk2d is now known as Sk1d
16:20 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
16:21 πŸ”— signius has quit IRC (Read error: Operation timed out)
16:23 πŸ”— mistym has joined #archiveteam
16:25 πŸ”— Sk1d has joined #archiveteam
16:33 πŸ”— DFJustin fwiw archivebot uploads in 5gb intervals so you don't actually need 10tb of free space
16:34 πŸ”— DFJustin tasks that run for months can be an issue though as machines need maintenance etc
16:35 πŸ”— signius has joined #archiveteam
16:37 πŸ”— DFJustin so if there's some way to feed in pieces of it one at a time (subdirectories are ideal)
16:51 πŸ”— Start has quit IRC (Disconnected.)
16:57 πŸ”— danneh_ has quit IRC (Ping timeout: 260 seconds)
17:03 πŸ”— danneh_ has joined #archiveteam
17:09 πŸ”— Nertsy has quit IRC (Read error: Operation timed out)
17:17 πŸ”— mistym has quit IRC (Remote host closed the connection)
17:31 πŸ”— mistym has joined #archiveteam
17:35 πŸ”— sep332 has quit IRC (bye)
17:37 πŸ”— sep332 has joined #archiveteam
17:48 πŸ”— chfoo i can probably implement tor for archivebot sometime this week
17:50 πŸ”— arkiver chfoo: I do think 10TB websites shouldn't be done with archivebot
17:52 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
17:55 πŸ”— Sk1d has joined #archiveteam
18:02 πŸ”— Start has joined #archiveteam
18:02 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
18:04 πŸ”— Sk1d has joined #archiveteam
18:09 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
18:09 πŸ”— chfoo https://about.gitlab.com/2015/03/03/gitlab-acquires-gitorious/
18:12 πŸ”— Sk1d has joined #archiveteam
18:17 πŸ”— Sk2d has joined #archiveteam
18:20 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
18:22 πŸ”— rolfb has joined #archiveteam
18:22 πŸ”— Sk2d has quit IRC (Ping timeout: 265 seconds)
18:23 πŸ”— Sk1d has joined #archiveteam
18:24 πŸ”— rolfb Hi there. Gitorious has been acquired and gitorious.org will shut down at the end of May. Is there any way to preserve the data?
18:30 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
18:31 πŸ”— chfoo arkiver: i'm not really fond of using tor in the warrior because it will involve setting up the latest tor and http proxy and it's likely that a manual script runner will break something. i'm also worried about needing to set up the warriors to use bridges in case the isp blocks tor
18:31 πŸ”— chfoo but maybe someone with lots of bandwidth could set up a public tor proxy for archiveteam use
18:33 πŸ”— Sk1d has joined #archiveteam
18:39 πŸ”— chazchaz rolfb: Is there data that isn't already in the WayBackMachine?
18:39 πŸ”— Sk2d has joined #archiveteam
18:41 πŸ”— Start has quit IRC (Disconnected.)
18:41 πŸ”— chazchaz As far as I can see, everything they have there other than the repo for the community edition source code is private/paid subscriptopn based.
18:42 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
18:42 πŸ”— Sk2d is now known as Sk1d
18:43 πŸ”— chfoo git clone everything
18:45 πŸ”— chazchaz Wait, never mind, it appears they do host some repos
18:49 πŸ”— chazchaz Apparently, GitLab took enough paying ustomers that Gitorious can't support its self while offering free service.
18:50 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
18:51 πŸ”— abartov has joined #archiveteam
19:00 πŸ”— Sk2d has joined #archiveteam
19:01 πŸ”— kyan_ has joined #archiveteam
19:03 πŸ”— kyan has quit IRC (Read error: Operation timed out)
19:05 πŸ”— Sk1d- has joined #archiveteam
19:06 πŸ”— Sk2d has quit IRC (Ping timeout: 265 seconds)
19:09 πŸ”— Sk2d has joined #archiveteam
19:09 πŸ”— Sk2d is now known as Sk1d
19:11 πŸ”— Sk1d- has quit IRC (Read error: Operation timed out)
19:11 πŸ”— fenn "We don't want to move people's code to another organization without their permission." yes, their open-source, public code
19:14 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
19:14 πŸ”— Muad-Dib lol
19:14 πŸ”— Muad-Dib sad
19:18 πŸ”— Sk1d has joined #archiveteam
19:21 πŸ”— sankin has quit IRC (Leaving.)
19:22 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
19:25 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
19:31 πŸ”— Sk1d has joined #archiveteam
19:32 πŸ”— dashcloud has joined #archiveteam
19:33 πŸ”— rolfb chazchaz: sorry for not replying, i don't know what the waybackmachine has, but surely it would be more interesting to have the git repositories, and all the code on .org is available for download, over 100k repositories
19:35 πŸ”— yipdw Muad-Dib: maximum diskspace has no effect on maximum job size
19:35 πŸ”— yipdw the main problem with 10 TB is justifying shoving 10 TB into IA
19:36 πŸ”— yipdw also running up someone's bandwidth bill if empathy is something you believe in
19:43 πŸ”— BlueMaxim has joined #archiveteam
19:48 πŸ”— sep332 I don't think it's bandwidth that's the problem. It takes more than 30 seconds to start getting data for some of those links
19:57 πŸ”— yipdw I was referring also to the node operator's bill
19:57 πŸ”— yipdw OVH doesn't seem to care, DO seems to eventually
19:57 πŸ”— yipdw in any case a 10 TB job is really just a dick move at present time
20:01 πŸ”— Start has joined #archiveteam
20:03 πŸ”— ersi rolfb: waybackmachine = http://web.archive.org/
20:07 πŸ”— chfoo rolfb: are you the rolf the gitlab news is talking about?
20:17 πŸ”— chfoo a database and data dump of everything straight from the source would be the most ideal
20:23 πŸ”— chfoo second option would be a backdoor for archiveteam
20:24 πŸ”— aschmitz has quit IRC (Read error: Operation timed out)
20:28 πŸ”— Start has quit IRC (Disconnected.)
20:31 πŸ”— aschmitz has joined #archiveteam
20:49 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
20:55 πŸ”— dashcloud has joined #archiveteam
21:13 πŸ”— Ymgve__ has joined #archiveteam
21:18 πŸ”— Nertsy has joined #archiveteam
21:18 πŸ”— Ymgve has quit IRC (Ping timeout: 506 seconds)
21:19 πŸ”— cbb has joined #archiveteam
21:20 πŸ”— Ymgve has joined #archiveteam
21:22 πŸ”— Ymgve__ has quit IRC (Ping timeout: 506 seconds)
21:26 πŸ”— Ymgve has quit IRC (Remote host closed the connection)
21:26 πŸ”— Ymgve has joined #archiveteam
21:28 πŸ”— Start has joined #archiveteam
21:29 πŸ”— Start has quit IRC (Read error: Connection reset by peer)
21:46 πŸ”— Start has joined #archiveteam
21:58 πŸ”— Ctrl-S if it's diskspace that's the problem i can donate a few hundred bucks for drives for that FA dump
22:03 πŸ”— Sk2d has joined #archiveteam
22:04 πŸ”— mistym has quit IRC (Remote host closed the connection)
22:04 πŸ”— rolfb chfoo: i am
22:04 πŸ”— Sk1d has quit IRC (Read error: Operation timed out)
22:04 πŸ”— Sk2d is now known as Sk1d
22:06 πŸ”— mistym has joined #archiveteam
22:08 πŸ”— SN4T14_ has quit IRC (Read error: Connection reset by peer)
22:09 πŸ”— Sk1d has quit IRC (Ping timeout: 265 seconds)
22:09 πŸ”— SN4T14 has joined #archiveteam
22:11 πŸ”— rolfb Ctrl-S: was that directed at me?
22:11 πŸ”— Ctrl-S no
22:11 πŸ”— Sk1d has joined #archiveteam
22:12 πŸ”— rolfb ok :)
22:12 πŸ”— Ctrl-S I odn't think so
22:12 πŸ”— rolfb Ctrl-S: what was it about?
22:12 πŸ”— Ctrl-S art hosting site backup someone's made with pretty much all the stuff that was deleted from the site included
22:13 πŸ”— Ctrl-S ~10 TB was estimated
22:14 πŸ”— chfoo rolfb: is possible to just upload the repos directly to archive.org?
22:14 πŸ”— rolfb chfoo: we have root, so I guess we can do whatever we want? we don't have much in terms of space to create images though
22:15 πŸ”— xmc how much temporary space would you need?
22:15 πŸ”— rolfb xmc: we have 4.5 TB of data
22:16 πŸ”— xmc oh, so a reasonable amount
22:16 πŸ”— rolfb always reasonable ;-)
22:16 πŸ”— xmc :)
22:16 πŸ”— Smiley I have a slooooooooow 2Tb
22:16 πŸ”— xmc you could probably fire up an amazon instance with a bunch of storage for a few dozen bucks
22:16 πŸ”— xmc and stream it to that for packaging
22:17 πŸ”— schbirid has quit IRC (Leaving)
22:19 πŸ”— Smiley the b/w in out tho??
22:19 πŸ”— Start has quit IRC (Disconnected.)
22:20 πŸ”— rolfb Smiley: bandwidth is adjustable
22:20 πŸ”— rolfb atleast on our side
22:21 πŸ”— Smiley Nod
22:21 πŸ”— Smiley but costs to export from amazon can be wild...
22:21 πŸ”— rolfb we could possibly send physical disks
22:21 πŸ”— Smiley oooooooooo
22:21 πŸ”— rolfb but how would it be made available after?
22:21 πŸ”— Smiley SketchCow could maybe accept physical disks
22:21 πŸ”— Smiley well, if you have disks I'd think IA would host it
22:21 πŸ”— Ctrl-S send disks to IA, IA uploads from the disks
22:21 πŸ”— Smiley it's just the fact their storage costs like $1000/Tb
22:21 πŸ”— Ctrl-S that much?
22:22 πŸ”— Smiley yah due to duplication etc etc
22:22 πŸ”— xmc ten cents a gig a month
22:22 πŸ”— * Smiley can't remember exactly
22:22 πŸ”— xmc IA or S3?
22:22 πŸ”— xmc thousand gigs is a hundred bucks a month
22:22 πŸ”— xmc ish
22:23 πŸ”— xmc killer is transit from AWS, they estimate about 500 bux to get 5T out of AWS
22:23 πŸ”— sep332 IA is $2k/TB. not per year, that's forever.
22:23 πŸ”— DFJustin for ia you have to keep in mind it's amortized out to infinity because you have to replace drives every couple years
22:23 πŸ”— xmc aye
22:23 πŸ”— Panasonic has quit IRC (Ping timeout: 370 seconds)
22:23 πŸ”— chfoo #archiveteam-bs
22:24 πŸ”— rolfb sep332: meaning that if we send disks to IA, we need to pay them $9k to preserve the data?
22:24 πŸ”— Smiley no
22:24 πŸ”— Ctrl-S no, they have to pay that
22:24 πŸ”— Smiley if you send htem disks, they'd be happy
22:24 πŸ”— rolfb ok, ok
22:24 πŸ”— Smiley if we want them to store the data for us, we might need to look at fundraising...
22:24 πŸ”— Ctrl-S you only need the disks, if they can't find the space i presume they'd just keep the data somewhere less expensive
22:25 πŸ”— Ctrl-S like in a cupboard
22:25 πŸ”— rolfb but ... how would the git repositories be made available?
22:25 πŸ”— xmc best practice for git repos is to export git bundles
22:25 πŸ”— Ctrl-S zip of each repo, infopage as html as well?
22:25 πŸ”— DFJustin IA has been very generous about doing pretty much anything we send them for free, the dollar figures are just to keep things in perspective
22:25 πŸ”— xmc then an IA item would consist of a git bundle and all the other stuff from the repo
22:25 πŸ”— xmc rolfb: what services exactly do you have for each repo?
22:26 πŸ”— xmc i mean, what stuff do you store
22:26 πŸ”— rolfb not much aside from the repository
22:26 πŸ”— xmc so not a wiki/bugtracker/filedump like github does
22:27 πŸ”— rolfb there's a wiki
22:27 πŸ”— rolfb but that's also a repository
22:27 πŸ”— xmc great
22:27 πŸ”— xmc so if i were doing this
22:27 πŸ”— xmc i would create one IA item per repo, containing two git bundles, one each of the source code and of the wiki
22:27 πŸ”— xmc git bundles are, conveniently, bzip'd
22:28 πŸ”— xmc but i'm sure you already know that :)
22:28 πŸ”— rolfb xmc, just to complicate things ... we have repositories by project
22:28 πŸ”— xmc project?
22:28 πŸ”— rolfb example https://gitorious.org/gitorious/
22:28 πŸ”— xmc ahh
22:28 πŸ”— rolfb but the project name could be metadata for a repo
22:29 πŸ”— xmc right
22:29 πŸ”— BlueMaxim has quit IRC (Ping timeout: 370 seconds)
22:29 πŸ”— xmc i'd say put e.g. https://gitorious.org/gitorious/libdolt/ into http://archive.org/details/gitoriousexport_gitorious_libdolt
22:30 πŸ”— xmc so the item names you're creating would be gitoriousexport_$(project)_$(repo)
22:30 πŸ”— xmc and then you'd add various metadata fields to the item as well
22:30 πŸ”— xmc how's this sound?
22:30 πŸ”— rolfb sounds good
22:30 πŸ”— xmc cool :)
22:31 πŸ”— xmc you can use almost any characters in IA item names, but it's best practice to restrict to [-_A-Za-z0-9]
22:31 πŸ”— xmc and .
22:31 πŸ”— rolfb i'm pretty sure we have similar restrictions ... as names are used as urls
22:31 πŸ”— xmc yeah
22:31 πŸ”— xmc i've not heard of any characters except / breaking things ... but *shrug*
22:32 πŸ”— rolfb but, how do we create an IA bundle?
22:32 πŸ”— xmc ia bundle?
22:32 πŸ”— rolfb item
22:32 πŸ”— DFJustin https://pypi.python.org/pypi/internetarchive
22:32 πŸ”— xmc there's a python toolo .. yes
22:32 πŸ”— rolfb thanks
22:33 πŸ”— xmc if you have all the items have a shared name prefix, or an identical metadata field, someone at IA can put them into a special collection
22:34 πŸ”— rolfb is there a problem uploading 122k bundles? or should we rather send disks?
22:34 πŸ”— rolfb ia items*
22:35 πŸ”— xmc 122,000 items / 4.5T? should be fine, i guess? especially if spread out over a month or so
22:36 πŸ”— rolfb yup, something like that
22:36 πŸ”— xmc the script that processes uploads will hold your upload until it's allocated space, which usually takes a few tens of seconds
22:36 πŸ”— xmc so you might want to look into mild parallelism
22:36 πŸ”— rolfb is this channel logged somewhere?
22:36 πŸ”— xmc yes
22:36 πŸ”— xmc also, i'm not an IA person
22:36 πŸ”— DFJustin http://badcheese.com/~steve/atlogs/?chan=archiveteam
22:36 πŸ”— xmc just a satisfied customer
22:36 πŸ”— Ctrl-S i can give logs if you need them
22:37 πŸ”— xmc rolfb: thanks for being a cool, forward-thinking person <3
22:37 πŸ”— rolfb my client has been logging so i'm all good for relaying information to the experts in my team
22:38 πŸ”— xmc sweet
22:38 πŸ”— rolfb xmc: since you are not an IA person, who do I verify that I can do this with?
22:38 πŸ”— xmc SketchCow is an IA employee
22:38 πŸ”— xmc i'd expect him to be in irc within the next few hours
22:38 πŸ”— rolfb it's already past my bedtime
22:38 πŸ”— rolfb <- norwegian
22:39 πŸ”— xmc ahhh, yes
22:39 πŸ”— rolfb xmc: also, thanks for the kind words
22:39 πŸ”— xmc i know a finn elsewhere on efnet who went to bed an hour ago
22:39 πŸ”— rolfb trying to make the best of a bad situation
22:39 πŸ”— xmc you're a good sight better than most people in your situation
22:39 πŸ”— Ctrl-S ^this
22:40 πŸ”— rolfb thanks, i'm just glad there is an alternative like IA
22:40 πŸ”— rolfb xmc: will you be staying around till SketchCow arrives?
22:40 πŸ”— DFJustin jscott@archive.org is his email
22:40 πŸ”— xmc i'll be in and out. i'm working, and in a few hours i'll be going to beer
22:40 πŸ”— rolfb ok, great. is it ok that I email him directly then?
22:41 πŸ”— xmc but i'm in irc most of my waking life
22:41 πŸ”— xmc yeah, go for it
22:41 πŸ”— rolfb ok, any names I can use as referrals for getting in touch?
22:41 πŸ”— rolfb or just use nicknames?
22:41 πŸ”— xmc irc names is good
22:42 πŸ”— DFJustin saying #archiveteam is probably good enough
22:42 πŸ”— xmc "some people with @ before their name"
22:42 πŸ”— xmc :P
22:42 πŸ”— sep332 "i'm trying to rescue my shit"
22:43 πŸ”— sep332 http://archiveteam.org/images/e/e6/Archiveteam.jpg
22:44 πŸ”— rolfb :)
22:48 πŸ”— Panasonic has joined #archiveteam
22:52 πŸ”— dashcloud has quit IRC (Read error: Operation timed out)
22:54 πŸ”— BlueMaxim has joined #archiveteam
22:59 πŸ”— dashcloud has joined #archiveteam
22:59 πŸ”— rolfb email sent
22:59 πŸ”— rolfb thanks again everyone
23:01 πŸ”— mistym has quit IRC (Remote host closed the connection)
23:12 πŸ”— rolfb has quit IRC (Linkinus - http://linkinus.com)
23:20 πŸ”— mistym has joined #archiveteam
23:21 πŸ”— Start has joined #archiveteam
23:22 πŸ”— Panasonic has quit IRC (Ping timeout: 606 seconds)

irclogger-viewer