[00:25] HCross: you made just one :P [00:25] or I did [01:30] *** cmaldonad has joined #internetarchive.bak [01:52] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [01:53] *** cmaldonad has joined #internetarchive.bak [02:16] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [02:20] *** cmaldonad has joined #internetarchive.bak [02:24] *** cmaldonad has quit IRC (Client Quit) [02:42] *** cmaldonad has joined #internetarchive.bak [02:44] *** cmaldonad has quit IRC (Client Quit) [02:47] *** cmaldonad has joined #internetarchive.bak [02:56] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [03:00] *** cmaldonad has joined #internetarchive.bak [03:10] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [03:20] *** cmaldonad has joined #internetarchive.bak [03:30] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [03:39] *** VADemon has quit IRC (Quit: left4dead) [04:04] *** cmaldonad has joined #internetarchive.bak [04:22] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [04:35] *** cmaldonad has joined #internetarchive.bak [04:54] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [05:05] lol this 60+ GB .MOV file [05:05] 2015_02_26_VICTORY_FOR_THE_NET_FULL_EVENT.MOV [05:38] *** kyan has quit IRC (Remote host closed the connection) [06:52] *** Start has quit IRC (Quit: Disconnected.) [06:52] *** Start has joined #internetarchive.bak [08:04] *** atomotic has joined #internetarchive.bak [08:29] *** atomotic_ has joined #internetarchive.bak [08:29] *** atomotic has quit IRC (Ping timeout: 260 seconds) [09:46] mornings. 444G ~50% of my first chunk of iabak'ing done now. yay. [11:53] *** atomotic_ has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [12:44] *** atomotic has joined #internetarchive.bak [14:03] *** atomotic has quit IRC (Ping timeout: 260 seconds) [14:15] *** atomotic has joined #internetarchive.bak [14:43] *** atomotic has quit IRC (Ping timeout: 260 seconds) [14:51] *** atomotic has joined #internetarchive.bak [15:20] *** cmaldonad has joined #internetarchive.bak [15:28] *** kyan has joined #internetarchive.bak [15:43] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:43] *** kyan has quit IRC (Remote host closed the connection) [16:11] *** kyan has joined #internetarchive.bak [17:30] *** Start has quit IRC (Remote host closed the connection) [17:31] *** Start has joined #internetarchive.bak [17:36] *** Start has quit IRC (Read error: Connection reset by peer) [17:39] *** Start has joined #internetarchive.bak [17:52] *** atomotic has joined #internetarchive.bak [18:28] *** cmaldonad has quit IRC (Quit: This computer has gone to sleep) [18:37] HEY THERE [18:37] So, I'm about to go to DataHoarder [18:37] So expect that influx. [18:43] oh great [18:43] i can't wait for the new season of Grand Designs [18:44] Hence the warning [18:44] Plus we need to be concocting the FAQs anyway [18:44] yep [18:44] Once we go past 1,000 participants [18:44] We're dealing with scale issues anyway, that will be one (massive variant opinions) [18:45] scale issues in which part of the system? [18:48] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [18:57] there are definitely rough spots w/r/t concurrent creation and on-lining of shards, but I think we have all the primitives to solve that [18:57] (for example, one problem is the lack of any sort of locking on the shard counter, which is one reason SHARD13 got skipped) [18:57] but there's solutions to explore there [18:58] *** db48x has joined #internetarchive.bak [18:59] actually, I kinda like that there is no SHARD13 [18:59] much like I like buildings that have no floor 13 [19:00] one solution is to not fix that :P [19:13] lol [19:18] heh [19:19] what's the actual problem? [19:23] *** cmaldonad has joined #internetarchive.bak [19:25] *** cmaldonad has quit IRC (Client Quit) [19:28] https://www.reddit.com/r/DataHoarder/comments/5cxtfl/a_plea_to_datahoarders_to_lend_space_for_the/ [19:38] SketchCow: see pm [19:57] *** VADemon has joined #internetarchive.bak [20:05] db48x: oh I was reading over the backlog and mulling over ways we can use machines to help us keep shard numbers straight [20:06] it seems to me that this is a problem that can be solved with optimistic locking [20:06] aka "oh, the git commit failed, I guess I'd better fetch/merge" [20:06] no big changes, just formalizing shard onlining workflow. i'll have to set up a shard myself before actually saying more [20:11] :) [20:11] could just be as simple as a text file in the repository [20:12] ye [20:12] s [20:12] ever use org-mode? [20:13] I have not used emacs much at all, but every day I get more evidence I should atone of my evil ways [20:13] | SHARD12 | db48x | active | all archiveteam_fire items from 2011-2015 | [20:13] and do so [20:13] :) [20:13] but seriously that's like getting me to learn Dvorak [20:13] the keyboard layout not the composer [20:14] you don't even need emacs to edit an org-mode file [20:14] true [20:14] oh, there's a vim plugin for orgmode [20:14] nice [20:14] never mind, I can keep on my decadent path [20:15] heh [20:15] I'm getting ready to disembark [20:15] bbiab [20:15] ok [20:19] *** db48x has quit IRC (Ping timeout: 255 seconds) [20:46] *** db48x has joined #internetarchive.bak [21:42] is the stats page really slow for everyone, or is that my internet connection? [21:43] uh, also ALL.html was last regenerated in May [21:44] oh, maybe the script was changed to generate index.html instead of ALL.html? [21:44] that would explain it [21:47] very slow in general for me [21:48] hmm [22:11] It appears that one of the stats page is not working for some images. [22:11] I can't make changes [22:18] *** Rye has quit IRC (Remote host closed the connection) [22:25] SketchCow: which one? [22:27] *** Xibalba has joined #internetarchive.bak [22:30] The "graph over time" one. [22:30] I see a broken one [22:32] well, that's something we can control [22:33] what kind of hardware do you recommend running this? [22:33] is this on a shard page, or an individual's stat page? [22:33] Xibalba: it doesn't need much cpu or memory at all [22:34] ah, http://iabak.archiveteam.org:8080/ give me an error 500 [22:34] same when trying to render a grap [22:34] h [22:37] db48x: I was wonder, how well would this work say on a raspberry pi 3? [22:39] Xibalba: ask Medowar [22:39] *** db48x has quit IRC (Remote host closed the connection) [22:40] *** db48x has joined #internetarchive.bak [22:50] *** db48x has quit IRC (Ping timeout: 255 seconds) [22:54] *** db48x has joined #internetarchive.bak [22:54] Someone offered a service with 1pb of space towards this [22:54] Hence my discussion of shardmasters being at the ready [22:55] I would assign someone to work with this group so it happens quickly [22:55] we'll fill a substantial amount of that with the archivebot shards [22:56] well, if you consider (128 / 1000) TB a substantial amount, anyway [22:57] er ((128 / 1000) * 100)% etc etc units feh feh [22:58] *** cmaldonad has joined #internetarchive.bak [22:58] http://iabak.archiveteam.org:8080/render/?width=900&height=500&_salt=1428621391.124&target=legendValue(alias(color(scale(divideSeries(diffSeries(sumSeries(keepLastValue(iabak.shardstats.filecount.*))%2CsumSeries(keepLastValue(iabak.shardstats.numcopies.0.*)%2CkeepLastValue(iabak.shardstats.numcopies.1.*)%2CkeepLastValue(iabak.shardstats.numcopies.2.*)))%2CsumSeries(keepLastValue(iabak.shards [22:58] tats.filecount.*)))%2C100)%2C%27%2300dd00%27)%2C%27%3E%3D3%20backups%27)%2C%27last%27)&target=legendValue(alias(color(scale(divideSeries(sumSeries(keepLastValue(iabak.shardstats.numcopies.2.*))%2CsumSeries(keepLastValue(iabak.shardstats.filecount.*)))%2C100)%2C%27%2393dd93%27)%2C%272%20backups%27)%2C%27last%27)&target=legendValue(alias(color(scale(divideSeries(sumSeries(keepLastValue(iabak.s [22:58] hardstats.numcopies.1.*))%2CsumSeries(keepLastValue(iabak.shardstats.filecount.*)))%2C100)%2C%27%23e89393%27)%2C%271%20backup%27)%2C%27last%27)&target=legendValue(alias(color(scale(divideSeries(sumSeries(keepLastValue(iabak.shardstats.numcopies.0.*))%2CsumSeries(keepLastValue(iabak.shardstats.filecount.*)))%2C100)%2C%27red%27)%2C%27IA%20only%27)%2C%27last%27)&areaMode=stacked&from=-1weeks&vt [22:59] itle=%25&yMax=100&yMin=0&title=Overall%20progress%2C%20%25 [22:59] Sorry, that's the URL that's dying when I connect to it [23:01] yea, all requests to iabak.archiveteam.org:8080 fail [23:01] and I can't ssh into the server for whatever reason [23:02] i can look into it in a bit [23:03] It'll help as we add more people, especially big hitters [23:12] hmm [23:12] root@iabak:/etc/apache2/sites-enabled# systemctl restart apache2 [23:12] Failed to restart apache2.service: Activation of org.freedesktop.systemd1 timed out [23:12] See system logs and 'systemctl status apache2.service' for details. [23:12] that's fun [23:13] huh, there are also an asston of zombie git processes [23:14] where the precise definition of 'asston' is [23:14] $ ps waxu | grep 'Z.*\[git' | wc -l [23:14] 6059 [23:14] should I just kick iabak in the head and see if we can diagnose this when it's not in such a wonky state [23:15] heh [23:15] well [23:19] those shouldn't affect apache [23:20] I don't think we have a graceful way to shut down the server, but doing it ungracefully won't be any worse than any other kind of transient network error, as far as the clients are concerned [23:20] they shouldn't affect apache, it just occurred to me that something looks off [23:20] interestingly, there's nothing in the graphite error log [23:21] oh wait, no, there is [23:21] I was looking at the wrong one [23:22] how is there a wrong one? [23:22] the most recent log file has a .1 suffix, as if there's something funky with log rotation [23:22] I was expecting it to be graphite_error.log [23:23] :P [23:23] we should find a way to send it all to journald and let it sort it out [23:23] at some point [23:23] in any case, https://gist.githubusercontent.com/yipdw/930b00f4d2bdb1fec86e475e52aea050/raw/39be1045a9a367759494b6a4b51b00fe8283e5d3/gistfile1.txt [23:25] [Mon Nov 14 17:23:13.406498 2016] [wsgi:error] [pid 13397] [remote 50.172.238.89:49760] File "/usr/lib/python2.7/dist-packages/django/apps/registry.py", line 124, in check_apps_ready [23:25] [Mon Nov 14 17:23:13.406507 2016] [wsgi:error] [pid 13397] [remote 50.172.238.89:49760] raise AppRegistryNotReady("Apps aren't loaded yet.") [23:26] apparently apps aren't loaded yet [23:26] yeah [23:26] I'm not yet familiar enough with Django to know what that means [23:27] but maaybe the internet knows [23:27] well, the front fell off, for one [23:29] *** VADemon has quit IRC (Quit: left4dead) [23:43] *** dingo has joined #internetarchive.bak [23:44] hi there [23:45] dingo: howdy [23:46] Came here from the post to /r/datahoarder. having a few issues getting a node up. here is the errors I'm getting https://imgur.com/a/InUyj [23:48] interesting [23:48] you're running it as root, which is not normally advisable [23:48] is this a special case of some type? [23:49] I was just playing around tbh, spun up a vm on my server [23:50] Once I got a feel for how it worked I was going to put it on an actual VM and give it 2TB [23:50] ah, ok [23:50] *** sevs has joined #internetarchive.bak [23:50] so the ssl cert error could be a misconfiguration [23:51] try just running "git clone https://github.com/ArchiveTeam/IA.BAK/" and see if you get the same error [23:51] what OS is this? it looks really stripped down [23:52] dingo the last error needs `cpan install CGI` to solve [23:52] like, CGI.pm missing is pretty interesting [23:52] Cloning that works fine, and the OS is just CentOS minimal [23:52] oh [23:52] ok [23:53] on ubuntu 16.04 I had to install CGI, not what I call a stripped OS [23:54] (but then it fails with "fatal: refusing to merge unrelated histories" which may be caused by an up to date git with different defaults) [23:55] hi, currently trying to set this up and wondering where to set the name that appears in the stats - would prefer if it wasn't my full fqdn etc.