[00:01] *** zenguy_pc has joined #archiveteam [00:05] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [00:15] *** segundo2 has joined #archiveteam [00:16] *** primus104 has quit IRC (Leaving.) [00:20] *** segundo2 has quit IRC (Ping timeout: 240 seconds) [00:22] *** zenguy_pc has joined #archiveteam [00:25] *** rduser has quit IRC (Read error: Operation timed out) [00:32] *** rduser has joined #archiveteam [00:38] *** Wyatts has quit IRC (Remote host closed the connection) [00:39] *** Wyatts has joined #archiveteam [00:41] *** BlueMaxim has joined #archiveteam [00:49] *** superkuh_ is now known as superkuh [01:11] *** Fusl has quit IRC (Ping timeout: 240 seconds) [01:20] *** Aranje has quit IRC (Quit: Three sheets to the wind) [01:32] *** aaaaaaaaa has quit IRC (Leaving) [01:51] *** dashcloud has quit IRC (Read error: Connection reset by peer) [01:52] *** dashcloud has joined #archiveteam [02:26] boop [02:26] lotta questions [02:28] hi, that musictube item is spam bullshit [02:29] Don't upload spam, problwm solved. [02:31] so i mistakenly deleted 1995-12-01 to 1995-12-16 of mbc newsdesk before uploading [02:32] i'm redownloading them now [02:32] it will take over a hour [03:44] *** Froggypwn has quit IRC (Ping timeout: 252 seconds) [03:45] *** Froggypwn has joined #archiveteam [04:21] *** JesseW has joined #archiveteam [04:28] *** chfoo0 has joined #archiveteam [04:30] *** chfoo has quit IRC (Ping timeout: 306 seconds) [04:31] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [04:41] *** chfoo0 is now known as chfoo [04:55] *** Fletcher has quit IRC (Ping timeout: 252 seconds) [04:56] *** diacope has quit IRC (Ping timeout: 252 seconds) [05:00] *** scyther has joined #archiveteam [05:07] *** zenguy_pc has joined #archiveteam [05:08] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [05:13] *** scyther has quit IRC (Quit: Leaving) [05:14] *** diacope has joined #archiveteam [05:25] *** zenguy_pc has joined #archiveteam [05:27] *** Fletcher has joined #archiveteam [06:09] *** Fusl has joined #archiveteam [06:29] *** Fusl has quit IRC (Contact: http://hallowe.lt/) [06:30] *** JesseW has quit IRC (Read error: Operation timed out) [06:30] *** Fusl has joined #archiveteam [06:31] *** phuzion has quit IRC (Read error: Operation timed out) [06:40] *** phuzion has joined #archiveteam [07:16] *** JesseW has joined #archiveteam [07:31] *** schbirid has joined #archiveteam [08:13] *** atomotic has joined #archiveteam [08:13] *** primus104 has joined #archiveteam [08:14] *** JesseW has quit IRC (Read error: Operation timed out) [08:45] *** primus104 has quit IRC (Leaving.) [08:53] *** russss__ has quit IRC (Remote host closed the connection) [08:53] *** JSharp has quit IRC (Remote host closed the connection) [08:53] *** zyphlar has quit IRC (Remote host closed the connection) [08:53] *** Boltsie has quit IRC (Remote host closed the connection) [08:56] *** arkiver2 has joined #archiveteam [08:57] *** russss__ has joined #archiveteam [09:01] *** JSharp has joined #archiveteam [09:05] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [09:08] *** arkiver2 has joined #archiveteam [09:14] *** zyphlar has joined #archiveteam [09:48] *** Boltsie has joined #archiveteam [10:29] *** primus104 has joined #archiveteam [10:33] *** RedType has quit IRC (Ping timeout: 483 seconds) [10:42] Blingee has started! [10:53] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [10:54] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [10:59] *** primus104 has quit IRC (Leaving.) [11:00] *** RedType has joined #archiveteam [11:01] *** arkiver2 has joined #archiveteam [11:03] chfoo: I think you can move project of choice to URLTeam or Blingee [11:56] *** PurpleSym has joined #archiveteam [12:00] *** scyther has joined #archiveteam [12:13] *** no2pencil has joined #archiveteam [12:17] *** primus104 has joined #archiveteam [12:28] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [12:44] *** atomotic has joined #archiveteam [12:45] *** zenguy_pc has joined #archiveteam [12:49] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [13:27] *** oli has joined #archiveteam [13:43] *** primus104 has quit IRC (Leaving.) [13:49] *** kris33 has joined #archiveteam [13:50] *** kris33 has quit IRC (Client Quit) [13:52] *** RichardG has quit IRC (Ping timeout: 252 seconds) [13:53] Upload to archive.org of Skillfeed continues its long, lonesome journey [13:56] Out of curiosity, what kinda speeds and file sizes are we talking? [14:16] FOS lives inside the network. Everything's huge and fast. [14:17] Especially with 30-40 simultaneous cconnections aren't ruining its upstream. [14:17] Are you new? [14:17] I feel like we have about six people who are new. [14:23] Moderatly new. I was here during the Sourceforge thing. It was joepie91 who got me here from LET [14:23] * joepie91 waves at HCross [14:31] * HCross waves back [14:40] *** scyther has quit IRC (Quit: Leaving) [14:42] *** BlueMaxim has quit IRC (Quit: Leaving) [14:48] Well, welcome then. [14:50] Thank you [14:50] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:49] *** primus104 has joined #archiveteam [15:50] *** JesseW has joined #archiveteam [15:57] *** lytv has quit IRC (Ping timeout: 240 seconds) [15:57] *** lytv has joined #archiveteam [16:07] *** xk_id_ has quit IRC (Remote host closed the connection) [16:19] *** JesseW has quit IRC (Read error: Operation timed out) [16:27] *** primus104 has quit IRC (Leaving.) [16:46] *** lbft has quit IRC (Read error: Operation timed out) [16:52] *** scyther has joined #archiveteam [16:58] *** lbft has joined #archiveteam [17:16] *** primus104 has joined #archiveteam [17:36] *** Lord_Nigh has quit IRC (ZNC - http://znc.in) [17:40] *** RichardG has joined #archiveteam [17:57] *** ete has joined #archiveteam [18:04] *** atomotic has joined #archiveteam [18:15] *** habi has joined #archiveteam [18:15] *** habi has left [18:16] *** xk_id has joined #archiveteam [18:22] *** limebyte has joined #archiveteam [18:22] *** nertzy has joined #archiveteam [18:22] Hey, gus i would like to help ya with indexing and getting stuff on Linux [18:23] how do i do that? [18:30] limebyte: ohai - there's lots of ways to get involved, really (also, did you come from vpsboard/lowendtalk by any chance? I think I recognize your nick...) [18:30] booth [18:30] limebyte: are you looking to contribute resources or time/work? (or both?) [18:30] heh, right :P [18:30] resources like storage/bandwidth cpu [18:30] etc [18:31] limebyte: on a server? [18:31] sure [18:31] mostly wheezy [18:32] limebyte: alright, so we have the archiveteam warrior, which automatically picks projects and does tasks for them... but you probably don't want to run that on a server, because it requires virtualbox [18:32] limebyte: so there's a few options, generally: [18:32] 1) manually running warrior scripts - when a project appears that you want to contribute to, follow the setup instructions to get it running, and it'll go off and do tasks for that project [18:33] 2) supplying an archivebot node (archivebot is an on-demand archival IRC bot, basically), but this requires a bunch of RAM (probably 2GB+?) and longer-term guaranteed availability (several months, iirc) [18:34] 3) being an rsync target server - this is occasionally needed, not really centrally announced anywhere... they function as collection servers, and occasionally we'll run out of space and need more rsync target servers; this requires a lot of disk space, ability to handle a lot of connections at once, and a lot of bandwidth (eg. for Blip we had it basically bursting a 1gbps connection for an entire weekend) [18:34] option 1 doesn't require any availability - you can just shut it down whenever needed, and it also doesn't require very many resources [18:34] usually only like 5GB of disk _at most_, a bit of RAM, and a bit of bandwidth [18:35] 2 and 3 [18:35] actually i dont know if 10TB on Gbit [18:35] and 2x2 TB is enought [18:35] for an archivebot node, if it's going to be available for a while, it certainly should be [18:35] thats not the issue [18:35] for an rsync target, it depends on the project (but I don't think we have any projects needing rsync servers right now) [18:36] paging yipdw [18:36] okay how do i setup the warrior then? [18:36] it needs virtualbox? [18:36] limebyte: the warrior, yes. if you run the scripts manually, then no - it's just a Python script [18:36] confused [18:36] limebyte: the warrior is just a Debian image that downloads and runs the scripts for you, basically :P [18:36] ah yeah, i am okay with python, should be good on a non root user [18:36] yeah [18:37] limebyte: okay, so for the warrior scripts, just pick any project from the archiveteam site frontpage that you want to contribute to: http://archiveteam.org/index.php?title=Main_Page [18:37] (they may not all be actively running) [18:37] and in the sidebar there should be a repository link [18:37] for each project [18:37] just go there, and follow the README - it has instructions for most major distros [18:37] that will set it up under an unprivileged 'archiveteam' user [18:37] so you can pretty much just copypaste them and enter your own nickname [18:37] hmmm [18:38] manual setup each project? [18:38] nice would be something like automated [18:38] limebyte: yeah - that's what the warrior normally does, but we don't have a non-virtualbox version of that (and it only runs one at a time) [18:38] I'm working on it [18:38] :p [18:38] why virtualbox based? [18:38] limebyte: okay, looks like Blingee is still running [18:38] as is urlteam [18:39] limebyte: the idea of the Warrior is to make it pretty much zero-setup for random people [18:39] to run on their desktop, etc [18:39] download image, load into virtualbox, start it, forget about it [18:39] but they do have like ppm limits and bw limits? [18:39] Windows doesn't have a very nice filesystem, so a Linux system is needed for it [18:39] how do you mean? [18:39] pages per minute [18:39] not like its pooling 500 per sec [18:40] that depends on the site - the scripts usually account for that [18:40] i guess does need some setup like tor? [18:40] each project has its own scripts [18:40] ah [18:40] with its own rate limits, etc [18:40] as an end user, you just need to run it :P [18:40] think of it as a voluntary botnet [18:40] a bit like SETI@Home [18:41] joepie91, you getting to deep into it [18:41] :D [18:41] lemme install python and lets see [18:41] limebyte: have a look at the instructions here: https://github.com/ArchiveTeam/blingee-grab#distribution-specific-setup [18:41] blingee is a currently active project [18:42] limebyte: and if you want to run multiple projects, then you can set up the urlteam project also: https://github.com/ArchiveTeam/terroroftinytown-client-grab#for-debianubuntu [18:42] limebyte, I have a feeling you may know who I am [18:42] _however_, for the second script, you'd have to change the address in the commands to 127.0.0.2 instead of 127.0.0.1, or they'll conflict [18:42] *** Lord_Nigh has joined #archiveteam [18:43] limebyte: and don't forget to set a nickname [18:43] :p [18:43] i will run a lot [18:43] *** balrog sets mode: +o Lord_Nigh [18:43] until HDD or CPU under load [18:43] limebyte: let me know whether you can get everything up and running [18:43] Yacy actually burning the shit out the one node [18:43] limebyte: probably won't happen with the warrior projects, at all :P [18:43] they're usually pretty light-weight [18:43] limebyte, when projects start compressing stuff then you may notice CPU [18:43] old HDD [18:43] so [18:43] yacy is killing it [18:44] okay then [18:45] limebyte: after you're done with the warrior script setup, if you can have it available for a few months at least, you'll want to talk to yipdw about setting up an archivebot node - probably best to just PM him directly [18:45] okay [18:45] he can see the uptime etc? [18:45] joepie91, I asked him the other day and he said he didnt need more [18:45] lol [18:46] limebyte, you would have to send him an SSH cert [18:46] HCross: huh? we had queue issues recently [18:46] It was recently I asked [18:46] root user or what? [18:46] limebyte: iirc archivebot still uses an SSH tunnel for talking to the master server [18:46] doesn't require access to your server though, afaik [18:49] fuck [18:49] aaaaaaa [18:49] dang [18:53] limebyte: what broke? [18:54] should run bro [18:55] i can install multiple over the same user ya? [18:56] which port does it listen? [18:56] on localhost? [18:56] limebyte: I forgot :P but you can track progress on the leaderboard [18:56] limebyte: you can set up multiple, but they will need to listen on different addresses [18:56] limebyte: http://tracker.archiveteam.org/blingee/ <- that's where the progress for blingee goes [18:57] limebyte: also, you may want to join #archiveteam-bs - this channel is usually just for short "OMG SOMETHING IS GOING DOWN" discussions, longer discussions go to -bs [18:57] just have one ip [18:57] sadly [18:57] datashitty [18:57] limebyte: local IP I mean [18:57] 127.0.0.x [18:57] the --address flag [18:57] why lol? [18:57] ? [18:57] or you do --port 8080 then --port 8081 [18:58] easy [18:58] ya [18:58] change port not ip [18:58] Ive done that on all the others [18:58] HCross: then i get to explain to people why it's conflicting with their custom daemon on 8089... ;) [18:58] --concurent? [18:59] does that mean the crawlers? [18:59] yes, dont do too many [18:59] limebyte: that's how many jobs it'll work on at the same time [18:59] hmmm [18:59] quad core idle [18:59] any suggestions? [18:59] * joepie91 just runs it at 20 on servers [18:59] but i have only 1 external ip maybe get blocked for to much jobs [18:59] good [18:59] not sure if blingee does IP throttling [19:00] eh [19:00] now it dies instant [19:00] if i try to restart after i killed the process [19:01] limebyte, take screen out of the command then you can see why it died [19:01] limebyte: oh - don't just kill the process, please. use `touch STOP` in the repository to let it stop gracefully [19:01] and what HCross said [19:01] (if you kill it, jobs will get stuck) [19:01] bad [19:01] seems like its broken now [19:02] eh [19:02] works again [19:03] weired [19:03] however its up and running now i guess [19:03] limebyte: join #archiveteam-bs? :P [19:03] right [19:03] bs = bullshit? [19:03] yeah :P [19:03] takes ages [19:04] to get up 20 workez [19:04] sucks [19:04] yep, we dont want to DDoS blingee, although I have a feeling that uni isnt liking that archivebot job ive got [19:05] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [19:05] Yeah, please move over to #archiveteam-bs [19:05] for general conversations and long-running talks that aren't strictly now-archiving-this-project-right-here-things [19:11] *** aaaaaaaaa has joined #archiveteam [19:15] What projects can I currently help with? [19:17] no2pencil: blingee and urlteam are currently active [19:17] no2pencil: see -bs :) [19:23] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [19:24] chfoo: if skillfeed doesn't undo the changes they made to their website we're done [19:24] so for now skillfeed can be removed from the json [19:28] Skillfeed uploads on FOS going swimmingly now. [19:34] Nice! [19:35] blingee has sarted again, if we're going too fast for FOS to keep up I'll slow it down [19:44] *** nertzy has joined #archiveteam [19:50] OK. [19:51] I've got blingee sorting and uploading now. [19:56] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [20:06] *** schbirid has quit IRC (Quit: Leaving) [20:23] can anyone check why this item was blocked: http://archive.org/details/GamingAnywhere [20:23] virustotal flagged it apparently [20:24] https://www.virustotal.com/file/996d221e7ac08550ce8f8c7b5d693699223b611fe6faa36a98fd06811d283da9/analysis/1370911134/ [20:25] is there a way to free-up that id? i'm the uploader [20:25] email info@archive.org [20:27] useretail: https://www.virustotal.com/en/file/996d221e7ac08550ce8f8c7b5d693699223b611fe6faa36a98fd06811d283da9/analysis/1370911134/ [20:28] the scan results [20:28] fresh results: https://www.virustotal.com/en/file/996d221e7ac08550ce8f8c7b5d693699223b611fe6faa36a98fd06811d283da9/analysis/1442867247/ [20:29] that's ridiculous [20:29] triggering some heuristics it seems [20:31] i can't upload stuff because smartest av thinks there's a virus [20:31] great [20:31] useretail: it's uploaded and stored, just not publicly accessible [20:32] uploading wasn't complete [20:32] *upload [20:34] SketchCow: how do you deal with situations like this one? [20:38] *** Jon has quit IRC (Read error: Operation timed out) [20:41] *** jmtd has joined #archiveteam [20:42] *** edsu_ has quit IRC (Read error: Operation timed out) [20:47] What. [20:48] *** scyther has quit IRC (Quit: Leaving) [20:50] SketchCow: uploading a file that is false-positively reported by virustotal [20:51] *** edsu has joined #archiveteam [20:53] undarked [20:56] thanks [20:56] *** PurpleSym has quit IRC (Remote host closed the connection) [20:56] ------------------------------------------- [20:56] Anybody here familiar with setting up a testing harness so we can do some needed JSMESS work? [20:56] ------------------------------------------- [21:14] *** nertzy has joined #archiveteam [21:48] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [23:00] http://www.publishersweekly.com/pw/by-topic/digital/content-and-e-books/article/68130-oyster-is-shutting-down-operations.html [23:01] Scripts for GameFront are ready, will test abit more tomorrow but we'll start the GameFront project very soon [23:04] balrog: will get on it [23:04] we can discovery the books through https://www.oysterbooks.com/explore [23:04] btw are there any ongoing activities for google code? [23:04] yes [23:05] code was last updated a day ago, need to add support for some more json files and then we're done [23:05] or well, ready to start [23:13] I'm not sure if we already have a channel for gamefront, but we need one [23:13] we also need a wiki page, so if someone would like to create a page for gamefront, please do :) [23:43] useretail: Our security guy does not agree with you that it's false positives. [23:44] So it's darked until further notice. [23:53] *** c_b has joined #archiveteam [23:53] SketchCow: reanalysis of the offending file component of that zip shows a detection rate of 0/56