#archiveteam 2017-03-03,Fri

↑back Search

Time Nickname Message
00:04 🔗 odemg has joined #archiveteam
00:06 🔗 VADemon TMM: meanwhile your robots.txt does not define it (on the main domain)?
00:07 🔗 TMM VADemon, I only just added one today as a kind of kneejerk to the issue
00:07 🔗 TMM VADemon, if archivebot had supported it I'd have done something a little nicer
00:08 🔗 VADemon Archivebot jobs are run manually by people issuing the command. Can someone add a reminder to #archivebot to be merciful with sleep times for crawling?
00:09 🔗 odemg has quit IRC (Remote host closed the connection)
00:09 🔗 TMM VADemon, I realize they are ran manually, but if there's a robots.txt with a crawl-delay it'd be nice if it could honor it
00:12 🔗 xmc default crawl params for archivebot is 3 threads, fetching links with an aggregate target of 250-375 between requests on average
00:12 🔗 xmc er, ms
00:12 🔗 TMM yeah, some of those changesets take 20+ seconds to generate
00:12 🔗 TMM someone put a whole firefox sourcetree on there :)
00:13 🔗 xmc that'll do it
00:13 🔗 xmc most websites are less cpu-intensive than yours
00:21 🔗 Stilett0 has joined #archiveteam
00:39 🔗 travisbel has joined #archiveteam
00:40 🔗 travisbel Anyone here familiar with the IMDB archive (https://archive.org/details/archiveteam_imdb)?
00:46 🔗 pnJay has joined #archiveteam
00:46 🔗 travisbel has quit IRC (Quit: Page closed)
00:46 🔗 travisbel has joined #archiveteam
00:46 🔗 pnJay has quit IRC (Client Quit)
00:46 🔗 travisbel has quit IRC (Client Quit)
00:51 🔗 pnJay has joined #archiveteam
00:51 🔗 odemg has joined #archiveteam
00:54 🔗 SketchCow Travis, you are going to get into so much trouble, you know that right
00:56 🔗 Aranje has quit IRC (Ping timeout: 506 seconds)
00:58 🔗 SketchCow Do you want to know your next 60 days or do you want it spoiled
01:13 🔗 lesderid has quit IRC (Ping timeout: 260 seconds)
01:14 🔗 Ymgve has quit IRC (Ping timeout: 506 seconds)
01:16 🔗 Silvan has quit IRC (Read error: Connection reset by peer)
01:18 🔗 SilSte has joined #archiveteam
01:19 🔗 Meeh has quit IRC (Remote host closed the connection)
01:19 🔗 Meeh has joined #archiveteam
01:20 🔗 Meeh has quit IRC (Remote host closed the connection)
01:20 🔗 Meeh has joined #archiveteam
01:22 🔗 Meeh has quit IRC (Remote host closed the connection)
01:22 🔗 Meeh has joined #archiveteam
01:22 🔗 icedice has quit IRC (Quit: Leaving)
01:23 🔗 Meeh has quit IRC (Remote host closed the connection)
01:23 🔗 Meeh has joined #archiveteam
01:24 🔗 Meeh has quit IRC (Remote host closed the connection)
01:29 🔗 lesderid has joined #archiveteam
01:35 🔗 VADemon has quit IRC (Quit: left4dead)
01:45 🔗 Meeh has joined #archiveteam
01:45 🔗 Ymgve has joined #archiveteam
01:47 🔗 Meeh has quit IRC (Remote host closed the connection)
01:47 🔗 Meeh has joined #archiveteam
01:47 🔗 Meeh has quit IRC (Remote host closed the connection)
01:47 🔗 Meeh has joined #archiveteam
01:48 🔗 Meeh has quit IRC (Remote host closed the connection)
01:50 🔗 Meeh has joined #archiveteam
01:50 🔗 Meeh has quit IRC (Remote host closed the connection)
01:50 🔗 Meeh has joined #archiveteam
01:51 🔗 Meeh has quit IRC (Remote host closed the connection)
01:54 🔗 Meeh has joined #archiveteam
01:54 🔗 Meeh has quit IRC (Remote host closed the connection)
01:55 🔗 Meeh has joined #archiveteam
02:05 🔗 j08nY has quit IRC (Quit: Leaving)
02:08 🔗 HCross2 has quit IRC (Quit: Connection closed for inactivity)
02:09 🔗 pnJay has quit IRC (Leaving)
02:13 🔗 namespace has joined #archiveteam
02:16 🔗 Stilett0 has quit IRC (Read error: Operation timed out)
02:23 🔗 ndiddy has joined #archiveteam
02:33 🔗 TMM has quit IRC (Ping timeout: 255 seconds)
02:35 🔗 TMM has joined #archiveteam
02:49 🔗 TMM has quit IRC (Ping timeout: 255 seconds)
02:50 🔗 TMM has joined #archiveteam
03:11 🔗 Stilett0 has joined #archiveteam
04:18 🔗 ndiddy has quit IRC (Read error: Connection reset by peer)
04:42 🔗 pizzaiolo has left
04:42 🔗 maelstrom has quit IRC (Quit: Leaving)
05:08 🔗 Sk1d has quit IRC (Ping timeout: 194 seconds)
05:18 🔗 squires has joined #archiveteam
05:22 🔗 HCross2 has joined #archiveteam
05:49 🔗 topdownji has quit IRC (Remote host closed the connection)
05:50 🔗 topdownji has joined #archiveteam
06:35 🔗 SketchCow P.S. FOS has been moving (not WARCing, moving) items out of the imdb dropbox for a week straight
07:01 🔗 SilSte has quit IRC (Read error: Connection reset by peer)
07:03 🔗 SilSte has joined #archiveteam
07:06 🔗 kris33 has joined #archiveteam
07:34 🔗 no2pencil has joined #archiveteam
07:34 🔗 no2pencil SketchCow: sending you good thoughts man, read about your ordeal on Twitter, sorry man.
07:34 🔗 schbirid has joined #archiveteam
07:54 🔗 QBcrusher has quit IRC (Read error: Connection reset by peer)
08:04 🔗 odemg has quit IRC (Remote host closed the connection)
08:05 🔗 kyounko has quit IRC (Ping timeout: 246 seconds)
08:06 🔗 SketchCow I'm fiiiiine
08:26 🔗 BlueMaxim has quit IRC (Read error: Operation timed out)
08:30 🔗 ravetcofx has quit IRC (Ping timeout: 245 seconds)
09:07 🔗 atomotic has joined #archiveteam
09:09 🔗 bwn has quit IRC (Read error: Operation timed out)
09:12 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
09:31 🔗 TMM has quit IRC (Quit: Ex-Chat)
10:18 🔗 ZexaronS has quit IRC (Quit: Leaving)
10:24 🔗 TMM has joined #archiveteam
10:25 🔗 jspiros has quit IRC (Read error: Operation timed out)
10:26 🔗 SadDM has quit IRC (Read error: Operation timed out)
10:50 🔗 gui7 has joined #archiveteam
10:50 🔗 gui7 can we make sure this stuff is saved? :) https://www.reddit.com/r/learnprogramming/comments/5x3ihz/uc_berkeley_to_remove_free_lecture_videos_from/?st=iztp3ugy&sh=3e8552dc
10:51 🔗 tobbez gui7: #berklost
10:52 🔗 gui7 thanks
10:53 🔗 atomotic has joined #archiveteam
11:00 🔗 eightfold has joined #archiveteam
11:04 🔗 eightfold has quit IRC (Ping timeout: 260 seconds)
11:08 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
11:26 🔗 jspiros has joined #archiveteam
11:26 🔗 SadDM has joined #archiveteam
11:26 🔗 swebb sets mode: +o SadDM
11:30 🔗 pizzaiolo has joined #archiveteam
11:39 🔗 atomotic has joined #archiveteam
11:43 🔗 atomotic has quit IRC (Client Quit)
12:00 🔗 bwn has joined #archiveteam
12:23 🔗 TMM has quit IRC (Quit: Ex-Chat)
12:24 🔗 LastNinja has joined #archiveteam
12:25 🔗 icedice has joined #archiveteam
12:32 🔗 bwn has quit IRC (Read error: Operation timed out)
12:40 🔗 Silvan has joined #archiveteam
12:41 🔗 SilSte has quit IRC (Read error: Operation timed out)
12:52 🔗 bwn has joined #archiveteam
13:15 🔗 gibigiana has quit IRC (Ping timeout: 370 seconds)
13:15 🔗 gibigiana has joined #archiveteam
13:17 🔗 bwn has quit IRC (Read error: Operation timed out)
13:44 🔗 khaoohs has joined #archiveteam
14:09 🔗 pnJay has joined #archiveteam
14:17 🔗 Gfy has quit IRC (Ping timeout: 250 seconds)
14:54 🔗 yonatan has joined #archiveteam
15:20 🔗 Boppen has joined #archiveteam
15:57 🔗 C4K3 has joined #archiveteam
15:58 🔗 yonatan has quit IRC (Quit: Page closed)
16:00 🔗 C4K3_ has quit IRC (Ping timeout: 250 seconds)
16:18 🔗 gui7 has quit IRC (Ping timeout: 250 seconds)
16:31 🔗 Gfy has joined #archiveteam
16:37 🔗 bwn has joined #archiveteam
17:03 🔗 Aranje has joined #archiveteam
17:04 🔗 gui7 has joined #archiveteam
17:10 🔗 fie has quit IRC (Ping timeout: 244 seconds)
17:15 🔗 mls has joined #archiveteam
17:28 🔗 fie has joined #archiveteam
17:34 🔗 Smiley has quit IRC (Quit: Lost terminal)
17:34 🔗 ravetcofx has joined #archiveteam
17:34 🔗 Smiley has joined #archiveteam
17:48 🔗 odemg has joined #archiveteam
18:26 🔗 mundus201 Hi, I know this isnt the right channel but im not getting much support elsewhere, Im tryinng to run the warrior with docker but docker is saying that it cannot connect to the daemon. Ive tried restarting the docker service, reinstalling it, killing it, and deleting the pid but I cant seem to get it to work, this is what it says when I try docker daemon -D, http://pastebin.com/raw/B7wCC9r0
18:29 🔗 joepie91 mundus201: sounds like a Docker problem
18:30 🔗 mundus201 hmm looks like when I stopped the service I was able to start the daemon
18:31 🔗 mundus201 but still have issues connecting
18:50 🔗 VADemon has joined #archiveteam
18:53 🔗 mundus201 yeah im dumb, I add to add my user to the docker group
19:04 🔗 macker docker needs to run as root?
19:05 🔗 mundus201 Yeah
19:05 🔗 macker guess it makes sense, Ive only p[layed around
19:05 🔗 macker sorry I cant help
19:05 🔗 mundus201 Ive got it fixed now
19:05 🔗 mundus201 now im having issues with my warrior
19:05 🔗 mundus201 "Error communicating with tracker: 404 Client Error: No items available currently. Don't worry; more items will be made soon. Try again later. for url: http://tracker.archiveteam.org:1337/api/get."
19:22 🔗 Morbus has joined #archiveteam
19:24 🔗 ThisAsYou mundus201: I usually run docker with sudo unless it's a dev machine.
19:37 🔗 mls mundus201: urlteam can be like that, items will come your way
19:37 🔗 mundus201 ah okay
19:38 🔗 mundus201 This is my first time running the warrior
20:07 🔗 QBcrusher has joined #archiveteam
20:19 🔗 icedice https://torrentfreak.com/torrent-legend-mininova-will-shut-down-for-good-170226/
20:20 🔗 icedice ^ any plans for this?
20:33 🔗 redlob has quit IRC (Read error: Operation timed out)
20:34 🔗 redlob has joined #archiveteam
20:35 🔗 nightpool icedice: we did some archivebot grabs of the public content.
20:35 🔗 icedice Ok
20:35 🔗 nightpool Althought it doesn't look like they went all the way through.... hm.
20:37 🔗 peep_ has joined #archiveteam
20:37 🔗 peep_ has quit IRC (Client Quit)
20:44 🔗 anonymoos has joined #archiveteam
20:46 🔗 nightpool icedice: looking at the logs, we were IP banned for requesting too fast, so we're re-running the grab much slower, and we'll see how it goes
20:47 🔗 icedice ok
20:56 🔗 bRick5772 has joined #archiveteam
21:18 🔗 redlob has quit IRC (Read error: Operation timed out)
21:20 🔗 REiN^ has quit IRC (Max SendQ exceeded)
21:20 🔗 REiN^ has joined #archiveteam
21:25 🔗 redlob has joined #archiveteam
21:33 🔗 icedice has quit IRC (Read error: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac)
21:34 🔗 icedice has joined #archiveteam
21:36 🔗 j08nY has joined #archiveteam
21:45 🔗 odemg has quit IRC (Remote host closed the connection)
21:45 🔗 odemg has joined #archiveteam
22:02 🔗 pnJay has quit IRC (Quit: Page closed)
22:07 🔗 REiN^ has quit IRC (Max SendQ exceeded)
22:07 🔗 _desu___ has quit IRC (Max SendQ exceeded)
22:11 🔗 REiN^ has joined #archiveteam
22:16 🔗 pnJay has joined #archiveteam
23:01 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
23:05 🔗 bRick5772 has quit IRC (Quit: Leaving.)
23:10 🔗 Ravenloft has joined #archiveteam
23:13 🔗 dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
23:15 🔗 dashcloud has joined #archiveteam
23:20 🔗 odemg has quit IRC (Remote host closed the connection)
23:24 🔗 BlueMaxim has joined #archiveteam
23:25 🔗 Asparagir has quit IRC (Asparagir)
23:28 🔗 dashcloud has quit IRC (Read error: Operation timed out)
23:29 🔗 odemg has joined #archiveteam
23:32 🔗 dashcloud has joined #archiveteam
23:47 🔗 odemg has quit IRC (Remote host closed the connection)
23:50 🔗 odemg has joined #archiveteam
23:52 🔗 eightfold has joined #archiveteam
23:59 🔗 BlueMaxim has quit IRC (Quit: Leaving)

irclogger-viewer