| Time |
Nickname |
Message |
|
00:01
🔗
|
|
chfoo has quit IRC (chfoo) |
|
00:07
🔗
|
|
chfoo has joined #archiveteam |
|
00:25
🔗
|
|
kyan has quit IRC (Ping timeout: 258 seconds) |
|
00:37
🔗
|
|
Start has joined #archiveteam |
|
00:51
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
00:53
🔗
|
|
signius has quit IRC (Read error: Operation timed out) |
|
00:55
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
01:02
🔗
|
|
signius has joined #archiveteam |
|
01:20
🔗
|
|
RedType has quit IRC (Quit: leaving) |
|
01:20
🔗
|
|
RedType has joined #archiveteam |
|
01:32
🔗
|
|
username1 has joined #archiveteam |
|
01:34
🔗
|
|
schbirid2 has quit IRC (Read error: Operation timed out) |
|
01:38
🔗
|
|
RedType has quit IRC (Quit: womp) |
|
01:38
🔗
|
|
RedType has joined #archiveteam |
|
01:42
🔗
|
|
mistym has joined #archiveteam |
|
02:08
🔗
|
|
kyan has joined #archiveteam |
|
02:38
🔗
|
|
Ymgve has quit IRC () |
|
03:12
🔗
|
|
ikreymer has quit IRC (Remote host closed the connection) |
|
03:15
🔗
|
|
dcmorton has quit IRC (Quit: ZNC - http://znc.in) |
|
03:18
🔗
|
|
dcmorton has joined #archiveteam |
|
03:38
🔗
|
|
VADemon has quit IRC (Quit: left4dead) |
|
03:38
🔗
|
|
tsp_ has quit IRC (Ping timeout: 606 seconds) |
|
03:39
🔗
|
|
tsp_ has joined #archiveteam |
|
03:44
🔗
|
|
ikreymer has joined #archiveteam |
|
04:19
🔗
|
|
antomatic has joined #archiveteam |
|
04:20
🔗
|
|
aNthraXx has quit IRC (Read error: Operation timed out) |
|
04:20
🔗
|
|
antomati_ has quit IRC (Read error: Operation timed out) |
|
04:20
🔗
|
|
Sk1d has quit IRC (Read error: Operation timed out) |
|
04:21
🔗
|
|
Sk1d has joined #archiveteam |
|
04:21
🔗
|
|
cadbury_ has quit IRC (Read error: Operation timed out) |
|
04:22
🔗
|
|
brayden has quit IRC (Read error: Operation timed out) |
|
04:22
🔗
|
|
aaaaaaaaa has quit IRC (Leaving) |
|
04:22
🔗
|
|
caber has quit IRC (Read error: Operation timed out) |
|
04:24
🔗
|
|
ikreymer_ has joined #archiveteam |
|
04:25
🔗
|
|
caber has joined #archiveteam |
|
04:26
🔗
|
|
ikreymer has quit IRC (Read error: Operation timed out) |
|
04:26
🔗
|
|
cadbury_ has joined #archiveteam |
|
04:31
🔗
|
|
aNthraXx has joined #archiveteam |
|
04:34
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
04:35
🔗
|
|
aNthraXx has quit IRC (Read error: Operation timed out) |
|
04:36
🔗
|
|
cadbury_ has quit IRC (Read error: Operation timed out) |
|
04:41
🔗
|
|
cadbury_ has joined #archiveteam |
|
04:42
🔗
|
|
brayden has joined #archiveteam |
|
04:42
🔗
|
|
aNthraXx has joined #archiveteam |
|
04:52
🔗
|
|
useretail has quit IRC (Dreaming in digital. Living in real-time. Thinking in binary. Talking in IP.) |
|
05:02
🔗
|
|
mistym has joined #archiveteam |
|
05:11
🔗
|
|
kyan has quit IRC (Quit: Leaving) |
|
05:15
🔗
|
|
mrfoo has joined #archiveteam |
|
05:18
🔗
|
|
ikreymer_ has quit IRC () |
|
05:43
🔗
|
|
hive-mind has quit IRC (Ping timeout: 252 seconds) |
|
05:43
🔗
|
|
wp494 has quit IRC (Ping timeout: 740 seconds) |
|
06:04
🔗
|
|
wp494 has joined #archiveteam |
|
06:08
🔗
|
|
hive-mind has joined #archiveteam |
|
06:40
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
07:37
🔗
|
|
primus104 has joined #archiveteam |
|
07:41
🔗
|
|
mistym has joined #archiveteam |
|
07:51
🔗
|
|
mistym has quit IRC (Ping timeout: 483 seconds) |
|
07:53
🔗
|
|
wp494 has quit IRC (LOUD UNNECESSARY QUIT MESSAGES) |
|
07:54
🔗
|
|
wp494 has joined #archiveteam |
|
08:40
🔗
|
|
atomotic has joined #archiveteam |
|
08:57
🔗
|
|
marvinw has quit IRC (Ping timeout: 252 seconds) |
|
09:04
🔗
|
|
atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) |
|
09:09
🔗
|
|
marvinw has joined #archiveteam |
|
09:49
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
10:33
🔗
|
|
Rotab has quit IRC (Ping timeout: 198 seconds) |
|
10:35
🔗
|
|
Rotab has joined #archiveteam |
|
11:20
🔗
|
|
mistym has joined #archiveteam |
|
11:24
🔗
|
|
mistym has quit IRC (Ping timeout: 252 seconds) |
|
11:44
🔗
|
|
Ymgve has joined #archiveteam |
|
12:24
🔗
|
|
username1 is now known as schbirid |
|
12:29
🔗
|
|
wp494 has quit IRC (Ping timeout: 740 seconds) |
|
12:40
🔗
|
BlueMaxim |
https://www.kickstarter.com/projects/2146199819/zx-spectrum-a-visual-compendium-by-bitmap-books all my money. all of it. |
|
12:42
🔗
|
|
sankin has joined #archiveteam |
|
12:47
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
|
13:12
🔗
|
|
primus104 has joined #archiveteam |
|
13:13
🔗
|
ersi |
Oh jeez |
|
13:13
🔗
|
* |
ersi wipes mouth |
|
13:41
🔗
|
SketchCow |
----------------------------- |
|
13:41
🔗
|
SketchCow |
REMINDER: FOS GOES DOWN FOR AN HOUR OR TWO |
|
13:41
🔗
|
SketchCow |
MAIN SYSTEM IS GETTING SOME UPGRADES |
|
13:42
🔗
|
SketchCow |
SHOULD RETURN IN NO TIME AND WILL RESUME |
|
13:42
🔗
|
SketchCow |
----------------------------- |
|
13:42
🔗
|
SketchCow |
Time to down - about 3 hours |
|
13:57
🔗
|
|
useretail has joined #archiveteam |
|
14:02
🔗
|
|
philpem has joined #archiveteam |
|
14:09
🔗
|
|
mistym has joined #archiveteam |
|
14:22
🔗
|
|
mistym has quit IRC (Read error: Operation timed out) |
|
14:31
🔗
|
|
mistym has joined #archiveteam |
|
14:48
🔗
|
|
primus has joined #archiveteam |
|
15:07
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
15:24
🔗
|
|
Stiletto has quit IRC () |
|
15:35
🔗
|
|
Wilki has joined #archiveteam |
|
15:35
🔗
|
Wilki |
hello |
|
15:35
🔗
|
SketchCow |
Who is uploading Atari books to the FTP dump? |
|
15:38
🔗
|
Wilki |
!a http://rethinkvps.com/ |
|
15:40
🔗
|
Wilki |
It looks like rethinkvps.com might be going under, whats the possibility of getting it archived for future reference? |
|
15:40
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
15:42
🔗
|
|
Wilki has quit IRC (Quit: Page closed) |
|
15:49
🔗
|
SketchCow |
What part of it? |
|
15:54
🔗
|
|
[Beta] has quit IRC (Ping timeout: 240 seconds) |
|
15:55
🔗
|
|
aaaaaaaaa has joined #archiveteam |
|
16:12
🔗
|
|
Stiletto has joined #archiveteam |
|
16:23
🔗
|
|
t4nk391 has joined #archiveteam |
|
16:24
🔗
|
t4nk391 |
hey is anyone active here? |
|
16:24
🔗
|
t4nk391 |
atm? |
|
16:28
🔗
|
|
mistym has joined #archiveteam |
|
16:40
🔗
|
schbirid |
things not to ask on irc #3 |
|
16:41
🔗
|
ohhdemgir |
can I ask a question? |
|
16:42
🔗
|
ohhdemgir |
I've got a question, is anyone here |
|
16:42
🔗
|
t4nk391 |
yup |
|
16:42
🔗
|
t4nk391 |
im here fire away |
|
16:42
🔗
|
ohhdemgir |
XD t4nk391 I'm just taking the piss cc schbirid |
|
16:42
🔗
|
ersi |
Jeez, just ask your question instead of asking to ask a question |
|
16:42
🔗
|
ohhdemgir |
ersi, i know right |
|
16:54
🔗
|
|
SimpBrain has joined #archiveteam |
|
17:03
🔗
|
aaaaaaaaa |
for some reason that made me think of https://www.youtube.com/watch?v=rEmTMkx7SLE |
|
17:03
🔗
|
|
philpem has quit IRC (Ping timeout: 252 seconds) |
|
17:05
🔗
|
aaaaaaaaa |
interesting fact, that was the first hit for "pink floyd is there anybody out there" rather than the song of the same name by them. |
|
17:08
🔗
|
Nemo_bis |
Playing Elastinen ft asa viidest ysiin-OUKbnDMHBXE.m4a. |
|
17:08
🔗
|
Nemo_bis |
whoops paste fail |
|
17:09
🔗
|
edsu_ |
https://www.youtube.com/watch?v=U13xOvDa19U |
|
17:10
🔗
|
t4nk391 |
this better be good xD |
|
17:11
🔗
|
edsu_ |
oh yeah |
|
17:12
🔗
|
aaaaaaaaa |
that works amazingly well |
|
17:13
🔗
|
edsu_ |
strange right? |
|
17:14
🔗
|
edsu_ |
erm, yeah |
|
17:14
🔗
|
edsu_ |
wrong channel :-) |
|
17:14
🔗
|
|
edsu_ is now known as edsu |
|
17:14
🔗
|
t4nk391 |
there are some good mashups out there but there are also bad ones |
|
17:14
🔗
|
yipdw |
=> #-bs |
|
17:18
🔗
|
SketchCow |
Is anyone active here? |
|
17:18
🔗
|
SketchCow |
Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | Is anyone active here? |
|
17:19
🔗
|
xmc |
---- |
|
17:19
🔗
|
t4nk391 |
whats the chance of a website kicking me if i crawl 644340 pages? trying to backup a site |
|
17:19
🔗
|
xmc |
no |
|
17:19
🔗
|
|
SketchCow changes topic to: Archive Team: We're not archive.org | http://archiveteam.org/ | lengthy/off-topic in #archiveteam-bs | Is anyone active here? |
|
17:19
🔗
|
xmc |
---- |
|
17:19
🔗
|
xmc |
t4nk391: depends on who runs it and what they decide |
|
17:20
🔗
|
aaaaaaaaa |
yeah, some crawls get into the millions of pages with no problems some only get a couple hundred before the hammer falls. |
|
17:20
🔗
|
|
signius has quit IRC (Read error: Operation timed out) |
|
17:21
🔗
|
t4nk391 |
I think the site was made in 2004, so i should be good until they check there logs |
|
17:22
🔗
|
t4nk391 |
does it make any difference if they use cloudflare? would they kick me / ban IP? |
|
17:23
🔗
|
xmc |
ah, now we're approaching an answerable question |
|
17:23
🔗
|
xmc |
i don't think cloudflare does anti-crawling |
|
17:23
🔗
|
xmc |
do they pay for cloudflare? |
|
17:24
🔗
|
t4nk391 |
i dont think so, they are having financial trouble. might shutdown on monday hense the crawl |
|
17:24
🔗
|
xmc |
go for it |
|
17:25
🔗
|
xmc |
cloudflare only does ddos protection on paid accounts, so if it did anti-crawling then it'd also only be for paid accounts |
|
17:26
🔗
|
t4nk391 |
@xmc: legend - thanks for the replies |
|
17:26
🔗
|
xmc |
sure thing |
|
17:27
🔗
|
xmc |
do you need pointers on how to do a high-quality site crawl? |
|
17:27
🔗
|
t4nk391 |
btw its a (crappy) private torrent tracker and i have 3 invites if anyone wants (bitsoup.me) |
|
17:28
🔗
|
t4nk391 |
pointers? not really, i have so far logged in with my script |
|
17:28
🔗
|
xmc |
ok |
|
17:29
🔗
|
xmc |
if you're comfortable writing lua code to make the crawler happen, we recommend using wpull to do the bulk of the work |
|
17:29
🔗
|
xmc |
wpull lets you write site-specific lua scripts and does most of the rest of things itself |
|
17:29
🔗
|
xmc |
if you're interested: https://github.com/chfoo/wpull |
|
17:30
🔗
|
t4nk391 |
oh nice, I wish i knew lua, using curl with php atm |
|
17:30
🔗
|
xmc |
lua's pretty straightforward |
|
17:31
🔗
|
xmc |
:) |
|
17:31
🔗
|
xmc |
here is, for example, a lua script to archive a phpbb3 forum https://github.com/ArchiveTeam/wget-lua-forum-scripts/blob/master/phpbb3.lua |
|
17:31
🔗
|
t4nk391 |
I have thought about using lua on top of C/C++ to make a game |
|
17:32
🔗
|
RedType |
that is a common use for lua |
|
17:32
🔗
|
yipdw |
Cloudflare does have crawler detection |
|
17:33
🔗
|
yipdw |
or at least they offer it |
|
17:33
🔗
|
|
signius has joined #archiveteam |
|
17:33
🔗
|
yipdw |
ArchiveBot hits that occasionally so you will want to watch your WARCs |
|
17:33
🔗
|
xmc |
hm, ok, it sounds like a thing that they would charge for |
|
17:34
🔗
|
t4nk391 |
oh ok, so use my vps, when crawling so i don't get a bad rep with cloudflare |
|
17:34
🔗
|
yipdw |
there seems to be at least one Cloudflare employee that has starred the ArchiveBot repo. I'm not sure what that implies |
|
17:34
🔗
|
xmc |
huh. |
|
17:35
🔗
|
yipdw |
t4nk391: we also have crawlers here, so unless you have a custom job and/or want to tinker around we can also run a crawl |
|
17:37
🔗
|
|
primus104 has joined #archiveteam |
|
17:37
🔗
|
t4nk391 |
@yipdw: ill keep that in mind |
|
17:37
🔗
|
xmc |
yipdw: sounds like site needs to be logged in to crawl |
|
17:37
🔗
|
yipdw |
oh whee |
|
17:37
🔗
|
xmc |
so technically speaking it's out of scope for archiveteam proper |
|
17:37
🔗
|
yipdw |
yes |
|
17:37
🔗
|
xmc |
being a private tracker |
|
17:37
🔗
|
t4nk391 |
@xmc i've covered that in my script, it also needs https |
|
17:37
🔗
|
* |
xmc nods |
|
17:38
🔗
|
xmc |
t4nk391: in irc you don't need to put an @ before someone's name to mention them. you can usually just type the first few characters of their name and press <tab> |
|
17:39
🔗
|
t4nk391 |
xmc: im just copying and pasting also using the browser clieny |
|
17:39
🔗
|
* |
xmc nods |
|
17:39
🔗
|
xmc |
no worries |
|
17:44
🔗
|
t4nk391 |
Whats an easy way to store HTML for mining later? |
|
17:44
🔗
|
yipdw |
WARC |
|
17:45
🔗
|
yipdw |
we should take this to #archiveteam-bs |
|
17:45
🔗
|
xmc |
imo here is fine, but i'm not going to stop you |
|
17:46
🔗
|
yipdw |
=> #-bs is an archiveteam meme |
|
17:46
🔗
|
t4nk391 |
should i stay or should i go? |
|
18:08
🔗
|
ersi |
yes |
|
18:09
🔗
|
xmc |
ersi is, as always, helpful |
|
18:09
🔗
|
ersi |
\:D/ |
|
18:10
🔗
|
xmc |
what a strange gesture |
|
18:14
🔗
|
|
Nystrom has joined #archiveteam |
|
18:16
🔗
|
t4nk391 |
Well im off to bed (6:19am here), have a good day/night all :D |
|
18:20
🔗
|
|
t4nk391 has quit IRC (Quit: Page closed) |
|
18:24
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
18:38
🔗
|
xmc |
gitorious rsync: http://zeppelin.xrtc.net/corp.xrtc.net/karbovanets.corp.xrtc.net/if_eth0.html |
|
18:38
🔗
|
xmc |
eta 10 days |
|
18:41
🔗
|
xmc |
same graph, different machine https://vpssd.com/munin/vpssd.com/jimmy.vpssd.com/if_p2p1_10.html |
|
18:56
🔗
|
|
Zebranky has quit IRC (Remote host closed the connection) |
|
18:57
🔗
|
xmc |
i'd like to stress that this is the first time I can remember since archiveteam started that someone has actually come to us proactively |
|
18:57
🔗
|
xmc |
since then i have graduated university, moved three times, and had four different jobs |
|
18:58
🔗
|
SketchCow |
-------------------------- |
|
18:58
🔗
|
SketchCow |
FOS.TEXTFILES.COM is back |
|
18:58
🔗
|
SketchCow |
-------------------------- |
|
18:58
🔗
|
xmc |
woooo |
|
18:59
🔗
|
SketchCow |
I'll get the crap going later today |
|
19:00
🔗
|
|
Nertsy has quit IRC (Read error: Operation timed out) |
|
19:01
🔗
|
xmc |
SketchCow: halo is still on hold, iirc, do you want to restart it sometime |
|
19:27
🔗
|
|
Nertsy has joined #archiveteam |
|
19:27
🔗
|
|
primus104 has joined #archiveteam |
|
19:34
🔗
|
|
Zebranky has joined #archiveteam |
|
19:35
🔗
|
|
scyther has joined #archiveteam |
|
19:50
🔗
|
|
lytv has quit IRC (Max SendQ exceeded) |
|
19:50
🔗
|
|
Fusl has quit IRC (Max SendQ exceeded) |
|
19:53
🔗
|
|
lytv has joined #archiveteam |
|
19:55
🔗
|
|
SN4T14_ has joined #archiveteam |
|
19:55
🔗
|
|
Fusl has joined #archiveteam |
|
20:00
🔗
|
|
SN4T14__ has quit IRC (Ping timeout: 369 seconds) |
|
20:03
🔗
|
|
SimpBrain has quit IRC (Quit: Leaving) |
|
20:44
🔗
|
|
sankin has quit IRC (Leaving.) |
|
20:44
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
20:50
🔗
|
|
SimpBrain has joined #archiveteam |
|
20:59
🔗
|
|
scyther has quit IRC (Leaving) |
|
21:18
🔗
|
|
philpem has joined #archiveteam |
|
21:25
🔗
|
|
Rotab has quit IRC (Read error: Connection reset by peer) |
|
21:37
🔗
|
SketchCow |
Yeah |
|
21:37
🔗
|
SketchCow |
I wonder what's filling it - lots of halo still coming in |
|
21:40
🔗
|
|
SimpBrain has quit IRC (Quit: Leaving) |
|
21:47
🔗
|
|
mistym has joined #archiveteam |
|
21:49
🔗
|
|
Rotab has joined #archiveteam |
|
21:51
🔗
|
|
pwnsrv has quit IRC (Ping timeout: 265 seconds) |
|
21:52
🔗
|
xmc |
hm, or is halo not on hold |
|
21:52
🔗
|
SketchCow |
I think we turned it on a little. |
|
21:53
🔗
|
xmc |
ok yeah it's letting thru 5 items per minute |
|
22:12
🔗
|
|
pwnsrv has joined #archiveteam |
|
22:38
🔗
|
|
mistym_ has joined #archiveteam |
|
22:44
🔗
|
|
Emcy has quit IRC (Ping timeout: 512 seconds) |
|
22:44
🔗
|
|
mistym has quit IRC (Read error: Operation timed out) |
|
22:55
🔗
|
|
wp494 has joined #archiveteam |
|
23:11
🔗
|
|
Mayonaise has quit IRC (Ping timeout: 362 seconds) |
|
23:13
🔗
|
|
RedType has quit IRC (Quit: leaving) |
|
23:14
🔗
|
|
RedType has joined #archiveteam |
|
23:14
🔗
|
|
sirdancea has joined #archiveteam |
|
23:16
🔗
|
|
RedType has quit IRC (Client Quit) |
|
23:26
🔗
|
|
lysobit has quit IRC (Ping timeout: 512 seconds) |
|
23:27
🔗
|
|
BlueMaxim has joined #archiveteam |
|
23:27
🔗
|
|
RedType has joined #archiveteam |
|
23:28
🔗
|
|
lysobit has joined #archiveteam |
|
23:36
🔗
|
|
Mayonaise has joined #archiveteam |
|
23:46
🔗
|
|
nertzy has joined #archiveteam |