| Time |
Nickname |
Message |
|
00:28
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
00:34
🔗
|
|
swebb has joined #archiveteam-bs |
|
00:52
🔗
|
|
mistym has joined #archiveteam-bs |
|
01:17
🔗
|
|
GLaDOS has quit IRC (Read error: Operation timed out) |
|
01:19
🔗
|
|
schbirid2 has quit IRC (Read error: Operation timed out) |
|
01:20
🔗
|
|
schbirid2 has joined #archiveteam-bs |
|
01:25
🔗
|
|
Ravenloft has joined #archiveteam-bs |
|
01:32
🔗
|
|
Coderjoe has quit IRC (Read error: Connection reset by peer) |
|
01:33
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
01:44
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
01:44
🔗
|
|
Coderjoe has joined #archiveteam-bs |
|
01:49
🔗
|
|
Start has joined #archiveteam-bs |
|
02:39
🔗
|
|
dan_ has quit IRC (Ping timeout: 252 seconds) |
|
02:48
🔗
|
|
dan_ has joined #archiveteam-bs |
|
03:47
🔗
|
|
aaaaaaaaa has quit IRC (Leaving) |
|
04:27
🔗
|
godane |
i'm going after iol.co.za website |
|
04:28
🔗
|
godane |
i can brute force the number id of the artciles |
|
04:28
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
04:28
🔗
|
godane |
they also redirect to the real links so there saved properly |
|
04:29
🔗
|
godane |
i only found out about the site cause of glenn becks old website links |
|
04:29
🔗
|
godane |
from 2001 |
|
04:30
🔗
|
godane |
last link on this page: http://archive.glennbeck.com/archives/01-31-01.shtml |
|
04:32
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
04:35
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
04:51
🔗
|
|
mistym has joined #archiveteam-bs |
|
04:52
🔗
|
|
Ravenloft has quit IRC (Remote host closed the connection) |
|
04:54
🔗
|
|
godane has quit IRC (Ping timeout: 492 seconds) |
|
04:59
🔗
|
|
godane has joined #archiveteam-bs |
|
04:59
🔗
|
Start |
wow, youtube's completely screwed up the leeroy jenkins video (https://www.youtube.com/watch?v=LkCNJRfSZBU) |
|
05:00
🔗
|
Start |
the video and sound have became very low quality |
|
05:01
🔗
|
Start |
and the audio cuts out during the LEEROY JENKINS battle cry |
|
05:04
🔗
|
|
zenguy_pc has quit IRC (Ping timeout: 258 seconds) |
|
05:04
🔗
|
|
zenguy_pc has joined #archiveteam-bs |
|
05:07
🔗
|
joepie91 |
Start: copyright! |
|
05:09
🔗
|
Start |
it's very weird how it was removed though, you can still hear a bit of the LEEROY part briefly cut in |
|
05:11
🔗
|
Start |
the top comment on this eli5 post guesses that the original video had an exotic audio codec and youtube screwed it up when converting the video to an html5 friendly format: http://www.reddit.com/r/explainlikeimfive/comments/357s6l/eli5_all_of_a_sudden_the_famous_video_of_leeroy/ |
|
05:17
🔗
|
Start |
fortunately there's still this upload of the video: https://www.youtube.com/watch?v=hooKVstzbz0 |
|
05:23
🔗
|
|
Sellyme has quit IRC (Quit: No Ping reply in 180 seconds.) |
|
05:25
🔗
|
|
Sellyme has joined #archiveteam-bs |
|
05:29
🔗
|
|
dan_ has quit IRC (Ping timeout: 252 seconds) |
|
05:35
🔗
|
|
dan_ has joined #archiveteam-bs |
|
05:38
🔗
|
|
RichardG has quit IRC (Ping timeout: 186 seconds) |
|
06:01
🔗
|
|
RichardG has joined #archiveteam-bs |
|
06:22
🔗
|
DFJustin |
I still have the (original?) wmv from back int he day |
|
06:23
🔗
|
Lord_Nigh |
upload plz |
|
06:25
🔗
|
DFJustin |
it's alread on ia https://archive.org/details/LeeroyJenkins |
|
06:27
🔗
|
DFJustin |
(same checksum) |
|
06:45
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
06:46
🔗
|
|
mistym has joined #archiveteam-bs |
|
06:57
🔗
|
|
phillipsj has joined #archiveteam-bs |
|
06:58
🔗
|
|
mistym has quit IRC (Read error: Operation timed out) |
|
07:26
🔗
|
|
godane has quit IRC (Read error: Operation timed out) |
|
07:37
🔗
|
|
dan_ has quit IRC (Ping timeout: 252 seconds) |
|
07:37
🔗
|
midas |
LEEEEEROY..Jenkins. |
|
07:38
🔗
|
midas |
i remember that vid |
|
07:42
🔗
|
|
dan_ has joined #archiveteam-bs |
|
08:06
🔗
|
|
primus104 has joined #archiveteam-bs |
|
08:48
🔗
|
|
mistym has joined #archiveteam-bs |
|
09:01
🔗
|
|
mistym has quit IRC (Read error: Operation timed out) |
|
09:07
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
09:10
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
10:02
🔗
|
|
balrog has quit IRC (Read error: Operation timed out) |
|
10:03
🔗
|
|
marnold has joined #archiveteam-bs |
|
10:03
🔗
|
|
lrkj_ has quit IRC (Read error: Operation timed out) |
|
10:03
🔗
|
|
Lord_Nigh has quit IRC (Write error: Broken pipe) |
|
10:03
🔗
|
|
lrkj has joined #archiveteam-bs |
|
10:05
🔗
|
|
logchfoo_ starts logging #archiveteam-bs at Fri May 08 10:05:15 2015 |
|
10:05
🔗
|
|
logchfoo_ has joined #archiveteam-bs |
|
10:06
🔗
|
|
marnold_ has quit IRC (Ping timeout: 600 seconds) |
|
10:06
🔗
|
|
GLaDOS has quit IRC (Read error: Operation timed out) |
|
10:07
🔗
|
|
lytv has joined #archiveteam-bs |
|
10:07
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
10:08
🔗
|
|
Ctrl-S has quit IRC (Read error: Operation timed out) |
|
10:19
🔗
|
|
nico_32 has joined #archiveteam-bs |
|
10:24
🔗
|
|
marvinw has joined #archiveteam-bs |
|
10:27
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
10:29
🔗
|
|
balrog has joined #archiveteam-bs |
|
10:36
🔗
|
|
ivan`_ has joined #archiveteam-bs |
|
10:41
🔗
|
|
xtr-107 has quit IRC (Ping timeout: 370 seconds) |
|
10:43
🔗
|
|
GLaDOS has quit IRC (Ping timeout: 252 seconds) |
|
10:43
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
11:01
🔗
|
|
S[h]O[r]T has joined #archiveteam-bs |
|
11:01
🔗
|
|
botpie91 has joined #archiveteam-bs |
|
11:02
🔗
|
|
sep332 has joined #archiveteam-bs |
|
11:03
🔗
|
|
Ctrl-S has joined #archiveteam-bs |
|
11:10
🔗
|
|
toad1 has joined #archiveteam-bs |
|
11:17
🔗
|
|
GLaDOS has quit IRC (Ping timeout: 252 seconds) |
|
11:17
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
11:41
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
|
11:53
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
11:57
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
12:04
🔗
|
|
zenguy_pc has quit IRC (Remote host closed the connection) |
|
12:06
🔗
|
|
zenguy_pc has joined #archiveteam-bs |
|
12:08
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
12:11
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
12:40
🔗
|
|
sankin has joined #archiveteam-bs |
|
13:05
🔗
|
|
zenguy_pc has quit IRC (Ping timeout: 258 seconds) |
|
13:25
🔗
|
|
zenguy_pc has joined #archiveteam-bs |
|
13:55
🔗
|
|
ivan`_ is now known as ivan` |
|
14:07
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
14:10
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
14:25
🔗
|
|
primus104 has joined #archiveteam-bs |
|
14:27
🔗
|
|
mistym has joined #archiveteam-bs |
|
14:33
🔗
|
|
Smiley has quit IRC (Read error: Connection reset by peer) |
|
14:33
🔗
|
|
Smiley has joined #archiveteam-bs |
|
14:51
🔗
|
|
xtr-201 has joined #archiveteam-bs |
|
15:21
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
15:52
🔗
|
|
rejon has quit IRC (Ping timeout: 258 seconds) |
|
15:55
🔗
|
|
sankin has quit IRC (Leaving.) |
|
16:02
🔗
|
|
GLaDOS has quit IRC (Read error: Operation timed out) |
|
16:05
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
16:09
🔗
|
|
mistym has joined #archiveteam-bs |
|
16:31
🔗
|
|
ivan` has left |
|
16:33
🔗
|
schbirid2 |
anyone able to play the videos on http://www.universal-music.de/rihanna/videos/detail/video:366120/american-oxygen (without flash)? |
|
16:35
🔗
|
phillipsj |
challenge accepted! (BTW I have not been able to play crunchyroll.com videos without flash) |
|
16:38
🔗
|
phillipsj |
looks like they do anti-adblocking tricks (breaking up the script tags for duobleclick.net) |
|
16:40
🔗
|
|
GLaDOS has quit IRC (Ping timeout: 252 seconds) |
|
16:41
🔗
|
|
godane has joined #archiveteam-bs |
|
16:42
🔗
|
schbirid2 |
ha, typical |
|
16:42
🔗
|
schbirid2 |
i even tried with tor browser though, no luck |
|
16:43
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
16:45
🔗
|
phillipsj |
got 403 forbidden on my first attempt. |
|
16:46
🔗
|
schbirid2 |
i always get 404 for the video file |
|
16:46
🔗
|
schbirid2 |
the amount of requests the page does is insane |
|
16:47
🔗
|
phillipsj |
first attempt was: "http://www.universal-music.de/rihanna/videos/player-config/video:359614/rihanna-kanye-west-paul-mccartney-fourfiveseconds/e" (pulled from meta property="og:video") |
|
16:47
🔗
|
schbirid2 |
ah, i only tried normally with the browser (firefox, chromium, operashitversion) |
|
16:48
🔗
|
|
SimpBrain has joined #archiveteam-bs |
|
16:49
🔗
|
phillipsj |
omg, the "secure_url" is the same only the the outer nest of the redirect has HTTPS |
|
16:51
🔗
|
|
aaaaaaaaa has joined #archiveteam-bs |
|
16:51
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
16:52
🔗
|
schbirid2 |
ff in vbox fails too |
|
16:52
🔗
|
schbirid2 |
i block some stuff in my router thuogh |
|
16:56
🔗
|
schbirid2 |
brb |
|
16:56
🔗
|
|
schbirid2 has quit IRC (Leaving) |
|
16:58
🔗
|
|
mistym has joined #archiveteam-bs |
|
16:59
🔗
|
|
schbirid has joined #archiveteam-bs |
|
17:00
🔗
|
|
mistym has quit IRC (Remote host closed the connection) |
|
17:01
🔗
|
|
mistym has joined #archiveteam-bs |
|
17:01
🔗
|
|
xtr-201 has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
Smiley has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
Start has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
DFJustin has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
SketchCow has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
RedType has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
twrist has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
Jonimus has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
SadDM has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
wacky has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
arkhive has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
useretail has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
dx- has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
jk[SVP] has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
chfoo- has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
rduser has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:01
🔗
|
|
underscor has quit IRC (ircd.choopa.net ircd.shaw.ca) |
|
17:02
🔗
|
schbirid |
no luck with ie either |
|
17:04
🔗
|
schbirid |
i give up |
|
17:04
🔗
|
|
mistym_ has joined #archiveteam-bs |
|
17:04
🔗
|
yipdw |
joepie91: https://medium.com/@b_k/https-the-end-of-an-era-c106acded474 |
|
17:05
🔗
|
|
xtr-201 has joined #archiveteam-bs |
|
17:05
🔗
|
|
Smiley has joined #archiveteam-bs |
|
17:05
🔗
|
|
Start has joined #archiveteam-bs |
|
17:05
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
17:05
🔗
|
|
SketchCow has joined #archiveteam-bs |
|
17:05
🔗
|
|
RedType has joined #archiveteam-bs |
|
17:05
🔗
|
|
twrist has joined #archiveteam-bs |
|
17:05
🔗
|
|
Jonimus has joined #archiveteam-bs |
|
17:05
🔗
|
|
SadDM has joined #archiveteam-bs |
|
17:05
🔗
|
|
wacky has joined #archiveteam-bs |
|
17:05
🔗
|
|
useretail has joined #archiveteam-bs |
|
17:05
🔗
|
|
arkhive has joined #archiveteam-bs |
|
17:05
🔗
|
|
dx- has joined #archiveteam-bs |
|
17:05
🔗
|
|
jk[SVP] has joined #archiveteam-bs |
|
17:05
🔗
|
|
underscor has joined #archiveteam-bs |
|
17:05
🔗
|
|
rduser has joined #archiveteam-bs |
|
17:05
🔗
|
|
chfoo- has joined #archiveteam-bs |
|
17:05
🔗
|
|
ircd.shaw.ca sets mode: +oo SketchCow underscor |
|
17:08
🔗
|
yipdw |
I thought that dovetailed nicely with your argument against mandatory HTTPS |
|
17:08
🔗
|
phillipsj |
looks like they use RTMP: http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol |
|
17:10
🔗
|
|
atlogbot has joined #archiveteam-bs |
|
17:10
🔗
|
wacky |
Yea, it is rtmp, there are some tools out there (rtmpdump) but it will be a pain in the ass as rtmp is the devil |
|
17:10
🔗
|
|
mistym has quit IRC (Read error: Operation timed out) |
|
17:10
🔗
|
wacky |
looks like rtmp over port 80 which is also interesting |
|
17:10
🔗
|
swebb |
Can I get op? |
|
17:11
🔗
|
phillipsj |
looks like they use flowplayer too: http://flash.flowplayer.org/plugins/streaming/rtmp.html |
|
17:11
🔗
|
schbirid |
ah, rtmp -> not fun to archive, retreat |
|
17:12
🔗
|
* |
phillipsj just wants a machine untainted my binary blobs. |
|
17:12
🔗
|
phillipsj |
*by |
|
17:12
🔗
|
schbirid |
yipdw: let me guess, a long rant about the CA situation? |
|
17:12
🔗
|
yipdw |
actually no |
|
17:12
🔗
|
yipdw |
you should read it, it's a rare essay about the social aspect of software and networks |
|
17:12
🔗
|
schbirid |
ok, convinced |
|
17:12
🔗
|
yipdw |
CAs do play into it but it's not the primary thing |
|
17:13
🔗
|
schbirid |
the style of thef irst lines turned me off |
|
17:13
🔗
|
schbirid |
"medium.com" "introvert" "essay" =) |
|
17:13
🔗
|
yipdw |
my personal stance on mandatory HTTPS is "I don't give a fuck because fuck computers" but on the other hand the internet does have real meaning beyond my fuck-computers stance |
|
17:14
🔗
|
schbirid |
i'd really like trust on first contact somehow |
|
17:14
🔗
|
|
xmc sets mode: +o swebb |
|
17:14
🔗
|
|
swebb sets mode: +o DFJustin |
|
17:14
🔗
|
|
swebb sets mode: +o SadDM |
|
17:14
🔗
|
|
swebb sets mode: +o balrog |
|
17:14
🔗
|
|
swebb sets mode: +o ersi |
|
17:15
🔗
|
yipdw |
and increasing organizational approval to do something which you didn't really need before isn't good |
|
17:16
🔗
|
schbirid |
ok, i gave up on that text. sorry but toolongwinded;turnedmeoff |
|
17:19
🔗
|
yipdw |
that said mandatory encryption is going to screw with me because I am not purchasing a wildcard cert for ninjawedding.org just for the ArchiveBot dashboard |
|
17:19
🔗
|
yipdw |
I guess "mandatory HTTPS" is the better phrase, encryption itself is no problem |
|
17:19
🔗
|
schbirid |
wasnt the mozilla plan for a really really long timeframe? |
|
17:20
🔗
|
yipdw |
I hope so, because that means it will never happen |
|
17:20
🔗
|
schbirid |
within which they will wait for things like the eff's ca etc |
|
17:24
🔗
|
aaaaaaaaa |
Mozilla's "plan" has to have been to gauge buy-in, they don't have the clout to do it themselves. |
|
17:24
🔗
|
yipdw |
Google does |
|
17:26
🔗
|
yipdw |
relatedly I found that openssl s_client in its default configuration does not work with SNI |
|
17:26
🔗
|
yipdw |
so that was fun |
|
17:26
🔗
|
yipdw |
you need to pass -serverName to make it work |
|
17:26
🔗
|
yipdw |
naturally the solution here is to get the entire internet on ipv6 and rofl |
|
17:28
🔗
|
schbirid |
fuck |
|
17:28
🔗
|
schbirid |
i had some netlabel in a tab open for months |
|
17:28
🔗
|
schbirid |
meaning to archive it |
|
17:28
🔗
|
schbirid |
and now |
|
17:28
🔗
|
schbirid |
http://www.starquakerecords.com |
|
17:28
🔗
|
|
Start has quit IRC (Disconnected.) |
|
17:28
🔗
|
yipdw |
browser extension to shove your current tab's URL to ArchiveBot |
|
17:28
🔗
|
schbirid |
ah http://starquakerecords.weebly.com/ |
|
17:29
🔗
|
yipdw |
IRC client in a browser, what could go wrong |
|
17:29
🔗
|
schbirid |
yay, its all there |
|
17:29
🔗
|
* |
schbirid archives |
|
17:29
🔗
|
|
Start has joined #archiveteam-bs |
|
17:30
🔗
|
|
Start has quit IRC (Client Quit) |
|
17:31
🔗
|
|
GLaDOS has quit IRC (Read error: Operation timed out) |
|
17:32
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
17:33
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
18:05
🔗
|
|
GLaDOS has quit IRC (Ping timeout: 252 seconds) |
|
18:08
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
18:11
🔗
|
|
mistym_ has quit IRC (Remote host closed the connection) |
|
18:12
🔗
|
joepie91 |
yipdw: that is a very, very good post |
|
18:13
🔗
|
joepie91 |
with a very interesting perspective |
|
18:16
🔗
|
|
RichardG has quit IRC (Read error: Connection reset by peer) |
|
18:17
🔗
|
|
GLaDOS has quit IRC (Read error: Operation timed out) |
|
18:18
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
18:18
🔗
|
|
RichardG has joined #archiveteam-bs |
|
18:19
🔗
|
schbirid |
so i thought, let's do angelfire tonight |
|
18:19
🔗
|
schbirid |
they have a sitemal.xml for each user |
|
18:19
🔗
|
schbirid |
inside are all their html pages, not images or anything else |
|
18:19
🔗
|
schbirid |
so what would be the best way to take them all (per user) and grab all their --page-requisites as well? |
|
18:20
🔗
|
schbirid |
they are often too many to simply pass a list of URLs to the wget invocation |
|
18:20
🔗
|
xmc |
sounds about right |
|
18:21
🔗
|
schbirid |
oh wait, i could just put them all into one file with one url per line and use -i |
|
18:21
🔗
|
schbirid |
let's try that |
|
18:21
🔗
|
joepie91 |
yeah, -i |
|
18:21
🔗
|
xmc |
why not a warrior job |
|
18:22
🔗
|
xmc |
wget will oom |
|
18:22
🔗
|
schbirid |
yeah, i can do it easily i think |
|
18:22
🔗
|
xmc |
ok yeah i see now |
|
18:22
🔗
|
xmc |
angelfire <3 |
|
18:22
🔗
|
joepie91 |
At Least It's Not Geocities |
|
18:23
🔗
|
schbirid |
every time i do something like "--reject-regex='(www.angelfire.com\/adm\/ad\/|www.angelfire.com\/doc\/images\/track\/ot_noscript\.gif)'" i feel really dumb |
|
18:29
🔗
|
|
GLaDOS has quit IRC (Ping timeout: 252 seconds) |
|
18:30
🔗
|
|
GLaDOS has joined #archiveteam-bs |
|
18:41
🔗
|
schbirid |
script is ready \o/ |
|
18:43
🔗
|
ersi |
Fire awaay! |
|
18:49
🔗
|
schbirid |
oh, 3895290 users, oops? |
|
18:50
🔗
|
xmc |
it'll be fiiiiine |
|
18:50
🔗
|
schbirid |
i'll split them per sitemap |
|
18:50
🔗
|
xmc |
we've done bigger projects :) |
|
18:50
🔗
|
xmc |
i'd say put one user per row into the tracker |
|
18:50
🔗
|
xmc |
this is what the tracker is designed to do! |
|
18:50
🔗
|
schbirid |
i am not writing tracker stuff. never learned that |
|
18:50
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
18:50
🔗
|
xmc |
ah |
|
18:50
🔗
|
xmc |
now's the time to learn! |
|
18:51
🔗
|
schbirid |
meh, not me! |
|
18:51
🔗
|
xmc |
:( |
|
18:51
🔗
|
xmc |
but this will actually work better |
|
18:51
🔗
|
schbirid |
docs anywhere? |
|
18:52
🔗
|
xmc |
you can crib from any of the warrior projects in https://github.com/archiveteam/ |
|
18:52
🔗
|
xmc |
baraza-grab for example |
|
18:52
🔗
|
xmc |
or lastfm-grab |
|
18:53
🔗
|
xmc |
this is iiuc the file you want to modify to make it your own: |
|
18:53
🔗
|
xmc |
https://github.com/ArchiveTeam/lastfm-grab/blob/master/lastfm.lua |
|
18:53
🔗
|
xmc |
actually, start from: https://github.com/ArchiveTeam/example-wpull-seesaw-project |
|
18:53
🔗
|
schbirid |
luaaaaaaa |
|
18:53
🔗
|
schbirid |
nah, i want to get it done. sorry |
|
18:53
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
18:53
🔗
|
xmc |
:( ok |
|
18:54
🔗
|
xmc |
done is better than perfect and unfinished |
|
18:58
🔗
|
|
primus104 has joined #archiveteam-bs |
|
19:04
🔗
|
joepie91 |
xmc: I'd say that's highly contextual |
|
19:04
🔗
|
joepie91 |
lol |
|
19:04
🔗
|
xmc |
right yes |
|
19:05
🔗
|
xmc |
let's just say that i trust schbirid to make well-formed and reasonably complete warcs |
|
19:05
🔗
|
xmc |
and i don't want to stand in front of them and say NOPE DO IT MY WAY |
|
19:05
🔗
|
joepie91 |
would certainly rather have a perfect and unfinished nuke, than I'd have a "done" one :D |
|
19:05
🔗
|
joepie91 |
but then, perhaps nukes aren't quite warcs |
|
19:06
🔗
|
xmc |
different goals |
|
19:06
🔗
|
schbirid |
warghs |
|
19:06
🔗
|
joepie91 |
depending on who you ask, of course... |
|
19:06
🔗
|
joepie91 |
:P |
|
19:08
🔗
|
schbirid |
lol, first user i hit, mp3z http://www.angelfire.com/dragon2/poofi_airboy/ |
|
19:11
🔗
|
schbirid |
oh great, infinite calendars http://filesha.angelfire.com/blog/index.blog?from=20231130 |
|
19:11
🔗
|
schbirid |
fuck, any suggestion how to handle those? |
|
19:12
🔗
|
joepie91 |
schbirid: "ignore it all", generally :P |
|
19:12
🔗
|
aaaaaaaaa |
schbirid: you may want to check archive.org because www.angelfire.com was grabbed with archivebot (but not subdomains) |
|
19:12
🔗
|
xmc |
add an ignore that matches regex 20[2-9]..... and so forth |
|
19:13
🔗
|
schbirid |
http://filesha.angelfire.com/blog/index.blog?from=090131 |
|
19:13
🔗
|
schbirid |
aaaaaaaaa: oh! i was looking at http://archiveteam.org/index.php?title=Angelfire |
|
19:14
🔗
|
schbirid |
nothing here https://archive.org/search.php?query=angelfire |
|
19:14
🔗
|
xmc |
wait is that ?from= parameter unixtime |
|
19:14
🔗
|
xmc |
lol |
|
19:14
🔗
|
xmc |
yeah uh maybe ignore it all |
|
19:17
🔗
|
aaaaaaaaa |
http://archive.fart.website/archivebot/viewer/job/9yhap |
|
19:17
🔗
|
winr4r |
is angelfire closing? |
|
19:20
🔗
|
schbirid |
aaaaaaaaa: i prefer human readable archives in addition :( |
|
19:20
🔗
|
schbirid |
err, standard archives. like 7z |
|
19:20
🔗
|
schbirid |
winr4r: you never know |
|
19:20
🔗
|
winr4r |
ah |
|
19:33
🔗
|
|
Start has joined #archiveteam-bs |
|
19:39
🔗
|
|
mistym has joined #archiveteam-bs |
|
19:54
🔗
|
|
SN4T14_ has joined #archiveteam-bs |
|
20:02
🔗
|
|
SN4T14 has quit IRC (Ping timeout: 512 seconds) |
|
20:04
🔗
|
schbirid |
test run overnight "Downloading sitemap 00 and all its ~16000 users, hold tight..." |
|
20:05
🔗
|
schbirid |
i gave up my "set -e" "-u" and "-o pipefail" experiments because everything just exited silently and some point |
|
20:06
🔗
|
winr4r |
16000? |
|
20:06
🔗
|
winr4r |
that looks way too low |
|
20:07
🔗
|
winr4r |
well, feels* |
|
20:08
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
20:08
🔗
|
schbirid |
just the first of 256 sitemaps |
|
20:08
🔗
|
aaaaaaaaa |
there are multiple sitemaps |
|
20:08
🔗
|
schbirid |
00 to ff, kinda freaky how that fit almost perfectly |
|
20:10
🔗
|
winr4r |
aaaaaaaaa: oh! |
|
20:10
🔗
|
winr4r |
i was going to say "ah", but that looks odd next to your username |
|
20:10
🔗
|
winr4r |
schbirid: yeah, that is |
|
20:11
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
20:19
🔗
|
Start |
http://arstechnica.com/information-technology/2015/05/prime-minister-of-singapore-shares-his-c-code-for-sudoku-solver/ |
|
20:21
🔗
|
|
schbirid has quit IRC (Read error: Operation timed out) |
|
20:22
🔗
|
winr4r |
neat af |
|
20:22
🔗
|
|
Start has quit IRC (Disconnected.) |
|
20:24
🔗
|
aaaaaaaaa |
my favorite complaint about that I saw was to the effect of "That is C code not C++, idiot doesn't know the difference." |
|
20:25
🔗
|
winr4r |
don't read the comments! |
|
20:28
🔗
|
winr4r |
on the off chance that he did actually get that wrong and that it wasn't a reporter transcribing him wrong, i'd still someone well-versed enough to write a sudoku solver over a mong like our PM who seriously hinted at banning encryption |
|
20:28
🔗
|
|
schbirid has joined #archiveteam-bs |
|
20:29
🔗
|
DFJustin |
it's a .cpp in the google drive folder |
|
20:29
🔗
|
DFJustin |
but there's no law that you have to use c++ features |
|
20:31
🔗
|
winr4r |
just tried, compiles as C |
|
20:33
🔗
|
joepie91 |
so it's a PM who can write polyglots? |
|
20:33
🔗
|
joepie91 |
bonus points! |
|
20:33
🔗
|
joepie91 |
:P |
|
20:33
🔗
|
winr4r |
haha |
|
20:34
🔗
|
aaaaaaaaa |
Technically, wouldn't it be objective-c too? |
|
20:34
🔗
|
schbirid |
kinda dumb how angelfire has users with no urls in the sitemaps |
|
20:34
🔗
|
joepie91 |
the consequences of not designing for remote API failures: http://i.imgur.com/2MOZxuu.png |
|
20:36
🔗
|
xmc |
:\ |
|
20:36
🔗
|
joepie91 |
also |
|
20:36
🔗
|
joepie91 |
vps-forge is hutting down |
|
20:36
🔗
|
joepie91 |
:( |
|
20:36
🔗
|
joepie91 |
shutting * |
|
20:36
🔗
|
schbirid |
jabba |
|
20:36
🔗
|
winr4r |
joepie91: at my last job we developed an iphone app that used a REST API (i was building the API bit) |
|
20:37
🔗
|
|
Start has joined #archiveteam-bs |
|
20:37
🔗
|
joepie91 |
https://vpsboard.com/topic/6979-vps-forge-is-shutting-down/ |
|
20:37
🔗
|
joepie91 |
xmc: that was my fuckup btw, I forgot my error handling there |
|
20:37
🔗
|
winr4r |
every now and then i would put something in there to return invalid JSON just to keep the app-developer guy on his toes |
|
20:37
🔗
|
joepie91 |
which is why I have this error reporting thing :P |
|
20:37
🔗
|
joepie91 |
winr4r: heh |
|
20:37
🔗
|
winr4r |
the first time i did it he was like "wuh the app crashed" and i didn't tell him i was deliberately introducing errors |
|
20:38
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
20:38
🔗
|
winr4r |
he worked it out after like, two hours and i was like "yes, i did that, now please sanity check your fucking data" |
|
20:38
🔗
|
joepie91 |
hahaha |
|
20:40
🔗
|
winr4r |
i suppose that was a dick move but as little as i understand objective-C, i saw zero sanity checks in his code |
|
20:40
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
20:41
🔗
|
xmc |
eh |
|
20:41
🔗
|
xmc |
ever heard of the netflix chaos monkey |
|
20:41
🔗
|
winr4r |
he learned to do sanity checks after that, started writing solid code, and got hired as an app developer for an insurance company a few weeks after i left that place |
|
20:42
🔗
|
winr4r |
moral of the story: be a dick, it makes other people better |
|
20:42
🔗
|
joepie91 |
winr4r: amen :P |
|
20:42
🔗
|
joepie91 |
but then again, I don't think anybody doubts I can be a dick for that same reason |
|
20:43
🔗
|
winr4r |
well, stronger* |
|
20:46
🔗
|
|
Rotab has quit IRC (Read error: Connection reset by peer) |
|
20:48
🔗
|
winr4r |
joepie91: as for vps-forge, the kind of people that are paying for a VPS probably know how to move their shit somewhere else |
|
20:48
🔗
|
joepie91 |
winr4r: I know, was just a PSA :) |
|
20:48
🔗
|
joepie91 |
may still be worth having a look though |
|
20:49
🔗
|
joepie91 |
3 week shutdown notice, container backups offered |
|
20:49
🔗
|
joepie91 |
but they've been around for 5 years without a hitch |
|
20:49
🔗
|
joepie91 |
so some may miss the notice |
|
20:49
🔗
|
* |
joepie91 still has to figure out where he's going to move his stuff |
|
20:50
🔗
|
winr4r |
joepie91: i'm with digitalocean, been very good so far |
|
20:51
🔗
|
|
Rotab has joined #archiveteam-bs |
|
20:51
🔗
|
joepie91 |
digitalocean always has been very meh to me |
|
20:51
🔗
|
winr4r |
as much as their advertising makes me want to crotch-kick whoever thought *that* was a good idea |
|
20:52
🔗
|
winr4r |
or at least, did, i'm not sure if they're still running with the "hurr i do not know how to server" hook |
|
20:52
🔗
|
joepie91 |
because 1) their perf really isn't all that impressive 2) I don't get the impression that I'll be talking to an actual human being whenever I need support 3) it's quite obvious that they're burning through VC money |
|
20:52
🔗
|
joepie91 |
and 4) their network is not exactly reliable across the board |
|
20:54
🔗
|
winr4r |
joepie91: 1) wfm 2) i don't care if my shit doesn't go offline 3) they can do that and i am happy because they're ten minutes, a credit card payment and an rsync command away from them losing their business |
|
20:55
🔗
|
joepie91 |
wfm? |
|
20:55
🔗
|
winr4r |
joepie91: works for me |
|
20:55
🔗
|
joepie91 |
ah |
|
20:55
🔗
|
winr4r |
a handy side-effect of leaving a screen(1) on my VPS on IRC is that i learn as soon as i get home if my VPS or its network has gone down |
|
20:55
🔗
|
winr4r |
happened once in months |
|
20:55
🔗
|
joepie91 |
right, I do care about support - when shit hits the fan (and it'll inevitably will at some point), I want a provider to be helpful rather than another obstacle to navigate around |
|
20:56
🔗
|
joepie91 |
that's not a reliable uptime monitor |
|
20:56
🔗
|
xmc |
apt-get install uptimed ! |
|
20:57
🔗
|
joepie91 |
I don't just mean system uptime |
|
20:57
🔗
|
joepie91 |
also network uptime |
|
20:57
🔗
|
joepie91 |
connecting via a screen won't catch network blips |
|
20:57
🔗
|
joepie91 |
while those can have real consequences |
|
20:57
🔗
|
xmc |
apt-get install uptimed smokeping |
|
20:57
🔗
|
joepie91 |
and DO had serious issues with network blips, a year ago anyway |
|
20:57
🔗
|
winr4r |
joepie91: yeah, obvious disclaimer is that nobody loses money if my VPS goes down |
|
20:57
🔗
|
joepie91 |
so unless they've fixed that... |
|
20:57
🔗
|
midas |
routing/connectivity/support/abuse |
|
20:57
🔗
|
joepie91 |
winr4r: same for me, doesn't mean I don't want reliability :) |
|
20:57
🔗
|
midas |
all important |
|
20:58
🔗
|
winr4r |
fair dos |
|
21:00
🔗
|
joepie91 |
anyhow, as far as I'm concerned, I don't really see DigitalOcean beating RamNode in any respect other than having a Singapore(?) location |
|
21:00
🔗
|
joepie91 |
the perf is worse, the price is about the same, the disk space is much less, the network is (or at least was) less reliable, the support isn't as good |
|
21:01
🔗
|
winr4r |
gotcha |
|
21:02
🔗
|
joepie91 |
(and RamNode isn't burning VC money either) |
|
21:02
🔗
|
joepie91 |
(proper, sustainably built business) |
|
21:02
🔗
|
joepie91 |
oh, and DO doesn't offer DDoS mitigation either |
|
21:02
🔗
|
winr4r |
(that really doesn't bother me, i don't mind bankrupting VVCs) |
|
21:03
🔗
|
winr4r |
-V |
|
21:03
🔗
|
joepie91 |
winr4r: I don't mind that either, but that's not my point |
|
21:04
🔗
|
joepie91 |
VC capital is almost a sure sign that there's no sustainable business |
|
21:04
🔗
|
joepie91 |
which means it can implode or "pivot" at any time (read: you as a customer get screwed over, packed in a fancy looking package) |
|
21:04
🔗
|
joepie91 |
just look at what Heroku is pulling now, for example |
|
21:04
🔗
|
joepie91 |
that's the kind of antics to generally be expected from business that either get started with VC capital or are still running off it |
|
21:05
🔗
|
xmc |
what's heroku up to? |
|
21:05
🔗
|
joepie91 |
xmc: tightening up their free plan |
|
21:05
🔗
|
xmc |
ah |
|
21:05
🔗
|
winr4r |
i don't agree, i treat hosting like i do supermarkets |
|
21:05
🔗
|
joepie91 |
I believe it translates to something along the lines of "you can use it 18 out of 24 hours a day" |
|
21:05
🔗
|
joepie91 |
which is just... what |
|
21:06
🔗
|
joepie91 |
winr4r: your business with a supermarket ends the moment you step away from the cash register |
|
21:06
🔗
|
winr4r |
cheap, and if it disappears there's really no shortage of other places to sell me the same thing |
|
21:06
🔗
|
joepie91 |
that is not the case for a hosting provider |
|
21:06
🔗
|
joepie91 |
your pot of instant coffee is not going to disappear from your storage cabinet, just because the supermarket you bought it from went bankrupt |
|
21:06
🔗
|
joepie91 |
yet that's /exactly/ what happens with hosting |
|
21:07
🔗
|
joepie91 |
they're really not comparable at all |
|
21:08
🔗
|
winr4r |
joepie91: the analogy was to point out that there's a lot of competition in the same market and that, taking the wider case, the kind of person that knows how to work a VPS is almost certainly going to have local copies of shit |
|
21:08
🔗
|
joepie91 |
winr4r: whether there is a lot of competition depends entirely on the kind of service |
|
21:08
🔗
|
joepie91 |
winr4r: for example, name a replacement for vps-forge |
|
21:09
🔗
|
joepie91 |
there goes 95% of the VPS companies right off the list, because they're not in Hetzner |
|
21:09
🔗
|
winr4r |
joepie91: Hetzner? |
|
21:09
🔗
|
joepie91 |
the remaining ones are typically either out of stock, severely restrict bandwidth usage, or are expensive |
|
21:09
🔗
|
joepie91 |
yes, Hetzner |
|
21:09
🔗
|
joepie91 |
a datacenter in Germany |
|
21:09
🔗
|
xmc |
hetzner is some datacenter iirc |
|
21:09
🔗
|
joepie91 |
with a very good network for EU |
|
21:10
🔗
|
joepie91 |
it's /extremely/ stable |
|
21:10
🔗
|
joepie91 |
like, "goes months without a single blip" stable |
|
21:10
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
21:11
🔗
|
winr4r |
ah gotcha |
|
21:12
🔗
|
joepie91 |
winr4r: what further complicates matters is that many VPS providers have node reboots with some regularity |
|
21:12
🔗
|
joepie91 |
so this |
|
21:12
🔗
|
joepie91 |
de1:~# uptime |
|
21:12
🔗
|
joepie91 |
00:22:59 up 243 days, 4:19, 1 user, load average: 0.00, 0.00, 0.00 |
|
21:12
🔗
|
joepie91 |
is very unlikely to be easy to find |
|
21:12
🔗
|
joepie91 |
taking into account other requirements |
|
21:12
🔗
|
winr4r |
eh, i just don't see how it's an issue, there's no shortage of VPS providers and while they might not be as good as vps-forge, their business model was unsustainable by their own admission |
|
21:13
🔗
|
joepie91 |
winr4r: there was never any such admission. |
|
21:13
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
21:13
🔗
|
joepie91 |
read the email carefully |
|
21:13
🔗
|
joepie91 |
in fact, I strongly suspect it comes down to "this is too taxing on our personal lives" |
|
21:13
🔗
|
joepie91 |
which is probably the #1 cause for shutdowns in the VPS industry for companies that have been around for a while |
|
21:14
🔗
|
joepie91 |
and the "not as good as" is exactly the problem |
|
21:14
🔗
|
winr4r |
joepie91: i factor that into "bad business model" |
|
21:14
🔗
|
joepie91 |
it being that good is part of my requirements |
|
21:14
🔗
|
joepie91 |
which are not fulfilled by others |
|
21:14
🔗
|
joepie91 |
I don't /care/ that there are worse options available, they do not fulfill my requirements |
|
21:14
🔗
|
joepie91 |
so no, you can't just arbitrarily replace a VPS |
|
21:14
🔗
|
joepie91 |
winr4r: it doesn't. |
|
21:15
🔗
|
joepie91 |
I'm kind of running out of ways to describe this |
|
21:15
🔗
|
joepie91 |
or explain this, rather |
|
21:16
🔗
|
joepie91 |
"there's other VPS companies" in no way logically results in "it is easy to replace any VPS" |
|
21:20
🔗
|
|
Start has quit IRC (Disconnected.) |
|
21:21
🔗
|
winr4r |
joepie91: we don't disagree as much as your wording suggests |
|
21:25
🔗
|
winr4r |
i treat VPSes like a dude handing out free pizzas, that's going to be fun while it lasts, but i am not going to go hungry when it ends |
|
21:27
🔗
|
winr4r |
and people with VPSes are the kind of people with enough technical knowledge that they have no excuse if they don't know how to make lunch when free pizza party ends |
|
21:28
🔗
|
winr4r |
(you can blame SketchCow for my love for stretched analogies) |
|
21:41
🔗
|
yipdw |
archivebot@archivebot-proto2:~$ w |
|
21:41
🔗
|
yipdw |
21:41:38 up 360 days, 4:51, 1 user, load average: 1.43, 1.51, 1.46 |
|
21:41
🔗
|
yipdw |
let's see if digitalocean can do one year |
|
21:43
🔗
|
joepie91 |
winr4r: right, that might be how you treat VPSes - my problem is that you seemed to be extending that attitude to "VPS users" in general, which is incorrect ;) |
|
21:47
🔗
|
winr4r |
17:47:20 up 182 days, 12:24, 1 user, load average: 0.00, 0.01, 0.05 |
|
21:47
🔗
|
winr4r |
half way there, yipdw |
|
21:47
🔗
|
yipdw |
I actually really hope DO does because ArchiveBot startup is not documented |
|
21:48
🔗
|
yipdw |
and I've punted on supervisory processes and startup procedures because I hate doing that |
|
21:48
🔗
|
yipdw |
one of these days I'll get around to it |
|
21:49
🔗
|
winr4r |
joepie91: maybe your social groups give you a different perspective, but all the people i've known with a VPS have been the kind that know how to do rsync and move their shit somewhere else in twenty minutes |
|
21:50
🔗
|
xmc |
i can move my shit elsewhere in a hurry but it'll take a while to unpack it and set up shop again |
|
21:50
🔗
|
xmc |
dns and so forth, at the very least |
|
21:50
🔗
|
joepie91 |
winr4r: but that's my point. you're deriving it from your own perspective without taking into account that other perspectives may (and probably do) exist |
|
21:50
🔗
|
joepie91 |
which is the problem with most blanket assertions |
|
21:51
🔗
|
ersi |
mmmh blankets |
|
21:51
🔗
|
winr4r |
joepie91: that's what discussion is for, finding other perspectives :) |
|
22:01
🔗
|
winr4r |
joepie91: tbh, i trust hosting providers so little that i wrote a static site generator for my motorsports stuff to make a migration to another provider as painless as possible |
|
22:02
🔗
|
|
SimpBrain has quit IRC (Quit: Leaving) |
|
22:04
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
22:15
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
22:40
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
22:44
🔗
|
|
BlueMaxim has joined #archiveteam-bs |
|
22:55
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
23:11
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
23:17
🔗
|
|
Fusl has quit IRC (Max SendQ exceeded) |
|
23:17
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
23:18
🔗
|
|
Fusl has joined #archiveteam-bs |
|
23:21
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
23:26
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
23:40
🔗
|
joepie91 |
sent a sales query to afterburst |
|
23:40
🔗
|
joepie91 |
let's see what happens |
|
23:40
🔗
|
joepie91 |
they've been around for a good while, I recall them being similar to VPS-Forge |
|
23:40
🔗
|
joepie91 |
also in Germany, but a diff network |
|
23:41
🔗
|
joepie91 |
cc winr4r |
|
23:51
🔗
|
winr4r |
joepie91: good luck :) |