Time |
Nickname |
Message |
00:10
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
00:11
π
|
|
MMovie has joined #archiveteam |
00:12
π
|
JesseW |
johtso: ping ivan` about archiving youtube |
00:14
π
|
johtso |
If it involves any kind of manual involvement it's probably not practical, it's a 24 hour live stream :) |
00:14
π
|
johtso |
I'm sure they'll archive it.. |
00:15
π
|
HCross |
johtso, look at using livestreamer and vlc media plater |
00:15
π
|
HCross |
player |
00:34
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
00:35
π
|
|
MMovie has joined #archiveteam |
00:37
π
|
hictooth |
I was wondering what the status of archiving fanfiction.net is? According to http://www.archiveteam.org/index.php?title=FanFiction.Net it's being saved, but I can't find out by who or where to. |
00:38
π
|
MrRadar |
Last September someone scraped every story from it and put it up as a torrent. |
00:39
π
|
MrRadar |
(Though they just saved each one as plain text, so maybe not the best job) |
00:39
π
|
MrRadar |
You can find the magnet link here: https://www.reddit.com/r/DataHoarder/comments/3jl3qm/nearly_complete_archive_of_fanfictionnet/ |
00:39
π
|
hictooth |
So it's not being actively archived now? |
00:40
π
|
MrRadar |
Not by us, as far as I know |
00:43
π
|
SimpBrain |
got tagged as saved |
00:44
π
|
JesseW |
Hm, probably should be changed to {{partiallysaved}} in that case |
00:44
π
|
JesseW |
Do you know if anyone tossed the torrent onto IA? |
00:44
π
|
MrRadar |
I did |
00:45
π
|
MrRadar |
https://archive.org/details/fanfiction.net_2015_09 |
00:46
π
|
JesseW |
ah, cool -- please do add that link to the wiki page |
00:48
π
|
JesseW |
also, if/when you get a chance, you could turn the link from the item description into a clickable link. |
00:49
π
|
MrRadar |
How do you do that? |
00:49
π
|
|
ndiddy has quit IRC (Read error: Operation timed out) |
00:50
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
00:51
π
|
|
MMovie has joined #archiveteam |
00:52
π
|
JesseW |
MrRadar: the description can use HTML |
00:52
π
|
MrRadar |
OK, I didn't know that |
00:52
π
|
JesseW |
i.e. <a href="http://blah.com">http://blah.com</a> |
00:52
π
|
JesseW |
yeah, one of many IA hidden features. :-) |
00:52
π
|
JesseW |
IDK what sanatizing they do (not much, I'd guess) |
00:54
π
|
JesseW |
Example (just tested): https://archive.org/details/fav-jesse_w |
00:55
π
|
MrRadar |
It looks like we did a scrape of the site back in 2014. Where did that data end up? |
00:55
π
|
MrRadar |
I'd like to add a link to it too |
00:55
π
|
|
robink has quit IRC (Ping timeout: 190 seconds) |
00:56
π
|
JesseW |
Hm, this is ... mildly alarming: https://catalogd.archive.org/log/466976952 |
00:56
π
|
JesseW |
Apparently the lack of sanatizing they do extends to these pages. :-) |
00:56
π
|
MrRadar |
Oh, yeah, that's bad |
00:57
π
|
MrRadar |
I'd hit info@ |
00:57
π
|
|
robink has joined #archiveteam |
00:57
π
|
MrRadar |
They probably just need to put an htmlspecialchars() call around the output |
01:00
π
|
MrRadar |
Hmm. It looks like that Fanfiction.net scrape was also uploaded by its original creator. |
01:00
π
|
|
robink has quit IRC (Ping timeout: 190 seconds) |
01:00
π
|
MrRadar |
Now that I do a search for it |
01:01
π
|
MrRadar |
The only difference is that mine has the original torrent file |
01:01
π
|
MrRadar |
Does the IA dedup by hash? I'd hate to have them storing this data 4 times |
01:01
π
|
|
robink has joined #archiveteam |
01:02
π
|
bsmith093 |
MrRadar: that was me, i think, and it now has an inventory file |
01:02
π
|
yipdw |
they're not storing it 4 times |
01:02
π
|
yipdw |
they're storing it 8 times |
01:02
π
|
yipdw |
at least |
01:02
π
|
|
robink has quit IRC (Remote host closed the connection) |
01:03
π
|
yipdw |
it would be interesting to see if any of those copies have the original version of Fifty Shades of Grey |
01:03
π
|
yipdw |
if none of them do that is a serious mark against all ofu s |
01:03
π
|
MrRadar |
bsmith093, this one: https://archive.org/details/FanfictionNearlyCompleteArchive ? |
01:03
π
|
bsmith093 |
MrRadar: yes, thst |
01:03
π
|
bsmith093 |
that |
01:04
π
|
MrRadar |
Haha, I should have searched first before uploading it |
01:04
π
|
MrRadar |
Did you create that scrape originally? |
01:04
π
|
bsmith093 |
MrRadar: which one did you do |
01:04
π
|
MrRadar |
I uploaded it to https://archive.org/details/fanfiction.net_2015_09 |
01:04
π
|
bsmith093 |
MrRadar: using fanficfare running through a list of all id numbers |
01:05
π
|
MrRadar |
OK |
01:05
π
|
bsmith093 |
you many want the inventory file to be able to search that |
01:05
π
|
MrRadar |
How should I credit you in my copy of the upload? |
01:06
π
|
bsmith093 |
list the other link. it's fine i just used another project somebody else built to scrape all of it |
01:07
π
|
bsmith093 |
MrRadar: while we're on the subject, how the hell do i extract one file from this archive. should it be taking forever? |
01:08
π
|
MrRadar |
Unlink zip and 7z files, TAR files don't have a catalog so if you want to extract a file it has to scan through the whole thing |
01:08
π
|
MrRadar |
To find it |
01:08
π
|
bsmith093 |
ugh |
01:10
π
|
bsmith093 |
MrRadar: if you want to rebuild it into a 7z file, you can, i would really like something i can extract a given file from in less than an hour |
01:15
π
|
snape |
yipdw, the original fifty shades was gone from ff by 2010. Wayback machine might have a copy, but it's robots.txt-excluded. There are still copies of it floating around the web tho, if you know the original title. |
01:15
π
|
yipdw |
so much for our efforts |
01:16
π
|
bsmith093 |
yipdw: if they did'nt throttle so hard, we could have gotten all of it in like a week |
01:20
π
|
yipdw |
I kid |
01:20
π
|
yipdw |
it's just that I've encountered situations where it's like "oh I wonder if we got that" and yet in the terabytes we drag in daily |
01:20
π
|
yipdw |
nope |
01:21
π
|
yipdw |
this happens sometimes in archivebot crawls |
01:21
π
|
yipdw |
maybe that's just what happens when you deal with something as ineffably huge as the web |
01:21
π
|
MrRadar |
Updated the Fanfiction.net page with references to the AT's 2012 scrape and bsmith093's 2015 one |
01:21
π
|
|
dserodio has quit IRC (Read error: Operation timed out) |
01:21
π
|
snape |
If it's any consolation, we likely have good representative samples of the early age of dinosaur erotica, and... whatever the next terrible trend will be. |
01:22
π
|
yipdw |
ARCHIVE TEAM TRENDSETTIN' 2016 \m/ |
01:23
π
|
dxrt |
simply amazing |
01:23
π
|
bsmith093 |
snape:well, thank FSM we have that! ;) |
01:24
π
|
|
dserodio has joined #archiveteam |
01:27
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
01:27
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
01:28
π
|
|
MMovie has joined #archiveteam |
01:28
π
|
JesseW |
OK, reported the lack of escaping to info@ |
01:30
π
|
|
dashcloud has joined #archiveteam |
01:31
π
|
bsmith093 |
JesseW: I checked that log, what was wrong? |
01:32
π
|
JesseW |
bsmith093: note the 2nd time the description was shown, after [description] => |
01:32
π
|
JesseW |
It uses the actual HTML, not escaped (as it is above) |
01:32
π
|
JesseW |
(and again below, by "with value:" |
01:33
π
|
bsmith093 |
oh, i see it now |
01:33
π
|
bsmith093 |
Also, is there an archive format that stores an index, because apparently tar doesn't |
01:35
π
|
HCross2 |
cdx |
01:35
π
|
MrRadar |
Well, that's for WARCs |
01:35
π
|
MrRadar |
For general files .zip or .7z are the go-to |
01:36
π
|
bsmith093 |
is there a thing i can use without having to un- and re-compress 300GB of files? |
01:37
π
|
MrRadar |
Not really. Part of it is also that if your .tar is also gzipped you would need to decompress the entire gzip stream up to each file |
01:37
π
|
MrRadar |
Even if you had an index |
01:37
π
|
MrRadar |
.tar.gz is not designed for random access |
01:38
π
|
bsmith093 |
anyone feel like being awesome? when i created that file, i thought it would actually be searchable easily. |
01:39
π
|
snape |
On a related note, I wonder if there's a list somewhere of the oldest continuously-active porn sites. The oldest one I could remember, from 2000, seems to have disappeared sometime last year. :/ |
01:41
π
|
JesseW |
snape: there may have been such a list on Wikipedia -- although it quite likely has been deleted by now; but if you look through old revisions, you may be able to find it. |
01:42
π
|
JesseW |
bsmith093: once I get done with the IA census (which should be pretty soon -- mostly just waiting on jjake uploading the results) I'm glad to recompress the fanfiction tarball as a zip. |
01:42
π
|
JesseW |
I have a good pipe, and enough free space. |
01:47
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
01:47
π
|
bsmith093 |
JesseW: thanks so much! BTW when I made the file, every single gui tool was choking on a folder that big, and i didn't actually have the space to sstore the final tar, so i compressed on the fly to fos. when i created the file, apparently i pushed in the whole path of the folder, so when you rebuild it, could you start with the Fanfiction folder, b |
01:47
π
|
bsmith093 |
uried in home/Desktop etc. that was my bad. |
01:49
π
|
JesseW |
yeah |
01:49
π
|
bsmith093 |
any way i have plenty of space now, mostly because i finally dumped the uncompressed files, and thats how i got started looking for omething to search a tar file |
01:49
π
|
JesseW |
I think my debian box should be OK handling it. |
01:49
π
|
JesseW |
Thank you for babysitting the script to make it! |
01:50
π
|
|
dashcloud has joined #archiveteam |
01:51
π
|
JesseW |
MrRadar: I improved the link to the 2012 scrape. |
01:51
π
|
bsmith093 |
np, i wasn't doing much anyway! also the inventory file is here, and you'll see the problem immediately https://archive.org/download/FanfictionNearlyCompleteArchive/inventory.txt |
01:51
π
|
MrRadar |
Thanks, JesseW |
01:52
π
|
JesseW |
argh, the *inventory* is nearly 800MB! |
01:53
π
|
JesseW |
The pipe I'm on right /now/ isn't so good -- I'll download that later. :-) |
01:55
π
|
bsmith093 |
JesseW:hey i had to leave that uncompressed, the whole point is so google can find it. |
02:10
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
02:12
π
|
|
MMovie has joined #archiveteam |
02:26
π
|
|
bsmith093 has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) |
02:35
π
|
|
JesseW has quit IRC (Quit: Leaving.) |
02:37
π
|
|
mafrasi2_ has quit IRC (Read error: Connection reset by peer) |
02:38
π
|
|
JesseW has joined #archiveteam |
02:38
π
|
|
mafrasi2 has joined #archiveteam |
02:38
π
|
|
yipdw_ has joined #archiveteam |
02:41
π
|
|
yipdw has quit IRC (Ping timeout: 506 seconds) |
02:54
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
02:55
π
|
|
MMovie has joined #archiveteam |
03:00
π
|
|
JesseW has quit IRC (Quit: Leaving.) |
03:05
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
03:06
π
|
|
MMovie has joined #archiveteam |
03:11
π
|
|
tomwsmf-a has quit IRC (Read error: Operation timed out) |
03:24
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
03:25
π
|
|
MMovie has joined #archiveteam |
03:42
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
03:43
π
|
|
Boppen has joined #archiveteam |
03:43
π
|
|
MMovie has joined #archiveteam |
03:48
π
|
|
bwn has quit IRC (Ping timeout: 492 seconds) |
03:58
π
|
|
ndiddy has joined #archiveteam |
04:03
π
|
|
xXx_ndidd has joined #archiveteam |
04:12
π
|
|
xXx_ndidd has quit IRC (Read error: Connection reset by peer) |
04:13
π
|
|
xXx_ndidd has joined #archiveteam |
04:16
π
|
|
Boppen has quit IRC (Ping timeout: 200 seconds) |
04:16
π
|
|
ndiddy has quit IRC (Read error: Operation timed out) |
04:21
π
|
|
JesseW has joined #archiveteam |
04:26
π
|
JesseW |
A games database that is looking for a home -- http://forum.kodi.tv/showthread.php?tid=261575 someone should suggest archive.org for them. |
04:34
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
04:35
π
|
JesseW |
Nemo_bis: it looks like the last wikiteam dump of the archiveteam wiki was in october 2015 -- could you make another one? |
04:36
π
|
JesseW |
(I'm asking you because you are listed as the uploader for https://archive.org/details/wiki-archiveteamorg ) |
04:36
π
|
|
MMovie has joined #archiveteam |
04:38
π
|
|
bsmith093 has joined #archiveteam |
04:49
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
04:50
π
|
|
MMovie has joined #archiveteam |
05:07
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
05:08
π
|
|
MMovie has joined #archiveteam |
05:10
π
|
|
xXx_ndidd has quit IRC (Read error: Connection reset by peer) |
05:21
π
|
|
Sk1d has quit IRC (Ping timeout: 250 seconds) |
05:24
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
05:25
π
|
|
MMovie has joined #archiveteam |
05:30
π
|
|
Sk1d has joined #archiveteam |
05:41
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
05:43
π
|
|
MMovie has joined #archiveteam |
05:45
π
|
|
myself has joined #archiveteam |
05:46
π
|
myself |
yo bitches |
05:46
π
|
myself |
http://www.ridethemindway.com/phones/ |
05:47
π
|
myself |
no idea where that came from or how long it'll be up, but something tells me, not forever |
05:47
π
|
|
myself has quit IRC (Client Quit) |
05:50
π
|
|
aksel has joined #archiveteam |
05:50
π
|
aksel |
Why did stypi shut down |
05:50
π
|
aksel |
? |
05:50
π
|
aksel |
Hello? |
05:50
π
|
aksel |
Is anyone here? |
05:50
π
|
aksel |
Why did Code.Stypi Shutdown |
05:51
π
|
aksel |
. |
05:51
π
|
|
aksel has quit IRC (Client Quit) |
06:01
π
|
|
atank1 has joined #archiveteam |
06:01
π
|
atank1 |
hello |
06:03
π
|
JesseW |
I'd never heard of Code.Stypi before. |
06:03
π
|
atank1 |
Code.stypi.com was a online source project that allowed programming languages on docs that you could edit with friends |
06:04
π
|
JesseW |
apparently I forgot it, as I created a wiki page for it back in Aug 2015: http://archiveteam.org/index.php?title=Stypi&action=history |
06:05
π
|
atank1 |
Wait you work with the the team?\ |
06:05
π
|
JesseW |
It doesn't look like we made any specific effort to save it. |
06:05
π
|
atank1 |
What happend to it? |
06:06
π
|
atank1 |
Why did they shut it down? |
06:06
π
|
atank1 |
Do you know why they shut Code.Stypi.com Down? |
06:06
π
|
JesseW |
Apparently whoever was running it decided to stop. I don't see any farewell notice, although apparently there was something saying it was going to die on Sept 3, 2015. |
06:07
π
|
atank1 |
oh |
06:07
π
|
atank1 |
is there an Archive with the websites source? like the code i would love to bring it back as my students are quite depressed. |
06:08
π
|
|
VADemon has quit IRC (Quit: left4dead) |
06:08
π
|
JesseW |
I don't think so. It doesn't look like the source for it was available. |
06:09
π
|
atank1 |
I decided to try and talk to somoene, I know it went down a while ago but. i want to try and get it back. |
06:09
π
|
atank1 |
Shit. |
06:09
π
|
atank1 |
Well uhh |
06:09
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
06:09
π
|
atank1 |
do you know who created it? |
06:09
π
|
JesseW |
I think there are alternatives, though. |
06:09
π
|
atank1 |
Such as? |
06:09
π
|
JesseW |
Collaborative text editing, certainly -- and I think collaborative programming too. Not sure of names offhand, though. |
06:10
π
|
|
MMovie has joined #archiveteam |
06:10
π
|
JesseW |
I'm digging around in the Wayback Machine copy, to see if I can dig up any relevant contact info. |
06:11
π
|
JesseW |
You can do the same. |
06:11
π
|
atank1 |
? |
06:11
π
|
JesseW |
e.g. https://web.archive.org/web/20130514113228/https://www.stypi.com/press |
06:11
π
|
atank1 |
._. |
06:12
π
|
atank1 |
I was asking around and someone said they would sell the source code for a couple thousand |
06:12
π
|
JesseW |
It looks like they were owned by Salesforce. So that'd be who you should contact. |
06:12
π
|
atank1 |
Bullshit lol |
06:12
π
|
JesseW |
Please let us know if you have any luck. |
06:12
π
|
atank1 |
Ok |
06:12
π
|
JesseW |
but they were aquired back in 2012 |
06:12
π
|
JesseW |
so it wasn't the aquasition that killed them |
06:13
π
|
JesseW |
and this gives their address (back in 2012) https://web.archive.org/web/20150320123654/https://code.stypi.com/privacy |
06:14
π
|
atank1 |
apparently someone i know attally knows where a source can be located |
06:14
π
|
JesseW |
neat! |
06:14
π
|
JesseW |
if you get a hold of it, please upload a copy to the Internet Archive |
06:15
π
|
JesseW |
Their tweets (all 42 of them) are hidden: https://twitter.com/stypi |
06:19
π
|
JesseW |
according to https://www.technologyreview.com/s/425690/google-wave-reincarnated/ the founders names were: Byron Milligan and Jason Chen. |
06:20
π
|
atank1 |
? |
06:20
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
06:20
π
|
JesseW |
You could try emailing/tweeting at them. |
06:22
π
|
|
Stiletto has joined #archiveteam |
06:22
π
|
|
MMovie has joined #archiveteam |
06:26
π
|
atank1 |
? |
06:26
π
|
atank1 |
grr |
06:28
π
|
JesseW |
? |
06:28
π
|
atank1 |
He lied to me |
06:29
π
|
JesseW |
your contact who said they had a copy of the source code? damm, that sucks |
06:29
π
|
atank1 |
Yep |
06:30
π
|
JesseW |
I can't say I'm surprised, but I'm sorry to hear it. |
06:34
π
|
atank1 |
Attually |
06:34
π
|
atank1 |
Since all my students robotic programming is gone... |
06:34
π
|
atank1 |
well |
06:35
π
|
atank1 |
i guess i just have to break the news |
06:35
π
|
atank1 |
they wanted me to grab their code for them |
06:36
π
|
atank1 |
oh dear |
06:36
π
|
atank1 |
i hope this does not get me fired |
06:37
π
|
JesseW |
I hope not. That's an awful place to get stuck in. |
06:37
π
|
JesseW |
Beware The Cloud. |
06:37
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
06:37
π
|
JesseW |
Always Have Multiple Local Backups |
06:38
π
|
|
dashcloud has joined #archiveteam |
06:39
π
|
atank1 |
We did |
06:40
π
|
atank1 |
but last week the Servers got attacked by Crpytowall |
06:40
π
|
JesseW |
OH FUCK. That REALLY sucks. |
06:40
π
|
atank1 |
ik |
06:41
π
|
JesseW |
BTW, thank you *VERY MUCH* for telling your story here. It's shit like this that demonstrates why what we do (or failed to do in this case) is important. |
06:42
π
|
atank1 |
anyways imma go |
06:42
π
|
bsmith093 |
JesseW: not to distract from the suck of cryptowall, but did you ever get that tar --> zip thing going? |
06:43
π
|
atank1 |
fuck im lagging my dads internet |
06:43
π
|
atank1 |
i should get off |
06:43
π
|
bsmith093 |
atank1: hope it goes well |
06:43
π
|
atank1 |
My dad aint happy |
06:43
π
|
JesseW |
It'll be a few days -- I want to keep my new IA census workstuff around until jjake gets the census stuff uploaded. |
06:44
π
|
atank1 |
I borrow the internet from my dad next door lol |
06:44
π
|
bsmith093 |
JesseW: k then, thanks |
06:44
π
|
atank1 |
cya |
06:44
π
|
JesseW |
bsmith093 but I should be able to start downloading the file at least |
06:44
π
|
|
atank1 has quit IRC () |
06:44
π
|
JesseW |
Well, that was a dammed sob story. :-( |
06:44
π
|
bsmith093 |
yeah that sucks, who *lies* about having source code? |
06:51
π
|
yipdw_ |
"Most importantly, Stypi will continue to be the Stypi you know. Our users will continue to have access to this great service, community, and innovation." |
06:51
π
|
yipdw_ |
nice |
06:51
π
|
JesseW |
Where is that from? |
06:51
π
|
yipdw_ |
https://web.archive.org/web/20130514120823/http://blog.stypi.com/ |
06:52
π
|
yipdw_ |
more specifically https://web.archive.org/web/20130325111746/http://blog.stypi.com/2012/05/stypi-joins-salesforce-com/ |
06:52
π
|
JesseW |
well, it wasn't being bought that killed them -- they didn't die till 3 years later. |
06:53
π
|
JesseW |
bsmith093: fanfiction download started. |
06:53
π
|
yipdw_ |
true but Bram Mooleenar switching jobs to Google didn't kill vim 3 years later |
06:54
π
|
bsmith093 |
JesseW: thanks. do, Stypi didn't actually say they were dumping anything. they usually make a point of that. |
06:54
π
|
bsmith093 |
startups in general, i mean |
06:54
π
|
JesseW |
and that illustrates the difference between a piece of FOSS standalone software and a proprietary service |
06:55
π
|
yipdw_ |
I mean it's nothing that nobody in here doesn't know |
06:55
π
|
JesseW |
bsmith093: they did say they were deleting it: "All documents that have not been downloaded to an archive by that time will be deleted. " |
06:55
π
|
yipdw_ |
it really sucks for atank1 and unless someone here happens to have an archive they may just be out of luck |
06:56
π
|
JesseW |
bsmith093: ETA on the fanfict download is 2days, 5 hours. :-) |
06:56
π
|
bsmith093 |
JesseW: just curious, what isp do you have? |
06:57
π
|
bsmith093 |
theres also a torrent file |
06:58
π
|
JesseW |
bsmith093: Wave G in Seattle |
06:59
π
|
JesseW |
Hm, I suppose I'll switch over to the torrent. |
06:59
π
|
bsmith093 |
JesseW: never heard of them, any good? |
06:59
π
|
JesseW |
They used to be called CondoInternet |
07:00
π
|
JesseW |
I've been very happy with them. Only complaint is that they keep sending me junk mail offering me a discount to sign up -- after I've already signed up. :-) |
07:00
π
|
bsmith093 |
how fast |
07:01
π
|
bsmith093 |
also will someone with ops please add http://www.ridethemindway.com/phones/ to archivebot yipdw_ SketchCow ersi |
07:01
π
|
JesseW |
bsmith093 already in there |
07:02
π
|
JesseW |
check the dashboard |
07:02
π
|
JesseW |
bsmith093: I pay for 100 Mbps |
07:02
π
|
bsmith093 |
great, it looks like 80s era phone docs |
07:02
π
|
bsmith093 |
those are rare |
07:03
π
|
bsmith093 |
so do i, new isp, faster than twc, for much cheaper |
07:03
π
|
JesseW |
It's already grabbed 5 GB, with about 1000 files to go |
07:03
π
|
JesseW |
I'm just delighted to not have to deal with the big ISPs. Those folks are simply unpleasant to deal with. |
07:05
π
|
bsmith093 |
mine's been down once since i got them, like ~2 years ago, for maybe a half hour, when i called them, they ACTUALLY TOLD ME WHY!! |
07:05
π
|
JesseW |
that is excellent, yeah |
07:07
π
|
bsmith093 |
JesseW: also grab the inventory file, that might not be int the torrent yet, i don't know how fast that updates |
07:09
π
|
JesseW |
It's not in the torrent I'm using (with hash d934709d1c7f1bf26d826718804de5f7a53757dc) |
07:10
π
|
bsmith093 |
it's on the page though. |
07:10
π
|
JesseW |
I know. I'll grab it afterward -- I mean, I can regenerate it myself once I have the data. :-) |
07:13
π
|
JesseW |
The torrent ETA is 1 day, 19 hours right now. |
07:13
π
|
JesseW |
or 1 day, 14 hours. |
07:27
π
|
|
vitzli has joined #archiveteam |
07:32
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
07:34
π
|
|
MMovie has joined #archiveteam |
07:37
π
|
|
JesseW has quit IRC (Quit: Leaving.) |
07:43
π
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
07:45
π
|
|
DFJustin has quit IRC (Remote host closed the connection) |
07:46
π
|
|
dashcloud has joined #archiveteam |
07:50
π
|
|
metalcamp has joined #archiveteam |
08:00
π
|
|
DFJustin has joined #archiveteam |
08:00
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
08:01
π
|
|
MMovie has joined #archiveteam |
08:07
π
|
|
brayden has quit IRC (Read error: Connection reset by peer) |
08:07
π
|
|
brayden has joined #archiveteam |
08:17
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
08:18
π
|
|
MMovie has joined #archiveteam |
08:36
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
08:38
π
|
|
MMovie has joined #archiveteam |
08:54
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
08:56
π
|
|
MMovie has joined #archiveteam |
09:07
π
|
|
Tomcat_ has joined #archiveteam |
09:19
π
|
Burak |
How can I add into wget -H -D domains, that looks like this - imagesX.fotosik.pl, where X is number from 1 to 99? |
09:25
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
09:27
π
|
|
MMovie has joined #archiveteam |
09:29
π
|
|
Zei-Pii has joined #archiveteam |
09:38
π
|
|
vitzli has quit IRC (Leaving) |
09:41
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
09:42
π
|
|
MMovie has joined #archiveteam |
09:48
π
|
|
bwn has joined #archiveteam |
09:58
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
09:58
π
|
|
MMovie has joined #archiveteam |
10:04
π
|
|
hictooth has quit IRC (Ping timeout: 255 seconds) |
10:13
π
|
joepie91 |
https://twitter.com/trulloapp/status/702225155464900608 |
10:13
π
|
joepie91 |
"Thank you all for your support & app love. Sadly, we're shutting down Trullo. We are grateful for your contributions and we'll miss you. :(" |
10:13
π
|
joepie91 |
well |
10:13
π
|
joepie91 |
looks like they didn't waste any time |
10:13
π
|
joepie91 |
DNS doesn't resolve anymore |
10:14
π
|
zhongfu |
wow |
10:14
π
|
joepie91 |
https://www.producthunt.com/tech/trullo |
10:16
π
|
joepie91 |
well |
10:16
π
|
joepie91 |
this has got to be one of the most dickish shutdowns |
10:16
π
|
joepie91 |
I think they just beat Yahoo |
10:16
π
|
joepie91 |
shutdown announcement with no notice |
10:16
π
|
joepie91 |
DNS gone 5 days later |
10:22
π
|
|
hictooth has joined #archiveteam |
10:23
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
10:23
π
|
PurpleSym |
Is there some kind of βDNS archiveβ you could go back to, fetch the IP address and see if the server still works? |
10:23
π
|
|
hictooth has quit IRC (Client Quit) |
10:25
π
|
|
MMovie has joined #archiveteam |
10:25
π
|
joepie91 |
PurpleSym: potentially, sec |
10:26
π
|
joepie91 |
PurpleSym: https://www.robtex.com/?dns=trullo.com |
10:26
π
|
joepie91 |
so yeah |
10:26
π
|
joepie91 |
52.24.188.223 and 52.24.194.8 |
10:27
π
|
PurpleSym |
Also: https://dnshistory.org/dns-records/trullo.com |
10:27
π
|
joepie91 |
AWS |
10:27
π
|
joepie91 |
IPs non-responsive |
10:27
π
|
joepie91 |
PurpleSym: robtex is more complet e:P |
10:28
π
|
PurpleSym |
Indeed. |
10:28
π
|
joepie91 |
I <3 robtex |
10:28
π
|
joepie91 |
the NSA does too, for obvious reasons |
10:29
π
|
PurpleSym |
We should get a copy. |
10:29
π
|
PurpleSym |
Anyway, was worth a shot⦠|
10:30
π
|
joepie91 |
PurpleSym: copy of? |
10:32
π
|
PurpleSym |
Robtex. |
10:32
π
|
joepie91 |
heh. |
10:32
π
|
joepie91 |
robtex is big |
10:32
π
|
joepie91 |
I still need to eventually talk to the guy and see if some kind of feed can be negotiated |
10:32
π
|
joepie91 |
he's supposedly working on an API for 'qualified organizations' but it's not entirely clear to me what that would mean |
10:33
π
|
joepie91 |
but |
10:33
π
|
joepie91 |
-bs |
10:45
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
10:46
π
|
|
MMovie has joined #archiveteam |
11:03
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
11:04
π
|
|
MMovie has joined #archiveteam |
11:16
π
|
|
winterfox has quit IRC (Remote host closed the connection) |
11:21
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
11:23
π
|
|
MMovie has joined #archiveteam |
11:41
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
11:43
π
|
|
MMovie has joined #archiveteam |
11:57
π
|
|
schbirid has joined #archiveteam |
11:59
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
12:00
π
|
|
MMovie has joined #archiveteam |
12:13
π
|
|
philpem has joined #archiveteam |
12:16
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
12:17
π
|
|
MMovie has joined #archiveteam |
12:34
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
12:35
π
|
|
MMovie has joined #archiveteam |
12:53
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
12:54
π
|
arkiver |
Is someone interested in scannig FTPs for the FTP project? |
12:54
π
|
arkiver |
Instructions and FTPs are here http://archiveteam.org/index.php?title=FTP/List |
12:54
π
|
|
MMovie has joined #archiveteam |
12:54
π
|
arkiver |
This is only scanning the FTP and creating a list of items for the grab. This won't take a lot of diskspace. |
12:56
π
|
arkiver |
It might take a lot of time, depending on the number of files and the speed of the FTP |
13:03
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
13:05
π
|
|
MMovie has joined #archiveteam |
13:10
π
|
|
snape has quit IRC (Hey! Where'd my controlling terminal go?) |
13:18
π
|
HCross |
Shall I kick a scan off on ftp://ftp.cup.cam.ac.uk - seems to have a lot of stuff on books published by Cambridge University Press |
13:20
π
|
HCross |
nvm, its done |
13:21
π
|
HCross |
or it isnt |
13:36
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
13:37
π
|
|
philpem has quit IRC (Ping timeout: 260 seconds) |
13:37
π
|
|
MMovie has joined #archiveteam |
13:55
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
13:56
π
|
|
MMovie has joined #archiveteam |
14:13
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
14:14
π
|
PurpleSym |
arkiver: Would you accept the output of `ncftpls -R`? |
14:15
π
|
arkiver |
I'm not sure what kind of output that gives |
14:15
π
|
|
MMovie has joined #archiveteam |
14:15
π
|
arkiver |
URL and filesize? |
14:15
π
|
arkiver |
if yes, then I can convert it, if no, then no |
14:16
π
|
arkiver |
Though using the ftp-queue scripts would be best for this (scripts will be more optimized in the future) |
14:16
π
|
PurpleSym |
Output looks like this: http://pastebin.com/t6vcPHbD |
14:17
π
|
arkiver |
Looks like URL can be generated and size is also there |
14:17
π
|
arkiver |
so I should be able to convert it |
14:17
π
|
arkiver |
Why would you use that command rather then the ftp-queue script? |
14:18
π
|
PurpleSym |
I donβt see why a script needed to be written for that in the first place :) |
14:19
π
|
arkiver |
The script wil make sure only new files are size changed files are added to the itemlists |
14:20
π
|
arkiver |
Previously scanned FTPs are also in /archive/, so they can be used for that if they are scanned again |
14:20
π
|
arkiver |
It creates smaller lists of 200 MB of FTP files |
14:20
π
|
PurpleSym |
`cat old new | sort | uniq` |
14:21
π
|
arkiver |
Also tests for the server response if a file or folder does not exist |
14:30
π
|
|
megaminxw has quit IRC (Quit: Leaving.) |
14:30
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
14:31
π
|
|
MMovie has joined #archiveteam |
14:35
π
|
|
ohhdemgir has quit IRC (Read error: Operation timed out) |
14:47
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
14:48
π
|
|
MMovie has joined #archiveteam |
14:48
π
|
|
zhongfu has quit IRC (Remote host closed the connection) |
14:59
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
15:00
π
|
|
test_ has joined #archiveteam |
15:00
π
|
test_ |
uhh hello? |
15:00
π
|
arkiver |
hi |
15:01
π
|
test_ |
can i request something for deletion |
15:01
π
|
test_ |
personal info |
15:01
π
|
arkiver |
Requests for deletion of something should be sent to info@archive.org |
15:01
π
|
test_ |
ok thanks |
15:01
π
|
|
MMovie has joined #archiveteam |
15:01
π
|
test_ |
will do, bye |
15:01
π
|
|
test_ has quit IRC (Client Quit) |
15:11
π
|
|
scyther has joined #archiveteam |
15:31
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
15:33
π
|
|
MMovie has joined #archiveteam |
15:50
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
15:52
π
|
|
MMovie has joined #archiveteam |
16:00
π
|
|
snape has joined #archiveteam |
16:06
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
16:07
π
|
|
MMovie has joined #archiveteam |
16:25
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
16:26
π
|
|
MMovie has joined #archiveteam |
16:30
π
|
|
ats has quit IRC (Quit: Let's see if Linux 4.4.3 has working NFS again...) |
16:36
π
|
|
ats has joined #archiveteam |
16:40
π
|
|
Zei-Pii has quit IRC (Read error: Connection reset by peer) |
16:41
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
16:42
π
|
|
MMovie has joined #archiveteam |
16:52
π
|
joepie91 |
arkiver: whatever happened to the dump of open FTPs that I had a while ago? |
16:52
π
|
joepie91 |
:p |
16:54
π
|
|
philpem has joined #archiveteam |
17:10
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
17:12
π
|
|
MMovie has joined #archiveteam |
17:30
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
17:30
π
|
|
scyther_ has joined #archiveteam |
17:31
π
|
|
MMovie has joined #archiveteam |
17:32
π
|
|
scyther has quit IRC (Ping timeout: 250 seconds) |
17:45
π
|
arkiver |
joepie91: I'll look into that! |
17:47
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
17:49
π
|
arkiver |
joepie91: do you still have that dump? |
17:50
π
|
|
MMovie has joined #archiveteam |
17:52
π
|
joepie91 |
arkiver: eh, might have, but my files are a mess atm |
17:59
π
|
|
zhongfu has joined #archiveteam |
18:00
π
|
joepie91 |
arkiver: remind me of the filename? |
18:06
π
|
|
zhongfu has quit IRC (Remote host closed the connection) |
18:11
π
|
|
JesseW has joined #archiveteam |
18:15
π
|
|
metalcamp has quit IRC (Ping timeout: 252 seconds) |
18:16
π
|
|
zhongfu has joined #archiveteam |
18:17
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
18:18
π
|
|
MMovie has joined #archiveteam |
18:36
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
18:37
π
|
|
MMovie has joined #archiveteam |
18:41
π
|
|
scyther_ has quit IRC (Read error: Connection reset by peer) |
18:54
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
18:56
π
|
|
MMovie has joined #archiveteam |
19:04
π
|
|
godane has quit IRC (Read error: Operation timed out) |
19:07
π
|
|
victor has joined #archiveteam |
19:11
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
19:12
π
|
|
MMovie has joined #archiveteam |
19:29
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
19:31
π
|
|
MMovie has joined #archiveteam |
19:33
π
|
|
Infreq has quit IRC (Ping timeout: 258 seconds) |
19:34
π
|
|
Infreq has joined #archiveteam |
19:35
π
|
|
zino_ has joined #archiveteam |
19:37
π
|
|
Burak has quit IRC (Ping timeout: 255 seconds) |
19:38
π
|
|
schbirid has quit IRC (hub.efnet.us irc.Prison.NET) |
19:38
π
|
|
zino has quit IRC (hub.efnet.us irc.Prison.NET) |
19:38
π
|
|
vOYtEC has quit IRC (hub.efnet.us irc.Prison.NET) |
19:38
π
|
|
achip has quit IRC (hub.efnet.us irc.Prison.NET) |
19:42
π
|
|
schbirid2 has joined #archiveteam |
19:57
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
19:57
π
|
|
achip has joined #archiveteam |
19:58
π
|
|
MMovie has joined #archiveteam |
19:58
π
|
|
vOYtEC has joined #archiveteam |
20:02
π
|
|
Burak has joined #archiveteam |
20:11
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
20:12
π
|
|
ndiddy has joined #archiveteam |
20:12
π
|
|
megaminxw has joined #archiveteam |
20:13
π
|
|
MMovie has joined #archiveteam |
20:29
π
|
|
metalcamp has joined #archiveteam |
20:47
π
|
|
Burak has quit IRC (Ping timeout: 255 seconds) |
20:48
π
|
|
JesseW has quit IRC (Quit: Leaving.) |
20:48
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
20:49
π
|
|
MMovie has joined #archiveteam |
20:50
π
|
|
scyther has joined #archiveteam |
21:04
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
21:05
π
|
|
MMovie has joined #archiveteam |
21:06
π
|
|
Burak has joined #archiveteam |
21:22
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
21:23
π
|
|
MMovie has joined #archiveteam |
21:26
π
|
|
metalcamp has quit IRC (Ping timeout: 252 seconds) |
21:32
π
|
|
bwn has quit IRC (Ping timeout: 246 seconds) |
21:39
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
21:40
π
|
|
MMovie has joined #archiveteam |
21:43
π
|
|
Tomcat_ has quit IRC (Remote host closed the connection) |
21:51
π
|
|
Boppen has joined #archiveteam |
21:54
π
|
|
mismatch_ has joined #archiveteam |
22:03
π
|
|
schbirid2 has quit IRC (Quit: Leaving) |
22:07
π
|
|
Boppen has quit IRC (hub.se irc.du.se) |
22:11
π
|
|
bwn has joined #archiveteam |
22:25
π
|
|
Boppen has joined #archiveteam |
22:27
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
22:29
π
|
|
MMovie has joined #archiveteam |
22:30
π
|
|
JesseW has joined #archiveteam |
22:30
π
|
|
scyther has quit IRC (Read error: Connection reset by peer) |
22:30
π
|
|
bwn has quit IRC (Read error: Operation timed out) |
22:34
π
|
|
scyther has joined #archiveteam |
22:44
π
|
|
Boppen has quit IRC (Ping timeout: 200 seconds) |
22:47
π
|
|
Boppen has joined #archiveteam |
22:53
π
|
|
Boppen has quit IRC (hub.se irc.du.se) |
22:56
π
|
|
megaminxw has quit IRC (Quit: Leaving.) |
22:59
π
|
|
mismatch_ has quit IRC (Ping timeout: 499 seconds) |
23:06
π
|
|
rduser has quit IRC (Ping timeout: 260 seconds) |
23:06
π
|
|
Rickster has quit IRC (Ping timeout: 260 seconds) |
23:08
π
|
|
Famicoman has quit IRC (Ping timeout: 260 seconds) |
23:09
π
|
|
Simpbrai_ has quit IRC (Remote host closed the connection) |
23:10
π
|
|
bauruine has quit IRC (Ping timeout: 260 seconds) |
23:10
π
|
|
mismatch_ has joined #archiveteam |
23:10
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
23:11
π
|
|
rduser has joined #archiveteam |
23:12
π
|
|
MMovie has joined #archiveteam |
23:12
π
|
|
bauruine has joined #archiveteam |
23:13
π
|
|
Rickster has joined #archiveteam |
23:14
π
|
|
Simpbrai_ has joined #archiveteam |
23:27
π
|
|
MMovie has quit IRC (Read error: Operation timed out) |
23:28
π
|
|
MMovie has joined #archiveteam |
23:43
π
|
arkiver |
BnA-Robin: if you're interested in another project to run you might like FTP |
23:43
π
|
arkiver |
restarted today and we don't have a lot of people running it yet |
23:46
π
|
|
Boppen has joined #archiveteam |
23:48
π
|
arkiver |
SketchCow: What do you think of saving LiveJournal? We can make it a long running project, maybe over a year, so it won't need a lot of resources |
23:49
π
|
arkiver |
If you give the go we'll have a project running soon for livejournal |
23:54
π
|
|
bwn has joined #archiveteam |
23:54
π
|
|
scyther has quit IRC (Quit: Leaving) |