| Time |
Nickname |
Message |
|
00:47
🔗
|
|
Stiletto has quit IRC () |
|
01:07
🔗
|
arkiver |
thanks chfoo yipdw |
|
01:10
🔗
|
|
j08nY has quit IRC (Remote host closed the connection) |
|
01:12
🔗
|
|
BlueMaxim has joined #archiveteam-bs |
|
01:15
🔗
|
|
Stilett0 has joined #archiveteam-bs |
|
01:15
🔗
|
|
Stilett0 has quit IRC (Client Quit) |
|
01:17
🔗
|
|
Stilett0 has joined #archiveteam-bs |
|
01:18
🔗
|
|
Stilett0 is now known as Stiletto |
|
01:21
🔗
|
|
ndiddy has quit IRC () |
|
01:28
🔗
|
|
ndiddy has joined #archiveteam-bs |
|
01:30
🔗
|
|
schbirid2 has joined #archiveteam-bs |
|
01:33
🔗
|
|
schbirid has quit IRC (Read error: Operation timed out) |
|
01:46
🔗
|
|
REiN^ has quit IRC (Max SendQ exceeded) |
|
01:47
🔗
|
|
BlueMaxim has quit IRC (Read error: Operation timed out) |
|
01:47
🔗
|
|
REiN^ has joined #archiveteam-bs |
|
02:12
🔗
|
|
Stiletto has quit IRC (Read error: Operation timed out) |
|
02:12
🔗
|
|
Stilett0 has joined #archiveteam-bs |
|
02:28
🔗
|
SketchCow |
OK, I'm fixing all the New Computer Express stuff the guy's uploading. |
|
02:28
🔗
|
SketchCow |
https://archive.org/details/NewComputerExpress000 will be the first one that finishes, I bet |
|
02:34
🔗
|
|
pizzaiolo has quit IRC (Quit: pizzaiolo) |
|
03:18
🔗
|
|
BlueMaxim has joined #archiveteam-bs |
|
04:31
🔗
|
|
Odd0002 has quit IRC (Remote host closed the connection) |
|
04:56
🔗
|
|
Sk1d has quit IRC (Ping timeout: 250 seconds) |
|
05:03
🔗
|
|
Sk1d has joined #archiveteam-bs |
|
06:22
🔗
|
|
fie has quit IRC (Ping timeout: 506 seconds) |
|
06:31
🔗
|
|
fie has joined #archiveteam-bs |
|
06:37
🔗
|
|
powerArch has quit IRC (Remote host closed the connection) |
|
06:48
🔗
|
|
vitzli has joined #archiveteam-bs |
|
08:09
🔗
|
|
Honno has joined #archiveteam-bs |
|
08:11
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
08:18
🔗
|
godane |
i'm asking the retromags people if they will look at the New Computer Express on IA |
|
08:19
🔗
|
godane |
i figure they would make a edit version of it and also a smaller size cbz for there site |
|
09:16
🔗
|
|
SHODAN_UI has joined #archiveteam-bs |
|
09:55
🔗
|
godane |
SketchCow: the pdf derive sucks ass on this one: https://archive.org/details/TNM_The_Apple_Collection_Catalog |
|
09:55
🔗
|
godane |
there is like no text in the pdf at all |
|
09:56
🔗
|
godane |
the only derive that works well is normal the jp2.zip files |
|
10:14
🔗
|
|
j08nY has joined #archiveteam-bs |
|
10:56
🔗
|
|
BlueMaxim has quit IRC (Read error: Operation timed out) |
|
10:57
🔗
|
|
BlueMaxim has joined #archiveteam-bs |
|
11:02
🔗
|
|
fie has quit IRC (Quit: Leaving) |
|
11:16
🔗
|
|
fie has joined #archiveteam-bs |
|
11:39
🔗
|
|
SHODAN_UI has quit IRC (Remote host closed the connection) |
|
12:13
🔗
|
|
robinak has joined #archiveteam-bs |
|
12:13
🔗
|
|
robink has quit IRC (Read error: Connection reset by peer) |
|
13:01
🔗
|
|
C4K3 has quit IRC (Quit: leaving) |
|
13:01
🔗
|
|
C4K3 has joined #archiveteam-bs |
|
13:04
🔗
|
|
BlueMaxim has quit IRC (Read error: Connection reset by peer) |
|
13:32
🔗
|
|
fie has quit IRC (Read error: Operation timed out) |
|
13:33
🔗
|
|
yuitimoth has quit IRC (Remote host closed the connection) |
|
13:34
🔗
|
|
yuitimoth has joined #archiveteam-bs |
|
13:40
🔗
|
|
SHODAN_UI has joined #archiveteam-bs |
|
13:42
🔗
|
|
j08nY has quit IRC (Read error: Operation timed out) |
|
14:09
🔗
|
|
j08nY has joined #archiveteam-bs |
|
15:09
🔗
|
|
pizzaiolo has joined #archiveteam-bs |
|
15:30
🔗
|
rocode |
What is a good upload rate limit on uploading to IA? I noticed some of my grabs are offline, so I need to get them on IA before I get hit by a bus, but I don't want to overload the server with concurrent uploads. Is there a guideline? |
|
16:22
🔗
|
Kaz |
if you use the s3 interface, it'll throw 429's at you if you go too fast |
|
16:23
🔗
|
Kaz |
as for bandwidth, I'm sure newsgrabber was sending in the 500-700mbit/s at it pretty constantly |
|
16:29
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
16:31
🔗
|
|
godane has quit IRC (Read error: Operation timed out) |
|
16:37
🔗
|
JAA |
I'm grabbing Tanobb now. It's quite slow, probably at least partially because their servers are located in Japan, but well, I'll try to get as much as possible before they shut down. |
|
16:42
🔗
|
|
godane has joined #archiveteam-bs |
|
17:10
🔗
|
|
godane has quit IRC (Quit: Leaving.) |
|
17:37
🔗
|
|
TheLovina has joined #archiveteam-bs |
|
18:00
🔗
|
|
RichardG has quit IRC (Read error: Connection reset by peer) |
|
18:17
🔗
|
|
RichardG has joined #archiveteam-bs |
|
18:42
🔗
|
|
godane has joined #archiveteam-bs |
|
18:57
🔗
|
|
pizzaiolo has quit IRC (Read error: Operation timed out) |
|
18:58
🔗
|
|
Florian_ has joined #archiveteam-bs |
|
18:58
🔗
|
|
Florian_ has quit IRC (Client Quit) |
|
19:05
🔗
|
|
SHODAN_UI has quit IRC (Remote host closed the connection) |
|
19:32
🔗
|
|
jrwr has joined #archiveteam-bs |
|
19:53
🔗
|
odemg |
timmc, Kaz as ero was built primarily for reddit we can assume that a very high percentage of the links were posted on reddit, this allows us to grep datasets like this one - http://files.pushshift.io/reddit/ - for eroshare links and download them |
|
19:54
🔗
|
Kaz |
yeah, that should make it a ton easier to work through |
|
19:54
🔗
|
odemg |
should I get to work pulling the links out of that data? |
|
19:54
🔗
|
Kaz |
only seems to go up to april though |
|
19:55
🔗
|
Kaz |
probbaly not a bad idea to start working it through, if we do end up grabbing |
|
19:55
🔗
|
odemg |
we can worry about that afterwards, ero was born when? I know it's only bee around a few years at this point so no need to grab earlier files |
|
19:56
🔗
|
Kaz |
you probably know better than me when ero started.. :) |
|
19:56
🔗
|
Kaz |
domain is 10 years old |
|
19:57
🔗
|
Kaz |
highly doubt it's been used for that long, somehow. |
|
19:57
🔗
|
odemg |
.... whois on the domain say its was regged on 2006-11-30 |
|
19:57
🔗
|
odemg |
wut |
|
19:57
🔗
|
Kaz |
actually, #nofap is probably the best place if we're actually going to start grabbing |
|
20:02
🔗
|
|
ZexaronS has joined #archiveteam-bs |
|
20:02
🔗
|
odemg |
Kaz, explain? |
|
20:04
🔗
|
Kaz |
Project channel |
|
20:06
🔗
|
jrwr |
I got my Archive Team Warrior Stickers in |
|
20:06
🔗
|
jrwr |
they are nice! |
|
20:10
🔗
|
|
wp494 has quit IRC (Read error: Operation timed out) |
|
20:17
🔗
|
|
icedice has joined #archiveteam-bs |
|
20:20
🔗
|
timmc |
odemg: Ah, interesting. |
|
20:21
🔗
|
jrwr |
https://goo.gl/photos/MnPQMLAe1ixzjedV9 |
|
20:21
🔗
|
jrwr |
Works very well on my mug |
|
20:26
🔗
|
|
pizzaiolo has joined #archiveteam-bs |
|
20:42
🔗
|
|
Pudsey has joined #archiveteam-bs |
|
20:58
🔗
|
|
SHODAN_UI has joined #archiveteam-bs |
|
21:07
🔗
|
|
pizzaiolo has quit IRC (Ping timeout: 506 seconds) |
|
21:20
🔗
|
|
icedice has quit IRC (Quit: Leaving) |
|
21:28
🔗
|
|
pizzaiolo has joined #archiveteam-bs |
|
21:30
🔗
|
|
SilSte has quit IRC (Remote host closed the connection) |
|
21:31
🔗
|
|
SilSte has joined #archiveteam-bs |
|
21:33
🔗
|
|
Pudsey has quit IRC (Remote host closed the connection) |
|
21:40
🔗
|
jrwr |
SilSte: Howd |
|
21:40
🔗
|
|
Silas has joined #archiveteam-bs |
|
21:40
🔗
|
jrwr |
So Silas |
|
21:40
🔗
|
jrwr |
can you hover over the invaild config icon at the bottom |
|
21:40
🔗
|
jrwr |
what does it say |
|
21:41
🔗
|
|
pizzaiolo has quit IRC (Read error: Operation timed out) |
|
21:41
🔗
|
Kaz |
jrwr: re cygwin port, I guess it's a good idea if it's stable and works |
|
21:41
🔗
|
Silas |
its just a warning about there not being enough video memory to go into fullscreen or seamless |
|
21:41
🔗
|
jrwr |
Ok |
|
21:41
🔗
|
Kaz |
that said, WSL might be better? albeit only win10 support |
|
21:41
🔗
|
jrwr |
damn, that should boot then |
|
21:41
🔗
|
|
pizzaiolo has joined #archiveteam-bs |
|
21:42
🔗
|
jrwr |
Kaz: true but since its pretty simple on the scripts, and would want it to even work on win7/win8 |
|
21:42
🔗
|
jrwr |
make a little installer + some shortcuts to some scripts to turn off the warrior and such |
|
21:43
🔗
|
Silas |
i should check my bios to make sure vt-x is enabled in the first place |
|
21:43
🔗
|
jrwr |
the processor doesn't support it |
|
21:43
🔗
|
Silas |
oh |
|
21:43
🔗
|
Silas |
:/ |
|
21:44
🔗
|
jrwr |
VMware player might work in the this case, it can import the OVA |
|
21:44
🔗
|
jrwr |
Or |
|
21:44
🔗
|
jrwr |
a 2.99 Euro a month Virtual Machine Instance at scaleway works very well |
|
21:44
🔗
|
Silas |
ill try out vmware player |
|
21:45
🔗
|
jrwr |
Let us know how it goes, ill be here all night |
|
21:45
🔗
|
jrwr |
Kaz: if we keep the docker support, Ill work on a installer for it, wont be too hard at all. ill do some testing today on it, I'm pretty bored right now |
|
21:54
🔗
|
|
Odd0002 has joined #archiveteam-bs |
|
21:56
🔗
|
JAA |
Seriously though, the warrior is a ridiculous security risk (which is why I'd never let it anywhere near my machines). I'm not sure about the Docker container, though based on a quick look it seems to be based on a 3-month-old version of phusion/baseimage and probably also has some security issues. |
|
21:57
🔗
|
Silas |
I got the warrior running in VMWare Player and set to run whatever ArchiveTeam's Choice is, everything looks good! |
|
21:57
🔗
|
Silas |
Thanks jrwr |
|
21:57
🔗
|
jrwr |
Awesome! |
|
21:57
🔗
|
jrwr |
jrwr: Ill have the cygwin version only listen on 127.0.0.1 |
|
21:58
🔗
|
Odd0002 |
did you just talk to yourself? |
|
21:58
🔗
|
jrwr |
I did |
|
21:58
🔗
|
jrwr |
I ment JAA |
|
21:58
🔗
|
Odd0002 |
ah |
|
21:58
🔗
|
JAA |
Well, it still needs an internet connection... |
|
21:58
🔗
|
jrwr |
Ya |
|
21:59
🔗
|
jrwr |
and it is running random code from the internet |
|
21:59
🔗
|
jrwr |
:) |
|
22:00
🔗
|
JAA |
True. It also auto-updates scripts and fetches the wget-lua source via HTTP without a checksum check. :-| (Ping JensRex, you implemented that, didn't you? What happened with that?) |
|
22:00
🔗
|
JAA |
As I said, ridiculous security risk. |
|
22:01
🔗
|
jrwr |
it is a botnet after all |
|
22:02
🔗
|
JAA |
That's true, but it could be improved a lot. |
|
22:02
🔗
|
jrwr |
Yes |
|
22:03
🔗
|
jrwr |
I've though about doing it, making a new Virtual Machine, but I think that should be up to the project managers, as I have no say around here |
|
22:03
🔗
|
|
decay has quit IRC (Read error: Operation timed out) |
|
22:03
🔗
|
|
decay has joined #archiveteam-bs |
|
22:12
🔗
|
jrwr |
wow, that was easy.. almost too easy |
|
22:21
🔗
|
|
Silas has quit IRC (Quit: help) |
|
22:21
🔗
|
|
Silas has joined #archiveteam-bs |
|
22:24
🔗
|
|
Silas has quit IRC (Client Quit) |
|
22:45
🔗
|
DFJustin |
we've had problems with people using cygwin in the past because of the case-insensitive filesystem and certain filenames being illegal on windows |
|
22:45
🔗
|
DFJustin |
that's one of the reasons the warrior was created in the first place |
|
22:45
🔗
|
DFJustin |
to have a consistent no-surprises environment |
|
22:47
🔗
|
|
Ravenloft has joined #archiveteam-bs |
|
22:48
🔗
|
Ravenloft |
http://www.os2museum.com/wp/rich-heimlichs-patch-set-overview/ |
|
22:48
🔗
|
DFJustin |
if you're not willing to run the warrior and you don't have access to a linux system then it's better to just let someone else do it rather than cause problems |
|
22:48
🔗
|
|
SHODAN_UI has quit IRC (Remote host closed the connection) |
|
22:48
🔗
|
DFJustin |
we usually don't have a shortage of warriors running |
|
22:50
🔗
|
jrwr |
This is true |
|
22:50
🔗
|
jrwr |
also wget-lua hates cygwin atm |
|
22:53
🔗
|
* |
jrwr is just trying to be helpful |
|
22:55
🔗
|
DFJustin |
yeah it's just that if it creates a mess and an admin has to clean it up that far outweighs the benefit of 1 more client running |
|
22:56
🔗
|
DFJustin |
usually the rate limiter is the site being archived or our staging servers rather than number of people crawling |
|
22:56
🔗
|
|
Silas has joined #archiveteam-bs |
|
22:57
🔗
|
|
Ravenloft has quit IRC () |
|
22:58
🔗
|
|
Ravenloft has joined #archiveteam-bs |
|
23:09
🔗
|
jrwr |
Ya i provided a staging server for pixiv |
|
23:09
🔗
|
jrwr |
it was crazy the amount of traffic inbound I was getting, poor FOS just cant keep up |
|
23:10
🔗
|
joepie91 |
jrwr: there's not really such a thing as "the project managers" :P |
|
23:11
🔗
|
joepie91 |
stuff gets done when somebody decides to do it |
|
23:11
🔗
|
Silas |
i love the fact there's a leaderboard |
|
23:11
🔗
|
Silas |
im a sucker for stat tracking lol |
|
23:12
🔗
|
jrwr |
Its a loose collection of people, There are people with access to infra and they tend to help when a need arises, those are the true "managers" of AT |
|
23:12
🔗
|
jrwr |
like I would save arkiver has been a amazing "manager" of savepixiv |
|
23:14
🔗
|
Ravenloft |
thats what she said |
|
23:19
🔗
|
|
j08nY has quit IRC (Remote host closed the connection) |
|
23:20
🔗
|
|
j08nY has joined #archiveteam-bs |
|
23:40
🔗
|
JAA |
What the hell, wpull is suddenly ignoring my --reject-regex options. O.o |
|
23:46
🔗
|
|
Silas has quit IRC (Quit: Page closed) |
|
23:48
🔗
|
JAA |
Ok, looks like it never worked, but now I seriously wonder why. |
|
23:52
🔗
|
JAA |
Ooh, you can only have one --reject-regex option. Well, that's... unexpected. |
|
23:59
🔗
|
|
Honno has quit IRC (Read error: Operation timed out) |