| Time |
Nickname |
Message |
|
00:02
🔗
|
|
Mateon1 has quit IRC (Read error: Operation timed out) |
|
00:02
🔗
|
|
Mateon1 has joined #archiveteam-bs |
|
00:33
🔗
|
|
C4K3_ has joined #archiveteam-bs |
|
00:33
🔗
|
|
C4K3 has quit IRC (Read error: Connection reset by peer) |
|
00:38
🔗
|
|
Fletcher has quit IRC (Remote host closed the connection) |
|
00:46
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
00:55
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
01:04
🔗
|
|
underscor has quit IRC (No Ping reply in 180 seconds.) |
|
01:04
🔗
|
|
atomicthu has quit IRC (No Ping reply in 180 seconds.) |
|
01:05
🔗
|
|
underscor has joined #archiveteam-bs |
|
01:09
🔗
|
|
nyany has quit IRC (Remote host closed the connection) |
|
01:10
🔗
|
|
nyany has joined #archiveteam-bs |
|
01:21
🔗
|
|
atomicthu has joined #archiveteam-bs |
|
01:55
🔗
|
|
Famicoman has quit IRC (Remote host closed the connection) |
|
01:57
🔗
|
|
Famicoman has joined #archiveteam-bs |
|
01:59
🔗
|
|
nyany has quit IRC (Remote host closed the connection) |
|
01:59
🔗
|
|
nyany has joined #archiveteam-bs |
|
02:10
🔗
|
|
qw3rty114 has quit IRC (Ping timeout: 600 seconds) |
|
02:31
🔗
|
|
zyphlar_ has joined #archiveteam-bs |
|
02:33
🔗
|
|
qw3rty114 has joined #archiveteam-bs |
|
02:41
🔗
|
|
godane has quit IRC (Ping timeout: 252 seconds) |
|
02:55
🔗
|
|
godane has joined #archiveteam-bs |
|
03:05
🔗
|
arkiver |
bmcginty: afaik no (?), but we will get on it |
|
03:32
🔗
|
|
wp494_ has joined #archiveteam-bs |
|
03:37
🔗
|
|
wp494 has quit IRC (Read error: Operation timed out) |
|
03:48
🔗
|
|
qw3rty115 has joined #archiveteam-bs |
|
03:52
🔗
|
|
qw3rty114 has quit IRC (Read error: Operation timed out) |
|
04:08
🔗
|
|
nyaomi has quit IRC (Ping timeout: 268 seconds) |
|
04:10
🔗
|
|
odemg has quit IRC (Read error: Operation timed out) |
|
04:15
🔗
|
|
odemg has joined #archiveteam-bs |
|
04:41
🔗
|
|
zyphlar_ has quit IRC (Quit: Connection closed for inactivity) |
|
04:42
🔗
|
|
wp494_ is now known as wp494 |
|
04:45
🔗
|
|
nyaomi has joined #archiveteam-bs |
|
04:54
🔗
|
|
nyaomi has quit IRC (Ping timeout: 244 seconds) |
|
05:08
🔗
|
|
nyaomi has joined #archiveteam-bs |
|
05:32
🔗
|
|
Lord_Nigh has quit IRC (Read error: Operation timed out) |
|
05:32
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
05:35
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
05:36
🔗
|
|
Lord_Nigh has joined #archiveteam-bs |
|
05:38
🔗
|
|
Lord_Nigh has quit IRC (Read error: Operation timed out) |
|
05:41
🔗
|
|
nyaomi has quit IRC (Read error: Connection reset by peer) |
|
05:44
🔗
|
|
nyaomi has joined #archiveteam-bs |
|
05:46
🔗
|
|
Lord_Nigh has joined #archiveteam-bs |
|
06:02
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
06:04
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
06:47
🔗
|
|
Mayonaise has quit IRC (Read error: Operation timed out) |
|
06:58
🔗
|
|
Mayonaise has joined #archiveteam-bs |
|
08:06
🔗
|
godane |
SketchCow: tons of cds isos in community texts: https://archive.org/details/@claunia |
|
09:03
🔗
|
|
MrRadar_ has joined #archiveteam-bs |
|
09:04
🔗
|
|
MrRadar has quit IRC (Read error: Operation timed out) |
|
10:01
🔗
|
|
SilSte has quit IRC (Remote host closed the connection) |
|
10:02
🔗
|
|
SilSte has joined #archiveteam-bs |
|
10:07
🔗
|
|
RichardG has quit IRC (Ping timeout: 268 seconds) |
|
10:11
🔗
|
|
dashcloud has quit IRC (Ping timeout: 260 seconds) |
|
10:13
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
11:32
🔗
|
|
BlueMax has quit IRC (Read error: Connection reset by peer) |
|
12:00
🔗
|
eientei95 |
Sanqui: Want me to archive their patreon? |
|
12:03
🔗
|
|
godane has quit IRC (Read error: Operation timed out) |
|
12:12
🔗
|
Sanqui |
eientei95: With log-in access? Please do |
|
12:19
🔗
|
eientei95 |
Sanqui: Would a JSON export of the posts be good enough? COntains links to the images and attachments |
|
12:20
🔗
|
Sanqui |
Can those be accessed without a 403? |
|
12:22
🔗
|
eientei95 |
I believe so, let me check |
|
12:23
🔗
|
eientei95 |
Hm. nope |
|
12:24
🔗
|
Sanqui |
Then it's up to you to archive the whole thing |
|
13:05
🔗
|
|
fie has quit IRC (Leaving) |
|
13:09
🔗
|
|
Lord_Nigh has quit IRC (Read error: Operation timed out) |
|
13:11
🔗
|
|
Lord_Nigh has joined #archiveteam-bs |
|
13:37
🔗
|
|
RichardG has joined #archiveteam-bs |
|
13:40
🔗
|
|
Lord_Nigh has quit IRC (Ping timeout: 268 seconds) |
|
13:47
🔗
|
|
Lord_Nigh has joined #archiveteam-bs |
|
14:43
🔗
|
betamax |
What's the best program to make a image of a CD containing Mac OS 9-era files? |
|
14:43
🔗
|
betamax |
Ideally cross platform - or suggestions for both Windows and Mac |
|
14:44
🔗
|
betamax |
(I'm in contact with someone who has a CD of Mac files, but I don't know their setup). |
|
15:37
🔗
|
|
Darkstar has quit IRC (Ping timeout: 506 seconds) |
|
15:42
🔗
|
|
Darkstar has joined #archiveteam-bs |
|
15:43
🔗
|
|
Boppen has quit IRC (Quit: Nettalk6 - www.ntalk.de) |
|
15:59
🔗
|
|
lindalap_ has joined #archiveteam-bs |
|
15:59
🔗
|
|
lindalap has quit IRC (Read error: Connection reset by peer) |
|
16:00
🔗
|
|
lindalap_ is now known as lindalap |
|
16:10
🔗
|
|
schbirid has joined #archiveteam-bs |
|
16:13
🔗
|
|
C4K3_ has quit IRC (leaving) |
|
16:14
🔗
|
|
C4K3 has joined #archiveteam-bs |
|
16:22
🔗
|
|
SilSte has quit IRC (Ping timeout: 633 seconds) |
|
16:22
🔗
|
|
Fusl has quit IRC (K-Lined) |
|
16:23
🔗
|
|
Fusl has joined #archiveteam-bs |
|
16:32
🔗
|
JAA |
So, trying to grab the GIGA forums. I can't get any faster than ~150 threads per minute from one machine for some reason, even at very high concurrency (it's not CPU-limited, the process is only using 50-60 % of one core). And that's not fast enough; it'd take 6 full days to scan all ~1.3 million thread IDs, and that would only cover the first page of each thread. I'll try to split it up into multiple |
|
16:33
🔗
|
astrid |
betamax: bin/cue, and then iso |
|
16:33
🔗
|
JAA |
parallel processes to see if that is any faster. |
|
16:33
🔗
|
JAA |
(If we end up doing a warrior project, this is obviously obsolete.) |
|
16:34
🔗
|
schbirid |
why bother scanning? just -m -reject and go |
|
16:36
🔗
|
JAA |
They don't list all threads in the indices, it seems. |
|
16:36
🔗
|
JAA |
For example, the frontpage says there are 1684 threads in "GIGA.DE allgemein", but when you go into those forums, it only shows three threads. |
|
16:36
🔗
|
schbirid |
&daysprune=-1 |
|
16:37
🔗
|
schbirid |
Anzeige-Eigenschaften -> Alter -> von Anfang an |
|
16:37
🔗
|
JAA |
Yeah, right. |
|
16:37
🔗
|
JAA |
Hmm |
|
16:38
🔗
|
|
atrocity has quit IRC (Read error: Operation timed out) |
|
16:40
🔗
|
JAA |
Setting up the ignores gets more complex though with a real recursive grab. |
|
16:40
🔗
|
JAA |
With the scanning strategy, I can just let it grab only showthread.php and the actual thread links and don't need to worry about infinite tunnels etc. |
|
16:42
🔗
|
JAA |
I might grab the indices like that though for browsability. |
|
16:59
🔗
|
betamax |
astrid: I've found that the person has a PC, what's the best (free) Windows software for making bin / cue files |
|
17:00
🔗
|
betamax |
(I tried ImgBurn, but that doesn't recognise the disc) |
|
17:00
🔗
|
astrid |
i haven't worked with ripping discs much so i can't give you any useful pointers, sorry |
|
17:01
🔗
|
astrid |
i might suggest http://cue.tools/wiki/CUETools |
|
17:19
🔗
|
|
atrocity has joined #archiveteam-bs |
|
17:31
🔗
|
schbirid |
betamax: https://github.com/saramibreak/DiscImageCreator |
|
17:43
🔗
|
|
bithippo has joined #archiveteam-bs |
|
17:57
🔗
|
JAA |
As expected, I get much higher throughput with parallel processes (at the cost of retrieving some images, stylesheets etc. several times). Six processes give me ~750 threads per minute. Let's see how long it takes until GIGA bans me. |
|
18:03
🔗
|
schbirid |
nice |
|
18:07
🔗
|
JAA |
Oh, just realised that I'll probably grab some threads multiple times. Hmm. |
|
18:08
🔗
|
JAA |
In the worst case, I might actually be grabbing *everything* multiple times. |
|
18:12
🔗
|
|
Darkstar has quit IRC (Ping timeout: 480 seconds) |
|
18:20
🔗
|
|
DFJustin has quit IRC (Ping timeout: 260 seconds) |
|
18:23
🔗
|
|
Darkstar has joined #archiveteam-bs |
|
18:37
🔗
|
|
DFJustin has joined #archiveteam-bs |
|
19:24
🔗
|
|
bithippo has quit IRC (Textual IRC Client: www.textualapp.com) |
|
19:46
🔗
|
|
jschwart has joined #archiveteam-bs |
|
19:52
🔗
|
|
Pixi has quit IRC (Ping timeout: 255 seconds) |
|
20:02
🔗
|
|
Pixi has joined #archiveteam-bs |
|
20:13
🔗
|
|
SilSte has joined #archiveteam-bs |
|
20:24
🔗
|
JAA |
Ew, the wiki doesn't support SVG. :-( |
|
20:31
🔗
|
|
schbirid has quit IRC (Quit: Leaving) |
|
20:41
🔗
|
JAA |
Still doing almost exactly 750 threads per minute (749.3 over the past half hour). :-) |
|
20:42
🔗
|
|
jschwart has quit IRC (Quit: Konversation terminated!) |
|
20:45
🔗
|
JAA |
I'm quite surprised that they haven't banned me yet, to be honest. 120 concurrent connections from a single IP... |
|
20:55
🔗
|
arkiver |
JAA: did you find any limits? |
|
20:57
🔗
|
JAA |
arkiver: Nada. Only my server. |
|
20:58
🔗
|
|
dashcloud has quit IRC (Ping timeout: 252 seconds) |
|
21:02
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
21:22
🔗
|
arkiver |
JAA: awesome, sounds good |
|
21:47
🔗
|
JAA |
Found a broken thread though: http://forum.giga.de/showthread.php?t=104517 |
|
21:47
🔗
|
JAA |
The only error so far. No timeouts, connection closures, whatever. |
|
21:50
🔗
|
|
BlueMax has joined #archiveteam-bs |
|
21:59
🔗
|
|
Mateon1 has quit IRC (Ping timeout: 268 seconds) |
|
21:59
🔗
|
|
Mateon1 has joined #archiveteam-bs |
|
22:11
🔗
|
|
godane has joined #archiveteam-bs |
|
22:31
🔗
|
|
Atom has quit IRC (Read error: Connection reset by peer) |
|
22:39
🔗
|
|
m007a83_ has joined #archiveteam-bs |
|
22:41
🔗
|
|
m007a83 has quit IRC (Ping timeout: 252 seconds) |
|
22:53
🔗
|
|
Mayonaise has quit IRC (Read error: Operation timed out) |
|
22:58
🔗
|
|
Mayonaise has joined #archiveteam-bs |
|
23:02
🔗
|
|
RichardG has quit IRC (Read error: Connection reset by peer) |
|
23:04
🔗
|
|
RichardG has joined #archiveteam-bs |
|
23:29
🔗
|
godane |
SketchCow: any update with Mank? |
|
23:30
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
23:30
🔗
|
|
dashcloud has joined #archiveteam-bs |
|
23:52
🔗
|
|
Atom has joined #archiveteam-bs |