| Time |
Nickname |
Message |
|
00:29
🔗
|
|
JesseW has joined #archiveteam |
|
00:34
🔗
|
Zandro |
I notice that the "PEOPLE ALSO FOUND" section below the detail body does not exclude the currently viewed collection |
|
00:35
🔗
|
Zandro |
it puts the fear of duping in the uploader |
|
00:36
🔗
|
Zandro |
oh, sorry. this is not IA |
|
00:36
🔗
|
JesseW |
Zandro: feel free to discuss this in #internetarchive , if you'd like |
|
00:43
🔗
|
|
jspiros has quit IRC (Ping timeout: 186 seconds) |
|
00:56
🔗
|
|
jspiros has joined #archiveteam |
|
00:59
🔗
|
|
xk_id has quit IRC (Remote host closed the connection) |
|
00:59
🔗
|
|
wyatt8740 has quit IRC (Remote host closed the connection) |
|
01:03
🔗
|
|
wyatt8740 has joined #archiveteam |
|
01:09
🔗
|
|
aaaaaaaa_ has joined #archiveteam |
|
01:09
🔗
|
|
aaaaaaaaa has quit IRC (Read error: Connection reset by peer) |
|
01:17
🔗
|
|
aaaaaaaa_ is now known as aaaaaaaaa |
|
01:17
🔗
|
|
xk_id has joined #archiveteam |
|
01:21
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
01:30
🔗
|
|
JesseW has quit IRC (Leaving.) |
|
01:30
🔗
|
|
JesseW has joined #archiveteam |
|
01:33
🔗
|
|
oli has quit IRC (Read error: Operation timed out) |
|
01:39
🔗
|
|
oli has joined #archiveteam |
|
01:46
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
01:47
🔗
|
|
c_b2 has joined #archiveteam |
|
01:47
🔗
|
|
JesseW has joined #archiveteam |
|
01:51
🔗
|
|
c_b has quit IRC (Read error: Operation timed out) |
|
01:53
🔗
|
|
c_b2 is now known as c_b |
|
02:11
🔗
|
|
c_b has quit IRC (c_b) |
|
02:12
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
02:18
🔗
|
Zandro |
looking into my upload, it will not complete the derive step due to its attempt to download from the dead swarm. |
|
02:22
🔗
|
|
khaoohs has joined #archiveteam |
|
02:24
🔗
|
Zandro |
transmission 8h41s idle... hope it times out before the 3 day limit for the next derive task |
|
02:25
🔗
|
|
khaoohs__ has quit IRC (Ping timeout: 240 seconds) |
|
03:00
🔗
|
|
RichardG_ has joined #archiveteam |
|
03:00
🔗
|
|
RichardG has quit IRC (Read error: Connection reset by peer) |
|
03:01
🔗
|
|
khaoohs has quit IRC (Read error: Operation timed out) |
|
03:01
🔗
|
|
slyphic_ has quit IRC (Read error: Operation timed out) |
|
03:01
🔗
|
|
atlogbot has quit IRC (Read error: Operation timed out) |
|
03:02
🔗
|
|
no2pencil has quit IRC (Read error: Operation timed out) |
|
03:03
🔗
|
|
no2pencil has joined #archiveteam |
|
03:03
🔗
|
|
swebb has quit IRC (Ping timeout: 369 seconds) |
|
03:04
🔗
|
|
vOYtEC has quit IRC (Ping timeout: 369 seconds) |
|
03:04
🔗
|
|
ohhdemgir has quit IRC (Read error: Operation timed out) |
|
03:05
🔗
|
|
vOYtEC has joined #archiveteam |
|
03:05
🔗
|
|
dserodio has quit IRC (Read error: Operation timed out) |
|
03:05
🔗
|
|
chazchaz has quit IRC (Read error: Operation timed out) |
|
03:07
🔗
|
|
mistym has quit IRC (Ping timeout: 369 seconds) |
|
03:07
🔗
|
|
slyphic has joined #archiveteam |
|
03:07
🔗
|
|
atlogbot has joined #archiveteam |
|
03:10
🔗
|
|
Laverne has quit IRC (Ping timeout: 369 seconds) |
|
03:12
🔗
|
|
khaoohs has joined #archiveteam |
|
03:13
🔗
|
|
robink has quit IRC (Ping timeout: 492 seconds) |
|
03:13
🔗
|
|
slyphic_ has joined #archiveteam |
|
03:13
🔗
|
|
atlogbot has quit IRC (Ping timeout: 369 seconds) |
|
03:14
🔗
|
|
Laverne has joined #archiveteam |
|
03:14
🔗
|
|
dserodio has joined #archiveteam |
|
03:14
🔗
|
|
robink has joined #archiveteam |
|
03:15
🔗
|
|
atlogbot has joined #archiveteam |
|
03:16
🔗
|
|
swebb has joined #archiveteam |
|
03:17
🔗
|
|
mistym has joined #archiveteam |
|
03:18
🔗
|
|
chazchaz has joined #archiveteam |
|
03:27
🔗
|
|
slyphic has quit IRC (Read error: Operation timed out) |
|
03:32
🔗
|
|
wp494_ has joined #archiveteam |
|
03:38
🔗
|
|
wp494 has quit IRC (Ping timeout: 483 seconds) |
|
03:39
🔗
|
|
slyphic_ has quit IRC (Read error: Operation timed out) |
|
03:39
🔗
|
|
chazchaz has quit IRC (Read error: Operation timed out) |
|
03:39
🔗
|
|
Laverne has quit IRC (Read error: Operation timed out) |
|
03:40
🔗
|
|
mistym has quit IRC (Ping timeout: 369 seconds) |
|
03:40
🔗
|
|
slyphic has joined #archiveteam |
|
03:40
🔗
|
|
chazchaz has joined #archiveteam |
|
03:41
🔗
|
|
Laverne has joined #archiveteam |
|
03:42
🔗
|
|
mistym has joined #archiveteam |
|
04:07
🔗
|
|
wutno has quit IRC (Read error: Operation timed out) |
|
04:10
🔗
|
|
Wyatts has quit IRC (Remote host closed the connection) |
|
04:14
🔗
|
|
Wyatts has joined #archiveteam |
|
04:27
🔗
|
|
aaaaaaaaa has quit IRC (Leaving) |
|
04:42
🔗
|
|
JesseW has joined #archiveteam |
|
04:48
🔗
|
|
khaoohs_ has joined #archiveteam |
|
04:54
🔗
|
|
khaoohs has quit IRC (Ping timeout: 483 seconds) |
|
05:12
🔗
|
|
oli has quit IRC (Read error: Operation timed out) |
|
05:15
🔗
|
|
oli has joined #archiveteam |
|
05:18
🔗
|
|
scyther has joined #archiveteam |
|
05:48
🔗
|
|
wyatt8740 has quit IRC (Read error: Operation timed out) |
|
05:56
🔗
|
|
scyther has quit IRC (Read error: Connection reset by peer) |
|
06:02
🔗
|
|
wyatt8740 has joined #archiveteam |
|
06:03
🔗
|
|
primus104 has joined #archiveteam |
|
06:23
🔗
|
|
habi has joined #archiveteam |
|
06:23
🔗
|
|
habi has left |
|
06:39
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
07:10
🔗
|
|
xk_id has quit IRC (Read error: Operation timed out) |
|
07:11
🔗
|
|
PurpleSym has joined #archiveteam |
|
07:13
🔗
|
|
trs80 has quit IRC (Ping timeout: 186 seconds) |
|
07:14
🔗
|
|
trs80 has joined #archiveteam |
|
07:18
🔗
|
|
schbirid has joined #archiveteam |
|
07:30
🔗
|
|
trs80 has quit IRC (Ping timeout: 186 seconds) |
|
07:30
🔗
|
|
trs80 has joined #archiveteam |
|
07:41
🔗
|
|
atomotic has joined #archiveteam |
|
07:52
🔗
|
|
jspiros has quit IRC (Ping timeout: 186 seconds) |
|
07:59
🔗
|
|
wp494_ is now known as wp494 |
|
08:31
🔗
|
|
jspiros has joined #archiveteam |
|
08:32
🔗
|
|
xk_id has joined #archiveteam |
|
08:33
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
08:41
🔗
|
|
RedType has quit IRC (Ping timeout: 252 seconds) |
|
08:43
🔗
|
|
Elegance_ has quit IRC (Read error: Operation timed out) |
|
08:46
🔗
|
|
zenguy_pc has quit IRC (Read error: Connection reset by peer) |
|
08:48
🔗
|
|
brayden has joined #archiveteam |
|
08:49
🔗
|
|
Elegance has joined #archiveteam |
|
08:58
🔗
|
|
MMovie has joined #archiveteam |
|
09:01
🔗
|
|
MMovie2 has quit IRC (Ping timeout: 306 seconds) |
|
09:02
🔗
|
|
zenguy_pc has joined #archiveteam |
|
09:05
🔗
|
|
brayden has quit IRC (Quit: Leaving) |
|
09:20
🔗
|
|
brayden has joined #archiveteam |
|
09:35
🔗
|
|
RedType has joined #archiveteam |
|
09:43
🔗
|
|
wp494 has quit IRC (Read error: Connection reset by peer) |
|
09:56
🔗
|
|
wp494 has joined #archiveteam |
|
09:57
🔗
|
|
RedType has quit IRC (Ping timeout: 483 seconds) |
|
10:15
🔗
|
|
RedType has joined #archiveteam |
|
10:33
🔗
|
|
anomie has quit IRC (Read error: Connection reset by peer) |
|
10:39
🔗
|
|
Wyatts has quit IRC (Remote host closed the connection) |
|
10:41
🔗
|
|
anomie has joined #archiveteam |
|
10:43
🔗
|
|
Wyatts has joined #archiveteam |
|
10:55
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
11:02
🔗
|
|
primus104 has joined #archiveteam |
|
11:16
🔗
|
|
dashcloud has quit IRC (Ping timeout: 483 seconds) |
|
11:18
🔗
|
|
dashcloud has joined #archiveteam |
|
11:25
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
11:42
🔗
|
|
Boppen has quit IRC (Read error: Connection reset by peer) |
|
11:43
🔗
|
|
Boppen has joined #archiveteam |
|
11:44
🔗
|
|
arkiver2 has joined #archiveteam |
|
11:48
🔗
|
|
atomotic has joined #archiveteam |
|
12:18
🔗
|
|
primus104 has joined #archiveteam |
|
12:25
🔗
|
|
vitzli has joined #archiveteam |
|
12:27
🔗
|
|
BlueMaxim has quit IRC (Quit: Leaving) |
|
12:31
🔗
|
|
scyther has joined #archiveteam |
|
12:35
🔗
|
|
arkiver2 has quit IRC (Ping timeout: 252 seconds) |
|
13:11
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
13:18
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
13:18
🔗
|
|
arkiver2 has joined #archiveteam |
|
13:27
🔗
|
|
arkiver2 has quit IRC (Ping timeout: 252 seconds) |
|
13:30
🔗
|
|
arkiver2 has joined #archiveteam |
|
13:37
🔗
|
|
PurpleSym has quit IRC (Remote host closed the connection) |
|
13:45
🔗
|
|
RichardG_ is now known as RichardG |
|
13:46
🔗
|
|
arkiver2 has quit IRC (Ping timeout: 252 seconds) |
|
13:48
🔗
|
|
habi has joined #archiveteam |
|
13:51
🔗
|
|
K4k has joined #archiveteam |
|
13:57
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
14:03
🔗
|
|
PurpleSym has joined #archiveteam |
|
14:15
🔗
|
|
habi has left |
|
14:50
🔗
|
|
scyther has quit IRC (Read error: Connection reset by peer) |
|
15:09
🔗
|
|
vitzli has joined #archiveteam |
|
15:11
🔗
|
|
Stilett0 has joined #archiveteam |
|
15:13
🔗
|
|
Stiletto has quit IRC (Ping timeout: 306 seconds) |
|
15:13
🔗
|
arkiver |
cfarence: only http://market.envato.com/ is closing or other parts too? |
|
15:14
🔗
|
arkiver |
cfarence: and all these 8 sites (http://themeforest.net/, etc) are closing too and are part of the envato market? |
|
15:26
🔗
|
joepie91 |
arkiver: just the Flash site |
|
15:26
🔗
|
joepie91 |
"activeden" |
|
15:27
🔗
|
arkiver |
so only http://activeden.net/ |
|
15:34
🔗
|
|
Alice_ has joined #archiveteam |
|
15:52
🔗
|
|
Alice_ has quit IRC (Ping timeout: 240 seconds) |
|
16:08
🔗
|
|
JesseW has joined #archiveteam |
|
16:10
🔗
|
limebyte |
wow |
|
16:10
🔗
|
limebyte |
my archiveteam stopped pushing |
|
16:26
🔗
|
|
wutno has joined #archiveteam |
|
16:34
🔗
|
|
JesseW has quit IRC (Read error: Operation timed out) |
|
16:42
🔗
|
|
primus104 has joined #archiveteam |
|
17:11
🔗
|
|
SimpBrain has joined #archiveteam |
|
17:49
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
17:52
🔗
|
|
nertzy has joined #archiveteam |
|
18:00
🔗
|
|
mksplg has joined #archiveteam |
|
18:08
🔗
|
|
nertzy has quit IRC (Quit: This computer has gone to sleep) |
|
18:08
🔗
|
SketchCow |
Minor thing: We're freaking IA a little. |
|
18:08
🔗
|
SketchCow |
Just a little. |
|
18:21
🔗
|
|
winr4r has joined #archiveteam |
|
18:22
🔗
|
joepie91 |
SketchCow: why's that? |
|
18:26
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
18:28
🔗
|
SketchCow |
Doubled intake |
|
18:29
🔗
|
SketchCow |
I think we're one of the factors, not the factor. |
|
18:34
🔗
|
SketchCow |
--------------------------------------------------- |
|
18:34
🔗
|
SketchCow |
GRAB THINGIVERSE IMMEDIATELY |
|
18:34
🔗
|
SketchCow |
--------------------------------------------------- |
|
18:34
🔗
|
SketchCow |
I M M E D I A T E L Y |
|
18:34
🔗
|
SketchCow |
I know godane has done it, I know others can. Move. Top priority. |
|
18:37
🔗
|
|
habi has joined #archiveteam |
|
18:38
🔗
|
arkiver |
OK! |
|
18:39
🔗
|
arkiver |
SketchCow: what's happenning over at thingiverse? |
|
18:39
🔗
|
SketchCow |
I don't entirely know |
|
18:39
🔗
|
arkiver |
https://www.thingiverse.com/thing:192937 |
|
18:39
🔗
|
SketchCow |
I think they're going to kill it |
|
18:40
🔗
|
joepie91 |
arkiver: need any help with grabbing script work? |
|
18:40
🔗
|
arkiver |
it's just empty |
|
18:40
🔗
|
Zandro |
got a backend fetch fail on first attempt to load dashboard, not sure if related |
|
18:40
🔗
|
|
RichardG has joined #archiveteam |
|
18:40
🔗
|
arkiver |
joepie91: I think I'll be fine, I'll ping you if I need any help |
|
18:40
🔗
|
joepie91 |
uhh |
|
18:40
🔗
|
joepie91 |
HTTP 500 error |
|
18:40
🔗
|
joepie91 |
https://www.thingiverse.com/thing:1008750 |
|
18:40
🔗
|
arkiver |
Will first find out if we can actually get the stuff still |
|
18:41
🔗
|
arkiver |
Looking at the site I should be able to have the scripts ready very fast if there's still anything to download |
|
18:41
🔗
|
joepie91 |
arkiver: I see 500 everywhere... |
|
18:41
🔗
|
arkiver |
yeah |
|
18:41
🔗
|
SketchCow |
arkiver: We have a thingiverse downloader. |
|
18:41
🔗
|
arkiver |
maybe that's just the html pages |
|
18:41
🔗
|
schbirid |
it worked seconds ago |
|
18:41
🔗
|
schbirid |
err minutes |
|
18:42
🔗
|
|
habi1 has joined #archiveteam |
|
18:42
🔗
|
arkiver |
SketchCow: where is it? |
|
18:42
🔗
|
joepie91 |
arkiver: it's slow and eventually 500s. I'm guessing a backend service is unavailable |
|
18:42
🔗
|
|
habi has quit IRC (Read error: Connection reset by peer) |
|
18:43
🔗
|
SketchCow |
Luckily I'm buddies with Bre Pettis. Hitting him up for info. |
|
18:43
🔗
|
SketchCow |
(Bre is no longer associated with the company) |
|
18:44
🔗
|
arkiver |
stuff is still available https://thingiverse-production-new.s3.amazonaws.com/zipfiles/5f/47/93/7a/4f/SCH_13mm.zip |
|
18:49
🔗
|
|
beardicus has quit IRC (Quit: bye now) |
|
18:49
🔗
|
arkiver |
SketchCow: I know how to get the stuff |
|
18:49
🔗
|
arkiver |
well not the html, the files |
|
18:50
🔗
|
arkiver |
getting scripts ready now |
|
18:52
🔗
|
|
beardicus has joined #archiveteam |
|
18:54
🔗
|
arkiver |
tested a bit more |
|
18:55
🔗
|
arkiver |
we can't get the zip files |
|
18:55
🔗
|
arkiver |
we can get the individual models |
|
18:55
🔗
|
joepie91 |
arkiver: why's that? |
|
18:56
🔗
|
arkiver |
well, the zip are redirected to through a link that is not available (500) |
|
18:56
🔗
|
arkiver |
the individual models are still downloadable through an other link |
|
18:56
🔗
|
arkiver |
that's still workin |
|
18:56
🔗
|
arkiver |
working* |
|
18:56
🔗
|
joepie91 |
arkiver: but you just linked a zip? |
|
18:57
🔗
|
arkiver |
yeah, I found it through godane's grabs |
|
18:57
🔗
|
arkiver |
just to check if their files weren't deleted yet |
|
18:57
🔗
|
joepie91 |
ah |
|
18:57
🔗
|
joepie91 |
arkiver: how are you getting them? |
|
18:57
🔗
|
arkiver |
http://www.thingiverse.com/download:183243 |
|
18:57
🔗
|
arkiver |
for now we're getting the inidividual models, the other things will be looked at later |
|
18:57
🔗
|
arkiver |
most important info first |
|
18:58
🔗
|
schbirid |
might be worth checking which files are already at IA |
|
18:58
🔗
|
joepie91 |
arkiver: and what did you use to try and get at the ZIPs? |
|
18:58
🔗
|
joepie91 |
(just wanting to explore a bit) |
|
18:58
🔗
|
arkiver |
http://www.thingiverse.com/thing:79692/zip |
|
18:58
🔗
|
arkiver |
that zip should contain http://www.thingiverse.com/download:183243 and http://www.thingiverse.com/download:183244 |
|
18:58
🔗
|
arkiver |
from https://webcache.googleusercontent.com/search?q=cache:www.thingiverse.com/thing:79692 |
|
18:59
🔗
|
joepie91 |
I see |
|
18:59
🔗
|
joepie91 |
thanks, going to have a poke around |
|
18:59
🔗
|
arkiver |
chfoo: can you please add thingiverse to the project.json and add a FOS rsync? |
|
19:00
🔗
|
arkiver |
joepie91: thanks! |
|
19:04
🔗
|
joepie91 |
uh oh |
|
19:04
🔗
|
joepie91 |
arkiver: I think they just went down? |
|
19:04
🔗
|
arkiver |
what? |
|
19:05
🔗
|
arkiver |
they work for me |
|
19:05
🔗
|
joepie91 |
getting cloudflare errors |
|
19:05
🔗
|
joepie91 |
even on the download: link |
|
19:05
🔗
|
arkiver |
scripts almost ready |
|
19:05
🔗
|
joepie91 |
arkiver: so account for HTTP 522 errors |
|
19:05
🔗
|
arkiver |
yes |
|
19:05
🔗
|
arkiver |
I'l continue the grab on 522 error |
|
19:05
🔗
|
arkiver |
errors |
|
19:07
🔗
|
arkiver |
actually |
|
19:07
🔗
|
arkiver |
what shall I do continue or abort on 522 |
|
19:07
🔗
|
arkiver |
I think I'll do one file per item |
|
19:07
🔗
|
joepie91 |
arkiver: well, they are supposedly temporary. pause and retry? |
|
19:07
🔗
|
arkiver |
so I can jsut do abort |
|
19:08
🔗
|
joepie91 |
yeah |
|
19:08
🔗
|
arkiver |
(was first doing 10) |
|
19:08
🔗
|
joepie91 |
it's back for me |
|
19:10
🔗
|
arkiver |
ok |
|
19:10
🔗
|
joepie91 |
arkiver: and down again, now a HTTP 521 |
|
19:10
🔗
|
joepie91 |
Origin Down |
|
19:11
🔗
|
|
habi1 has left |
|
19:12
🔗
|
arkiver |
ok, scripts working |
|
19:12
🔗
|
joepie91 |
arkiver: want me to spin up a script? |
|
19:12
🔗
|
arkiver |
wait |
|
19:14
🔗
|
|
aaaaaaaaa has joined #archiveteam |
|
19:14
🔗
|
arkiver |
I need an rsync target |
|
19:14
🔗
|
joepie91 |
arkiver: how much data are we talking? |
|
19:14
🔗
|
joepie91 |
ballpark |
|
19:15
🔗
|
arkiver |
No idea |
|
19:15
🔗
|
arkiver |
I don't think more then 1T |
|
19:15
🔗
|
joepie91 |
because I just got a new box with 4TB disk and 10TB traffic @ 1gbps |
|
19:15
🔗
|
joepie91 |
but it's single-disk |
|
19:15
🔗
|
arkiver |
ok, please PM me the rsync target |
|
19:15
🔗
|
joepie91 |
and it's a new disk |
|
19:15
🔗
|
arkiver |
:) |
|
19:15
🔗
|
joepie91 |
so it's not _guaranteed_ that it won't fail |
|
19:15
🔗
|
arkiver |
hmm |
|
19:15
🔗
|
joepie91 |
(disks usually fail either very early or very late) |
|
19:16
🔗
|
joepie91 |
arkiver: do we have any alternative RAID-backed rsync targets? |
|
19:16
🔗
|
joepie91 |
or is it either this one or none? |
|
19:16
🔗
|
joepie91 |
(cc Kenshin ) |
|
19:16
🔗
|
arkiver |
------------------------------ |
|
19:16
🔗
|
arkiver |
IMPORTANT: |
|
19:16
🔗
|
arkiver |
We need a reliable rsync target ASAP |
|
19:16
🔗
|
arkiver |
------------------------------ |
|
19:16
🔗
|
arkiver |
hmm |
|
19:16
🔗
|
arkiver |
I'll use the gamefront rsync |
|
19:16
🔗
|
arkiver |
SketchCow ^ |
|
19:17
🔗
|
arkiver |
and joepie91 ^ |
|
19:17
🔗
|
joepie91 |
arkiver: I'll set up an rsync target on my box anyway - if you end up not having anything else, you at least have a fallback that hopefully won't fail |
|
19:18
🔗
|
arkiver |
ok! |
|
19:18
🔗
|
arkiver |
preparing items now |
|
19:21
🔗
|
|
scyther has joined #archiveteam |
|
19:22
🔗
|
arkiver |
items being added |
|
19:22
🔗
|
arkiver |
I'll pause blingee |
|
19:23
🔗
|
arkiver |
we have started! |
|
19:23
🔗
|
arkiver |
it's not yet in the warrior, but it can be run manually |
|
19:23
🔗
|
joepie91 |
arkiver: link to repo? |
|
19:23
🔗
|
arkiver |
http://tracker.archiveteam.org/blingee/ |
|
19:23
🔗
|
arkiver |
https://github.com/ArchiveTeam/thingiverse-grab |
|
19:23
🔗
|
joepie91 |
wrong tracker |
|
19:23
🔗
|
joepie91 |
:P |
|
19:23
🔗
|
arkiver |
oops |
|
19:23
🔗
|
arkiver |
http://tracker.archiveteam.org/thingiverse/ |
|
19:23
🔗
|
joepie91 |
arkiver: you're missing the README btw |
|
19:23
🔗
|
arkiver |
yeah |
|
19:24
🔗
|
arkiver |
will be added now |
|
19:24
🔗
|
arkiver |
wasn't vital |
|
19:25
🔗
|
limebyte |
wow |
|
19:25
🔗
|
limebyte |
need grabber |
|
19:25
🔗
|
limebyte |
I SETUP |
|
19:26
🔗
|
limebyte |
not linked to wiki |
|
19:26
🔗
|
limebyte |
shame |
|
19:26
🔗
|
arkiver |
SketchCow: grab of thiingiverse started |
|
19:26
🔗
|
joepie91 |
arkiver: let me know when you've added the README and checked that the list of deps is correct etc |
|
19:26
🔗
|
arkiver |
Only files currently |
|
19:27
🔗
|
arkiver |
joepie91: I'll just use the template |
|
19:27
🔗
|
limebyte |
what does actually happen |
|
19:27
🔗
|
limebyte |
if i run my googlecode grab |
|
19:27
🔗
|
limebyte |
and it isnt started? |
|
19:27
🔗
|
limebyte |
does it start when itstarts? |
|
19:29
🔗
|
joepie91 |
limebyte: most likely |
|
19:29
🔗
|
limebyte |
so the warrior idles |
|
19:29
🔗
|
limebyte |
unti you press play or what? |
|
19:30
🔗
|
joepie91 |
limebyte: not sure what you mean, but probably better suited for -bs |
|
19:30
🔗
|
arkiver |
readme added. |
|
19:30
🔗
|
arkiver |
we have 1.6 million files to go |
|
19:30
🔗
|
arkiver |
give it all you have I'd say |
|
19:30
🔗
|
limebyte |
okay |
|
19:33
🔗
|
joepie91 |
arkiver: you typoed the repo name |
|
19:33
🔗
|
joepie91 |
missing -grab |
|
19:33
🔗
|
joepie91 |
arkiver: hold, PR incoming |
|
19:34
🔗
|
limebyte |
ya |
|
19:34
🔗
|
limebyte |
didnt worked |
|
19:34
🔗
|
limebyte |
shame on you |
|
19:34
🔗
|
* |
arkiver is afk for 20 minutes |
|
19:34
🔗
|
joepie91 |
arkiver: https://github.com/ArchiveTeam/thingiverse-grab/pull/1 |
|
19:34
🔗
|
joepie91 |
accept pls |
|
19:34
🔗
|
joepie91 |
:P |
|
19:35
🔗
|
joepie91 |
danke |
|
19:35
🔗
|
bentpins |
429 is the rate limited status I think |
|
19:35
🔗
|
arkiver |
bitte |
|
19:35
🔗
|
arkiver |
hmm, joepie91: maybe we should go with the 10 or 100 files/item |
|
19:36
🔗
|
arkiver |
not sure if the tracker can hold |
|
19:36
🔗
|
limebyte |
hmmm |
|
19:36
🔗
|
arkiver |
currently doing 1200/min |
|
19:37
🔗
|
limebyte |
wow |
|
19:37
🔗
|
joepie91 |
poop, another error |
|
19:37
🔗
|
joepie91 |
arkiver: wait 1 min please |
|
19:37
🔗
|
arkiver |
ok |
|
19:38
🔗
|
bentpins |
Getting a decent amount of 503s |
|
19:38
🔗
|
joepie91 |
arkiver: new PR |
|
19:38
🔗
|
joepie91 |
arkiver: hmm. won't it interfere with the gamefront data? |
|
19:38
🔗
|
arkiver |
gamefront didn't start yet |
|
19:38
🔗
|
joepie91 |
ahh |
|
19:38
🔗
|
joepie91 |
right |
|
19:39
🔗
|
arkiver |
and all thiingiverse warc's have prefix thingiverse- |
|
19:39
🔗
|
joepie91 |
right :P |
|
19:39
🔗
|
joepie91 |
arkiver: filesizes look right to you? |
|
19:39
🔗
|
arkiver |
yes |
|
19:39
🔗
|
joepie91 |
ok |
|
19:40
🔗
|
godane |
i'm awake |
|
19:41
🔗
|
arkiver |
joepie91: most are very small and compress very well, example: https://www.thingiverse.com/download:220747 |
|
19:41
🔗
|
godane |
i will tell you that thingiverse has about 90k things between 2009 to 2013 |
|
19:42
🔗
|
joepie91 |
arkiver: makes sense |
|
19:47
🔗
|
Zandro |
wikipedia: There were 25,000 designs uploaded to Thingiverse as of November 2012[4] and more than 100,000 in June 2013.[5] The 400000th Thing was published on the 19 July 2014[6] |
|
19:48
🔗
|
joepie91 |
exponential growth |
|
19:48
🔗
|
joepie91 |
probably safe to assume a few million by now |
|
19:48
🔗
|
joepie91 |
lol |
|
19:51
🔗
|
aaaaaaaaa |
they may have been losing money on each upload, but made up for it in volume |
|
19:52
🔗
|
Baljem |
'Tracker returned status code 500. The tracker has probably malfunctioned.' on thingiverse-grab - or is it just me? |
|
19:52
🔗
|
Baljem |
ah, off it goes again. sorry! |
|
19:58
🔗
|
joepie91 |
we break the tracker again? |
|
19:58
🔗
|
joepie91 |
lol |
|
19:58
🔗
|
arkiver |
1.6 million |
|
20:00
🔗
|
|
SimpBrain has quit IRC (Quit: Leaving) |
|
20:02
🔗
|
Baljem |
is it --context-value shared:rsync_threads=N to change the number of rsync jobs at once? I seem to be stuck on 1 which looks like a bit of a bottleneck |
|
20:02
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
20:07
🔗
|
aaaaaaaaa |
IIRC that is ignored for some reason. Plus, overloading the rsync target causes other problems. |
|
20:09
🔗
|
|
RichardG has joined #archiveteam |
|
20:10
🔗
|
|
wyatt8760 has joined #archiveteam |
|
20:10
🔗
|
|
wyatt8760 is now known as wyatt|cla |
|
20:10
🔗
|
|
wyatt|cla is now known as wyattclas |
|
20:10
🔗
|
|
wyattclas is now known as wyatt8760 |
|
20:15
🔗
|
|
RedType has quit IRC (Ping timeout: 1730 seconds) |
|
20:20
🔗
|
|
SmileyG has quit IRC (Ping timeout: 240 seconds) |
|
20:25
🔗
|
wyatt8760 |
what is the proper way to make a m3u for uploading to archive.org? |
|
20:26
🔗
|
wyatt8760 |
because i just noticed the m3u I uploaded had hard paths to files in it -_- |
|
20:28
🔗
|
|
Smiley has joined #archiveteam |
|
20:31
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
20:46
🔗
|
|
Morbus has quit IRC (Quit: http://www.disobey.com/) |
|
20:47
🔗
|
schbirid |
just remove the paths in a text editor ;) |
|
20:47
🔗
|
|
schbirid has quit IRC (Quit: Leaving) |
|
20:49
🔗
|
|
RichardG has joined #archiveteam |
|
20:50
🔗
|
|
Morbus has joined #archiveteam |
|
20:53
🔗
|
|
zenguy_pc has quit IRC (Read error: Connection reset by peer) |
|
20:55
🔗
|
|
wyatt8760 has quit IRC (Read error: Operation timed out) |
|
20:57
🔗
|
|
wyatt8760 has joined #archiveteam |
|
21:10
🔗
|
|
zenguy_pc has joined #archiveteam |
|
21:12
🔗
|
mksplg |
https://twitter.com/thingiverse/status/646769701285175297 |
|
21:15
🔗
|
|
scyther has quit IRC (Read error: Connection reset by peer) |
|
21:16
🔗
|
|
PurpleSym has quit IRC (Remote host closed the connection) |
|
21:16
🔗
|
|
ersi has quit IRC (Read error: Operation timed out) |
|
21:17
🔗
|
limebyte |
wow |
|
21:17
🔗
|
limebyte |
such Archiveteam DDOS |
|
21:17
🔗
|
joepie91 |
nah |
|
21:17
🔗
|
limebyte |
i hope i dont get abuse |
|
21:17
🔗
|
joepie91 |
this one is not our fault |
|
21:17
🔗
|
joepie91 |
:) |
|
21:17
🔗
|
limebyte |
sure? :D |
|
21:17
🔗
|
joepie91 |
yeah |
|
21:17
🔗
|
joepie91 |
it was breaking before we even had a script |
|
21:18
🔗
|
limebyte |
is there Archiveteam 2.0 outside? |
|
21:18
🔗
|
limebyte |
maybe |
|
21:18
🔗
|
joepie91 |
lol |
|
21:18
🔗
|
joepie91 |
but, -bs |
|
21:18
🔗
|
joepie91 |
:p |
|
21:19
🔗
|
limebyte |
strict rules |
|
21:19
🔗
|
limebyte |
eh |
|
21:22
🔗
|
|
K4k has quit IRC (Ping timeout: 186 seconds) |
|
21:26
🔗
|
wp494 |
jumping in on thingiverse now |
|
21:26
🔗
|
wp494 |
yay, my 2.5mbit u/l line can finally be put to good use aside from personal backups! |
|
21:32
🔗
|
|
ersi has joined #archiveteam |
|
21:54
🔗
|
|
wyatt8760 has quit IRC (Read error: Operation timed out) |
|
22:01
🔗
|
wyatt8740 |
schbirid: do you mean make them look in the current folder (e.g. './filename.flac')? |
|
22:02
🔗
|
wyatt8740 |
or would it lack the / |
|
22:02
🔗
|
wyatt8740 |
*./ |
|
22:03
🔗
|
aaaaaaaaa |
I believe that if they are in the same directory, you can just do the file name |
|
22:15
🔗
|
matthusby |
Looks like tracker isn't pleased with us |
|
22:18
🔗
|
joepie91 |
matthusby: that has a tendency of happening :P |
|
22:18
🔗
|
joepie91 |
matthusby: kind of amusing - the one part of archiveteam infra that's prone to failure, is the Ruby thing |
|
22:18
🔗
|
joepie91 |
(iirc it was Ruby anyway) |
|
22:20
🔗
|
xmc |
hm, what seems to be wrong with the tracker |
|
22:21
🔗
|
joepie91 |
xmc: been seeing some slowdowns. we're probably just hammering it |
|
22:21
🔗
|
joepie91 |
lots of tiny tasks |
|
22:21
🔗
|
xmc |
hm ok |
|
22:21
🔗
|
joepie91 |
oh |
|
22:21
🔗
|
joepie91 |
it's actually Really Down now |
|
22:21
🔗
|
joepie91 |
@ xmc |
|
22:22
🔗
|
joepie91 |
The Phusion Passenger application server encountered an error while starting your web application. |
|
22:22
🔗
|
xmc |
hahaha |
|
22:23
🔗
|
aaaaaaaaa |
it is being updated |
|
22:23
🔗
|
aaaaaaaaa |
https://github.com/ArchiveTeam/universal-tracker/commit/4770a65e12091cb4b3c1921efe3b823241abf728 |
|
22:24
🔗
|
|
vegbrasil has joined #archiveteam |
|
22:24
🔗
|
arkiver |
aaaaaaaaa: oh heh |
|
22:25
🔗
|
aaaaaaaaa |
in other news, looks like global admins can now update the projects.json |
|
22:25
🔗
|
aaaaaaaaa |
err, in related news |
|
22:26
🔗
|
joepie91 |
oh |
|
22:26
🔗
|
joepie91 |
good news everybody! :P |
|
22:44
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
22:46
🔗
|
|
RichardG has joined #archiveteam |
|
22:53
🔗
|
|
RichardG has quit IRC (Ping timeout: 255 seconds) |
|
22:55
🔗
|
|
RichardG has joined #archiveteam |
|
23:03
🔗
|
|
RichardG has quit IRC (Ping timeout: 362 seconds) |
|
23:04
🔗
|
yipdw |
typically when you have a tracker failure it's not the app, it's the datastore |
|
23:04
🔗
|
yipdw |
but this is pointless demarcation |
|
23:04
🔗
|
|
RichardG has joined #archiveteam |
|
23:07
🔗
|
|
RichardG has quit IRC (Remote host closed the connection) |
|
23:15
🔗
|
|
cfarence has quit IRC (Quit: Page closed) |
|
23:16
🔗
|
SketchCow |
Thiniverse is back |
|
23:32
🔗
|
wp494 |
<100K items to go on thingiverse tracker |
|
23:37
🔗
|
godane |
so you guys are grabbing it now? |
|
23:37
🔗
|
godane |
i hope it grabs images and the zip file |
|
23:37
🔗
|
SketchCow |
We are, but I'm concerned we're not doing the full thing. |
|
23:37
🔗
|
SketchCow |
But let's emergency grab right now |
|
23:37
🔗
|
godane |
ok |
|
23:37
🔗
|
godane |
i will still do my grabs |
|
23:38
🔗
|
godane |
my code: wget -x -i index.txt -p --content-disposition --no-check-certificate --warc-file=$website-thing-id-$start-to-$end-$(date +%Y%m%d) --warc-cdx -H --domains=thingiverse.com,s3.amazonaws.com --reject-regex="(target_thumb|/icon-flag.png|/mb-images/|/img/|/fonts/)" -w 0.3 -E --warc-max-size=1G -o wget.log |
|
23:39
🔗
|
godane |
you have to have a wait or you will get block |
|
23:39
🔗
|
godane |
but the block is short luckly |
|
23:40
🔗
|
|
RedType has joined #archiveteam |
|
23:43
🔗
|
joepie91 |
wp494: not sure it's all the items |
|
23:43
🔗
|
joepie91 |
seems only 500k |
|
23:43
🔗
|
joepie91 |
I suspect there's more |
|
23:55
🔗
|
godane |
so 10 lost episodes of JIBS News General 2008-02 is saved |
|
23:55
🔗
|
godane |
2008-02-10 episode couldn't be saved with vlc |
|
23:56
🔗
|
godane |
it was dead dead |
|
23:56
🔗
|
godane |
the lost episodes are only dead when grabbing in mplayer |