#archiveteam-bs 2017-07-26,Wed

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***schbirid2 has quit IRC (Read error: Operation timed out) [00:00]
Stiletto has quit IRC ()
kristian_ has joined #archiveteam-bs
[00:06]
schbirid has quit IRC (Ping timeout: 255 seconds) [00:17]
nyany has quit IRC (Leaving)
schbirid has joined #archiveteam-bs
Stiletti has joined #archiveteam-bs
Stiletti is now known as Stiletto
[00:26]
...... (idle for 28mn)
bitspillAll of my Roblox workers are either on 500 errors or rsync max connections(120) [00:59]
***kristian_ has quit IRC (Quit: Leaving) [01:04]
.... (idle for 18mn)
nyany has joined #archiveteam-bs [01:22]
..... (idle for 24mn)
j08nY has quit IRC (Remote host closed the connection) [01:46]
schbirid has quit IRC (Ping timeout: 255 seconds) [01:56]
wp494chfoo, any way you can nudge your rsync connection max up a bit
oh wait it's on FOS
nvm
I'm dumb
[02:02]
***schbirid has joined #archiveteam-bs [02:09]
kristian_ has joined #archiveteam-bs
TheLovina has quit IRC (Read error: Operation timed out)
TheLovina has joined #archiveteam-bs
[02:16]
.... (idle for 16mn)
ld1 has quit IRC (Ping timeout: 260 seconds)
ld1 has joined #archiveteam-bs
[02:36]
...... (idle for 26mn)
kristian_ has quit IRC (Quit: Leaving) [03:03]
.... (idle for 19mn)
mundus201http://www.archiveteam.org/ is returning 509 [03:22]
***schbirid has quit IRC (Read error: Operation timed out) [03:36]
schbirid has joined #archiveteam-bs
pizzaiolo has quit IRC (Quit: pizzaiolo)
[03:48]
wp494wait a bit
try again
[03:53]
.... (idle for 18mn)
***Stiletto has quit IRC () [04:11]
..... (idle for 22mn)
schbirid has quit IRC (Read error: Operation timed out) [04:33]
schbirid has joined #archiveteam-bs
Sk1d has quit IRC (Ping timeout: 194 seconds)
[04:45]
Sk1d has joined #archiveteam-bs [04:52]
mundus201been nearly 2 hours now [05:03]
.......... (idle for 48mn)
***schbirid has quit IRC (Ping timeout: 255 seconds) [05:51]
schbirid has joined #archiveteam-bs [06:04]
.... (idle for 17mn)
Honno has joined #archiveteam-bs
ld1 has quit IRC (Ping timeout: 260 seconds)
ld1 has joined #archiveteam-bs
[06:21]
AoedeIs there a channel for Roblox? [06:28]
...... (idle for 29mn)
wp494nope, this is it [06:57]
Aoedeokay, thanks [06:57]
...... (idle for 26mn)
midasarchiveteam, running out of bandwith sinds 2017 ;) [07:23]
wp494and rsync connections on FOS too
(BTW SketchCow, any chance of upping the limit a bit or is 120 all we're getting?)
[07:25]
........ (idle for 36mn)
midasthat box is probably dying under that load anyway [08:02]
wp494possibly
might need another target that can handle much higher
[08:05]
.... (idle for 17mn)
***j08nY has joined #archiveteam-bs [08:22]
schbirid has quit IRC (Ping timeout: 255 seconds) [08:27]
schbirid has joined #archiveteam-bs
tuluu has quit IRC (Ping timeout: 260 seconds)
[08:41]
....... (idle for 30mn)
kurt_ has quit IRC (Read error: Operation timed out)
Igloo_ has quit IRC (Read error: Operation timed out)
mgrytbak has quit IRC (Read error: Operation timed out)
tapedrive has quit IRC (Read error: Operation timed out)
tapedrive has joined #archiveteam-bs
kurt has joined #archiveteam-bs
Igloo has joined #archiveteam-bs
ItsYoda has quit IRC (Quit: rippppp to the yoda you used to know!)
mgrytbak has joined #archiveteam-bs
Hecatz has quit IRC (Ping timeout: 268 seconds)
ItsYoda has joined #archiveteam-bs
schbirid has quit IRC (Read error: Operation timed out)
Hecatz has joined #archiveteam-bs
[09:14]
zhongfu has quit IRC (Ping timeout: 260 seconds)
schbirid has joined #archiveteam-bs
Ravenloft has quit IRC (Ping timeout: 260 seconds)
zhongfu has joined #archiveteam-bs
[09:34]
.... (idle for 17mn)
schbirid2 has joined #archiveteam-bs
sun_shine has joined #archiveteam-bs
efsnable has joined #archiveteam-bs
[09:55]
sun_shineI'm interested in archiving a pyramid scheme's website [09:57]
***schbirid has quit IRC (Read error: Operation timed out) [09:58]
sun_shineSaid pyramid scheme has "virtual parties", the IDs of which are incremented integers. Over the past month the average is 1 party every 11.4 seconds.
The URLs to archive follow a pattern like http://example.scam/{user_id}/party/{party_id}/view
but a GET request to http://example.scam/party/{party_id}/view will return a 302 to the correct URL with the username
[09:59]
My question is somewhat about whether this is worth pointing ArchiveBot at and then secondarily how to best handle either a range or just a list of URLs covering a given time period. [10:08]
***DFJustin has quit IRC (Ping timeout: 260 seconds)
DFJustin has joined #archiveteam-bs
swebb sets mode: +o DFJustin
BlueMaxim has quit IRC (Read error: Operation timed out)
BlueMaxim has joined #archiveteam-bs
[10:13]
bitBaron has quit IRC (Read error: Operation timed out)
ja0Hai has quit IRC (Ping timeout: 260 seconds)
ja0Hai has joined #archiveteam-bs
[10:26]
.... (idle for 17mn)
zhongfu has quit IRC (Ping timeout: 260 seconds)
zhongfu has joined #archiveteam-bs
[10:44]
zhongfu has quit IRC (Read error: Connection reset by peer)
zhongfu has joined #archiveteam-bs
[10:51]
midasyou can add a list of urls using !archiveonly < https://www.example.com/some-file.txt
https://archivebot.readthedocs.io/en/latest/commands.html#archiveonly-file
cc sun_shine
[11:02]
***sun_shine has quit IRC (Ping timeout: 245 seconds) [11:17]
....... (idle for 33mn)
GLaDOSim working on bringing up another rsync target [11:50]
***j08nY has quit IRC (Quit: Leaving) [11:59]
BlueMaxim has quit IRC (Quit: Leaving)
username1 has joined #archiveteam-bs
schbirid2 has quit IRC (Read error: Operation timed out)
[12:11]
SketchCow120 is about all we can do.
That machine gets mega-hit all the time.
[12:24]
***schbirid2 has joined #archiveteam-bs
username1 has quit IRC (Read error: Operation timed out)
[12:29]
.... (idle for 18mn)
ld1 has quit IRC (Ping timeout: 260 seconds)
ld1 has joined #archiveteam-bs
[12:48]
SketchCowMy guy who usually gives me 10-15 new CD-ROMs with images every two weeks, stumbled on a collection of Russian Warez CDs.
We already have a lot of Russian Warez CDs, but this one, Triada, is complete and massive.
He has a stupid fast pipe to us, it's usually a case of uploading the materials pretty quickly.
But he's been at it over a day and a half.
151 CDs and DVDs, and 180gb so far.
[12:51]
GLaDOSwow, that's pretty damn neat [12:56]
...... (idle for 29mn)
***TheLovina has quit IRC (Ping timeout: 1208 seconds) [13:25]
Stiletti has joined #archiveteam-bs [13:35]
DarkstarI remember there was a way to view the job log (derive tasks etc.) of an item that was uploaded to IA. anyone know which URL I need to reach this?
it was a site with a couple tables with small, colored cells which contained links to the raw job logs etc. Maybe it was even external to archive.org, I'm not sure
[13:42]
midasnope, it's on the https://monitor.archive.org/ page
my bad, wrong box
[13:55]
***username1 has joined #archiveteam-bs [13:58]
midasbut i think you might need admin rights for it [14:00]
Darkstarno, I have definitely seen these logs for some of my uploads before [14:01]
PurpleSymDarkstar: archive.org/history/<item> ? [14:02]
midasgot a link to your upload? [14:02]
Darkstaryes, exactly. thanks @PurpleSym! [14:02]
midasthe only one i know is https://catalogd.archive.org/ :x [14:03]
***schbirid2 has quit IRC (Read error: Operation timed out)
username1 has quit IRC (Read error: Operation timed out)
schbirid has joined #archiveteam-bs
Stiletto has joined #archiveteam-bs
Stiletti has quit IRC (Ping timeout: 260 seconds)
[14:05]
mlsI'll drop some concurrent on Roblox until another rsync target becomes available [14:21]
DarkstarHm, who is this Jeff Kaplan and why does he mark perfectly valid KryoFlux dumps on archive.org as "spam"? ;-)
(not my upload though, so I'm probably not the right person to contact him about it)
[14:24]
efsnabledorkstar [14:25]
midasJeff is the dude @ archive.org [14:26]
efsnablehi midas
remember me
u lesbo freak
[14:27]
Darkstarmidas: "the dude"? I thought that's Jason :) [14:28]
midasefsnable: no, should i? [14:29]
GLaDOSi think we have a soundcloud employee in here..
or that guy that runs/ran twitpic
can't tell, they're all so, so angry
[14:29]
midasoh yeah the twitpic dude, he was really angry
i liked him
[14:30]
efsnableim gay [14:31]
midasi don't care what you are. [14:32]
efsnablei sell weed to elementary school kids [14:32]
midasi dont care what you do either [14:32]
efsnablethey caught me
im going to prison soon
for 5 years
[14:33]
midasok [14:33]
efsnablemidas r u a chick ?
can i suck u
[14:33]
***yipdw sets mode: +b *!bossgt100@70.39.109.163
efsnable was kicked by yipdw (efsnable)
[14:35]
mlsAw I'm too late with the popcorn
mls *sulks*
[14:37]
midasty yipdw [14:42]
***kittymeow has joined #archiveteam-bs [14:43]
DFJustinDarkstar: I poked jason [14:43]
kittymeowDoes anyone know a good way of archiving a page that needs a login/cookie with an external site like archive.is or webcitation or wayback [14:44]
DFJustinkaplan is an (the only?) admin who has to go through all the tons of crap coming in daily, sometimes he makes a mistake [14:44]
DarkstarDFJustin: thanks. this is about samna-ami-kryoflux by the way. I didn't realize that there is actually a real person sifting through all the uploads all day long :) [14:45]
kittymeowWhere sure a single person logged on can save it, but then there's no way to know if that person edited the page they archived to add or change stuff
so it'd be really good if there was anything like that that would act as a proxy or something and then archive it
While you are logged in
[14:45]
DFJustinhmm can https://webrecorder.io/ do that? (I haven't tried) [14:47]
kittymeowexample https://register.thesecretworld.com/account/paidservice/ctrl/offer the whole of the http://tswshop.funcom.com site is only visible while logged in, but it contains prices
thanks I'll try
[14:48]
***ld1 has quit IRC (Ping timeout: 260 seconds)
ld1 has joined #archiveteam-bs
[14:50]
kittymeow"Webrecorder MaintenanceWebrecorder is being upgraded!Please come back soon!" :( [14:52]
yipdwyou can run it on your own computers
https://github.com/webrecorder/webrecorder
[14:52]
***godane has quit IRC (Quit: Leaving.)
PurpleSym sets mode: +o midas
[14:54]
kittymeowI use https://addons.mozilla.org/addon/scrapbook for stuff like that it's really good.. but the point is if it's on client side instead of a remote site, it's hard to prove with controversial or money related stuff that the person archiving it didn't edit it before sharing the archive files [14:55]
***qw3rty3 has joined #archiveteam-bs [14:59]
kittymeowThis could apply to a lot of politiucal stuff on facebook etc too, more and more stuff is locked behind "you need an account to view this" as corporations get more confident of a monpoly that they know people will feel forced to do it [15:03]
***Zebranky has quit IRC (Ping timeout: 633 seconds)
Zebranky has joined #archiveteam-bs
[15:04]
GLaDOSoh boy, instantly the uploads start [15:07]
***schbirid has quit IRC (Read error: Operation timed out) [15:08]
mlsGLaDOS: Oh so you've noticed eh? ;-) [15:11]
GLaDOSwell the second i added combine harvester as a target, about 150 pipelines connected [15:11]
mlsIsn't there a round robin or load balancing thingamabob? [15:12]
GLaDOSthere is in the tracker, but it only works if there's more than one target
before it was only FOS
[15:13]
mlsAh right, I see now (actually paying attention to the wall of text) [15:14]
GLaDOSso hopefully that should keep things smooth [15:14]
Jonimuskittymeow: actually I think https://www.taricorp.net/2016/web-history-warc/ this might be interesting to you, it uses your firefox cookies to get data from sites that require login. [15:14]
mlsI don't know what the current threshold is on the tracker, but I have a guess it's enough to congest both rsync targets like easy looking at the avg upload size [15:16]
GLaDOSwell right now the limit is at 150 items/minute, i'm not sure why it's set at that so i'm leaving it
may i suggest #robloxd
[15:16]
mlsBy all means [15:18]
***Stiletto has quit IRC ()
schbirid has joined #archiveteam-bs
schbirid2 has joined #archiveteam-bs
schbirid has quit IRC (Read error: Operation timed out)
[15:21]
ld1JAA: Lots of `.woff` are being pulled. Maybe doubles that could be left out.
cc arkiver
[15:27]
***qw3rty3 has quit IRC (Nettalk6 - www.ntalk.de)
schbirid2 has quit IRC (Ping timeout: 255 seconds)
[15:31]
pizzaiolo has joined #archiveteam-bs
pizzaiolo has left
[15:40]
schbirid2 has joined #archiveteam-bs
Stiletti has joined #archiveteam-bs
godane has joined #archiveteam-bs
svchfoo1 sets mode: +o godane
[15:46]
.... (idle for 18mn)
ld1 has quit IRC (Ping timeout: 260 seconds)
ld1 has joined #archiveteam-bs
[16:10]
...... (idle for 26mn)
Stiletti is now known as Stiletto [16:36]
...... (idle for 26mn)
username1 has joined #archiveteam-bs
schbirid2 has quit IRC (Read error: Operation timed out)
[17:02]
xmcalso i pay an unknowable amount of money to host the tracker and a few archivebot pipelines :) [17:18]
SketchCowHuzzah [17:20]
***Retroity has joined #archiveteam-bs [17:23]
pizzaiolo has joined #archiveteam-bs
Retroity has quit IRC (Quit: Page closed)
[17:31]
ReimuHaku has quit IRC (Ping timeout: 250 seconds) [17:45]
ReimuHaku has joined #archiveteam-bs [17:51]
ReimuHaku has quit IRC (Ping timeout: 245 seconds) [17:57]
ReimuHaku has joined #archiveteam-bs [18:03]
ndiddy-pi has joined #archiveteam-bs
ndiddy has quit IRC (Read error: Connection reset by peer)
ndiddy-pi is now known as ndiddy
ReimuHaku has quit IRC (Ping timeout: 245 seconds)
RichardG has quit IRC (Ping timeout: 370 seconds)
ReimuHaku has joined #archiveteam-bs
[18:14]
Aranje has joined #archiveteam-bs [18:29]
....... (idle for 33mn)
ld1_ has joined #archiveteam-bs
ld1 has quit IRC (Ping timeout: 260 seconds)
[19:02]
ld1_ is now known as ld1 [19:13]
............... (idle for 1h11mn)
username1 has quit IRC (Ping timeout: 255 seconds)
balrog has quit IRC (Ping timeout: 260 seconds)
[20:24]
username1 has joined #archiveteam-bs [20:37]
username1 has quit IRC (Quit: Leaving)
Stiletto has quit IRC (Read error: Operation timed out)
[20:44]
......... (idle for 42mn)
godane has quit IRC (Ping timeout: 260 seconds) [21:27]
godane has joined #archiveteam-bs
svchfoo1 sets mode: +o godane
[21:40]
........ (idle for 37mn)
svchfoo3 has quit IRC (Quit: Closing) [22:18]
RichardG has joined #archiveteam-bs [22:25]
SilSte has quit IRC (Read error: Operation timed out) [22:39]
SilSte has joined #archiveteam-bs
Silvan has joined #archiveteam-bs
Silvan has quit IRC (Read error: Connection reset by peer)
SilSte has quit IRC (Read error: Operation timed out)
[22:50]
SilSte has joined #archiveteam-bs [23:01]
..... (idle for 23mn)
ranma"Anyways.. while I know the MAME effort has been going on a long time, and in most cases they have been left alone by the original manufacturer/IP Rights Holder, they really need to be careful with what they're doing now; these manufaturers still have big legal departments, are more than willing to go for the throat, and companies like Namco and Nintendo in particular don't fool around
with this stuff. Copying ROMs is one thing, but when you're cracking the custom ASICs and other security devices they designed into the game hardware to prevent more or less what they're actually trying to do? That's one step away from producing knock-offs of the actual game. It's been a long time since I was involved in the coin-op game industry, but I'm sure that there are still plenty
of countries that would welcome with open arms even arcade games from 10 to 20 years ago, being better than what they have right now."
i hope those rippers are at least doing a minimal job to stay anonymous
from https://arstechnica.com/gaming/2017/07/mame-devs-are-cracking-open-arcade-chips-to-get-around-drm/
nothing pisses me off more than when, say, a ROM translation or fan project reskinning/new content creators get shut down with a C & D
because they didn't even like try to stay anonymous
[23:24]
..... (idle for 23mn)
***zyphlar has joined #archiveteam-bs [23:49]
Odd0002 has joined #archiveteam-bs [23:59]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)