#archiveteam-bs 2017-06-23,Fri

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***dashcloud has quit IRC (Ping timeout: 260 seconds)
j08nY has quit IRC (Quit: Leaving)
dashcloud has joined #archiveteam-bs
[00:08]
dashcloudLord_Nigh: for what it's worth, I use an SVHS vcr and an AVT-8710 (which has TBC) for VHS capture [00:12]
SketchCowNobody complains about anyone [00:13]
jrwrI figured SketchCow, we are using a nginx proxy to reduce connections to the API
since we where maxing it out
[00:16]
***dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
dashcloud has joined #archiveteam-bs
[00:29]
..... (idle for 23mn)
BlueMaxim has joined #archiveteam-bs [00:52]
nyanyare there any projects worth going for at the moment other than urlteam?
sorry, haven't been involved lately.
[00:57]
***dashcloud has quit IRC (Read error: Operation timed out) [01:00]
dashcloud has joined #archiveteam-bs [01:07]
voidsta2/6 [01:08]
***bitBaron has quit IRC (Quit: My computer has gone to sleep. ZZZzzz…)
kristian_ has quit IRC (Quit: Leaving)
[01:09]
jrwrnyany: NewsGrabber [01:10]
ivanhttps://twitter.com/_jared/status/878036763805048832 Basho probably dead [01:11]
jrwrhttp://basho.com/
check out the link near the bottom middle that says Basho’s Riak® is a distributed NoSQL database that delivers:
its Broken
how in the hell do you have a professional website with a broken link
[01:13]
timmcTheir tech blog has some good posts, e.g. http://basho.com/posts/technical/vector-clocks-revisited-part-2-dotted-version-vectors/
I know someone who used to work there.
[01:15]
***Stilett0 has quit IRC (Ping timeout: 268 seconds) [01:21]
nyanyjrwr: got it [01:30]
godanehey SketchCow
i'm closer to been done with my tape captures
i got the 2 full selfs of tapes gone
i'm just down to a cupboard
of tapes
[01:41]
jrwrAudio/Video/Data [01:42]
***Soni has joined #archiveteam-bs [01:46]
MrRadarIf your system is stationary a better solution might be a cantenna or equivalent [01:46]
jrwrYa
Most APs are about 100-200mW
that Mikrotik is rated for outdoor use :)
[01:46]
Soniwhy not add grounded tinfoil padding to my house and a cell booster? [01:46]
jrwr:)
Its what I use, and have been so far impressed as hell by it
[01:47]
.... (idle for 17mn)
***Aranje has joined #archiveteam-bs [02:04]
dashcloud has quit IRC (Read error: Operation timed out) [02:13]
CrusherIsn't a 1000mW AP a good way to have the FCC / CRTC knocking at your door? [02:15]
***Asparagir has joined #archiveteam-bs [02:16]
Froggingnot if you have an amateur radio license and are using the right frequency :)
(part of one of the ham bands intersects wifi :p)
[02:16]
jrwrCrusher: https://routerboard.com/RB951Ui-2HnD [02:19]
CrusherHue [02:23]
***zenguy has quit IRC (Read error: Operation timed out)
Stilett0 has joined #archiveteam-bs
[02:23]
CrusherThat's naughty :P [02:23]
jrwrHehehe
And I can set it to non-us channels
CH18 HERE I COME
[02:24]
***zenguy has joined #archiveteam-bs [02:25]
CrusherUh oh... Why did my server stop contributing to imzy...
That's not good
[02:26]
jrwrI use this little guy Crusher mounted to a pith helmet https://routerboard.com/RBmAP2nD
https://jrwr.io/doku.php?id=projects:hatnas
[02:30]
***dashcloud has joined #archiveteam-bs [02:32]
Froggingthat is neat [02:33]
CrusherCombine that with the guy who made the solar charging hat Kickstarter
And the runtime becomes forever :P
[02:34]
jrwrRight now its 29 Hours [02:35]
Crusher(yes, he used LiFePo4 cells) [02:35]
jrwrwith that battery pack [02:35]
CrusherThat one is using LiFe cells right?
Not straight Lithium ion?
[02:35]
jrwrLi-ion
its a 27k mWh Battery
fits in my pocket very well
[02:36]
CrusherOh ok
Pocket mounted battery
[02:36]
jrwrLong 6ft cables [02:37]
CrusherI'm not putting a lion battery on my head xD [02:37]
Soniyay for docker
never used it before but it seems to have worked
[02:37]
CrusherBrb, switching to WiFi [02:37]
jrwrCrusher: its a Public File Sharing system [02:37]
***Crusher has quit IRC (Bye)
Crusher has joined #archiveteam-bs
[02:37]
jrwrSo its a public file sharing system
I take it to Conventions and things
and wear it around walmart sometimes
[02:40]
Frogginghehe [02:40]
jrwrit has a PHP based interface
a little menu on the pi it self to do admin things
(uses 2FA OTC to unlock)
and is pretty secure, even has SSL + HTTP/2
[02:40]
SoniI can hear my wifi and it's annoying [02:41]
jrwrYou can hear 2.4Ghz?! [02:41]
Froggingcan you hear radio stations
that'd be super cool
[02:41]
Sonimy speakers kinda pick up wifi and basically any radio signals [02:41]
jrwrOh [02:42]
Soniso I hear annoying noise [02:42]
jrwrYou need a magnet puck [02:42]
Sonibut that's how I know it's working [02:42]
Froggingneed ferrites [02:42]
SoniI need a faraday cage tbh
or uh
shielded cables
[02:42]
jrwrAnyway, I call it HatNAS [02:44]
Frogginggood name [02:44]
jrwrI figured
Im kind of broke, or would of gotten more drives for it
Need a way to hack a voltagereg on the pi so I can talk USB Charger and get more volts
if I can get 14 @ 2A (what QC2 is rated for)
and down convert to 5v 1.5A Stable
that would be legit
[02:47]
robogoatjrwr: you're going to defcon? [02:51]
jrwrYes [02:51]
robogoatrobogoat is jealous. [02:51]
jrwrIm going to go super broke doing it [02:51]
robogoatBunch of buddies are headed out for that week. [02:51]
jrwrWhole paycheck [02:51]
***Crusher has quit IRC (Read error: Connection reset by peer)
Crusher has joined #archiveteam-bs
Crusher has quit IRC (Client Quit)
[02:51]
robogoatI hate vegas, but seeing some folks that have come distances would be fun. [02:52]
jrwrI am going to a CTF this weekend
my first one
[02:52]
robogoatCool, have fun! [02:53]
jrwrIm with a local CitySec style group
so I have backup, we meetup once a month, damn good fellows
[02:53]
robogoatNice, yeah,
there is a local defcon group and 2600 meetup in my area.
And I host a sparsely attended cryptoparty.
[02:54]
***Crusher has joined #archiveteam-bs [02:55]
jrwrWe have about 10 guys show up in a very nice tavern and talk shit for about 8 hours [02:55]
robogoatNice. [02:55]
jrwrWe hang out in a slack all day as well, about 200 of us [02:56]
robogoatWhoa, that's quite a crew.
There's no local infosec slack here AFAIK.
We have a general technology slack people hang out in, though.
[02:56]
jrwrYa, Ours is called STLSec [03:01]
CrusherHey, imzy is really speeding up
Half way done
[03:04]
.... (idle for 17mn)
***th1x has joined #archiveteam-bs [03:21]
Soniaw shit *plugs holes* [03:28]
CrusherMundus, stop making us look bad!
:P
[03:29]
Soniholes plugged
idk why I operate my firewall in blacklist mode .-.
[03:30]
...... (idle for 28mn)
***qw3rty2 has joined #archiveteam-bs [03:58]
qw3rty has quit IRC (Read error: Operation timed out) [04:04]
........ (idle for 35mn)
Crusher_ has joined #archiveteam-bs
Crusher has quit IRC (Read error: Connection reset by peer)
Sk1d has quit IRC (Ping timeout: 194 seconds)
[04:39]
Sk1d has joined #archiveteam-bs [04:46]
...... (idle for 26mn)
Crusher has joined #archiveteam-bs
Crusher_ has quit IRC (Read error: Connection reset by peer)
[05:12]
..... (idle for 21mn)
CrusherIs there any reason for the checkip() function to not be run only once when the warrior starts?
Currently it seems to run during every thread, which seems inefficient given that "am I behind a proxy / firewall" is not going to change between threads
[05:33]
***pizzaiolo has quit IRC (Quit: pizzaiolo) [05:34]
CrusherI'm sure the sites used as examples would appreciate not having our packets hit them every time we try to scrape a file [05:38]
.... (idle for 16mn)
***Famicoman has quit IRC (Ping timeout: 260 seconds) [05:54]
Famicoman has joined #archiveteam-bs [06:01]
...... (idle for 28mn)
j08nY has joined #archiveteam-bs [06:29]
..... (idle for 20mn)
JAACrusher: The IP check only looks up the IPs for those hosts, it doesn't actually contact them (as far as I know).
It's really just "Do these hosts resolve to different IPs?", not a thorough check whether those pages return specific content or something like that.
[06:49]
..... (idle for 21mn)
***th1x_ has joined #archiveteam-bs [07:11]
th1x has quit IRC (Read error: Operation timed out)
fie has quit IRC (Ping timeout: 600 seconds)
[07:16]
...... (idle for 26mn)
fie has joined #archiveteam-bs [07:42]
..... (idle for 24mn)
bsmith093 has quit IRC (Ping timeout: 250 seconds)
j08nY has quit IRC (Quit: Leaving)
SilSte has quit IRC (hub.efnet.us hub.dk)
dan- has quit IRC (hub.efnet.us hub.dk)
robinak has quit IRC (hub.efnet.us hub.dk)
eprillios has quit IRC (hub.efnet.us hub.dk)
Lord_Nigh has quit IRC (hub.efnet.us hub.dk)
ReimuHaku has quit IRC (hub.efnet.us hub.dk)
RedType has quit IRC (hub.efnet.us hub.dk)
pikhq has quit IRC (hub.efnet.us hub.dk)
mgrytbak has quit IRC (hub.efnet.us hub.dk)
acridAxid has quit IRC (hub.efnet.us hub.dk)
joepie91 has quit IRC (hub.efnet.us hub.dk)
cf has quit IRC (hub.efnet.us hub.dk)
chfoo has quit IRC (hub.efnet.us hub.dk)
tapedrive has quit IRC (hub.efnet.us hub.dk)
underscor has quit IRC (hub.efnet.us hub.dk)
espes__ has quit IRC (hub.efnet.us hub.dk)
kvieta has quit IRC (hub.efnet.us hub.dk)
klg has quit IRC (hub.efnet.us hub.dk)
[08:06]
schbirid has joined #archiveteam-bs
bsmith093 has joined #archiveteam-bs
[08:17]
bsmith093 has quit IRC (Ping timeout: 250 seconds) [08:29]
bsmith093 has joined #archiveteam-bs [08:36]
BlueMaxim has quit IRC (Quit: Leaving) [08:41]
Lord_Nigh has joined #archiveteam-bs [08:47]
werethles has joined #archiveteam-bs [08:57]
werethlesSo.
I ran the "pip install --upgrade seesaw" But what directory do I cd to?
[08:57]
JAAThe repository. I.e. git clone https://github.com/ArchiveTeam/imzy-grab.git && cd imzy-grab
You'll probably run into issues when you run get-wget-lua.sh though. Coincidentally, I just helped someone with that yesterday. See here: http://archive.fart.website/bin/irclogger_log/archiveteam-bs?date=2017-06-22,Thu&sel=347#l343
(Obviously, you can ignore the OpenSSL parts if you use GnuTLS.)
[08:58]
werethlesSo I can't have the script auto choose a project? [09:04]
JAANo, that's not possible.
The scripts will also not update automatically. If you're getting an "out of date" error, you'll have to stop them, run git pull, and restart.
[09:05]
werethlesOhhhh
Okay.
[09:05]
JAAThe warrior is more of a "set up and forget"-type of thing, while the scripts offer more control. [09:06]
werethlesThat seems super easy to automate with chron though
Yeah.
[09:06]
JAAYeah, it can certainly be done. [09:06]
werethlesAll righty. I'll just have chron auto re-pull every 24hrs or so
Thanks a bunch!
[09:06]
JAAYou'll want something like `touch STOP && wait <insert PID of the pipeline process> && git pull && run-pipeline pipeline.py ...`.
`touch STOP` initiates the stop, but the scripts will finish the current items first, so you need to wait for the process. Then update and restart.
[09:07]
werethlesGood call! [09:09]
JAAOr you could set something up which first checks for updates and only runs this if there are changes.
Something something git fetch && git diff master origin/master
[09:09]
***SHODAN_UI has joined #archiveteam-bs
werethles has quit IRC (Quit: Page closed)
[09:20]
.... (idle for 18mn)
thuban4 has joined #archiveteam-bs [09:42]
zenguy has quit IRC (Ping timeout: 245 seconds)
thuban3 has quit IRC (Read error: Operation timed out)
zenguy has joined #archiveteam-bs
sun_shine has joined #archiveteam-bs
[09:47]
sun_shineHi. There's a long-running "ex-mormon" forum that will close July located lifeaftermormonism.net . Can someone help point archivebot at it? [09:52]
***dan- has joined #archiveteam-bs [10:00]
........ (idle for 35mn)
SHODAN_UI has quit IRC (Remote host closed the connection) [10:35]
JAAThe countries around Qatar have demanded (again, I believe) that Al-Jazeera be closed in 10 days: http://www.aljazeera.com/news/2017/06/arab-states-issue-list-demands-qatar-crisis-170623022133024.html
There was an ArchiveBot job a bit over two weeks ago. :-)
... but only for the English version. I'll queue the others.
[10:35]
..... (idle for 20mn)
***C4K3 has quit IRC (Read error: Connection reset by peer)
C4K3 has joined #archiveteam-bs
icedice has joined #archiveteam-bs
[11:00]
.... (idle for 17mn)
icedice has quit IRC (Ping timeout: 268 seconds) [11:22]
.... (idle for 16mn)
icedice has joined #archiveteam-bs [11:38]
JAATIL: the referrer matters in wpull. If you recursively grab https://example.com/a and /b, and /a contains a link to (or in another way adds) /b/foo to the queue, /b/foo will not be retrieved, but if it's added from /b, it will. [11:42]
Sanquigood to know, thanks [11:45]
JAAI'm not sure what happens if both /a and /b link to /b/foo. I assume that whichever discovers the link first will win. [11:46]
***kvieta has joined #archiveteam-bs
espes__ has joined #archiveteam-bs
tapedrive has joined #archiveteam-bs
chfoo has joined #archiveteam-bs
cf has joined #archiveteam-bs
joepie91 has joined #archiveteam-bs
acridAxid has joined #archiveteam-bs
mgrytbak has joined #archiveteam-bs
pikhq has joined #archiveteam-bs
RedType has joined #archiveteam-bs
klg has joined #archiveteam-bs
underscor has joined #archiveteam-bs
ReimuHaku has joined #archiveteam-bs
eprillios has joined #archiveteam-bs
robinak has joined #archiveteam-bs
SilSte has joined #archiveteam-bs
hub.dk sets mode: +o underscor
swebb sets mode: +o underscor
pizzaiolo has joined #archiveteam-bs
[11:52]
.... (idle for 17mn)
Crusher_ has joined #archiveteam-bs [12:12]
Crusher has quit IRC (Read error: Operation timed out) [12:17]
JAAHaha, I was wondering why my wpull tests were so slow. Turns out, I had a time.sleep(5) in the get_urls hook. [12:21]
***Jonison has joined #archiveteam-bs
Hiccup has joined #archiveteam-bs
[12:22]
Hiccupmoved to -bs [12:22]
Crusher_Unless my connection to the tracker died
It looks like the tracker died
Imzy is still up
[12:28]
***Hiccup has quit IRC (Quit: Page closed) [12:29]
Crusher_Hmm. There's still a few uploads trickling in, but something is borked
And it's back... That was odd
[12:30]
Fhew, For a second there it looked like that was the end of the downloads [12:36]
***icedice has quit IRC (Quit: Leaving)
SHODAN_UI has joined #archiveteam-bs
[12:40]
JAACrusher_: That's been happening from time to time at least since yesterday evening (UTC). [12:42]
Crusher_Huh. Any idea what's going on?
A hiccup at imzy or the tracker?
[12:42]
***icedice has joined #archiveteam-bs [12:43]
JAAImzy [12:43]
Crusher_Maybe it's some sort of routinely run backup that's pushing it over the edge with our scraping [12:44]
JAAI have to say though that I'm a bit surprised they're still online, given how firmly greenie said yesterday that they're probably going to shut down around 00:00 EST (05:00 UTC, 7h 45m ago). [12:44]
Crusher_So for anyone actually in that region, 01:00 ;P
But yeah, that is a welcome time extension
[12:46]
JAARight. (He did say EST though.) [12:47]
Crusher_Just a few more hours is all we need [12:47]
JAAWell, we'll have to requeue lots of items as well.
But yeah, I won't complain. :-)
[12:47]
Crusher_how many items are out right now?
Oh wow
That's bad xD
Make that more than a few hours
[12:48]
JAAI wonder how many of those are mundus's. ;-) [12:48]
Crusher_To be fair, I think a good number of them are failing
Right before I left, I noticed my logs were returning a good number a dead ones
[12:49]
JAAYeah, I know. [12:50]
***REiN^ has quit IRC (Max SendQ exceeded) [12:50]
JAAI see about as many fails as successes on my machines. [12:50]
Crusher_I wonder if they *are* in the middle of shutting down [12:51]
***REiN has joined #archiveteam-bs [12:51]
Crusher_Awfully slow shutdown, but possible [12:51]
JAAOr we're overloading their database servers again. [12:52]
Crusher_Eh, the site is still responding snappily [12:52]
JAAMuch of the site is probably cached. [12:53]
Crusher_True... [12:53]
timmcyou're busting through all the caches though [12:53]
JAAYeah, obviously [12:54]
***icedice has quit IRC (Read error: Connection reset by peer) [12:54]
JAAI was trying to say: the site working fine is not a surprise and doesn't contradict the errors we see on the grab. [12:54]
timmcah, yeah [12:54]
Crusher_So what's the difference between the site and our grabs, are we grabbing the raw posts? [12:56]
JAAWe're iterating over *all posts*. [12:56]
Crusher_Ah. [12:56]
***Jonison has quit IRC (Read error: Connection reset by peer) [12:56]
JAAThe site will just retrieve a post once, then serve it from cache (until the cache expires), so refreshing will be very fast. [12:57]
Crusher_Right. Alright, well off to a compsci exam. [12:58]
JAAGood luck! [12:58]
***Crusher has joined #archiveteam-bs [12:58]
timmchave fun! [13:00]
***user0815 has quit IRC (Ping timeout: 268 seconds)
Crusher_ has quit IRC (Ping timeout: 246 seconds)
[13:01]
.... (idle for 15mn)
godane has quit IRC (Ping timeout: 492 seconds) [13:17]
timmcJAA: The shutdown was planned for last night, but it'll happen tonight instead since ArchiveTeam is pretty close.
Or, well, that's the plan as I hear it. If the DB goes down or something, it probably won't be brought back up.
What is the "out" number on the tracker?
[13:22]
JAAItems currently claimed by a worker. This includes items that have failed. Once the todo counter is down to zero, we'll requeue all items which are "out".
So yeah, we still have quite a bit to do.
[13:27]
timmcinteresting, got it [13:28]
***th1x_ has quit IRC (Leaving) [13:29]
.............. (idle for 1h5mn)
icedice has joined #archiveteam-bs [14:34]
.......... (idle for 49mn)
j08nY has joined #archiveteam-bs [15:23]
icedice has quit IRC (Quit: Leaving) [15:30]
....... (idle for 32mn)
C4K3_ has joined #archiveteam-bs
C4K3 has quit IRC (Read error: Operation timed out)
[16:02]
C4K3 has joined #archiveteam-bs
C4K3_ has quit IRC (Read error: Operation timed out)
[16:16]
bitBaron has joined #archiveteam-bs
bitBaron_ has joined #archiveteam-bs
bitBaron has quit IRC (Read error: Operation timed out)
[16:27]
CrusherOh god, I'm about to use a Pentium 4 to backup all my files that are stored on my school's network drive
I should have thought about this one sooner...
[16:40]
***kristian_ has joined #archiveteam-bs [16:43]
.... (idle for 16mn)
bitBaron_ has quit IRC (Read error: Operation timed out)
bitBaron has joined #archiveteam-bs
[16:59]
timmcImzy has *at most* 11 hours of life left, but looking at the tracker's graphs, at the fastest rate yet it would still take 15 hours to get everything.
Are the URLs shuffled before being distributed to warriors?
[17:03]
CrusherWhat is the port that's taxing their server the most, the sheer number of lookups? Or the actual downloading?
Part*
[17:13]
***ZexaronS has quit IRC (Quit: Leaving) [17:21]
kristian_ has quit IRC (Quit: Leaving) [17:30]
timmcNot entirely sure, but I think it's DB load.
I wonder if walking the URLs in chronological order would help. :-P
[17:43]
Kalrothhaha [17:57]
timmc(Presumably that would help with cache/disk locality and whatnot.) [18:00]
CrusherHypothetical, has anyone considered the option of buying the disks off of companies closing down?
Obviously wouldn't work for an aquihire
[18:11]
***odemg has quit IRC (Quit: Leaving)
Crusher_ has joined #archiveteam-bs
Crusher has quit IRC (Ping timeout: 246 seconds)
[18:24]
ZexaronS has joined #archiveteam-bs [18:41]
JAAtimmc: Yes, as far as I know, jobs are handed out randomly. And I don't think it's possible to prioritise certain jobs (e.g. do users before posts). [18:54]
........ (idle for 39mn)
***Crusher has joined #archiveteam-bs
Crusher_ has quit IRC (Read error: Connection reset by peer)
Crusher has quit IRC (Read error: Connection reset by peer)
[19:33]
bwn has quit IRC (Max SendQ exceeded)
bwn has joined #archiveteam-bs
[19:46]
...... (idle for 28mn)
kristian_ has joined #archiveteam-bs [20:16]
..... (idle for 23mn)
schbiridoh god, i made a 30MB torrent file
at 64mb piece size
not sure i can improve on that :\
[20:39]
MrRadarWow, how much data do you have in there? [20:41]
schbirid~400GB with 400k files. i guess i should zip some directories to reduce the num of files [20:42]
Kazhaha
I know deluge falls over at a couple of thousand files, i guess rtorrent is your only option at that sort of size
[20:43]
schbiridoh it loaded it rtorrent just fine [20:43]
***robinak has quit IRC (Read error: Connection reset by peer) [20:44]
Kalrothyeah, imzy returning 0 now, they hating on us
i'm guessing it's dead, jim
[20:49]
***RedType has left [20:50]
Kalrothnevermind, it's back now, at least returning non-zero status codes on some pages [20:51]
***BartoCH has quit IRC (Ping timeout: 260 seconds)
dashcloud has quit IRC (Read error: Operation timed out)
[20:52]
nyanysite is slow Kalroth [20:54]
***dashcloud has joined #archiveteam-bs [20:54]
Kalrothyeah, i'm guessing they're preparing to flip the switch [20:55]
***robink has joined #archiveteam-bs [20:56]
JAAI doubt anything changed since a few hours ago, see the discussion around 13:00 UTC. [20:56]
***BartoCH has joined #archiveteam-bs
dashcloud has quit IRC (Remote host closed the connection)
[20:59]
nyany{"message": "Uh oh, something went wrong! Sorry about that. We've been alerted and will be on it shortly", "errors": []}
that's what I'm seeing
[20:59]
timmcKalroth: If anything is happening, it's weffey restarting things to bring them *back* to life. :-)
The site going read-only was, as far as I know, the only preparation for site-death.
[20:59]
nyanyI've got all the hardware I can to give it one good push
Looks like things are slowly coming back to life
[21:00]
timmcMy understanding is that the main limit right now is the SQL connection or the DB engine or something close to that. [21:01]
***kristian_ has quit IRC (Quit: Leaving) [21:01]
Kalrothtimmc: alright, sounds good
(not that you're having issues, but that it isn't shutting down just yet :)
[21:02]
timmcHaha, I just learned that Zork made a series of scheduled posts before the site went read-only, so they've been popping up periodically since then. :-D
they're all about ghosts and goodbyes and whatnot
[21:08]
nyanyi think my new favorite number
is 500
[21:11]
xmci like 16384 [21:12]
schbiridare >4GB zip files ok nowadays? [21:12]
nyanyxmc: Ah, see ya like it large mate\
lol
[21:13]
SoniI prefer 2520 [21:19]
MrRadarschbirid: Generally yeah, unless you need to deal with Windows XP-era systems [21:20]
schbiridyay [21:20]
MrRadarLOL, apparently Android didn't add native Zip64 support to their Java runtime until 6.0. WTF Google [21:21]
nyanyGGG [21:22]
MrRadar(Granted, many android phones don't have more than 4 GB of free space on their internal storage...) [21:22]
JAAAnd according to Wikipedia (but without specifying a source), OS X Yosemite can create ZIP64 archives but not extract them, at least through the GUI. [21:33]
***BartoCH has quit IRC (Read error: Connection reset by peer)
BartoCH has joined #archiveteam-bs
Petri152 has quit IRC (Read error: Operation timed out)
[21:34]
greenie20:59 < timmc> Kal.roth: If anything is happening, it's weffey restarting things to bring them *back* to life. :-) <- weffey left on her roadtrip about 3 & 1/2 hours ago, I'm not sure will be at a computer for another 4 hours or so. I think those alert emails are actually turned off now
and yeah the site is super slow right now. if they werent in the process of shutting down, would be measures to take but yeah, are fairly limited options right now.
[21:38]
timmcgreenie: Ah, so it's just the site recovering from hiccups on its own? [21:52]
greenieI don't know, I'm sorry. I don't have access anymore to the stuff that alerts me and tells me why-ish its fussy [21:52]
timmc*nod*
sounds like AT is on their own to figure out appropriate load, though
[21:54]
greenielikely, yeah. [21:56]
JAASo I'm about to archive the Tilt API data. Some of that data is also used when you access the website and click on certain things or scroll to the bottom, and it involves pagination. In some cases, it just appends ?page=2 etc. to the URL, in others it also includes the number of results per page (?page=2&rows=50). The former is much easier to set up, obviously. Should I try to go for maximum playback c
ompatibility or take the easy route?
[21:57]
***pie__ has joined #archiveteam-bs [21:59]
pie__BS! :D
anywhay
pie__ pokes joepie91
[22:00]
joepie91pie__: "I think Gog stores the old versions for compatibility reasons but they don't keep the old installers public. You could try asking support if they can provide you a dowload link for whatever version you want to play."
ref https://www.gog.com/forum/terraria/i_want_to_get_the_old_version_of_terraria
[22:00]
pie__pie__ scratches head
you think theyd give me all 20 previous versions? :P
[22:01]
joepie91lol
you could ask
[22:01]
pie__I could, I don't think they would though...wonder if the game devs would oblige? xD
oh hey https://www.reddit.com/r/CrossCode/comments/36d1fq/anyone_interested_in_the_old_demo_versions/
now how to i talk to this guy
thats two years old, i hope he still has them
[22:01]
Sonican we archive MC servers? I'd love to find beta 1.7_03 server and stuff
bukkit or vanilla, but preferably vanilla
[22:06]
pie__you mean the server program? [22:07]
Sonithe minecraft_server.jar [22:07]
pie__isnt bukkit open source? so you can just get whichever of that [22:07]
Sonibukkit is dead, all the files are gone
no beta 1.7_03
[22:07]
pie__on the other hand iirc you can directly download old mc versions from whatever the name of the company was that does minecraft?
oh shit, really?
[22:07]
SoniI had it. once.
but then my HDD died
[22:07]
pie__damn. [22:07]
Soninot server versions, only client versions [22:08]
pie__oh. that doesnt make sense though
because you need the client and server to have compatible versions?
[22:08]
Soniyou can play singleplayer [22:09]
pie__im just guessing but the server cores should be the same since the client version can also start servers? not that that helps you. [22:09]
Soninot pre-1.3
release 1.3
http://minecraft.gamepedia.com/1.3.1
look for "Made singleplayer internally use a multiplayer server."
before that, client and server were separate beasts
[22:09]
pie__wow is it just me or did crosscode get more expensive? good game tho >.> [22:12]
***SHODAN_UI has quit IRC (Remote host closed the connection) [22:14]
Lord_Nighhttps://www.theregister.co.uk/2017/06/23/windows_10_leak/ <- i forsee a very very short future for betaarchive now :(
do we have a mirror of that?
i mean, not this stuff
but betaarchive in general
[22:14]
JAA"not *just* this stuff" FTFY [22:16]
***Petri152 has joined #archiveteam-bs [22:16]
JAABut yes, we should definitely grab a copy. [22:17]
pie__archiveteam hug of death
"The leaked code is Microsoft's Shared Source Kit: according to people who have seen its contents, it includes the source to the base Windows 10 hardware drivers plus Redmond's PnP code, its USB and Wi-Fi stacks, its storage drivers, and ARM-specific OneCore kernel code." welllllll thats not what id have thought most interesting but what do i know
[22:17]
Lord_Nighhttps://www.theregister.co.uk/2017/06/22/two_men_arrested_probe_microsoft_networks_hack/ might be related [22:18]
JAAThere are a few things on IA, but no "full" mirror as far as I can see, unless it's disguised somehow. [22:19]
Lord_Nighit might be blacked out [22:19]
JAATrue [22:19]
pie__i would be very interested in that leak but sheesh who has 8tb lying around? >.> [22:20]
Lord_Nighi know someone named "kazblox"? (not sure of nick) maybe was involved at betaarchive, so if we can get in contact with him maybe we can arrange a mirror [22:24]
joepie91Lord_Nigh: please do [22:24]
Lord_Nighwow, whoever it was deleted their github user profile completely
kazblox at least
[22:25]
pie__apparently this is a thing:
https://www.reddit.com/r/Steam/comments/611h5e/guide_how_to_download_older_versions_of_a_game_on/?st=j4af9vw3&sh=8e9d5630 https://steamdb.info/app/368340/history/
[22:25]
Lord_Nighwait, WHAT? you can DO THAT? [22:27]
pie__in which case i may ask a favor of someone trying to do that for me for crosscode because i dont have a steam or anything....its a good game, worth the money :PPPP
of course that depends on the person
[22:29]
Lord_Nighwow, and you can even get the versions of the gta games before they had to remove the copyrighted music because licenses expired
i gather this hole will be patched very quickly
[22:29]
pie__lol [22:29]
Lord_Nighif it hasn't already [22:29]
arkiverAnyone here has the windows 10 leaks? [22:32]
Lord_Nighiirc the first version of owlboy at least on gog had a debugger .pdb file in it (its an xna .net game anyway so you can decompile it without that, but that makes it way easier)
not sure if the first steam version had it or not
[22:32]
pie__im not actually sure how to interpret this steamdb stuff
cant tell how many versions of CC it has
[22:32]
Lord_Nighpie__: it lets you download all old versions of a game assuming you have the valid release id for that version
kinda like how with picroma's cube world you can grab old versions off the update server if you have a valid GUID for it
i fed ALL that stuff into archivebot over a year ago though
at least the guids i could find
[22:33]
pie__oh thats cute re cube world
* re:
[22:34]
Lord_Nighassuming valve doesn't patch this this means i can probably delete 400GB+ of 'old versions of steam games' i have archievd locally [22:35]
pie__> assuming <
>> archiveteam <<
>>> delete <<<
:P
[22:35]
***godane has joined #archiveteam-bs [22:36]
Lord_Nighif you want me to upload all that shit somewhere, gimme a server... actually its probably easier to send a harddrive via snail mail
or a usb flash drive or 5
[22:36]
timmctimmc attaches a 2 TB spinning rust drive to a pigeon [22:37]
pie__yeah if i had $$ lol
im just joking, im not equipped for any serious librarian stuff :P
[22:37]
Lord_Nightimmc: that won't work, but if you put it in a basket and tie it to about 3 dozen pigeons... [22:38]
pie__maybe timmc has russian pigeons [22:38]
Lord_Nighproblem is packet loss from pigeon predation [22:38]
FroggingI'd build another NAS just to store that juicy windows leak
maybe a tape drive would be cheaper at that point, not sure
[22:38]
Lord_Nighhttps://steamdb.info/blog/steam-music-download-soundtracks/ is more exploiting of download_depot steam console command
this exploit btw doens't let you download games you don't own
i don't think
it only lets you download old versions with known ids of games
[22:39]
FroggingI wish mainstream software publishers cared more about providing historic versions
s/mainstream/consumer/
[22:40]
Lord_Nighhmm. i wonder if i can get the no man's sky steam prototype, the one from july 2016...
... i would try it, but if theres one thing i definitely do not want to do is get my steam account with like $2000 in games banned...
[22:40]
***odemg has joined #archiveteam-bs
Crusher_ has joined #archiveteam-bs
[22:41]
Crusher_Hope I didn't miss anything good in the last couple hours [22:44]
JAAJust a juicy Microsoft leak.
Which we haven't obtained (yet).
https://www.theregister.co.uk/2017/06/23/windows_10_leak/
[22:44]
Crusher_Assuming we are grabbing an archive of that, you want another connection on that? [22:46]
FroggingIt's limited access, we can't just yet [22:47]
Lord_Nighhttps://steamdb.info/app/275850/ and https://steamdb.info/depot/275851/manifests/ <- the 20170317 version has about 300mb of unknown data that was dropped in the next version
also the earliest versions of starbound on steam have full debugging symbol information compiled in
so theres a ton of "cool stuff" to be found using this download_depot trick
[22:47]
JAAInterestingly enough, the relevant forum post on their forums doesn't list the Shared Source Kit anymore: https://www.betaarchive.com/forum/viewtopic.php?p=422843#p422843 [22:48]
Lord_Nighdamage control
oh, and the VERY FIRST VERSION of skyrim on steam doesn't even have steam integration DRM compiled in AT ALL
its drm free completely by accident
[22:48]
Froggingooooooh [22:49]
pie__Lord_Nigh, please gentleman do your stuff
quietly.
[22:49]
Lord_Nighi'm actually not touching this stuff personally
since i don't want my personal steam account to get banned
just pointing stuff out
[22:50]
pie__oh right. :I
is there a policy against downloading older things?
[22:50]
Crusher_I'm not touching games with a 10 ft pole [22:50]
Lord_Nighi don't think so? [22:50]
pie__soooo is this like "hacking" or something?
why would you get banned for this
[22:51]
Lord_Nighi think steam actually does it automatically if you have updates disabled, have an old version installed and do the 'verify local files' command
it probably downloads the depot
or at least a set of hashes for all the files
[22:51]
pie__Lord_Nigh, any idea what this dev package thing means?: https://steamdb.info/sub/66910/ (from https://steamdb.info/depot/368341/subs/) [22:52]
JAAInteresting discussion thread about that leak: https://www.betaarchive.com/forum/viewtopic.php?f=72&t=37282 [22:52]
Lord_Nighno idea about dev package [22:52]
pie__what i mean to say is i dont really see through the architecture of this at all [22:52]
Lord_Nighpie__: it probably won't install unless you have a "license" to it on your account, it probably won't download the depot [22:53]
pie__no idea how depos and packages and manifests relate
Lord_Nigh, yeah thats what i figured
[22:53]
Lord_Nighi think when you do a download_depot appid depotid manifestid command it checks on valves servers to see if you've "bought" that product first, and if you did then it lets you download it [22:53]
pie__it wouldnt really make sense otherwise [22:53]
Lord_Nighits possible it will download it anyway and then try to decrypt it and give a "license error" [22:54]
pie__is download_depot part of the steam binaries? [22:54]
Lord_Nighyes [22:54]
pie__oh >steam console
i just cant read
[22:54]
Lord_Nighgo to the url field in steam
and you can launch the console from there
[22:54]
Kazlooping back to the MS leak - it's 32TB or so
Kaz pokes odemg
[22:55]
pie__Kaz, didnt the article say its 8tb compresseD? [22:56]
timmcIf you can even confirm it exists. :-) [22:56]
Kazah yeah..
well, I don't even have 8tb free atm :(
[22:56]
JAAKaz: See the links to the Beta Archive forums I posted above. [22:58]
pie__Lord_Nigh, to be fair it doesnt look to me like this method is going to get blocked [22:58]
Lord_Nigher, you can't normally get to the steam url field, but in the system browser open steam://open/console and steam will use that
pie__: i agree
[22:58]
pie__Lord_Nigh, also just noticed i didnt rad half of what you said, might have been bad internet lag
*erad
*read :I
[22:58]
Lord_Nighthis probably won't be blocked, though companies could in theory decide to force-take-down certain manifests [22:59]
Kazlooks disappointing [22:59]
Lord_Nighsteam has a way of revoking manifests, i.e. i think that's what happened here: https://www.windowscentral.com/forza-horizon-3-developer-pc-build-accidentally-released-public [23:00]
pie__now the question is if theres a properly automated download_depot script somewhere that doesnt require screwing around in guis all over the place [23:00]
JAAWell, it's still very interesting. But yeah, not "32 TB" interesting. [23:00]
pie__oh. this is a thing. http://crosscode.gamepedia.com/Steam_depots [23:00]
Lord_Nigher, wait, that was an xbox one leak, not steam related at all [23:01]
pie__yeah this method seems to have been around for at least two years
wonder if theres any way to get those protected manifest IDs :/ but obviously there shouldnt be.
[23:02]
joepie91starting to look like el reg dun been trolled? [23:09]
Lord_Nighyeah, dunno
the no man's sky experimental branch has a known password: "3xperimental" iirc
a bunch of other games have known passwords for beta/other branches
[23:09]
pie__what do you mean?
"this branch requires a password"
whatever that means. (shown in steamdb)
[23:09]
joepie91https://boards.4chan.org/g/thread/61048732#p61049660
cc Lord_Nigh
this looks like a panic baletion?
baleetion*
I'm guessing that it drew too much attention
[23:12]
***bmcginty has quit IRC (Read error: Operation timed out)
bmcginty has joined #archiveteam-bs
[23:15]
Lord_Nighsometimes private beta passwords for steam stuff leak or people figure them out by trial and error: https://steamcommunity.com/app/40800/discussions/0/133261124636030329/ [23:20]
pie__where does it ask you for the password? [23:21]
Lord_Nighright click on super meat boy in your library and go to properties
go to the 'betas' tab
[23:21]
pie__oh.
i am literally just now installing steam (but ive seen it before a bit)
[23:21]
***bmcginty has quit IRC (Ping timeout: 268 seconds)
bmcginty has joined #archiveteam-bs
[23:28]
pie__Lord_Nigh, is it feasible that i write a python script that watches the download dir, copies the stuff out when dl-ing is finished and then i manually enter the next manifest, ad infinitum? :/ [23:30]
Lord_Nighyes
it might be possible to do wit with an autohotkey script too
[23:31]
pie__im on linux [23:31]
Lord_Nighfor moving cursor and selecting fields
ah
[23:31]
pie__this is going to be annoying
well linux has xdotool
man i dont want to code this right now lol
alternatively, this appears to be a thing >.> https://github.com/SteamRE/DepotDownloader
oh except its for windows only :I
[23:32]
***schbirid2 has joined #archiveteam-bs [23:33]
pie__might run under wine... [23:33]
***schbirid has quit IRC (Read error: Operation timed out) [23:35]
pie__inb4 account stolen
(with nothing on it)
and of course i hit a bug https://github.com/SteamRE/DepotDownloader/issues/16
[23:39]
Lord_Nighwant a killing floor key?
humble was giving them out
and it expires july 10, i already own it
[23:42]
Froggingoh hey it's on linux too
too bad I don't really play multiplayer games
[23:42]
Lord_Nighpie__: want it? [23:43]
pie__i probably dont have the specs lmao
pie__ checks YT vid
ok fine i might have specs fro half-life level stuff
[23:44]
***octarine has joined #archiveteam-bs [23:45]
Froggingis it Source Engine? [23:45]
pie__no clue
im sooo bad at FPSs xD
[23:45]
Lord_Nighbetter to have someone use the free key than have it expire by itself in 2 weeks [23:45]
pie__this seems like a mildly terrible game haha [23:46]
***odemg has quit IRC (Read error: Operation timed out) [23:46]
pie__eh, thanks but i think ill pass. [23:50]
Lord_Nighif you didn't use the key, can i give it to someone else? [23:54]
pie__yeah
appreciate the gesture though :P
[23:55]
Lord_NighFrogging: do you want it? [23:56]
FroggingNah, not my kind of game. but thans
thanks
[23:56]
pie__any idea where steam saves this stuff [23:56]
Froggingsomeone in #steamlug on freenode might but most probably have it already :p [23:57]
Lord_Nighon windows its in program files x86/steam/steamapps/etc [23:57]
pie__oh hey steam accepts bitcoin [23:59]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)