#newsgrabber 2017-06-21,Wed

Logs of this channel are not protected. You can protect them by a password.

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)


WhoWhatWhen
***kyan has quit IRC (Remote host closed the connection) [02:25]
........... (idle for 54mn)
jrwrKaz: iM HERE
Whats up
[03:19]
....................... (idle for 1h50mn)
***Aranje has quit IRC (Quit: Three sheets to the wind) [05:10]
Aranje has joined #newsgrabber [05:19]
...... (idle for 27mn)
Aranje has quit IRC (Three sheets to the wind) [05:46]
.............................. (idle for 2h28mn)
jrwr has quit IRC (Read error: Operation timed out) [08:14]
........................... (idle for 2h14mn)
logchfoo2 starts logging #newsgrabber at Wed Jun 21 10:28:19 2017
logchfoo2 has joined #newsgrabber
[10:28]
............................................................. (idle for 5h3mn)
Aoede has joined #newsgrabber [15:31]
AoedeHi. So how do I use thiss, just run worker_script.py? [15:33]
KazAoede: have you run through the install instructions in the readme?
wait hang on, there's no worker_script.py for newsgrabber-warrior
definitely don't run that
[15:35]
AoedeWait, I was looking at wrong git repo
Sorry :P
[15:38]
........... (idle for 50mn)
https://pastebin.com/UZt2Tg6U
this happens for some tasks, others work fine
[16:29]
***crusher__ has joined #newsgrabber [16:37]
crusher__i'm here from the wiki link, where can i get the scraper scripts? [16:38]
never mind, i found the git. where should i point to for the rsync server? [16:47]
Aoedecrusher__: https://github.com/ArchiveTeam/NewsGrabber-Warrior [16:51]
***Hecatz has joined #newsgrabber [16:53]
.... (idle for 17mn)
Kazscripts have changed a fair few times
-warrior is the one you want
[17:10]
JAASounds like the wiki page needs an update then.
And the topic in this channel
[17:12]
***kurt sets mode: +o Kaz
Kaz changes topic to: http://newsgrabber.harrycross.me:29000 | https://github.com/ArchiveTeam/NewsGrabber-Warrior
Kaz changes topic to: https://github.com/ArchiveTeam/NewsGrabber-Warrior // https://github.com/ArchiveTeam/Newsgrabber-Discovery
[17:13]
......... (idle for 40mn)
jrwr has joined #newsgrabber [17:54]
jrwrstupid scaleway restarted one of my instances [17:54]
crusher__i don't suppose there is a way to run the scripts on windows is there xD
Bah, may as well install a distro on that machine
[17:55]
jrwrcrusher__: virtual machine [17:56]
MrRadarcrusher__: They would probably with the Linux compatibilty thing Windows 10 has
But a VM is probably a better choice
[17:57]
crusher__jrwr other than that :P [17:57]
jrwrWindows Linux Subsystem
or try reallly hard in cygwin
[17:58]
crusher__Eh...
This machine won't be doing anything other than archive team for the foreseeable future
[17:58]
jrwrLinux
Ubuntu 16.04 works well
[17:59]
crusher__yeah
i'll probably install ubuntu server and throw lxde on to make it a bit lighter
i don't feel like fighting with slitaz today, so that's as thin as it is going to get xD
[17:59]
.... (idle for 19mn)
JAAIf you're comfortable enough with the command line, why not skip the window system entirely and just run a headless system? It doesn't get much lighter than that.
Then connect from another machine via ssh and run the scripts in tmux or screen.
[18:19]
crusher__i could, and i probably should
but lxde is so bloody light
it hardly matters xD
it also does make my life a bit easier :P
[18:29]
JAASure. I'm so used to the terminal that it'd actually make it harder for me, but to each their own. :-) [18:31]
HCross2https://hastebin.com/nepeyirepa.css M247...
These people are supposed to know what they are doing
[18:40]
MrRadarJAA: The Unix Haters Handbook is still mostly accurage when it comes to X11: "most computers running X run just four programs: xterm, xload, xclock, and a window manager"
You just need to swap "Chromium" for "xclock" (since the window manager includes a clock these days) and it's accurate
[18:43]
.......... (idle for 47mn)
AoedeFinally got it working. Sometimes I really hate python... [19:30]
.... (idle for 19mn)
KazHCross2: routing looks ace [19:49]
***crusher__ is now known as crusher [19:58]
HCross2Kaz: it's literally one side of the road to the other via several European cities. Several times :p [20:01]
crusherlol [20:06]
KazHaha, I hadn't even noticed that
That's actually really impressive
[20:19]
HCross2Kaz: http://bgp.he.net/AS9009#_bogons it gets better [20:22]
KazHahaha what
Why
Wtf are they doing
[20:23]
HCross2They're also announcing rfc1918 space to the public internet [20:23]
crushererr...
doesn't that break
a lot?
[20:24]
KazNobody will accept the routes
Nobody (hopefully)
[20:25]
AoedeO.o [20:25]
HCross2At somepoint someone will.. and we'll be in a world of hurt [20:25]
crusherthey'd basically get depeered on that then? [20:25]
HCross2Or rather.. one of M247s core will [20:25]
crusherforgive me if i'm using the wrong terminology, i'm fascinated by networking, but by no means an expert.
so assuming someone accepts their route, what happens?
[20:26]
HCross2Everyone who is either connected to that person who accepts directly.. or also hasn't configured their stuff right will send that data to M24Seven.. who won't probably have the capacity.. and will promptly fall ovet
Over
And then I'll ask where half of my infrastructure has just gone
[20:28]
crusherall the public schools in my province share one giant 10.x.x.x network...
(internal)
[20:28]
KazSame in my area.. Sadly [20:29]
HCross2Kaz: it's a mess over here :p
We've got NAT behind NAT behind NAT in some cases
[20:30]
crusheryeesh
how do you do anything?
[20:30]
MrRadarIPv6 can't happen soon enough
(I'm count myself as lucky that I have it at home)
[20:31]
HCross2crusher: I wait for a couple of months.. and the broadband supplier looses their contract
MrRadar: one of the biggest UK ISPs still doesn't do it
[20:31]
crusherDoesn't look like ipv6 is going to happen any time soon at least where i am
My ISP is Bell Canada, I have an external address and i'm lucky enough they don't block or filter any ports
[20:31]
HCross2Eh. We've got government mandated filters here [20:34]
crusherwhich is odd compared to the flip side of the di-oploly here in canada [20:34]
HCross2Heck.. my phone provider blocks archive.org
I've had a row with them before
[20:34]
MrRadarO_o
WHY???
And who?
[20:34]
HCross2Three UK [20:35]
crusherRogers is the other big one here, they don't block sites, but they do block mail ports, filter others and they have gotten in huge amounts of trouble for picking on p2p traffic [20:35]
HCross2ive not done age ID with them as prepay is fine for me https://usercontent.irccloud-cdn.com/file/2trI9bao/Screenshot_20170621-213528.png [20:35]
crusherwtf? [20:36]
MrRadarOK, that makes a bit more sense [20:36]
crusherthat's insane to me [20:36]
MrRadarMy phone provider, T-Mobile US has a similar "feature" but you don't need to provide ID or anything to turn it off [20:37]
crushercensorship even for mature content reasons should be an opt-in.
never an opt-out unless *maybe* the product is specifically for youth
[20:37]
HCross2What's better is that they need a credit card to validate.. I don't need to borrow so I don't have one [20:38]
crusheryour banked might offer a prepaid linked to your debit
bank*
[20:39]
HCross2Eh. I've just tried with my Monzo MasterCard.. let's see if that fools them [20:40]
MrRadarEh, having a 0 annual fee credit card is a good idea even if you never plan to carry a balance. They're useful for travel (car rental and hotel deposits) and shopping at shadier online shops [20:40]
crusherI've got a "mastercard" that i can instantly dump funds onto directly from my bank account [20:40]
HCross2crusher: government makes all filtering opt out [20:40]
crusherEw [20:40]
HCross2crusher: that's what Monzo is for me [20:41]
crusherHCross2: At least it is opt out
HCross2: i guess thats better than nothing...
[20:41]
HCross2For now.. who knows that the new government may do [20:41]
MrRadarYeah. Ill-considered is probably the nicest way to describe the policy proposals I've seen floated by them
Regarding the Internet and encryption
[20:43]
crusherCan you spot the scraper? : http://imgur.com/a/1u0vi
xD
[20:44]
...... (idle for 28mn)
***_Crusher_ has joined #newsgrabber
crusher has quit IRC (Quit: Page closed)
_Crusher_ is now known as crusher
[21:12]
......... (idle for 40mn)
crusher has quit IRC (Ping timeout: 492 seconds) [21:52]
Crusher has joined #newsgrabber [21:59]
Kazhmm, me and underscor both put in a 16618mb item very recently (within 2 items of eachother)
coincidence, or lists being reused?
[22:00]
***Crusher_ has joined #newsgrabber
Crusher has quit IRC (Read error: Connection reset by peer)
[22:09]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)