#archiveteam-ot 2020-01-26,Sun

↑back Search

Time Nickname Message
00:09 🔗 Frogging I doubt the type of video matters for rate limiting
00:09 🔗 Raccoon it's definitely not rate limiting. ie, one video every 20-40 minutes
00:10 🔗 Raccoon do you get a captcha for https://www.youtube.com/watch?v=2zEIQuRg-m4
00:33 🔗 Frogging no
00:48 🔗 OrIdow6 I'm on a VPN and Youtube-dl works fine on there for me
00:49 🔗 OrIdow6 Maybe you're using the same IP address as someone who's been mass-downloading or something
00:53 🔗 Raccoon maybe
01:21 🔗 thuban3 has joined #archiveteam-ot
01:24 🔗 thuban2 has quit IRC (Read error: Operation timed out)
01:33 🔗 qw3rty_ has quit IRC (Read error: Connection reset by peer)
01:34 🔗 qw3rty_ has joined #archiveteam-ot
02:38 🔗 godane has joined #archiveteam-ot
03:07 🔗 kiska has quit IRC (Remote host closed the connection)
03:07 🔗 Flashfire has quit IRC (Remote host closed the connection)
03:08 🔗 kiska has joined #archiveteam-ot
03:08 🔗 Flashfire has joined #archiveteam-ot
03:08 🔗 svchfoo3 sets mode: +o kiska
03:08 🔗 svchfoo1 sets mode: +o kiska
04:13 🔗 DogsRNice has quit IRC (Read error: Connection reset by peer)
04:22 🔗 atphoenix I don't ever remember seeing YT captcha me
04:23 🔗 atphoenix Google search has, though
04:23 🔗 atphoenix I wonder how ivan deals with it.
04:27 🔗 qw3rty__ has joined #archiveteam-ot
04:31 🔗 qw3rty_ has quit IRC (Ping timeout: 276 seconds)
04:44 🔗 marked1 use clean IPs would be the simplest
04:47 🔗 duh has quit IRC (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.)
04:48 🔗 legoktm has joined #archiveteam-ot
04:57 🔗 cerca has quit IRC (Remote host closed the connection)
06:26 🔗 britmob has quit IRC (Read error: Connection reset by peer)
06:28 🔗 britmob has joined #archiveteam-ot
07:05 🔗 dhyan_nat has joined #archiveteam-ot
07:43 🔗 ShellyRol has quit IRC (Read error: Connection reset by peer)
07:45 🔗 ShellyRol has joined #archiveteam-ot
07:45 🔗 qw3rty_ has joined #archiveteam-ot
07:46 🔗 lunik1 has quit IRC (Read error: Connection reset by peer)
07:46 🔗 qw3rty has joined #archiveteam-ot
07:47 🔗 Flashfire has quit IRC (Read error: Connection reset by peer)
07:47 🔗 Dallas has quit IRC (Read error: Connection reset by peer)
07:47 🔗 marked1 has quit IRC (Read error: Connection reset by peer)
07:47 🔗 benjinsmi has joined #archiveteam-ot
07:48 🔗 qw3rty__ has quit IRC (Ping timeout: 276 seconds)
07:48 🔗 OrIdow6 has quit IRC (Ping timeout: 276 seconds)
07:48 🔗 atphoenix has quit IRC (Ping timeout: 276 seconds)
07:48 🔗 benjins has quit IRC (Ping timeout: 276 seconds)
07:49 🔗 atphoenix has joined #archiveteam-ot
07:50 🔗 qw3rty_ has quit IRC (Ping timeout: 276 seconds)
07:50 🔗 marked1 has joined #archiveteam-ot
07:52 🔗 marked10 has joined #archiveteam-ot
07:59 🔗 marked1 has quit IRC (Ping timeout: 492 seconds)
08:00 🔗 marked101 has joined #archiveteam-ot
08:07 🔗 marked10 has quit IRC (Ping timeout: 492 seconds)
08:07 🔗 marked101 is now known as marked10
08:08 🔗 marked108 has joined #archiveteam-ot
08:09 🔗 marked108 has quit IRC (Client Quit)
08:10 🔗 marked104 has joined #archiveteam-ot
08:11 🔗 marked104 has quit IRC (Client Quit)
08:12 🔗 marked105 has joined #archiveteam-ot
08:13 🔗 marked105 has quit IRC (Client Quit)
08:14 🔗 marked103 has joined #archiveteam-ot
08:15 🔗 marked103 has quit IRC (Client Quit)
08:16 🔗 marked10 has quit IRC (Ping timeout: 492 seconds)
08:33 🔗 OrIdow6 has joined #archiveteam-ot
09:03 🔗 OrIdow6 has quit IRC (Ping timeout: 276 seconds)
09:35 🔗 OrIdow6 has joined #archiveteam-ot
11:09 🔗 OrIdow6 has quit IRC (Ping timeout: 276 seconds)
11:29 🔗 BlueMax has quit IRC (Read error: Connection reset by peer)
11:41 🔗 OrIdow6 has joined #archiveteam-ot
12:28 🔗 LowLevelM has joined #archiveteam-ot
12:49 🔗 X-Scale` has joined #archiveteam-ot
12:56 🔗 X-Scale has quit IRC (Ping timeout: 610 seconds)
12:56 🔗 X-Scale` is now known as X-Scale
13:01 🔗 X-Scale` has joined #archiveteam-ot
13:07 🔗 X-Scale has quit IRC (Ping timeout: 610 seconds)
13:07 🔗 X-Scale` is now known as X-Scale
13:48 🔗 mystonlin has joined #archiveteam-ot
13:52 🔗 mystonlin Thanks very much @nyany, it would be great if you could replace the citation links at the bottom of the page which currently both point to example.com for both, with the relevant forum posts: https://mystonline.com/forums/viewtopic.php?f=1&t=29240 and https://forums.cyan.com/viewtopic.php?f=7&t=1608 Thanks again!
13:54 🔗 mystonlin has quit IRC (Remote host closed the connection)
15:32 🔗 igloo25 has joined #archiveteam-ot
15:50 🔗 nyany JAA: That was exactly it. Seems to work fine now.
15:50 🔗 nyany in RE: the reference links
15:51 🔗 Raccoon atphoenix: btw, the youtube captcha is exactly the same as google search captcha, even the google domain
16:33 🔗 britmob has quit IRC (Read error: Connection reset by peer)
16:33 🔗 marked10 has joined #archiveteam-ot
16:35 🔗 britmob has joined #archiveteam-ot
16:59 🔗 MaximeleG has joined #archiveteam-ot
17:05 🔗 MaximeleG has quit IRC (Quit: MaximeleG)
17:07 🔗 MaximeleG has joined #archiveteam-ot
17:07 🔗 thuban3 has quit IRC (Read error: Connection reset by peer)
17:09 🔗 thuban3 has joined #archiveteam-ot
17:22 🔗 DogsRNice has joined #archiveteam-ot
17:24 🔗 MaximeleG has quit IRC (Quit: MaximeleG)
17:30 🔗 godane1 has joined #archiveteam-ot
17:31 🔗 godane has quit IRC (Read error: Connection reset by peer)
17:35 🔗 thuban4 has joined #archiveteam-ot
17:41 🔗 thuban3 has quit IRC (Read error: Operation timed out)
17:50 🔗 Atom__ has quit IRC (Quit: Atom__)
17:54 🔗 systwi Is anyone here familiar with perl?
17:55 🔗 systwi I wanted to try out https://metacpan.org/pod/release/BSUGARS/IPC-Shareable-0.60/lib/IPC/Shareable.pm , it looks like it allows you to store a variable and reference it within any shell process
17:56 🔗 systwi I'd love to use that in a bash script I'm working on, but I have absolutely no clue on how to use it. I've read through the page but perl is so foreign to me I don't know what I'm looking at
17:58 🔗 thuban has joined #archiveteam-ot
18:01 🔗 thuban4 has quit IRC (Ping timeout: 276 seconds)
18:29 🔗 marked108 has joined #archiveteam-ot
18:29 🔗 asdf01012 has joined #archiveteam-ot
18:30 🔗 marked10 has quit IRC (Read error: Operation timed out)
18:30 🔗 marked108 is now known as marked10
18:32 🔗 asdf0101 has quit IRC (Read error: Operation timed out)
18:32 🔗 asdf01012 is now known as asdf0101
18:51 🔗 marked10 systwi : I know perl. I'm not sure you want to use it from bash. Try to at least find a bash native way first.
18:51 🔗 marked10 is now known as marked1
18:56 🔗 JAA I sense an XY problem.
19:08 🔗 superkuh has quit IRC (Quit: the neuronal action potential is an electrical manipulation of reversible abrupt phase changes in the lipid bilaye)
19:40 🔗 qw3rty has quit IRC (Read error: Operation timed out)
19:59 🔗 atphoenix #archiveteam needs ODAH!!!!
20:01 🔗 atphoenix from hackint at -ot: The Best Ever Put-Downs From House Speaker John Berco https://www.youtube.com/watch?v=kAM-YW-6vdU <-- ODAH! ODAH! (Order! Order!) keeping #archiveteam on topic
20:02 🔗 superkuh has joined #archiveteam-ot
20:08 🔗 atphoenix maybe we could have a talking JohnBercoBot (JBB) in #archiveteam. Yells ODAH! after (some criteria). Step 1 is clear (create bot). Step 2 (the criteria) is nebulous. Step 3 Profit. (southpark gnomes references)
20:09 🔗 marked1 true. systwi: perl is rarely the best choice. what was the original task you're trying to get done, what did you try, and where did you get stuck?
20:19 🔗 systwi marked1: Thanks for the input. In my bash script I want it to make sure it's the only instance running, and if a second instance is running, wait until the first one is finished. I read online a lot of people suggest using temporary files, however there is the chance that a sudden power outage could shut down the computer improperly. Then, when it's booted back up the temporary files are still there. A new run of the sc
20:19 🔗 systwi ript may see the temp file still there and think another iteration is running, when it's not
20:20 🔗 thuban systwi: the usual method in this case is to apply `pgrep`
20:20 🔗 systwi Yes, but this is a bash script. As far as I know the process will only be listed as "bash"
20:21 🔗 systwi Which could easily get confused if I have other Terminal windows open doing other things, which is very common
20:21 🔗 SootBectr Try flock https://www.systutorials.com/docs/linux/man/1-flock/
20:22 🔗 SootBectr sorry that site is full of 3rd party js, it's just the man page for flock which you likely have installed already though
20:22 🔗 systwi Oh yeah I installed flock a long time ago (I'm using macOS so I had to use brew)
20:23 🔗 systwi I'm reading the man page right now
20:23 🔗 marked1 what is the task you were trying to solve though, bash might not be a good language for it
20:23 🔗 qw3rty has joined #archiveteam-ot
20:23 🔗 systwi The only thing is, though, that still means I have to rely on files and not something in memory
20:23 🔗 systwi like a variable
20:24 🔗 systwi I am writing a tool to archive everything for a specified YT channel. That's the brief summary, I have been working on this tool for about 7 months now, I reeeally prefer not to switch languages at this point lol
20:25 🔗 systwi Everything has gone perfect so far
20:26 🔗 dhyan_nat has quit IRC (Read error: Operation timed out)
20:26 🔗 jodizzle has quit IRC (Read error: Operation timed out)
20:26 🔗 Frogging has quit IRC (Read error: Operation timed out)
20:26 🔗 Frogging has joined #archiveteam-ot
20:26 🔗 girst has quit IRC (Read error: Operation timed out)
20:26 🔗 jodizzle has joined #archiveteam-ot
20:26 🔗 dxrt_ has quit IRC (Read error: Operation timed out)
20:26 🔗 Craigle has quit IRC (Read error: Operation timed out)
20:27 🔗 systwiALT has joined #archiveteam-ot
20:27 🔗 systwiALT Sorry about that, connection timeout, hope I didn't miss anything
20:27 🔗 paul2520 has quit IRC (Read error: Operation timed out)
20:28 🔗 Auctus has quit IRC (Read error: Operation timed out)
20:28 🔗 Auctus has joined #archiveteam-ot
20:28 🔗 Raccoon` has joined #archiveteam-ot
20:28 🔗 asdf0101 has quit IRC (Read error: Operation timed out)
20:28 🔗 SootBectr put the lockfile in /tmp and it should achieve what you specify
20:29 🔗 LowLevelM has quit IRC (Read error: Operation timed out)
20:29 🔗 atphoeni1 has joined #archiveteam-ot
20:29 🔗 luckcolor has quit IRC (Read error: Operation timed out)
20:29 🔗 systwiALT That's definitely a good idea, however what about the off chance another file with the same name already exists?
20:29 🔗 keith20 has quit IRC (Read error: Operation timed out)
20:29 🔗 luckcolor has joined #archiveteam-ot
20:30 🔗 systwiALT That's why I thought having something in memory would prevent any issues
20:30 🔗 girst has joined #archiveteam-ot
20:30 🔗 systwiALT There's also the possibility of the user having a full disk
20:30 🔗 systwiALT I'm just trying to cover all bases
20:30 🔗 atphoenix has quit IRC (Read error: Operation timed out)
20:30 🔗 atphoeni1 is now known as atphoenix
20:31 🔗 mr_archiv @systwi to solve your temporary problem you can put the files in a RAM-disk (/dev/shm or /tmp depending on configuration). This will be lost upon power-off solving your problem.
20:31 🔗 nicolas17 if the disk is full you're not gonna download any YouTube videos anyway
20:31 🔗 marked1 trying to invent something nobody has thought of before is going to cause more bugs than following what's considered normal practice
20:31 🔗 mr_archiv SootBectr beat me to it. I was scrolled up.
20:32 🔗 Raccoon has quit IRC (Read error: Operation timed out)
20:32 🔗 Raccoon` is now known as Raccoon
20:32 🔗 systwiALT The idea of a RAM disk is a good idea, idk if macOS's /tmp folder _is_ a ram disk in disguise. I can't reboot to check, too much stuff open atm lol.
20:32 🔗 mr_archiv Also to ensure a unique file name first create a folder for your program and append a unix timestamp to the end of the filename.
20:33 🔗 systwiALT nicolas17: True, but the user could download it to a drive with free space
20:33 🔗 Tenebrae has quit IRC (Ping timeout: 864 seconds)
20:33 🔗 systwiALT I was going to use a uuid
20:33 🔗 systwi has quit IRC (Read error: Operation timed out)
20:33 🔗 systwiALT But I can also append a unix timestamp to that :)
20:33 🔗 mr_archiv Also have you tried youtube-dl? It can download an entire channel out of the box?
20:33 🔗 thuban systwiALT: if you really want to avoid lockfiles, you can invoke script.sh as /bin/bash script.sh (instead of ./script.sh) and use `pgrep -f`. but this will have a similar problem in that a the process name could theoretically exist; at some point you have to trust your system.
20:34 🔗 systwiALT mr_archiv: Yes, youtube-dl is a critical component, but I am logging the history of channels as well. One grab in 2016 could be identical to a grab in 2019, but the user could have changed their channel banner. My script will track all of that
20:34 🔗 thuban s/a the/a similar/
20:35 🔗 mr_archiv Got it
20:35 🔗 atphoenix I also lost my EFnet connection for 5 minutes. For others thinking they have gaps, see https://archive.fart.website/bin/irclogger_log/archiveteam-ot?date=2020-01-26,Sun
20:35 🔗 SootBectr To ensure a unique filename, use mktemp
20:36 🔗 systwiALT Thanks for the tips everyone
20:36 🔗 systwiALT Didn't know about mktemp too, nice
20:36 🔗 kiska has quit IRC (Ping timeout: 622 seconds)
20:36 🔗 systwiALT I guess to be absolutely safe, I can; 1, check the disk space is > enough to make a temp file
20:37 🔗 systwiALT 2, create the temp file with mktemp and flock it
20:37 🔗 systwiALT Then any new script iterations will be on hold until the file is freed and removed
20:38 🔗 systwiALT Now, regarding perl, I can't remember exactly what it was in my script, but I did use it (something like `perl -c "code here"`) and it worked perfectly
20:38 🔗 Wingy has quit IRC (Read error: Operation timed out)
20:38 🔗 systwiALT That's why I thought maybe doing something similar with a non-tempfile-reliant solution would work
20:39 🔗 thuban er, isn't the point of mktemp the randomization, i.e. that successive invocations will _not_ clobber each other?
20:40 🔗 nicolas17 each mktemp will return a different name, yes
20:40 🔗 MrRadar2 has quit IRC (Read error: Operation timed out)
20:40 🔗 systwiALT I just also think of the possibility of the user unable to write to /tmp (would probably cause WAY more issues though)
20:41 🔗 systwiALT It's probably me just being too paranoid
20:49 🔗 astrid atphoenix: no bots that talk in #archiveteam
20:50 🔗 nicolas17 where did a bot talk?
20:50 🔗 atphoenix no bot talked. I was trying to make a humorous comment.
21:00 🔗 godane1 has quit IRC (Ping timeout: 255 seconds)
21:06 🔗 Tenebrae has joined #archiveteam-ot
21:07 🔗 dhyan_nat has joined #archiveteam-ot
21:07 🔗 asdf0101 has joined #archiveteam-ot
21:07 🔗 LowLevelM has joined #archiveteam-ot
21:07 🔗 dxrt_ has joined #archiveteam-ot
21:08 🔗 keith20 has joined #archiveteam-ot
21:08 🔗 MrRadar2 has joined #archiveteam-ot
21:08 🔗 svchfoo3 sets mode: +o dxrt_
21:08 🔗 svchfoo1 sets mode: +o dxrt_
21:09 🔗 Wingy has joined #archiveteam-ot
21:14 🔗 godane has joined #archiveteam-ot
21:15 🔗 atphoenix has quit IRC (Read error: Connection reset by peer)
21:16 🔗 Frogging has quit IRC (Read error: Connection reset by peer)
21:16 🔗 atphoenix has joined #archiveteam-ot
21:17 🔗 thuban has quit IRC (Ping timeout: 276 seconds)
21:17 🔗 paul2520 has joined #archiveteam-ot
21:17 🔗 thuban has joined #archiveteam-ot
21:18 🔗 Frogging has joined #archiveteam-ot
21:18 🔗 qw3rty_ has joined #archiveteam-ot
21:18 🔗 qw3rty has quit IRC (Ping timeout: 276 seconds)
21:23 🔗 Raccoon has quit IRC (Ping timeout: 276 seconds)
21:24 🔗 Raccoon has joined #archiveteam-ot
21:24 🔗 OrIdow6 has quit IRC (Ping timeout: 276 seconds)
21:26 🔗 qw3rty_ has quit IRC (Read error: Connection reset by peer)
21:27 🔗 qw3rty has joined #archiveteam-ot
21:39 🔗 OrIdow6 has joined #archiveteam-ot
21:46 🔗 thuban1 has joined #archiveteam-ot
21:47 🔗 Ctrl has joined #archiveteam-ot
21:51 🔗 thuban has quit IRC (Read error: Operation timed out)
21:55 🔗 kiska has joined #archiveteam-ot
21:56 🔗 svchfoo3 sets mode: +o kiska
21:56 🔗 svchfoo1 sets mode: +o kiska
21:59 🔗 BlueMax has joined #archiveteam-ot
22:06 🔗 Craigle has joined #archiveteam-ot
22:07 🔗 dhyan_nat has quit IRC (Read error: Operation timed out)
22:17 🔗 NickN00b has joined #archiveteam-ot
22:50 🔗 antomati_ has joined #archiveteam-ot
22:50 🔗 qw3rty_ has joined #archiveteam-ot
22:51 🔗 ranma_ has joined #archiveteam-ot
22:53 🔗 benjins has joined #archiveteam-ot
22:54 🔗 yawkat` has joined #archiveteam-ot
22:54 🔗 Fionera_ has joined #archiveteam-ot
22:54 🔗 sknebel_ has joined #archiveteam-ot
22:55 🔗 qw3rty has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 britmob has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 X-Scale has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 benjinsmi has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 ShellyRol has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 dashcloud has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 ranma has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 sknebel has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 Kenshin has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 Polylith has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 Fionera has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 yawkat has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 chfoo has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 betamax has quit IRC (irc.efnet.nl efnet.deic.eu)
22:55 🔗 antomatic has quit IRC (irc.efnet.nl efnet.deic.eu)
22:56 🔗 betamax_ has joined #archiveteam-ot
22:56 🔗 chfoo_ has joined #archiveteam-ot
22:56 🔗 britmob_ has joined #archiveteam-ot
22:58 🔗 RKenshin has joined #archiveteam-ot
23:05 🔗 X-Scale` has joined #archiveteam-ot
23:07 🔗 Polylith_ has joined #archiveteam-ot
23:08 🔗 thuban1 has quit IRC (Read error: Connection reset by peer)
23:09 🔗 thuban1 has joined #archiveteam-ot
23:10 🔗 dashcloud has joined #archiveteam-ot
23:10 🔗 RKenshin is now known as Kenshin
23:12 🔗 ShellyRol has joined #archiveteam-ot
23:47 🔗 chfoo_ is now known as chfoo

irclogger-viewer