| Time |
Nickname |
Message |
|
00:36
🔗
|
|
caff has quit IRC (Read error: Operation timed out) |
|
00:39
🔗
|
|
BlueMax has joined #archiveteam-ot |
|
01:41
🔗
|
w0rmhole |
anyone heard of brightcove? |
|
01:41
🔗
|
w0rmhole |
total piece of shit imo |
|
01:41
🔗
|
w0rmhole |
pain in the ass to get a video url |
|
02:05
🔗
|
|
Mateon1 has quit IRC (Read error: Operation timed out) |
|
02:05
🔗
|
|
Mateon1 has joined #archiveteam-ot |
|
03:12
🔗
|
|
odemg has quit IRC (Ping timeout: 260 seconds) |
|
05:27
🔗
|
|
Kaz has quit IRC (Read error: Connection reset by peer) |
|
05:27
🔗
|
|
Kaz has joined #archiveteam-ot |
|
05:41
🔗
|
ivan |
w0rmhole: youtube-dl can often handle those things but yeah I use the devtools when that doesn't work |
|
05:50
🔗
|
w0rmhole |
ivan: devtools? |
|
05:57
🔗
|
mal |
browser developer tools. hit f12 |
|
05:57
🔗
|
mal |
network tab, filter to media (usually) |
|
05:58
🔗
|
w0rmhole |
oh those |
|
05:58
🔗
|
w0rmhole |
yeah i tried those too |
|
05:58
🔗
|
w0rmhole |
couldnt grab the url |
|
07:39
🔗
|
w0rmhole |
anyone here use zfs? |
|
07:39
🔗
|
w0rmhole |
i was wondering, can i enable deduplication while i've already got data on my zpool? |
|
07:39
🔗
|
w0rmhole |
data which i particularly dont want to lose (yes i have backups but still) |
|
08:14
🔗
|
ivan |
isn't zfs dedup very memory-intensive |
|
08:16
🔗
|
ivan |
if you really hate hard links and need reflinks they're available in xfs (newest format) and btrfs |
|
08:17
🔗
|
ivan |
then you don't need an incredibly roundabout dedup scheme |
|
08:18
🔗
|
w0rmhole |
yeah dedup is, and i just looked it up a while ago and it sounds like 24GB isnt enough ram x_x |
|
08:18
🔗
|
w0rmhole |
i dont mind sticking to hardlinks |
|
09:02
🔗
|
|
chferfa has quit IRC () |
|
09:30
🔗
|
|
caff has joined #archiveteam-ot |
|
09:54
🔗
|
|
caff has quit IRC (Quit: smell ya later) |
|
09:54
🔗
|
|
ivan has quit IRC (Read error: Operation timed out) |
|
09:54
🔗
|
|
jspiros has quit IRC (Read error: Operation timed out) |
|
09:55
🔗
|
|
ivan has joined #archiveteam-ot |
|
09:55
🔗
|
|
svchfoo3 sets mode: +o ivan |
|
09:55
🔗
|
dxrt |
Does the python internetarchive package spit anything out if I upload the same file? |
|
09:56
🔗
|
dxrt |
I dont want duplicates |
|
09:56
🔗
|
dxrt |
within the same identifier |
|
09:57
🔗
|
|
bithippo has quit IRC (Ping timeout: 246 seconds) |
|
09:58
🔗
|
|
JAA has quit IRC (Ping timeout: 246 seconds) |
|
09:58
🔗
|
|
chfoo has quit IRC (Ping timeout: 246 seconds) |
|
09:59
🔗
|
Flashfire |
I’m not sure I would hope so |
|
09:59
🔗
|
dxrt |
I can just upload the same file again and again without it complaining |
|
09:59
🔗
|
|
wp494 has quit IRC (Ping timeout: 492 seconds) |
|
09:59
🔗
|
dxrt |
i'm not sure if it's overwriting it.. or what.. |
|
10:01
🔗
|
|
wp494 has joined #archiveteam-ot |
|
10:04
🔗
|
|
chfoo has joined #archiveteam-ot |
|
10:05
🔗
|
|
svchfoo1 sets mode: +o chfoo |
|
10:21
🔗
|
|
wp494 has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:21
🔗
|
|
jut has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:21
🔗
|
|
zino has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:21
🔗
|
|
arkiver has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:21
🔗
|
|
faolingfa has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:21
🔗
|
|
astrid has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:21
🔗
|
|
MrRadar has quit IRC (hub.efnet.us irc.servercentral.net) |
|
10:29
🔗
|
bakJAA |
dxrt: "Note that ia upload makes a backup of any files that are clobbered. They are saved to a directory in the item named history/files/. The files are named in the format $key.~N~. These files can be deleted like normal files. You can also prevent the backup from happening on clobbers by adding -H x-archive-keep-old-version:0 to your command." |
|
10:29
🔗
|
bakJAA |
https://internetarchive.readthedocs.io/en/latest/cli.html#upload |
|
10:35
🔗
|
dxrt |
So if understand correctly, it will keep one version of the file and send the rest to a history directory? |
|
10:35
🔗
|
dxrt |
Is clobbered a technical word, I'm a bit confused by its meaning in this.. |
|
10:38
🔗
|
bakJAA |
"Clobbering" just means "overwriting an existing file". |
|
10:38
🔗
|
bakJAA |
And I'd think it keeps all versions. |
|
10:43
🔗
|
dxrt |
Keeps multiple files with the same name in the same directory? hmm? |
|
10:47
🔗
|
bakJAA |
Nope, but I think the N in "$key.~N~" is an integer which is incremented for each version. I might be wrong though. Did you check the history/files directory on your item? |
|
10:57
🔗
|
|
JAA has joined #archiveteam-ot |
|
10:57
🔗
|
|
svchfoo3 sets mode: +o JAA |
|
10:59
🔗
|
|
jspiros has joined #archiveteam-ot |
|
11:00
🔗
|
dxrt |
Just waiting for this derive to finish. |
|
11:02
🔗
|
|
bakJAA sets mode: +o JAA |
|
11:15
🔗
|
dxrt |
Ok well it just keeps one file in the main directory and increments the rest under /history/files/file.txt.~1~ ~2~ etc. |
|
11:15
🔗
|
dxrt |
That's good. Thanks. |
|
11:20
🔗
|
JAA |
:-) |
|
11:24
🔗
|
JAA |
w0rmhole: The rule of thumb is 5 GB of RAM per TB of storage for ZFS dedupe. And that's only the dedupe table itself... |
|
11:26
🔗
|
JAA |
Here's a good article on ZFS dedupe: https://constantin.glez.de/2011/07/27/zfs-to-dedupe-or-not-dedupe/ |
|
12:06
🔗
|
|
astrid has joined #archiveteam-ot |
|
12:06
🔗
|
|
MrRadar has joined #archiveteam-ot |
|
12:06
🔗
|
|
wp494 has joined #archiveteam-ot |
|
12:06
🔗
|
|
jut has joined #archiveteam-ot |
|
12:06
🔗
|
|
zino has joined #archiveteam-ot |
|
12:06
🔗
|
|
arkiver has joined #archiveteam-ot |
|
12:06
🔗
|
|
faolingfa has joined #archiveteam-ot |
|
12:06
🔗
|
|
irc.servercentral.net sets mode: +o zino |
|
12:24
🔗
|
|
JAA has quit IRC (Ping timeout: 246 seconds) |
|
12:27
🔗
|
|
jspiros has quit IRC (Ping timeout: 492 seconds) |
|
12:55
🔗
|
|
BlueMax has quit IRC (Ping timeout: 633 seconds) |
|
13:22
🔗
|
|
jspiros has joined #archiveteam-ot |
|
13:23
🔗
|
|
JAA has joined #archiveteam-ot |
|
13:23
🔗
|
|
svchfoo1 sets mode: +o JAA |
|
13:24
🔗
|
|
bakJAA sets mode: +o JAA |
|
13:26
🔗
|
|
chferfa has joined #archiveteam-ot |
|
13:40
🔗
|
VoynichCr |
so is JAA going to release the twitter scraper? cooool |
|
13:41
🔗
|
VoynichCr |
now we only need a seconday bot which takes a !twitter command and generates the list of tweets, and write !ao < twitter-list on channel |
|
13:50
🔗
|
JAA |
VoynichCr: The ideal solution would be to integrate it into ArchiveBot directly. |
|
13:53
🔗
|
VoynichCr |
sure |
|
13:54
🔗
|
VoynichCr |
there was a channel to upload videos from tweets, what was it? does it work? |
|
13:54
🔗
|
JAA |
#videobot, but it's broken. |
|
13:58
🔗
|
VoynichCr |
man, there are many twitter accounts i want to archive, though the 3000 tweets limit is a pitty |
|
14:16
🔗
|
|
wp494 has quit IRC (Read error: Operation timed out) |
|
14:17
🔗
|
|
wp494 has joined #archiveteam-ot |
|
15:45
🔗
|
|
odemg has joined #archiveteam-ot |
|
16:45
🔗
|
VoynichCr |
this article deserves a reading https://www.wired.com/story/how-to-design-beacons-for-humanitys-afterlife/ |
|
17:40
🔗
|
|
odemg has quit IRC (Ping timeout: 260 seconds) |
|
17:52
🔗
|
|
odemg has joined #archiveteam-ot |
|
18:18
🔗
|
|
betamax_ is now known as betamax |
|
18:44
🔗
|
|
caff has joined #archiveteam-ot |
|
19:41
🔗
|
|
odemg has quit IRC (Ping timeout: 260 seconds) |
|
19:53
🔗
|
|
odemg has joined #archiveteam-ot |
|
20:51
🔗
|
|
icedice has joined #archiveteam-ot |
|
21:11
🔗
|
ivan |
https://twitter.com/pwnsdx/status/1038821975089664001 nice Chrome DoS |
|
21:16
🔗
|
ivan |
VoynichCr: https://twitter.com/search?q=from%3AUSER&src=typd will return everything |
|
21:19
🔗
|
JAA |
I like the i<1/0. |
|
21:48
🔗
|
|
odemg has quit IRC (Ping timeout: 260 seconds) |
|
22:00
🔗
|
|
odemg has joined #archiveteam-ot |
|
22:19
🔗
|
|
sep332 has quit IRC (Ping timeout: 600 seconds) |
|
22:26
🔗
|
|
n00b641 has joined #archiveteam-ot |
|
22:51
🔗
|
|
sep332 has joined #archiveteam-ot |
|
23:25
🔗
|
|
n00b641 has quit IRC (Quit: Page closed) |
|
23:41
🔗
|
|
caff has quit IRC (Read error: Operation timed out) |
|
23:48
🔗
|
|
odemg has quit IRC (Ping timeout: 260 seconds) |