[00:29] *** matthusby has quit IRC (Remote host closed the connection) [00:34] *** matthusby has joined #archiveteam [00:41] *** matthusby has quit IRC (Remote host closed the connection) [00:41] *** matthusby has joined #archiveteam [00:46] *** Sk1d has quit IRC (Read error: Operation timed out) [00:48] *** satoshi has quit IRC (Read error: Connection reset by peer) [00:49] *** Sk1d has joined #archiveteam [00:49] *** matthusby has quit IRC (Remote host closed the connection) [00:51] *** satoshi has joined #archiveteam [00:56] *** matthusby has joined #archiveteam [00:58] *** Sk1d has quit IRC (Read error: Operation timed out) [01:01] *** Sk1d has joined #archiveteam [01:07] *** matthusby has quit IRC (Remote host closed the connection) [01:09] *** Sk1d has quit IRC (Read error: Operation timed out) [01:11] *** Sk1d has joined #archiveteam [01:19] *** SmileyG has joined #archiveteam [01:20] *** Smiley has quit IRC (Read error: Operation timed out) [01:26] *** matthusby has joined #archiveteam [01:32] *** matthusby has quit IRC (Remote host closed the connection) [01:33] *** enowaldo has quit IRC (Read error: Operation timed out) [01:52] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [01:59] *** matthusby has joined #archiveteam [02:08] *** matthusby has quit IRC (Remote host closed the connection) [02:10] *** wp494 has quit IRC (Read error: Operation timed out) [02:10] *** wp494 has joined #archiveteam [02:11] *** ripdog has quit IRC (Remote host closed the connection) [02:11] *** enowaldo has joined #archiveteam [02:18] *** Sk1d has quit IRC (Read error: Operation timed out) [02:20] *** Sk1d has joined #archiveteam [02:31] *** bitBaron has joined #archiveteam [02:32] *** matthusby has joined #archiveteam [02:42] *** Sk1d has quit IRC (Read error: Operation timed out) [02:45] *** Sk1d has joined #archiveteam [02:46] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [02:59] *** pizzaiolo has quit IRC (Quit: pizzaiolo) [03:08] *** Mateon1 has quit IRC (Ping timeout: 615 seconds) [03:08] *** Mateon1 has joined #archiveteam [03:14] *** satoshi has quit IRC (Read error: Connection reset by peer) [03:18] *** matthusby has quit IRC (Remote host closed the connection) [03:19] *** satoshi has joined #archiveteam [03:22] *** ripdog has joined #archiveteam [03:31] *** ripdog has quit IRC (Ping timeout: 246 seconds) [03:53] *** inzom has joined #archiveteam [03:55] *** ripdog has joined #archiveteam [03:58] *** icedice has quit IRC (Quit: Leaving) [04:00] *** Sk1d has quit IRC (Read error: Operation timed out) [04:02] *** Sk1d has joined #archiveteam [04:26] *** Sk1d has quit IRC (Read error: Operation timed out) [04:26] *** Sk1d has joined #archiveteam [04:28] *** odemgi has joined #archiveteam [04:29] *** qw3rty118 has joined #archiveteam [04:31] *** odemgi_ has quit IRC (Ping timeout: 252 seconds) [04:31] *** Sk1d has quit IRC (Read error: Operation timed out) [04:34] *** qw3rty117 has quit IRC (Ping timeout: 600 seconds) [04:34] *** Sk1d has joined #archiveteam [04:37] *** odemg has quit IRC (Ping timeout: 615 seconds) [04:44] *** chimyatta has joined #archiveteam [04:58] *** redlob has quit IRC (Read error: Operation timed out) [04:58] *** matthusby has joined #archiveteam [05:07] *** matthusby has quit IRC (Read error: Operation timed out) [05:11] *** RKenshin has joined #archiveteam [05:13] *** stratum`_ has joined #archiveteam [05:17] *** Harzilein has quit IRC (Ping timeout: 492 seconds) [05:17] *** jtvjan has quit IRC (Ping timeout: 492 seconds) [05:17] *** stratum` has quit IRC (Ping timeout: 492 seconds) [05:17] *** Kenshin has quit IRC (Ping timeout: 492 seconds) [05:17] *** RKenshin is now known as Kenshin [05:19] *** Sk1d has quit IRC (Read error: Operation timed out) [05:19] *** jtvjan has joined #archiveteam [05:22] *** Sk1d has joined #archiveteam [05:43] *** enowaldo has quit IRC (Read error: Operation timed out) [05:45] *** SmileyG has quit IRC (Read error: Operation timed out) [05:56] *** Smiley has joined #archiveteam [06:07] *** Sk1d has quit IRC (Read error: Operation timed out) [06:10] *** Sk1d has joined #archiveteam [06:32] *** deevious has joined #archiveteam [06:46] *** Phoen1x has joined #archiveteam [07:00] *** enowaldo has joined #archiveteam [07:05] *** enowaldo has quit IRC (Ping timeout: 265 seconds) [07:06] *** Sk1d has quit IRC (Read error: Operation timed out) [07:08] *** Sk1d has joined #archiveteam [07:09] Hey anyone here have a lot of experience with how Internet Archive makes their item data available? Trying to script downloading several TB of stuff from them and I'm wondering if I can assume every file in an item will be distributed across the same two servers [07:10] Alternatively, is there any API to query what servers have a file? Just hitting the main download URL repeatedly until it returns the second server seem inelegant [07:11] And yes, I know this is not IA, but they seem to have fuck all in the way of community and I figured you guys might know [07:13] Phoen1x: yes, each item is fully on two servers. it's why you really shouldn't put more than a few dozen gigabytes in an item - it makes it harder for them to balance out items across servers. [07:14] #internetarchive is where we talk about 'how tf IA seems to work' [07:14] i don't know an external way to find which servers hold an item. maybe in the xml or the logs in the item, but that sounds fragile and undocumented. [07:15] aye, no more conversation about this in the announcements channel, moving to #internetarchive [07:19] *** wise_flow has joined #archiveteam [07:23] *** wiseflowe has quit IRC (Read error: Operation timed out) [07:47] *** Exairnous has quit IRC (Ping timeout: 252 seconds) [07:51] *** redlob has joined #archiveteam [08:01] Internet Archive: "Fuck All in the Way of Community" [08:02] Related to IA, I've made a new script set called YOU HAD ONE JOB, which finds accounts that are uploading things that all go to the same place, and every two hours it sweeps through and moves them. [08:03] It is because I am lazy and also the open collections are bloated. I have over 80 such accounts now in the ONEJOB list, which will pull them out of open collections and put them into the right place, including several intense youtube mirroring projects. [08:03] I will probably add more as I go. [09:02] *** enowaldo has joined #archiveteam [09:09] *** VADemon_ has quit IRC (Read error: Connection reset by peer) [09:14] *** enowaldo has quit IRC (Ping timeout: 612 seconds) [09:19] *** matthusby has joined #archiveteam [09:25] *** matthusby has quit IRC (Ping timeout: 360 seconds) [10:09] *** PhrackD has quit IRC (Read error: Operation timed out) [10:09] *** PhrackD has joined #archiveteam [10:59] *** BlueMax has quit IRC (Quit: Leaving) [11:18] *** wp494 has quit IRC (Read error: Operation timed out) [11:19] *** wp494 has joined #archiveteam [11:30] *** Boppen has joined #archiveteam [11:43] *** enowaldo has joined #archiveteam [11:48] *** enowaldo has quit IRC (Ping timeout: 265 seconds) [11:59] *** ubahn has joined #archiveteam [12:23] *** ubahn has quit IRC (Read error: Connection reset by peer) [12:24] *** ubahn has joined #archiveteam [12:30] *** ubahn has quit IRC (Read error: Connection reset by peer) [12:31] *** ubahn has joined #archiveteam [12:34] *** godane has quit IRC (Ping timeout: 265 seconds) [12:39] *** Sk1d has quit IRC (Read error: Operation timed out) [12:41] *** odemgi has quit IRC (Remote host closed the connection) [12:42] *** Sk1d has joined #archiveteam [12:50] *** enowaldo has joined #archiveteam [12:51] *** godane has joined #archiveteam [12:53] *** bitBaron has joined #archiveteam [12:55] *** enowaldo has quit IRC (Ping timeout: 265 seconds) [13:04] *** ubahn has quit IRC (Quit: ubahn) [13:20] *** enowaldo has joined #archiveteam [13:32] *** matthusby has joined #archiveteam [13:42] *** ubahn has joined #archiveteam [14:08] *** ubahn_ has joined #archiveteam [14:10] *** ubahn has quit IRC (Ping timeout: 252 seconds) [14:41] *** Sk1d has quit IRC (Read error: Operation timed out) [14:44] *** Sk1d has joined #archiveteam [15:07] *** bitBaron has quit IRC (My computer has gone to sleep. 😴😪ZZZzzz…) [15:16] *** bitBaron has joined #archiveteam [15:29] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [15:30] *** enowaldo has quit IRC (Read error: Operation timed out) [15:51] *** deevious has quit IRC (Quit: deevious) [15:56] *** bitBaron has joined #archiveteam [16:03] *** ubahn_ has quit IRC (Quit: ubahn_) [16:08] *** enowaldo has joined #archiveteam [16:09] *** LFlare has joined #archiveteam [16:20] *** enowaldo has quit IRC (Read error: Operation timed out) [16:21] *** enowaldo has joined #archiveteam [16:35] *** enowaldo has quit IRC (Read error: Operation timed out) [16:37] *** martini has joined #archiveteam [16:37] *** ubahn has joined #archiveteam [16:48] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [16:54] *** schbirid has joined #archiveteam [16:56] *** bitBaron has joined #archiveteam [16:59] *** enowaldo has joined #archiveteam [17:11] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [17:26] *** ubahn has quit IRC (Quit: ubahn) [17:33] *** LFlare has quit IRC (Quit: The Lounge - https://thelounge.chat) [17:45] *** enowaldo has quit IRC (Read error: Operation timed out) [17:56] *** Dimtree has quit IRC (Quit: Peace) [17:57] *** bitBaron has joined #archiveteam [18:23] *** Atom-- has joined #archiveteam [18:28] *** Atom has quit IRC (Read error: Operation timed out) [18:38] *** enowaldo has joined #archiveteam [18:55] *** CCTrollz has joined #archiveteam [18:55] So EFNet is not a fan of VPNs it would seem [18:57] *** CCTrollz has quit IRC (Client Quit) [19:14] *** Exairnous has joined #archiveteam [19:15] *** ubahn has joined #archiveteam [19:56] *** Raccoon has quit IRC (Ping timeout: 1212 seconds) [20:02] *** Raccoon has joined #archiveteam [20:03] *** schbirid has quit IRC (Remote host closed the connection) [20:09] *** wp494 has quit IRC (Ping timeout: 265 seconds) [20:10] *** wp494 has joined #archiveteam [20:35] *** m007a83 has quit IRC (Read error: Connection reset by peer) [20:48] *** ubahn_ has joined #archiveteam [20:56] *** ubahn has quit IRC (Ping timeout: 615 seconds) [21:00] *** redlob has quit IRC (Ping timeout: 615 seconds) [21:08] *** redlob has joined #archiveteam [21:14] *** chimyatta has quit IRC (Read error: Connection reset by peer) [21:14] *** C4K3 has quit IRC (Remote host closed the connection) [21:17] *** fredgido has quit IRC (Read error: Operation timed out) [21:19] *** Dimtree has joined #archiveteam [21:37] *** m007a83 has joined #archiveteam [21:38] *** pizzaiolo has joined #archiveteam [21:42] *** Medo42 has quit IRC (Quit: Allons-y!) [21:44] *** Medo42 has joined #archiveteam [22:28] *** VADemon has joined #archiveteam [22:31] *** martini has quit IRC (Quit: No Reasson) [22:32] *** MR9K has quit IRC (Read error: Connection reset by peer) [22:33] *** MR9K has joined #archiveteam [22:44] *** godane has quit IRC (Read error: Operation timed out) [22:57] *** Sk1d has quit IRC (Read error: Operation timed out) [23:00] *** Sk1d has joined #archiveteam [23:05] *** godane has joined #archiveteam [23:11] *** Sk1d has quit IRC (Read error: Operation timed out) [23:11] *** ubahn_ has quit IRC (Quit: ubahn_) [23:13] *** fredgido has joined #archiveteam [23:14] *** Sk1d has joined #archiveteam [23:42] *** Mateon1 has quit IRC (Remote host closed the connection) [23:43] *** Mateon1 has joined #archiveteam [23:52] *** stratum`_ has quit IRC (Max SendQ exceeded) [23:52] *** stratum` has joined #archiveteam [23:55] *** odemgi has joined #archiveteam