[00:24] *** Ravenloft has joined #archiveteam-bs [00:43] *** kyan_ has joined #archiveteam-bs [00:43] *** kyan has quit IRC (Read error: Connection reset by peer) [01:16] *** primus104 has quit IRC (Leaving.) [01:27] *** Fusl has quit IRC (Ping timeout: 186 seconds) [01:39] *** Fusl has joined #archiveteam-bs [02:35] *** Start has joined #archiveteam-bs [03:35] *** JesseW has joined #archiveteam-bs [03:48] *** ripvanwin has joined #archiveteam-bs [03:51] *** JesseW has quit IRC (Read error: Operation timed out) [03:54] *** JesseW has joined #archiveteam-bs [03:59] i'm starting to upload Kevin Pollak Chat Show [04:08] *** lbft has quit IRC (Ping timeout: 258 seconds) [04:08] *** lbft has joined #archiveteam-bs [04:17] *** vitzli has joined #archiveteam-bs [04:19] *** ripvanwin has quit IRC (Leaving) [04:40] *** zenguy_pc has quit IRC (Read error: Connection reset by peer) [04:41] *** zenguy_pc has joined #archiveteam-bs [05:15] *** BlueMaxim has joined #archiveteam-bs [05:33] *** JesseW has quit IRC (Read error: Operation timed out) [05:39] *** JesseW has joined #archiveteam-bs [06:50] *** JesseW has quit IRC (Read error: Operation timed out) [07:17] Is it possible to get a list of items in the collection via API requests? aaand I found answer myself: use advanced search to return list as json - https://blog.archive.org/2012/04/26/downloading-in-bulk-using-wget/ [07:18] I thought that there is a native way of querying collection, like http://archive.org/metadata/COLLECTIONANME [07:50] *** primus104 has joined #archiveteam-bs [08:26] *** arkiver2 has joined #archiveteam-bs [08:38] *** brayden_ has joined #archiveteam-bs [08:38] *** swebb sets mode: +o brayden_ [08:39] *** primus104 has quit IRC (Leaving.) [08:40] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [10:13] *** Ravenloft has quit IRC (Remote host closed the connection) [10:13] vitzli: there is. https://blog.archive.org/2013/07/04/metadata-api/ [10:13] vitzli: and https://archive.org/help/json.php [10:13] vitzli: and a bunch of other stuff, which the `ia` lib/tool combines [10:13] :P [10:15] ./metadata/ stuff gives only information about creator and size, not individual items, but maybe I am doing something wrong [10:16] but using advancedsearch.php works [10:19] i.e. archive.org/metadata/ephemera/ - where ephemera is a collection - just gives the list of service files (.xml, .sqlite and their md5's), not the list of items in collection. [10:22] ahh. yes [10:22] you are right [10:22] indeed needs a collection: search [10:25] *** primus104 has joined #archiveteam-bs [11:31] *** robink has quit IRC (Read error: Connection reset by peer) [11:32] *** schbirid has joined #archiveteam-bs [11:32] *** robink has joined #archiveteam-bs [11:37] *** robink has quit IRC (Read error: Connection reset by peer) [11:37] *** robink has joined #archiveteam-bs [11:44] *** primus104 has quit IRC (Leaving.) [12:04] *** robink has quit IRC (Ping timeout: 492 seconds) [12:05] *** robink has joined #archiveteam-bs [12:53] *** robink has quit IRC (Quit: No Ping reply in 210 seconds.) [12:53] *** robink has joined #archiveteam-bs [13:18] *** vitzli has quit IRC (Quit: Leaving) [13:25] *** robink has quit IRC (Read error: Connection reset by peer) [13:38] There are a bunch of people within IA working on a new version of search [13:38] There's one I use but few others can [13:39] ia search --itemlist "collection:computermagazines" [13:43] *** Start has quit IRC (Quit: Disconnected.) [13:54] *** kyan_ has quit IRC (Quit: This computer has gone to sleep) [13:54] *** kyan_ has joined #archiveteam-bs [13:55] *** kyan_ has quit IRC (Client Quit) [13:55] *** kyan_ has joined #archiveteam-bs [13:55] *** kyan_ has quit IRC (Client Quit) [13:56] *** BlueMaxim has quit IRC (Quit: Leaving) [14:08] *** robink has joined #archiveteam-bs [14:17] *** primus104 has joined #archiveteam-bs [14:21] *** robink has quit IRC (Ping timeout: 492 seconds) [14:33] *** robink has joined #archiveteam-bs [14:34] this may sound crazy, but does anyone archiving irc logs? [14:35] useretail: they end up in archivebot every now and then, but I don't think it's happening actively yet [14:36] *** Start has joined #archiveteam-bs [14:45] many people here have logging enabled... [14:45] I know for sure e I do [14:53] *** primus104 has quit IRC (Leaving.) [15:12] *** robink has quit IRC (Ping timeout: 492 seconds) [15:19] *** Start has quit IRC (Quit: Disconnected.) [15:20] *** primus104 has joined #archiveteam-bs [15:24] *** robink has joined #archiveteam-bs [15:31] *** robink has quit IRC (Read error: Connection reset by peer) [15:33] *** robink has joined #archiveteam-bs [15:39] *** vitzli has joined #archiveteam-bs [15:46] *** JesseW has joined #archiveteam-bs [15:53] *** Start has joined #archiveteam-bs [16:00] *** primus104 has quit IRC (Leaving.) [16:07] *** Start has quit IRC (Quit: Disconnected.) [16:14] *** JesseW has quit IRC (Read error: Operation timed out) [16:15] *** Stiletto has quit IRC (Read error: Operation timed out) [16:22] *** kyan has joined #archiveteam-bs [16:22] *** wyatt8740 has quit IRC (Remote host closed the connection) [16:23] *** wyatt8740 has joined #archiveteam-bs [16:43] *** arkiver2 has joined #archiveteam-bs [16:46] i'm up to 2015-06-18 of medium.com [16:46] *** wyatt8740 has quit IRC (Remote host closed the connection) [16:47] *** wyatt8740 has joined #archiveteam-bs [17:00] *** arkiver2 has quit IRC (Ping timeout: 252 seconds) [17:07] *** useretail has quit IRC (Read error: Operation timed out) [17:09] *** wyatt8740 has quit IRC (Remote host closed the connection) [17:10] *** wyatt8740 has joined #archiveteam-bs [17:11] *** toad1 has quit IRC (Read error: Operation timed out) [17:12] *** lbft has quit IRC (Ping timeout: 600 seconds) [17:13] *** lbft has joined #archiveteam-bs [17:14] *** Gfy has quit IRC (Ping timeout: 364 seconds) [17:15] *** lbft has quit IRC (Read error: Operation timed out) [17:16] *** lbft has joined #archiveteam-bs [17:16] *** kvieta has quit IRC (Ping timeout: 600 seconds) [17:17] *** phuzion has quit IRC (Ping timeout: 600 seconds) [17:19] *** dashcloud has quit IRC (Read error: Operation timed out) [17:22] *** dashcloud has joined #archiveteam-bs [17:22] *** phuzion has joined #archiveteam-bs [17:25] *** brayden__ has joined #archiveteam-bs [17:25] *** swebb sets mode: +o brayden__ [17:25] *** toad1 has joined #archiveteam-bs [17:25] *** kvieta has joined #archiveteam-bs [17:26] *** useretai- has joined #archiveteam-bs [17:26] *** Gfy has joined #archiveteam-bs [17:31] *** brayden_ has quit IRC (Read error: Operation timed out) [17:32] *** Start has joined #archiveteam-bs [17:37] *** primus104 has joined #archiveteam-bs [17:38] *** Start has quit IRC (Quit: Disconnected.) [17:51] *** vitzli has quit IRC (Quit: Leaving) [18:20] *** PurpleSym has joined #archiveteam-bs [18:36] wpull does not add redirect targets (status 302) to its database, does it? [18:44] it will follow redirects, yes [18:56] It does, yes. [18:56] But it downloads the same file over and over. [18:56] If it is a redirect target. [19:07] poke chfoo for more information [19:07] it could be an error [19:08] we see this behavior in archivebot also but there's duplication detection that seems to ameliorate the problem [19:08] *** nico_32_ has joined #archiveteam-bs [19:08] *** Coderjoe has joined #archiveteam-bs [19:08] *** tephra_ has joined #archiveteam-bs [19:08] Is the deduplication an archivebot feature? [19:09] it's a wpull plugin that archivebot drags in [19:09] *** SketchCo1 has joined #archiveteam-bs [19:09] *** swebb sets mode: +o SketchCo1 [19:10] Ok. I’ll check the github issue tracker first. [19:11] *** Fusl has quit IRC (hub.se efnet.port80.se) [19:11] *** nico_32 has quit IRC (hub.se efnet.port80.se) [19:11] *** Kazzy has quit IRC (hub.se efnet.port80.se) [19:11] *** kevin has quit IRC (hub.se efnet.port80.se) [19:11] *** edsu has quit IRC (hub.se efnet.port80.se) [19:11] *** sigkell has quit IRC (hub.se efnet.port80.se) [19:11] *** Lord_Nigh has quit IRC (hub.se efnet.port80.se) [19:11] *** Coderjoe_ has quit IRC (hub.se efnet.port80.se) [19:11] *** dan- has quit IRC (hub.se efnet.port80.se) [19:11] *** anomie has quit IRC (hub.se efnet.port80.se) [19:11] *** Fletcher has quit IRC (hub.se efnet.port80.se) [19:11] *** diacope has quit IRC (hub.se efnet.port80.se) [19:11] *** SketchCow has quit IRC (hub.se efnet.port80.se) [19:11] *** tephra has quit IRC (hub.se efnet.port80.se) [19:11] *** goekesmi has quit IRC (hub.se efnet.port80.se) [19:11] *** zyphlar has quit IRC (hub.se efnet.port80.se) [19:11] *** _desu_ has quit IRC (hub.se efnet.port80.se) [19:11] *** JSharp has quit IRC (hub.se efnet.port80.se) [19:11] *** deathy has quit IRC (hub.se efnet.port80.se) [19:11] *** Rickster has quit IRC (hub.se efnet.port80.se) [19:11] *** afics has quit IRC (hub.se efnet.port80.se) [19:11] *** Ctrl-S has quit IRC (hub.se efnet.port80.se) [19:11] *** Muad-Dib has quit IRC (hub.se efnet.port80.se) [19:11] *** GLaDOS has quit IRC (hub.se efnet.port80.se) [19:11] *** Boltsie has quit IRC (hub.se efnet.port80.se) [19:11] *** arkiver has quit IRC (hub.se efnet.port80.se) [19:12] *** dan-- has joined #archiveteam-bs [19:12] *** Kazzy_ has joined #archiveteam-bs [19:16] *** edsu_ has joined #archiveteam-bs [19:16] *** swebb sets mode: +o edsu_ [19:16] *** goekesmi_ has joined #archiveteam-bs [19:21] *** Fusl_ has joined #archiveteam-bs [19:22] *** schbirid has quit IRC (Read error: Operation timed out) [19:26] *** Kazzy_ is now known as Kazzy [19:27] *** Fusl_ is now known as Fusl [19:38] *** schbirid has joined #archiveteam-bs [19:42] *** Lord_Nigh has joined #archiveteam-bs [19:44] *** schbirid has quit IRC (Quit: Leaving) [19:46] *** aaaaaaaaa has joined #archiveteam-bs [19:46] *** swebb sets mode: +o aaaaaaaaa [19:49] *** Stiletto has joined #archiveteam-bs [20:08] *** PurpleSym has quit IRC (Quit: WeeChat 1.1.1) [20:10] *** SketchCo1 is now known as SketchCow [20:15] *** kyan has quit IRC (Quit: This computer has gone to sleep) [20:31] *** kyan has joined #archiveteam-bs [21:00] *** Fletcher has joined #archiveteam-bs [21:00] *** diacope has joined #archiveteam-bs [21:00] *** Ctrl-S has joined #archiveteam-bs [21:00] *** afics has joined #archiveteam-bs [21:00] *** Boltsie has joined #archiveteam-bs [21:00] *** zyphlar has joined #archiveteam-bs [21:00] *** kevin has joined #archiveteam-bs [21:00] *** deathy has joined #archiveteam-bs [21:00] *** _desu_ has joined #archiveteam-bs [21:00] *** JSharp has joined #archiveteam-bs [21:00] *** sigkell has joined #archiveteam-bs [21:00] *** anomie has joined #archiveteam-bs [21:00] *** Rickster has joined #archiveteam-bs [21:00] *** Muad-Dib has joined #archiveteam-bs [21:00] *** GLaDOS has joined #archiveteam-bs [21:00] *** arkiver has joined #archiveteam-bs [21:00] *** efnet.port80.se sets mode: +o arkiver [21:10] *** aaaaaaaaa has quit IRC (Read error: Connection reset by peer) [21:10] *** Start has joined #archiveteam-bs [21:16] *** Start has quit IRC (Quit: Disconnected.) [21:22] *** Fletcher has quit IRC (hub.se efnet.port80.se) [21:22] *** diacope has quit IRC (hub.se efnet.port80.se) [21:22] *** Ctrl-S has quit IRC (hub.se efnet.port80.se) [21:22] *** afics has quit IRC (hub.se efnet.port80.se) [21:22] *** zyphlar has quit IRC (hub.se efnet.port80.se) [21:22] *** kevin has quit IRC (hub.se efnet.port80.se) [21:22] *** Boltsie has quit IRC (hub.se efnet.port80.se) [21:22] *** deathy has quit IRC (hub.se efnet.port80.se) [21:22] *** _desu_ has quit IRC (hub.se efnet.port80.se) [21:22] *** JSharp has quit IRC (hub.se efnet.port80.se) [21:22] *** sigkell has quit IRC (hub.se efnet.port80.se) [21:22] *** anomie has quit IRC (hub.se efnet.port80.se) [21:22] *** Rickster has quit IRC (hub.se efnet.port80.se) [21:22] *** Muad-Dib has quit IRC (hub.se efnet.port80.se) [21:22] *** GLaDOS has quit IRC (hub.se efnet.port80.se) [21:22] *** arkiver has quit IRC (hub.se efnet.port80.se) [21:25] *** nico_32_ is now known as nico_32 [21:29] *** anomie_ has joined #archiveteam-bs [21:33] *** Arkiver2 has joined #archiveteam-bs [21:38] *** anomie_ is now known as anomie [21:45] *** Arkiver2 is now known as arkiver [21:55] *** aaaaaaaaa has joined #archiveteam-bs [21:55] *** swebb sets mode: +o aaaaaaaaa [22:01] *** aaaaaaaaa sets mode: +ooo chfoo joepie91 yipdw [22:03] *** aaaaaaaaa sets mode: +oo godane midas [22:06] *** wyatt8740 has quit IRC (Remote host closed the connection) [22:41] *** wyatt8740 has joined #archiveteam-bs [23:01] *** wyatt8740 has quit IRC (Remote host closed the connection) [23:05] *** wyatt8740 has joined #archiveteam-bs [23:56] *** Start has joined #archiveteam-bs