[01:28] *** edsu has quit IRC (Read error: Operation timed out) [01:35] *** edsu has joined #internetarchive [01:39] *** X-Scale has quit IRC (Ping timeout: 240 seconds) [01:40] *** [X-Scale] has joined #internetarchive [03:44] *** nicolas17 has joined #internetarchive [03:44] hi, what's the user-agent used by the wayback machine crawler? [03:47] I'm trying to archive http://www.casarosada.gob.ar/informacion/eventos-destacados-presi/39048-el-gobierno-firmo-convenios-con-la-casa-ana-frank-para-promover-el-dialogo-y-la-tolerancia and I'm getting a blank page [03:48] so I'm wondering if they block IA [03:48] https://archive.org/details/archive.org_bot [03:48] "User Agent archive.org_bot is used for our wide crawl of the web" [03:48] I'm using the "Save Page Now" thing, will that do things differently? [03:48] i'm not sure [03:49] they do block curl (response becomes "

" if the user-agent is curl) [03:50] that's positively adversarial [03:50] but curl -A "archive.org_bot" gives the normal page [03:50] so I dunno [04:47] *** X-Scale has joined #internetarchive [04:48] *** [X-Scale] has quit IRC (Ping timeout: 240 seconds) [06:14] *** Stilett0 has quit IRC (Ping timeout: 250 seconds) [06:22] *** nicolas17 has quit IRC (Quit: Konversation terminated!) [07:34] *** atomotic has joined #internetarchive [08:00] *** X-Scale has quit IRC (Quit: Try HydraIRC -> http://www.hydrairc.com <-) [08:23] *** atomotic has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…) [08:28] *** atomotic has joined #internetarchive [09:33] *** atomotic has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…) [09:34] *** kyounko has joined #internetarchive [10:08] *** atomotic has joined #internetarchive [14:00] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:21] *** nicolas17 has joined #internetarchive [15:30] *** atomotic has joined #internetarchive [16:51] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [17:14] *** Stilett0 has joined #internetarchive [20:31] *** SketchCow has quit IRC (Quit: Lost terminal) [20:38] *** SketchCow has joined #internetarchive [23:19] *** butchster has quit IRC (Ping timeout: 492 seconds) [23:25] *** kyan has joined #internetarchive