[01:00] Nemo_bis: it seems wikis are now disabling Special:Export too [01:00] which breaks screenscraping [01:00] at least the way the script does it [01:10] what fucks [01:10] any idea why? [05:36] balrog: "now"? It's not a new thing, any specific wiki in your mind? [16:30] Nemo_bis: http://wikirtishchevo.shoutwiki.com/wiki/Ð¡Ð»ÑÐ¶ÐµÐ±Ð½Ð°Ñ:ÐÐºÑÐ¿Ð¾ÑÑ [16:30] balrog: all of shoutwiki? [16:30] not sure if /all/ [16:31] let me check a couple more [16:31] It's new since january for that wiki https://archive.org/details/wiki-wikirtishchevoshoutwikicom_w [16:32] Nemo_bis: seems like all of shoutwiki, indeed [16:32] O_o [16:32] I checked like five wikis [16:32] no Special:Export on any [16:33] http://blog.shoutwiki.com/2012/06/new-feature-instant-database-dumps/ no longer [16:33] isn't that something else? [16:33] hah, 404 [16:34] Yes, I was being optimistic "maybe they disabled Special:Export because it was superseded by regular server-side dumps" [16:34] :/ [16:35] Well, no need to worry now, I'll poke/beat them in 6 months from now when our dumps start rotting and updates will be very needed [16:36] In the meanwhile we can ger (more) ready with https://github.com/WikiTeam/wikiteam/issues/44 [16:36] * get [17:08] hey Nemo_bis, would you be so kind as to add me to the wikiteam github org? [17:08] username 'chronomex [17:08] ' [17:10] cmx: thanks for asking but I have no such power, ask emijrp [17:10] ok [17:10] will do [17:22] time to port this thing to requests [17:22] it's nowhere near as difficult as any of you think [17:22] just one concern [17:22] what's the best way to handle cookies? [17:22] a global variable for the cookiejar? [21:57] Nemo_bis: are you around? [21:57] balrog: yes but going to bed soonish [21:57] ah ok [21:57] so far: ported to Requests; fixed more unicode bugs [21:57] balrog: did you find an answer for the cookies? [21:57] whee :) [21:57] No, I was asking what's recommended [21:58] Requests requires the cookiejar to be passed when you call requests.post() or get() [21:58] I was trying to think if a global cookie could have pollute anything [21:58] Not sure what's the cleanest way to do that [21:58] would be a global cookiejar [21:58] It would be cleared from one dump to the next anyway, right? [21:58] yes of course [21:58] it would only exist while the session is running [21:58] Yep [21:59] I was just wondering if there's a cleaner way to do it than with a python global variable [21:59] passing it in the config dict looks messy too [21:59] but would work; thing is, is it ok to add stuff to that dict? [21:59] Consider we already pass around a global "config" I guess this won't be much worse [21:59] my question is: add it to the global 'config' or add a global var [21:59] Better not, because sometimes we pass it from one script/"class" to the other [22:00] hmm what do you mean? [22:00] Global var looks better [22:00] oh, config gets passed from altogether different scripts? [22:00] I mean when launcher.py calls dumpgenerator.py and stuff like that, IIRC [22:00] ah ok [22:00] Though I may misremember [22:04] hmm or I can just use a session maybe [22:04] would that be better? [22:05] I wouldn't nitpick too much, global variable is ok. [22:05] keep-alive works within a session [22:05] which is a great thing since it reduces server load greatly [22:05] That sounds nice [22:13] can you tell emijrp what I'm doing? [22:13] I don't want to duplicate work [22:19] He's already working on other stuff [22:20] like? [22:20] I see him pushing stuff to the repo [22:20] I mean, non-intersecting work [22:20] Except perhaps some of the unicode fixes :) [22:23] I added a note on https://github.com/WikiTeam/wikiteam/issues/104