[07:54] alard: do you have a Yahoo! search API key? [07:55] No. [07:56] Do I need one? [07:56] alard: I'm trying to get someone (or myself?) run http://www.cs.brown.edu/~pavlo/mediawiki/ again [07:56] to list all the mediawikis around [07:58] alard: do your scripts for Google use keys or are they not needed? I don't understand the requirements for http://graffiti.cs.brown.edu/svn/graffiti/src/webcrawler/crawler/googlecrawler.py [07:58] The Google script doesn't need a key. You do need a (free) ipv6 tunnel. [10:39] alard: I don't know if the script supports it though [10:39] emijrp: can't Pavlo's crawler be used with Google only?one doesn't need a key for that [10:44] try it, im busy with WLM [10:57] Nemo_bis: You could look if you can modify the script. From what I've seen it's a two-step process: it does a random search for "This is MediaWiki"-like phrases, that's something that you could do with any google crawl script, and the second part is looking at each url you've found to see if it is really a MediaWiki-wiki. [12:22] alard: is that google.search it imports a standard library of some sort? [12:23] oh, this one it seems http://web.archive.org/web/20110625003240/http://blackcodeseo.com/google-search-api-for-python/