so i couldn't get bluebird card to work with petreon without registering it
and bluebird.com couldn't take my info after i filled everything out
https://archive.org/details/disneynews&tab=collection
i noticed that awhile ago
SketchCow: i'm setting up my patreon page so can get more vhs tapes to digitize
SketchCow: https://www.patreon.com/godane
My Dead Format scraper isn't even close to done yet, but it already discovered 10.9k users (out of 12.3k total according to the homepage). :-)
here are all the tapes on archive.org that i digitize so far: https://pastebin.com/SAzZth7J
nice job!
i have a patreon page to get money to buy tapes off ebay: https://www.patreon.com/godane
JAA: make sure you doublecheck that it's actually getting all results :)
JAA: the scraper I wrote was for a search that allowed like 50 results max, so the variance in letter usage made quite an impact
if you can get more results out of your target, the adapting thing might indeed not be necessary
my twitter account: https://twitter.com/ArchiveGodane
i put a twit out to help get my patreon campaign going
i hope when SketchCow gets better he can retweet my campaign
i really suck at social networking stuff anyways
joepie91: The problem isn't that certain search terms don't work. I could just make 26 queries for a* through z* and handle the pagination. But that would be extremely slow because it takes the server a very long time to retrieve those records from the database.
Also, searches for bla*, blac*, and black* are almost equally slow. But searching for blacka*, blackb*, etc. obviously won't find records with the word "black". So I can't really go too deep either.
I rewrote my scraper earlier today. It now uses aiohttp and multiple connections. In less than three hours, it has already surpassed the progress my other script has made since yesterday.
and i keep getting cockblocked by wpull bugs :(
Yeah, I'm pretty glad I didn't use wpull for this one.
Hmm, I guess I should've used multiple aiohttp sessions.
https://i.mundus.xyz/2ZEbM7.png
mundus: lol
yep
mundus: maybe put a robots.txt so google doesn't crawl it
good idea
the future is here: https://twitter.com/a_antonellis/status/912428669230043136
that's heckin rad