This week Guido Van Rossum (author of Python) put out a call for experts at asynchronous programming to collaborate on a new API.
Blog
-
Asynchronous support in Python
Python Concurrent Big picture October 20, 2012
-
Threading with webkit
Javascript Webkit Qt Python Example Concurrent Efficiency December 30, 2011
In a previous post I showed how to scrape a list of webpages. That is fine for small crawls but will take too long otherwise. Here is an updated example that downloads the content in multiple threads.
-
Crawling with threads
Concurrent Python April 10, 2011
The bottleneck for web scraping is generally bandwidth - the time waiting for webpages to download. This delay can be minimized by downloading multiple webpages concurrently in separate threads.