There is a nice website screenshots.com that hosts historic screenshots for many websites. This post will show how to generate our own historic screenshots with python.
Blog
-
Generating a website screenshot history
Webkit Python Qt Opensource January 03, 2013
-
Automating webkit
Python Webkit Qt Example February 14, 2012
I have received some inquiries about using webkit for web scraping, so here is an example using the webscraping module:
-
Threading with webkit
Javascript Webkit Qt Python Example Concurrent Efficiency December 30, 2011
In a previous post I showed how to scrape a list of webpages. That is fine for small crawls but will take too long otherwise. Here is an updated example that downloads the content in multiple threads.
-
Scraping multiple JavaScript webpages with webkit
Javascript Webkit Qt Python Example December 06, 2011
I made an earlier post about using webkit to process the JavaScript in a webpage so you can access the resulting HTML. A few people asked how to apply this to multiple webpages, so here it is:
-
Webpage screenshots with webkit
Webkit Qt Python Screenshot Example September 20, 2011
For a recent project I needed to render screenshots of webpages. Here is my solution using webkit:
-
Scraping JavaScript webpages with webkit
Javascript Webkit Qt Python March 12, 2010
In the previous post I covered how to tackle JavaScript based websites with Chickenfoot. Chickenfoot is great but not perfect because it: