|
It was written in Python and can be utilized for data mining, monitoring and automated testing.
Here are some key features of "Scrapy":
· Simple - Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way
· Productive - Just write the rules to extract the data from web pages and let Scrapy crawl the entire web site for you
· Fast - Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server
· Extensible - Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core
· Portable - Scrapy runs on Linux, Windows, Mac and BSD
· Open Source and 100% Python -Scrapy is completely written in Python, which makes it very easy to hack
· Well-tested - Scrapy has an extensive test suite with very good code coverage.
Requirements:
· Python 2.5 or higher
· Twisted 2.5.0 or higher
· libxml2 2.6.28 or higher
· pyopenssl
Via: Scrapy 0.16.1
0 Comment:
Post a Comment