Releasing crau, a command-line tool to archive the Web
Published on
Originally published on Twitter on October 26, 2019
Topics: Web scraping, Command line, Python
I've just released crau (the way Brazilians say "crawl") version 0.1.0: the easiest command-line tool for archiving the #Web and playing archives: you just need a list of URLs. Install by running `pip install crau`, issues on https://GitHub.com/turicas/crau #scraping #python #datascience
The motivation to start this project was a lack of easy and robust software to archive URLs - I just wanted to execute one command without thinking and get a WARC file. It still has a lot of improvements to be made, but works already.