We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
A simple crawler. It requires python 3. Build: $ virtualenv -p python3 .env $ . .env/bin/activate $ pip install -r requirements.txt Usage: $ python crawler.py http://<root url>
A very simple crawler, nothing you would use in production