Doccle to Paperless Integration

This project provides an integration between Doccle and Paperless, allowing you to automatically download new documents from Doccle and upload them to Paperless for easy organization and management.

Doccle no longer accepts basic auth for document downloads, so this tool logs in through the Doccle web interface with Playwright and downloads via the briefcase endpoint. Playwright is therefore a required dependency.

Features

• Automatically checks for new documents in Doccle at regular intervals
• Downloads new documents as PDF files with friendly filenames
• Saves document metadata as JSON files
• Uploads downloaded documents to Paperless
• Archives documents in Doccle after successful upload to Paperless
• Detects documents that require itsme/eID and reports them instead of retrying forever
• Reads every page of your Doccle inbox, not just the first
• Provides a health check endpoint for monitoring the application's status
• Supports running the application locally or using Docker
• Optional notifications via webhook or Pushover
• Rotating log files with configurable retention

Documents this tool cannot download

Doccle serves some documents only behind an itsme or eID step-up login, which cannot be automated. Typical examples are tax assessments from FOD Financiën and pension statements from Sigedis.

These are detected automatically. You get one notification per document, it is recorded in the failure tracker, and it is not retried. Download those documents manually from Doccle and archive them there.

TODO

• Assign owner, tags, correspondents, and document types from Doccle's metadata
• Better scheduling
• Better status API

Prerequisites

• Python 3.12 or later (the Docker image ships 3.12)
• Docker and Docker Compose (optional, for running the application in a container)
• Doccle account with valid credentials
• Paperless instance with API access

Installation and Configuration

1. Clone the repository and navigate to the project directory.

2. Copy .env.sample to .env and fill in your Doccle and Paperless credentials.

3. For Docker usage, copy docker-compose.sample.yaml to docker-compose.yaml and adjust the environment and volumes to taste.

Configuration reference

Variable	Default	Purpose
DOCCLE_USERNAME	–	Doccle login (required)
DOCCLE_PASSWORD	–	Doccle password (required)
PAPERLESS_URL	–	Base URL of your Paperless instance (required)
PAPERLESS_TOKEN	–	Paperless API token (required)
FETCH_INTERVAL	3600	Seconds between runs
LOG_LEVEL	INFO	DEBUG, INFO, WARNING or ERROR
DATA_DIR	data	Where the login cookie and failure tracker are kept
MAX_DOC_RETRIES	3	Attempts before a document is left alone
RETRY_COOLDOWN_HOURS	72	How long before a given-up document is tried again
DOCCLE_FORCE_LOGIN	unset	Set to true to ignore cached cookies
WEBHOOK_URL	unset	Receives a JSON POST with {message, title}
PUSHOVER_USER / PUSHOVER_TOKEN	unset	Pushover credentials

Persistent state

DATA_DIR holds the cached Doccle session cookie and failed_documents.json. Mount it (the sample compose file maps ./data) so a container rebuild does not force a fresh login or lose the record of documents that need manual handling.

Usage

Running Locally

1. Install the required dependencies:

   pip install -r requirements.txt
   playwright install chromium
   

2. Run the application:

   python main.py
   

   On first run, Playwright authenticates via the Doccle web interface (headless). Cookies are cached in $DATA_DIR/.doccle_cookies.json and reused for 12 hours.

Running with Docker

1. Build the Docker image and start the container:

   docker compose up -d
   

The application will periodically check for new documents in Doccle, download them, and upload them to Paperless.

Manual Document Import

In case some documents fail to import automatically, you can manually retry importing all downloaded documents from the downloaded_documents folder:

1. Ensure your .env file includes PAPERLESS_URL and PAPERLESS_TOKEN.

2. Run the manual import script:

   python import_downloaded.py
   

Note: Depending on your Paperless instance settings, you may receive an OK response from the Paperless API, but the document might not import successfully if it already exists in Paperless. In such cases, check the Paperless application for any in-app error messages.

Testing

Run the test suite:

pip install pytest
python -m pytest tests/ -v

Logging

Logs are written to the logs directory with automatic rotation:

• doccle.log - Main application log (rotates at 10MB, keeps 5 backups)
• doccle_errors.log - Errors only

Set LOG_LEVEL in your .env file to control verbosity (DEBUG, INFO, WARNING, ERROR).

Notifications

Optional push notifications on document import success/failure:

Webhook - Set WEBHOOK_URL to receive JSON POST with {message, title}.

Pushover - Set PUSHOVER_USER and PUSHOVER_TOKEN for mobile notifications.

Monitoring

The application provides a health check endpoint at /health that returns the current status and uptime of the application.

Troubleshooting

"Failed to download or invalid PDF": Doccle stopped accepting basic auth for downloads. Update to v0.2.0 or later, which logs in with Playwright instead.

Login fails: set LOG_LEVEL=DEBUG and check $DATA_DIR/login_error.png, a screenshot taken when the login page does not look as expected.

A document never imports: it most likely needs itsme/eID. Check the logs for "Secured document" and download it manually.

Contributing

Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.

License

This project is licensed under the GNU General Public License v3.0.

Acknowledgements

• Steve Gilissen, author of py-doccle
• Doccle for providing the document management platform
• Paperless-NGX for the community-supported open-source document management system