Adhere to JOSS requirements and more analyses/examples

CONTRIBUTING.md

@@ -0,0 +1,60 @@
+# Contributing
+
+Thanks for considering a contribution to `paperscraper`. Contributions are
+welcome when they improve reliability, documentation, tests, supported scholarly
+metadata sources, or the ergonomics of existing workflows.
+
+## Ways to Contribute
+
+- Report bugs with a minimal reproducible example.
+- Improve documentation, examples, or error messages.
+- Add tests for existing behavior.
+- Fix source-specific API breakage.
+- Propose new metadata sources or retrieval fallbacks.
+
+For larger changes, please open an issue first so the design, maintenance cost,
+and API surface can be discussed before implementation.
+
+## Development Setup
+
+The project uses `uv` for local development:
+
+```sh
+uv sync --group dev
+uv run python -c "import paperscraper"
+```
+
+Run formatting, linting, and tests before opening a pull request:
+
+```sh
+uv run ruff format paperscraper
+uv run ruff check paperscraper
+uv run isort paperscraper
+uv run pytest paperscraper
+```
+
+Some tests exercise external scholarly services and can be slow or sensitive to
+rate limits. The GitHub Actions test suite is the source of truth for release
+readiness; it can take a long time because it verifies source and wheel installs
+against the supported workflows.
+
+## Pull Request Expectations
+
+- Keep changes focused and avoid unrelated refactors.
+- Add or update tests when changing behavior.
+- Update documentation and examples when changing user-facing APIs.
+- Do not commit API keys, credentials, downloaded server dumps, PDFs, or other
+  large generated artifacts.
+- Preserve backward compatibility unless a breaking change has been discussed.
+
+## Release Notes
+
+This project does not maintain a separate `CHANGELOG.md`. Release notes are kept
+with versioned GitHub releases and PyPI release history. Maintainers summarize
+notable changes there when publishing a new package version.
+
+## Conduct
+
+Please keep issues and pull requests respectful, technical, and actionable. The
+maintainers may close or moderate interactions that are abusive, off-topic, or
+not aligned with the project scope.

GOVERNANCE.md

@@ -0,0 +1,53 @@
+# Governance
+
+`paperscraper` is maintained as an open-source research software project. Its
+scope is to provide practical tools for scholarly metadata collection,
+publication search, full-text retrieval where access is available, citation
+analysis, and related reproducible examples.
+
+## Maintainers
+
+The project maintainers are responsible for reviewing contributions, deciding on
+API changes, publishing releases, and keeping the package aligned with its
+research-software scope. The current maintainers are the package authors listed
+in `pyproject.toml` and on PyPI.
+
+## Decision Making
+
+Small fixes can be reviewed and merged by a maintainer when tests and
+documentation are adequate. Larger changes, new dependencies, new data sources,
+or compatibility-breaking changes should be discussed in an issue before a pull
+request is opened.
+
+Maintainers aim for consensus. If consensus is not possible, final decisions are
+made by the active maintainers based on project scope, maintenance burden,
+backward compatibility, testability, and benefit to users.
+
+## Contribution Review
+
+Pull requests are evaluated for:
+
+- Correctness and reproducibility.
+- Compatibility with existing public APIs.
+- Test coverage appropriate to the change.
+- Documentation for user-facing behavior.
+- Respect for source platform terms, publisher restrictions, and credential
+  safety.
+
+Maintainers may request changes, split large pull requests, or decline changes
+that would add substantial maintenance cost without clear benefit to the core
+use cases.
+
+## Releases
+
+Releases are made by maintainers when a set of changes is ready for users. The
+project uses versioned Git tags and publishes packages to PyPI. Release notes are
+provided through GitHub releases and PyPI release history rather than a separate
+changelog file.
+
+## Project Direction
+
+Priorities are guided by user reports, contributor interest, upstream API
+changes, documentation quality, and the needs of reproducible literature and
+bibliometric analyses. The project favors reliable, well-documented workflows
+over broad but weakly maintained source coverage.

README.md

@@ -17,6 +17,9 @@ It provides a streamlined interface to scrape metadata, retrieve citation counts
 from [Google Scholar](https://scholar.google.com/), query journal impact factors,
 and run simple postprocessing and plotting routines for meta-analysis.
 
+<p align="center">
+  <img src="assets/preprints_per_year.png" alt="Preprints per year" width="340">
+</p>
 
 ## Table of Contents
 
@@ -26,8 +29,8 @@ and run simple postprocessing and plotting routines for meta-analysis.
 2. [Examples](#examples)
    - [Paper Keyword Analysis](#paper-keyword-analysis)
    - [PDF Retrieval](#pdf-retrieval)
-   - [Self-Citation Analysis](#self-citation-analysis)
    - [Scholar Metrics Analysis](#scholar-metrics-analysis)
+   - [Self-Citation Analysis](#self-citation-analysis)
 3. [Citation](#citation)
 4. [Contributors](#contributors)
 
@@ -63,7 +66,7 @@ uv run python -c "import paperscraper"
 However, to scrape publication data from the preprint servers [bioRxiv](https://www.biorxiv.org),
 [medRxiv](https://www.medrxiv.org/) and [chemRxiv](https://chemrxiv.org/), the setup is
 different. The entire history of papers is downloaded and stored in the `server_dumps`
-folder in JSONL format (one paper per line). This takes a while, as of November 2025:
+folder in JSONL format (one paper per line).
 
 ```py
 from paperscraper.get_dumps import biorxiv, medrxiv, chemrxiv
@@ -140,8 +143,8 @@ The README keeps examples short. The
 
 - [Paper Keyword Analysis](https://jannisborn.github.io/paperscraper/examples/paper-keyword-analysis/)
 - [PDF Retrieval](https://jannisborn.github.io/paperscraper/examples/pdf-retrieval/)
-- [Self-Citation Analysis](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/)
 - [Scholar Metrics Analysis](https://jannisborn.github.io/paperscraper/examples/scholar-metrics-analysis/)
+- [Self-Citation Analysis](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/)
 
 ### Paper keyword analysis
 
@@ -191,24 +194,6 @@ Output: `True` when the file was saved.
 See the [PDF retrieval example](https://jannisborn.github.io/paperscraper/examples/pdf-retrieval/)
 for batch downloads, fallbacks, publisher API keys, and downstream PDF analysis.
 
-### Self-citation analysis
-
-Estimate paper-level self-citations and self-references:
-
-```py
-from paperscraper.citations import self_citations_paper, self_references_paper
-
-doi = "10.1038/s41586-023-06600-9"
-self_citations_paper(doi).citation_score
-self_references_paper(doi).reference_score
-```
-
-Output: `3.192` and `5.05`, the mean self-citation and self-reference percentages
-across paper authors.
-
-See the [self-citation analysis example](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/)
-for paper- and author-level workflows using [Semantic Scholar](https://www.semanticscholar.org/).
-
 ### Scholar metrics analysis
 
 Get paper citation counts and journal metrics:
@@ -245,8 +230,38 @@ See the [scholar metrics analysis example](https://jannisborn.github.io/paperscr
 for [Google Scholar](https://scholar.google.com/), [Semantic Scholar](https://www.semanticscholar.org/),
 researcher metrics, and journal impact factors.
 
+### Self-citation analysis
+
+Estimate paper-level self-citations and self-references:
+
+```py
+from paperscraper.citations import self_citations_paper, self_references_paper
+
+doi = "10.1038/s41586-023-06600-9"
+self_citations_paper(doi).citation_score
+self_references_paper(doi).reference_score
+```
+
+Output: `3.192` and `5.05`, the mean self-citation and self-reference percentages
+across paper authors.
+
+The documentation example also includes a small researcher-level benchmark with
+self-citation and self-reference trends by discipline, career-stage group, and
+an overall average across disciplines:
+
+<p align="center">
+  <img src="assets/self_citation_grouped_by_discipline.png" alt="Self-citation ratios grouped by discipline" width="620">
+</p>
+
+<p align="center">
+  <img src="assets/self_reference_grouped_by_discipline.png" alt="Self-reference ratios grouped by discipline" width="620">
+</p>
+
+See the [self-citation analysis example](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/)
+for paper- and author-level workflows using [Semantic Scholar](https://www.semanticscholar.org/).
+
 ## Citation
-If you use `paperscraper`, please cite a paper that motivated our development of this tool.
+If you scrape papers with paperscraper, please cite the paperscraper paper :)
 
 ```bibtex
 @article{born2021trends,
@@ -261,6 +276,13 @@ If you use `paperscraper`, please cite a paper that motivated our development of
 }
 ```
 
+## Contributing and support
+
+Contribution guidelines are in [CONTRIBUTING.md](CONTRIBUTING.md), support
+expectations are in [SUPPORT.md](SUPPORT.md), and project decision-making is
+summarized in [GOVERNANCE.md](GOVERNANCE.md). Release notes are maintained with
+GitHub releases and PyPI release history rather than in a separate changelog.
+
 ## Contributors
 Thanks to the following contributors:
 

SUPPORT.md

@@ -0,0 +1,42 @@
+# Support
+
+The best place to get support is the public issue tracker:
+
+https://github.com/jannisborn/paperscraper/issues
+
+Before opening an issue, please check the documentation:
+
+https://jannisborn.github.io/paperscraper/
+
+## Bug Reports
+
+For bugs, include:
+
+- The installed `paperscraper` version.
+- The Python version and operating system.
+- A minimal code snippet or command that reproduces the issue.
+- The full traceback or error message.
+- Whether the issue depends on a specific external service, DOI, author ID, or
+  preprint server.
+
+## Usage Questions
+
+Questions about documented workflows are welcome. Please include enough context
+to reproduce the problem locally. For source-specific failures, note that
+external APIs, rate limits, captchas, publisher access rules, and platform terms
+can affect behavior independently of `paperscraper`.
+
+## Security and Credentials
+
+Do not post API keys, publisher credentials, AWS credentials, or private data in
+public issues. If a report requires private credentials or describes a sensitive
+security issue, contact the maintainers privately through their public GitHub or
+package metadata profiles and include only the minimum information needed to
+triage the issue.
+
+## Maintenance Scope
+
+The project can support package behavior, documented APIs, metadata parsing,
+retrieval fallbacks, and reproducible examples. It cannot guarantee access to
+paywalled content, bypass publisher restrictions, solve Google Scholar captchas,
+or control upstream API availability.

docs/examples/getting-started.md

@@ -0,0 +1,114 @@
+# Getting Started
+
+![Preprints per year](https://raw.githubusercontent.com/jannisborn/paperscraper/main/assets/preprints_per_year.png){ width="420" }
+
+The volume of scientific preprints is increasing, making reproducible
+meta-analyses and automated literature monitoring more useful. `paperscraper`
+helps turn searches over publication metadata into local, inspectable JSONL
+files that can be reused for downstream analysis.
+
+## Installation
+
+```console
+pip install paperscraper
+```
+
+or, with [uv](https://docs.astral.sh/uv/):
+
+```console
+uv add paperscraper
+```
+
+This is enough to query [PubMed](https://pubmed.ncbi.nlm.nih.gov/),
+[arXiv](https://arxiv.org/), or [Google Scholar](https://scholar.google.com/).
+
+## Local Development
+
+```console
+uv sync
+```
+
+This installs the project and dev tooling into `.venv`. Use `uv run` to execute
+commands, for example:
+
+```console
+uv run python -c "import paperscraper"
+```
+
+## Download xRxiv Dumps
+
+To scrape publication data from [bioRxiv](https://www.biorxiv.org/),
+[medRxiv](https://www.medrxiv.org/), and [chemRxiv](https://chemrxiv.org/), first
+download local metadata dumps. The entire history of papers is stored in the
+`server_dumps` folder in JSONL format, with one paper per line.
+
+```py
+from paperscraper.get_dumps import biorxiv, medrxiv, chemrxiv
+
+chemrxiv()  # Takes <15min -> +50K papers (~30 MB file)
+medrxiv()   # Takes <5min -> +100K papers (~200 MB file)
+biorxiv()   # Takes <1h -> +450K papers (~800 MB file)
+```
+
+After downloading dumps, restart the Python interpreter so that
+`paperscraper.load_dumps` can discover the new files.
+
+If you experience API connection issues, retries and request behavior can be
+tuned:
+
+```py
+biorxiv(
+    max_retries=12,
+    request_timeout=(5.0, 45.0),
+    retry_backoff_seconds=1.0,
+    max_workers=8,
+    window_days=30,
+)
+```
+
+You can also scrape xRxiv sources for specific dates:
+
+```py
+medrxiv(start_date="2023-04-01", end_date="2023-04-08")
+```
+
+The resulting `.jsonl` file is labelled according to the current date, and later
+local searches will use that file. Use
+`paperscraper.utils.get_server_dumps_dir()` to inspect the active dump
+directory.
+
+## arXiv Local Dump
+
+Local search can be faster than using the
+[arXiv API](https://info.arxiv.org/help/api/index.html), especially for many
+queries. `paperscraper` provides two arXiv dump backends: [Kaggle](https://www.kaggle.com/)
+and the [`arxiv`](https://pypi.org/project/arxiv/) package. The default is
+`kaggle` because it is much faster. Before using it, authenticate with your
+Kaggle account:
+
+```sh
+kaggle auth login
+```
+
+```py
+from paperscraper.get_dumps import arxiv
+
+arxiv(start_date="2019-01-01", end_date="2026-12-31")
+```
+
+The `kaggle` backend bulk-downloads all of arXiv. For small API-backed dumps,
+use the `api` backend:
+
+```py
+from paperscraper.get_dumps import arxiv
+
+arxiv(start_date="2024-01-01", end_date="2024-01-04", backend="api")
+```
+
+Afterwards you can search the local arXiv dump like the other xRxiv dumps:
+
+```py
+from paperscraper.arxiv import get_and_dump_arxiv_papers
+
+get_and_dump_arxiv_papers(..., backend="local")
+```

docs/examples/paper-keyword-analysis.md

@@ -81,7 +81,7 @@ papers = get_arxiv_papers_api('all:"quantum machine learning"', max_results=25)
 ## bioRxiv, medRxiv, and chemRxiv
 
 Download local xRxiv dumps once using the
-[setup instructions](../README.md#download-xrxiv-dumps), then restart Python so
+[setup instructions](getting-started.md#download-xrxiv-dumps), then restart Python so
 `paperscraper.load_dumps` can discover the new files.
 
 For local xRxiv dumps, use `XRXivQuery` directly: