diff --git a/README.md b/README.md index 9797eba..a1a9f7a 100644 --- a/README.md +++ b/README.md @@ -17,10 +17,6 @@ It provides a streamlined interface to scrape metadata, retrieve citation counts from [Google Scholar](https://scholar.google.com/), query journal impact factors, and run simple postprocessing and plotting routines for meta-analysis. -

- Preprints per year -

- ## Table of Contents 1. [Getting Started](#getting-started) diff --git a/docs/README.md b/docs/README.md deleted file mode 120000 index 32d46ee..0000000 --- a/docs/README.md +++ /dev/null @@ -1 +0,0 @@ -../README.md \ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..e7a8044 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,89 @@ +[![build](https://github.com/jannisborn/paperscraper/actions/workflows/test_tip.yml/badge.svg?branch=main)](https://github.com/jannisborn/paperscraper/actions/workflows/test_tip.yml?query=branch%3Amain) +[![build](https://github.com/jannisborn/paperscraper/actions/workflows/test_pypi.yml/badge.svg?branch=main)](https://github.com/jannisborn/paperscraper/actions/workflows/test_pypi.yml?query=branch%3Amain) +[![build](https://github.com/jannisborn/paperscraper/actions/workflows/docs.yml/badge.svg?branch=main)](https://jannisborn.github.io/paperscraper/) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![PyPI version](https://badge.fury.io/py/paperscraper.svg)](https://badge.fury.io/py/paperscraper) +[![Downloads](https://static.pepy.tech/badge/paperscraper)](https://pepy.tech/project/paperscraper) +[![codecov](https://codecov.io/github/jannisborn/paperscraper/branch/main/graph/badge.svg?token=Clwi0pu61a)](https://codecov.io/github/jannisborn/paperscraper) + +# paperscraper + +`paperscraper` is a Python package for reproducible searches over scholarly +metadata, accessible full-text retrieval, citation lookup, and small +bibliometric workflows across PubMed, arXiv, bioRxiv, medRxiv, and ChemRxiv. + +```sh +pip install paperscraper +``` + +or: + +```sh +uv add paperscraper +``` + +## What It Does + +
+ +- **Search scholarly metadata** + + Query PubMed and arXiv directly, or search local JSONL dumps from arXiv, + bioRxiv, medRxiv, and ChemRxiv with one nested keyword convention. + + [:octicons-arrow-right-24: Paper keyword analysis](examples/paper-keyword-analysis.md) + +- **Build local preprint dumps** + + Download local xRxiv snapshots once, then run reproducible repeated searches + without depending on live search results for every query. + + [:octicons-arrow-right-24: Getting started](examples/getting-started.md) + +- **Retrieve accessible full text** + + Save PDFs or XML from DOI metadata using direct links and supported fallback + paths when access is available. + + [:octicons-arrow-right-24: PDF retrieval](examples/pdf-retrieval.md) + +- **Inspect citation behavior** + + Query citation counts, author metrics, journal impact factors, and + paper-level or researcher-level self-citation and self-reference rates. + + [:octicons-arrow-right-24: Scholar metrics](examples/scholar-metrics-analysis.md) + + [:octicons-arrow-right-24: Self-citation analysis](examples/self-citation-analysis.md) + +
+ +## Quick Example + +```py +from paperscraper import dump_queries + +ai = ["Artificial intelligence", "Machine learning"] +qc = ["Quantum computing", "Quantum information", "Quantum algorithm"] +chemistry = ["Chemistry", "Chemical", "Molecule", "Materials science"] + +dump_queries([[ai, qc, chemistry]], ".") +``` + +Nested lists encode Boolean logic: outer lists are combined with `AND`, while +inner lists define synonyms combined with `OR`. + +## Where To Go Next + +- Start with [Getting Started](examples/getting-started.md) for installation and + local dump setup. +- Use [Paper Keyword Analysis](examples/paper-keyword-analysis.md) for + multi-source literature trend workflows. +- Use [PDF Retrieval](examples/pdf-retrieval.md) for full-text download options + and supported fallbacks. +- Use [Scholar Metrics Analysis](examples/scholar-metrics-analysis.md) for + citation counts, author metrics, and journal metrics. +- Use [Self-Citation Analysis](examples/self-citation-analysis.md) for + self-citation and self-reference workflows. + +API details are available under [API Documentation](api/index.md). diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index 8fe558b..0000000 --- a/docs/index.md +++ /dev/null @@ -1,19 +0,0 @@ -# paperscraper - -`paperscraper` is a Python package for scraping publication metadata and full -text files from [PubMed](https://pubmed.ncbi.nlm.nih.gov/), [arXiv](https://arxiv.org/), -[medRxiv](https://www.medrxiv.org/), [bioRxiv](https://www.biorxiv.org/), and -[chemRxiv](https://chemrxiv.org/). - -It provides a streamlined interface to query publication metadata, retrieve -citation counts, search journal metrics, download accessible PDFs or XML files, -and run postprocessing workflows for literature meta-analysis. - -For installation and local dump setup, start with the -[Getting Started example](examples/getting-started.md). - -For common workflows, see the examples for -[paper keyword analysis](examples/paper-keyword-analysis.md), -[PDF retrieval](examples/pdf-retrieval.md), -[scholar metrics analysis](examples/scholar-metrics-analysis.md), and -[self-citation analysis](examples/self-citation-analysis.md). diff --git a/mkdocs.yml b/mkdocs.yml index dc4cc50..6ae8b20 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -49,7 +49,7 @@ theme: language: en nav: - - Overview: index.md + - Overview: README.md - API Documentation: - Overview: api/index.md - paperscraper: