diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..1934fb5 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,60 @@ +# Contributing + +Thanks for considering a contribution to `paperscraper`. Contributions are +welcome when they improve reliability, documentation, tests, supported scholarly +metadata sources, or the ergonomics of existing workflows. + +## Ways to Contribute + +- Report bugs with a minimal reproducible example. +- Improve documentation, examples, or error messages. +- Add tests for existing behavior. +- Fix source-specific API breakage. +- Propose new metadata sources or retrieval fallbacks. + +For larger changes, please open an issue first so the design, maintenance cost, +and API surface can be discussed before implementation. + +## Development Setup + +The project uses `uv` for local development: + +```sh +uv sync --group dev +uv run python -c "import paperscraper" +``` + +Run formatting, linting, and tests before opening a pull request: + +```sh +uv run ruff format paperscraper +uv run ruff check paperscraper +uv run isort paperscraper +uv run pytest paperscraper +``` + +Some tests exercise external scholarly services and can be slow or sensitive to +rate limits. The GitHub Actions test suite is the source of truth for release +readiness; it can take a long time because it verifies source and wheel installs +against the supported workflows. + +## Pull Request Expectations + +- Keep changes focused and avoid unrelated refactors. +- Add or update tests when changing behavior. +- Update documentation and examples when changing user-facing APIs. +- Do not commit API keys, credentials, downloaded server dumps, PDFs, or other + large generated artifacts. +- Preserve backward compatibility unless a breaking change has been discussed. + +## Release Notes + +This project does not maintain a separate `CHANGELOG.md`. Release notes are kept +with versioned GitHub releases and PyPI release history. Maintainers summarize +notable changes there when publishing a new package version. + +## Conduct + +Please keep issues and pull requests respectful, technical, and actionable. The +maintainers may close or moderate interactions that are abusive, off-topic, or +not aligned with the project scope. diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 0000000..5263986 --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,53 @@ +# Governance + +`paperscraper` is maintained as an open-source research software project. Its +scope is to provide practical tools for scholarly metadata collection, +publication search, full-text retrieval where access is available, citation +analysis, and related reproducible examples. + +## Maintainers + +The project maintainers are responsible for reviewing contributions, deciding on +API changes, publishing releases, and keeping the package aligned with its +research-software scope. The current maintainers are the package authors listed +in `pyproject.toml` and on PyPI. + +## Decision Making + +Small fixes can be reviewed and merged by a maintainer when tests and +documentation are adequate. Larger changes, new dependencies, new data sources, +or compatibility-breaking changes should be discussed in an issue before a pull +request is opened. + +Maintainers aim for consensus. If consensus is not possible, final decisions are +made by the active maintainers based on project scope, maintenance burden, +backward compatibility, testability, and benefit to users. + +## Contribution Review + +Pull requests are evaluated for: + +- Correctness and reproducibility. +- Compatibility with existing public APIs. +- Test coverage appropriate to the change. +- Documentation for user-facing behavior. +- Respect for source platform terms, publisher restrictions, and credential + safety. + +Maintainers may request changes, split large pull requests, or decline changes +that would add substantial maintenance cost without clear benefit to the core +use cases. + +## Releases + +Releases are made by maintainers when a set of changes is ready for users. The +project uses versioned Git tags and publishes packages to PyPI. Release notes are +provided through GitHub releases and PyPI release history rather than a separate +changelog file. + +## Project Direction + +Priorities are guided by user reports, contributor interest, upstream API +changes, documentation quality, and the needs of reproducible literature and +bibliometric analyses. The project favors reliable, well-documented workflows +over broad but weakly maintained source coverage. diff --git a/README.md b/README.md index 9833b5d..9797eba 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,9 @@ It provides a streamlined interface to scrape metadata, retrieve citation counts from [Google Scholar](https://scholar.google.com/), query journal impact factors, and run simple postprocessing and plotting routines for meta-analysis. +

+ Preprints per year +

## Table of Contents @@ -26,8 +29,8 @@ and run simple postprocessing and plotting routines for meta-analysis. 2. [Examples](#examples) - [Paper Keyword Analysis](#paper-keyword-analysis) - [PDF Retrieval](#pdf-retrieval) - - [Self-Citation Analysis](#self-citation-analysis) - [Scholar Metrics Analysis](#scholar-metrics-analysis) + - [Self-Citation Analysis](#self-citation-analysis) 3. [Citation](#citation) 4. [Contributors](#contributors) @@ -63,7 +66,7 @@ uv run python -c "import paperscraper" However, to scrape publication data from the preprint servers [bioRxiv](https://www.biorxiv.org), [medRxiv](https://www.medrxiv.org/) and [chemRxiv](https://chemrxiv.org/), the setup is different. The entire history of papers is downloaded and stored in the `server_dumps` -folder in JSONL format (one paper per line). This takes a while, as of November 2025: +folder in JSONL format (one paper per line). ```py from paperscraper.get_dumps import biorxiv, medrxiv, chemrxiv @@ -140,8 +143,8 @@ The README keeps examples short. The - [Paper Keyword Analysis](https://jannisborn.github.io/paperscraper/examples/paper-keyword-analysis/) - [PDF Retrieval](https://jannisborn.github.io/paperscraper/examples/pdf-retrieval/) -- [Self-Citation Analysis](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/) - [Scholar Metrics Analysis](https://jannisborn.github.io/paperscraper/examples/scholar-metrics-analysis/) +- [Self-Citation Analysis](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/) ### Paper keyword analysis @@ -191,24 +194,6 @@ Output: `True` when the file was saved. See the [PDF retrieval example](https://jannisborn.github.io/paperscraper/examples/pdf-retrieval/) for batch downloads, fallbacks, publisher API keys, and downstream PDF analysis. -### Self-citation analysis - -Estimate paper-level self-citations and self-references: - -```py -from paperscraper.citations import self_citations_paper, self_references_paper - -doi = "10.1038/s41586-023-06600-9" -self_citations_paper(doi).citation_score -self_references_paper(doi).reference_score -``` - -Output: `3.192` and `5.05`, the mean self-citation and self-reference percentages -across paper authors. - -See the [self-citation analysis example](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/) -for paper- and author-level workflows using [Semantic Scholar](https://www.semanticscholar.org/). - ### Scholar metrics analysis Get paper citation counts and journal metrics: @@ -245,8 +230,38 @@ See the [scholar metrics analysis example](https://jannisborn.github.io/paperscr for [Google Scholar](https://scholar.google.com/), [Semantic Scholar](https://www.semanticscholar.org/), researcher metrics, and journal impact factors. +### Self-citation analysis + +Estimate paper-level self-citations and self-references: + +```py +from paperscraper.citations import self_citations_paper, self_references_paper + +doi = "10.1038/s41586-023-06600-9" +self_citations_paper(doi).citation_score +self_references_paper(doi).reference_score +``` + +Output: `3.192` and `5.05`, the mean self-citation and self-reference percentages +across paper authors. + +The documentation example also includes a small researcher-level benchmark with +self-citation and self-reference trends by discipline, career-stage group, and +an overall average across disciplines: + +

+ Self-citation ratios grouped by discipline +

+ +

+ Self-reference ratios grouped by discipline +

+ +See the [self-citation analysis example](https://jannisborn.github.io/paperscraper/examples/self-citation-analysis/) +for paper- and author-level workflows using [Semantic Scholar](https://www.semanticscholar.org/). + ## Citation -If you use `paperscraper`, please cite a paper that motivated our development of this tool. +If you scrape papers with paperscraper, please cite the paperscraper paper :) ```bibtex @article{born2021trends, @@ -261,6 +276,13 @@ If you use `paperscraper`, please cite a paper that motivated our development of } ``` +## Contributing and support + +Contribution guidelines are in [CONTRIBUTING.md](CONTRIBUTING.md), support +expectations are in [SUPPORT.md](SUPPORT.md), and project decision-making is +summarized in [GOVERNANCE.md](GOVERNANCE.md). Release notes are maintained with +GitHub releases and PyPI release history rather than in a separate changelog. + ## Contributors Thanks to the following contributors: diff --git a/SUPPORT.md b/SUPPORT.md new file mode 100644 index 0000000..f8284b8 --- /dev/null +++ b/SUPPORT.md @@ -0,0 +1,42 @@ +# Support + +The best place to get support is the public issue tracker: + +https://github.com/jannisborn/paperscraper/issues + +Before opening an issue, please check the documentation: + +https://jannisborn.github.io/paperscraper/ + +## Bug Reports + +For bugs, include: + +- The installed `paperscraper` version. +- The Python version and operating system. +- A minimal code snippet or command that reproduces the issue. +- The full traceback or error message. +- Whether the issue depends on a specific external service, DOI, author ID, or + preprint server. + +## Usage Questions + +Questions about documented workflows are welcome. Please include enough context +to reproduce the problem locally. For source-specific failures, note that +external APIs, rate limits, captchas, publisher access rules, and platform terms +can affect behavior independently of `paperscraper`. + +## Security and Credentials + +Do not post API keys, publisher credentials, AWS credentials, or private data in +public issues. If a report requires private credentials or describes a sensitive +security issue, contact the maintainers privately through their public GitHub or +package metadata profiles and include only the minimum information needed to +triage the issue. + +## Maintenance Scope + +The project can support package behavior, documented APIs, metadata parsing, +retrieval fallbacks, and reproducible examples. It cannot guarantee access to +paywalled content, bypass publisher restrictions, solve Google Scholar captchas, +or control upstream API availability. diff --git a/assets/preprints_per_year.png b/assets/preprints_per_year.png new file mode 100644 index 0000000..18a12b1 Binary files /dev/null and b/assets/preprints_per_year.png differ diff --git a/assets/self_citation_grouped_by_discipline.png b/assets/self_citation_grouped_by_discipline.png new file mode 100644 index 0000000..3d71fe0 Binary files /dev/null and b/assets/self_citation_grouped_by_discipline.png differ diff --git a/assets/self_reference_grouped_by_discipline.png b/assets/self_reference_grouped_by_discipline.png new file mode 100644 index 0000000..e5e15ee Binary files /dev/null and b/assets/self_reference_grouped_by_discipline.png differ diff --git a/docs/examples/getting-started.md b/docs/examples/getting-started.md new file mode 100644 index 0000000..1ef0d0f --- /dev/null +++ b/docs/examples/getting-started.md @@ -0,0 +1,114 @@ +# Getting Started + +![Preprints per year](https://raw.githubusercontent.com/jannisborn/paperscraper/main/assets/preprints_per_year.png){ width="420" } + +The volume of scientific preprints is increasing, making reproducible +meta-analyses and automated literature monitoring more useful. `paperscraper` +helps turn searches over publication metadata into local, inspectable JSONL +files that can be reused for downstream analysis. + +## Installation + +```console +pip install paperscraper +``` + +or, with [uv](https://docs.astral.sh/uv/): + +```console +uv add paperscraper +``` + +This is enough to query [PubMed](https://pubmed.ncbi.nlm.nih.gov/), +[arXiv](https://arxiv.org/), or [Google Scholar](https://scholar.google.com/). + +## Local Development + +```console +uv sync +``` + +This installs the project and dev tooling into `.venv`. Use `uv run` to execute +commands, for example: + +```console +uv run python -c "import paperscraper" +``` + +## Download xRxiv Dumps + +To scrape publication data from [bioRxiv](https://www.biorxiv.org/), +[medRxiv](https://www.medrxiv.org/), and [chemRxiv](https://chemrxiv.org/), first +download local metadata dumps. The entire history of papers is stored in the +`server_dumps` folder in JSONL format, with one paper per line. + +```py +from paperscraper.get_dumps import biorxiv, medrxiv, chemrxiv + +chemrxiv() # Takes <15min -> +50K papers (~30 MB file) +medrxiv() # Takes <5min -> +100K papers (~200 MB file) +biorxiv() # Takes <1h -> +450K papers (~800 MB file) +``` + +After downloading dumps, restart the Python interpreter so that +`paperscraper.load_dumps` can discover the new files. + +If you experience API connection issues, retries and request behavior can be +tuned: + +```py +biorxiv( + max_retries=12, + request_timeout=(5.0, 45.0), + retry_backoff_seconds=1.0, + max_workers=8, + window_days=30, +) +``` + +You can also scrape xRxiv sources for specific dates: + +```py +medrxiv(start_date="2023-04-01", end_date="2023-04-08") +``` + +The resulting `.jsonl` file is labelled according to the current date, and later +local searches will use that file. Use +`paperscraper.utils.get_server_dumps_dir()` to inspect the active dump +directory. + +## arXiv Local Dump + +Local search can be faster than using the +[arXiv API](https://info.arxiv.org/help/api/index.html), especially for many +queries. `paperscraper` provides two arXiv dump backends: [Kaggle](https://www.kaggle.com/) +and the [`arxiv`](https://pypi.org/project/arxiv/) package. The default is +`kaggle` because it is much faster. Before using it, authenticate with your +Kaggle account: + +```sh +kaggle auth login +``` + +```py +from paperscraper.get_dumps import arxiv + +arxiv(start_date="2019-01-01", end_date="2026-12-31") +``` + +The `kaggle` backend bulk-downloads all of arXiv. For small API-backed dumps, +use the `api` backend: + +```py +from paperscraper.get_dumps import arxiv + +arxiv(start_date="2024-01-01", end_date="2024-01-04", backend="api") +``` + +Afterwards you can search the local arXiv dump like the other xRxiv dumps: + +```py +from paperscraper.arxiv import get_and_dump_arxiv_papers + +get_and_dump_arxiv_papers(..., backend="local") +``` diff --git a/docs/examples/paper-keyword-analysis.md b/docs/examples/paper-keyword-analysis.md index a147d17..b773c81 100644 --- a/docs/examples/paper-keyword-analysis.md +++ b/docs/examples/paper-keyword-analysis.md @@ -81,7 +81,7 @@ papers = get_arxiv_papers_api('all:"quantum machine learning"', max_results=25) ## bioRxiv, medRxiv, and chemRxiv Download local xRxiv dumps once using the -[setup instructions](../README.md#download-xrxiv-dumps), then restart Python so +[setup instructions](getting-started.md#download-xrxiv-dumps), then restart Python so `paperscraper.load_dumps` can discover the new files. For local xRxiv dumps, use `XRXivQuery` directly: diff --git a/docs/examples/self-citation-analysis.md b/docs/examples/self-citation-analysis.md index ddd8dae..cf3d033 100644 --- a/docs/examples/self-citation-analysis.md +++ b/docs/examples/self-citation-analysis.md @@ -1,42 +1,64 @@ # Self-Citation Analysis -The citations submodule uses Semantic Scholar paper, citation, and reference -metadata to estimate how often authors cite or reference their own work. Set -`SS_API_KEY` to increase throughput: +Citation counts and h-index values are useful summary statistics, but they can +be blurred by excessive self-citation. `paperscraper` helps inspect this effect +with paper-level and author-level self-citation and self-reference analyses. + +The citation workflow uses [Semantic Scholar](https://www.semanticscholar.org/) +paper, citation, and reference metadata. It can run without authentication, but +larger analyses should use a [Semantic Scholar API key](https://www.semanticscholar.org/product/api): ```sh export SS_API_KEY=YOUR_API_KEY ``` -Citation and reference counts can change as Semantic Scholar updates its records. - -## Paper-Level Self-Links +## Self-References of a Paper -Estimate self-citations and self-references for a paper: +Start from a paper DOI or Semantic Scholar paper ID. Self-references ask: among +the papers referenced by this paper, how many include one of the paper's own +authors? ```pycon ->>> from paperscraper.citations import self_citations_paper, self_references_paper +>>> from paperscraper.citations import self_references_paper >>> doi = "10.1038/s41586-023-06600-9" ->>> self_citations = self_citations_paper(doi) ->>> self_citations.num_citations -141 # Total citations. ->>> self_citations.citation_score -3.192 # Mean self-citation percentage across paper authors. >>> self_references = self_references_paper(doi) >>> self_references.num_references -33 # Total references. +33 >>> self_references.reference_score -5.05 # Mean self-reference percentage across paper authors. +5.05 +``` + +The score is the mean self-reference percentage across the authors of the paper. +The per-author values are available directly: + +```pycon +>>> self_references.self_references +{ + "Abhishek Sharma": 3.03, + "Dániel Czégel": 0.0, + "Michael Lachmann": 0.0, + "C. Kempes": 0.0, + "S. I. Walker": 6.06, + "Leroy Cronin": 21.21, +} ``` -Both functions accept either one DOI/Semantic Scholar paper ID or a list. A -single input returns one result object; a list returns a list of result objects. +Here, `21.21` means that 21.21% of the paper's references include Leroy Cronin +as an author. -## Author Breakdown +## Self-Citations of a Paper -Print the per-author self-link percentages: +The same idea can be turned around. Instead of looking at the references made by +the paper, self-citations look at later papers that cite it and ask how often +those citing papers include one of the original paper's authors. ```pycon +>>> from paperscraper.citations import self_citations_paper +>>> self_citations = self_citations_paper(doi) +>>> self_citations.num_citations +141 +>>> self_citations.citation_score +3.192 >>> self_citations.self_citations { "Abhishek Sharma": 3.55, @@ -45,21 +67,23 @@ Print the per-author self-link percentages: "C. Kempes": 3.55, "S. I. Walker": 4.96, "Leroy Cronin": 4.96, -} # Percentage of citations that include each paper author. ->>> self_references.self_references -{ - "Abhishek Sharma": 3.03, - "Dániel Czégel": 0.0, - "Michael Lachmann": 0.0, - "C. Kempes": 0.0, - "S. I. Walker": 6.06, - "Leroy Cronin": 21.21, -} # Percentage of references that include each paper author. +} ``` -## Author-Level Summary +Here, `4.96` means that 4.96% of the papers citing the focal paper include +Leroy Cronin as an author. + +Both `self_references_paper` and `self_citations_paper` accept either one +DOI/Semantic Scholar paper ID or a list. A single input returns one result +object; a list returns a list of result objects. + +## Researcher-Level Tendencies + +Now imagine doing this for every paper of an author. This gives a researcher- +level tendency for self-references and self-citations, aggregated over that +author's publication record. -For an author-level summary, use `Researcher`. Full author analyses can take +For author-level workflows, use `Researcher`. Full author analyses can take longer for large publication lists, so this example limits the run to one paper. ```pycon @@ -71,36 +95,147 @@ longer for large publication lists, so this example limits the run to one paper. >>> result.name "K. Wijk" >>> result.self_citation_ratio -0.0 # Mean self-citation percentage across the selected papers. +0.0 >>> result.self_reference_ratio -4.65 # Mean self-reference percentage across the selected papers. +4.65 >>> result.num_citations 10 >>> result.num_references 43 >>> result.self_references {"Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 4.65} +>>> result +ResearcherResult( + name="K. Wijk", + self_reference_ratio=4.65, + self_citation_ratio=0.0, + num_references=43, + num_citations=10, + self_references={ + "Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 4.65, + }, + self_citations={ + "Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 0.0, + }, + ssaid=2289839817, + orcid=None, +) ``` -## Unified Paper Interface - -Use `SelfLinkClient` when you want self-citations and self-references through one -paper-level object. Paper inputs can be DOIs, Semantic Scholar paper IDs, or -titles; use `mode` when you want to disambiguate. +The example above uses Klaas Wijk's Semantic Scholar author profile and pins the +analysis to one paper to keep the runtime short. + +## Researcher-per-Discipline Self-Link Analysis + +The plots below summarize a small benchmark analysis of self-links grouped by +discipline and researcher category. The processed table uses three categories: +historic awardees (`PastSenior`) for Nobel Prize, Fields Medal, and Turing +Award winners before 2000, represented here by 1993-1998 laureates; recent +awardees (`TodaySenior`) for comparable senior prize winners since 2018; and +rising researchers (`TodayJunior`) for 2025 rising-star researchers selected as +plausible future candidates for such prizes. + +The plots show mean ± standard deviation by discipline and category. Individual +researchers are overlaid as points, and the separated `Overall` block averages +the five discipline-level means for each researcher cohort. + +![Self-citation ratios grouped by discipline](https://raw.githubusercontent.com/jannisborn/paperscraper/main/assets/self_citation_grouped_by_discipline.png){ width="760" } + +![Self-reference ratios grouped by discipline](https://raw.githubusercontent.com/jannisborn/paperscraper/main/assets/self_reference_grouped_by_discipline.png){ width="760" } + +The strongest statistically supported signal is in self-citations rather than +self-references. A discipline-stratified permutation test finds that +rising researchers have higher self-citation ratios than the other two groups by +5.79 percentage points (`p < 0.0001`, Benjamini-Hochberg `q = 0.0010`). +Pairwise Mann-Whitney checks show the same pattern against both historic +awardees (`p = 0.0007`) and recent awardees (`p = 0.0044`). + +Here, "percentage points" means an absolute difference in percentages: a change +from 5% to 10% is +5 percentage points. The `q` value is the p-value after +Benjamini-Hochberg correction across the tested career-stage and discipline +contrasts. "Discipline-stratified" means that the class labels are shuffled only +within each discipline, so the junior effect is not merely a consequence of one +discipline having both more junior researchers and a different baseline +self-citation rate. + +Across disciplines, mathematics has the clearest effect: after stratifying by +career-stage group, mathematicians have higher self-citation ratios than the +other disciplines by 4.91 percentage points (`p = 0.0020`, `q = 0.0245`). +The visually plausible self-reference trends are weaker. Rising researchers +have lower, not higher, self-reference ratios by 4.36 +percentage points, but this is only nominal after correction (`p = 0.0121`, +`q = 0.0724`). Pooling self-citations and self-references into a combined +standardized score over the two rates does not produce a robust class or +discipline result. + +These results should be read as associations, not causal claims. The higher +rising-researcher self-citation ratio may partly reflect shorter citation windows: +younger papers have had less time to accumulate independent citations, so early +citations may contain a larger fraction of follow-up work from the same authors. +The aggregate benchmark controls for discipline composition in the tests above, +but it does not model paper age, citation-window length, or field-specific +citation velocity. + +## Unified Interface + +Use `SelfLinkClient` when you want self-citations and self-references through +one object. Paper inputs can be DOIs, Semantic Scholar paper IDs, or titles; use +`mode` when you want to disambiguate. ```pycon >>> from paperscraper.citations import SelfLinkClient ->>> client = SelfLinkClient("10.1038/s41586-023-06600-9", mode="paper") ->>> client.extract() ->>> result = client.get_result() ->>> result.title -"Assembly theory explains and quantifies selection and evolution" ->>> result.citation_score -3.192 # Mean self-citation percentage across paper authors. ->>> result.reference_score -5.05 # Mean self-reference percentage across paper authors. +>>> paper_client = SelfLinkClient("10.1038/s41586-023-06600-9", mode="paper") +>>> paper_client.extract() +>>> paper_client.get_result() +PaperResult( + ssid="10.1038/s41586-023-06600-9", + title="Assembly theory explains and quantifies selection and evolution", + num_citations=141, + self_citations={ + "Abhishek Sharma": 3.55, + "Dániel Czégel": 0.71, + "Michael Lachmann": 1.42, + "C. Kempes": 3.55, + "S. I. Walker": 4.96, + "Leroy Cronin": 4.96, + }, + citation_score=3.192, + num_references=33, + self_references={ + "Abhishek Sharma": 3.03, + "Dániel Czégel": 0.0, + "Michael Lachmann": 0.0, + "C. Kempes": 0.0, + "S. I. Walker": 6.06, + "Leroy Cronin": 21.21, + }, + reference_score=5.05, +) ``` -`SelfLinkClient(..., mode="author")` is also available for author-level runs. -Use `Researcher` directly when you want to limit the paper list before running, -as shown in the author-level example above. +`SelfLinkClient(..., mode="author")` wraps the same `Researcher` object used +above and returns a `ResearcherResult`. Use `Researcher` directly when you want +to limit or inspect the paper list before running the analysis; the example below +does the same pinning through the client for compact output. + +```pycon +>>> author_client = SelfLinkClient("2289839817", mode="author") +>>> author_client.object.ssids = ["2c1edb95c07643a834c9d4f8f2acedfecfe894de"] +>>> author_client.extract() +>>> author_client.get_result() +ResearcherResult( + name="K. Wijk", + self_reference_ratio=4.65, + self_citation_ratio=0.0, + num_references=43, + num_citations=10, + self_references={ + "Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 4.65, + }, + self_citations={ + "Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 0.0, + }, + ssaid=2289839817, + orcid=None, +) +``` diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..8fe558b --- /dev/null +++ b/docs/index.md @@ -0,0 +1,19 @@ +# paperscraper + +`paperscraper` is a Python package for scraping publication metadata and full +text files from [PubMed](https://pubmed.ncbi.nlm.nih.gov/), [arXiv](https://arxiv.org/), +[medRxiv](https://www.medrxiv.org/), [bioRxiv](https://www.biorxiv.org/), and +[chemRxiv](https://chemrxiv.org/). + +It provides a streamlined interface to query publication metadata, retrieve +citation counts, search journal metrics, download accessible PDFs or XML files, +and run postprocessing workflows for literature meta-analysis. + +For installation and local dump setup, start with the +[Getting Started example](examples/getting-started.md). + +For common workflows, see the examples for +[paper keyword analysis](examples/paper-keyword-analysis.md), +[PDF retrieval](examples/pdf-retrieval.md), +[scholar metrics analysis](examples/scholar-metrics-analysis.md), and +[self-citation analysis](examples/self-citation-analysis.md). diff --git a/mkdocs.yml b/mkdocs.yml index 44ced1c..dc4cc50 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -49,7 +49,7 @@ theme: language: en nav: - - Overview: README.md + - Overview: index.md - API Documentation: - Overview: api/index.md - paperscraper: @@ -62,10 +62,11 @@ nav: - paperscraper.scholar: api/scholar.md - paperscraper.xrxiv: api/xrxiv.md - Examples: + - Getting Started: examples/getting-started.md - Paper Keyword Analysis: examples/paper-keyword-analysis.md - PDF Retrieval: examples/pdf-retrieval.md - - Self-Citation Analysis: examples/self-citation-analysis.md - Scholar Metrics Analysis: examples/scholar-metrics-analysis.md + - Self-Citation Analysis: examples/self-citation-analysis.md # Copyright copyright: MIT License