Skip to content

Commit 4699893

Browse files
committed
feat: adopt src layout, harden resolution pipeline, expand test suite
- move the package to src/onecite with unchanged public module and CLI names - add candidate-suggestion flow, anti-hallucination benchmark, and doctor checks - expand offline test coverage and CI lint scope - add LICENSE.txt/Licence.txt aliases, uv.lock, and packaging cleanups
1 parent cbea632 commit 4699893

92 files changed

Lines changed: 8323 additions & 1338 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/docs.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
- name: Build documentation
3131
run: |
3232
cd docs
33-
sphinx-build -b html . _build/html
33+
sphinx-build -W -b html . _build/html
3434
3535
- name: Upload documentation artifacts
3636
uses: actions/upload-artifact@v4

.github/workflows/tests.yml

Lines changed: 44 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ jobs:
1313
fail-fast: false
1414
matrix:
1515
os: [ubuntu-latest, windows-latest, macos-latest]
16-
python-version: ['3.10', '3.11']
16+
python-version: ['3.10', '3.11', '3.12']
1717

1818
steps:
1919
- uses: actions/checkout@v4
@@ -30,7 +30,15 @@ jobs:
3030
3131
- name: Lint with flake8
3232
run: |
33-
flake8 onecite tests --statistics --count
33+
flake8 src/onecite tests --statistics --count
34+
35+
- name: Check formatting with Black
36+
run: |
37+
black --check src/onecite tests
38+
39+
- name: Type-check with mypy
40+
run: |
41+
mypy src/onecite
3442
3543
- name: Test with pytest
3644
run: |
@@ -39,6 +47,7 @@ jobs:
3947
- name: Run deterministic OneCite quality gates
4048
run: |
4149
onecite benchmark --json
50+
onecite benchmark --anti-hallucination --json
4251
onecite doctor --json
4352
4453
- name: Upload coverage to Codecov
@@ -50,3 +59,36 @@ jobs:
5059
flags: unittests
5160
name: codecov-umbrella
5261
fail_ci_if_error: false
62+
63+
build-and-install:
64+
runs-on: ubuntu-latest
65+
66+
steps:
67+
- uses: actions/checkout@v4
68+
69+
- name: Set up Python 3.12
70+
uses: actions/setup-python@v5
71+
with:
72+
python-version: '3.12'
73+
74+
- name: Build and validate distributions
75+
run: |
76+
python -m pip install --upgrade pip
77+
python -m pip install build twine
78+
python -m build --outdir dist
79+
python -m twine check dist/*
80+
tar -tf dist/onecite-*.tar.gz | grep -Eq '/LICENSE$'
81+
82+
- name: Test the built wheel outside the repository
83+
shell: bash
84+
run: |
85+
wheel_path="$(python -c 'from pathlib import Path; print(next(Path("dist").glob("*.whl")).resolve())')"
86+
clean_root="$(mktemp -d)"
87+
python -m venv "$clean_root/venv"
88+
"$clean_root/venv/bin/python" -m pip install --upgrade pip
89+
"$clean_root/venv/bin/python" -m pip install "$wheel_path"
90+
cd "$clean_root"
91+
"$clean_root/venv/bin/python" -c 'import onecite; assert onecite.__version__ == "0.2.0"'
92+
"$clean_root/venv/bin/onecite" doctor --json
93+
"$clean_root/venv/bin/onecite" benchmark --json
94+
"$clean_root/venv/bin/onecite" benchmark --anti-hallucination --json

.zenodo.json

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,32 @@
11
{
2-
"title": "OneCite: Citation Management and Academic Reference Toolkit",
3-
"description": "OneCite is a command-line tool and Python library that automates citation management. It accepts multiple input formats (DOI, arXiv ID, PMID, paper titles, etc.) and produces clean BibTeX entries.",
2+
"title": "OneCite: Auditable citation normalization for research workflows",
3+
"description": "OneCite is a command-line tool and Python library that normalizes messy scholarly references into auditable, source-resolved BibTeX and CSL-JSON. Strong identifiers follow documented metadata-service routes; ambiguous plain-text references are returned as ranked candidates for review rather than auto-promoted by the processing workflow.",
44
"creators": [
55
{
6-
"name": "He, Zhiang",
7-
"affiliation": "Independent Developer"
6+
"name": "He, Zhiang"
87
}
98
],
109
"keywords": [
11-
"citation management",
10+
"citation",
1211
"bibliography",
1312
"bibtex",
1413
"academic research",
15-
"reference management",
14+
"reference normalization",
1615
"doi",
1716
"arxiv",
17+
"reproducibility",
18+
"metadata",
1819
"python"
1920
],
2021
"license": "MIT",
22+
"version": "0.2.0",
2123
"upload_type": "software",
2224
"access_right": "open",
2325
"related_identifiers": [
2426
{
2527
"identifier": "https://github.com/HzaCode/OneCite",
2628
"relation": "isSupplementTo",
2729
"scheme": "url"
28-
},
29-
{
30-
"identifier": "https://pypi.org/project/onecite/",
31-
"relation": "isIdenticalTo",
32-
"scheme": "url"
3330
}
3431
]
3532
}
36-

CHANGELOG.md

Lines changed: 152 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,72 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
3131
JSON, doctor JSON, and template JSON envelopes used by automation and CI.
3232
- `build>=1.0` is now included in the development extra so the documented
3333
wheel build check works after `pip install -e ".[dev]"`.
34+
- `onecite benchmark --anti-hallucination`: a labelled, fully-offline evaluation
35+
of OneCite's non-fabrication property (resolution rate on real identifiers;
36+
non-fabrication rate on ambiguous text and fabricated DOIs; mismatch
37+
detection rate on real DOIs paired with a different paper's title), also
38+
exposed as `onecite.run_anti_hallucination_eval()`. Pipeline crashes are
39+
recorded as `error` and never count as correct.
40+
- Text/DOI consistency warning: when the descriptive text around a resolved
41+
DOI clearly describes a different work (the classic hallucinated title+DOI
42+
pairing), `process` still resolves from the authoritative DOI but attaches
43+
a non-blocking `text_metadata_mismatch` warning, surfaced in the report,
44+
the JSON/NDJSON envelopes, and the CLI summary. Consistency requires
45+
*positive* overlap — the text must contain most of the title's words or
46+
most of the authors' family names; character-level fuzz alone is not
47+
accepted as evidence (it has a blind spot for short titles like "Deep
48+
learning"). A year cited in the text that contradicts the resolved
49+
metadata by more than five years raises the required overlap further.
50+
- Auditable failure reports: failed entries now carry the original input
51+
text (`raw_text`) and a `reason` code (`doi_not_found`, `source_error`,
52+
`pmid_unresolved`, `isbn_unresolved`, `no_strong_identifier`, and more) so
53+
a safely rejected identifier, an ambiguous reference, and an unavailable
54+
source can be told apart; identification-stage failures are no longer
55+
mislabelled as `enrichment_failed`.
56+
- DOI-level deduplication: when the same work appears several times in a
57+
batch under different spellings (bare DOI, PMID, formatted citation), it
58+
is emitted once and the repeats are reported as `duplicates` (with the
59+
emitted entry's cite key) instead of re-emitted under suffixed keys.
60+
- `onecite benchmark --anti-hallucination` now refuses `--cases` and
61+
`--min-success-rate` instead of silently ignoring them.
62+
63+
- `onecite suggest` now searches arXiv directly alongside CrossRef and
64+
Semantic Scholar. arXiv covers the CS/ML venues that CrossRef does not
65+
index (e.g. NeurIPS/ICML), which previously made famous conference papers
66+
unfindable when Semantic Scholar was rate-limited; the title-field query
67+
is derived from the segment before the first comma of the citation.
68+
- CSL-JSON output: `onecite process --output-format csl-json` emits a valid
69+
CSL-JSON array (plain Unicode, structured author names, mapped item types)
70+
for downstream tools that consume CSL-JSON; a development fixture verifies
71+
Pandoc 3.10 consumption of representative emitted items, while Quarto,
72+
standalone citeproc, and reference-manager import workflows are not
73+
separately validated in this release;
74+
`process_references(output_format="csl-json")` returns one CSL item per
75+
result. Deduplication, warnings, and failure reporting apply unchanged.
76+
- Honest suggestion scoring: when the query explicitly cites a year, a
77+
candidate contradicting it by more than five years is penalized and
78+
flagged (`year_conflict` in `score_breakdown`) — same-title later works
79+
(commentaries, book chapters, reprints) no longer outrank on title
80+
similarity alone. Year and venue scores are now gated by title similarity,
81+
so unrelated works sharing only a year or a phrase fragment no longer
82+
climb the ranking.
83+
- `onecite suggest` now discloses source health: each suggestion carries a
84+
`sources` list with the status of the always-consulted scholarly indexes
85+
(CrossRef, Semantic Scholar) and per-source candidate counts. When a
86+
source is rate-limited or errors, the suggestion status becomes
87+
`candidates_found_incomplete`/`no_candidates_incomplete` and the CLI
88+
prints a notice — an incomplete candidate list is no longer presented as
89+
exhaustive. Semantic Scholar rate limits are retried once with backoff.
3490

3591
### Changed
92+
- Moved the importable package from the repository root to
93+
`src/onecite`, adopting the standard `src/` repository layout while
94+
retaining the public `onecite` module and CLI names.
95+
- Installed data files are now limited to artifacts with an actual consumer
96+
(the OneCite Skill file and the benchmark baseline, both located by
97+
`onecite doctor`). Documentation is no longer installed into
98+
`sys.prefix/share` — it ships in the source distribution and lives on the
99+
documentation site; the wheel drops from 56 to 33 files.
36100
- Default pytest runs now exclude live external-API checks;
37101
live checks are explicitly marked with `pytest.mark.live` so the
38102
default suite is deterministic and offline.
@@ -52,10 +116,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
52116
supports the current pyproject license contract.
53117
- Reformatted the package and tests with Black, removed stale imports and
54118
unused locals, aligned flake8 with Black for long lines, and made CI run
55-
the same `flake8 onecite tests` check documented in the OneCite
119+
the same `flake8 src/onecite tests` check documented in the OneCite
56120
Skill release checklist.
57121
- Aligned the OneCite Skill with the repository's actual Roadmap source
58-
in the `README.md` Roadmap section and the `flake8 onecite tests`
122+
in the `README.md` Roadmap section and the `flake8 src/onecite tests`
59123
validation check.
60124

61125
### Removed
@@ -79,6 +143,92 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/), and this
79143
strong-identifier-only contract of `process`.
80144

81145
### Fixed
146+
- The bundled DataCite and Zenodo fixtures used a dead Dryad DOI and a
147+
fabricated Zenodo title, so `benchmark --live` failed on real APIs even
148+
though the offline run passed. Both now mirror real, long-lived records
149+
(`10.5061/dryad.8515`; Zenodo `3233118` = nibabel 2.4.1), and the golden
150+
+ anti-hallucination suites both passed live in their then-current forms on
151+
2026-07-03. Those historical fixtures are regression smoke checks, not a
152+
general effectiveness estimate; the later 11-case development smoke suite
153+
is tracked separately. Live baseline records were added to
154+
`benchmarks/`.
155+
- The test-suite mock responses are now derived from the bundled offline
156+
fixtures instead of hand-maintaining a second copy of the same DOIs'
157+
metadata; the two copies had already drifted (`URL` and citation counts
158+
missing on one side, `ISSN` on the other) and consistency tests now pin
159+
the derivation.
160+
- The documentation changelog (`docs/changelog.rst`) had silently drifted
161+
from `CHANGELOG.md` (16 Unreleased entries missing) and the Unreleased
162+
section contained a duplicated `### Fixed` heading; both are fixed and
163+
the docs page now states that `CHANGELOG.md` is canonical.
164+
- `process_references` no longer requires the `interactive_callback`
165+
argument. The parameter was never invoked by the pipeline — `process` is
166+
strictly non-interactive — yet the signature forced every caller to pass a
167+
dummy lambda and the docs described selection behavior that did not exist.
168+
It is now optional, documented as never invoked, and retained only for
169+
backward compatibility; the Python API examples were corrected accordingly.
170+
- Google Books lookups now retry with backoff on HTTP 429/5xx instead of
171+
failing the ISBN entry on the first rate-limit response.
172+
- `onecite process --template` now rejects unknown template names with the
173+
list of valid presets instead of silently falling back to the default
174+
template; the Python-level `TemplateLoader` fallback logs a warning.
175+
- Explicitly labelled PMIDs embedded in citation text (e.g.
176+
`"Author (2015). Deep learning. PMID:26017442"`) now resolve, matching
177+
the long-standing behavior for DOIs embedded in text. Unlabelled numbers
178+
inside prose remain ambiguous and are not extracted.
179+
- The arXiv suggestion source now queries the `https` endpoint directly
180+
(the `http` URL 301-redirected on every call), retries transient
181+
429/5xx responses with a short backoff, and truthfully discloses a
182+
still-throttled source as `rate_limited` instead of `error`.
183+
- The test suite now fails loudly on any unmocked network call from a
184+
non-`live` test (autouse guard in `conftest.py`). This exposed unit
185+
tests that had been silently hitting live APIs whenever a suggestion
186+
source was missing from their mock list, and cut the full-suite runtime
187+
from ~43s to ~6s.
188+
- Eliminated a duplicate CrossRef API call per DOI entry: the work object
189+
fetched during DOI verification is now reused for enrichment instead of
190+
requesting the same DOI from CrossRef a second time — halving CrossRef
191+
load and saving up to a second of latency per entry in live runs.
192+
- Cite-key generation is now LaTeX-safe and crash-free: accented author
193+
names are ASCII-folded (Müller → Muller), characters with no ASCII form
194+
(CJK) are dropped instead of emitted into `\cite{...}` keys, and an
195+
integer `year` (as DataCite's `publicationYear` delivers) no longer
196+
raises `TypeError`.
197+
- CSL-JSON values now strip *all* LaTeX case-protection braces instead of
198+
mangling inner ones (`{ResNet}: ...` previously became the unbalanced
199+
`ResNet}: ...`), and BibTeX `--` page ranges are normalized to plain
200+
`-` as CSL expects.
201+
- An unwritable `--output` path no longer discards computed results or
202+
misattributes the IO failure as a processing failure: `process` and
203+
`suggest` now emit the computed output to stdout with a clear error on
204+
stderr and exit `1`, so expensive live-API work is never lost.
205+
- `onecite doctor` now verifies the bundled anti-hallucination dataset as
206+
part of the `benchmark_resources` check, so a broken install cannot
207+
report healthy benchmark resources while the core safety-evaluation
208+
asset is missing.
209+
- Removed the undocumented `sugget` CLI alias (a development-time typo
210+
for `suggest` that leaked into the subcommand list).
211+
- `onecite suggest --input-type bib` no longer sends a Python dict repr to
212+
the scholarly indexes when a BibTeX entry carries a DOI: the search query
213+
is now always built from the structured title/author/year fields.
214+
- OneCite's own BibTeX output now survives re-processing through
215+
`--input-type bib` byte-identically for all entry kinds, locked in by
216+
round-trip tests. Two defects were closed: bibtexparser silently dropped
217+
non-standard entry types, so OneCite could not re-parse its own
218+
`@software` entries; and a non-empty BibTeX file that parsed to zero
219+
entries produced an empty "success" (exit 0, empty output) instead of a
220+
loud `ParseError`.
221+
- The Sphinx documentation now builds with zero warnings (fixed broken
222+
section underlines in the changelog and FAQ, removed the nonexistent
223+
`_static` path and the deprecated `display_version` theme option), and
224+
the docs CI build runs with `-W` so new warnings fail the build.
225+
- The mypy configuration declared in `pyproject.toml` is now actually
226+
enforced: all 70 outstanding type errors were fixed and `mypy src/onecite`
227+
runs in CI. This closed several latent crash paths — an `HTTPError`
228+
handler that dereferenced a possibly-absent `response` object,
229+
`json.loads(None)` on empty structured-data script tags in publisher
230+
pages, a possibly-unset Google Books retry response, and unguarded
231+
`.get()` calls on possibly-`None` metadata dicts in the enricher.
82232
- Corrected the benchmark Nature DQN DOI fixture from
83233
`10.1038/nature14539` to `10.1038/nature14236`, and added regression
84234
coverage to catch future DOI-title-author mismatches in bundled

CITATION.cff

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
cff-version: 1.2.0
22
message: "If you use this software, please cite it as below."
33
type: software
4-
title: "OneCite: Citation Management and Academic Reference Toolkit"
4+
title: "OneCite: Auditable citation normalization for research workflows"
55
authors:
66
- family-names: "He"
77
given-names: "Zhiang"
88
email: "ang@hezhiang.com"
99
repository-code: "https://github.com/HzaCode/OneCite"
1010
url: "https://github.com/HzaCode/OneCite"
11-
abstract: "OneCite is a command-line tool and Python library that automates citation management. It accepts multiple input formats (DOI, arXiv ID, PMID, paper titles, etc.) and produces clean BibTeX entries."
11+
abstract: "OneCite is a command-line tool and Python library that normalizes messy scholarly references into auditable, source-resolved BibTeX and CSL-JSON. Strong identifiers follow documented metadata-service routes; ambiguous plain-text references are returned as ranked candidates for review rather than auto-promoted by the processing workflow."
1212
keywords:
1313
- citation
1414
- bibliography
@@ -19,6 +19,4 @@ keywords:
1919
- arxiv
2020
- research
2121
license: MIT
22-
version: "0.1.1"
23-
date-released: "2026-04-17"
24-
22+
version: "0.2.0"

CONTRIBUTING.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ pip install -e ".[dev]"
5151
pytest
5252

5353
# Check code style
54-
flake8 onecite tests
54+
flake8 src/onecite tests
5555
```
5656

5757
## How to Contribute
@@ -188,7 +188,7 @@ def example_function(param1: str, param2: int) -> bool:
188188
pytest
189189

190190
# Run specific test file
191-
pytest tests/test_core.py
191+
pytest tests/test_core_unit.py
192192

193193
# Run with coverage
194194
pytest --cov=onecite --cov-report=html
@@ -255,4 +255,3 @@ If you have questions or need help:
255255
- Email ang@hezhiang.com
256256

257257
Thank you for contributing to OneCite! 🎉
258-

LICENSE.txt

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
MIT License
2+
3+
Copyright (c) 2024-2025 He Zhiang
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.
22+

Licence.txt

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
MIT License
2+
3+
Copyright (c) 2024-2025 He Zhiang
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.
22+

0 commit comments

Comments
 (0)