feat: Domain config through file, Sandhi improvements, and Kannada numeral supportFeature domain file - #15
Merged
Merged
Conversation
- Add factory methods for domain configs (DomainConfig.legal(), medical(), financial(), technical()) - Bundle config files in package (src/dicterrors/config/) - Add 200+ legal terms (English abbreviations, Malayalam terms, Latin phrases) - Fix word boundary bug preventing substring matches - Support multi-word phrases as single tokens (Police Station, Prima facie) - Update all examples and docs to use new factory methods Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…t agnostic word boundary definition
- Add normalize_kannada_numerals() function to convert Kannada digits (೦-೯) to Arabic numerals for semantic matching - Fix correct token aggregation in measure_batch.py (properly accumulate correct counts instead of calculating from total - errors) - Expand legal domain vocabulary with 148 new Malayalam legal terms
- Add `use_sandhi: bool = True` param to align_arrays, text_error_rates, token_error_rates, and compute_sample_errors for optional Sandhi detection - Add Sandhi Detection toggle to visualizer sidebar - Tune mismatch penalty multiplier from 0.5 to 0.2 for better alignment scoring - Update README.md and CLAUDE.md API docs with use_sandhi param and visualizer toggles Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
domain config files added under src/dicterrors/config/.
terminology.
for non-agglutinative languages. Sandhi Detection toggle added to visualizer sidebar. Mismatch penalty multiplier tuned from 0.5 → 0.2. README and CLAUDE.md updated to reflect new API params.
Test plan