fix: only show the dump/redistribute reminder for local soil-id DB writes#367
Merged
Merged
Conversation
…rites The "Next: regenerate the dump ... and redistribute it" footer only applies when --write targets the dedicated local soil-id database (named "soil_id"), which gets dumped into the soil-id-db image for local dev + CI. A --write against the staging/prod primary database applies the change directly -- nothing to dump or redistribute -- so gate the reminder on current_database() = 'soil_id'.
johannesparty
added a commit
to techmatters/terraso-backend
that referenced
this pull request
Jun 12, 2026
Picks up techmatters/soil-id-algorithm#367 (the wrb_descriptions_sync footer now only prints for local soil-id DB writes). The installed soil_id package is unchanged from 2026-06-11.1 — only the archive URL + hash move; no runtime effect.
johannesparty
added a commit
to techmatters/terraso-backend
that referenced
this pull request
Jun 12, 2026
* feat: admin action to mint long-lived partner refresh tokens Add 'Mint 1-year' / 'Mint 10-year partner refresh token' actions to the User admin (2a). An admin first creates a dedicated service-account user, then selects it and runs the action to mint a long-lived refresh token to hand to the partner. The partner exchanges it at /auth/tokens for short-lived access tokens to call the soil-ID API. The token is shown once in the admin UI and never logged or persisted (secrets policy). Revoke by unchecking 'Active' on the user, which the refresh endpoint already enforces. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: support adding email-based users in the Django admin The User model is email-based (USERNAME_FIELD = email, username removed), but UserAdmin inherited DjangoUserAdmin's add_form / add_fieldsets, which reference a username field. Opening /admin/core/user/add/ therefore raised FieldError(Unknown field(s) (username)). Bind an email-based creation form and add_fieldsets so service accounts (and any user) can be created from the admin. Pre-existing bug, surfaced by the partner service-account flow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: accept Munsell color (string or numeric) in the soil-ID API The soilMatches query previously took soil color only as CIELAB. Add two optional alternatives on each depth entry: colorMunsell (a Munsell string like "7.5YR 5/4" or "N 5/") and colorMunsellNumeric (hue/value/chroma, hue on the 0-100 scale). Color sources are tried in order LAB, string, numeric. Conversion is centralized in a new apps/soil_id/munsell module (munsell_to_lab moved here from apps/export, plus a new munsell_string_to_lab), backed by the soil-id LandPKS_munsell_rgb_lab lookup table. A Munsell value that can't be converted (e.g. an out-of-gamut, photo-derived color) is ignored for that depth rather than failing the request. Fully additive; colorLAB and existing callers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add partner soil-ID access sample (scripts + README) Self-contained example of calling the soil-ID API as a partner service account: Python and shell scripts that read a long-lived refresh token from ~/secrets/terraso/tokens.json, exchange it for an access token (caching it), run a soilMatches query, and write the result. README documents the request fields including the new Munsell color inputs. Generated output (soil_id_response.json) and README.pdf are gitignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(export): send Munsell to the soil-ID API instead of pre-converting to LAB fetch_soil_id converted stored Munsell (hue/value/chroma) to colorLAB before calling the soilMatches query, duplicating the conversion the API now performs itself. Pass colorMunsellNumeric through and let the resolver convert, removing the munsell_to_lab call and import from the export app. Behavior-preserving: a valid color yields the same LAB (same munsell_to_lab, same inputs); an out-of-gamut color produces no color for that depth either way (export previously skipped it; the resolver now ignores it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: extract shared decode_hue helper for the 0-100 Munsell hue The 0-100 continuous hue -> (substep, family) decode block, plus the hue-family array, was duplicated in apps/soil_id/munsell.munsell_to_lab and apps/export/transformers.render_munsell_hue. Extract it as munsell.decode_hue and call it from both; drop the duplicated non_neutral_color_hues constant. Behavior-preserving; covered by the existing munsell/export tests plus a new decode_hue unit test (incl. the 10X and 0/100 boundary cases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: delegate Munsell->LAB lookup to the soil-id library Use soil_id.color.munsell_to_lab against the library's preloaded MUNSELL_COLOR_REF/MUNSELL_REF instead of re-parsing LandPKS_munsell_rgb_lab.csv into a second in-memory table here. The app-specific 0-100 hue decode and Munsell-string parsing stay as thin wrappers; behaviour is unchanged (values rounded as before, library np.float64 cast back to plain float). Removes ~35 lines (_load_munsell_lab_table + locking) and the duplicate CSV load in the backend process. REQUIRES the soil-id-algorithm change adding munsell_to_lab to be merged and the soil-id pin in requirements/base.in bumped past f4fdadd, then 'make lock'. Until then the backend will not import. Verified against the local library (PYTHONPATH override): full unit suite passes (775). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: add USE_LOCAL_SOILID flag to run against a local soil-id checkout Adds a docker-compose.local-soilid.yml override that mounts a local ../soil-id-algorithm checkout and shadows the pinned release via PYTHONPATH, plus two ways to enable it without repinning requirements or rebuilding: USE_LOCAL_SOILID=true make run # convenience flag make run DC_EXTRA_FILES="-f docker-compose.local-soilid.yml" # general hook DC_EXTRA_FILES is threaded through DC_FILE_ARG so it applies to run/test/bash/ lint alike. Useful while a soil-id change (e.g. munsell_to_lab) is unreleased; stop using it once requirements is repinned to the merged version. Override the checkout path with SOIL_ID_SRC. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: add run-local-soilid convenience target Wrapper for USE_LOCAL_SOILID=true make run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: move the Munsell consistency test to the soil-id library The cross-platform Munsell->LAB consistency check (and its mobile-client fixture) validates soil-id's lookup table, so it now lives in soil-id next to that table and the munsell_to_lab it exercises. The backend keeps its own unit tests for the app-specific layer (0-100 hue decoding and Munsell string parsing) in tests/soil_id/test_munsell.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: ignore wrb-out/ (wrb_descriptions_sync diff output) * build: repin soil-id to 2026-06-12.1 (script-only dump-reminder fix) Picks up techmatters/soil-id-algorithm#367 (the wrb_descriptions_sync footer now only prints for local soil-id DB writes). The installed soil_id package is unchanged from 2026-06-11.1 — only the archive URL + hash move; no runtime effect. * docs(partner-access): stream response to stdout; add management; Kenya sample - Print the GraphQL response to stdout and route status/progress to stderr, so `| jq` / `> file` stays clean — the script no longer writes a response file. - Rename the input to soil_id_query_params.json; drop the generated response file and its .gitignore entry. - Request the new `management` field alongside `description` in both scripts. - Switch the sample lookup to a Kenya (global) location, exercising the global description/management. - README updated to match (Files, Run, Result shape). * chore(partner-access): remove unused .gitignore Nothing in this directory is generated anymore — the response now goes to stdout (no response file) and there's no README.pdf — so the .gitignore has nothing to ignore. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
johannesparty
added a commit
to techmatters/terraso-backend
that referenced
this pull request
Jun 12, 2026
* feat: admin action to mint long-lived partner refresh tokens Add 'Mint 1-year' / 'Mint 10-year partner refresh token' actions to the User admin (2a). An admin first creates a dedicated service-account user, then selects it and runs the action to mint a long-lived refresh token to hand to the partner. The partner exchanges it at /auth/tokens for short-lived access tokens to call the soil-ID API. The token is shown once in the admin UI and never logged or persisted (secrets policy). Revoke by unchecking 'Active' on the user, which the refresh endpoint already enforces. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: support adding email-based users in the Django admin The User model is email-based (USERNAME_FIELD = email, username removed), but UserAdmin inherited DjangoUserAdmin's add_form / add_fieldsets, which reference a username field. Opening /admin/core/user/add/ therefore raised FieldError(Unknown field(s) (username)). Bind an email-based creation form and add_fieldsets so service accounts (and any user) can be created from the admin. Pre-existing bug, surfaced by the partner service-account flow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: accept Munsell color (string or numeric) in the soil-ID API The soilMatches query previously took soil color only as CIELAB. Add two optional alternatives on each depth entry: colorMunsell (a Munsell string like "7.5YR 5/4" or "N 5/") and colorMunsellNumeric (hue/value/chroma, hue on the 0-100 scale). Color sources are tried in order LAB, string, numeric. Conversion is centralized in a new apps/soil_id/munsell module (munsell_to_lab moved here from apps/export, plus a new munsell_string_to_lab), backed by the soil-id LandPKS_munsell_rgb_lab lookup table. A Munsell value that can't be converted (e.g. an out-of-gamut, photo-derived color) is ignored for that depth rather than failing the request. Fully additive; colorLAB and existing callers are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add partner soil-ID access sample (scripts + README) Self-contained example of calling the soil-ID API as a partner service account: Python and shell scripts that read a long-lived refresh token from ~/secrets/terraso/tokens.json, exchange it for an access token (caching it), run a soilMatches query, and write the result. README documents the request fields including the new Munsell color inputs. Generated output (soil_id_response.json) and README.pdf are gitignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(export): send Munsell to the soil-ID API instead of pre-converting to LAB fetch_soil_id converted stored Munsell (hue/value/chroma) to colorLAB before calling the soilMatches query, duplicating the conversion the API now performs itself. Pass colorMunsellNumeric through and let the resolver convert, removing the munsell_to_lab call and import from the export app. Behavior-preserving: a valid color yields the same LAB (same munsell_to_lab, same inputs); an out-of-gamut color produces no color for that depth either way (export previously skipped it; the resolver now ignores it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: extract shared decode_hue helper for the 0-100 Munsell hue The 0-100 continuous hue -> (substep, family) decode block, plus the hue-family array, was duplicated in apps/soil_id/munsell.munsell_to_lab and apps/export/transformers.render_munsell_hue. Extract it as munsell.decode_hue and call it from both; drop the duplicated non_neutral_color_hues constant. Behavior-preserving; covered by the existing munsell/export tests plus a new decode_hue unit test (incl. the 10X and 0/100 boundary cases). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: delegate Munsell->LAB lookup to the soil-id library Use soil_id.color.munsell_to_lab against the library's preloaded MUNSELL_COLOR_REF/MUNSELL_REF instead of re-parsing LandPKS_munsell_rgb_lab.csv into a second in-memory table here. The app-specific 0-100 hue decode and Munsell-string parsing stay as thin wrappers; behaviour is unchanged (values rounded as before, library np.float64 cast back to plain float). Removes ~35 lines (_load_munsell_lab_table + locking) and the duplicate CSV load in the backend process. REQUIRES the soil-id-algorithm change adding munsell_to_lab to be merged and the soil-id pin in requirements/base.in bumped past f4fdadd, then 'make lock'. Until then the backend will not import. Verified against the local library (PYTHONPATH override): full unit suite passes (775). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: add USE_LOCAL_SOILID flag to run against a local soil-id checkout Adds a docker-compose.local-soilid.yml override that mounts a local ../soil-id-algorithm checkout and shadows the pinned release via PYTHONPATH, plus two ways to enable it without repinning requirements or rebuilding: USE_LOCAL_SOILID=true make run # convenience flag make run DC_EXTRA_FILES="-f docker-compose.local-soilid.yml" # general hook DC_EXTRA_FILES is threaded through DC_FILE_ARG so it applies to run/test/bash/ lint alike. Useful while a soil-id change (e.g. munsell_to_lab) is unreleased; stop using it once requirements is repinned to the merged version. Override the checkout path with SOIL_ID_SRC. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build: add run-local-soilid convenience target Wrapper for USE_LOCAL_SOILID=true make run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: move the Munsell consistency test to the soil-id library The cross-platform Munsell->LAB consistency check (and its mobile-client fixture) validates soil-id's lookup table, so it now lives in soil-id next to that table and the munsell_to_lab it exercises. The backend keeps its own unit tests for the app-specific layer (0-100 hue decoding and Munsell string parsing) in tests/soil_id/test_munsell.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: ignore wrb-out/ (wrb_descriptions_sync diff output) * build: repin soil-id to 2026-06-12.1 (script-only dump-reminder fix) Picks up techmatters/soil-id-algorithm#367 (the wrb_descriptions_sync footer now only prints for local soil-id DB writes). The installed soil_id package is unchanged from 2026-06-11.1 — only the archive URL + hash move; no runtime effect. * docs(partner-access): stream response to stdout; add management; Kenya sample - Print the GraphQL response to stdout and route status/progress to stderr, so `| jq` / `> file` stays clean — the script no longer writes a response file. - Rename the input to soil_id_query_params.json; drop the generated response file and its .gitignore entry. - Request the new `management` field alongside `description` in both scripts. - Switch the sample lookup to a Kenya (global) location, exercising the global description/management. - README updated to match (Files, Run, Result shape). * chore(partner-access): remove unused .gitignore Nothing in this directory is generated anymore — the response now goes to stdout (no response file) and there's no README.pdf — so the .gitignore has nothing to ignore. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
--writefooter ("Next: regenerate the dump … and redistribute it") is only meaningful for the dedicated local soil-id DB (namedsoil_id), which is dumped into thesoil-id-dbimage for local dev + CI.A
--writeagainst the staging/prod primary database applies the change directly — there's nothing to dump or redistribute — yet the line printed there too, which was confusing during the prod sync.Fix: capture
current_database()after the rebuild and only print the reminder when it'ssoil_id.🤖 Generated with Claude Code