Skip to content

feat/measurements-uploader - #23

Merged
xameYeoh merged 11 commits into
mainfrom
feat/measurements-uploader
Oct 19, 2026
Merged

xameYeoh merged 11 commits into
mainfrom
feat/measurements-uploader

Conversation

@xameYeoh

@xameYeoh xameYeoh commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds the client side of uploading mobile sessions to /api/v3: creating the session, sending measurements in batches as binary frames, and finishing it. The code is three layered classes in
data/network/upload/. Each one can be called again after a failure and picks up where it stopped.

The class that decides when uploads run (the outbox-observing loop, plus WorkManager / beginBackgroundTask as a backstop) is not built here. Its design is agreed and written down in
docs/API_V3.md → When the uploader runs.

What's in it

MobileMeasurementPayload: binary frame encoder

  • Writes the v3 mobile measurement format: header 0xAB 0xBA + uint16 frame count, then one frame per measurement (int32 epoch seconds, uint8 sensor_type_id, float32 value, float64 lat,
    float64 lng), then a trailing XOR checksum.
  • Rejects an empty batch, more than MaxFrames = 3000 frames, and any sensor_type_id outside 1..255.

MobileMeasurementUploader: sends one session's outbox

  • Takes the session's pending rows in batches of up to 3000 and POSTs each batch to /mobile_sessions/{uuid}/measurements.
  • A row is removed from the outbox (acknowledged) only after the server accepts its batch. If a request fails, the exception propagates and the unsent rows stay queued.
  • A row it can't send (its stream has no sensor_type_id, or the row has no location) is skipped and logged as an error. It stays in the outbox and is never dropped silently.

MobileSessionUpload: brings one session up to date

Runs create (once) → upload outbox → /finish (once, after stop). Each step is skipped if it was already done:

  • Create runs only while some local stream still has no sensor_type_id. The ids from the response are written back to the local streams.
  • 409 recovery: a 409 on create means an earlier create reached the server but its response was lost. In that case the ids are read back with GET /mobile_sessions/{uuid}.
  • Finish sends the phone's own end_time as finished_at, then sets the new is_finished_on_server flag.

API / DTO changes

  • CreateMobileSessionRequestDto now has a flat body (the session wrapper is gone) and adds device { mac_address, model }, tag_list and a required streams.
  • createSession now returns CreateMobileSessionResponseDto (streams[] { sensor_name, sensor_type_id }).
  • New finishSession(token, uuid, finishedAt) and the v3MobileSessionFinish endpoint.
  • GET session/list DTOs now include streams, keyed by sensor_name. This is what the 409 read-back uses.

Persistence

  • sessions.is_finished_on_server column, markFinishedOnServer query, and a matching method on SessionRepository.
  • Renamed contribute → is_contributing_to_crowd_map. The JSON name stays contribute via @SerialName.

Boolean naming refactor (mechanical)

Boolean fields across the app now read as questions: granted → isGranted, darkMode → isDarkMode, contribute → isContributingToCrowdMap, and the same in auth, settings, mic and custom-server. No behavior change.

Decisions recorded in docs

  • docs/API_V3.md
    • Every stream must be sent in the create call: v3 can't add a stream later, and a frame needs a sensor_type_id.
    • The create body is flat; a 409 is recovered with the GET read-back.
    • The client sends its own finished_at, so upload and /finish can run in either order.
    • No client-side clock checks.
    • Upload scheduling: runs live, driven by the outbox, 60 s batching while recording, exponential backoff, platform backstops. iOS force-quit limit noted.
  • docs/DOMAIN_KNOWLEDGE.md
    • Location stamping when a measurement is stored, with the legacy fallback chain (V2MobileMeasurementsInserter).
    • When the legacy app triggered sync.
    • The app uploads mobile measurements only (AB3 SD-sync is the one exception, not built yet).
  • CLAUDE.md: when a business rule isn't covered by /docs/, take it from the legacy app's code and record it in /docs/.

⚠️ Notes for reviewers

  • Schema change without a migration: a column was renamed and is_finished_on_server was added directly in Session.sq, with no .sqm file. That's fine before release, but any existing local install needs a clean reinstall or app-data wipe.
  • Nothing calls MobileSessionUpload yet. That comes with the trigger loop in a follow-up.

Tests

  • MobileMeasurementPayloadTest: byte layout, checksum, input limits.
  • MobileMeasurementUploaderTest: batching at 3000, acknowledge only on success, skipped unsendable rows, no token → nothing sent.
  • MobileSessionUploadTest: create once and write the ids back, 409 read-back, skip create when ids exist, finish only after stop and only once, resume after a partial failure.
  • Shared fakes in UploadFakes.kt; existing tests updated for the boolean renames.

xameYeoh and others added 11 commits October 16, 2026 10:21
Record the legacy last-known-location fallback for mobile measurements,
that the app uploads mobile measurements only (AB3 SD sync excepted),
and the drain-before-finish rule that removes the finished_at open
decision. CLAUDE.md: take undocumented business logic from the legacy app.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nary encoder

Add binary payload encoder and test suite for mobile measurement session uploads.

- Introduce `MobileFrame` data model and `MobileMeasurementPayload.encode` for encoding frames into the expected binary format (magic header `0xABBA`, 16-bit count, 25 bytes per frame, and trailing XOR checksum).
- Enforce validation rules: batch size capped at 3000 frames, non-empty list required, and `sensorTypeId` bounded to uint8.
- Add `MobileMeasurementPayloadTest` to verify byte-by-byte encoding accuracy, timestamp truncation to whole seconds, frame limits, and checksum generation.
BE agreed to accept the client's finished_at on mobile /finish. Every
frame is stamped by the same phone clock at or before end_time, so
upload and /finish may run in any order and the uploader never reads
finished_at back. Replaces the drain-before-finish rule.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v3 cannot add a stream to an existing session, and frames need the
sensor_type_id from the create response. Streams without one are
skipped by the uploader and logged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Phone clocks are network-synced and the legacy app never pre-checked
timestamps. An invalid_epoch batch throws, stays queued, and is surfaced
to the user and crash reporting.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…dy, 409 recovery

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
finishSynced -> isFinishedOnServer, contribute -> isContributingToCrowdMap
(domain, DB columns, DTOs; wire key stays "contribute" via @SerialName).
No DB migration: schema is unreleased.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AppPreferences: is{CrowdMap,LocationTracking,PushNotifications}Enabled,
isDarkMode, isWifiOnlySync. AuthFormState: isPasswordVisible, isSubmitting.
CustomServerForm: hasHostError/hasPortError; Testing.isTakingTooLong.
AirBeamDevice.canReportOwnState, MicrophonePermission.isGranted,
Deletion.AwaitingCode.isRejected. Storage keys unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…cument upload triggers

Upload is no longer tied to backend sync: v3 has no sync_with_versioning,
and mobile measurements now go up live, driven by the outbox. Docs record
the legacy triggers and the new loop/backstop design.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@xameYeoh
xameYeoh merged commit 3988dc7 into main Oct 19, 2026
2 checks passed
@xameYeoh
xameYeoh deleted the feat/measurements-uploader branch October 19, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant