Skip to content

Add selective CI and manual Selectel UI tests with device availability waiting - #31

Closed
andre487 wants to merge 13 commits into
mainfrom
feat/selectel-ui-tests
Closed

andre487 wants to merge 13 commits into
mainfrom
feat/selectel-ui-tests

Conversation

@andre487

@andre487 andre487 commented Sep 7, 2026 •

Copy link
Copy Markdown
Owner

PRs select automatic checks from the full PR diff while Selectel device UI tests run manually before merging. Python-only changes skip Android and UI tests; documentation-only changes skip test jobs. Shared CI/build inputs and unknown paths conservatively run all suites. The required Change scope check prevents classification failures from silently bypassing tests.

  • Required UI: HONOR X8c / Android 15, 12 cases. Additional UI: Android 11, 13, 16 and 17, four devices and 48 cases without overlap. Both use a fixed catalog. Publish separate commit statuses on the tested SHA; without Android-impacting changes, mark UI as not required and skip APK builds and rentals even on manual dispatch.
  • Wait up to 10 minutes for the selected device, checking availability every 30 seconds before creating a paid lease. SELECTEL_DEVICE_WAIT_SECONDS accepts 0–900 seconds. Never retry an ambiguous rental POST. Extend device jobs to 40 minutes for waiting, tests and cleanup.
  • Build ARM64 debug application/test APKs before renting. Run at most two additional devices concurrently in GitHub and sequentially locally. Preserve reports, screenshots and lease journals; cleanup and independent recovery verify owned slots without touching existing rentals. PR jobs have no release signing material.
  • Add a standard-library Python launcher using native gh commands for PR selection, dispatch and CI reruns. Support dry-run, --yes/-y to skip the final confirmation, and stale-head detection. Keep authentication in gh. Run Python contracts and pinned Black/isort checks through Fastlane.
  • Cover navigation, activity recreation and profile drafts in English/Russian at 100%/200% font scale. Synchronize English/Russian instructions and skip artifact comments when Android builds intentionally do not run.

Validation: 47 Python tests and Black/isort checks pass through Fastlane. Workflow YAML parses; 14 mocked status scenarios pass, including intentional skips, failures, cancellation and preserved CI rerun results. The real gh launcher dry-run with -y selected the updated required profile without launching jobs. Full PR diff classification correctly still enables Android for this PR.

Latest UI failure analysis (before this push): run 34149140033 failed at acquisition because Galaxy A14 / API 35 had no available device. No lease was created and instrumentation did not run. At the catalog check, HONOR X8c / API 35 had three free devices versus zero A14s; availability can change. Earlier A14 runs passed all 12 cases, while other attempts encountered remote ADB offline. The subsequent X8c run 34150734050 rented successfully but remained ADB offline; instrumentation was skipped and the paid lease and key were released successfully.

Align session handling with the current Selectel API: use v3 users/devices and remote-connect for assignment and session creation/cleanup; accept remoteConnectUrl without the old success flag. Verify operational status and readiness before and after assignment. Follow the documented ADB sequence: connect once, then poll devices -l for the exact endpoint, accepting an initial authentication warning and preserving the transport while offline. Report intermediate session-cleanup failures while still removing the paid rental. Contract tests cover the v3 lifecycle, empty assignment responses, readiness and delayed ADB readiness without disconnect loops. A later run reached the v3 ADB session but remained offline and then absent; cleanup succeeded. The primary-key correction below was subsequently verified locally against Selectel.

Fix ADB identity selection: register the public half of the actual primary ~/.android/adbkey, deriving it with adb pubkey. Android preference environment variables do not relocate the ADB primary key; ADB_VENDOR_KEYS only adds signing keys and does not replace the public identity sent during authentication. Preserve existing local private keys and existing Selectel registrations (HTTP 409); remove only registrations created by this run. Keep a separate ADB server port and clear inherited server-socket overrides.

Validation before changing production code: a local TCP ADB authentication model with the installed real ADB binary rejected the old setup because the presented public key differed from the registered vendor key (offline). The same model accepted the corrected setup (device). Afterwards, a single real HONOR X8c rental from this machine connected successfully through v3; adb shell getprop confirmed API 35 and model ABR-LX1. Cleanup succeeded; a separate API read confirmed device removal (404) and absence of the run-created key registration. This was a connection smoke check, not a complete UI instrumentation run. All 47 Python tests and Black/isort checks pass locally.

@github-actions

github-actions Bot commented Sep 7, 2026 •

Copy link
Copy Markdown

APK artifacts

Built from commit 033b1214 by CI run #116.
Artifacts expire after 14 days. Neither APK uses the MegaProxy release key.

@andre487
andre487 marked this pull request as ready for review September 7, 2026 16:12
@andre487 andre487 changed the title Add Selectel real-device UI tests to pull requests Add manually required Selectel UI tests with a fixed device matrix Sep 7, 2026
@andre487 andre487 changed the title Add manually required Selectel UI tests with a fixed device matrix Add manual required and additional Selectel UI tests Sep 7, 2026
@andre487 andre487 changed the title Add manual required and additional Selectel UI tests Add selective CI and manual Selectel UI tests with device availability waiting Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant