[DRAFT] Add RLE Support#47694
Conversation
novaturient95
requested review from
bobogogo1990,
dargilco,
glharper,
howieleung,
kingernupur,
nick863,
trangevi and
trrwilson
as code owners
There was a problem hiding this comment.
Pull request overview
This draft PR adds a new "RLE" (Reinforcement Learning Environment) client surface to azure-ai-projects. It introduces gym-style sync (RLEEnvironment) and async (AsyncRLEEnvironment) clients that lease a hosted sandbox from an RLE control plane and then drive it via reset/step/state/health/metadata/schema, plus supporting result/state models (RLEStepResult, RLEEnvState, RLEError). These symbols are exported at the top-level (azure.ai.projects) and async (azure.ai.projects.aio) namespaces via the _patch.py __all__ mechanism, with a sample and unit tests for the pure helpers.
Notably, the implementation uses its own HTTP stack (urllib for sync, aiohttp for async) and a raw bearer token rather than the azure-core transport/credential pipeline used elsewhere in the package. The PR is still draft: the description is the unfilled template, and there is no CHANGELOG entry for this new public feature.
Changes:
- Add sync/async RLE clients and models (
_patch_rle.py,_patch_rle_async.py) with bespoke HTTP transport and bearer-token auth. - Export the new RLE symbols from the sync and async public namespaces.
- Add a sample (sync only) and unit tests covering the pure helper functions.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
azure/ai/projects/operations/_patch_rle.py |
New sync RLE client/models; includes unused is_terminal/_TERMINAL_STATUSES and raw-token auth over a urllib transport. |
azure/ai/projects/aio/operations/_patch_rle_async.py |
New async RLE client built on aiohttp; mirrors sync logic including the same lease-loop gap. |
azure/ai/projects/_patch.py |
Re-exports sync RLE symbols at the top-level namespace. |
azure/ai/projects/aio/_patch.py |
Re-exports async RLE symbols at the top-level async namespace. |
azure/ai/projects/operations/_patch.py |
Adds RLE symbols to operations __all__. |
azure/ai/projects/aio/operations/_patch.py |
Adds async RLE symbols to operations __all__. |
samples/reinforcement_learning/sample_rle_code_rl_environment.py |
New sample driving a Code-RL environment via the sync client. |
tests/reinforcement_learning/test_rle_models.py |
Unit tests for the pure helpers (from_wire, coerce_action) and public exports. |
| while not lease.is_ready: | ||
| if lease.is_failed: | ||
| raise RLEError(f"sandbox {created.id} failed to start: {lease.error or 'unknown error'}") |
| if token: | ||
| self._headers["Authorization"] = f"Bearer {token}" |
| def reset(self, seed: Optional[int] = None, episode_id: Optional[str] = None, **extra: Any) -> RLEStepResult: | ||
| """Start a new episode and return the initial observation.""" | ||
| body: Dict[str, Any] = { | ||
| key: value for key, value in {"seed": seed, "episode_id": episode_id, **extra}.items() if value is not None | ||
| } | ||
| return RLEStepResult.from_wire(self._request("POST", "/reset", body)) | ||
|
|
||
| @distributed_trace | ||
| def step(self, action: Any = None, **action_kwargs: Any) -> RLEStepResult: | ||
| """Apply an action and return the resulting observation, reward, and done state.""" | ||
| return RLEStepResult.from_wire(self._request("POST", "/step", {"action": coerce_action(action, action_kwargs)})) |
Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines