Skip to content

Add a general ansible validation action#3684

Merged
ipspace merged 1 commit into
ipspace:devfrom
roc-ops:ocnos-ansible-action
Jul 22, 2026
Merged

Add a general ansible validation action#3684
ipspace merged 1 commit into
ipspace:devfrom
roc-ops:ocnos-ansible-action

Conversation

@roc-ops

@roc-ops roc-ops commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Add a general ansible validation action

Closes #3676.

netlab validate reads device state either by running a show/exec command over the node connection, or from an external source (suzieq). Both assume the platform can return command output to a non-interactive caller. Some network OSes only expose their operational data through a vendor Ansible module — their interactive CLI rejects non-interactive command execution — so neither existing path can reach them, and today such a device cannot be validated at all.

This PR adds an ansible validation action to close that gap. It is a validation data source, a peer of netsim/cli/validate/suzieq.pynot a connection method. netsim/cli/connect.py is unchanged; interactive netlab connect is untouched. As discussed on #3676, it is implemented as a first-class, reusable action rather than a device-specific plugin, so any platform in the same situation can adopt it.

How it works

  • A device declares the module it validates through, e.g.:
    netlab_validate:
      ansible_module: ipinfusion.ocnos.ocnos_command
  • A test can invoke it directly in a validate entry:
    validate:
      adj:
        device: <device>
        ansible: show ip ospf neighbor
        valid: "'Full' in stdout"
    or a device validation plugin can supply the command from an ansible_<test>() function and assert on the result in valid_<test>().
  • netsim/cli/validate/ansible.py runs the module ad-hoc against the node using the netlab-generated inventory, and returns parsed JSON when the command emits it (... | json), otherwise the raw CLI text in stdout for a text assertion to match.

The action is registered everywhere a validation action is expected — the topology schema (attributes.yml), the augment/validation checks and their hint strings (augment/validate.py, hints.yml), and the test/plugin dispatchers — so ansible: is accepted and executed like any built-in action, and it's documented in docs/topology/validate.md.

Files

  • new netsim/cli/validate/ansible.py — the validation source (peer of suzieq.py)
  • netsim/cli/validate/tests.py, utils.py, plugin.py — dispatch + action registration
  • netsim/augment/validate.py, netsim/defaults/attributes.yml, netsim/defaults/hints.yml — accept ansible: as a first-class validation action (schema, checks, and user-facing hints)
  • docs/topology/validate.md — documentation
  • tests/coverage/errors/validation-checks.log — updated golden for the new action's error text

Notes

The first consumer is the IP Infusion OcNOS device (submitted separately), whose restricted CLI (cmlsh) returns a non-zero exit on every non-interactive SSH command and so can't be reached by the standard show/exec transports. The action itself is device-agnostic. Verified live against a real OcNOS lab (OSPFv2/OSPFv3, BGP and IS-IS validators passing through the action).

@ipspace ipspace left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly nits, the only thing that would be nice to have is a more informative printout on failure.

return err_value
command = ' '.join(v_cmd)

a_module = topology.defaults.devices[node.device].netlab_validate.ansible_module

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my only real questionable point in this PR. On one hand, it would be nice to place this into an existing dictionary (for example, intofeatures, i.e. features.validate.ansible_module or some such). OTOH, we already use the device.netlab_validate dictionary to extend wait times (although netlab_validate is so far never used in device definition).

I don't have a strong opinion either way, so whatever works for you is fine ;)

Comment thread netsim/cli/validate/ansible.py Outdated
# (hosts.yml + ansible.cfg in the lab directory). Pass the command as a single
# quoted string -- ansible ad-hoc rejects a bare list literal ("does not support
# raw params").
cmd = ['ansible', n_name, '-m', str(a_module), '-a', 'commands=%r' % command, '--one-line']

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might have to support a more generic command format. While most networking "something_command" modules use "commands" argument, the "command" module uses none. However, we can change that later if needed.

print(f'Preparing to execute {command!r} on {n_name} via Ansible ({a_module})')

try:
out = subprocess.run(cmd, capture_output=True, text=True, timeout=60)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have "cli/external_commands/run_command" wrapper around subprocess.run in case you want to use that one. Maybe it's simpler to just go with "subprocess.run" here.

Comment thread netsim/cli/validate/ansible.py Outdated
out = subprocess.run(cmd, capture_output=True, text=True, timeout=60)
except Exception as ex: # noqa: BLE001
report.log_failure(f'Ansible action failed for "{command}" on {n_name}',
more_data=str(ex), topology=topology)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might make sense to include the actual command executed in "more_data", so the user can run the command manually to see what's wrong -- similar to how we do it in "execute_netlab_config" in devices.py (same directory)


raw = out.stdout or ''
# ansible --one-line: "<host> | SUCCESS => { ...json... }" (or FAILED!/UNREACHABLE!)
if ' => ' not in raw:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I knew the output would be messy ;)

@roc-ops
roc-ops force-pushed the ocnos-ansible-action branch from 6c39574 to 20dfa97 Compare July 22, 2026 14:41
@roc-ops

roc-ops commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the quick review — glad it's mostly nits! All four addressed:

1. More informative failure output. Good call. The failure path now surfaces the exact ad-hoc ansible … command in more_data (mirroring execute_netlab_config in netsim/cli/validate/devices.py), so you can copy-paste and re-run it by hand. Each failure branch — subprocess error, no parseable output, JSON parse error, and a module FAILED!/UNREACHABLE! — now prints a clean one-line reason plus the runnable command instead of dumping the raw --one-line blob. (And yes, the old output was messy — thanks for the nudge 🙂.)

2. subprocess wrapper. I looked at cli/external_commands.run_command but kept the plain subprocess.run here: the action needs the module's stdout even when ansible exits non-zero (a FAILED!/UNREACHABLE! still returns a parseable … => {json} payload we read the error msg from), then inspects that payload rather than a bool/exit code. run_command's default check=True raises on non-zero (losing stdout), and the return_exitcode + CAPTURED_STDOUT-global route is more convoluted than a direct capture_output=True — so I went with subprocess.run, as you said was fine.

3. Generic command format. Handled the common split now (low-effort): <os>_command modules get commands='<cmd>', while the plain command/shell modules (no commands= arg) get the bare command line, selected by the module's last name component, with a comment that other arg shapes can be added later.

4. ansible_module placement. Kept it in device.netlab_validate.ansible_module — it sits alongside the existing netlab_validate.<test>.wait device settings used across the integration tests, so grouping it there felt consistent. Happy to move it to features.validate if you'd prefer; just say the word.

Add an `ansible` validation action for `netlab validate`: a validation data
source (a peer of the SuzieQ action in netsim/cli/validate/suzieq.py), not a
device connection method. It fetches show output by running the command
through a device's Ansible module (network_cli), for platforms whose CLI
cannot be driven with non-interactive SSH commands.

A test selects it with an `ansible` action (or a validation-plugin
`ansible_<test>()` function) that supplies the show command; the device names
the module to run it through in
`defaults.devices.<device>.netlab_validate.ansible_module`. The result is
parsed as JSON when the command emits JSON, otherwise the CLI text is returned
in `stdout` for a `valid:` expression or a plugin `valid_<test>()` to
screen-scrape.

- netsim/cli/validate/ansible.py: the action implementation (get_result()).
- netsim/cli/validate/tests.py: dispatch arm for the ansible action.
- netsim/cli/validate/utils.py, plugin.py: register ansible in the test and
  plugin action lookups.
- netsim/augment/validate.py, netsim/defaults/attributes.yml: make ansible a
  first-class validation test action in the schema and augment checks, so any
  topology can use `ansible: <show command>` with `valid:` directly.
- netsim/defaults/hints.yml: include ansible in the validation hint messages.
- docs/topology/validate.md: document the ansible action.

Closes ipspace#3676.
@roc-ops
roc-ops force-pushed the ocnos-ansible-action branch from 20dfa97 to 9cb0a83 Compare July 22, 2026 14:57

@ipspace ipspace left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@ipspace
ipspace merged commit 68e04f3 into ipspace:dev Jul 22, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native post-deployment validation for a device whose CLI has no one-shot command mode over SSH (e.g. OcNOS cmlsh) — proposal + design question

2 participants