diff --git a/examples/hand_detector/README.md b/examples/hand_detector/README.md new file mode 100644 index 00000000..fb324a3c --- /dev/null +++ b/examples/hand_detector/README.md @@ -0,0 +1,275 @@ +# MediaPipe Palm Detector + +This example reconstructs the MediaPipe palm detector as a PyTorch module, +quantizes it with TICO WrapQ, and exports static Circle models with an NHWC +input boundary. + +The example is organized around four entry points: + +```text +convert.py Convert source TFLite weights and graph metadata to PyTorch artifacts. +export.py Export floating-point or calibrated quantized Circle models. +analyze.py Run reusable numerical quantization analyses. +verify.py Verify torch.export and Circle artifacts. +``` + +Model-independent numerical analysis lives in `tico.quantization.analysis`. +The example only supplies palm-detector model loading, data normalization, and +output-boundary selection. + +## Model interface + +The exported model accepts one image tensor: + +```text +shape: [1, 192, 192, 3] +layout: NHWC +range: [0, 1] +dtype: float32 before quantized export +``` + +It returns: + +```text +regressors: [1, 2016, 18] +classifiers: [1, 2016, 1] +``` + +The PyTorch implementation uses NCHW internally. `NHWCInputAdapter` keeps the +external ABI explicit while Circle layout optimization removes redundant +internal layout transitions. + +## Setup + +Build and install TICO first: + +```bash +./ccex build +./ccex install --cpu_only +``` + +Install the example dependencies: + +```bash +python -m pip install -r examples/hand_detector/requirements.txt +``` + +Run commands from the repository root with `python -m`. This keeps package +imports stable and avoids depending on the current working directory. + +## Convert the source TFLite model + +```bash +python -m examples.hand_detector.convert \ + /path/to/hand_detector.tflite +``` + +The default outputs are: + +```text +examples/hand_detector/hand_detector_spec.json +examples/hand_detector/hand_detector_float.pt +``` + +The converter currently supports the operator subset used by the supplied palm +detector. It is not a general TFLite-to-PyTorch frontend. + +## Export Circle models + +### Floating point + +```bash +python -m examples.hand_detector.export float \ + --output examples/hand_detector/hand_detector_float.circle +``` + +The command verifies the NHWC input, layout optimization, and the two +`RESIZE_BILINEAR` operators unless `--skip-verification` is supplied. + +### Quantized + +Use representative tensors produced by the same preprocessing path as runtime +inputs: + +```bash +python -m examples.hand_detector.export quantized \ + --calibration-dir /path/to/calibration_npy \ + --bits 8 16 +``` + +The quantization policies are: + +| Tensor role | UINT8 | INT16 | +|---|---|---| +| Image and activations | per-tensor asymmetric | per-tensor symmetric | +| Conv/depthwise weights | per-channel asymmetric | per-channel symmetric | +| PReLU slope | per-channel asymmetric | per-channel symmetric | +| Convolution bias | INT32 | INT64 | + +Synthetic inputs are available only for smoke tests: + +```bash +python -m examples.hand_detector.export quantized \ + --synthetic-calibration-samples 32 \ + --bits 8 +``` + +## Quantization analysis + +### A/B/C/D ablation + +The standard profiles isolate the major quantization error sources: + +```text +A output-only +B weight-only with floating-point activations and outputs +C internal activation-only with floating-point weights and outputs +D full quantization +``` + +Run all four profiles from one calibrated candidate: + +```bash +python -m examples.hand_detector.analyze ablation \ + --calibration-dir /path/to/calibration_npy \ + --evaluation-dir /path/to/evaluation_npy \ + --bits 8 +``` + +The same API is reusable from Python: + +```python +from tico.quantization.analysis import ( + QuantizationAblation, + QuantizationBoundaries, + QuantizationProfile, + SiteSelector, +) +``` + +A model adapter only needs to define which observer sites represent final model +outputs. Parameter and internal-activation profiles are derived from observer +roles. + +### Output clipping + +Compare MinMax, fixed percentile, and calibration-L1 clipping while leaving all +internal model computation in floating point: + +```bash +python -m examples.hand_detector.analyze output-clipping \ + --calibration-dir /path/to/calibration_npy \ + --evaluation-dir /path/to/evaluation_npy \ + --bits 8 +``` + +This reports calibration and evaluation MAE, selected ranges, affine qparams, +saturation, and integer-code utilization. The L1 candidate is selected only +from calibration outputs. + +### Activation observer sweep + +Keep per-channel MinMax weight quantization fixed and compare activation range +estimators: + +```bash +python -m examples.hand_detector.analyze observer-sweep \ + --calibration-dir /path/to/calibration_npy \ + --evaluation-dir /path/to/evaluation_npy \ + --bits 8 \ + --percentiles 99.9 99.99 99.999 +``` + +`PercentileObserver` uses bounded sampling, so it does not retain every value +from every activation tensor. + +## Calibration and evaluation data + +Supported NumPy shapes are: + +```text +[192, 192, 3] +[1, 192, 192, 3] +[3, 192, 192] +[1, 3, 192, 192] +``` + +Integer arrays are converted to float32 and divided by 255. Floating-point +arrays are assumed to already use the model input range. + +Calibration and evaluation may point to the same directory for numerical-floor +analysis, but policy selection and final reporting should use disjoint data. +Use offsets to split a naturally sorted directory: + +```bash +python -m examples.hand_detector.analyze observer-sweep \ + --calibration-dir /path/to/npy \ + --calibration-offset 0 \ + --calibration-limit 200 \ + --evaluation-dir /path/to/npy \ + --evaluation-offset 200 \ + --evaluation-limit 79 \ + --require-disjoint +``` + +For frames extracted from video, split by source video or capture session rather +than adjacent frame number. + +## Verification + +Verify the PyTorch export graph: + +```bash +python -m examples.hand_detector.verify torch +``` + +Verify a floating-point Circle model: + +```bash +python -m examples.hand_detector.verify circle \ + examples/hand_detector/hand_detector_float.circle +``` + +Verify a quantized Circle model: + +```bash +python -m examples.hand_detector.verify quantized \ + examples/hand_detector/exported/hand_detector_uint8.circle \ + --bits 8 +``` + +## Internal support modules + +Implementation helpers are under `_support/` and are not separate user-facing +commands: + +```text +_support/circle.py +_support/conversion.py +_support/data.py +_support/quantization.py +_support/tflite_flatbuffer.py +_support/verify_circle_layout.py +_support/verify_circle_resize.py +_support/verify_quantized_circle.py +``` + +## Tests + +Run reusable analysis tests: + +```bash +python -m unittest discover -s test/quantization/analysis -v +python -m unittest discover -s test/quantization/wrapq -p "test_control.py" -v +python -m unittest discover -s test/quantization/wrapq/observers \ + -p "test_percentile.py" -v +``` + +Run the model example tests: + +```bash +python -m examples.hand_detector.test_hand_detector +``` + +See `docs/layout_optimization.md` for the Circle layout-region optimization design +and `THIRD_PARTY_NOTICES.md` for source-model attribution. diff --git a/examples/hand_detector/THIRD_PARTY_NOTICES.md b/examples/hand_detector/THIRD_PARTY_NOTICES.md new file mode 100644 index 00000000..a90870f3 --- /dev/null +++ b/examples/hand_detector/THIRD_PARTY_NOTICES.md @@ -0,0 +1,14 @@ +MediaPipe Hand Tracking Models +------------------------------ + +This project includes PyTorch conversions of the MediaPipe Hand Tracking +palm detection and hand landmark models. + +The original models are licensed under the Apache License, Version 2.0. + +Modifications: +- Converted the original TensorFlow Lite models to PyTorch modules. +- Adapted tensor layouts and operator implementations for PyTorch. +- Added quantization and model-debugging support. +- The converted and quantized models may produce results different from + the original MediaPipe models. diff --git a/examples/hand_detector/__init__.py b/examples/hand_detector/__init__.py new file mode 100644 index 00000000..9f572121 --- /dev/null +++ b/examples/hand_detector/__init__.py @@ -0,0 +1,15 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""MediaPipe palm-detector conversion and quantization example.""" diff --git a/examples/hand_detector/_support/__init__.py b/examples/hand_detector/_support/__init__.py new file mode 100644 index 00000000..3e9b2144 --- /dev/null +++ b/examples/hand_detector/_support/__init__.py @@ -0,0 +1,15 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Internal helpers for the hand-detector example entry points.""" diff --git a/examples/hand_detector/_support/analysis.py b/examples/hand_detector/_support/analysis.py new file mode 100644 index 00000000..4c60a359 --- /dev/null +++ b/examples/hand_detector/_support/analysis.py @@ -0,0 +1,83 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Hand-detector-specific adapters for reusable quantization analysis.""" + +from __future__ import annotations + +from typing import Any + +from examples.hand_detector.hand_detector import HandDetector, NHWCInputAdapter +from tico.quantization.analysis import QuantizationBoundaries, SiteSelector +from tico.quantization.wrapq.observers.percentile import PercentileObserver + +from torch import nn + + +OUTPUT_NAMES = ("regressors", "classifiers") + + +def output_boundaries(model: nn.Module) -> QuantizationBoundaries: + """Select the final output-domain observers from a prepared detector.""" + detector, prefix = _find_detector(model) + output_paths: list[str] = [] + output_tensors = set(detector.output_tensors) + for layer_index, operation in enumerate(detector.operations): + produced = {int(value) for value in operation["outputs"]} + if produced & output_tensors: + module_path = ( + f"{prefix}layers.{layer_index}" if prefix else f"layers.{layer_index}" + ) + output_paths.append(module_path) + if not output_paths: + raise RuntimeError("No detector layer produces a configured model output.") + selector = SiteSelector.module_paths(*output_paths) & SiteSelector.observer_names( + "act_out" + ) + return QuantizationBoundaries(outputs=selector) + + +def summarize_percentile_observers(model: nn.Module) -> list[dict[str, Any]]: + """Return ranges and qparams for every percentile activation observer.""" + summaries: list[dict[str, Any]] = [] + for module_name, module in model.named_modules(): + if not isinstance(module, PercentileObserver): + continue + scale, zero_point = module.compute_qparams() + summaries.append( + { + "module": module_name, + "observer_name": module.name, + "observed_minimum": float(module.min_val.detach().cpu()), + "observed_maximum": float(module.max_val.detach().cpu()), + "clip_minimum": float(module.clip_min_val.detach().cpu()), + "clip_maximum": float(module.clip_max_val.detach().cpu()), + "scale": float(scale.detach().cpu()), + "zero_point": int(zero_point.detach().cpu()), + "sampled_value_count": module.sampled_value_count, + "percentile": module.percentile, + } + ) + return summaries + + +def _find_detector(model: nn.Module) -> tuple[HandDetector, str]: + if isinstance(model, NHWCInputAdapter): + return model.detector, "detector." + if isinstance(model, HandDetector): + return model, "" + detector = getattr(model, "detector", None) + if isinstance(detector, HandDetector): + return detector, "detector." + raise TypeError("Expected HandDetector or NHWCInputAdapter.") diff --git a/examples/hand_detector/_support/circle.py b/examples/hand_detector/_support/circle.py new file mode 100644 index 00000000..e5e6c628 --- /dev/null +++ b/examples/hand_detector/_support/circle.py @@ -0,0 +1,62 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Circle-side layout optimization helpers for the hand detector.""" + +from __future__ import annotations + +from pathlib import Path + +from tico.circle.document import CircleDocument +from tico.circle.passes import ( + CirclePassContext, + CirclePassManager, + CirclePassManagerResult, + CirclePassStrategy, + EliminateTransposeBoundedLayoutRegionPass, + RemoveRedundantLayoutOpsPass, +) +from tico.circle.passes.cleanup import CompactIndicesPass, DeadCodeEliminationPass +from tico.utils.model import CircleModel + + +def optimize_layout_transitions( + circle_model: CircleModel, +) -> tuple[CircleModel, CirclePassManagerResult]: + """Optimize layout round trips in a serialized Circle model.""" + document = CircleDocument.from_bytes(circle_model.circle_binary) + pipeline = CirclePassManager( + [ + EliminateTransposeBoundedLayoutRegionPass(), + RemoveRedundantLayoutOpsPass(), + DeadCodeEliminationPass(), + CompactIndicesPass(), + ], + strategy=CirclePassStrategy.RESTART, + ) + result = pipeline.run(document, CirclePassContext()) + document.verify(raise_on_error=True) + return CircleModel(document.to_bytes()), result + + +def save_layout_optimized_circle( + circle_model: CircleModel, + output_path: str | Path, +) -> tuple[Path, CirclePassManagerResult]: + """Optimize one Circle model and save the resulting binary.""" + optimized, result = optimize_layout_transitions(circle_model) + output = Path(output_path) + output.parent.mkdir(parents=True, exist_ok=True) + optimized.save(output) + return output, result diff --git a/examples/hand_detector/_support/conversion.py b/examples/hand_detector/_support/conversion.py new file mode 100644 index 00000000..7009ec1c --- /dev/null +++ b/examples/hand_detector/_support/conversion.py @@ -0,0 +1,277 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Convert the supplied TFLite hand detector into a static PyTorch model.""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path +from typing import Any + +import numpy as np +import torch +from examples.hand_detector._support.tflite_flatbuffer import OperatorInfo, TFLiteModel + +from examples.hand_detector.hand_detector import HandDetector + + +PADDING_SAME = 0 +FUSED_ACTIVATION_NONE = 0 + + +def _same_padding( + input_size: int, + output_size: int, + kernel_size: int, + stride: int, + dilation: int, +) -> tuple[int, int]: + """Return the before and after padding used by TFLite SAME convolution.""" + effective_kernel = (kernel_size - 1) * dilation + 1 + total = max((output_size - 1) * stride + effective_kernel - input_size, 0) + before = total // 2 + return before, total - before + + +def _conv_config( + model: TFLiteModel, + operation: OperatorInfo, + *, + depthwise: bool, +) -> dict[str, Any]: + """Build a PyTorch Conv2d configuration from one TFLite convolution.""" + input_tensor = model.tensors[operation.inputs[0]] + weight_tensor = model.tensors[operation.inputs[1]] + output_tensor = model.tensors[operation.outputs[0]] + options = operation.options + input_channels = int(input_tensor.shape[3]) + if depthwise: + kernel_h, kernel_w = int(weight_tensor.shape[1]), int(weight_tensor.shape[2]) + output_channels = int(weight_tensor.shape[3]) + groups = input_channels + else: + output_channels = int(weight_tensor.shape[0]) + kernel_h, kernel_w = int(weight_tensor.shape[1]), int(weight_tensor.shape[2]) + groups = 1 + stride_h, stride_w = int(options["stride_h"]), int(options["stride_w"]) + dilation_h, dilation_w = int(options["dilation_h"]), int(options["dilation_w"]) + padding = "same" if int(options["padding"]) == PADDING_SAME else "valid" + if padding == "same": + top, bottom = _same_padding( + int(input_tensor.shape[1]), + int(output_tensor.shape[1]), + kernel_h, + stride_h, + dilation_h, + ) + left, right = _same_padding( + int(input_tensor.shape[2]), + int(output_tensor.shape[2]), + kernel_w, + stride_w, + dilation_w, + ) + else: + left = right = top = bottom = 0 + return { + "in_channels": input_channels, + "out_channels": output_channels, + "kernel_size": [kernel_h, kernel_w], + "stride": [stride_h, stride_w], + "dilation": [dilation_h, dilation_w], + "groups": groups, + "has_bias": len(operation.inputs) >= 3 and operation.inputs[2] >= 0, + "padding": padding, + "pad": [left, right, top, bottom], + } + + +def _decode_constant_map(model: TFLiteModel) -> dict[int, np.ndarray[Any, Any]]: + """Map DEQUANTIZE outputs to FP32 arrays and preserve direct constants.""" + constants: dict[int, np.ndarray[Any, Any]] = {} + for operation in model.operators: + if operation.name != "DEQUANTIZE": + continue + source = operation.inputs[0] + constants[operation.outputs[0]] = model.tensor_array(source).astype(np.float32) + for index, tensor in enumerate(model.tensors): + if model.buffers[tensor.buffer_index]: + constants.setdefault(index, model.tensor_array(index)) + return constants + + +def _convert_channel_pad(paddings_nhwc: np.ndarray[Any, Any]) -> list[int]: + """Convert TFLite NHWC paddings to the order accepted by torch.nn.functional.pad.""" + if paddings_nhwc.shape != (4, 2): + raise ValueError(f"Expected [4, 2] paddings, got {paddings_nhwc.shape}") + n, h, w, c = paddings_nhwc.astype(np.int64).tolist() + if n != [0, 0]: + raise ValueError("Batch padding is not supported by this static converter") + return [w[0], w[1], h[0], h[1], c[0], c[1]] + + +def build_specification( + model: TFLiteModel, +) -> tuple[dict[str, Any], dict[int, np.ndarray[Any, Any]]]: + """Build the JSON graph specification and decoded constant mapping.""" + constants = _decode_constant_map(model) + operations: list[dict[str, Any]] = [] + for operation in model.operators: + if operation.name == "DEQUANTIZE": + continue + if any( + int(value) != FUSED_ACTIVATION_NONE + for key, value in operation.options.items() + if key == "fused_activation" + ): + raise NotImplementedError( + f"Operator {operation.index} uses a fused activation that is " + "not represented separately" + ) + config: dict[str, Any] = {} + if operation.name == "CONV_2D": + config = _conv_config(model, operation, depthwise=False) + elif operation.name == "DEPTHWISE_CONV_2D": + config = _conv_config(model, operation, depthwise=True) + elif operation.name == "PRELU": + config = {"channels": int(model.tensors[operation.inputs[0]].shape[3])} + elif operation.name == "MAX_POOL_2D": + input_tensor = model.tensors[operation.inputs[0]] + output_tensor = model.tensors[operation.outputs[0]] + options = operation.options + kernel_h, kernel_w = int(options["filter_h"]), int(options["filter_w"]) + stride_h, stride_w = int(options["stride_h"]), int(options["stride_w"]) + if int(options["padding"]) == PADDING_SAME: + top, bottom = _same_padding( + int(input_tensor.shape[1]), + int(output_tensor.shape[1]), + kernel_h, + stride_h, + 1, + ) + left, right = _same_padding( + int(input_tensor.shape[2]), + int(output_tensor.shape[2]), + kernel_w, + stride_w, + 1, + ) + if any((left, right, top, bottom)): + raise NotImplementedError( + "This model requires padded max pooling, which is not expected" + ) + config = { + "kernel_size": [kernel_h, kernel_w], + "stride": [stride_h, stride_w], + } + elif operation.name == "PAD": + config = {"pad": _convert_channel_pad(constants[operation.inputs[1]])} + elif operation.name == "RESIZE_BILINEAR": + size = constants[operation.inputs[1]].astype(np.int64).reshape(-1) + config = { + "size": [int(size[0]), int(size[1])], + "align_corners": bool(operation.options["align_corners"]), + "half_pixel_centers": bool(operation.options["half_pixel_centers"]), + } + elif operation.name == "RESHAPE": + shape = constants[operation.inputs[1]].astype(np.int64).reshape(-1).tolist() + config = { + "shape": [int(value) for value in shape], + "nhwc_memory_order": len(model.tensors[operation.inputs[0]].shape) == 4, + } + elif operation.name == "CONCATENATION": + rank = len(model.tensors[operation.inputs[0]].shape) + axis = int(operation.options["axis"]) + if rank == 4: + axis = [0, 2, 3, 1][axis] + config = {"axis": axis} + elif operation.name == "ADD": + config = {} + else: + raise NotImplementedError( + f"Unsupported operator {operation.name} at index {operation.index}" + ) + operations.append( + { + "index": operation.index, + "name": operation.name, + "inputs": [int(value) for value in operation.inputs if value >= 0], + "outputs": [int(value) for value in operation.outputs], + "config": config, + } + ) + specification = { + "format_version": 1, + "source": model.path.name, + "input_layout": "NCHW", + "inputs": [int(value) for value in model.inputs], + "outputs": [int(value) for value in model.outputs], + "operations": operations, + } + return specification, constants + + +def load_parameters( + pytorch_model: HandDetector, + specification: dict[str, Any], + constants: dict[int, np.ndarray[Any, Any]], +) -> None: + """Load converted TFLite constants into the generated PyTorch modules.""" + with torch.no_grad(): + for operation, layer in zip(specification["operations"], pytorch_model.layers): + name = operation["name"] + inputs = operation["inputs"] + if name == "CONV_2D": + weight = torch.from_numpy(constants[int(inputs[1])]).permute(0, 3, 1, 2) + layer.conv.weight.copy_(weight) + if layer.conv.bias is not None: + layer.conv.bias.copy_(torch.from_numpy(constants[int(inputs[2])])) + elif name == "DEPTHWISE_CONV_2D": + weight = torch.from_numpy(constants[int(inputs[1])]).permute(3, 0, 1, 2) + layer.conv.weight.copy_(weight) + if layer.conv.bias is not None: + layer.conv.bias.copy_(torch.from_numpy(constants[int(inputs[2])])) + elif name == "PRELU": + alpha = torch.from_numpy(constants[int(inputs[1])]).reshape(-1) + layer.weight.copy_(alpha) + + +def parse_args() -> argparse.Namespace: + """Parse command-line arguments.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("tflite", type=Path) + parser.add_argument("--spec", type=Path, default=Path("hand_detector_spec.json")) + parser.add_argument("--weights", type=Path, default=Path("hand_detector_float.pt")) + return parser.parse_args() + + +def main() -> None: + """Convert the TFLite graph and write a specification and state dictionary.""" + args = parse_args() + tflite_model = TFLiteModel(args.tflite) + specification, constants = build_specification(tflite_model) + pytorch_model = HandDetector(specification) + load_parameters(pytorch_model, specification, constants) + args.spec.write_text(json.dumps(specification, indent=2), encoding="utf-8") + torch.save(pytorch_model.state_dict(), args.weights) + parameter_count = sum(parameter.numel() for parameter in pytorch_model.parameters()) + print(f"Wrote {args.spec}") + print(f"Wrote {args.weights}") + print(f"Parameters: {parameter_count:,}") + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/_support/data.py b/examples/hand_detector/_support/data.py new file mode 100644 index 00000000..a3de2924 --- /dev/null +++ b/examples/hand_detector/_support/data.py @@ -0,0 +1,91 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Input loading helpers for hand-detector calibration and evaluation.""" + +from __future__ import annotations + +import re +from pathlib import Path + +import numpy as np +import torch + + +_NUMERIC_SUFFIX = re.compile(r"(\d+)$") + + +def normalize_input_array(array: np.ndarray) -> torch.Tensor: + """Convert one supported NumPy image layout into NHWC float32 format.""" + value = np.asarray(array) + if value.ndim == 3: + value = value[None, ...] + if value.shape == (1, 3, 192, 192): + value = np.transpose(value, (0, 2, 3, 1)) + elif value.shape != (1, 192, 192, 3): + raise ValueError( + "Expected [192,192,3], [1,192,192,3], [3,192,192], or " + f"[1,3,192,192], got {value.shape}." + ) + if np.issubdtype(value.dtype, np.integer): + value = value.astype(np.float32) / 255.0 + else: + value = value.astype(np.float32) + return torch.from_numpy(np.ascontiguousarray(value)) + + +def load_npy_inputs( + directory: Path, + limit: int | None = None, + *, + offset: int = 0, + pattern: str = "palm*.npy", +) -> list[torch.Tensor]: + """Load a naturally sorted slice of representative input arrays.""" + if offset < 0: + raise ValueError("offset must be non-negative.") + if limit is not None and limit <= 0: + raise ValueError("limit must be positive when provided.") + paths = sorted(directory.glob(pattern), key=_natural_path_key) + paths = paths[offset : None if limit is None else offset + limit] + if not paths: + raise FileNotFoundError( + f"No input arrays matched {pattern!r} under {directory} at offset {offset}." + ) + return [normalize_input_array(np.load(path)) for path in paths] + + +def list_npy_inputs( + directory: Path, + *, + pattern: str = "palm*.npy", +) -> list[Path]: + """Return naturally sorted input paths without loading their contents.""" + return sorted(directory.glob(pattern), key=_natural_path_key) + + +def make_synthetic_inputs(count: int, seed: int) -> list[torch.Tensor]: + """Create deterministic NHWC [0, 1] inputs for smoke tests only.""" + if count <= 0: + raise ValueError("count must be positive.") + generator = torch.Generator().manual_seed(seed) + return [torch.rand(1, 192, 192, 3, generator=generator) for _ in range(count)] + + +def _natural_path_key(path: Path) -> tuple[str, int, str]: + stem = path.stem + match = _NUMERIC_SUFFIX.search(stem) + if match is None: + return stem, -1, path.name + return stem[: match.start()], int(match.group(1)), path.name diff --git a/examples/hand_detector/_support/quantization.py b/examples/hand_detector/_support/quantization.py new file mode 100644 index 00000000..ea5a7e81 --- /dev/null +++ b/examples/hand_detector/_support/quantization.py @@ -0,0 +1,203 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""WrapQ preparation and Circle export helpers for the hand detector.""" + +from __future__ import annotations + +import copy +from pathlib import Path +from typing import Mapping, Sequence + +import tico +import torch + +from examples.hand_detector._support.circle import save_layout_optimized_circle +from tico.ops import Concat, ResizeBilinear2d +from tico.quantization import convert as freeze_quantization, prepare, QuantStub +from tico.quantization.config.ptq import PTQConfig +from tico.quantization.config.specs import affine +from tico.quantization.wrapq.dtypes import DType +from tico.quantization.wrapq.observers.base import ObserverBase +from tico.quantization.wrapq.observers.minmax import MinMaxObserver +from tico.quantization.wrapq.qscheme import QScheme +from tico.quantization.wrapq.wrappers.nn.quant_conv2d import QuantConv2d +from tico.quantization.wrapq.wrappers.nn.quant_maxpool2d import QuantMaxPool2d +from tico.quantization.wrapq.wrappers.nn.quant_prelu import QuantPReLU +from tico.quantization.wrapq.wrappers.ops.quant_concat import QuantConcat +from tico.quantization.wrapq.wrappers.ops.quant_resize_bilinear import ( + QuantResizeBilinear2d, +) +from tico.quantization.wrapq.wrappers.quant_stub import QuantStubWrapper +from torch import nn + + +_FLOAT_MODULE_TYPES = ( + QuantStub, + nn.Conv2d, + nn.MaxPool2d, + nn.PReLU, + Concat, + ResizeBilinear2d, +) +_QUANT_MODULE_TYPES = ( + QuantStubWrapper, + QuantConv2d, + QuantMaxPool2d, + QuantPReLU, + QuantConcat, + QuantResizeBilinear2d, +) +_SUPPORTED_BIT_WIDTHS = (8, 16) + + +def validate_bit_width(bit_width: int) -> None: + """Reject bit widths unsupported by this example backend profile.""" + if bit_width not in _SUPPORTED_BIT_WIDTHS: + raise ValueError( + f"Expected one of {_SUPPORTED_BIT_WIDTHS}, but received {bit_width}." + ) + + +def quantization_name(bit_width: int) -> str: + """Return the lowercase dtype name for one example bit width.""" + validate_bit_width(bit_width) + return "uint8" if bit_width == 8 else "int16" + + +def quantization_label(bit_width: int) -> str: + """Return the uppercase dtype label for one example bit width.""" + return quantization_name(bit_width).upper() + + +def make_ptq_config( + bit_width: int, + *, + activation_observer: type[ObserverBase] = MinMaxObserver, + activation_observer_kwargs: Mapping[str, object] | None = None, +) -> PTQConfig: + """Create the example PTQ policy with a selectable activation observer.""" + validate_bit_width(bit_width) + if bit_width == 8: + dtype = DType.uint(8) + activation_qscheme = QScheme.PER_TENSOR_ASYMM + weight_qscheme = QScheme.PER_CHANNEL_ASYMM + else: + dtype = DType.int(16) + activation_qscheme = QScheme.PER_TENSOR_SYMM + weight_qscheme = QScheme.PER_CHANNEL_SYMM + + return PTQConfig( + activation=affine( + dtype, + qscheme=activation_qscheme, + observer=activation_observer, + **dict(activation_observer_kwargs or {}), + ), + weight=affine( + dtype, + qscheme=weight_qscheme, + observer=MinMaxObserver, + ), + strict_wrap=False, + ) + + +def calibrate(model: nn.Module, samples: Sequence[torch.Tensor]) -> None: + """Collect observer statistics from representative NHWC inputs.""" + if not samples: + raise ValueError("Calibration requires at least one input sample.") + model.eval() + with torch.inference_mode(): + for sample in samples: + model(sample) + + +def prepare_quantized_candidate( + float_model: nn.Module, + bit_width: int, + *, + activation_observer: type[ObserverBase] = MinMaxObserver, + activation_observer_kwargs: Mapping[str, object] | None = None, +) -> nn.Module: + """Clone, WrapQ-prepare, and validate one hand-detector candidate.""" + candidate = copy.deepcopy(float_model).eval() + expected_wrappers = sum( + isinstance(module, _FLOAT_MODULE_TYPES) for module in candidate.modules() + ) + candidate = prepare( + candidate, + make_ptq_config( + bit_width, + activation_observer=activation_observer, + activation_observer_kwargs=activation_observer_kwargs, + ), + inplace=True, + ) + actual_wrappers = sum( + isinstance(module, _QUANT_MODULE_TYPES) for module in candidate.modules() + ) + if actual_wrappers != expected_wrappers: + raise RuntimeError( + f"Expected {expected_wrappers} quantization wrappers for " + f"{quantization_label(bit_width)}, but found {actual_wrappers}." + ) + return candidate + + +def quantize_candidate( + float_model: nn.Module, + bit_width: int, + calibration_samples: Sequence[torch.Tensor], + *, + activation_observer: type[ObserverBase] = MinMaxObserver, + activation_observer_kwargs: Mapping[str, object] | None = None, +) -> nn.Module: + """Prepare, calibrate, and freeze one hand-detector candidate.""" + candidate = prepare_quantized_candidate( + float_model, + bit_width, + activation_observer=activation_observer, + activation_observer_kwargs=activation_observer_kwargs, + ) + calibrate(candidate, calibration_samples) + candidate = freeze_quantization(candidate, inplace=True) + return candidate.eval() + + +def get_example_inputs(model: nn.Module) -> tuple[torch.Tensor, ...]: + """Return static example inputs exposed by a converted detector module.""" + if not hasattr(model, "get_example_inputs"): + raise TypeError("The hand detector must expose get_example_inputs().") + inputs = model.get_example_inputs() # type: ignore[attr-defined] + if not isinstance(inputs, tuple) or not inputs: + raise TypeError("get_example_inputs() must return a non-empty tuple.") + if not all(isinstance(value, torch.Tensor) for value in inputs): + raise TypeError("Every example input must be a Tensor.") + return inputs + + +def export_quantized_circle( + quantized_model: nn.Module, + output_path: str | Path, +) -> Path: + """Export, layout-optimize, and save a frozen fake-quantized Circle model.""" + with torch.inference_mode(): + circle_model = tico.convert( + quantized_model.eval(), + get_example_inputs(quantized_model), + ) + output, result = save_layout_optimized_circle(circle_model, output_path) + print(f"Circle layout optimization reported {result.changes} changes.") + return output diff --git a/examples/hand_detector/_support/tflite_flatbuffer.py b/examples/hand_detector/_support/tflite_flatbuffer.py new file mode 100644 index 00000000..5ab4a9ca --- /dev/null +++ b/examples/hand_detector/_support/tflite_flatbuffer.py @@ -0,0 +1,359 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Minimal FlatBuffer reader for the TensorFlow Lite fields used by this model.""" + +from __future__ import annotations + +import struct +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +import numpy as np + + +TENSOR_DTYPES: dict[int, np.dtype[Any]] = { + 0: np.dtype(np.float32), + 1: np.dtype(np.float16), + 2: np.dtype(np.int32), + 3: np.dtype(np.uint8), + 4: np.dtype(np.int64), + 6: np.dtype(np.bool_), + 7: np.dtype(np.int16), + 9: np.dtype(np.int8), + 10: np.dtype(np.float64), + 12: np.dtype(np.uint64), + 15: np.dtype(np.uint32), + 16: np.dtype(np.uint16), +} + +BUILTIN_OPERATOR_NAMES = { + 0: "ADD", + 2: "CONCATENATION", + 3: "CONV_2D", + 4: "DEPTHWISE_CONV_2D", + 6: "DEQUANTIZE", + 17: "MAX_POOL_2D", + 22: "RESHAPE", + 23: "RESIZE_BILINEAR", + 34: "PAD", + 54: "PRELU", +} + + +class FlatBufferReader: + """Read scalar, vector, string, and table fields from a FlatBuffer byte array.""" + + def __init__(self, data: bytes) -> None: + """Store the immutable FlatBuffer payload.""" + self.data = data + + def _unpack(self, fmt: str, offset: int) -> Any: + """Unpack one scalar value at an absolute byte offset.""" + return struct.unpack_from(fmt, self.data, offset)[0] + + def u8(self, offset: int) -> int: + """Read an unsigned 8-bit integer.""" + return self.data[offset] + + def i8(self, offset: int) -> int: + """Read a signed 8-bit integer.""" + return int(self._unpack(" int: + """Read an unsigned 16-bit integer.""" + return int(self._unpack(" int: + """Read a signed 32-bit integer.""" + return int(self._unpack(" int: + """Read a signed 64-bit integer.""" + return int(self._unpack(" int: + """Read an unsigned 32-bit integer.""" + return int(self._unpack(" float: + """Read a 32-bit floating-point value.""" + return float(self._unpack(" int: + """Return the absolute position of the FlatBuffer root table.""" + return self.u32(0) + + def field(self, table: int, slot: int) -> int | None: + """Return the absolute scalar field position for a table slot.""" + vtable = table - self.i32(table) + vtable_size = self.u16(vtable) + entry = vtable + 4 + 2 * slot + if entry + 2 > vtable + vtable_size: + return None + relative = self.u16(entry) + return None if relative == 0 else table + relative + + def indirect(self, offset: int) -> int: + """Follow a FlatBuffer uoffset.""" + return offset + self.u32(offset) + + def table(self, parent: int, slot: int) -> int | None: + """Return a nested table position.""" + field = self.field(parent, slot) + return None if field is None else self.indirect(field) + + def scalar_i8(self, table: int, slot: int, default: int = 0) -> int: + """Read an int8 table field with a default.""" + field = self.field(table, slot) + return default if field is None else self.i8(field) + + def scalar_i32(self, table: int, slot: int, default: int = 0) -> int: + """Read an int32 table field with a default.""" + field = self.field(table, slot) + return default if field is None else self.i32(field) + + def scalar_u32(self, table: int, slot: int, default: int = 0) -> int: + """Read a uint32 table field with a default.""" + field = self.field(table, slot) + return default if field is None else self.u32(field) + + def scalar_bool(self, table: int, slot: int, default: bool = False) -> bool: + """Read a boolean table field with a default.""" + field = self.field(table, slot) + return default if field is None else bool(self.u8(field)) + + def vector(self, table: int, slot: int) -> tuple[int, int] | None: + """Return a vector data position and element count.""" + field = self.field(table, slot) + if field is None: + return None + vector = self.indirect(field) + return vector + 4, self.u32(vector) + + def vector_i32(self, table: int, slot: int) -> list[int]: + """Read an int32 vector.""" + vector = self.vector(table, slot) + if vector is None: + return [] + data, size = vector + return [self.i32(data + 4 * index) for index in range(size)] + + def vector_i64(self, table: int, slot: int) -> list[int]: + """Read an int64 vector.""" + vector = self.vector(table, slot) + if vector is None: + return [] + data, size = vector + return [self.i64(data + 8 * index) for index in range(size)] + + def vector_f32(self, table: int, slot: int) -> list[float]: + """Read a float32 vector.""" + vector = self.vector(table, slot) + if vector is None: + return [] + data, size = vector + return [self.f32(data + 4 * index) for index in range(size)] + + def vector_u8(self, table: int, slot: int) -> bytes: + """Read a byte vector.""" + vector = self.vector(table, slot) + if vector is None: + return b"" + data, size = vector + return self.data[data : data + size] + + def vector_tables(self, table: int, slot: int) -> list[int]: + """Read a vector of nested tables.""" + vector = self.vector(table, slot) + if vector is None: + return [] + data, size = vector + return [self.indirect(data + 4 * index) for index in range(size)] + + def string(self, table: int, slot: int) -> str | None: + """Read a UTF-8 string field.""" + field = self.field(table, slot) + if field is None: + return None + string = self.indirect(field) + size = self.u32(string) + return self.data[string + 4 : string + 4 + size].decode("utf-8", "replace") + + +@dataclass(frozen=True) +class TensorInfo: + """Describe one TFLite tensor.""" + + shape: tuple[int, ...] + tensor_type: int + buffer_index: int + name: str + + +@dataclass(frozen=True) +class OperatorInfo: + """Describe one TFLite operator and its decoded builtin options.""" + + index: int + name: str + inputs: tuple[int, ...] + outputs: tuple[int, ...] + options: dict[str, Any] + + +class TFLiteModel: + """Parse the subset of the TFLite schema needed by the MediaPipe detector.""" + + def __init__(self, path: str | Path) -> None: + """Read and decode the supported portions of one TFLite model.""" + self.path = Path(path) + self.data = self.path.read_bytes() + self.reader = FlatBufferReader(self.data) + if self.data[4:8] != b"TFL3": + raise ValueError(f"{self.path} is not a TFLite FlatBuffer") + self.root = self.reader.root_table() + self.operator_codes = self._parse_operator_codes() + self.buffers = self._parse_buffers() + subgraphs = self.reader.vector_tables(self.root, 2) + if len(subgraphs) != 1: + raise ValueError(f"Expected one subgraph, found {len(subgraphs)}") + self.subgraph = subgraphs[0] + self.tensors = self._parse_tensors() + self.inputs = tuple(self.reader.vector_i32(self.subgraph, 1)) + self.outputs = tuple(self.reader.vector_i32(self.subgraph, 2)) + self.operators = self._parse_operators() + + def _parse_operator_codes(self) -> list[int]: + """Decode the builtin operator code table.""" + result: list[int] = [] + for table in self.reader.vector_tables(self.root, 1): + deprecated = self.reader.scalar_i8(table, 0, 0) + builtin = self.reader.scalar_i32(table, 3, deprecated) + if builtin == 0 and deprecated != 0: + builtin = deprecated + result.append(builtin) + return result + + def _parse_buffers(self) -> list[bytes]: + """Decode all inline buffer payloads.""" + return [ + self.reader.vector_u8(table, 0) + for table in self.reader.vector_tables(self.root, 4) + ] + + def _parse_tensors(self) -> list[TensorInfo]: + """Decode tensor metadata from the only subgraph.""" + result: list[TensorInfo] = [] + for table in self.reader.vector_tables(self.subgraph, 0): + result.append( + TensorInfo( + shape=tuple(self.reader.vector_i32(table, 0)), + tensor_type=self.reader.scalar_i8(table, 1, 0), + buffer_index=self.reader.scalar_u32(table, 2, 0), + name=self.reader.string(table, 3) or "", + ) + ) + return result + + def _decode_options(self, name: str, table: int | None) -> dict[str, Any]: + """Decode builtin options used by one supported operator.""" + if table is None: + return {} + reader = self.reader + if name == "CONV_2D": + return { + "padding": reader.scalar_i8(table, 0, 0), + "stride_w": reader.scalar_i32(table, 1, 1), + "stride_h": reader.scalar_i32(table, 2, 1), + "fused_activation": reader.scalar_i8(table, 3, 0), + "dilation_w": reader.scalar_i32(table, 4, 1), + "dilation_h": reader.scalar_i32(table, 5, 1), + } + if name == "DEPTHWISE_CONV_2D": + return { + "padding": reader.scalar_i8(table, 0, 0), + "stride_w": reader.scalar_i32(table, 1, 1), + "stride_h": reader.scalar_i32(table, 2, 1), + "depth_multiplier": reader.scalar_i32(table, 3, 1), + "fused_activation": reader.scalar_i8(table, 4, 0), + "dilation_w": reader.scalar_i32(table, 5, 1), + "dilation_h": reader.scalar_i32(table, 6, 1), + } + if name == "MAX_POOL_2D": + return { + "padding": reader.scalar_i8(table, 0, 0), + "stride_w": reader.scalar_i32(table, 1, 1), + "stride_h": reader.scalar_i32(table, 2, 1), + "filter_w": reader.scalar_i32(table, 3, 1), + "filter_h": reader.scalar_i32(table, 4, 1), + "fused_activation": reader.scalar_i8(table, 5, 0), + } + if name == "ADD": + return {"fused_activation": reader.scalar_i8(table, 0, 0)} + if name == "CONCATENATION": + return { + "axis": reader.scalar_i32(table, 0, 0), + "fused_activation": reader.scalar_i8(table, 1, 0), + } + if name == "RESHAPE": + return {"new_shape": reader.vector_i32(table, 0)} + if name == "RESIZE_BILINEAR": + # TFLite keeps two deprecated fields before the active options: + # slot 0: new_height + # slot 1: new_width + # slot 2: align_corners + # slot 3: half_pixel_centers + return { + "align_corners": reader.scalar_bool(table, 2, False), + "half_pixel_centers": reader.scalar_bool(table, 3, False), + } + return {} + + def _parse_operators(self) -> list[OperatorInfo]: + """Decode operators in execution order.""" + result: list[OperatorInfo] = [] + for index, table in enumerate(self.reader.vector_tables(self.subgraph, 3)): + opcode_index = self.reader.scalar_u32(table, 0, 0) + builtin = self.operator_codes[opcode_index] + name = BUILTIN_OPERATOR_NAMES.get(builtin) + if name is None: + raise NotImplementedError( + f"Builtin operator code {builtin} at index {index} is unsupported" + ) + result.append( + OperatorInfo( + index=index, + name=name, + inputs=tuple(self.reader.vector_i32(table, 1)), + outputs=tuple(self.reader.vector_i32(table, 2)), + options=self._decode_options(name, self.reader.table(table, 4)), + ) + ) + return result + + def tensor_array(self, tensor_index: int) -> np.ndarray[Any, Any]: + """Return a constant tensor as a NumPy array.""" + tensor = self.tensors[tensor_index] + try: + dtype = TENSOR_DTYPES[tensor.tensor_type] + except KeyError as exc: + raise NotImplementedError( + f"Unsupported TFLite tensor type {tensor.tensor_type}" + ) from exc + payload = self.buffers[tensor.buffer_index] + if not payload: + raise ValueError(f"Tensor {tensor_index} does not contain constant data") + return np.frombuffer(payload, dtype=dtype).reshape(tensor.shape).copy() diff --git a/examples/hand_detector/_support/verify_circle_layout.py b/examples/hand_detector/_support/verify_circle_layout.py new file mode 100644 index 00000000..3fd5233c --- /dev/null +++ b/examples/hand_detector/_support/verify_circle_layout.py @@ -0,0 +1,376 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Validate the NHWC input ABI and removed Transpose round trips in Circle.""" + +from __future__ import annotations + +import argparse +import struct +from dataclasses import asdict, dataclass +from pathlib import Path +from typing import Iterable + +from examples.hand_detector._support.tflite_flatbuffer import FlatBufferReader + + +ADD = 0 +TRANSPOSE = 39 +EXPECTED_INPUT_SHAPE = (1, 192, 192, 3) + + +@dataclass(frozen=True) +class OperatorInfo: + """Describe the Circle operator fields needed by the layout verifier.""" + + builtin_code: int + inputs: tuple[int, ...] + outputs: tuple[int, ...] + + +@dataclass(frozen=True) +class LayoutVerificationSummary: + """Describe the observable layout properties of one Circle model.""" + + path: str + size_bytes: int + input_shapes: tuple[tuple[int, ...], ...] + transpose_count: int + add_count: int + consecutive_inverse_transpose_pairs: int + transpose_add_round_trips: int + + def to_dict(self) -> dict[str, object]: + """Return a JSON-serializable representation of the summary.""" + + return asdict(self) + + +def _parse_operator_codes(reader: FlatBufferReader, root: int) -> list[int]: + """Decode builtin operator codes from a Circle model.""" + + result: list[int] = [] + for table in reader.vector_tables(root, 1): + deprecated = reader.scalar_i8(table, 0, 0) + builtin = reader.scalar_i32(table, 3, deprecated) + if builtin == 0 and deprecated != 0: + builtin = deprecated + result.append(builtin) + return result + + +def _parse_tensor_shapes( + reader: FlatBufferReader, + subgraph: int, +) -> list[tuple[int, ...]]: + """Decode every tensor shape in one Circle subgraph.""" + + return [ + tuple(reader.vector_i32(table, 0)) + for table in reader.vector_tables(subgraph, 0) + ] + + +def _parse_tensor_buffers(reader: FlatBufferReader, subgraph: int) -> list[int]: + """Decode every tensor buffer index in one Circle subgraph.""" + + return [ + reader.scalar_u32(table, 2, 0) for table in reader.vector_tables(subgraph, 0) + ] + + +def _parse_operators( + reader: FlatBufferReader, + subgraph: int, + operator_codes: list[int], +) -> list[OperatorInfo]: + """Decode operator types and tensor connections from one subgraph.""" + + result: list[OperatorInfo] = [] + for table in reader.vector_tables(subgraph, 3): + opcode_index = reader.scalar_u32(table, 0, 0) + result.append( + OperatorInfo( + builtin_code=operator_codes[opcode_index], + inputs=tuple(reader.vector_i32(table, 1)), + outputs=tuple(reader.vector_i32(table, 2)), + ) + ) + return result + + +def _const_i32_data( + reader: FlatBufferReader, + root: int, + tensor_buffers: list[int], + tensor_index: int, +) -> tuple[int, ...] | None: + """Decode one inline INT32 constant tensor.""" + + if tensor_index < 0 or tensor_index >= len(tensor_buffers): + return None + buffer_index = tensor_buffers[tensor_index] + buffers = reader.vector_tables(root, 4) + if buffer_index <= 0 or buffer_index >= len(buffers): + return None + payload = reader.vector_u8(buffers[buffer_index], 0) + if not payload or len(payload) % 4 != 0: + return None + return tuple( + struct.unpack_from(" bool: + """Return whether composing two permutations produces the identity.""" + + first_values = tuple(first) + second_values = tuple(second) + if len(first_values) != len(second_values): + return False + if sorted(first_values) != list(range(len(first_values))): + return False + if sorted(second_values) != list(range(len(second_values))): + return False + return all( + first_values[second_values[index]] == index + for index in range(len(second_values)) + ) + + +def _build_edges( + operators: list[OperatorInfo], +) -> tuple[dict[int, int], dict[int, list[int]]]: + """Build tensor producer and consumer indexes.""" + + producers: dict[int, int] = {} + consumers: dict[int, list[int]] = {} + for operator_index, operator in enumerate(operators): + for tensor_index in operator.outputs: + if tensor_index >= 0: + producers[tensor_index] = operator_index + for tensor_index in operator.inputs: + if tensor_index >= 0: + consumers.setdefault(tensor_index, []).append(operator_index) + return producers, consumers + + +def _transpose_permutation( + operator: OperatorInfo, + *, + reader: FlatBufferReader, + root: int, + tensor_buffers: list[int], +) -> tuple[int, ...] | None: + """Return the constant permutation used by one Transpose operator.""" + + if operator.builtin_code != TRANSPOSE or len(operator.inputs) < 2: + return None + return _const_i32_data( + reader, + root, + tensor_buffers, + operator.inputs[1], + ) + + +def _count_consecutive_inverse_pairs( + operators: list[OperatorInfo], + producers: dict[int, int], + *, + reader: FlatBufferReader, + root: int, + tensor_buffers: list[int], +) -> int: + """Count consecutive inverse Transpose operator pairs.""" + + count = 0 + for operator in operators: + if operator.builtin_code != TRANSPOSE or not operator.inputs: + continue + producer_index = producers.get(operator.inputs[0]) + if producer_index is None: + continue + producer = operators[producer_index] + first = _transpose_permutation( + producer, + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + second = _transpose_permutation( + operator, + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + if first is not None and second is not None and _is_inverse(first, second): + count += 1 + return count + + +def _count_transpose_add_round_trips( + operators: list[OperatorInfo], + producers: dict[int, int], + consumers: dict[int, list[int]], + *, + reader: FlatBufferReader, + root: int, + tensor_buffers: list[int], +) -> int: + """Count inverse Transpose round trips whose middle operator is ADD.""" + + count = 0 + for operator in operators: + if operator.builtin_code != ADD: + continue + if len(operator.inputs) != 2 or len(operator.outputs) != 1: + continue + input_producers = [producers.get(index) for index in operator.inputs] + if any(index is None for index in input_producers): + continue + input_transposes = [operators[int(index)] for index in input_producers] + first = _transpose_permutation( + input_transposes[0], + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + second = _transpose_permutation( + input_transposes[1], + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + if first is None or second is None or first != second: + continue + output_consumers = consumers.get(operator.outputs[0], []) + if len(output_consumers) != 1: + continue + output_transpose = operators[output_consumers[0]] + inverse = _transpose_permutation( + output_transpose, + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + if inverse is not None and _is_inverse(first, inverse): + count += 1 + return count + + +def verify_circle_layout( + path: str | Path, + *, + expected_transpose_count: int = 0, +) -> dict[str, object]: + """Validate the hand detector's NHWC input and optimized layout graph.""" + + circle_path = Path(path) + data = circle_path.read_bytes() + if len(data) < 8 or data[4:8] != b"CIR0": + raise ValueError(f"{circle_path} does not contain a Circle CIR0 identifier") + + reader = FlatBufferReader(data) + root = reader.root_table() + subgraphs = reader.vector_tables(root, 2) + if len(subgraphs) != 1: + raise RuntimeError(f"Expected one subgraph, found {len(subgraphs)}.") + subgraph = subgraphs[0] + operator_codes = _parse_operator_codes(reader, root) + tensor_shapes = _parse_tensor_shapes(reader, subgraph) + tensor_buffers = _parse_tensor_buffers(reader, subgraph) + operators = _parse_operators(reader, subgraph, operator_codes) + input_indices = tuple(reader.vector_i32(subgraph, 1)) + input_shapes = tuple(tensor_shapes[index] for index in input_indices) + if input_shapes != (EXPECTED_INPUT_SHAPE,): + raise RuntimeError( + "Expected one NHWC graph input with shape " + f"{list(EXPECTED_INPUT_SHAPE)}, found {input_shapes}." + ) + + producers, consumers = _build_edges(operators) + inverse_pairs = _count_consecutive_inverse_pairs( + operators, + producers, + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + add_round_trips = _count_transpose_add_round_trips( + operators, + producers, + consumers, + reader=reader, + root=root, + tensor_buffers=tensor_buffers, + ) + if inverse_pairs: + raise RuntimeError( + f"Found {inverse_pairs} consecutive inverse Transpose pairs." + ) + if add_round_trips: + raise RuntimeError( + f"Found {add_round_trips} Transpose-ADD-Transpose round trips." + ) + transpose_count = sum(operator.builtin_code == TRANSPOSE for operator in operators) + if transpose_count != expected_transpose_count: + raise RuntimeError( + f"Expected {expected_transpose_count} Transpose operators, " + f"found {transpose_count}." + ) + + summary = LayoutVerificationSummary( + path=str(circle_path), + size_bytes=len(data), + input_shapes=input_shapes, + transpose_count=transpose_count, + add_count=sum(operator.builtin_code == ADD for operator in operators), + consecutive_inverse_transpose_pairs=inverse_pairs, + transpose_add_round_trips=add_round_trips, + ) + return summary.to_dict() + + +def parse_args() -> argparse.Namespace: + """Parse command-line arguments.""" + + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("circle", type=Path) + parser.add_argument( + "--expected-transpose-count", + type=int, + default=0, + help="Require this exact number of Circle Transpose operators.", + ) + return parser.parse_args() + + +def main() -> None: + """Run the Circle layout verifier and print its summary.""" + + args = parse_args() + summary = verify_circle_layout( + args.circle, + expected_transpose_count=args.expected_transpose_count, + ) + print(f"Verified NHWC input shape {list(EXPECTED_INPUT_SHAPE)}.") + print("Verified zero consecutive inverse Transpose pairs.") + print("Verified zero Transpose-ADD-Transpose round trips.") + print("Verified Circle Transpose operator count: " f"{summary['transpose_count']}.") + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/_support/verify_circle_resize.py b/examples/hand_detector/_support/verify_circle_resize.py new file mode 100644 index 00000000..acecf699 --- /dev/null +++ b/examples/hand_detector/_support/verify_circle_resize.py @@ -0,0 +1,103 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Count Circle RESIZE_BILINEAR operators in an exported model.""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +from examples.hand_detector._support.tflite_flatbuffer import FlatBufferReader + + +RESIZE_BILINEAR_BUILTIN_CODE = 23 + + +def parse_operator_codes(reader: FlatBufferReader, root: int) -> list[int]: + """Decode builtin operator codes from a Circle model table.""" + result: list[int] = [] + for table in reader.vector_tables(root, 1): + deprecated_code = reader.scalar_i8(table, 0, 0) + builtin_code = reader.scalar_i32(table, 3, deprecated_code) + if builtin_code == 0 and deprecated_code != 0: + builtin_code = deprecated_code + result.append(builtin_code) + return result + + +def read_resize_bilinear_options(path: Path) -> list[tuple[bool, bool]]: + """Return coordinate options of all first-subgraph RESIZE_BILINEAR nodes.""" + data = path.read_bytes() + if len(data) < 8 or data[4:8] != b"CIR0": + raise ValueError(f"{path} does not contain a Circle CIR0 identifier") + reader = FlatBufferReader(data) + root = reader.root_table() + operator_codes = parse_operator_codes(reader, root) + subgraphs = reader.vector_tables(root, 2) + if not subgraphs: + raise ValueError("The Circle model does not contain a subgraph") + + result: list[tuple[bool, bool]] = [] + for operator in reader.vector_tables(subgraphs[0], 3): + opcode_index = reader.scalar_u32(operator, 0, 0) + if operator_codes[opcode_index] != RESIZE_BILINEAR_BUILTIN_CODE: + continue + options = reader.table(operator, 4) + if options is None: + raise ValueError("RESIZE_BILINEAR does not contain builtin options") + # ResizeBilinearOptions keeps deprecated new_height/new_width in slots 0/1. + align_corners = reader.scalar_bool(options, 2, False) + half_pixel_centers = reader.scalar_bool(options, 3, False) + result.append((align_corners, half_pixel_centers)) + return result + + +def count_resize_bilinear(path: Path) -> int: + """Return the number of RESIZE_BILINEAR operators in the first subgraph.""" + return len(read_resize_bilinear_options(path)) + + +def parse_args() -> argparse.Namespace: + """Parse command-line arguments.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("circle", type=Path) + parser.add_argument("--expected-count", type=int, default=2) + return parser.parse_args() + + +def main() -> None: + """Validate the expected number of Circle ResizeBilinear operators.""" + args = parse_args() + options = read_resize_bilinear_options(args.circle) + actual_count = len(options) + if actual_count != args.expected_count: + raise RuntimeError( + f"Expected {args.expected_count} RESIZE_BILINEAR operators, " + f"found {actual_count}" + ) + expected_options = [(False, True)] * args.expected_count + if options != expected_options: + raise RuntimeError( + "Unexpected Circle ResizeBilinear options: " + f"expected {expected_options}, found {options}" + ) + print( + f"Verified {actual_count} Circle RESIZE_BILINEAR operators with " + f"alignCorners=False and halfPixelCenters=True in {args.circle}." + ) + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/_support/verify_quantized_circle.py b/examples/hand_detector/_support/verify_quantized_circle.py new file mode 100644 index 00000000..e0c1d8e2 --- /dev/null +++ b/examples/hand_detector/_support/verify_quantized_circle.py @@ -0,0 +1,602 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Validate a quantized hand-detector Circle model without running inference.""" + +from __future__ import annotations + +import argparse +from dataclasses import asdict, dataclass +from pathlib import Path +from typing import Any, Iterable + +from examples.hand_detector._support.tflite_flatbuffer import FlatBufferReader + + +ADD = 0 +CONCATENATION = 2 +CONV_2D = 3 +DEPTHWISE_CONV_2D = 4 +DEQUANTIZE = 6 +MAX_POOL_2D = 17 +RESHAPE = 22 +RESIZE_BILINEAR = 23 +PAD = 34 +TRANSPOSE = 39 +PRELU = 54 +SLICE = 65 +QUANTIZE = 114 + +PADDING_SAME = 0 + +TENSOR_FLOAT32 = 0 +TENSOR_INT32 = 2 +TENSOR_UINT8 = 3 +TENSOR_INT64 = 4 +TENSOR_INT16 = 7 + +TENSOR_TYPE_NAMES = { + TENSOR_FLOAT32: "FLOAT32", + TENSOR_INT32: "INT32", + TENSOR_UINT8: "UINT8", + TENSOR_INT64: "INT64", + TENSOR_INT16: "INT16", +} + +# This table captures target-backend constraints for data-preserving operators. +# ResizeBilinear is intentionally False because the target NPU accepts distinct +# affine qparams for the input and output tensors. +_DATA_OPERATOR_REQUIRES_SHARED_QPARAMS = { + MAX_POOL_2D: True, + RESHAPE: True, + RESIZE_BILINEAR: False, + PAD: True, + TRANSPOSE: True, + SLICE: True, +} + +OPERATOR_NAMES = { + ADD: "ADD", + CONCATENATION: "CONCATENATION", + CONV_2D: "CONV_2D", + DEPTHWISE_CONV_2D: "DEPTHWISE_CONV_2D", + DEQUANTIZE: "DEQUANTIZE", + MAX_POOL_2D: "MAX_POOL_2D", + RESHAPE: "RESHAPE", + RESIZE_BILINEAR: "RESIZE_BILINEAR", + PAD: "PAD", + TRANSPOSE: "TRANSPOSE", + PRELU: "PRELU", + SLICE: "SLICE", + QUANTIZE: "QUANTIZE", +} + + +@dataclass(frozen=True) +class QuantizationInfo: + """Describe one tensor's affine quantization metadata.""" + + scales: tuple[float, ...] + zero_points: tuple[int, ...] + quantized_dimension: int + + +@dataclass(frozen=True) +class CircleTensorInfo: + """Describe the Circle tensor fields needed by the verifier.""" + + shape: tuple[int, ...] + tensor_type: int + buffer_index: int + name: str + quantization: QuantizationInfo | None + + +@dataclass(frozen=True) +class CircleOperatorInfo: + """Describe the Circle operator fields needed by the verifier.""" + + builtin_code: int + inputs: tuple[int, ...] + outputs: tuple[int, ...] + options_table: int | None + + +def _parse_operator_codes(reader: FlatBufferReader, root: int) -> list[int]: + """Decode builtin operator codes from a Circle model.""" + result: list[int] = [] + for table in reader.vector_tables(root, 1): + deprecated = reader.scalar_i8(table, 0, 0) + builtin = reader.scalar_i32(table, 3, deprecated) + if builtin == 0 and deprecated != 0: + builtin = deprecated + result.append(builtin) + return result + + +def _parse_quantization( + reader: FlatBufferReader, + tensor_table: int, +) -> QuantizationInfo | None: + """Decode one optional Circle QuantizationParameters table.""" + table = reader.table(tensor_table, 4) + if table is None: + return None + scales = tuple(reader.vector_f32(table, 2)) + zero_points = tuple(reader.vector_i64(table, 3)) + if not scales and not zero_points: + return None + return QuantizationInfo( + scales=scales, + zero_points=zero_points, + quantized_dimension=reader.scalar_i32(table, 6, 0), + ) + + +def _parse_tensors( + reader: FlatBufferReader, + subgraph: int, +) -> list[CircleTensorInfo]: + """Decode tensor shapes, types, names, buffers, and quantization metadata.""" + result: list[CircleTensorInfo] = [] + for table in reader.vector_tables(subgraph, 0): + result.append( + CircleTensorInfo( + shape=tuple(reader.vector_i32(table, 0)), + tensor_type=reader.scalar_i8(table, 1, 0), + buffer_index=reader.scalar_u32(table, 2, 0), + name=reader.string(table, 3) or "", + quantization=_parse_quantization(reader, table), + ) + ) + return result + + +def _parse_operators( + reader: FlatBufferReader, + subgraph: int, + operator_codes: list[int], +) -> list[CircleOperatorInfo]: + """Decode operator inputs, outputs, builtin codes, and option tables.""" + result: list[CircleOperatorInfo] = [] + for table in reader.vector_tables(subgraph, 3): + opcode_index = reader.scalar_u32(table, 0, 0) + result.append( + CircleOperatorInfo( + builtin_code=operator_codes[opcode_index], + inputs=tuple(reader.vector_i32(table, 1)), + outputs=tuple(reader.vector_i32(table, 2)), + options_table=reader.table(table, 4), + ) + ) + return result + + +def _expected_tensor_type(bit_width: int) -> int: + """Return the Circle tensor type required for one configured bit width.""" + if bit_width == 8: + return TENSOR_UINT8 + if bit_width == 16: + return TENSOR_INT16 + raise ValueError(f"Unsupported bit width: {bit_width}") + + +def _expected_bias_type(bit_width: int) -> int: + """Return the accumulator-backed Circle bias type for one bit width.""" + if bit_width == 8: + return TENSOR_INT32 + if bit_width == 16: + return TENSOR_INT64 + raise ValueError(f"Unsupported bit width: {bit_width}") + + +def _type_name(tensor_type: int) -> str: + """Return a readable Circle tensor type name.""" + return TENSOR_TYPE_NAMES.get(tensor_type, str(tensor_type)) + + +def _require_quantized_tensor( + tensor: CircleTensorInfo, + expected_type: int, + *, + context: str, + expected_qparam_count: int | None = None, + expected_axis: int | None = None, +) -> None: + """Validate one tensor's integer type and affine quantization metadata.""" + if tensor.tensor_type != expected_type: + raise RuntimeError( + f"{context} must be {_type_name(expected_type)}, but " + f"{tensor.name!r} is {_type_name(tensor.tensor_type)}." + ) + quantization = tensor.quantization + if quantization is None: + raise RuntimeError(f"{context} {tensor.name!r} has no quantization metadata.") + if not quantization.scales: + raise RuntimeError(f"{context} {tensor.name!r} has no quantization scale.") + if not quantization.zero_points: + raise RuntimeError(f"{context} {tensor.name!r} has no zero point.") + if len(quantization.scales) != len(quantization.zero_points): + raise RuntimeError( + f"{context} {tensor.name!r} has mismatched scale and zero-point " + "vector lengths." + ) + if expected_type == TENSOR_INT16 and any( + zero_point != 0 for zero_point in quantization.zero_points + ): + raise RuntimeError( + f"{context} {tensor.name!r} must use zero point 0 for symmetric INT16." + ) + if expected_qparam_count is not None: + if len(quantization.scales) != expected_qparam_count: + raise RuntimeError( + f"{context} {tensor.name!r} must contain " + f"{expected_qparam_count} qparams, but contains " + f"{len(quantization.scales)}." + ) + if expected_axis is not None: + if quantization.quantized_dimension != expected_axis: + raise RuntimeError( + f"{context} {tensor.name!r} must use quantized dimension " + f"{expected_axis}, but uses {quantization.quantized_dimension}." + ) + + +def _require_same_qparams( + tensors: Iterable[CircleTensorInfo], + *, + context: str, +) -> None: + """Require every tensor to use the exact same affine quantization parameters.""" + values = list(tensors) + if len(values) < 2: + return + reference = values[0].quantization + if reference is None: + raise RuntimeError(f"{context} reference tensor has no quantization metadata.") + for tensor in values[1:]: + if tensor.quantization != reference: + raise RuntimeError( + f"{context} requires identical scale and zero point, but " + f"{values[0].name!r} and {tensor.name!r} differ." + ) + + +def _tensor_indices(indices: Iterable[int]) -> list[int]: + """Drop optional negative tensor indices from an operator input list.""" + return [index for index in indices if index >= 0] + + +def _require_per_tensor_data( + tensors: list[CircleTensorInfo], + indices: Iterable[int], + expected_type: int, + *, + context: str, +) -> list[CircleTensorInfo]: + """Validate and return data tensors with one affine qparam each.""" + result = [tensors[index] for index in _tensor_indices(indices)] + for tensor in result: + _require_quantized_tensor( + tensor, + expected_type, + context=context, + expected_qparam_count=1, + ) + return result + + +def verify_quantized_circle( + path: str | Path, + bit_width: int, + *, + expected_resize_count: int = 2, + expected_same_padding_conv_count: int = 33, + expected_pad_count: int = 3, + expected_max_pool_count: int = 4, + expected_concat_count: int = 2, +) -> dict[str, Any]: + """Validate graph I/O, operator tensors, biases, and resize options.""" + circle_path = Path(path) + data = circle_path.read_bytes() + if len(data) < 8 or data[4:8] != b"CIR0": + raise ValueError(f"{circle_path} does not contain a Circle CIR0 identifier") + + reader = FlatBufferReader(data) + root = reader.root_table() + operator_codes = _parse_operator_codes(reader, root) + subgraphs = reader.vector_tables(root, 2) + if len(subgraphs) != 1: + raise RuntimeError(f"Expected one subgraph, found {len(subgraphs)}.") + subgraph = subgraphs[0] + tensors = _parse_tensors(reader, subgraph) + operators = _parse_operators(reader, subgraph, operator_codes) + inputs = tuple(reader.vector_i32(subgraph, 1)) + outputs = tuple(reader.vector_i32(subgraph, 2)) + + expected_type = _expected_tensor_type(bit_width) + expected_bias_type = _expected_bias_type(bit_width) + _require_per_tensor_data( + tensors, + inputs, + expected_type, + context="Graph input", + ) + _require_per_tensor_data( + tensors, + outputs, + expected_type, + context="Graph output", + ) + + counts = {name: 0 for name in OPERATOR_NAMES.values()} + resize_options: list[tuple[bool, bool]] = [] + conv_weight_count = 0 + depthwise_weight_count = 0 + bias_count = 0 + prelu_slope_count = 0 + same_padding_conv_count = 0 + + for operator_index, operator in enumerate(operators): + name = OPERATOR_NAMES.get( + operator.builtin_code, + f"BUILTIN_{operator.builtin_code}", + ) + if name in counts: + counts[name] += 1 + if operator.builtin_code == DEQUANTIZE: + raise RuntimeError( + f"Operator {operator_index} is DEQUANTIZE; the graph is not " + "fully integer-quantized." + ) + + if operator.builtin_code in (CONV_2D, DEPTHWISE_CONV_2D): + if len(operator.inputs) < 2 or not operator.outputs: + raise RuntimeError(f"{name} has incomplete tensor connections.") + if operator.options_table is None: + raise RuntimeError(f"{name} does not contain builtin options.") + # Padding.SAME is enum value 0 and is also the FlatBuffer field + # default. FlatBuffers may therefore omit the field entirely when + # the serialized value is SAME. Use the schema default instead of + # a sentinel so an omitted field is decoded as SAME. + padding = reader.scalar_i8(operator.options_table, 0, PADDING_SAME) + if padding == PADDING_SAME: + same_padding_conv_count += 1 + _require_per_tensor_data( + tensors, + [operator.inputs[0], operator.outputs[0]], + expected_type, + context=f"{name} activation", + ) + + weight = tensors[operator.inputs[1]] + if operator.builtin_code == CONV_2D: + expected_axis = 0 + expected_channels = weight.shape[0] + conv_weight_count += 1 + else: + expected_axis = 3 + expected_channels = weight.shape[3] + depthwise_weight_count += 1 + _require_quantized_tensor( + weight, + expected_type, + context=f"{name} weight", + expected_qparam_count=expected_channels, + expected_axis=expected_axis, + ) + + if len(operator.inputs) >= 3 and operator.inputs[2] >= 0: + bias = tensors[operator.inputs[2]] + _require_quantized_tensor( + bias, + expected_bias_type, + context=f"{name} bias", + expected_qparam_count=expected_channels, + expected_axis=0, + ) + bias_count += 1 + continue + + if operator.builtin_code == PRELU: + if len(operator.inputs) != 2 or not operator.outputs: + raise RuntimeError( + "PRELU must contain input, slope, and output tensors." + ) + activation_tensors = _require_per_tensor_data( + tensors, + [operator.inputs[0], operator.outputs[0]], + expected_type, + context="PRELU activation", + ) + slope = tensors[operator.inputs[1]] + if len(slope.shape) != 1: + raise RuntimeError( + f"PRELU slope {slope.name!r} must be rank 1, but has " + f"shape {slope.shape}." + ) + expected_channels = activation_tensors[0].shape[-1] + if slope.shape[0] != expected_channels: + raise RuntimeError( + f"PRELU slope {slope.name!r} contains {slope.shape[0]} " + f"values, but the channel-last input contains " + f"{expected_channels} channels." + ) + _require_quantized_tensor( + slope, + expected_type, + context="PRELU slope", + expected_qparam_count=expected_channels, + expected_axis=0, + ) + prelu_slope_count += 1 + continue + + if operator.builtin_code == ADD: + _require_per_tensor_data( + tensors, + [*operator.inputs, *operator.outputs], + expected_type, + context="ADD tensor", + ) + continue + + if operator.builtin_code == CONCATENATION: + connected = _require_per_tensor_data( + tensors, + [*operator.inputs, *operator.outputs], + expected_type, + context="CONCATENATION tensor", + ) + _require_same_qparams(connected, context="CONCATENATION") + continue + + if operator.builtin_code in _DATA_OPERATOR_REQUIRES_SHARED_QPARAMS: + if not operator.inputs or not operator.outputs: + raise RuntimeError(f"{name} has incomplete tensor connections.") + connected = _require_per_tensor_data( + tensors, + [operator.inputs[0], operator.outputs[0]], + expected_type, + context=f"{name} data tensor", + ) + if _DATA_OPERATOR_REQUIRES_SHARED_QPARAMS[operator.builtin_code]: + _require_same_qparams(connected, context=name) + + if operator.builtin_code == RESIZE_BILINEAR: + if operator.options_table is None: + raise RuntimeError( + "RESIZE_BILINEAR does not contain builtin options." + ) + # The schema keeps deprecated new_height/new_width in slots 0/1. + align_corners = reader.scalar_bool( + operator.options_table, + 2, + False, + ) + half_pixel_centers = reader.scalar_bool( + operator.options_table, + 3, + False, + ) + resize_options.append((align_corners, half_pixel_centers)) + continue + + if operator.builtin_code == QUANTIZE: + if not operator.outputs: + raise RuntimeError("QUANTIZE has no output tensor.") + _require_per_tensor_data( + tensors, + operator.outputs, + expected_type, + context="QUANTIZE output", + ) + + if counts["CONV_2D"] == 0: + raise RuntimeError("The Circle graph does not contain CONV_2D.") + if counts["DEPTHWISE_CONV_2D"] == 0: + raise RuntimeError("The Circle graph does not contain DEPTHWISE_CONV_2D.") + if counts["PRELU"] == 0: + raise RuntimeError("The Circle graph does not contain PRELU.") + if same_padding_conv_count != expected_same_padding_conv_count: + raise RuntimeError( + f"Expected {expected_same_padding_conv_count} SAME-padded convolution " + f"operators, found {same_padding_conv_count}." + ) + if counts["PAD"] != expected_pad_count: + raise RuntimeError( + f"Expected {expected_pad_count} explicit PAD operators, " + f"found {counts['PAD']}." + ) + if counts["MAX_POOL_2D"] != expected_max_pool_count: + raise RuntimeError( + f"Expected {expected_max_pool_count} MAX_POOL_2D operators, " + f"found {counts['MAX_POOL_2D']}." + ) + if counts["CONCATENATION"] != expected_concat_count: + raise RuntimeError( + f"Expected {expected_concat_count} CONCATENATION operators, " + f"found {counts['CONCATENATION']}." + ) + if len(resize_options) != expected_resize_count: + raise RuntimeError( + f"Expected {expected_resize_count} RESIZE_BILINEAR operators, " + f"found {len(resize_options)}." + ) + expected_options = [(False, True)] * expected_resize_count + if resize_options != expected_options: + raise RuntimeError( + f"Expected ResizeBilinear options {expected_options}, " + f"found {resize_options}." + ) + + quantized_tensor_count = sum( + tensor.tensor_type == expected_type and tensor.quantization is not None + for tensor in tensors + ) + return { + "path": str(circle_path), + "size_bytes": len(data), + "bit_width": bit_width, + "tensor_type": _type_name(expected_type), + "graph_inputs": len(inputs), + "graph_outputs": len(outputs), + "quantized_tensors": quantized_tensor_count, + "conv_weights": conv_weight_count, + "depthwise_weights": depthwise_weight_count, + "quantized_biases": bias_count, + "prelu_slopes": prelu_slope_count, + "same_padding_convolutions": same_padding_conv_count, + "operator_counts": counts, + "resize_options": [list(value) for value in resize_options], + "input_tensors": [asdict(tensors[index]) for index in inputs], + "output_tensors": [asdict(tensors[index]) for index in outputs], + } + + +def parse_args() -> argparse.Namespace: + """Parse command-line arguments.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("circle", type=Path) + parser.add_argument("--bits", type=int, required=True, choices=[8, 16]) + parser.add_argument("--expected-resize-count", type=int, default=2) + parser.add_argument("--expected-same-padding-conv-count", type=int, default=33) + parser.add_argument("--expected-pad-count", type=int, default=3) + parser.add_argument("--expected-max-pool-count", type=int, default=4) + parser.add_argument("--expected-concat-count", type=int, default=2) + return parser.parse_args() + + +def main() -> None: + """Validate one quantized Circle model and print its summary.""" + args = parse_args() + summary = verify_quantized_circle( + args.circle, + args.bits, + expected_resize_count=args.expected_resize_count, + expected_same_padding_conv_count=args.expected_same_padding_conv_count, + expected_pad_count=args.expected_pad_count, + expected_max_pool_count=args.expected_max_pool_count, + expected_concat_count=args.expected_concat_count, + ) + print( + f"Verified {summary['tensor_type']} Circle model with " + f"{summary['quantized_tensors']} quantized tensors and " + f"{summary['operator_counts']['RESIZE_BILINEAR']} " + f"RESIZE_BILINEAR operators: {args.circle}" + ) + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/analyze.py b/examples/hand_detector/analyze.py new file mode 100644 index 00000000..7a49420f --- /dev/null +++ b/examples/hand_detector/analyze.py @@ -0,0 +1,503 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Run numerical quantization analyses for the MediaPipe palm detector.""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path +from typing import Any + +from examples.hand_detector._support.analysis import ( + output_boundaries, + OUTPUT_NAMES, + summarize_percentile_observers, +) +from examples.hand_detector._support.data import ( + list_npy_inputs, + load_npy_inputs, + make_synthetic_inputs, +) +from examples.hand_detector._support.quantization import ( + quantization_name, + quantize_candidate, +) +from examples.hand_detector.hand_detector import load_nhwc_hand_detector +from tico.quantization.analysis import ( + AffineQuantizationPolicy, + build_clipping_candidates, + collect_output_calibration_data, + evaluate_clipping_candidates, + evaluate_models, + make_output_adapter, + QuantizationAblation, + QuantizationProfile, +) +from tico.quantization.wrapq.observers.minmax import MinMaxObserver +from tico.quantization.wrapq.observers.percentile import PercentileObserver + + +DIRECTORY = Path(__file__).resolve().parent +DEFAULT_PERCENTILES = (99.0, 99.5, 99.9, 99.95, 99.99, 99.995, 99.999) +DEFAULT_TAIL_PERCENTAGES = (0.0, 0.001, 0.005, 0.01, 0.05, 0.1, 0.5) +OUTPUT_ADAPTER = make_output_adapter(OUTPUT_NAMES) + + +def parse_args() -> argparse.Namespace: + """Parse the analysis subcommand and its arguments.""" + parser = argparse.ArgumentParser(description=__doc__) + subparsers = parser.add_subparsers(dest="command", required=True) + + ablation = subparsers.add_parser( + "ablation", + help="Run output-only, weight-only, activation-only, and full PTQ.", + ) + _add_model_arguments(ablation) + _add_dataset_arguments(ablation, evaluation=True) + ablation.add_argument("--bits", type=int, default=8, choices=[8, 16]) + ablation.add_argument( + "--report-json", + type=Path, + default=DIRECTORY / "reports" / "ablation.json", + ) + + clipping = subparsers.add_parser( + "output-clipping", + help="Compare MinMax, percentile, and L1 output clipping ranges.", + ) + _add_model_arguments(clipping) + _add_dataset_arguments(clipping, evaluation=True) + clipping.add_argument("--bits", type=int, default=8, choices=[8, 16]) + clipping.add_argument( + "--percentiles", + type=float, + nargs="+", + default=list(DEFAULT_PERCENTILES), + ) + clipping.add_argument( + "--tail-percentages", + type=float, + nargs="+", + default=list(DEFAULT_TAIL_PERCENTAGES), + ) + clipping.add_argument("--skip-l1-search", action="store_true") + clipping.add_argument("--max-values-per-output", type=int, default=1_000_000) + clipping.add_argument("--sampling-seed", type=int, default=20260803) + clipping.add_argument( + "--report-json", + type=Path, + default=DIRECTORY / "reports" / "output_clipping.json", + ) + + observer_sweep = subparsers.add_parser( + "observer-sweep", + help="Compare MinMax and percentile activation observers in full PTQ.", + ) + _add_model_arguments(observer_sweep) + _add_dataset_arguments(observer_sweep, evaluation=True) + observer_sweep.add_argument("--bits", type=int, default=8, choices=[8, 16]) + observer_sweep.add_argument( + "--percentiles", + type=float, + nargs="+", + default=list(DEFAULT_PERCENTILES), + ) + observer_sweep.add_argument("--max-samples", type=int, default=131_072) + observer_sweep.add_argument("--samples-per-batch", type=int, default=4_096) + observer_sweep.add_argument("--sampling-seed", type=int, default=20260803) + observer_sweep.add_argument( + "--report-json", + type=Path, + default=DIRECTORY / "reports" / "observer_sweep.json", + ) + return parser.parse_args() + + +def _add_model_arguments(parser: argparse.ArgumentParser) -> None: + parser.add_argument( + "--weights", + type=Path, + default=DIRECTORY / "hand_detector_float.pt", + ) + parser.add_argument( + "--spec", + type=Path, + default=DIRECTORY / "hand_detector_spec.json", + ) + + +def _add_dataset_arguments( + parser: argparse.ArgumentParser, + *, + evaluation: bool, +) -> None: + parser.add_argument("--calibration-dir", type=Path) + parser.add_argument("--calibration-offset", type=int, default=0) + parser.add_argument("--calibration-limit", type=int) + parser.add_argument("--synthetic-calibration-samples", type=int, default=32) + if evaluation: + parser.add_argument("--evaluation-dir", type=Path) + parser.add_argument("--evaluation-offset", type=int, default=0) + parser.add_argument("--evaluation-limit", type=int) + parser.add_argument("--synthetic-evaluation-samples", type=int, default=8) + parser.add_argument( + "--require-disjoint", + action="store_true", + help="Reject overlapping calibration and evaluation file selections.", + ) + + +def main() -> None: + """Dispatch the selected quantization analysis.""" + args = parse_args() + if args.command == "ablation": + _run_ablation(args) + elif args.command == "output-clipping": + _run_output_clipping(args) + elif args.command == "observer-sweep": + _run_observer_sweep(args) + else: + raise RuntimeError(f"Unhandled analysis command: {args.command}") + + +def _run_ablation(args: argparse.Namespace) -> None: + float_model = load_nhwc_hand_detector(args.weights, args.spec).eval() + calibration, evaluation, data_metadata = _load_datasets(args) + candidate = quantize_candidate(float_model, args.bits, calibration) + runner = QuantizationAblation( + float_model, + candidate, + boundaries=output_boundaries(candidate), + output_adapter=OUTPUT_ADAPTER, + ) + report = runner.run( + evaluation, + metadata={ + **data_metadata, + "dtype": quantization_name(args.bits), + }, + ) + _print_ablation(report.to_dict()) + output = report.write_json(args.report_json) + print(f"\nWrote {output}") + + +def _run_output_clipping(args: argparse.Namespace) -> None: + float_model = load_nhwc_hand_detector(args.weights, args.spec).eval() + calibration, evaluation, data_metadata = _load_datasets(args) + policy = ( + AffineQuantizationPolicy.uint8() + if args.bits == 8 + else AffineQuantizationPolicy.int16() + ) + calibration_data = collect_output_calibration_data( + float_model, + calibration, + output_adapter=OUTPUT_ADAPTER, + max_values_per_output=args.max_values_per_output, + seed=args.sampling_seed, + ) + candidates = { + data.name: build_clipping_candidates( + data, + policy, + percentiles=args.percentiles, + tail_percentages=args.tail_percentages, + include_l1_search=not args.skip_l1_search, + ) + for data in calibration_data + } + evaluated = evaluate_clipping_candidates( + float_model, + evaluation, + calibration_data, + candidates, + policy, + output_adapter=OUTPUT_ADAPTER, + ) + _print_output_clipping(calibration_data, evaluated, policy.name) + report = { + "analysis": "output_clipping", + "metadata": { + **data_metadata, + "dtype": policy.name, + "max_values_per_output": args.max_values_per_output, + "sampling_seed": args.sampling_seed, + }, + "calibration_outputs": [ + { + "name": data.name, + "observed_minimum": data.observed_minimum, + "observed_maximum": data.observed_maximum, + "total_value_count": data.total_value_count, + "sampled_value_count": data.sampled_value_count, + } + for data in calibration_data + ], + "outputs": { + name: [candidate.to_dict() for candidate in output_candidates] + for name, output_candidates in evaluated.items() + }, + } + _write_json(args.report_json, report) + + +def _run_observer_sweep(args: argparse.Namespace) -> None: + _validate_percentiles(args.percentiles) + float_model = load_nhwc_hand_detector(args.weights, args.spec).eval() + calibration, evaluation, data_metadata = _load_datasets(args) + results: dict[str, dict[str, Any]] = {} + + minmax = quantize_candidate( + float_model, + args.bits, + calibration, + activation_observer=MinMaxObserver, + ) + results["minmax"] = { + "observer": "MinMaxObserver", + "outputs": evaluate_models( + float_model, + minmax, + evaluation, + output_adapter=OUTPUT_ADAPTER, + ), + "observer_details": [], + } + + for percentile in args.percentiles: + name = f"percentile_{percentile:g}".replace(".", "_") + candidate = quantize_candidate( + float_model, + args.bits, + calibration, + activation_observer=PercentileObserver, + activation_observer_kwargs={ + "percentile": percentile, + "max_samples": args.max_samples, + "samples_per_batch": args.samples_per_batch, + "seed": args.sampling_seed, + }, + ) + results[name] = { + "observer": "PercentileObserver", + "percentile": percentile, + "outputs": evaluate_models( + float_model, + candidate, + evaluation, + output_adapter=OUTPUT_ADAPTER, + ), + "observer_details": summarize_percentile_observers(candidate), + } + + _print_observer_sweep(results, quantization_name(args.bits)) + _write_json( + args.report_json, + { + "analysis": "activation_observer_sweep", + "metadata": { + **data_metadata, + "dtype": quantization_name(args.bits), + "percentiles": args.percentiles, + "max_samples": args.max_samples, + "samples_per_batch": args.samples_per_batch, + }, + "results": results, + }, + ) + + +def _load_datasets( + args: argparse.Namespace, +) -> tuple[list[Any], list[Any], dict[str, Any]]: + if args.calibration_dir is None: + calibration = make_synthetic_inputs( + args.synthetic_calibration_samples, + seed=20260806, + ) + calibration_paths: set[Path] = set() + print("Using synthetic calibration inputs for a smoke test only.") + else: + calibration = load_npy_inputs( + args.calibration_dir, + args.calibration_limit, + offset=args.calibration_offset, + ) + calibration_paths = _selected_paths( + args.calibration_dir, + args.calibration_offset, + args.calibration_limit, + ) + + if args.evaluation_dir is None: + evaluation = make_synthetic_inputs( + args.synthetic_evaluation_samples, + seed=20260807, + ) + evaluation_paths: set[Path] = set() + print("Using synthetic evaluation inputs for a smoke test only.") + else: + evaluation = load_npy_inputs( + args.evaluation_dir, + args.evaluation_limit, + offset=args.evaluation_offset, + ) + evaluation_paths = _selected_paths( + args.evaluation_dir, + args.evaluation_offset, + args.evaluation_limit, + ) + + overlap = calibration_paths & evaluation_paths + if overlap: + message = ( + f"Calibration and evaluation selections overlap by {len(overlap)} files. " + "This is acceptable for numerical floor analysis, but not for selecting " + "or reporting a final quantization policy." + ) + if args.require_disjoint: + raise ValueError(message) + print(f"WARNING: {message}") + + return ( + calibration, + evaluation, + { + "calibration_samples": len(calibration), + "evaluation_samples": len(evaluation), + "synthetic_calibration": args.calibration_dir is None, + "synthetic_evaluation": args.evaluation_dir is None, + "overlapping_files": len(overlap), + }, + ) + + +def _selected_paths( + directory: Path, + offset: int, + limit: int | None, +) -> set[Path]: + paths = list_npy_inputs(directory) + selected = paths[offset : None if limit is None else offset + limit] + return {path.resolve() for path in selected} + + +def _validate_percentiles(percentiles: list[float]) -> None: + if not percentiles or any(not 0.0 < value <= 100.0 for value in percentiles): + raise ValueError("Percentiles must be in the interval (0, 100].") + + +def _print_ablation(report: dict[str, Any]) -> None: + print("\nQuantization A/B/C/D ablation") + print( + f"{'profile':18s} {'REG_MAE':>13s} {'REG_COS':>13s} " + f"{'CLS_MAE':>13s} {'CLS_COS':>13s} {'SITES':>7s}" + ) + parity = report["float_parity"] + _print_output_row("float-parity", parity, 0) + for profile in ( + QuantizationProfile.OUTPUT_ONLY, + QuantizationProfile.WEIGHT_ONLY, + QuantizationProfile.ACTIVATION_ONLY, + QuantizationProfile.FULL, + ): + result = report["profiles"][profile.value] + _print_output_row( + f"{profile.value}:{result['label']}", + result["outputs"], + result["enabled_site_count"], + ) + + +def _print_output_row(label: str, outputs: dict[str, Any], sites: int) -> None: + regressors = outputs["regressors"] + classifiers = outputs["classifiers"] + print( + f"{label:18s} " + f"{float(regressors['mae']):13.6e} " + f"{float(regressors['cosine_similarity']):13.9f} " + f"{float(classifiers['mae']):13.6e} " + f"{float(classifiers['cosine_similarity']):13.9f} " + f"{sites:7d}" + ) + + +def _print_output_clipping(calibration_data, evaluated, dtype_name: str) -> None: + print(f"\n{dtype_name.upper()} final-output clipping analysis") + print("All internal model computation remains floating point.") + data_by_name = {data.name: data for data in calibration_data} + for name, candidates in evaluated.items(): + data = data_by_name[name] + print( + f"\n{name}: sampled {data.sampled_value_count:,} / " + f"{data.total_value_count:,} calibration values; raw range " + f"[{data.observed_minimum:.6e}, {data.observed_maximum:.6e}]" + ) + print( + f"{'candidate':18s} {'CLIP_MIN':>13s} {'CLIP_MAX':>13s} " + f"{'SCALE':>11s} {'CAL_MAE':>11s} {'EVAL_MAE':>11s} {'SAT(%)':>10s}" + ) + ranked = sorted( + candidates, + key=lambda item: float(item.evaluation_error["mae"]), + ) + for index, item in enumerate(ranked): + marker = "*" if index == 0 else " " + print( + f"{marker}{item.candidate.name:17s} " + f"{item.candidate.minimum:13.4e} " + f"{item.candidate.maximum:13.4e} " + f"{float(item.quantizer['scale']):11.4e} " + f"{float(item.candidate.calibration_error['mae']):11.4e} " + f"{float(item.evaluation_error['mae']):11.4e} " + f"{100.0 * float(item.quantizer['saturation_ratio']):10.5f}" + ) + + +def _print_observer_sweep(results: dict[str, dict[str, Any]], dtype_name: str) -> None: + print(f"\n{dtype_name.upper()} activation observer sweep") + print( + f"{'observer':24s} {'REG_MAE':>13s} {'REG_COS':>13s} " + f"{'CLS_MAE':>13s} {'CLS_COS':>13s}" + ) + ranked = sorted( + results.items(), + key=lambda item: float(item[1]["outputs"]["regressors"]["mae"]), + ) + for index, (name, result) in enumerate(ranked): + marker = "*" if index == 0 else " " + outputs = result["outputs"] + print( + f"{marker}{name:23s} " + f"{float(outputs['regressors']['mae']):13.6e} " + f"{float(outputs['regressors']['cosine_similarity']):13.9f} " + f"{float(outputs['classifiers']['mae']):13.6e} " + f"{float(outputs['classifiers']['cosine_similarity']):13.9f}" + ) + + +def _write_json(path: Path, value: dict[str, Any]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text( + json.dumps(value, indent=2, allow_nan=False), + encoding="utf-8", + ) + print(f"\nWrote {path}") + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/convert.py b/examples/hand_detector/convert.py new file mode 100644 index 00000000..5c54a660 --- /dev/null +++ b/examples/hand_detector/convert.py @@ -0,0 +1,72 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Convert the MediaPipe palm-detector TFLite model into PyTorch artifacts.""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +import torch + +from examples.hand_detector._support.conversion import ( + build_specification, + load_parameters, +) +from examples.hand_detector._support.tflite_flatbuffer import TFLiteModel +from examples.hand_detector.hand_detector import HandDetector + + +DIRECTORY = Path(__file__).resolve().parent + + +def parse_args() -> argparse.Namespace: + """Parse source and destination artifact paths.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("tflite", type=Path) + parser.add_argument( + "--spec", + type=Path, + default=DIRECTORY / "hand_detector_spec.json", + ) + parser.add_argument( + "--weights", + type=Path, + default=DIRECTORY / "hand_detector_float.pt", + ) + return parser.parse_args() + + +def main() -> None: + """Convert the source graph and save a specification and state dictionary.""" + args = parse_args() + source = TFLiteModel(args.tflite) + specification, constants = build_specification(source) + model = HandDetector(specification) + load_parameters(model, specification, constants) + + args.spec.parent.mkdir(parents=True, exist_ok=True) + args.weights.parent.mkdir(parents=True, exist_ok=True) + args.spec.write_text(json.dumps(specification, indent=2), encoding="utf-8") + torch.save(model.state_dict(), args.weights) + parameter_count = sum(parameter.numel() for parameter in model.parameters()) + print(f"Wrote {args.spec}") + print(f"Wrote {args.weights}") + print(f"Parameters: {parameter_count:,}") + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/docs/layout_optimization.md b/examples/hand_detector/docs/layout_optimization.md new file mode 100644 index 00000000..be989144 --- /dev/null +++ b/examples/hand_detector/docs/layout_optimization.md @@ -0,0 +1,147 @@ +# NHWC input and Circle-side layout-region optimization + +The PyTorch detector remains an ordinary NCHW implementation. Circle export uses +`NHWCInputAdapter` to expose an NHWC image input with shape +`[1, 192, 192, 3]`. + +The adapter places the input `QuantStub` before the NHWC-to-NCHW permutation. +Consequently, UINT8 and INT16 exports attach the input affine qparam directly to +the NHWC Circle input tensor. + +## Circle optimization pipeline + +After serialization, the example runs this Circle-to-Circle pipeline: + +```text +EliminateTransposeBoundedLayoutRegionPass +RemoveRedundantLayoutOpsPass +DeadCodeEliminationPass +CompactIndicesPass +``` + +The pass manager uses `CirclePassStrategy.RESTART`. A rewrite can expose a new +inverse Transpose pair or dead object, and restart scheduling allows the earlier +passes to run again until the complete sequence reaches a fixed point. + +`EliminateTransposeBoundedLayoutRegionPass` finds connected Circle regions made +only of layout-convertible operators: + +```text +ADD +PAD with a constant rank-by-two padding tensor +``` + +Every external data input to a candidate region must enter through the same +Transpose permutation. Every external data output must leave through the +inverse permutation. The pass then executes the complete region directly in the +source layout: + +```text +source-layout tensors + -> Transpose(P) + -> ADD/PAD region + -> Transpose(P^-1) + -> source-layout tensors +``` + +becomes: + +```text +source-layout tensors + -> ADD/PAD region + -> source-layout tensors +``` + +For PAD, the rank-by-two constant is cloned and its rows are reordered to match +the source-layout axes. Cloning avoids changing another operator that may share +the original constant buffer. + +The detector contains five such regions: + +- three downsample residual regions containing PAD and ADD; +- two decoder regions containing two connected ADD operators and an NHWC side + path between them. + +Together, the regions bypass all 19 Circle Transpose operators. Dead-code +elimination removes the unused Transpose nodes, and index compaction removes the +unused permutation and padding constants. + +## Safety conditions + +A region is rewritten only when all of the following conditions hold: + +- every internal operator is supported by the region pass; +- binary ADD inputs and outputs have identical shapes, so broadcasting is not + involved; +- PAD uses a constant INT32 tensor with shape `[rank, 2]`; +- every input boundary uses one common permutation; +- every output boundary uses the inverse permutation; +- no region-layout tensor is exposed directly as a graph or signature output; +- every unsupported external consumer is separated by the expected inverse + Transpose; +- every boundary Transpose preserves tensor type and affine qparams; +- quantized activations have exactly one scale and one zero point; +- every region data tensor is unquantized or uses per-tensor activation qparams. + +Per-channel activation qparams are deliberately rejected. The target activation +policy is per-tensor, so the rewrite never has to remap a quantized axis. +Conv2D and DepthwiseConv2D weight qparams remain per-channel and are unrelated +to this activation-layout transformation. + +## Export + +Floating-point export: + +```bash +python examples/hand_detector/export_float_circle.py \ + --output examples/hand_detector/exported/hand_detector_float.circle +``` + +Quantized export: + +```bash +python examples/hand_detector/export_quantized_circle.py \ + --calibration-dir /path/to/calibration_npy \ + --bits 8 16 \ + --output-dir examples/hand_detector/exported +``` + +Calibration and evaluation arrays are normalized to NHWC by `input_data.py`. +The accepted source shapes remain: + +```text +[192, 192, 3] +[1, 192, 192, 3] +[3, 192, 192] +[1, 3, 192, 192] +``` + +## Verify the exported graph + +```bash +python examples/hand_detector/verify_circle_layout.py \ + examples/hand_detector/exported/hand_detector_float.circle +``` + +The verifier checks: + +- one Circle input with shape `[1, 192, 192, 3]`; +- no consecutive inverse Transpose pair; +- no remaining Transpose-ADD-Transpose round trip; +- exactly zero Circle Transpose operators by default. + +## Use the Circle pass from the CLI + +The pass is opt-in and does not change the default `tico.convert` pipeline: + +```bash +tico-circle optimize input.circle \ + --passes \ +eliminate-transpose-bounded-layout-region,remove-redundant-layout-ops,dce,compact \ + --strategy restart \ + -o output.circle +``` + +Changing the external input ABI remains an explicit model-authoring decision via +`NHWCInputAdapter`. The Circle pass performs only semantics-preserving internal +graph rewrites. diff --git a/examples/hand_detector/export.py b/examples/hand_detector/export.py new file mode 100644 index 00000000..d86d6617 --- /dev/null +++ b/examples/hand_detector/export.py @@ -0,0 +1,195 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Export floating-point or calibrated quantized palm-detector Circle models.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +from pathlib import Path +from typing import Any + +import tico +import torch + +from examples.hand_detector._support.circle import save_layout_optimized_circle +from examples.hand_detector._support.data import load_npy_inputs, make_synthetic_inputs +from examples.hand_detector._support.quantization import ( + export_quantized_circle, + quantization_label, + quantization_name, + quantize_candidate, +) +from examples.hand_detector._support.verify_circle_layout import verify_circle_layout +from examples.hand_detector._support.verify_circle_resize import ( + read_resize_bilinear_options, +) +from examples.hand_detector._support.verify_quantized_circle import ( + verify_quantized_circle, +) +from examples.hand_detector.hand_detector import load_nhwc_hand_detector + + +DIRECTORY = Path(__file__).resolve().parent + + +def parse_args() -> argparse.Namespace: + """Parse the export mode and its options.""" + parser = argparse.ArgumentParser(description=__doc__) + subparsers = parser.add_subparsers(dest="mode", required=True) + + float_parser = subparsers.add_parser("float", help="Export the FP32 model.") + _add_model_arguments(float_parser) + float_parser.add_argument( + "--output", + type=Path, + default=DIRECTORY / "hand_detector_float.circle", + ) + float_parser.add_argument("--skip-verification", action="store_true") + + quantized = subparsers.add_parser( + "quantized", + help="Calibrate and export UINT8 and/or INT16 models.", + ) + _add_model_arguments(quantized) + quantized.add_argument("--calibration-dir", type=Path) + quantized.add_argument("--calibration-offset", type=int, default=0) + quantized.add_argument("--calibration-limit", type=int) + quantized.add_argument("--synthetic-calibration-samples", type=int, default=32) + quantized.add_argument( + "--bits", + type=int, + nargs="+", + default=[8, 16], + choices=[8, 16], + ) + quantized.add_argument("--output-dir", type=Path, default=DIRECTORY / "exported") + quantized.add_argument("--output-prefix", default="hand_detector") + quantized.add_argument( + "--manifest-json", + type=Path, + default=DIRECTORY / "exported" / "manifest.json", + ) + quantized.add_argument("--skip-verification", action="store_true") + return parser.parse_args() + + +def _add_model_arguments(parser: argparse.ArgumentParser) -> None: + parser.add_argument( + "--weights", + type=Path, + default=DIRECTORY / "hand_detector_float.pt", + ) + parser.add_argument( + "--spec", + type=Path, + default=DIRECTORY / "hand_detector_spec.json", + ) + + +def main() -> None: + """Dispatch floating-point or quantized export.""" + args = parse_args() + if args.mode == "float": + _export_float(args) + elif args.mode == "quantized": + _export_quantized(args) + else: + raise RuntimeError(f"Unhandled export mode: {args.mode}") + + +def _export_float(args: argparse.Namespace) -> None: + model = load_nhwc_hand_detector(args.weights, args.spec).eval() + with torch.inference_mode(): + circle_model = tico.convert(model, model.get_example_inputs()) + output, result = save_layout_optimized_circle(circle_model, args.output) + print(f"Wrote {output}") + print(f"Circle layout optimization reported {result.changes} changes.") + if args.skip_verification: + return + layout = verify_circle_layout(output) + resize_options = read_resize_bilinear_options(output) + expected = [(False, True), (False, True)] + if resize_options != expected: + raise RuntimeError( + f"Expected ResizeBilinear options {expected}, found {resize_options}." + ) + print(f"Verified Circle input shapes: {layout['input_shapes']}") + print(f"Remaining Circle Transpose operators: {layout['transpose_count']}") + print("Verified 2 Circle RESIZE_BILINEAR operators.") + + +def _export_quantized(args: argparse.Namespace) -> None: + model = load_nhwc_hand_detector(args.weights, args.spec).eval() + if args.calibration_dir is None: + calibration = make_synthetic_inputs( + args.synthetic_calibration_samples, + seed=20260728, + ) + print("Using synthetic calibration inputs for a smoke test only.") + else: + calibration = load_npy_inputs( + args.calibration_dir, + args.calibration_limit, + offset=args.calibration_offset, + ) + + args.output_dir.mkdir(parents=True, exist_ok=True) + models: dict[str, dict[str, Any]] = {} + for bit_width in args.bits: + print(f"Preparing {quantization_label(bit_width)} model...") + candidate = quantize_candidate(model, bit_width, calibration) + dtype_name = quantization_name(bit_width) + output = args.output_dir / f"{args.output_prefix}_{dtype_name}.circle" + export_quantized_circle(candidate, output) + if args.skip_verification: + summary: dict[str, Any] = { + "path": str(output), + "size_bytes": output.stat().st_size, + "verification_skipped": True, + } + else: + summary = verify_quantized_circle(output, bit_width) + summary["layout"] = verify_circle_layout(output) + summary["sha256"] = _sha256(output) + models[dtype_name] = summary + print(f"Wrote {output} ({output.stat().st_size} bytes).") + + manifest = { + "input_layout": "NHWC", + "input_shape": [1, 192, 192, 3], + "calibration_samples": len(calibration), + "synthetic_calibration": args.calibration_dir is None, + "models": models, + } + args.manifest_json.parent.mkdir(parents=True, exist_ok=True) + args.manifest_json.write_text( + json.dumps(manifest, indent=2, allow_nan=False), + encoding="utf-8", + ) + print(f"Wrote {args.manifest_json}") + + +def _sha256(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as stream: + for chunk in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +if __name__ == "__main__": + main() diff --git a/examples/hand_detector/hand_detector.py b/examples/hand_detector/hand_detector.py new file mode 100644 index 00000000..a72f5d6b --- /dev/null +++ b/examples/hand_detector/hand_detector.py @@ -0,0 +1,205 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Static PyTorch reconstruction of the MediaPipe palm detector.""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +import torch +import torch.nn.functional as F + +from tico.ops import Concat, ResizeBilinear2d, SamePaddingConv2d +from tico.quantization import QuantStub +from torch import nn + + +class ConvNode(nn.Module): + """Apply one regular, depthwise, VALID, or SAME Conv2d operation.""" + + def __init__(self, config: dict[str, Any]) -> None: + """Create a Conv2d node from the static converted configuration.""" + super().__init__() + padding = config.get("padding") + if padding is None: + padding = "same" if any(config.get("pad", ())) else "valid" + if padding not in {"same", "valid"}: + raise ValueError(f"Unsupported convolution padding: {padding!r}") + + conv_type = SamePaddingConv2d if padding == "same" else nn.Conv2d + self.conv = conv_type( + in_channels=int(config["in_channels"]), + out_channels=int(config["out_channels"]), + kernel_size=tuple(config["kernel_size"]), + stride=tuple(config["stride"]), + padding=0, + dilation=tuple(config["dilation"]), + groups=int(config["groups"]), + bias=bool(config["has_bias"]), + ) + + def forward(self, input_: torch.Tensor) -> torch.Tensor: + """Run the configured convolution.""" + return self.conv(input_) + + +class ChannelPadNode(nn.Module): + """Apply constant zero padding after converting NHWC padding to NCHW order.""" + + def __init__(self, pad: list[int]) -> None: + """Store constant NCHW padding values.""" + super().__init__() + self.pad = tuple(int(value) for value in pad) + + def forward(self, input_: torch.Tensor) -> torch.Tensor: + """Pad an NCHW tensor.""" + return F.pad(input_, self.pad) + + +class HandDetector(nn.Module): + """Execute the converted static graph with NCHW input tensors.""" + + def __init__(self, specification: dict[str, Any]) -> None: + """Construct modules for every operation in the static specification.""" + super().__init__() + self.specification = specification + self.input_tensor = int(specification["inputs"][0]) + self.output_tensors = tuple(int(value) for value in specification["outputs"]) + self.operations = tuple(specification["operations"]) + self.input_quantizer = QuantStub() + layers: list[nn.Module] = [] + for operation in self.operations: + name = operation["name"] + config = operation["config"] + if name in {"CONV_2D", "DEPTHWISE_CONV_2D"}: + layers.append(ConvNode(config)) + elif name == "PRELU": + layers.append(nn.PReLU(int(config["channels"]))) + elif name == "MAX_POOL_2D": + layers.append( + nn.MaxPool2d( + kernel_size=tuple(config["kernel_size"]), + stride=tuple(config["stride"]), + ) + ) + elif name == "PAD": + layers.append(ChannelPadNode(config["pad"])) + elif name == "RESIZE_BILINEAR": + layers.append( + ResizeBilinear2d( + tuple(config["size"]), + align_corners=bool(config["align_corners"]), + half_pixel_centers=bool(config["half_pixel_centers"]), + ) + ) + elif name == "CONCATENATION": + layers.append(Concat(dim=int(config["axis"]))) + else: + layers.append(nn.Identity()) + self.layers = nn.ModuleList(layers) + + def _forward_core( + self, + input_: torch.Tensor, + ) -> tuple[torch.Tensor, torch.Tensor]: + """Run the static detector graph from an already quantized NCHW tensor.""" + values: dict[int, torch.Tensor] = {self.input_tensor: input_} + for operation, layer in zip(self.operations, self.layers): + name = operation["name"] + inputs = operation["inputs"] + output = int(operation["outputs"][0]) + config = operation["config"] + if name in { + "CONV_2D", + "DEPTHWISE_CONV_2D", + "PRELU", + "MAX_POOL_2D", + "PAD", + "RESIZE_BILINEAR", + }: + values[output] = layer(values[int(inputs[0])]) + elif name == "ADD": + values[output] = values[int(inputs[0])] + values[int(inputs[1])] + elif name == "RESHAPE": + source = values[int(inputs[0])] + if bool(config["nhwc_memory_order"]): + source = source.permute(0, 2, 3, 1) + values[output] = source.reshape(tuple(config["shape"])) + elif name == "CONCATENATION": + values[output] = layer(tuple(values[int(index)] for index in inputs)) + else: + raise RuntimeError(f"Unsupported converted operation: {name}") + return values[self.output_tensors[0]], values[self.output_tensors[1]] + + def forward(self, input_: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]: + """Run the detector from an NCHW input tensor.""" + return self._forward_core(self.input_quantizer(input_)) + + def forward_nhwc(self, input_: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]: + """Quantize an NHWC input before converting it to the internal NCHW layout.""" + quantized = self.input_quantizer(input_) + return self._forward_core(quantized.permute(0, 3, 1, 2)) + + def get_example_inputs(self) -> tuple[torch.Tensor]: + """Return the static NCHW example input used by direct model export.""" + return (torch.zeros(1, 3, 192, 192, dtype=torch.float32),) + + +class NHWCInputAdapter(nn.Module): + """Expose an NHWC input ABI while preserving the NCHW detector implementation.""" + + def __init__(self, detector: HandDetector) -> None: + """Store the detector whose input boundary should be exported as NHWC.""" + super().__init__() + self.detector = detector + + def forward(self, input_: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]: + """Run the wrapped detector from one NHWC input tensor.""" + return self.detector.forward_nhwc(input_) + + def get_example_inputs(self) -> tuple[torch.Tensor]: + """Return the static NHWC example input used by Circle export.""" + return (torch.zeros(1, 192, 192, 3, dtype=torch.float32),) + + +def load_hand_detector( + weights: str | Path, + specification: str | Path, + *, + map_location: str | torch.device = "cpu", +) -> HandDetector: + """Construct the detector and load a converted state dictionary.""" + spec = json.loads(Path(specification).read_text(encoding="utf-8")) + model = HandDetector(spec) + state = torch.load(weights, map_location=map_location, weights_only=True) + model.load_state_dict(state, strict=True) + return model + + +def load_nhwc_hand_detector( + weights: str | Path, + specification: str | Path, + *, + map_location: str | torch.device = "cpu", +) -> NHWCInputAdapter: + """Load the detector and expose an NHWC input ABI for Circle export.""" + detector = load_hand_detector( + weights, + specification, + map_location=map_location, + ) + return NHWCInputAdapter(detector) diff --git a/examples/hand_detector/hand_detector_spec.json b/examples/hand_detector/hand_detector_spec.json new file mode 100644 index 00000000..3d13e4d2 --- /dev/null +++ b/examples/hand_detector/hand_detector_spec.json @@ -0,0 +1,3424 @@ +{ + "format_version": 1, + "source": "hand_detector.tflite", + "input_layout": "NCHW", + "inputs": [ + 0 + ], + "outputs": [ + 279, + 276 + ], + "operations": [ + { + "index": 2, + "name": "CONV_2D", + "inputs": [ + 0, + 387, + 346 + ], + "outputs": [ + 141 + ], + "config": { + "in_channels": 3, + "out_channels": 32, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 2, + 2 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "same", + "pad": [ + 1, + 2, + 1, + 2 + ] + } + }, + { + "index": 4, + "name": "PRELU", + "inputs": [ + 141, + 344 + ], + "outputs": [ + 142 + ], + "config": { + "channels": 32 + } + }, + { + "index": 7, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 142, + 353, + 408 + ], + "outputs": [ + 143 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 32, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 10, + "name": "CONV_2D", + "inputs": [ + 143, + 336, + 328 + ], + "outputs": [ + 144 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 11, + "name": "ADD", + "inputs": [ + 142, + 144 + ], + "outputs": [ + 145 + ], + "config": {} + }, + { + "index": 13, + "name": "PRELU", + "inputs": [ + 145, + 310 + ], + "outputs": [ + 146 + ], + "config": { + "channels": 32 + } + }, + { + "index": 15, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 146, + 319, + 408 + ], + "outputs": [ + 147 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 32, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 18, + "name": "CONV_2D", + "inputs": [ + 147, + 297, + 291 + ], + "outputs": [ + 148 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 19, + "name": "ADD", + "inputs": [ + 146, + 148 + ], + "outputs": [ + 149 + ], + "config": {} + }, + { + "index": 21, + "name": "PRELU", + "inputs": [ + 149, + 404 + ], + "outputs": [ + 150 + ], + "config": { + "channels": 32 + } + }, + { + "index": 23, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 150, + 295, + 408 + ], + "outputs": [ + 151 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 32, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 26, + "name": "CONV_2D", + "inputs": [ + 151, + 407, + 368 + ], + "outputs": [ + 152 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 27, + "name": "ADD", + "inputs": [ + 150, + 152 + ], + "outputs": [ + 153 + ], + "config": {} + }, + { + "index": 29, + "name": "PRELU", + "inputs": [ + 153, + 351 + ], + "outputs": [ + 154 + ], + "config": { + "channels": 32 + } + }, + { + "index": 31, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 154, + 388, + 408 + ], + "outputs": [ + 155 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 32, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 34, + "name": "CONV_2D", + "inputs": [ + 155, + 369, + 334 + ], + "outputs": [ + 156 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 35, + "name": "ADD", + "inputs": [ + 154, + 156 + ], + "outputs": [ + 157 + ], + "config": {} + }, + { + "index": 37, + "name": "PRELU", + "inputs": [ + 157, + 315 + ], + "outputs": [ + 158 + ], + "config": { + "channels": 32 + } + }, + { + "index": 39, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 158, + 355, + 408 + ], + "outputs": [ + 159 + ], + "config": { + "in_channels": 32, + "out_channels": 32, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 2, + 2 + ], + "dilation": [ + 1, + 1 + ], + "groups": 32, + "has_bias": true, + "padding": "same", + "pad": [ + 1, + 2, + 1, + 2 + ] + } + }, + { + "index": 42, + "name": "CONV_2D", + "inputs": [ + 159, + 303, + 313 + ], + "outputs": [ + 160 + ], + "config": { + "in_channels": 32, + "out_channels": 64, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 43, + "name": "MAX_POOL_2D", + "inputs": [ + 158 + ], + "outputs": [ + 161 + ], + "config": { + "kernel_size": [ + 2, + 2 + ], + "stride": [ + 2, + 2 + ] + } + }, + { + "index": 44, + "name": "PAD", + "inputs": [ + 161, + 9 + ], + "outputs": [ + 162 + ], + "config": { + "pad": [ + 0, + 0, + 0, + 0, + 0, + 32 + ] + } + }, + { + "index": 45, + "name": "ADD", + "inputs": [ + 162, + 160 + ], + "outputs": [ + 163 + ], + "config": {} + }, + { + "index": 47, + "name": "PRELU", + "inputs": [ + 163, + 293 + ], + "outputs": [ + 164 + ], + "config": { + "channels": 64 + } + }, + { + "index": 50, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 164, + 284, + 324 + ], + "outputs": [ + 165 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 64, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 53, + "name": "CONV_2D", + "inputs": [ + 165, + 396, + 405 + ], + "outputs": [ + 166 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 54, + "name": "ADD", + "inputs": [ + 164, + 166 + ], + "outputs": [ + 167 + ], + "config": {} + }, + { + "index": 56, + "name": "PRELU", + "inputs": [ + 167, + 386 + ], + "outputs": [ + 168 + ], + "config": { + "channels": 64 + } + }, + { + "index": 58, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 168, + 392, + 324 + ], + "outputs": [ + 169 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 64, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 61, + "name": "CONV_2D", + "inputs": [ + 169, + 374, + 370 + ], + "outputs": [ + 170 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 62, + "name": "ADD", + "inputs": [ + 168, + 170 + ], + "outputs": [ + 171 + ], + "config": {} + }, + { + "index": 64, + "name": "PRELU", + "inputs": [ + 171, + 338 + ], + "outputs": [ + 172 + ], + "config": { + "channels": 64 + } + }, + { + "index": 66, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 172, + 357, + 324 + ], + "outputs": [ + 173 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 64, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 69, + "name": "CONV_2D", + "inputs": [ + 173, + 339, + 321 + ], + "outputs": [ + 174 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 70, + "name": "ADD", + "inputs": [ + 172, + 174 + ], + "outputs": [ + 175 + ], + "config": {} + }, + { + "index": 72, + "name": "PRELU", + "inputs": [ + 175, + 301 + ], + "outputs": [ + 176 + ], + "config": { + "channels": 64 + } + }, + { + "index": 74, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 176, + 304, + 324 + ], + "outputs": [ + 177 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 64, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 77, + "name": "CONV_2D", + "inputs": [ + 177, + 288, + 282 + ], + "outputs": [ + 178 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 78, + "name": "ADD", + "inputs": [ + 176, + 178 + ], + "outputs": [ + 179 + ], + "config": {} + }, + { + "index": 80, + "name": "PRELU", + "inputs": [ + 179, + 410 + ], + "outputs": [ + 180 + ], + "config": { + "channels": 64 + } + }, + { + "index": 82, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 180, + 400, + 324 + ], + "outputs": [ + 181 + ], + "config": { + "in_channels": 64, + "out_channels": 64, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 2, + 2 + ], + "dilation": [ + 1, + 1 + ], + "groups": 64, + "has_bias": true, + "padding": "same", + "pad": [ + 1, + 2, + 1, + 2 + ] + } + }, + { + "index": 85, + "name": "CONV_2D", + "inputs": [ + 181, + 362, + 390 + ], + "outputs": [ + 182 + ], + "config": { + "in_channels": 64, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 86, + "name": "MAX_POOL_2D", + "inputs": [ + 180 + ], + "outputs": [ + 183 + ], + "config": { + "kernel_size": [ + 2, + 2 + ], + "stride": [ + 2, + 2 + ] + } + }, + { + "index": 87, + "name": "PAD", + "inputs": [ + 183, + 8 + ], + "outputs": [ + 184 + ], + "config": { + "pad": [ + 0, + 0, + 0, + 0, + 0, + 64 + ] + } + }, + { + "index": 88, + "name": "ADD", + "inputs": [ + 184, + 182 + ], + "outputs": [ + 185 + ], + "config": {} + }, + { + "index": 90, + "name": "PRELU", + "inputs": [ + 185, + 372 + ], + "outputs": [ + 186 + ], + "config": { + "channels": 128 + } + }, + { + "index": 93, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 186, + 361, + 380 + ], + "outputs": [ + 187 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 96, + "name": "CONV_2D", + "inputs": [ + 187, + 342, + 354 + ], + "outputs": [ + 188 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 97, + "name": "ADD", + "inputs": [ + 186, + 188 + ], + "outputs": [ + 189 + ], + "config": {} + }, + { + "index": 99, + "name": "PRELU", + "inputs": [ + 189, + 323 + ], + "outputs": [ + 190 + ], + "config": { + "channels": 128 + } + }, + { + "index": 101, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 190, + 326, + 380 + ], + "outputs": [ + 191 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 104, + "name": "CONV_2D", + "inputs": [ + 191, + 307, + 305 + ], + "outputs": [ + 192 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 105, + "name": "ADD", + "inputs": [ + 190, + 192 + ], + "outputs": [ + 193 + ], + "config": {} + }, + { + "index": 107, + "name": "PRELU", + "inputs": [ + 193, + 286 + ], + "outputs": [ + 194 + ], + "config": { + "channels": 128 + } + }, + { + "index": 109, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 194, + 401, + 380 + ], + "outputs": [ + 195 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 112, + "name": "CONV_2D", + "inputs": [ + 195, + 381, + 398 + ], + "outputs": [ + 196 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 113, + "name": "ADD", + "inputs": [ + 194, + 196 + ], + "outputs": [ + 197 + ], + "config": {} + }, + { + "index": 115, + "name": "PRELU", + "inputs": [ + 197, + 379 + ], + "outputs": [ + 198 + ], + "config": { + "channels": 128 + } + }, + { + "index": 117, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 198, + 366, + 380 + ], + "outputs": [ + 199 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 120, + "name": "CONV_2D", + "inputs": [ + 199, + 349, + 377 + ], + "outputs": [ + 200 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 121, + "name": "ADD", + "inputs": [ + 198, + 200 + ], + "outputs": [ + 201 + ], + "config": {} + }, + { + "index": 123, + "name": "PRELU", + "inputs": [ + 201, + 359 + ], + "outputs": [ + 202 + ], + "config": { + "channels": 128 + } + }, + { + "index": 125, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 202, + 331, + 380 + ], + "outputs": [ + 203 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 2, + 2 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 1, + 2, + 1, + 2 + ] + } + }, + { + "index": 128, + "name": "CONV_2D", + "inputs": [ + 203, + 312, + 341 + ], + "outputs": [ + 204 + ], + "config": { + "in_channels": 128, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 129, + "name": "MAX_POOL_2D", + "inputs": [ + 202 + ], + "outputs": [ + 205 + ], + "config": { + "kernel_size": [ + 2, + 2 + ], + "stride": [ + 2, + 2 + ] + } + }, + { + "index": 130, + "name": "PAD", + "inputs": [ + 205, + 7 + ], + "outputs": [ + 206 + ], + "config": { + "pad": [ + 0, + 0, + 0, + 0, + 0, + 128 + ] + } + }, + { + "index": 131, + "name": "ADD", + "inputs": [ + 206, + 204 + ], + "outputs": [ + 207 + ], + "config": {} + }, + { + "index": 133, + "name": "PRELU", + "inputs": [ + 207, + 325 + ], + "outputs": [ + 208 + ], + "config": { + "channels": 256 + } + }, + { + "index": 136, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 208, + 292, + 330 + ], + "outputs": [ + 209 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 139, + "name": "CONV_2D", + "inputs": [ + 209, + 403, + 287 + ], + "outputs": [ + 210 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 140, + "name": "ADD", + "inputs": [ + 208, + 210 + ], + "outputs": [ + 211 + ], + "config": {} + }, + { + "index": 142, + "name": "PRELU", + "inputs": [ + 211, + 399 + ], + "outputs": [ + 212 + ], + "config": { + "channels": 256 + } + }, + { + "index": 144, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 212, + 367, + 330 + ], + "outputs": [ + 213 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 147, + "name": "CONV_2D", + "inputs": [ + 213, + 350, + 382 + ], + "outputs": [ + 214 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 148, + "name": "ADD", + "inputs": [ + 212, + 214 + ], + "outputs": [ + 215 + ], + "config": {} + }, + { + "index": 150, + "name": "PRELU", + "inputs": [ + 215, + 364 + ], + "outputs": [ + 216 + ], + "config": { + "channels": 256 + } + }, + { + "index": 152, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 216, + 332, + 330 + ], + "outputs": [ + 217 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 155, + "name": "CONV_2D", + "inputs": [ + 217, + 316, + 347 + ], + "outputs": [ + 218 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 156, + "name": "ADD", + "inputs": [ + 216, + 218 + ], + "outputs": [ + 219 + ], + "config": {} + }, + { + "index": 158, + "name": "PRELU", + "inputs": [ + 219, + 345 + ], + "outputs": [ + 220 + ], + "config": { + "channels": 256 + } + }, + { + "index": 160, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 220, + 298, + 330 + ], + "outputs": [ + 221 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 163, + "name": "CONV_2D", + "inputs": [ + 221, + 296, + 329 + ], + "outputs": [ + 222 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 164, + "name": "ADD", + "inputs": [ + 220, + 222 + ], + "outputs": [ + 223 + ], + "config": {} + }, + { + "index": 166, + "name": "PRELU", + "inputs": [ + 223, + 309 + ], + "outputs": [ + 224 + ], + "config": { + "channels": 256 + } + }, + { + "index": 168, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 224, + 409, + 330 + ], + "outputs": [ + 225 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 2, + 2 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 1, + 2, + 1, + 2 + ] + } + }, + { + "index": 171, + "name": "CONV_2D", + "inputs": [ + 225, + 389, + 289 + ], + "outputs": [ + 226 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 172, + "name": "MAX_POOL_2D", + "inputs": [ + 224 + ], + "outputs": [ + 227 + ], + "config": { + "kernel_size": [ + 2, + 2 + ], + "stride": [ + 2, + 2 + ] + } + }, + { + "index": 173, + "name": "ADD", + "inputs": [ + 227, + 226 + ], + "outputs": [ + 228 + ], + "config": {} + }, + { + "index": 175, + "name": "PRELU", + "inputs": [ + 228, + 384 + ], + "outputs": [ + 229 + ], + "config": { + "channels": 256 + } + }, + { + "index": 177, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 229, + 371, + 330 + ], + "outputs": [ + 230 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 180, + "name": "CONV_2D", + "inputs": [ + 230, + 337, + 365 + ], + "outputs": [ + 231 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 181, + "name": "ADD", + "inputs": [ + 229, + 231 + ], + "outputs": [ + 232 + ], + "config": {} + }, + { + "index": 183, + "name": "PRELU", + "inputs": [ + 232, + 348 + ], + "outputs": [ + 233 + ], + "config": { + "channels": 256 + } + }, + { + "index": 185, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 233, + 320, + 330 + ], + "outputs": [ + 234 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 188, + "name": "CONV_2D", + "inputs": [ + 234, + 300, + 333 + ], + "outputs": [ + 235 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 189, + "name": "ADD", + "inputs": [ + 233, + 235 + ], + "outputs": [ + 236 + ], + "config": {} + }, + { + "index": 191, + "name": "PRELU", + "inputs": [ + 236, + 317 + ], + "outputs": [ + 237 + ], + "config": { + "channels": 256 + } + }, + { + "index": 193, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 237, + 280, + 330 + ], + "outputs": [ + 238 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 196, + "name": "CONV_2D", + "inputs": [ + 238, + 411, + 314 + ], + "outputs": [ + 239 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 197, + "name": "ADD", + "inputs": [ + 237, + 239 + ], + "outputs": [ + 240 + ], + "config": {} + }, + { + "index": 199, + "name": "PRELU", + "inputs": [ + 240, + 294 + ], + "outputs": [ + 241 + ], + "config": { + "channels": 256 + } + }, + { + "index": 201, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 241, + 394, + 330 + ], + "outputs": [ + 242 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 204, + "name": "CONV_2D", + "inputs": [ + 242, + 375, + 406 + ], + "outputs": [ + 243 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 205, + "name": "ADD", + "inputs": [ + 241, + 243 + ], + "outputs": [ + 244 + ], + "config": {} + }, + { + "index": 207, + "name": "PRELU", + "inputs": [ + 244, + 385 + ], + "outputs": [ + 245 + ], + "config": { + "channels": 256 + } + }, + { + "index": 208, + "name": "RESIZE_BILINEAR", + "inputs": [ + 245, + 4 + ], + "outputs": [ + 246 + ], + "config": { + "size": [ + 12, + 12 + ], + "align_corners": false, + "half_pixel_centers": true + } + }, + { + "index": 211, + "name": "CONV_2D", + "inputs": [ + 246, + 358, + 311 + ], + "outputs": [ + 247 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 213, + "name": "PRELU", + "inputs": [ + 247, + 352 + ], + "outputs": [ + 248 + ], + "config": { + "channels": 256 + } + }, + { + "index": 214, + "name": "ADD", + "inputs": [ + 224, + 248 + ], + "outputs": [ + 249 + ], + "config": {} + }, + { + "index": 216, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 249, + 340, + 330 + ], + "outputs": [ + 250 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 219, + "name": "CONV_2D", + "inputs": [ + 250, + 306, + 335 + ], + "outputs": [ + 251 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 220, + "name": "ADD", + "inputs": [ + 249, + 251 + ], + "outputs": [ + 252 + ], + "config": {} + }, + { + "index": 222, + "name": "PRELU", + "inputs": [ + 252, + 318 + ], + "outputs": [ + 253 + ], + "config": { + "channels": 256 + } + }, + { + "index": 224, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 253, + 285, + 330 + ], + "outputs": [ + 254 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 256, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 227, + "name": "CONV_2D", + "inputs": [ + 254, + 397, + 299 + ], + "outputs": [ + 255 + ], + "config": { + "in_channels": 256, + "out_channels": 256, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 228, + "name": "ADD", + "inputs": [ + 253, + 255 + ], + "outputs": [ + 256 + ], + "config": {} + }, + { + "index": 230, + "name": "PRELU", + "inputs": [ + 256, + 281 + ], + "outputs": [ + 257 + ], + "config": { + "channels": 256 + } + }, + { + "index": 233, + "name": "CONV_2D", + "inputs": [ + 257, + 378, + 412 + ], + "outputs": [ + 258 + ], + "config": { + "in_channels": 256, + "out_channels": 6, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "same", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 234, + "name": "RESHAPE", + "inputs": [ + 258, + 6 + ], + "outputs": [ + 259 + ], + "config": { + "shape": [ + 1, + -1, + 1 + ], + "nhwc_memory_order": true + } + }, + { + "index": 237, + "name": "CONV_2D", + "inputs": [ + 257, + 376, + 391 + ], + "outputs": [ + 260 + ], + "config": { + "in_channels": 256, + "out_channels": 108, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "same", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 238, + "name": "RESHAPE", + "inputs": [ + 260, + 5 + ], + "outputs": [ + 261 + ], + "config": { + "shape": [ + 1, + -1, + 18 + ], + "nhwc_memory_order": true + } + }, + { + "index": 239, + "name": "RESIZE_BILINEAR", + "inputs": [ + 257, + 3 + ], + "outputs": [ + 262 + ], + "config": { + "size": [ + 24, + 24 + ], + "align_corners": false, + "half_pixel_centers": true + } + }, + { + "index": 242, + "name": "CONV_2D", + "inputs": [ + 262, + 360, + 290 + ], + "outputs": [ + 263 + ], + "config": { + "in_channels": 256, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 244, + "name": "PRELU", + "inputs": [ + 263, + 373 + ], + "outputs": [ + 264 + ], + "config": { + "channels": 128 + } + }, + { + "index": 245, + "name": "ADD", + "inputs": [ + 202, + 264 + ], + "outputs": [ + 265 + ], + "config": {} + }, + { + "index": 247, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 265, + 343, + 380 + ], + "outputs": [ + 266 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 250, + "name": "CONV_2D", + "inputs": [ + 266, + 327, + 356 + ], + "outputs": [ + 267 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 251, + "name": "ADD", + "inputs": [ + 265, + 267 + ], + "outputs": [ + 268 + ], + "config": {} + }, + { + "index": 253, + "name": "PRELU", + "inputs": [ + 268, + 322 + ], + "outputs": [ + 269 + ], + "config": { + "channels": 128 + } + }, + { + "index": 255, + "name": "DEPTHWISE_CONV_2D", + "inputs": [ + 269, + 308, + 380 + ], + "outputs": [ + 270 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 5, + 5 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 128, + "has_bias": true, + "padding": "same", + "pad": [ + 2, + 2, + 2, + 2 + ] + } + }, + { + "index": 258, + "name": "CONV_2D", + "inputs": [ + 270, + 402, + 302 + ], + "outputs": [ + 271 + ], + "config": { + "in_channels": 128, + "out_channels": 128, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "valid", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 259, + "name": "ADD", + "inputs": [ + 269, + 271 + ], + "outputs": [ + 272 + ], + "config": {} + }, + { + "index": 261, + "name": "PRELU", + "inputs": [ + 272, + 283 + ], + "outputs": [ + 273 + ], + "config": { + "channels": 128 + } + }, + { + "index": 264, + "name": "CONV_2D", + "inputs": [ + 273, + 383, + 395 + ], + "outputs": [ + 274 + ], + "config": { + "in_channels": 128, + "out_channels": 2, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "same", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 265, + "name": "RESHAPE", + "inputs": [ + 274, + 6 + ], + "outputs": [ + 275 + ], + "config": { + "shape": [ + 1, + -1, + 1 + ], + "nhwc_memory_order": true + } + }, + { + "index": 266, + "name": "CONCATENATION", + "inputs": [ + 275, + 259 + ], + "outputs": [ + 276 + ], + "config": { + "axis": 1 + } + }, + { + "index": 269, + "name": "CONV_2D", + "inputs": [ + 273, + 363, + 393 + ], + "outputs": [ + 277 + ], + "config": { + "in_channels": 128, + "out_channels": 36, + "kernel_size": [ + 1, + 1 + ], + "stride": [ + 1, + 1 + ], + "dilation": [ + 1, + 1 + ], + "groups": 1, + "has_bias": true, + "padding": "same", + "pad": [ + 0, + 0, + 0, + 0 + ] + } + }, + { + "index": 270, + "name": "RESHAPE", + "inputs": [ + 277, + 5 + ], + "outputs": [ + 278 + ], + "config": { + "shape": [ + 1, + -1, + 18 + ], + "nhwc_memory_order": true + } + }, + { + "index": 271, + "name": "CONCATENATION", + "inputs": [ + 278, + 261 + ], + "outputs": [ + 279 + ], + "config": { + "axis": 1 + } + } + ] +} diff --git a/examples/hand_detector/requirements.txt b/examples/hand_detector/requirements.txt new file mode 100644 index 00000000..77509df4 --- /dev/null +++ b/examples/hand_detector/requirements.txt @@ -0,0 +1,3 @@ +numpy>=1.24 +torch>=2.6 +# Install Samsung/TICO separately from PyPI or source. diff --git a/examples/hand_detector/test_hand_detector.py b/examples/hand_detector/test_hand_detector.py new file mode 100644 index 00000000..5bbdbc85 --- /dev/null +++ b/examples/hand_detector/test_hand_detector.py @@ -0,0 +1,259 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Unit tests for the converted hand detector and NHWC export adapter.""" + +from __future__ import annotations + +import json +import unittest + +from collections import Counter +from pathlib import Path + +import numpy as np +import torch + +from examples.hand_detector.hand_detector import ( + load_hand_detector, + load_nhwc_hand_detector, +) +from tico.ops import Concat, ResizeBilinear2d, SamePaddingConv2d + + +DIRECTORY = Path(__file__).resolve().parent + + +def scalar_resize_bilinear_asymmetric( + input_: np.ndarray, + output_size: tuple[int, int], +) -> np.ndarray: + """Compute TFLite false/false ResizeBilinear with scalar loops.""" + + batch_size, input_h, input_w, channels = input_.shape + output_h, output_w = output_size + output = np.empty( + (batch_size, output_h, output_w, channels), + dtype=np.float32, + ) + height_scale = input_h / output_h + width_scale = input_w / output_w + for batch in range(batch_size): + for output_y in range(output_h): + source_y = output_y * height_scale + y0 = int(np.floor(source_y)) + y1 = min(y0 + 1, input_h - 1) + y_weight = source_y - y0 + for output_x in range(output_w): + source_x = output_x * width_scale + x0 = int(np.floor(source_x)) + x1 = min(x0 + 1, input_w - 1) + x_weight = source_x - x0 + top = ( + input_[batch, y0, x0] * (1.0 - x_weight) + + input_[batch, y0, x1] * x_weight + ) + bottom = ( + input_[batch, y1, x0] * (1.0 - x_weight) + + input_[batch, y1, x1] * x_weight + ) + output[batch, output_y, output_x] = ( + top * (1.0 - y_weight) + bottom * y_weight + ) + return output + + +class ResizeBilinearTest(unittest.TestCase): + """Validate eager semantics and the opaque torch.export representation.""" + + def test_custom_op_matches_scalar_asymmetric_coordinates(self) -> None: + """Compare the central custom op with a scalar TFLite reference.""" + + generator = np.random.default_rng(20260728) + source = generator.standard_normal((1, 3, 4, 2), dtype=np.float32) + expected = scalar_resize_bilinear_asymmetric(source, (6, 8)) + actual = torch.ops.circle_custom.resize_bilinear.default( + torch.from_numpy(source), + [6, 8], + False, + False, + ).numpy() + np.testing.assert_allclose(actual, expected, rtol=0.0, atol=1.0e-6) + + def test_module_exports_as_one_custom_operator(self) -> None: + """Check that one facade module becomes one opaque custom operator.""" + + module = ResizeBilinear2d((12, 12)).eval() + exported = torch.export.export( + module, + (torch.zeros(1, 8, 6, 6),), + strict=True, + ) + targets = [ + str(node.target) + for node in exported.graph.nodes + if node.op == "call_function" + ] + self.assertEqual(targets.count("circle_custom.resize_bilinear.default"), 1) + + +class HandDetectorTest(unittest.TestCase): + """Validate model structure, outputs, and the NHWC adapter.""" + + @classmethod + def setUpClass(cls) -> None: + """Load the NCHW detector and NHWC adapter once for all tests.""" + + cls.model = load_hand_detector( + DIRECTORY / "hand_detector_float.pt", + DIRECTORY / "hand_detector_spec.json", + ).eval() + cls.nhwc_model = load_nhwc_hand_detector( + DIRECTORY / "hand_detector_float.pt", + DIRECTORY / "hand_detector_spec.json", + ).eval() + + def test_model_shapes_and_parameter_count(self) -> None: + """Check detector output shapes and the converted parameter count.""" + + with torch.inference_mode(): + regressors, classifiers = self.model(torch.zeros(1, 3, 192, 192)) + self.assertEqual(tuple(regressors.shape), (1, 2016, 18)) + self.assertEqual(tuple(classifiers.shape), (1, 2016, 1)) + self.assertEqual( + sum(parameter.numel() for parameter in self.model.parameters()), + 1_136_248, + ) + + def test_nhwc_adapter_matches_nchw_detector(self) -> None: + """Check that the input adapter changes only the external memory order.""" + + generator = torch.Generator().manual_seed(20260730) + input_nhwc = torch.rand(1, 192, 192, 3, generator=generator) + input_nchw = input_nhwc.permute(0, 3, 1, 2) + with torch.inference_mode(): + expected = self.model(input_nchw) + actual = self.nhwc_model(input_nhwc) + for expected_output, actual_output in zip(expected, actual): + torch.testing.assert_close(actual_output, expected_output) + + def test_nhwc_adapter_exposes_nhwc_example_input(self) -> None: + """Check the example input used as the exported Circle input ABI.""" + + example_inputs = self.nhwc_model.get_example_inputs() + self.assertEqual(len(example_inputs), 1) + self.assertEqual(tuple(example_inputs[0].shape), (1, 192, 192, 3)) + + def test_model_uses_quantizable_pool_and_concat_boundaries(self) -> None: + """Expose native MaxPool2d and TICO Concat modules to WrapQ.""" + + self.assertEqual( + sum( + isinstance(module, torch.nn.MaxPool2d) + for module in self.model.modules() + ), + 4, + ) + self.assertEqual( + sum(isinstance(module, Concat) for module in self.model.modules()), + 2, + ) + self.assertEqual( + sum( + isinstance(module, SamePaddingConv2d) for module in self.model.modules() + ), + 33, + ) + + def test_specification_contains_original_convolution_padding(self) -> None: + """Preserve every source Conv2D SAME or VALID padding option.""" + + specification = json.loads( + (DIRECTORY / "hand_detector_spec.json").read_text(encoding="utf-8") + ) + convolution_operations = [ + operation + for operation in specification["operations"] + if operation["name"] in {"CONV_2D", "DEPTHWISE_CONV_2D"} + ] + paddings = [ + operation["config"]["padding"] for operation in convolution_operations + ] + self.assertEqual(paddings.count("same"), 33) + self.assertEqual(paddings.count("valid"), 30) + + def test_specification_contains_original_resize_options(self) -> None: + """Check both source ResizeBilinear nodes and their exact options.""" + + specification = json.loads( + (DIRECTORY / "hand_detector_spec.json").read_text(encoding="utf-8") + ) + resize_operations = [ + operation + for operation in specification["operations"] + if operation["name"] == "RESIZE_BILINEAR" + ] + self.assertEqual(len(resize_operations), 2) + self.assertEqual( + [operation["config"]["size"] for operation in resize_operations], + [[12, 12], [24, 24]], + ) + for operation in resize_operations: + self.assertFalse(operation["config"]["align_corners"]) + self.assertTrue(operation["config"]["half_pixel_centers"]) + + def test_nhwc_export_contains_expected_input_and_resize_nodes(self) -> None: + """Check the NHWC placeholder and both custom ResizeBilinear nodes.""" + + exported = torch.export.export( + self.nhwc_model, + self.nhwc_model.get_example_inputs(), + strict=True, + ) + user_input_name = exported.graph_signature.user_inputs[0] + user_input = next( + node for node in exported.graph.nodes if node.name == user_input_name + ) + self.assertEqual(tuple(user_input.meta["val"].shape), (1, 192, 192, 3)) + + call_nodes = [ + node for node in exported.graph.nodes if node.op == "call_function" + ] + counts = Counter(str(node.target) for node in call_nodes) + self.assertEqual(counts["circle_custom.resize_bilinear.default"], 2) + self.assertEqual(counts["circle_custom.conv2d.padding"], 5) + self.assertEqual( + counts["circle_custom.depthwise_conv2d.padding"], + 28, + ) + self.assertEqual(counts["aten.pad.default"], 3) + self.assertEqual(counts["aten.cat.default"], 2) + resize_nodes = [ + node + for node in call_nodes + if str(node.target) == "circle_custom.resize_bilinear.default" + ] + self.assertEqual( + [ + (list(node.args[1]), bool(node.args[2]), bool(node.args[3])) + for node in resize_nodes + ], + [([12, 12], False, True), ([24, 24], False, True)], + ) + self.assertEqual(counts["aten.slice.Tensor"], 0) + self.assertEqual(counts["aten.mul.Tensor"], 0) + + +if __name__ == "__main__": + unittest.main() diff --git a/examples/hand_detector/verify.py b/examples/hand_detector/verify.py new file mode 100644 index 00000000..016db2e4 --- /dev/null +++ b/examples/hand_detector/verify.py @@ -0,0 +1,147 @@ +# Copyright (c) 2026 Samsung Electronics Co., Ltd. All Rights Reserved +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Verify torch.export or Circle artifacts produced by the hand-detector example.""" + +from __future__ import annotations + +import argparse +from collections import Counter +from pathlib import Path + +import torch + +from examples.hand_detector._support.verify_circle_layout import verify_circle_layout +from examples.hand_detector._support.verify_circle_resize import ( + read_resize_bilinear_options, +) +from examples.hand_detector._support.verify_quantized_circle import ( + verify_quantized_circle, +) +from examples.hand_detector.hand_detector import load_nhwc_hand_detector + + +DIRECTORY = Path(__file__).resolve().parent +RESIZE_TARGET = "circle_custom.resize_bilinear.default" + + +def parse_args() -> argparse.Namespace: + """Parse the artifact type and expected properties.""" + parser = argparse.ArgumentParser(description=__doc__) + subparsers = parser.add_subparsers(dest="mode", required=True) + + torch_parser = subparsers.add_parser("torch", help="Verify torch.export.") + torch_parser.add_argument( + "--weights", + type=Path, + default=DIRECTORY / "hand_detector_float.pt", + ) + torch_parser.add_argument( + "--spec", + type=Path, + default=DIRECTORY / "hand_detector_spec.json", + ) + + circle_parser = subparsers.add_parser("circle", help="Verify FP32 Circle layout.") + circle_parser.add_argument("circle", type=Path) + circle_parser.add_argument("--expected-resize-count", type=int, default=2) + + quantized = subparsers.add_parser( + "quantized", + help="Verify quantized Circle metadata and layout.", + ) + quantized.add_argument("circle", type=Path) + quantized.add_argument("--bits", type=int, required=True, choices=[8, 16]) + return parser.parse_args() + + +def main() -> None: + """Dispatch the selected verification path.""" + args = parse_args() + if args.mode == "torch": + _verify_torch(args.weights, args.spec) + elif args.mode == "circle": + _verify_circle(args.circle, args.expected_resize_count) + elif args.mode == "quantized": + summary = verify_quantized_circle(args.circle, args.bits) + summary["layout"] = verify_circle_layout(args.circle) + print(f"Verified {args.circle}") + else: + raise RuntimeError(f"Unhandled verification mode: {args.mode}") + + +def _verify_torch(weights: Path, spec: Path) -> None: + model = load_nhwc_hand_detector(weights, spec).eval() + exported = torch.export.export(model, model.get_example_inputs(), strict=True) + placeholders = [node for node in exported.graph.nodes if node.op == "placeholder"] + user_inputs = [ + node + for node in placeholders + if tuple(getattr(node.meta.get("val"), "shape", ())) == (1, 192, 192, 3) + ] + if len(user_inputs) != 1: + raise RuntimeError("Expected one NHWC image placeholder [1, 192, 192, 3].") + + call_nodes = [node for node in exported.graph.nodes if node.op == "call_function"] + counts = Counter(str(node.target) for node in call_nodes) + resize_nodes = [node for node in call_nodes if str(node.target) == RESIZE_TARGET] + expected_options = [([12, 12], False, True), ([24, 24], False, True)] + actual_options = [ + (list(node.args[1]), bool(node.args[2]), bool(node.args[3])) + for node in resize_nodes + ] + if actual_options != expected_options: + raise RuntimeError( + f"Expected ResizeBilinear options {expected_options}, " + f"found {actual_options}." + ) + expected_counts = { + "circle_custom.conv2d.padding": 5, + "circle_custom.depthwise_conv2d.padding": 28, + "aten.pad.default": 3, + "aten.cat.default": 2, + } + actual_counts = {target: counts[target] for target in expected_counts} + if actual_counts != expected_counts: + raise RuntimeError( + f"Expected structural counts {expected_counts}, found {actual_counts}." + ) + forbidden = { + "aten.slice.Tensor", + "aten.mul.Tensor", + "aten.upsample_bilinear2d.default", + "aten.upsample_bilinear2d.vec", + } + present = {target: counts[target] for target in forbidden if counts[target]} + if present: + raise RuntimeError(f"ResizeBilinear was unexpectedly decomposed: {present}") + print("Verified NHWC torch.export input and two opaque ResizeBilinear nodes.") + + +def _verify_circle(path: Path, expected_resize_count: int) -> None: + layout = verify_circle_layout(path) + options = read_resize_bilinear_options(path) + expected_options = [(False, True)] * expected_resize_count + if options != expected_options: + raise RuntimeError( + f"Expected ResizeBilinear options {expected_options}, found {options}." + ) + print(f"Verified {path}") + print(f"Input shapes: {layout['input_shapes']}") + print(f"Remaining Transpose operators: {layout['transpose_count']}") + print(f"ResizeBilinear operators: {len(options)}") + + +if __name__ == "__main__": + main()