[codex] Add v0.6 general-shape plan API

CHANGELOG.md

@@ -2,6 +2,48 @@
 
 All notable repository and research-artifact changes are recorded here.
 
+## 0.6.0 - 2026-08-11 General Shapes And Plan API
+
+### Added
+
+- CUDA-library-style C handle/descriptor/plan API with streams, explicit
+  workspace, status codes, selection queries, and structured logging.
+- Exact arbitrary-length complex FP32/FP64 FFT through Bluestein and exact
+  uint64 modular Bluestein NTT when the requested root domain exists.
+- Explicit zero-extended power-of-two embedding for all public operators.
+- Rank-two FFT, NTT, FWHT, zeta/Mobius, and Structured 2x2 composition through
+  per-axis plans and transpose boundaries.
+- Versioned V100 application profile table generated at build time, exact-key
+  user cache, bounded runtime measurement policy, and profile-miss guidance.
+- A `cubutterfly_plan_bench` CLI for logical/physical shape, batch, policy,
+  cache, workspace, and kernel-time inspection.
+- Per-axis static/cache/measurement selection for embedded and rank-two
+  compositions, selectable Hybrid2D Bluestein NTT cores, and complete direct
+  cuFFT versus Bluestein measurement for exact arbitrary FFT.
+- Structured 2x2 matrices and explicit composite algorithm IDs in the plan
+  benchmark surface.
+- Direct physical-output lowering for contiguous embedded and rank-two
+  compositions, eliminating the final full-buffer scatter while retaining the
+  general strided-output path.
+- Generated zero-extended input dispatch for warp-register FP32 FWHT and
+  Structured 2x2 units, with the runtime selecting the one-kernel fused-input
+  lowering for validated FWHT workloads and retaining pack fallback otherwise.
+
+### Current Evidence Boundary
+
+- All 49 CTest entries pass on V100/CUDA 11.8, including exact arbitrary
+  length, embedding, rank-two, strided fallback, cache, and generated boundary
+  candidate coverage.
+- Direct packed arbitrary FFT selects cuFFT at timing parity; this is
+  processing-unit abstention, not a claim of improving the vendor core.
+- Fused-input FWHT reaches 0.11759 ms for `32767 -> 32768,batch=256`, within
+  1.01x of the archived pre-padded Dao result, and uses zero composition
+  workspace.
+- Structured fused input is retained as a correct generated candidate but is
+  not selected because the measured V100 path regresses.
+- Cross-GPU selection transfer and universal specialist-library superiority
+  remain outside the release claim.
+
 ## 0.5.0 - 2026-08-08 Numeric-Regime Mapping
 
 ### Added

CITATION.cff

@@ -6,8 +6,8 @@ authors:
   - name: "TruNcat3"
 repository-code: "https://github.com/TruNcat3/cuButterfly"
 url: "https://github.com/TruNcat3/cuButterfly"
-version: 0.5.0
-date-released: 2026-08-08
+version: 0.6.0
+date-released: 2026-08-11
 keywords:
   - butterfly computation
   - GPU

CMakeLists.txt

@@ -4,7 +4,7 @@ if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
     set(CMAKE_CUDA_ARCHITECTURES 70 CACHE STRING "CUDA architectures to compile for")
 endif()
 
-project(cuButterfly VERSION 0.5.0 LANGUAGES CXX CUDA)
+project(cuButterfly VERSION 0.6.0 LANGUAGES C CXX CUDA)
 
 include(GNUInstallDirs)
 include(CMakePackageConfigHelpers)
@@ -144,11 +144,25 @@ execute_process(
 if(NOT CUBUTTERFLY_RUNTIME_SELECTOR_RESULT EQUAL 0)
     message(FATAL_ERROR "cuButterfly runtime selector generation failed: ${CUBUTTERFLY_RUNTIME_SELECTOR_ERROR}")
 endif()
+execute_process(
+    COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/scripts/generate_application_profiles.py"
+            --spec "${CMAKE_CURRENT_SOURCE_DIR}/config/v100_application_profiles.json"
+            --header "${CUNTT_GENERATED_DIR}/generated_application_profiles.hpp"
+    RESULT_VARIABLE CUBUTTERFLY_APPLICATION_PROFILE_RESULT
+    ERROR_VARIABLE CUBUTTERFLY_APPLICATION_PROFILE_ERROR)
+if(NOT CUBUTTERFLY_APPLICATION_PROFILE_RESULT EQUAL 0)
+    message(FATAL_ERROR "cuButterfly application profile generation failed: ${CUBUTTERFLY_APPLICATION_PROFILE_ERROR}")
+endif()
 include("${CUNTT_GENERATED_DIR}/design_points.cmake")
 
 add_library(cuntt STATIC
+    src/bluestein_fft.cu
+    src/bluestein_ntt.cu
+    src/c_api.cpp
+    src/direct_fft.cu
     src/butterfly.cu
     src/butterfly_reference.cpp
+    src/general_kernels.cu
     src/mapping_selector.cpp
     src/ntt.cu
     src/reference.cpp)
@@ -225,6 +239,10 @@ add_executable(cubutterfly_bench apps/cubutterfly_bench.cpp)
 target_link_libraries(cubutterfly_bench PRIVATE cuntt)
 target_compile_options(cubutterfly_bench PRIVATE -Wall -Wextra -Wpedantic)
 
+add_executable(cubutterfly_plan_bench apps/cubutterfly_plan_bench.cpp)
+target_link_libraries(cubutterfly_plan_bench PRIVATE cuntt)
+target_compile_options(cubutterfly_plan_bench PRIVATE -Wall -Wextra -Wpedantic)
+
 add_executable(cuntt_tests tests/test_ntt.cpp)
 target_link_libraries(cuntt_tests PRIVATE cuntt)
 target_compile_options(cuntt_tests PRIVATE -Wall -Wextra -Wpedantic)
@@ -236,6 +254,9 @@ target_compile_options(cubutterfly_tests PRIVATE -Wall -Wextra -Wpedantic)
 add_executable(cubutterfly_structured_reference_tests tests/test_structured_reference.cpp)
 target_link_libraries(cubutterfly_structured_reference_tests PRIVATE cuntt)
 target_compile_options(cubutterfly_structured_reference_tests PRIVATE -Wall -Wextra -Wpedantic)
+add_executable(cubutterfly_c_api_tests tests/test_c_api.c)
+target_link_libraries(cubutterfly_c_api_tests PRIVATE cuntt)
+target_compile_options(cubutterfly_c_api_tests PRIVATE -Wall -Wextra -Wpedantic)
 if(CUBUTTERFLY_ENABLE_CUFFTDX)
     target_compile_definitions(cubutterfly_tests PRIVATE CUBUTTERFLY_TEST_CUFFTDX=1)
 endif()
@@ -267,6 +288,7 @@ if(BUILD_TESTING)
     add_test(NAME cuntt_correctness COMMAND cuntt_tests)
     add_test(NAME cubutterfly_correctness COMMAND cubutterfly_tests)
     add_test(NAME cubutterfly_structured_reference COMMAND cubutterfly_structured_reference_tests)
+    add_test(NAME cubutterfly_c_api COMMAND cubutterfly_c_api_tests)
     add_test(NAME fft_design_space
              COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/tests/test_fft_design_space.py")
     add_test(NAME butterfly_design_space
@@ -312,6 +334,8 @@ if(BUILD_TESTING)
                      -P "${CMAKE_CURRENT_SOURCE_DIR}/tests/test_install_package.cmake")
     add_test(NAME runtime_selector_generator
              COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/tests/test_runtime_selector_generator.py")
+    add_test(NAME application_profile_generator
+             COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/tests/test_application_profiles.py")
     if(CUBUTTERFLY_ENABLE_CUFFTDX)
         add_test(NAME fft_fp64_cufftdx_block_correctness
                  COMMAND cubutterfly_bench --operator fft --precision fp64 --backend temporal-tile
@@ -450,7 +474,7 @@ install(TARGETS cuntt
     ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
     LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
     RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}")
-install(DIRECTORY include/cuntt DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}")
+install(DIRECTORY include/cuntt include/cubutterfly DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}")
 
 configure_package_config_file(
     "${CMAKE_CURRENT_SOURCE_DIR}/cmake/cuButterflyConfig.cmake.in"

README.md

@@ -18,6 +18,27 @@ processing-unit boundary, same-machine library comparisons, Nsight profiling
 scripts, and the raw CSV data used in the reports. It is a research artifact,
 not a drop-in replacement for cuFFT or a production cryptography library.
 
+The v0.6 API adds a CUDA-library-style C plan surface, exact
+non-power-of-two FFT/compatible NTT lowering, explicit power-of-two embedding,
+rank-two axis mapping, caller-owned workspace, static application profiles,
+one-time measured tuning, and exact-key cache reuse. See the
+[C Plan API](docs/c_api.md) and [General Shape Mapping](docs/general_shapes.md).
+The [general-shape comparison](docs/general_shape_results.md) records both the
+initial unfused lowering and the new composition selector. Exact packed FFT
+now selects direct cuFFT at parity, while modular NTT and embedded operators
+reuse selected physical cores. Composition boundaries are reported explicitly:
+contiguous output removes scatter, selected warp-register FWHT also fuses
+logical input and zero fill, and all other paths retain their documented
+materialized boundary.
+Contiguous embedded outputs also bypass the final scatter and expose that
+choice as `+direct-output` in the selected algorithm name.
+Generated FP32 warp-register FWHT can also consume the logical input and
+synthesize its zero tail inside the physical kernel. This one-kernel lowering
+is exposed as `+fused-input`; unsupported or measured-slower combinations keep
+the explicit pack path.
+The complete release boundary is summarized in the
+[v0.6 Implementation Status](docs/v0.6_implementation_status.md).
+
 ## Why This Design
 
 A fast butterfly kernel needs more than a fast butterfly instruction. It must
@@ -104,7 +125,10 @@ The historical `xor-zeta` API name is retained as a compatibility spelling for
 its original subset-zeta behavior. See the [Operator
 Catalog](docs/operator_catalog.md) for exact semantics and expansion priorities.
 
-Power-of-two lengths, batches, forward/inverse execution, normalization,
+The C++ kernel-level API covers power-of-two lengths. The unified C plan API
+also covers exact non-power-of-two FP32/FP64 FFT, compatible uint64 NTT,
+explicit power-of-two embedding for every operator, and rank-two axis
+composition. Batches, forward/inverse execution, normalization,
 in-place/out-of-place placement, padded batches, and strided elements are
 covered where listed in the [Feature Matrix](docs/cubutterfly_feature_coverage.md).
 Unsupported generated combinations fail explicitly rather than falling back to
@@ -164,6 +188,23 @@ all precisions and shapes.
 | 60-bit NTT, `logN=16`, natural order | 0.274104 ms | GPU-NTT 0.291133 ms | 1.062x | fused Hybrid2D |
 | 60-bit NTT, `logN=20`, native bit-reversed | 0.341320 ms | GPU-NTT 0.396186 ms | 1.161x | compact-stage mapping |
 
+### General-Shape Composition
+
+These v0.6 rows time the complete logical-input resident composition, not only
+the selected power-of-two core:
+
+| Workload | Initial composition | Selected composition | Result |
+|:--|--:|--:|:--|
+| exact FP32 FFT `N=1000,batch=64` | 0.02622 ms Bluestein | 0.00427 ms direct cuFFT | 0.998x matching cuFFT; processing-unit abstention |
+| exact rank-2 FP32 FFT `30x50,batch=64` | 0.07993 ms Bluestein | 0.00981 ms direct cuFFT | 1.000x matching cuFFT; processing-unit abstention |
+| embedded FP32 FWHT `32767 -> 32768,batch=256` | 0.73772 ms | **0.11759 ms** | 6.27x composition gain; within 1.01x of pre-padded Dao core |
+| embedded FP32 FWHT `255 -> 256,batch=65536` | 0.60309 ms | **0.16511 ms** | 3.65x composition gain |
+
+The FFT rows deliberately reuse the vendor exact-length unit and therefore do
+not claim to outperform cuFFT. See [General-Shape V100 Selection
+Results](docs/general_shape_results.md) for NTT, zeta, Structured, boundary,
+workspace, and raw-trial details.
+
 The generated local core, fused physical grouping, and vectorized two-pass
 composition bring the selected long FFT shapes to 95.4%-108.5% of cuFFT.
 Whole-transform cuFFTDx
@@ -356,7 +397,7 @@ software release as:
   author  = {TruNcat3},
   title   = {cuButterfly: Hardware-Mapped Space-Time Parallelism for Butterfly Computations on GPUs},
   year    = {2026},
-  version = {0.5.0},
+  version = {0.6.0},
   url     = {https://github.com/TruNcat3/cuButterfly}
 }
 ```

ROADMAP.md

@@ -3,6 +3,26 @@
 This roadmap separates implementation goals from evidence required for the
 research claim. Ordering may change after profiling or access to new hardware.
 
+## v0.6.0 Status: General Shapes And Professional Plan Surface
+
+Status: complete on the single-V100 release target. General-shape semantics,
+the professional plan surface, boundary selection, tests, raw evidence, and
+release documentation are checked in.
+The new C plan API separates standard exact-length semantics from explicit
+zero-extended embedding, lowers rank-two workloads to per-axis mappings, and
+adds exact static-profile/cache selection with actionable miss logs. Exact FFT
+selects between native cuFFT and Bluestein; modular Bluestein NTT is enabled
+only when its root requirements hold and now reuses the selected radix-4
+Hybrid2D core. Embedded and rank-two plans select and cache each physical axis.
+The initial Bluestein-only FFT reached 0.105x-0.179x of cuFFT; the new selector
+reaches parity by abstaining to direct cuFFT. Boundary fusion for embedded
+operators now removes contiguous output scatter, and selected FP32
+warp-register FWHT also fuses logical input and zero fill. The complete
+logical `32767 -> 32768,batch=256` FWHT path reaches 0.11759 ms, within 1.01x
+of the archived pre-padded Dao core. Structured fused input remains a generated
+but unselected candidate because it regresses on V100. See
+[v0.6 Implementation Status](docs/v0.6_implementation_status.md).
+
 ## v0.4.0 Status: Library And Resource-Model Integration
 
 Status: complete on V100. This release adds the public device API and package,
@@ -75,6 +95,11 @@ artifacts, and decision gates.
 
 ## Near Term Backlog
 
+- Search boundary residence jointly with processing unit, numeric contract,
+  logical padding ratio, batch alignment, and layout instead of promoting a
+  fusion family from one successful operator.
+- Add a padded-layout reuse contract for adjacent plan executions so an
+  application can retain physical data without repeated embedding work.
 - Extend the fixed-V100 regime model to stride, placement, direction,
   normalization, coefficient policy, and output-order neighborhoods using the
   same screen-follow-up-counter hierarchy.