Skip to content

Commit 1902874

Browse files
committed
Make templates external; add example config
Refactor bindgen to use external templates and provide an example setup. - README: document that templates are provided by the user's repo (example templates kept for reference) and update architecture/CLI/docs to reflect external template discovery. - CLI: change --config help to point to config.yaml, remove lang/platform overrides, pass config path to generator. - Codegen: discover templates under <config_dir>/template/*.j2, load mapping from config, use jinja2 to render templates; improved error messages. - Config: replace languages/platform fields with a mapping dict (BindgenConfig.mapping) and adjust load_config accordingly. - Parser: auto-discover entry headers under src/ and default include_paths if not set; add helpers to find project root and discover headers. - Add example: tools/bindgen/example with config.yaml, example template and README showing usage. - Remove built-in Dart templates (bindings.j2, helpers.j2, lang.yaml) since templates are now external. These changes let consumers supply their own language templates and mapping next to their config.yaml while keeping an example for validation.
1 parent 6ac78b8 commit 1902874

File tree

12 files changed

+166
-235
lines changed

12 files changed

+166
-235
lines changed

tools/bindgen/README.md

Lines changed: 41 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Bindgen 设计文档(Python)
22

3-
本目录目标:使用 Python 实现一个面向 C++ Headers 的 bindgen,自动解析头文件并生成多语言绑定(Dart / Swift / Rust 等)。各语言 API **统一调用 C API**,再由 C API 调用 C++ 实现,确保 ABI 稳定与可移植性。
3+
本目录目标:使用 Python 实现一个面向 C++ Headers 的 bindgen,自动解析头文件并生成多语言绑定。各语言 API **统一调用 C API**,再由 C API 调用 C++ 实现,确保 ABI 稳定与可移植性。模板不内置于本仓库,由使用方在自己的仓库提供;本仓库仅保留示例模板用于验证与参考
44

55
本文档说明核心需求、架构、数据模型、生成流程、CLI、可扩展性与里程碑计划。
66

@@ -9,7 +9,7 @@
99
**目标**
1010

1111
- 解析 C++ 公开 API(以头文件为主)
12-
- 生成多语言 bindings(Dart FFI、Swift bridging、Rust FFI
12+
- 生成多语言 bindings(模板由使用方仓库提供
1313
- 所有语言绑定统一调用 C API 层(避免直接绑定 C++ ABI)
1414
- 统一的中间表示(IR),减少多语言实现成本
1515
- 可配置、可扩展、可测试
@@ -38,15 +38,12 @@ C++ Headers
3838
v
3939
Code Generator (通用生成器)
4040
|
41-
+--------------------+--------------------+--------------------+
42-
| | |
43-
v v v
44-
Dart Templates Swift Templates Rust Templates
45-
| | |
46-
+---------+----------+---------+----------+---------+
47-
| |
48-
v v
49-
Output bindings (ffi glue per language)
41+
|
42+
v
43+
External Templates
44+
|
45+
v
46+
Bindings Output
5047
|
5148
v
5249
C API (extern "C")
@@ -59,8 +56,8 @@ C++ Headers
5956
- **Normalizer**:将 AST 规约为可生成的 IR(类型、函数、常量、结构体)
6057
- **IR**:语言无关的描述结构
6158
- **Code Generator**:通用生成器,负责加载模板与语言映射配置并输出 bindings
62-
- **Templates**语言模板 + 语言映射配置(类型映射、命名风格、ABI 约定)
63-
- **C API**:稳定 ABI 层,`extern "C"` 函数,供所有语言 bindings 调用
59+
- **Templates**使用方自定义模板 + 映射配置(类型映射、命名风格、ABI 约定)
60+
- **C API**:稳定 ABI 层,`extern "C"` 函数
6461
- **C++ Implementation**:真实实现层,仅由 C API 访问
6562

6663
## 输入与约束
@@ -149,7 +146,7 @@ C++ Headers
149146

150147
**绑定链路约束**
151148

152-
- 生成的 Dart/Swift/Rust bindings **只调用 C API**,不直接依赖 C++ 符号
149+
- 生成的 bindings **只调用 C API**,不直接依赖 C++ 符号
153150
- C API 负责:参数规约、ABI 稳定、与 C++ 实现交互
154151
- C++ 实现层可自由演进,但对外 ABI 需保持稳定
155152

@@ -176,38 +173,25 @@ tools/bindgen/
176173
codegen/
177174
generator.py # 通用生成器入口
178175
context.py # 模板上下文构建
179-
templates/
180-
dart/
181-
bindings.j2
182-
helpers.j2
183-
lang.yaml
184-
swift/
185-
bindings.j2
186-
modulemap.j2
187-
lang.yaml
188-
rust/
189-
bindings.j2
190-
lib.j2
191-
lang.yaml
176+
example/
177+
bindgen/
178+
config.yaml # 示例配置
179+
template/
180+
example.txt.j2 # 示例目标语言模板(用于验证与参考)
181+
README.md
192182
```
193183

194184
**语言映射配置(示例)**
195185

186+
语言相关配置统一写在 `config.yaml``mapping` 字段中:
187+
196188
```yaml
197-
language: dart
198-
types:
199-
void: Void
200-
int32: Int32
201-
uint32: Uint32
202-
float32: Float
203-
cstring: Pointer<Utf8>
204-
conventions:
205-
enum_as_int: true
206-
struct_layout: ffi.Struct
207-
callconv: native
208-
naming:
209-
function: camel
210-
constant: upper_snake
189+
mapping:
190+
language: example
191+
conventions:
192+
enum_as_int: true
193+
naming:
194+
function: camel
211195
```
212196
213197
**通用生成器伪代码**
@@ -227,63 +211,36 @@ render_to_files(templates, context, out_dir)
227211
- `functions`: 函数列表(已过滤)
228212
- `constants`: 常量列表
229213
- `mapping`: 语言映射配置(types/conventions/naming)
230-
- `helpers`: 预计算字段(如 dart ffi 类型名、rust type 名)
231214

232215
**优点**
233216

234217
- 新增语言只需新增模板 + 映射配置
235218
- 语言差异集中可见,维护成本低
236219
- 测试可按模板/映射分层
237220

238-
### Dart
239-
240-
- 生成 `ffi.DynamicLibrary` + `typedef` + `lookupFunction`
241-
- `struct` 映射为 `ffi.Struct`
242-
- `enum` 映射为 `int`
243-
- `cstring` -> `Pointer<Utf8>` + helper
244-
- 输出文件(建议):`bindings.dart`, `helpers.dart`
221+
### Templates (External)
245222

246-
### Swift
247-
248-
- C header 生成 module map(如需)
249-
- `struct` 使用 `@frozen struct``UnsafePointer`
250-
- `enum` -> Swift `enum` with rawValue
251-
- 暴露 C 函数桥接层
252-
- 输出文件(建议):`Bindings.swift`, `module.modulemap`(可选)
253-
254-
### Rust
255-
256-
- 生成 `extern "C"` block
257-
- `struct` -> `#[repr(C)] struct`
258-
- `enum` -> `#[repr(C)] enum` or `type alias` for constants
259-
- `cstring` -> `*const c_char`
260-
- 输出文件(建议):`lib.rs`, `bindings.rs`
223+
- 本仓库仅保留示例模板用于验证
224+
- 实际多语言绑定由使用方在自己的仓库提供模板与映射配置
261225

262226
## 配置与注解
263227

264228
### YAML 配置文件
265229

266230
```yaml
267-
entry_headers:
268-
- include/nativeapi.h
269-
include_paths:
270-
- include
271231
clang_flags:
272232
- -std=c11
273233
- -DNATIVEAPI_EXPORT
274-
languages:
275-
- dart
276-
- swift
277-
- rust
234+
mapping:
235+
language: example
278236
filters:
279237
export_macro: NATIVEAPI_EXPORT
280238
allowlist_regex: []
281239
denylist_regex: []
282-
platform:
283-
os: macos
284-
abi: clang
285240
```
286241
242+
说明:`entry_headers` 将自动从 `src/` 目录下收集所有 `.h`(忽略 `platform/`),`include_paths` 也会默认指向 `src/`,无需在配置中手动列出。模板目录默认读取 `config.yaml` 同路径下的 `template/`。
243+
287244
### 注解建议(可扩展)
288245

289246
- 使用宏或注释标注 API,如:
@@ -295,30 +252,32 @@ platform:
295252
## CLI 设计
296253

297254
```
298-
PYTHONPATH=tools python -m bindgen \
299-
--config tools/bindgen/bindgen.yaml \
300-
--out tools/bindgen/out
255+
cd tools/bindgen/example
256+
PYTHONPATH=../.. python3 -m bindgen \
257+
--config bindgen/config.yaml \
258+
--dump-ir out/ir.json \
259+
--out out
301260
```
302261
262+
示例运行后将使用 `bindgen/template` 中的模板,并把 IR 与生成结果都写到 `out`。
263+
303264
常用参数:
304265
305266
- `--config` 配置文件
306267
- `--out` 输出目录
307-
- `--lang` 指定语言(可覆盖配置)
308268
- `--dump-ir` 输出 IR JSON 便于调试
309-
- `--platform` 指定平台(如 `windows-msvc`、`linux-gnu`、`macos-clang`)
310269
311270
## 开发计划(里程碑)
312271
313272
1. **MVP**
314273
- 支持解析函数 + 基础类型
315-
- 生成 Dart bindings
274+
- 生成示例模板输出
316275
- IR JSON dump
317276
- 最小可用示例(单头文件 -> bindings)
318277
319278
2. **扩展**
320279
- 支持 struct、enum
321-
- 生成 Swift/Rust bindings
280+
- 按需扩展多语言模板
322281
323282
3. **增强**
324283
- 增加注解/宏过滤

tools/bindgen/bindgen.yaml

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,11 @@
1-
entry_headers:
2-
- include/nativeapi.h
3-
include_paths:
4-
- include
51
clang_flags:
62
- -x
73
- c++
84
- -std=c++17
95
- -DNATIVEAPI_EXPORT
10-
languages:
11-
- dart
6+
mapping:
7+
language: example
128
filters:
139
export_macro: NATIVEAPI_EXPORT
1410
allowlist_regex: []
1511
denylist_regex: []
16-
platform:
17-
os: macos
18-
abi: clang

tools/bindgen/cli.py

Lines changed: 6 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,23 @@
11
import argparse
2-
import json
3-
import sys
42
from pathlib import Path
53

4+
from .codegen.generator import generate_bindings
65
from .config import load_config
76
from .ir.serializer import dump_ir_json
8-
from .parser import parse_headers
97
from .normalizer import normalize_translation_unit
10-
from .codegen.generator import generate_bindings
8+
from .parser import parse_headers
119

1210

1311
def main(argv=None) -> int:
1412
parser = argparse.ArgumentParser(prog="bindgen")
15-
parser.add_argument("--config", required=True, help="Path to bindgen.yaml")
13+
parser.add_argument("--config", required=True, help="Path to config.yaml")
1614
parser.add_argument("--out", required=True, help="Output directory")
17-
parser.add_argument("--lang", action="append", help="Limit to one or more languages")
1815
parser.add_argument("--dump-ir", help="Write IR JSON to path")
19-
parser.add_argument("--platform", help="Override platform name")
2016

2117
args = parser.parse_args(argv)
2218

23-
cfg = load_config(Path(args.config))
24-
if args.lang:
25-
cfg.languages = args.lang
26-
if args.platform:
27-
cfg.platform.os = args.platform
19+
config_path = Path(args.config)
20+
cfg = load_config(config_path)
2821

2922
tu = parse_headers(cfg)
3023
module = normalize_translation_unit(tu, cfg)
@@ -34,6 +27,6 @@ def main(argv=None) -> int:
3427

3528
out_dir = Path(args.out)
3629
out_dir.mkdir(parents=True, exist_ok=True)
37-
generate_bindings(module, cfg, out_dir)
30+
generate_bindings(module, cfg, out_dir, config_path)
3831

3932
return 0

tools/bindgen/codegen/generator.py

Lines changed: 26 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
from __future__ import annotations
22

33
from pathlib import Path
4-
from typing import Dict
54

65
from ..config import BindgenConfig
76
from ..ir.model import IRModule
@@ -12,32 +11,32 @@ def _load_jinja():
1211
try:
1312
from jinja2 import Environment, FileSystemLoader # type: ignore
1413
except Exception as exc: # pragma: no cover
15-
raise RuntimeError("jinja2 is required. Install with `pip install jinja2`.") from exc
14+
raise RuntimeError(
15+
"jinja2 is required. Install with `pip install jinja2`."
16+
) from exc
1617
return Environment, FileSystemLoader
1718

1819

19-
def _load_yaml(path: Path) -> dict:
20-
try:
21-
import yaml # type: ignore
22-
except Exception as exc: # pragma: no cover
23-
raise RuntimeError("PyYAML is required for language configs.") from exc
24-
with path.open("r", encoding="utf-8") as f:
25-
return yaml.safe_load(f) or {}
26-
27-
28-
def generate_bindings(module: IRModule, cfg: BindgenConfig, out_dir: Path) -> None:
29-
for lang in cfg.languages:
30-
lang_dir = Path(__file__).resolve().parent.parent / "templates" / lang
31-
if not lang_dir.exists():
32-
raise RuntimeError(f"Unknown language template: {lang}")
33-
lang_cfg = _load_yaml(lang_dir / "lang.yaml")
34-
35-
Environment, FileSystemLoader = _load_jinja()
36-
env = Environment(loader=FileSystemLoader(str(lang_dir)), autoescape=False)
37-
context = build_context(module, lang_cfg)
38-
39-
for template_path in lang_dir.glob("*.j2"):
40-
template = env.get_template(template_path.name)
41-
output_name = template_path.stem
42-
rendered = template.render(**context)
43-
(out_dir / output_name).write_text(rendered + "\n", encoding="utf-8")
20+
def generate_bindings(
21+
module: IRModule, cfg: BindgenConfig, out_dir: Path, config_path: Path
22+
) -> None:
23+
# Templates are discovered from config.yaml sibling directory:
24+
# <config_dir>/template/*.j2
25+
config_template_root = config_path.resolve().parent / "template"
26+
if not config_template_root.exists():
27+
raise RuntimeError(f"Template directory not found: {config_template_root}")
28+
29+
mapping = dict(cfg.mapping)
30+
mapping.setdefault("language", "default")
31+
32+
Environment, FileSystemLoader = _load_jinja()
33+
env = Environment(
34+
loader=FileSystemLoader(str(config_template_root)), autoescape=False
35+
)
36+
context = build_context(module, mapping)
37+
38+
for template_path in config_template_root.glob("*.j2"):
39+
template = env.get_template(template_path.name)
40+
output_name = template_path.stem
41+
rendered = template.render(**context)
42+
(out_dir / output_name).write_text(rendered + "\n", encoding="utf-8")

0 commit comments

Comments
 (0)