Skip to content

Commit ebc55b7

Browse files
committed
Support per-file templates and IR JSON load
Add per-file template rendering, partial templates, and IR JSON import/export. - CLI: add --ir option to load an existing IR JSON (skips parsing) and keep --dump-ir support. - Generator: implement per-file templates (template/file/*.j2), partials support, output path computation, and rendering rules; render global templates once. Register handy Jinja2 filters (snake_case, camel_case, pascal_case, screaming_snake_case, kebab_case, strip/add prefix/suffix). - Serializer: add load_ir_json and many parsing helpers to reconstruct IR types/files from JSON. - Examples: add example per-file template and partial templates (type/enum/function/class/constant/alias) under example/bindgen/template to demonstrate usage. - README: document template directory layout, partials usage, per-file rendering rules and output path conventions. These changes enable faster iteration (load IR from JSON) and more flexible templating (per-file outputs and reusable partials).
1 parent 24ac439 commit ebc55b7

File tree

11 files changed

+617
-11
lines changed

11 files changed

+617
-11
lines changed

tools/bindgen/README.md

Lines changed: 53 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -202,10 +202,40 @@ tools/bindgen/
202202
bindgen/
203203
config.yaml # 示例配置
204204
template/
205-
example.txt.j2 # 示例目标语言模板(用于验证与参考)
205+
*.j2 # 全局模板(渲染一次)
206+
file/
207+
*.j2 # 按文件模板(每个 IR 文件渲染一次)
208+
partials/
209+
type.j2 # 结构体片段模板
210+
enum.j2 # 枚举片段模板
211+
function.j2 # 函数片段模板
212+
class.j2 # 类片段模板
213+
constant.j2 # 常量片段模板
214+
alias.j2 # 类型别名片段模板
206215
README.md
207216
```
208217

218+
**模板目录结构说明**
219+
220+
- `template/*.j2` - 全局模板,只渲染一次,生成单个输出文件
221+
- `template/file/*.j2` - 按文件模板,为每个 IR 源文件渲染一次
222+
- `template/partials/*.j2` - 片段模板(宏),可被其他模板导入使用
223+
224+
**使用 partials 片段模板**
225+
226+
片段模板定义了可复用的宏,用于渲染单个 IR 元素:
227+
228+
```jinja2
229+
{# 在 file/*.j2 中导入并使用 partials #}
230+
{% from 'partials/type.j2' import render_type %}
231+
{% from 'partials/enum.j2' import render_enum %}
232+
{% from 'partials/class.j2' import render_class %}
233+
234+
{% for item in types %}
235+
{{ render_type(item) }}
236+
{% endfor %}
237+
```
238+
209239
**语言映射配置(示例)**
210240

211241
语言相关配置统一写在 `config.yaml``mapping` 字段中:
@@ -240,11 +270,33 @@ render_to_files(templates, context, out_dir)
240270
- `constants`: 常量列表(扁平化)
241271
- `mapping`: 语言映射配置(types/conventions/naming)
242272

273+
**按文件模板渲染规则**
274+
275+
`template/file/*.j2` 中的模板会按 IR 的 `file_paths` 逐文件渲染:
276+
277+
- 渲染时 `types/enums/functions/classes/constants/aliases` 仅包含当前文件的内容
278+
- 额外提供全量列表:`all_types/all_enums/all_functions/all_classes/all_constants/all_aliases`
279+
- `file_path``file` 提供当前文件信息
280+
281+
**输出路径规则**
282+
283+
输出路径由 IR 文件路径和模板名称决定:
284+
285+
```
286+
out/<ir_dir>/<ir_stem>.<template_stem>
287+
```
288+
289+
示例:
290+
- IR 文件:`src/foundation/geometry.h` + 模板:`file/bindings.j2``out/src/foundation/geometry.bindings`
291+
- IR 文件:`src/window.h` + 模板:`file/dart.j2``out/src/window.dart`
292+
- IR 文件:`src/menu.h` + 模板:`file/rs.j2``out/src/menu.rs`
293+
243294
**优点**
244295

245296
- 新增语言只需新增模板 + 映射配置
246297
- 语言差异集中可见,维护成本低
247298
- 测试可按模板/映射分层
299+
- 支持一比一文件映射,保持源码结构
248300

249301
### Templates (External)
250302

tools/bindgen/cli.py

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
from .codegen.generator import generate_bindings
55
from .config import load_config
6-
from .ir.serializer import dump_ir_json
6+
from .ir.serializer import dump_ir_json, load_ir_json
77
from .normalizer import normalize_translation_unit
88
from .parser import parse_headers
99

@@ -12,15 +12,21 @@ def main(argv=None) -> int:
1212
parser = argparse.ArgumentParser(prog="bindgen")
1313
parser.add_argument("--config", required=True, help="Path to config.yaml")
1414
parser.add_argument("--out", required=True, help="Output directory")
15+
parser.add_argument("--ir", help="Load IR from existing JSON file (skips parsing)")
1516
parser.add_argument("--dump-ir", help="Write IR JSON to path")
1617

1718
args = parser.parse_args(argv)
1819

1920
config_path = Path(args.config)
2021
cfg = load_config(config_path)
2122

22-
tu = parse_headers(cfg)
23-
module = normalize_translation_unit(tu, cfg)
23+
if args.ir:
24+
# Load from existing IR JSON
25+
module = load_ir_json(Path(args.ir))
26+
else:
27+
# Parse headers and normalize
28+
tu = parse_headers(cfg)
29+
module = normalize_translation_unit(tu, cfg)
2430

2531
if args.dump_ir:
2632
dump_ir_json(module, Path(args.dump_ir))

tools/bindgen/codegen/generator.py

Lines changed: 195 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,126 @@
11
from __future__ import annotations
22

33
from pathlib import Path
4+
from typing import Any, Dict, List
45

56
from ..config import BindgenConfig
6-
from ..ir.model import IRModule
7+
from ..ir.model import IRFile, IRModule
78
from .context import build_context
89

910

1011
def _load_jinja():
1112
try:
12-
from jinja2 import Environment, FileSystemLoader # type: ignore
13+
from jinja2 import BaseLoader, Environment, FileSystemLoader, TemplateNotFound
1314
except Exception as exc: # pragma: no cover
1415
raise RuntimeError(
1516
"jinja2 is required. Install with `pip install jinja2`."
1617
) from exc
17-
return Environment, FileSystemLoader
18+
return Environment, FileSystemLoader, BaseLoader, TemplateNotFound
19+
20+
21+
def _build_file_context(
22+
file_path: str,
23+
ir_file: IRFile,
24+
global_context: Dict[str, Any],
25+
) -> Dict[str, Any]:
26+
"""Build context for a single file template."""
27+
from pathlib import PurePosixPath
28+
29+
p = PurePosixPath(file_path)
30+
31+
return {
32+
# Global context
33+
"module": global_context["module"],
34+
"files": global_context["files"],
35+
"file_paths": global_context["file_paths"],
36+
"mapping": global_context["mapping"],
37+
# All items (for cross-referencing)
38+
"all_types": global_context["types"],
39+
"all_enums": global_context["enums"],
40+
"all_functions": global_context["functions"],
41+
"all_classes": global_context["classes"],
42+
"all_constants": global_context["constants"],
43+
"all_aliases": global_context["aliases"],
44+
# Current file items
45+
"file": ir_file,
46+
"file_path": file_path,
47+
"file_dir": str(p.parent) if p.parent != PurePosixPath(".") else "",
48+
"file_name": p.name,
49+
"file_stem": p.stem,
50+
"file_ext": p.suffix,
51+
"types": ir_file.types,
52+
"enums": ir_file.enums,
53+
"functions": ir_file.functions,
54+
"classes": ir_file.classes,
55+
"constants": ir_file.constants,
56+
"aliases": ir_file.aliases,
57+
# Convenience flags
58+
"is_empty": (
59+
not ir_file.types
60+
and not ir_file.enums
61+
and not ir_file.functions
62+
and not ir_file.classes
63+
and not ir_file.constants
64+
and not ir_file.aliases
65+
),
66+
"has_types": bool(ir_file.types),
67+
"has_enums": bool(ir_file.enums),
68+
"has_functions": bool(ir_file.functions),
69+
"has_classes": bool(ir_file.classes),
70+
"has_constants": bool(ir_file.constants),
71+
"has_aliases": bool(ir_file.aliases),
72+
}
73+
74+
75+
def _register_partials(env, partials_dir: Path) -> None:
76+
"""Register partial templates as macros that can be included."""
77+
if not partials_dir.exists():
78+
return
79+
80+
# Partials are available via {% include 'partials/xxx.j2' %}
81+
# No special registration needed since we use FileSystemLoader
82+
83+
84+
def _compute_output_path(
85+
ir_file_path: str,
86+
template_name: str,
87+
out_dir: Path,
88+
) -> Path:
89+
"""
90+
Compute output path from IR file path and template name.
91+
92+
Rules:
93+
- IR path: src/foundation/geometry.h
94+
- Template: file/dart.j2 -> out/src/foundation/geometry.dart
95+
- Template: file/rs.j2 -> out/src/foundation/geometry.rs
96+
"""
97+
from pathlib import PurePosixPath
98+
99+
ir_path = PurePosixPath(ir_file_path)
100+
# Template stem becomes the new extension
101+
template_stem = Path(template_name).stem # e.g., "dart" from "dart.j2"
102+
103+
# Build output path: out_dir / ir_dir / ir_stem.template_stem
104+
output_name = f"{ir_path.stem}.{template_stem}"
105+
output_path = out_dir / ir_path.parent / output_name
106+
107+
return output_path
18108

19109

20110
def generate_bindings(
21111
module: IRModule, cfg: BindgenConfig, out_dir: Path, config_path: Path
22112
) -> None:
113+
"""
114+
Generate bindings using templates.
115+
116+
Template directory structure:
117+
- template/*.j2 - Global templates (rendered once)
118+
- template/file/*.j2 - Per-file templates (rendered for each IR file)
119+
- template/partials/*.j2 - Partial templates (type.j2, enum.j2, function.j2, etc.)
120+
121+
Partials can be included in file templates:
122+
{% include 'partials/type.j2' %}
123+
"""
23124
# Templates are discovered from config.yaml sibling directory:
24125
# <config_dir>/template/*.j2
25126
config_template_root = config_path.resolve().parent / "template"
@@ -29,14 +130,102 @@ def generate_bindings(
29130
mapping = dict(cfg.mapping)
30131
mapping.setdefault("language", "default")
31132

32-
Environment, FileSystemLoader = _load_jinja()
133+
Environment, FileSystemLoader, BaseLoader, TemplateNotFound = _load_jinja()
33134
env = Environment(
34135
loader=FileSystemLoader(str(config_template_root)), autoescape=False
35136
)
36-
context = build_context(module, mapping)
37137

138+
# Register any custom filters here if needed
139+
_register_filters(env)
140+
141+
global_context = build_context(module, mapping)
142+
143+
# 1. Render global templates (template/*.j2)
38144
for template_path in config_template_root.glob("*.j2"):
39145
template = env.get_template(template_path.name)
40146
output_name = template_path.stem
41-
rendered = template.render(**context)
147+
rendered = template.render(**global_context)
42148
(out_dir / output_name).write_text(rendered + "\n", encoding="utf-8")
149+
150+
# 2. Render per-file templates (template/file/*.j2)
151+
file_template_dir = config_template_root / "file"
152+
if file_template_dir.exists():
153+
file_templates = list(file_template_dir.glob("*.j2"))
154+
155+
for ir_file_path in global_context["file_paths"]:
156+
ir_file = global_context["files"][ir_file_path]
157+
file_context = _build_file_context(ir_file_path, ir_file, global_context)
158+
159+
for template_path in file_templates:
160+
template = env.get_template(f"file/{template_path.name}")
161+
output_path = _compute_output_path(
162+
ir_file_path, template_path.name, out_dir
163+
)
164+
165+
# Create output directory if needed
166+
output_path.parent.mkdir(parents=True, exist_ok=True)
167+
168+
rendered = template.render(**file_context)
169+
output_path.write_text(rendered + "\n", encoding="utf-8")
170+
171+
172+
def _register_filters(env) -> None:
173+
"""Register custom Jinja2 filters."""
174+
import re
175+
176+
def snake_case(s: str) -> str:
177+
"""Convert to snake_case."""
178+
s = re.sub(r"([A-Z]+)([A-Z][a-z])", r"\1_\2", s)
179+
s = re.sub(r"([a-z\d])([A-Z])", r"\1_\2", s)
180+
return s.lower().replace("-", "_")
181+
182+
def camel_case(s: str) -> str:
183+
"""Convert to camelCase."""
184+
parts = re.split(r"[_\-\s]+", s)
185+
if not parts:
186+
return s
187+
return parts[0].lower() + "".join(p.title() for p in parts[1:])
188+
189+
def pascal_case(s: str) -> str:
190+
"""Convert to PascalCase."""
191+
parts = re.split(r"[_\-\s]+", s)
192+
return "".join(p.title() for p in parts)
193+
194+
def screaming_snake_case(s: str) -> str:
195+
"""Convert to SCREAMING_SNAKE_CASE."""
196+
return snake_case(s).upper()
197+
198+
def kebab_case(s: str) -> str:
199+
"""Convert to kebab-case."""
200+
return snake_case(s).replace("_", "-")
201+
202+
def strip_prefix(s: str, prefix: str) -> str:
203+
"""Strip prefix from string."""
204+
if s.startswith(prefix):
205+
return s[len(prefix) :]
206+
return s
207+
208+
def strip_suffix(s: str, suffix: str) -> str:
209+
"""Strip suffix from string."""
210+
if s.endswith(suffix):
211+
return s[: -len(suffix)]
212+
return s
213+
214+
def add_prefix(s: str, prefix: str) -> str:
215+
"""Add prefix to string."""
216+
return prefix + s
217+
218+
def add_suffix(s: str, suffix: str) -> str:
219+
"""Add suffix to string."""
220+
return s + suffix
221+
222+
# Register all filters
223+
env.filters["snake_case"] = snake_case
224+
env.filters["camel_case"] = camel_case
225+
env.filters["pascal_case"] = pascal_case
226+
env.filters["screaming_snake_case"] = screaming_snake_case
227+
env.filters["kebab_case"] = kebab_case
228+
env.filters["strip_prefix"] = strip_prefix
229+
env.filters["strip_suffix"] = strip_suffix
230+
env.filters["add_prefix"] = add_prefix
231+
env.filters["add_suffix"] = add_suffix

0 commit comments

Comments
 (0)