diff --git a/.github/workflows/runtime-scripts.yml b/.github/workflows/runtime-scripts.yml new file mode 100644 index 000000000..9471c10c9 --- /dev/null +++ b/.github/workflows/runtime-scripts.yml @@ -0,0 +1,24 @@ +name: Runtime Scripts + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +jobs: + build: + runs-on: ubuntu-latest + defaults: + run: + working-directory: datascripts + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0 + with: + bun-version: 1.3.7 + - run: bun install --frozen-lockfile + - name: Typecheck and verify Runtime Script output + run: bun test tests/runtime-scripts.test.ts diff --git a/.gitignore b/.gitignore index c8c11aeef..233469761 100644 --- a/.gitignore +++ b/.gitignore @@ -53,16 +53,17 @@ __pycache__/ # never client bytes, but it is the operator's own derived data and stays on the operator's machine. /datascripts/generated/ /datascripts/node_modules/ +/datascripts/runtime-scripts/node_modules/ # Packages installed on a host (`lyracore packages add`) are local to that checkout. Unknown # Package directories stay out of Git status. Source files in the committed first-party Packages -# remain visible; a first-party Package that ever ships a Script Artifact gets its own negation -# under data/.generated/ then. None does today. +# remain visible. /packages/* !/packages/example/ !/packages/fire_nova/ -# A generated Package Delta and its Build Identity sidecar, even inside an allowlisted first-party -# Package above: `packages build` reproduces both from source on every run, and a Package Delta is -# never committed — the same rule `datascripts/generated/` follows above. +# What `lyracore packages build` writes into a first-party Package: Package Deltas, Script Artifacts +# and their Build Identity sidecars. Here they are output the sources next to them reproduce, so +# none is committed. A Package Delta is never committed anywhere (licensing firewall); the Official +# Package Collection does commit Script Artifacts, so an Operator installs prebuilt Lua without Bun. /packages/*/data/.generated/ diff --git a/CONTEXT.md b/CONTEXT.md index b879edacc..d88ab828e 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -494,11 +494,13 @@ records where a Package was installed from. _Avoid_: apply log, history, audit trail **Build Identity**: -The recorded inputs `packages build` writes next to a Package's generated artifact, in a sibling -file rather than inside it: hashes of the Package's Datascript source tree, the generated Module -typings, the Base Snapshot, the authoring library, and the pinned Datascript toolchain files, plus -the pinned Bun version and the artifact's own hash. `lyracore packages check` and preflight -recompute it against the checkout on disk and refuse, naming the input that changed. +The recorded inputs `packages build` writes next to a source-built artifact, in a sibling file +rather than inside it. A Package Delta records its Datascript source tree, generated Module +typings, Base Snapshot, authoring library, and pinned Datascript toolchain. A source-built Script +Artifact records its `scripts/` sources, Runtime Script Toolchain, Bun pin, and artifact hash. A +source-free prebuilt Script Artifact has no local author inputs or Build Identity; the authoritative +checker still parses and traces it. `lyracore packages check` and preflight recompute a present +identity against the checkout on disk and refuse, naming the input that changed. _Avoid_: identity file, build fingerprint, artifact metadata **Claim**: @@ -553,6 +555,20 @@ event: lower priority runs first and the script identifier breaks a tie, so ever plan in one order. _Avoid_: hook registration, subscription, listener +**Script Directive**: +A `@key value` comment line at the top of a Runtime Script source, declaring what the file cannot +say in its own code: `@event` and `@id` are required, `@priority` and `@enabled` have defaults. The +identifier is written down rather than derived, because it is durable — deriving it from a file +index would renumber a Package's scripts the moment an author added one. +_Avoid_: annotation, frontmatter, pragma, metadata header + +**Runtime Script Toolchain**: +The pinned compiler that turns a Package's `scripts/` sources into its Script Artifact: Bun plus +`typescript-to-lua`, its config, the hand-maintained Host API typings, and the emitter that keeps +generated Lua off the interpreter's known call-shape fault. It lives in +`datascripts/runtime-scripts/` and runs at author time only; an Operator installs the prebuilt Lua. +_Avoid_: transpiler, build pipeline, SDK + **Package Item Range**: The items family's Package Identifier Range: 7,000,000 to 7,999,999. Above every reserved band, and one whole decade above the Package Spell Range so the millions column stays a family-at-a-glance diff --git a/crates/lyracore-package-delta/src/bin/lyracore-delta-check.rs b/crates/lyracore-package-delta/src/bin/lyracore-delta-check.rs index c471165bd..b502414a4 100644 --- a/crates/lyracore-package-delta/src/bin/lyracore-delta-check.rs +++ b/crates/lyracore-package-delta/src/bin/lyracore-delta-check.rs @@ -1,14 +1,19 @@ -//! `lyracore-delta-check ...` — parse a set of Package Delta artifacts, trace them -//! together, and print the plan they form. +//! `lyracore-delta-check ...` — parse a set of generated Package artifacts, trace +//! them together, and print the plan they form. +//! +//! Two artifact kinds arrive, because a Package ships every kind it has into one generated +//! directory: Package Deltas and Script Artifacts. Each file is routed on its own `kind` member and +//! traced with its own kind, and both traces have to come out clear. //! //! This is the check a Datascript author gets after a build, before anything reaches a shard. It //! needs no client data, no database and no realm: it reads the named files and nothing else, which //! is why it can run on any machine in under a second. //! -//! Exit status is the whole answer: 0 when every artifact parses and the claims merge cleanly, -//! 1 when an artifact is refused or two Packages claim the same column of one row. The output names -//! the file and the exact claim in both cases, because the reader is the author who has to fix one -//! line of a Datascript. +//! Exit status is the whole answer: 0 when every artifact parses and the Packages agree, 1 when an +//! artifact is refused, two Packages claim the same column of one row, or two Packages claim one +//! Package twice, or two Packages claim one Runtime Script identity. The output names the file and +//! the exact conflict in every case, +//! because the reader is the author who has to fix one line of a Datascript or one script file. //! //! Trace ALL of a realm's enabled artifacts in one invocation. Conflicts exist BETWEEN Packages, so //! checking one file at a time can only ever prove that one file parses. @@ -19,7 +24,10 @@ use std::path::Path; use std::process::ExitCode; -use lyracore_package_delta::{trace, ClaimTrace, Operation, PackageDelta, HOOK_EVENT_NAMES}; +use lyracore_package_delta::{ + artifact_kind, trace, trace_scripts, ArtifactKind, ClaimTrace, Operation, PackageDelta, + ScriptArtifact, ScriptTrace, HOOK_EVENT_NAMES, +}; fn main() -> ExitCode { let args: Vec = std::env::args().skip(1).collect(); @@ -31,20 +39,31 @@ fn main() -> ExitCode { if paths.is_empty() || paths.iter().any(|a| a == "-h" || a == "--help") { eprintln!( "usage: lyracore-delta-check ...\n\n\ - Parses each Package Delta artifact, traces them together, and prints the plan.\n\ - Exits non-zero on a refused artifact or a Claim Conflict. Reads no client data and \ - no database.\n\n\ - Name every enabled Package's artifacts in ONE run: a Claim Conflict is between \ + Parses each generated Package artifact — a Package Delta or a Script Artifact — \ + traces each kind together, and prints the plan.\n\ + Exits non-zero on a refused artifact, a Claim Conflict or a Script Artifact \ + conflict. Reads no client data and no database.\n\n\ + Name every enabled Package's artifacts in ONE run: a conflict is between \ Packages, so one file alone cannot show one.\n\n\ lyracore-delta-check --print-events prints the Event Binding catalogue instead." ); return ExitCode::FAILURE; } - let mut deltas = Vec::with_capacity(paths.len()); + let mut deltas: Vec = Vec::new(); + let mut delta_paths: Vec<&String> = Vec::new(); + let mut scripts: Vec = Vec::new(); + let mut script_paths: Vec<&String> = Vec::new(); for path in &paths { match read_artifact(Path::new(path)) { - Ok(delta) => deltas.push(delta), + Ok(Artifact::Delta(delta)) => { + deltas.push(delta); + delta_paths.push(path); + } + Ok(Artifact::Script(artifact)) => { + scripts.push(artifact); + script_paths.push(path); + } Err(refusal) => { eprintln!("{refusal}"); return ExitCode::FAILURE; @@ -52,23 +71,48 @@ fn main() -> ExitCode { } } - let traced = trace(&deltas); - print!("{}", report(&paths, &deltas, &traced)); + let claim_trace = trace(&deltas); + let script_trace = trace_scripts(&scripts); + print!( + "{}", + report( + &delta_paths, + &deltas, + &claim_trace, + &script_paths, + &scripts, + &script_trace + ) + ); - if traced.is_clear() { + if claim_trace.is_clear() && script_trace.is_clear() { eprintln!( - "{} artifact(s) check out over {} row(s).", - deltas.len(), - traced.rows().len() + "{} artifact(s) check out over {} row(s) and {} Runtime Script(s).", + deltas.len() + scripts.len(), + claim_trace.rows().len(), + script_trace.scripts().len() ); - ExitCode::SUCCESS - } else { + return ExitCode::SUCCESS; + } + if !claim_trace.is_clear() { eprintln!( "{} claim conflict(s) between the named Packages — a human chooses.", - traced.conflicts().len() + claim_trace.conflicts().len() ); - ExitCode::FAILURE } + if !script_trace.is_clear() { + eprintln!( + "{} Script Artifact conflict(s) between the named Packages; a human chooses.", + script_trace.conflicts().len() + ); + } + ExitCode::FAILURE +} + +/// One generated artifact, already routed to the parser its own `kind` chose. +enum Artifact { + Delta(PackageDelta), + Script(ScriptArtifact), } /// Prints [`HOOK_EVENT_NAMES`], one event per line, in catalogue order, and exits. @@ -95,19 +139,37 @@ fn print_events(args: &[String]) -> ExitCode { ExitCode::SUCCESS } +/// Read one file and hand it to the parser its own `kind` names. +/// +/// Routing on `kind` rather than on the file name: a Package chooses its own artifact file names, +/// and the member inside the file is the only thing that says what it holds. A kind this build does +/// not read reaches [`PackageDelta::parse`], which refuses it by name. +/// /// A refusal names the FILE, so an author with several Packages open knows which one to fix. -fn read_artifact(path: &Path) -> Result { +fn read_artifact(path: &Path) -> Result { let json = std::fs::read_to_string(path) .map_err(|e| format!("{}: cannot read: {e}", path.display()))?; - PackageDelta::parse(&json).map_err(|e| format!("{}: {e}", path.display())) + let refuse = |e| format!("{}: {e}", path.display()); + match artifact_kind(&json) { + Some(ArtifactKind::Script) => ScriptArtifact::parse(&json).map(Artifact::Script), + _ => PackageDelta::parse(&json).map(Artifact::Delta), + } + .map_err(refuse) } /// The plan: every Package with its file and row counts, then every claimed row, then every /// conflict. The same shape the importer's Package Delta stage prints, so an author and an operator /// read the same report. -fn report(paths: &[String], deltas: &[PackageDelta], traced: &ClaimTrace) -> String { +fn report( + delta_paths: &[&String], + deltas: &[PackageDelta], + traced: &ClaimTrace, + script_paths: &[&String], + scripts: &[ScriptArtifact], + script_trace: &ScriptTrace, +) -> String { let mut out = String::from("\n=== Package Deltas ===\n"); - for (path, delta) in paths.iter().zip(deltas) { + for (path, delta) in delta_paths.iter().zip(deltas) { let counts = delta.claim_counts(); out.push_str(&format!( " {:<32} {:>3} updated {:>3} spells {:>3} effects {path}\n", @@ -139,6 +201,29 @@ fn report(paths: &[String], deltas: &[PackageDelta], traced: &ClaimTrace) -> Str for conflict in traced.conflicts() { out.push_str(&format!(" CONFLICT {conflict}\n")); } + + out.push_str("=== Runtime Scripts ===\n"); + for (path, artifact) in script_paths.iter().zip(scripts) { + out.push_str(&format!( + " {:<32} {:>3} script(s) {path}\n", + artifact.package(), + artifact.scripts().len() + )); + } + for traced in script_trace.scripts() { + let script = &traced.script; + out.push_str(&format!( + " {:>6} {:<32} {} priority {}{}\n", + script.script_id(), + script.name(), + script.event(), + script.priority(), + if script.enabled() { "" } else { " (disabled)" }, + )); + } + for conflict in script_trace.conflicts() { + out.push_str(&format!(" CONFLICT {conflict}\n")); + } out.push_str("======================\n\n"); out } diff --git a/crates/lyracore-package-delta/src/script.rs b/crates/lyracore-package-delta/src/script.rs index 4e918dfe0..1a7779906 100644 --- a/crates/lyracore-package-delta/src/script.rs +++ b/crates/lyracore-package-delta/src/script.rs @@ -53,9 +53,9 @@ //! //! # What it traces //! -//! [`trace_scripts`] puts several Packages' artifacts together. Two Packages shipping one -//! `script_id`, or one name, is a [`ScriptConflict`] naming both Packages. There is no merge case -//! and there are no priority numbers to break the tie with: `priority` orders scripts at an event, +//! [`trace_scripts`] puts several Packages' artifacts together. A Package appearing twice, or two +//! Packages shipping one `script_id` or name, is a [`ScriptConflict`]. There is no merge case and +//! there are no priority numbers to break the tie with: `priority` orders scripts at an event, //! never Packages against each other. use core::fmt; @@ -601,9 +601,14 @@ pub struct TracedScript { pub source_hash: SourceHash, } -/// Two Packages claiming one Runtime Script identity. Only a human can settle it. +/// Script Artifacts that cannot belong to one plan. Only a human can settle the inventory. #[derive(Debug, Clone, PartialEq, Eq)] pub enum ScriptConflict { + /// One Package supplied more than one Script Artifact. + Package { + /// The Package named by both artifacts. + package: PackageId, + }, /// Both Packages ship a script at this identifier. Id { /// The contested identifier. @@ -627,6 +632,12 @@ pub enum ScriptConflict { impl fmt::Display for ScriptConflict { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { + Self::Package { package } => { + write!( + f, + "package `{package}` appears in more than one Script Artifact" + ) + } Self::Id { script_id, first, @@ -668,7 +679,7 @@ impl ScriptTrace { &self.conflicts } - /// True when no two Packages disagree, which is the only state an applier may write. + /// True when every Package supplied one artifact and no two Packages disagree. #[must_use] pub fn is_clear(&self) -> bool { self.conflicts.is_empty() @@ -677,8 +688,8 @@ impl ScriptTrace { /// Merge several Packages' Script Artifacts into one plan, reporting every collision. /// -/// A script is never merged with another: the whole row belongs to one Package. So the only thing -/// to trace is identity — an identifier or a name that two Packages both claim. +/// A script is never merged with another: the whole row belongs to one Package. The tracer first +/// requires one artifact per Package, then checks each identifier and name across Packages. /// /// A conflicting script is still carried into [`ScriptTrace::scripts`], from whichever Package /// shipped it first. That keeps the plan describable in a report; it is not applyable, because an @@ -686,7 +697,15 @@ impl ScriptTrace { #[must_use] pub fn trace_scripts(artifacts: &[ScriptArtifact]) -> ScriptTrace { let mut trace = ScriptTrace::default(); + let mut packages: Vec = Vec::new(); for artifact in artifacts { + if packages.contains(&artifact.package) { + trace.conflicts.push(ScriptConflict::Package { + package: artifact.package.clone(), + }); + continue; + } + packages.push(artifact.package.clone()); for script in &artifact.scripts { if let Some(seen) = trace .scripts diff --git a/crates/lyracore-package-delta/tests/delta_check_script_artifacts.rs b/crates/lyracore-package-delta/tests/delta_check_script_artifacts.rs new file mode 100644 index 000000000..f3bd3a03d --- /dev/null +++ b/crates/lyracore-package-delta/tests/delta_check_script_artifacts.rs @@ -0,0 +1,165 @@ +//! `lyracore-delta-check ...` over a MIXED set of artifacts. +//! +//! A Package ships every artifact kind it has into one generated directory, so the set +//! `lyracore packages build` hands this binary holds Package Deltas and Script Artifacts together. +//! Each file is routed on its own `kind` member, each kind is traced with its own tracer, and the +//! exit status covers both. +//! +//! These run the real binary: the routing lives in its argument handling, not in the library. + +use std::process::{Command, Output}; + +const HASH: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"; + +fn run(paths: &[&std::path::Path]) -> Output { + Command::new(env!("CARGO_BIN_EXE_lyracore-delta-check")) + .args(paths) + .output() + .expect("spawn lyracore-delta-check") +} + +fn write(dir: &std::path::Path, name: &str, json: &str) -> std::path::PathBuf { + let path = dir.join(name); + std::fs::write(&path, json).expect("write artifact"); + path +} + +fn script_artifact(package: &str, script_id: u32, name: &str, event: &str) -> String { + format!( + r#"{{"kind":"script","version":1,"package":"{package}","source_hash":"{HASH}","scripts":[ + {{"script_id":{script_id},"name":"{name}","event":"{event}","priority":0, + "enabled":true,"source":"return 1"}}]}}"# + ) +} + +fn delta_artifact(package: &str, spell_id: u32) -> String { + format!( + r#"{{"version":1,"package":"{package}","source_hash":"{HASH}","claims":[ + {{"table":"game_spell","key":{{"spell_id":{spell_id}}},"operation":"update", + "fields":{{"cooldown_ms":{{"type":"u32","value":1500}}}}}}]}}"# + ) +} + +fn temp_dir(name: &str) -> std::path::PathBuf { + let dir = std::env::temp_dir().join(format!("lyracore-delta-check-{name}")); + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).expect("create temp dir"); + dir +} + +#[test] +fn a_script_artifact_and_a_package_delta_check_out_together() { + let dir = temp_dir("mixed"); + let delta = write(&dir, "spell.json", &delta_artifact("example.bolt", 133)); + let script = write( + &dir, + "bolt.script.json", + &script_artifact("example.bolt", 100_001, "bolt.greet", "on_login"), + ); + + let output = run(&[&delta, &script]); + + assert!(output.status.success()); + let report = String::from_utf8(output.stdout).unwrap(); + assert!(report.contains("=== Package Deltas ==="), "{report}"); + assert!(report.contains("=== Runtime Scripts ==="), "{report}"); + assert!(report.contains("bolt.greet"), "{report}"); +} + +/// A Package Event of the artifact's OWN Package is bindable; nothing has to teach this binary +/// about it, because the parser already knows whose event it is. +#[test] +fn a_package_event_of_the_artifacts_own_package_checks_out() { + let dir = temp_dir("package-event"); + let script = write( + &dir, + "bots.script.json", + &script_artifact( + "playerbots", + 100_100, + "playerbots.flee", + "playerbots.flee_at", + ), + ); + + assert!(run(&[&script]).status.success()); +} + +#[test] +fn an_event_outside_the_catalogue_refuses_the_run() { + let dir = temp_dir("unknown-event"); + let script = write( + &dir, + "bolt.script.json", + &script_artifact("example.bolt", 100_001, "bolt.greet", "on_sneeze"), + ); + + let output = run(&[&script]); + + assert!(!output.status.success()); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("on_sneeze"), "{stderr}"); + assert!( + stderr.contains(script.to_string_lossy().as_ref()), + "{stderr}" + ); +} + +/// The collision the tracer exists for: two Packages claiming one Runtime Script identity. It has +/// to fail the run exactly the way a Claim Conflict does. +#[test] +fn two_packages_shipping_one_script_id_fail_the_run() { + let dir = temp_dir("collision"); + let first = write( + &dir, + "first.script.json", + &script_artifact("example.first", 100_001, "first.greet", "on_login"), + ); + let second = write( + &dir, + "second.script.json", + &script_artifact("example.second", 100_001, "second.greet", "on_login"), + ); + + let output = run(&[&first, &second]); + + assert!(!output.status.success()); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("conflict"), "{stderr}"); +} + +#[test] +fn two_artifacts_for_one_package_fail_even_when_the_scripts_do_not_collide() { + let dir = temp_dir("duplicate-package"); + let first = write( + &dir, + "first.script.json", + &script_artifact("example.same", 100_001, "same.greet", "on_login"), + ); + let second = write( + &dir, + "second.script.json", + &script_artifact("example.same", 100_002, "same.leave", "on_logout"), + ); + + let output = run(&[&first, &second]); + + assert!(!output.status.success()); + let stdout = String::from_utf8(output.stdout).unwrap(); + assert!(stdout.contains("example.same"), "{stdout}"); + assert!(stdout.contains("more than one Script Artifact"), "{stdout}"); +} + +/// A kind this build does not read still meets the Package Delta parser, which names it rather than +/// complaining about a missing `claims` member. +#[test] +fn an_artifact_of_an_unknown_kind_is_refused_by_name() { + let dir = temp_dir("unknown-kind"); + let path = write(&dir, "weather.json", r#"{"kind":"weather","version":1}"#); + + let output = run(&[&path]); + + assert!(!output.status.success()); + let stderr = String::from_utf8(output.stderr).unwrap(); + assert!(stderr.contains("weather"), "{stderr}"); +} diff --git a/crates/lyracore-package-delta/tests/script_conflicts.rs b/crates/lyracore-package-delta/tests/script_conflicts.rs index d3f5c7cb1..ba5bc7c56 100644 --- a/crates/lyracore-package-delta/tests/script_conflicts.rs +++ b/crates/lyracore-package-delta/tests/script_conflicts.rs @@ -124,6 +124,21 @@ fn two_packages_shipping_one_name_is_a_conflict_naming_both() { assert!(matches!(trace.conflicts()[0], ScriptConflict::Name { .. })); } +#[test] +fn one_package_cannot_supply_two_artifacts_even_when_the_scripts_do_not_collide() { + let trace = trace_scripts(&[ + artifact("example.alpha", &[(100_001, "alpha.greet", "on_login")]), + artifact("example.alpha", &[(100_002, "alpha.leave", "on_logout")]), + ]); + + assert!(!trace.is_clear()); + assert_eq!(trace.scripts().len(), 1); + assert!(matches!( + trace.conflicts(), + [ScriptConflict::Package { package }] if package.as_str() == "example.alpha" + )); +} + /// The operator fixing a broken Package Inventory wants the whole list in one pass, not one /// refusal per re-run. #[test] diff --git a/datascripts/bun.lock b/datascripts/bun.lock index 90dab0098..f20d9239f 100644 --- a/datascripts/bun.lock +++ b/datascripts/bun.lock @@ -13,12 +13,21 @@ "typescript": "7.0.2", }, }, + "runtime-scripts": { + "name": "lyracore-script-compiler", + "dependencies": { + "typescript": "6.0.2", + "typescript-to-lua": "1.37.1", + }, + }, }, "packages": { "@types/bun": ["@types/bun@1.3.7", "", { "dependencies": { "bun-types": "1.3.7" } }, "sha512-lmNuMda+Z9b7tmhA0tohwy8ZWFSnmQm1UDWXtH5r9F7wZCfkeO3Jx7wKQ1EOiKq43yHts7ky6r8SDJQWRNupkA=="], "@types/node": ["@types/node@24.13.3", "", { "dependencies": { "undici-types": "~7.18.0" } }, "sha512-Dh8vAsV36ig5wa9OX4pXvMc9D3Veibfw2wix0CUwYODLD8nkj9UsLjASr49nPg+2eKzxhBV+v7L8pXvT4e639Q=="], + "@typescript-to-lua/language-extensions": ["@typescript-to-lua/language-extensions@1.19.0", "", {}, "sha512-Os5wOKwviTD4LeqI29N0btYOjokSJ97iCf45EOjIABlb5IwNQy7AE/AqZJobRw3ywHH8+KzJUMkEirWPzh2tUA=="], + "@typescript/typescript-aix-ppc64": ["@typescript/typescript-aix-ppc64@7.0.2", "", { "os": "aix", "cpu": "ppc64" }, "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ=="], "@typescript/typescript-darwin-arm64": ["@typescript/typescript-darwin-arm64@7.0.2", "", { "os": "darwin", "cpu": "arm64" }, "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA=="], @@ -63,24 +72,54 @@ "bun-types": ["bun-types@1.3.7", "", { "dependencies": { "@types/node": "*" } }, "sha512-qyschsA03Qz+gou+apt6HNl6HnI+sJJLL4wLDke4iugsE6584CMupOtTY1n+2YC9nGVrEKUlTs99jjRLKgWnjQ=="], + "enhanced-resolve": ["enhanced-resolve@5.8.2", "", { "dependencies": { "graceful-fs": "^4.2.4", "tapable": "^2.2.0" } }, "sha512-F27oB3WuHDzvR2DOGNTaYy0D5o0cnrv8TeI482VM4kYgQd/FT9lUQwuNsJ0oOHtBUq7eiW5ytqzp7nBFknL+GA=="], + + "es-errors": ["es-errors@1.3.0", "", {}, "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw=="], + + "function-bind": ["function-bind@1.1.2", "", {}, "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA=="], + + "graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="], + + "hasown": ["hasown@2.0.4", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A=="], + "headers-polyfill": ["headers-polyfill@4.0.3", "", {}, "sha512-IScLbePpkvO846sIwOtOTDjutRMWdXdJmXdMvk6gCBHxFO8d+QKOQedyZSxFTTFYRSmlgSTDtXqqq4pcenBXLQ=="], + "is-core-module": ["is-core-module@2.16.2", "", { "dependencies": { "hasown": "^2.0.3" } }, "sha512-evOr8xfXKxE6qSR0hSXL2r3sd7ALj8+7jQEUvPYcm5sgZFdJ+AYzT6yNmJenvIYQBgIGwfwz08sL8zoL7yq2BA=="], + + "lyracore-script-compiler": ["lyracore-script-compiler@workspace:runtime-scripts"], + "object-inspect": ["object-inspect@1.13.4", "", {}, "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew=="], + "path-parse": ["path-parse@1.0.7", "", {}, "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw=="], + + "picomatch": ["picomatch@2.3.2", "", {}, "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA=="], + "prettier": ["prettier@3.9.6", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g=="], "pure-rand": ["pure-rand@7.0.1", "", {}, "sha512-oTUZM/NAZS8p7ANR3SHh30kXB+zK2r2BPcEn/awJIbOvq82WoMN4p62AWWp3Hhw50G0xMsw1mhIBLqHw64EcNQ=="], + "resolve": ["resolve@1.22.12", "", { "dependencies": { "es-errors": "^1.3.0", "is-core-module": "^2.16.1", "path-parse": "^1.0.7", "supports-preserve-symlinks-flag": "^1.0.0" }, "bin": { "resolve": "bin/resolve" } }, "sha512-TyeJ1zif53BPfHootBGwPRYT1RUt6oGWsaQr8UyZW/eAm9bKoijtvruSDEmZHm92CwS9nj7/fWttqPCgzep8CA=="], + "safe-stable-stringify": ["safe-stable-stringify@2.5.0", "", {}, "sha512-b3rppTKm9T+PsVCBEOUR46GWI7fdOs00VKZ1+9c1EWDaDMvjQc6tUwuFyIprgGgTcWoVHSKrU8H31ZHA2e0RHA=="], + "source-map": ["source-map@0.7.6", "", {}, "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ=="], + "spacetimedb": ["spacetimedb@2.7.1", "", { "dependencies": { "base64-js": "^1.5.1", "headers-polyfill": "^4.0.3", "object-inspect": "^1.13.4", "prettier": "^3.3.3", "pure-rand": "^7.0.1", "safe-stable-stringify": "^2.5.0", "statuses": "^2.0.2", "url-polyfill": "^1.1.14" }, "peerDependencies": { "@angular/core": ">=17.0.0", "@tanstack/react-query": "^5.0.0", "react": "^18.0.0 || ^19.0.0-0 || ^19.0.0", "solid-js": "^1.6.0", "svelte": "^4.0.0 || ^5.0.0", "undici": "^6.19.2", "vue": "^3.3.0" }, "optionalPeers": ["@angular/core", "@tanstack/react-query", "react", "solid-js", "svelte", "undici", "vue"] }, "sha512-BuND5Q056fGe4YDzxOhTSdTnxb8lXJKTW0F5gscMR/GXRf+AoSIGfb9K811AhH0zklDBXCMfAwdgeL599Z8Ozg=="], "statuses": ["statuses@2.0.2", "", {}, "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw=="], + "supports-preserve-symlinks-flag": ["supports-preserve-symlinks-flag@1.0.0", "", {}, "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w=="], + + "tapable": ["tapable@2.3.3", "", {}, "sha512-uxc/zpqFg6x7C8vOE7lh6Lbda8eEL9zmVm/PLeTPBRhh1xCgdWaQ+J1CUieGpIfm2HdtsUpRv+HshiasBMcc6A=="], + "typescript": ["typescript@7.0.2", "", { "optionalDependencies": { "@typescript/typescript-aix-ppc64": "7.0.2", "@typescript/typescript-darwin-arm64": "7.0.2", "@typescript/typescript-darwin-x64": "7.0.2", "@typescript/typescript-freebsd-arm64": "7.0.2", "@typescript/typescript-freebsd-x64": "7.0.2", "@typescript/typescript-linux-arm": "7.0.2", "@typescript/typescript-linux-arm64": "7.0.2", "@typescript/typescript-linux-loong64": "7.0.2", "@typescript/typescript-linux-mips64el": "7.0.2", "@typescript/typescript-linux-ppc64": "7.0.2", "@typescript/typescript-linux-riscv64": "7.0.2", "@typescript/typescript-linux-s390x": "7.0.2", "@typescript/typescript-linux-x64": "7.0.2", "@typescript/typescript-netbsd-arm64": "7.0.2", "@typescript/typescript-netbsd-x64": "7.0.2", "@typescript/typescript-openbsd-arm64": "7.0.2", "@typescript/typescript-openbsd-x64": "7.0.2", "@typescript/typescript-sunos-x64": "7.0.2", "@typescript/typescript-win32-arm64": "7.0.2", "@typescript/typescript-win32-x64": "7.0.2" }, "bin": { "tsc": "bin/tsc" } }, "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA=="], + "typescript-to-lua": ["typescript-to-lua@1.37.1", "", { "dependencies": { "@typescript-to-lua/language-extensions": "1.19.0", "enhanced-resolve": "5.8.2", "picomatch": "^2.3.1", "resolve": "^1.15.1", "source-map": "^0.7.3" }, "peerDependencies": { "typescript": "6.0.2" }, "bin": { "tstl": "dist/tstl.js" } }, "sha512-iOgG7UcfBxPzr2E/pNfamz2IEEOqub6nyivDoj2eDV5XRtKrHroCcR0uitvVzbRMUAITP71PHMJS3bNKVWG42g=="], + "undici-types": ["undici-types@7.18.2", "", {}, "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w=="], "url-polyfill": ["url-polyfill@1.1.14", "", {}, "sha512-p4f3TTAG6ADVF3mwbXw7hGw+QJyw5CnNGvYh5fCuQQZIiuKUswqcznyV3pGDP9j0TSmC4UvRKm8kl1QsX1diiQ=="], + + "lyracore-script-compiler/typescript": ["typescript@6.0.2", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-bGdAIrZ0wiGDo5l8c++HWtbaNCWTS4UTv7RaTH/ThVIgjkveJt83m74bBHMJkuCbslY8ixgLBVZJIOiQlQTjfQ=="], } } diff --git a/datascripts/package.json b/datascripts/package.json index 6f2cb5f3f..cc987de1b 100644 --- a/datascripts/package.json +++ b/datascripts/package.json @@ -2,6 +2,9 @@ "name": "lyracore-datascripts", "private": true, "type": "module", + "workspaces": [ + "runtime-scripts" + ], "scripts": { "test": "bun test", "typecheck": "tsc --noEmit" diff --git a/datascripts/runtime-scripts/build-scripts.ts b/datascripts/runtime-scripts/build-scripts.ts new file mode 100644 index 000000000..ae8fa9a62 --- /dev/null +++ b/datascripts/runtime-scripts/build-scripts.ts @@ -0,0 +1,370 @@ +// Compile one Package's Runtime Scripts into its Script Artifact. +// +// bun run datascripts/runtime-scripts/build-scripts.ts +// +// `lyracore packages build` runs this once per enabled Package that carries a `scripts/` folder, +// the same way it runs a Datascript: one Bun subprocess, one artifact written, non-zero on any +// refusal and nothing written. Run it by hand the same way. +// +// # What a Package ships +// +// `packages//scripts/` holds the sources. A `.ts` file is compiled by the pinned +// `typescript-to-lua` under `tsconfig.json` next to this file; a `.lua` file is shipped unchanged, +// for the author who would rather write Lua. Both carry the same directive header: +// +// // @event on_login -- @event on_login +// // @id 100200 -- @id 100201 +// // @priority 10 -- @priority 10 +// // @enabled false -- @enabled false +// +// `@event` and `@id` are required; `@priority` defaults to 0 and `@enabled` to true. The directives +// stop at the first line that is neither blank nor a comment, so ordinary comments below them are +// just comments. +// +// # Why the identifier is written down +// +// A `script_id` is DURABLE: it is what `game_script` keys on and what a Package's own rows will +// point at. Deriving it from a sorted file index would renumber every later script the moment an +// author adds one alphabetically earlier, and two Packages would both start at the bottom of the +// band. So the author writes it, once, and a collision is a refusal that names both Packages +// instead of a silent renumber. The band is 100000..=999999. +// +// The script NAME is not written down: it is `.`, which is the one part of the +// identity that has an obviously right answer. +// +// # What checks what +// +// This builder refuses what it can name a FILE and a directive for. `lyracore-delta-check` is the +// authority on the artifact itself and runs afterwards over every Package at once, because a +// collision between Packages cannot be seen from inside one. The event catalogue arrives through +// `LYRACORE_HOOK_EVENTS` (the CLI fills it from `lyracore-delta-check --print-events`); without it +// this builder does not check event names and says so, and the validator still refuses an unknown +// one. + +import { createHash } from "node:crypto"; +import { existsSync, mkdtempSync, readdirSync, rmSync } from "node:fs"; +import { mkdir, mkdtemp, open, readFile, rename, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { basename, dirname, join, resolve } from "node:path"; + +/// Where a Package keeps its Runtime Script sources, relative to the Package folder. +const SCRIPTS_DIR = "scripts"; + +/// Where a Package's generated artifacts live, relative to the Package folder. +const GENERATED_DIR = "data/.generated"; + +/// The Package script band. Mirrors `is_package_script_id` in `lyracore-package-delta`. +const SCRIPT_ID_FLOOR = 100_000; +const SCRIPT_ID_CEIL = 999_999; + +/// One script, as its directives and its Lua describe it. +interface RuntimeScript { + readonly script_id: number; + readonly name: string; + readonly event: string; + readonly priority: number; + readonly enabled: boolean; + readonly source: string; +} + +/// A refusal an author can act on: it always names the file. +class ScriptRefusal extends Error {} + +function refuse(file: string, what: string): never { + throw new ScriptRefusal(`${file}: ${what}`); +} + +// ---- directives ---- + +/// The `@key value` lines at the top of a script file, before the first line that is neither blank +/// nor a comment. Both comment markers are accepted so one reader serves `.ts` and `.lua`. +function readDirectives(file: string, source: string): Map { + const directives = new Map(); + for (const line of source.split("\n")) { + const text = line.trim(); + if (text.length === 0) continue; + const comment = text.startsWith("//") ? text.slice(2) : text.startsWith("--") ? text.slice(2) : undefined; + if (comment === undefined) break; + const match = /^\s*@([a-z]+)\s+(\S+)\s*$/.exec(comment); + if (!match) continue; + const [, key, value] = match as unknown as [string, string, string]; + if (directives.has(key)) refuse(file, `\`@${key}\` is declared twice`); + directives.set(key, value); + } + return directives; +} + +function requiredDirective(file: string, directives: Map, key: string): string { + const value = directives.get(key); + if (value === undefined) { + refuse(file, `no \`@${key}\` directive. Every Runtime Script declares \`@event\` and \`@id\`.`); + } + return value; +} + +function scriptId(file: string, directives: Map): number { + const raw = requiredDirective(file, directives, "id"); + const id = Number(raw); + if (!Number.isInteger(id) || id < SCRIPT_ID_FLOOR || id > SCRIPT_ID_CEIL) { + refuse(file, `\`@id ${raw}\` is not a whole number in ${SCRIPT_ID_FLOOR}..=${SCRIPT_ID_CEIL}, the Package script band`); + } + return id; +} + +function priority(file: string, directives: Map): number { + const raw = directives.get("priority"); + if (raw === undefined) return 0; + const value = Number(raw); + if (!Number.isInteger(value)) refuse(file, `\`@priority ${raw}\` is not a whole number`); + return value; +} + +function enabled(file: string, directives: Map): boolean { + const raw = directives.get("enabled"); + if (raw === undefined) return true; + if (raw === "true") return true; + if (raw === "false") return false; + return refuse(file, `\`@enabled ${raw}\` is neither \`true\` nor \`false\``); +} + +/// The catalogue the Module dispatches, or `undefined` when the caller did not supply one. +function hookEvents(): Set | undefined { + const raw = process.env.LYRACORE_HOOK_EVENTS; + if (raw === undefined) return undefined; + return new Set(raw.split("\n").map((name) => name.trim()).filter((name) => name.length > 0)); +} + +/// A core event from the catalogue, or a Package Event this Package fires itself. +function checkEvent(file: string, event: string, packageName: string, catalogue: Set | undefined): void { + if (catalogue === undefined || catalogue.has(event)) return; + const own = event.startsWith(`${packageName}.`) && /^[a-z][a-z0-9_]*$/.test(event.slice(packageName.length + 1)); + if (own) return; + refuse( + file, + `\`@event ${event}\` is neither an event this build fires nor a Package Event of \`${packageName}\`. ` + + `The catalogue is: ${[...catalogue].join(", ")}. A Package Event is spelled \`${packageName}.\`.`, + ); +} + +// ---- compilation ---- + +/// The directory this file sits in: the pinned Runtime Script toolchain. +function toolchainDir(): string { + return dirname(new URL(import.meta.url).pathname); +} + +function repoRoot(): string { + return resolve(toolchainDir(), "..", ".."); +} + +function packagesRoot(): string { + return process.env.LYRACORE_PACKAGES_ROOT ?? join(repoRoot(), "packages"); +} + +/// Compile ONE `.ts` under `scriptsDir` into `outDir`, with the checked-in settings. +/// +/// One file per invocation, not one program per Package: every script declares `function script()` +/// at top level, and a shared program would make the second one a duplicate declaration. Separate +/// programs also match the Host, which runs each script as its own chunk with its own globals. +/// +/// The per-run config is the checked-in `tsconfig.json` with three members added and the plugin path +/// made absolute. It is NOT an `extends` of it: `typescript-to-lua` resolves a plugin path against +/// the config it was invoked with, so a relative one in a base config would resolve from the wrong +/// directory. Materialising the whole config keeps that resolution the toolchain's own business. +async function compile(scriptsDir: string, source: string, outDir: string): Promise { + const toolchain = toolchainDir(); + const config = JSON.parse(await readFile(join(toolchain, "tsconfig.json"), "utf8")) as { + compilerOptions: Record; + tstl: { luaPlugins: Array<{ name: string }> }; + files?: string[]; + }; + config.compilerOptions.rootDir = scriptsDir; + config.compilerOptions.outDir = outDir; + config.files = [join(toolchain, "runtime-script.d.ts"), source]; + config.tstl.luaPlugins = config.tstl.luaPlugins.map((plugin) => ({ + ...plugin, + name: resolve(toolchain, plugin.name), + })); + + const configPath = join(outDir, "tsconfig.json"); + await writeFile(configPath, JSON.stringify(config, null, 2)); + + const tstl = join(toolchain, "node_modules", "typescript-to-lua", "dist", "tstl.js"); + const result = Bun.spawnSync({ + cmd: ["bun", tstl, "-p", configPath], + stdout: "inherit", + stderr: "inherit", + cwd: repoRoot(), + }); + if (result.exitCode !== 0) { + throw new ScriptRefusal( + `typescript-to-lua refused ${source}. Its own diagnostics are above.`, + ); + } +} + +// ---- the artifact ---- + +/// The digest of the sources this artifact was generated from: each immediate regular `.ts` or +/// `.lua` file under `scripts/`, by sorted name, length-prefixed so no two inventories can collide. +/// +/// The toolchain is deliberately NOT in it. `source_hash` answers "which revision of this Package's +/// sources is on the Shard"; whether the compiler moved is the Build Identity sidecar's question. +async function sourceHash(scriptsDir: string, files: string[]): Promise { + const digest = createHash("sha256"); + for (const file of files) { + const bytes = await readFile(join(scriptsDir, file)); + digest.update(file); + digest.update("\0"); + digest.update(String(bytes.length)); + digest.update("\0"); + digest.update(bytes); + } + return digest.digest("hex"); +} + +/// The artifact, as `crates/lyracore-package-delta/src/script.rs` parses it. +/// +/// Indented rather than canonical: the Official Package Collection COMMITS Script Artifacts, and a +/// reviewer reads this file. The Module canonicalises it again before hashing, so how it is spelled +/// changes nothing it means. Member order and script order are still fixed, so two runs over one +/// source tree write the same bytes. +function renderArtifact(packageName: string, hash: string, scripts: RuntimeScript[]): string { + const ordered = [...scripts].sort((a, b) => a.script_id - b.script_id); + return `${JSON.stringify( + { + kind: "script", + version: 1, + package: packageName, + source_hash: hash, + scripts: ordered.map((script) => ({ + script_id: script.script_id, + name: script.name, + event: script.event, + priority: script.priority, + enabled: script.enabled, + source: script.source, + })), + }, + null, + 2, + )}\n`; +} + +/// Every script source a Package ships, in file-name order. +function scriptSources(scriptsDir: string): string[] { + const listing = existsSync(scriptsDir) ? readdirSync(scriptsDir, { withFileTypes: true }) : []; + return listing + .filter((entry) => entry.isFile() && (entry.name.endsWith(".ts") || entry.name.endsWith(".lua"))) + .map((entry) => entry.name) + .sort(); +} + +/// Flush `artifact` to a sibling temporary file, then replace `path` in one rename. +/// +/// The old artifact stays whole if creating, writing, flushing, closing, or renaming the temporary +/// file fails. The temporary directory is on the destination filesystem, so the final rename is +/// atomic rather than a copy followed by deletion. +async function writeArtifact(path: string, artifact: string): Promise { + const parent = dirname(path); + await mkdir(parent, { recursive: true }); + const temporaryDir = await mkdtemp(join(parent, ".script-artifact-")); + try { + const temporaryPath = join(temporaryDir, basename(path)); + const file = await open(temporaryPath, "wx"); + try { + await file.writeFile(artifact, "utf8"); + await file.sync(); + } finally { + await file.close(); + } + await rename(temporaryPath, path); + } finally { + rmSync(temporaryDir, { recursive: true, force: true }); + } +} + +/// Build one Package's Script Artifact and write it. Returns the path written. +export async function buildPackageScripts(packageName: string): Promise { + const packageDir = join(packagesRoot(), packageName); + const scriptsDir = join(packageDir, SCRIPTS_DIR); + const files = scriptSources(scriptsDir); + if (files.length === 0) { + throw new ScriptRefusal(`${scriptsDir}: holds no \`.ts\` or \`.lua\` Runtime Script`); + } + + const catalogue = hookEvents(); + if (catalogue === undefined) { + console.error( + "LYRACORE_HOOK_EVENTS is unset, so event names are not checked here. " + + "`lyracore-delta-check` still refuses an unknown one.", + ); + } + + // Directives first, for every file, so a bad `@id` or a collision is a refusal before the first + // compile rather than after the last one. + const declared: Array & { file: string }> = []; + for (const file of files) { + const path = join(scriptsDir, file); + const directives = readDirectives(path, await readFile(path, "utf8")); + const event = requiredDirective(path, directives, "event"); + checkEvent(path, event, packageName, catalogue); + declared.push({ + file, + script_id: scriptId(path, directives), + name: `${packageName}.${file.slice(0, file.lastIndexOf("."))}`, + event, + priority: priority(path, directives), + enabled: enabled(path, directives), + }); + } + for (const [index, script] of declared.entries()) { + const clash = declared.findIndex( + (other, at) => at < index && (other.script_id === script.script_id || other.name === script.name), + ); + if (clash !== -1) { + refuse(join(scriptsDir, script.file), `collides with ${declared[clash]?.file} on \`@id\` or name`); + } + } + + const outDir = mkdtempSync(join(tmpdir(), "lyracore-scripts-")); + try { + const scripts: RuntimeScript[] = []; + for (const { file, ...script } of declared) { + const path = join(scriptsDir, file); + const stem = file.slice(0, file.lastIndexOf(".")); + let source: string; + if (file.endsWith(".lua")) { + source = await readFile(path, "utf8"); + } else { + await compile(scriptsDir, path, outDir); + source = await readFile(join(outDir, `${stem}.lua`), "utf8").catch(() => { + refuse(path, "typescript-to-lua emitted no Lua for it"); + }); + } + if (source.trim().length === 0) refuse(path, "compiles to nothing; a Runtime Script needs Lua in it"); + scripts.push({ ...script, source }); + } + + const artifact = renderArtifact(packageName, await sourceHash(scriptsDir, files), scripts); + const path = join(packageDir, GENERATED_DIR, `${packageName}.script.json`); + await writeArtifact(path, artifact); + return path; + } finally { + rmSync(outDir, { recursive: true, force: true }); + } +} + +if (import.meta.main) { + const packageName = process.argv[2]; + if (!packageName) { + console.error("usage: bun run datascripts/runtime-scripts/build-scripts.ts "); + process.exit(1); + } + try { + console.log(await buildPackageScripts(packageName)); + } catch (cause) { + console.error(cause instanceof Error ? cause.message : String(cause)); + process.exit(1); + } +} diff --git a/datascripts/runtime-scripts/fuel-workload.lua b/datascripts/runtime-scripts/fuel-workload.lua new file mode 100644 index 000000000..6c8c42047 --- /dev/null +++ b/datascripts/runtime-scripts/fuel-workload.lua @@ -0,0 +1,33 @@ +-- Lua Library inline imports +local function __TS__ArrayMap(self, callbackfn, thisArg) + local result = {} + for i = 1, #self do + result[i] = callbackfn(thisArg, self[i], i - 1, self) + end + return result +end +-- End of Lua Library inline imports +local function ____tbl(t) + return t +end +local function script() + local names = {} + do + local i = 1 + while i <= 20 do + names[#names + 1] = "unit" .. tostring(i) + i = i + 1 + end + end + local shouted = __TS__ArrayMap( + names, + function(____, name) return string.upper(name) end + ) + local roster = table.concat(shouted, ",") + local actor = event.actor + if actor and #roster > 0 then + grant_xp(actor, 25) + end + return #roster +end +return script() diff --git a/datascripts/runtime-scripts/fuel-workload.ts b/datascripts/runtime-scripts/fuel-workload.ts new file mode 100644 index 000000000..e79cd4e3c --- /dev/null +++ b/datascripts/runtime-scripts/fuel-workload.ts @@ -0,0 +1,30 @@ +// @event on_login +// @id 100999 +// +// The transpiler-shaped workload the Runtime Script Host's Fuel Budget is sized against, in the +// language a Package author writes it in. +// +// It is not installed anywhere. The directives above are there only so a test can copy this file +// straight into a scratch Package and get a real Runtime Script out; no Package ships it. Its ONLY job is to be compiled by the pinned +// toolchain and to have its Lua committed next to it, so two tests can meet on the same bytes: +// `datascripts/tests/runtime-scripts.test.ts` recompiles it and refuses a stale `.lua`, and the +// Module's `the_generated_representative_script_fits_the_fuel_budget` runs that same `.lua` on the +// Host. What the budget is sized against is therefore real transpiler output, not a hand-written +// impression of it. +// +// It reaches for the shapes the lua library has to supply: an array built by pushing, a +// higher-order map, string work, a join, and a Host Verb. + +function script(): number | void { + const names: string[] = []; + for (let i = 1; i <= 20; i += 1) { + names.push("unit" + i); + } + const shouted = names.map((name) => name.toUpperCase()); + const roster = shouted.join(","); + const actor = event.actor; + if (actor && roster.length > 0) { + grant_xp(actor, 25); + } + return roster.length; +} diff --git a/datascripts/runtime-scripts/lua-emit.cjs b/datascripts/runtime-scripts/lua-emit.cjs new file mode 100644 index 000000000..88377f0f9 --- /dev/null +++ b/datascripts/runtime-scripts/lua-emit.cjs @@ -0,0 +1,152 @@ +// The two things this printer owns: the shape generated Lua must not take on the pinned +// interpreter, and the entry point every Runtime Script declares. +// +// # The table-constructor guard +// +// piccolo 0.3.3 passes an inline table constructor's ELEMENT COUNT as an extra argument when the +// constructor is the LAST argument of a call, so `f({7, 8, 9})` reaches a two-parameter `f` as +// `f(table, 3)`. `x = x or default` is the commonest shape in transpiler output, so generated Lua +// meets it constantly. Both facts are pinned in the Module by +// `piccolo_leaks_a_table_constructors_element_count_as_an_extra_argument`. +// +// Parentheses do not help: the leak is in how the call site counts arguments, not in value +// adjustment. Passing the table through a one-parameter function does, because that function drops +// the extra value and returns exactly one. So this printer rewrites `f(a, {…})` to +// `f(a, ____tbl({…}))` and prepends `____tbl` to every emitted file. +// +// A printer rather than a TypeScript visitor: it sees the whole Lua AST, including the calls tstl +// itself invents for classes, spreads and library helpers, which a visitor over TypeScript nodes +// never meets. +// +// # The entry point +// +// A Runtime Script answers its caller by RETURNING a number, and TypeScript has no top-level +// return. So every authored `.ts` script declares `function script()` and this printer closes the +// emitted file with `return script()`. `beforeTransform` refuses a file without one, where the +// diagnostic can name it, rather than leaving a chunk that calls nil at runtime. + +const ts = require("typescript"); +const tstl = require("typescript-to-lua"); +const lua = require("typescript-to-lua/dist/LuaAST"); + +/// The one-parameter passthrough. Named in the `____` space tstl reserves for its own locals. +const GUARD = "____tbl"; + +/// The function name every authored `.ts` Runtime Script declares. +const ENTRY = "script"; + +/// `params` with a trailing table constructor wrapped in the guard. Only the last argument is +/// rewritten: Lua adjusts only the last argument of a call to multiple values, and that is exactly +/// where the leak lands. +function guardTrailingTable(params) { + if (!params || params.length === 0) return params; + const last = params[params.length - 1]; + if (!lua.isTableExpression(last)) return params; + const wrapped = lua.createCallExpression(lua.createIdentifier(GUARD), [last]); + return [...params.slice(0, -1), wrapped]; +} + +/// `local function ____tbl(t) return t end`, prepended to every file whether or not a call needed +/// it. Emitting it unconditionally keeps the output a pure function of the source: whether the +/// guard is reachable never changes a byte anywhere else. +function guardDeclaration() { + const parameter = lua.createIdentifier("t"); + return lua.createVariableDeclarationStatement( + lua.createIdentifier(GUARD), + lua.createFunctionExpression( + lua.createBlock([lua.createReturnStatement([parameter])]), + [parameter], + ), + ); +} + +/// `return script()` — the last statement of every emitted file, so the chunk's return value is the +/// Script Answer. +function entryCall() { + return lua.createReturnStatement([ + lua.createCallExpression(lua.createIdentifier(ENTRY), []), + ]); +} + +class PiccoloPrinter extends tstl.LuaPrinter { + printCallExpression(expression) { + // The guard's own call must not be wrapped again, or printing never terminates. + if (lua.isIdentifier(expression.expression) && expression.expression.text === GUARD) { + return super.printCallExpression(expression); + } + return super.printCallExpression({ ...expression, params: guardTrailingTable(expression.params) }); + } + + printMethodCallExpression(expression) { + return super.printMethodCallExpression({ + ...expression, + params: guardTrailingTable(expression.params), + }); + } + + printFile(file) { + return super.printFile({ + ...file, + statements: [guardDeclaration(), ...file.statements, entryCall()], + }); + } +} + +function acceptsScriptAnswer(type) { + if (type.isUnion()) return type.types.every(acceptsScriptAnswer); + return (type.flags & (ts.TypeFlags.NumberLike | ts.TypeFlags.Void | ts.TypeFlags.Undefined | ts.TypeFlags.Never)) !== 0; +} + +/// Every non-declaration source file must declare `function script(): number | void` with no +/// parameters. The Host invokes it without arguments and reads only a numeric Script Answer. +function requireEntryPoint(program) { + const diagnostics = []; + const checker = program.getTypeChecker(); + for (const file of program.getSourceFiles()) { + if (file.isDeclarationFile) continue; + const declarations = file.statements.filter( + (statement) => + ts.isFunctionDeclaration(statement) && statement.name && statement.name.text === ENTRY, + ); + const implementations = declarations.filter((declaration) => declaration.body !== undefined); + if (implementations.length !== 1) { + diagnostics.push({ + file, + start: declarations[0]?.getStart(file) ?? 0, + length: declarations[0]?.getWidth(file) ?? 0, + category: ts.DiagnosticCategory.Error, + code: 0, + messageText: + `a Runtime Script declares exactly one concrete entry point as ` + + `\`function ${ENTRY}(): number | void\`. ` + + `The emitted Lua ends with \`return ${ENTRY}()\`, because TypeScript has no top-level ` + + "return and a Script Answer is the chunk's return value.", + }); + continue; + } + + const declaration = implementations[0]; + const signature = checker.getSignatureFromDeclaration(declaration); + const returnType = signature && checker.getReturnTypeOfSignature(signature); + if (declaration.parameters.length === 0 && returnType && acceptsScriptAnswer(returnType)) { + continue; + } + diagnostics.push({ + file, + start: declaration.getStart(file), + length: declaration.getWidth(file), + category: ts.DiagnosticCategory.Error, + code: 0, + messageText: + `\`${ENTRY}\` must take no parameters and return only a number or nothing. ` + + "The Runtime Script Host calls it without arguments and accepts only a numeric Script Answer.", + }); + } + return diagnostics; +} + +module.exports = { + beforeTransform: (program) => requireEntryPoint(program), + printer: (program, emitHost, fileName, file) => + new PiccoloPrinter(emitHost, program, fileName).print(file), +}; diff --git a/datascripts/runtime-scripts/package.json b/datascripts/runtime-scripts/package.json new file mode 100644 index 000000000..1b2ac926c --- /dev/null +++ b/datascripts/runtime-scripts/package.json @@ -0,0 +1,8 @@ +{ + "name": "lyracore-script-compiler", + "private": true, + "dependencies": { + "typescript": "6.0.2", + "typescript-to-lua": "1.37.1" + } +} diff --git a/datascripts/runtime-scripts/runtime-script.d.ts b/datascripts/runtime-scripts/runtime-script.d.ts new file mode 100644 index 000000000..c77b5fd27 --- /dev/null +++ b/datascripts/runtime-scripts/runtime-script.d.ts @@ -0,0 +1,62 @@ +/// The Runtime Script Host API, as TypeScript sees it. +/// +/// Ambient on purpose: the Host hands an Invocation globals, not a module, so an authored script +/// imports nothing and exports nothing. A file with an `import` or an `export` becomes a TypeScript +/// module, and `typescript-to-lua` then emits a chunk that returns its export table instead of the +/// Script Answer. +/// +/// Hand-maintained against `module/src/runtime_script.rs`. The EVENT names are not here: they are a +/// closed catalogue the Module's own build owns, and `lyracore packages build` checks a script's +/// `@event` directive against `lyracore-delta-check --print-events` rather than against a copy that +/// can drift. +/// +/// Everything a script may reach is below. A name outside this file is nil inside an Invocation. + +/// One creature or player the Host resolved for this Invocation. +/// +/// Opaque: a script cannot read a guid out of it and cannot mint one, so the only entities a script +/// can act on are the ones its event carried. Every field is a snapshot taken before the script +/// ran; writing to one changes nothing. +interface Entity { + readonly name: string; + readonly is_player: boolean; + readonly level: number; + readonly health: number; + readonly max_health: number; + readonly map_id: number; + readonly x: number; + readonly y: number; + readonly z: number; +} + +/// What this Invocation is running for. +interface ScriptEvent { + /// The event label, as the Script Artifact spells it. + readonly name: string; + /// Who caused the event, when the event carries one. + readonly actor?: Entity; + /// What the event acted on, when the event carries one. + readonly target?: Entity; +} + +declare const event: ScriptEvent; + +/// Stage a heal on `entity`, crediting heal-threat to `event.actor`. +declare function heal(entity: Entity, amount: number): void; + +/// Stage a System Message to one online player. +declare function send_chat(player: Entity, text: string): void; + +/// Stage an experience grant. +declare function grant_xp(player: Entity, amount: number): void; + +// Every authored `.ts` Runtime Script declares its own entry point: +// +// function script(): number | void { … } +// +// `lyracore packages build` appends `return script()` to the emitted Lua, so the number it returns +// is the Script Answer the asking Package reads back. Returning nothing answers nothing, and the +// caller keeps its own fallback. The build refuses a script file that declares no `script`. +// +// It is not declared here: an ambient declaration and the author's own implementation would be two +// declarations of one name. diff --git a/datascripts/runtime-scripts/tsconfig.json b/datascripts/runtime-scripts/tsconfig.json new file mode 100644 index 000000000..76f8dd18a --- /dev/null +++ b/datascripts/runtime-scripts/tsconfig.json @@ -0,0 +1,18 @@ +{ + "compilerOptions": { + "strict": true, + "target": "ES2020", + "lib": ["esnext"], + "types": [], + "noEmitOnError": true + }, + "tstl": { + "luaTarget": "5.4", + "luaLibImport": "inline", + "noHeader": true, + "noImplicitSelf": true, + "noImplicitGlobalVariables": true, + "sourceMapTraceback": false, + "luaPlugins": [{ "name": "./lua-emit.cjs" }] + } +} diff --git a/datascripts/tests/runtime-scripts.test.ts b/datascripts/tests/runtime-scripts.test.ts new file mode 100644 index 000000000..654d6a7dd --- /dev/null +++ b/datascripts/tests/runtime-scripts.test.ts @@ -0,0 +1,333 @@ +// The Runtime Script build, end to end: sources under a Package's `scripts/`, one Script Artifact +// out, and the same bytes every time. +// +// The committed `runtime-scripts/fuel-workload.lua` is where this suite meets the Module: +// `module/src/runtime_script.rs` runs those exact bytes on the Runtime Script Host and sizes the +// Fuel Budget against them. So the first test recompiles the workload and refuses a stale copy — +// otherwise the Module would be measuring Lua the toolchain no longer emits. + +import { expect, test } from "bun:test"; +import { + chmodSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; + +import { buildPackageScripts } from "../runtime-scripts/build-scripts.ts"; + +const TOOLCHAIN = join(import.meta.dir, "..", "runtime-scripts"); +const PACKAGE = "example.scripts"; +const EVENT = "on_login"; + +interface Built { + readonly artifact: string; + readonly scripts: Array<{ script_id: number; name: string; event: string; source: string }>; +} + +/// Build a scratch Package holding `files`, and return its artifact. +/// +/// A scratch root rather than `packages/`: the artifact is what is under test, and a build that +/// wrote into the checkout would make one test's output another test's input. +async function build( + files: Record, + events?: string[], + packageName = PACKAGE, +): Promise { + const root = mkdtempSync(join(tmpdir(), "lyracore-runtime-scripts-")); + const previousRoot = process.env.LYRACORE_PACKAGES_ROOT; + const previousEvents = process.env.LYRACORE_HOOK_EVENTS; + try { + const scripts = join(root, packageName, "scripts"); + mkdirSync(scripts, { recursive: true }); + for (const [name, source] of Object.entries(files)) { + const path = join(scripts, name); + mkdirSync(dirname(path), { recursive: true }); + writeFileSync(path, source); + } + process.env.LYRACORE_PACKAGES_ROOT = root; + if (events === undefined) delete process.env.LYRACORE_HOOK_EVENTS; + else process.env.LYRACORE_HOOK_EVENTS = events.join("\n"); + + const artifact = readFileSync(await buildPackageScripts(packageName), "utf8"); + return { artifact, scripts: JSON.parse(artifact).scripts }; + } finally { + if (previousRoot === undefined) delete process.env.LYRACORE_PACKAGES_ROOT; + else process.env.LYRACORE_PACKAGES_ROOT = previousRoot; + if (previousEvents === undefined) delete process.env.LYRACORE_HOOK_EVENTS; + else process.env.LYRACORE_HOOK_EVENTS = previousEvents; + rmSync(root, { recursive: true, force: true }); + } +} + +/// The two required directives, for the tests that are about something else. +function directives(event = EVENT, id = 100_200): string { + return `// @event ${event}\n// @id ${id}\n`; +} + +// ---- the workload the Module's Fuel Budget is sized against ---- + +test("the committed fuel workload Lua is what the pinned toolchain emits today", async () => { + const source = readFileSync(join(TOOLCHAIN, "fuel-workload.ts"), "utf8"); + const committed = readFileSync(join(TOOLCHAIN, "fuel-workload.lua"), "utf8"); + + const built = await build({ "fuel-workload.ts": source }); + + expect(built.scripts[0]?.source).toBe(committed); +}); + +test("every checked-in Package Runtime Script passes through the pinned compiler", async () => { + const packagesRoot = join(import.meta.dir, "..", "..", "packages"); + const packageScripts = readdirSync(packagesRoot, { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .sort((a, b) => a.name.localeCompare(b.name)) + .flatMap((entry) => { + const scriptsDir = join(packagesRoot, entry.name, "scripts"); + let sources; + try { + sources = readdirSync(scriptsDir, { withFileTypes: true }); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return []; + throw error; + } + const files = Object.fromEntries( + sources + .filter( + (source) => + source.isFile() && (source.name.endsWith(".ts") || source.name.endsWith(".lua")), + ) + .sort((a, b) => a.name.localeCompare(b.name)) + .map((source) => [source.name, readFileSync(join(scriptsDir, source.name), "utf8")]), + ); + return Object.keys(files).length === 0 ? [] : [{ package: entry.name, files }]; + }); + + expect(packageScripts.length).toBeGreaterThan(0); + for (const checkedIn of packageScripts) { + await build(checkedIn.files, undefined, checkedIn.package); + } +}); + +// ---- determinism ---- + +test("two builds of one source tree write the same bytes", async () => { + const files = { "greet.ts": `${directives()}function script(): number { return 7; }\n` }; + + expect((await build(files)).artifact).toBe((await build(files)).artifact); +}); + +test("the source digest covers only immediate regular TypeScript and Lua files", async () => { + const built = await build({ + "alpha.ts": `${directives(EVENT, 100_201)}function script(): void {}\n`, + "zeta.lua": "-- @event on_login\n-- @id 100202\nreturn 2\n", + "README.md": "ignored\n", + "nested/hidden.ts": `${directives(EVENT, 100_203)}function script(): void {}\n`, + }); + + expect(JSON.parse(built.artifact).source_hash).toBe( + "8395ead00aad341a7daa23658447385da94dabf6932b406cbfbdb5e2fd664002", + ); + expect(built.scripts.map((script) => script.name)).toEqual([ + `${PACKAGE}.alpha`, + `${PACKAGE}.zeta`, + ]); +}); + +test("the scripts are ordered by identifier, whatever order the files sort in", async () => { + const built = await build({ + "zeta.ts": `${directives(EVENT, 100_300)}function script(): void {}\n`, + "alpha.ts": `${directives(EVENT, 100_900)}function script(): void {}\n`, + }); + + expect(built.scripts.map((s) => s.script_id)).toEqual([100_300, 100_900]); + expect(built.scripts.map((s) => s.name)).toEqual([`${PACKAGE}.zeta`, `${PACKAGE}.alpha`]); +}); + +// ---- the piccolo call shape ---- + +test("a table constructor passed to a call is hoisted through the guard", async () => { + // piccolo 0.3.3 leaks a trailing constructor's element count as an extra argument. The Module + // pins both that fault and this fix; here we prove the emitter applies it. + const built = await build({ + "guard.ts": + `${directives()}function take(o: { a: number }): number { return o.a; }\n` + + "function script(): number { return take({ a: 1 }); }\n", + }); + + const lua = built.scripts[0]?.source ?? ""; + expect(lua).toContain("local function ____tbl(t)"); + expect(lua).toMatch(/____tbl\(\{/); +}); + +test("every emitted file ends by calling its entry point, so the chunk returns the Script Answer", async () => { + const built = await build({ + "answer.ts": `${directives()}function script(): number { return 3; }\n`, + }); + + expect(built.scripts[0]?.source.trimEnd().endsWith("return script()")).toBe(true); +}); + +// ---- hand-written Lua ---- + +test("a hand-written .lua script ships unchanged", async () => { + const lua = "-- @event on_login\n-- @id 100400\ngrant_xp(event.actor, 5)\n"; + + const built = await build({ "bonus.lua": lua }); + + expect(built.scripts[0]?.source).toBe(lua); + expect(built.scripts[0]?.name).toBe(`${PACKAGE}.bonus`); +}); + +// ---- refusals, each naming the file ---- + +test("a script declaring no entry point is refused", async () => { + expect(build({ "bare.ts": `${directives()}const x = 1;\n` })).rejects.toThrow( + /typescript-to-lua refused/, + ); +}); + +test("the entry point takes no parameters and returns only a number or nothing", async () => { + expect( + build({ + "parameter.ts": + `${directives()}function script(required: number): number { return required; }\n`, + }), + ).rejects.toThrow(/typescript-to-lua refused/); + expect( + build({ + "string.ts": `${directives()}function script(): string { return "wrong"; }\n`, + }), + ).rejects.toThrow(/typescript-to-lua refused/); +}); + +test("an optional entry-point parameter is refused", async () => { + expect( + build({ + "optional.ts": `${directives()}function script(value?: number): void {}\n`, + }), + ).rejects.toThrow(/typescript-to-lua refused/); +}); + +test("a rest entry-point parameter is refused", async () => { + expect( + build({ + "rest.ts": `${directives()}function script(...values: number[]): void {}\n`, + }), + ).rejects.toThrow(/typescript-to-lua refused/); +}); + +test("an overload cannot hide an invalid concrete entry point", async () => { + expect( + build({ + "overload.ts": + `${directives()}function script(): number;\n` + + `function script(value?: number): number | string { return value ?? "wrong"; }\n`, + }), + ).rejects.toThrow(/typescript-to-lua refused/); +}); + +test("a missing @event or @id is refused by name", async () => { + expect( + build({ "no-id.ts": "// @event on_login\nfunction script(): void {}\n" }), + ).rejects.toThrow(/`@id`/); + expect(build({ "no-event.ts": "// @id 100200\nfunction script(): void {}\n" })).rejects.toThrow( + /`@event`/, + ); +}); + +test("an identifier outside the Package script band is refused", async () => { + expect( + build({ "low.ts": `${directives(EVENT, 42)}function script(): void {}\n` }), + ).rejects.toThrow(/100000\.\.=999999/); +}); + +test("two scripts claiming one identifier are refused", async () => { + expect( + build({ + "a.ts": `${directives()}function script(): void {}\n`, + "b.ts": `${directives()}function script(): void {}\n`, + }), + ).rejects.toThrow(/collides/); +}); + +test("an unknown event is refused and a Package Event of the own Package is not", async () => { + const catalogue = ["on_login", "on_death"]; + + expect( + build({ "sneeze.ts": `${directives("on_sneeze")}function script(): void {}\n` }, catalogue), + ).rejects.toThrow(/on_sneeze/); + + const own = await build( + { "own.ts": `${directives(`${PACKAGE}.ember_spent`)}function script(): void {}\n` }, + catalogue, + ); + expect(own.scripts[0]?.event).toBe(`${PACKAGE}.ember_spent`); +}); + +test("a Package folder with no script sources is refused rather than writing an empty artifact", async () => { + expect(build({})).rejects.toThrow(/no `\.ts` or `\.lua`/); +}); + +test("a failed artifact write leaves the prior artifact unchanged", async () => { + const root = mkdtempSync(join(tmpdir(), "lyracore-runtime-scripts-write-")); + const previousRoot = process.env.LYRACORE_PACKAGES_ROOT; + const packageDir = join(root, PACKAGE); + const scripts = join(packageDir, "scripts"); + const generated = join(packageDir, "data", ".generated"); + const artifact = join(generated, `${PACKAGE}.script.json`); + try { + mkdirSync(scripts, { recursive: true }); + mkdirSync(generated, { recursive: true }); + writeFileSync( + join(scripts, "greet.lua"), + "-- @event on_login\n-- @id 100200\nreturn 1\n", + ); + writeFileSync(artifact, "prior artifact\n"); + chmodSync(generated, 0o555); + process.env.LYRACORE_PACKAGES_ROOT = root; + + await expect(buildPackageScripts(PACKAGE)).rejects.toThrow(); + + chmodSync(generated, 0o755); + expect(readFileSync(artifact, "utf8")).toBe("prior artifact\n"); + expect(readdirSync(generated)).toEqual([`${PACKAGE}.script.json`]); + } finally { + chmodSync(generated, 0o755); + if (previousRoot === undefined) delete process.env.LYRACORE_PACKAGES_ROOT; + else process.env.LYRACORE_PACKAGES_ROOT = previousRoot; + rmSync(root, { recursive: true, force: true }); + } +}); + +test("a failed atomic rename removes its sibling temporary directory", async () => { + const root = mkdtempSync(join(tmpdir(), "lyracore-runtime-scripts-rename-")); + const previousRoot = process.env.LYRACORE_PACKAGES_ROOT; + const packageDir = join(root, PACKAGE); + const scripts = join(packageDir, "scripts"); + const generated = join(packageDir, "data", ".generated"); + const artifact = join(generated, `${PACKAGE}.script.json`); + try { + mkdirSync(scripts, { recursive: true }); + mkdirSync(artifact, { recursive: true }); + writeFileSync( + join(scripts, "greet.lua"), + "-- @event on_login\n-- @id 100200\nreturn 1\n", + ); + writeFileSync(join(artifact, "prior"), "unchanged\n"); + process.env.LYRACORE_PACKAGES_ROOT = root; + + await expect(buildPackageScripts(PACKAGE)).rejects.toThrow(); + + expect(readFileSync(join(artifact, "prior"), "utf8")).toBe("unchanged\n"); + expect(readdirSync(generated)).toEqual([`${PACKAGE}.script.json`]); + } finally { + if (previousRoot === undefined) delete process.env.LYRACORE_PACKAGES_ROOT; + else process.env.LYRACORE_PACKAGES_ROOT = previousRoot; + rmSync(root, { recursive: true, force: true }); + } +}); diff --git a/datascripts/tsconfig.json b/datascripts/tsconfig.json index 70eb92362..124a1aad7 100644 --- a/datascripts/tsconfig.json +++ b/datascripts/tsconfig.json @@ -10,5 +10,5 @@ "skipLibCheck": true, "types": ["bun", "node"] }, - "include": ["lib", "src", "tests", "generated"] + "include": ["lib", "src", "tests", "generated", "runtime-scripts/build-scripts.ts"] } diff --git a/docs/development-cli.md b/docs/development-cli.md index 7a1b7ce08..1f8e8be9f 100644 --- a/docs/development-cli.md +++ b/docs/development-cli.md @@ -93,7 +93,7 @@ lyracore update | `client sync` | pack `patch-3.MPQ` and every enabled Package's addons, then install them into the configured client | | `client pack` | build the Client Artifact a player installs: package-authored content only, into a directory of your choosing | | `packages add` | install a Package from a folder on this machine, from a Git URL, or by bare name from the Official Package Collection, after a trust review and a confirmation | -| `packages build` | regenerate the Module schema typings, typecheck every Datascript against them, then emit and validate each enabled Package's Datascript-generated Package Delta | +| `packages build` | regenerate the Module schema typings, typecheck every Datascript against them, then emit and validate each enabled Package's Package Delta and Script Artifact | | `packages check` | verify every enabled Package's generated artifact against its recorded Build Identity, regenerating the Module typings fresh | | `packages config` | read a Package's key-values, or write one to every Shard of the fixture topology | | `packages disable` | move an enabled Package out of the build's sight, keeping it on disk | @@ -226,11 +226,11 @@ than the folder name — `my-package` and `my_package` fold onto the same module Then it prints a **Trust Review** and asks. The review is a deterministic, read-only scan of the candidate folder using a port of the build's own marker scan, so a commented-out or quoted marker registers nothing here either. It reports tables, reducers, hooks, tick passes, character-owned -sweeps, addons and client overrides. A Package cannot yet ship a Datascript of its own — the -authoring toolchain lives in `datascripts/` and is described under `packages build` below — so -Datascripts and Runtime Scripts get an explicit "none detected" row rather than silence. The review states -plainly that everything else in the Package's Rust is trusted code and that it is an inventory, not -a security guarantee. +sweeps, addons, client overrides and Runtime Script sources. Runtime Scripts are named because they +run on the realm once the Package is built. A Package cannot ship a Datascript of its own — the +authoring toolchain lives in `datascripts/`, outside any Package folder — so that row is an explicit +"none detected" rather than silence. The review states plainly that everything else in the Package's +Rust is trusted code and that it is an inventory, not a security guarantee. On confirmation the folder is **copied, never symlinked**: a linked Package would compile from a folder outside the checkout, so `preflight`, `publish` and `client sync` would each read whatever @@ -394,7 +394,7 @@ claim the Operator, so the local realm has to be up and claimed. was written, what stopped it, and what was never touched. Re-running the same command after the cause is fixed rewrites the Shards that already took the value, which changes nothing on them. -## `packages build` — Datascript typings, the typecheck gate, and Package Delta emission +## `packages build` — Datascript typings, the typecheck gate, and artifact emission ```bash ./lyracore packages build @@ -402,8 +402,9 @@ cause is fixed rewrites the Shards that already took the value, which changes no A **Datascript** is author-time TypeScript that describes game data. It is written against the Module's own schema, so the names and types in it cannot drift from the Module. `packages build` -enforces that and, once a Package has a Datascript, turns it into a validated Package Delta. A -version gate, then up to seven steps, in this order: +enforces that and, once a Package has a Datascript, turns it into a validated Package Delta. The +same command compiles a Package's **Runtime Scripts** into its Script Artifact. A version gate, +then up to eight steps, in this order: 0. `bun --version` must match the checkout's pin. A hard failure, not a warning: the next two steps run `bun install` and the locked `tsc` for real, against whatever Bun is on `PATH`. @@ -414,22 +415,30 @@ version gate, then up to seven steps, in this order: a build never silently resolves a newer dependency than the next author will get. 3. `tsc --noEmit` is the typecheck gate. Nothing is emitted; the answer is the exit code. -Steps 4 to 7 run only when an enabled Package carries a Datascript. A checkout with none builds -exactly as it did before those steps existed: +Steps 4 to 8 run only when an enabled Package carries a Datascript or a Runtime Script. A checkout +with neither builds exactly as it did before those steps existed: 4. The Base Snapshot must already exist at `datascripts/generated/base-snapshot.json`, or the build fails fast with the exact `lyracore-importer --spell-snapshot` command to build one, once, rather - than letting every Datascript fail with the same "cannot read" error in turn. + than letting every Datascript fail with the same "cannot read" error in turn. Skipped when no + Package has a Datascript: a Runtime Script reads no base data. 5. Every enabled Package's Datascripts run, one `bun run` subprocess per file, in name order. The first script to throw stops the build; later scripts and later Packages never run. -6. `lyracore-delta-check` traces every enabled Package's generated artifacts together, in one - invocation. This is the same authoritative Rust-side check `packages replay` runs before it - writes to a Shard, so a Claim Conflict between two Packages is caught by the one implementation - that also decides whether it may apply, not by a second, looser one. -7. A **Build Identity** sidecar is written next to each artifact that just validated: the hashes - `packages check` and preflight later recompute to tell whether the artifact is still current. - Written only after step 6 succeeds, so a sidecar never describes an artifact this build itself - would have refused. +6. Every enabled Package with a `scripts/` folder compiles its Runtime Scripts into one Script + Artifact, one `bun run` subprocess per Package, in folder-name order. The builder is handed the + Module's Event Binding catalogue, read from `lyracore-delta-check --print-events`, so a mistyped + `@event` fails with the file that holds it. Fail-fast, the same way step 5 is. +7. `lyracore-delta-check` traces every enabled Package's generated artifacts together, in one + invocation, Package Deltas and Script Artifacts alike. This is the same authoritative Rust-side + check `packages replay` runs before it writes to a Shard, so a Claim Conflict or a Runtime Script + collision between two Packages is caught by the one implementation that also decides whether it + may apply, not by a second, looser one. +8. A **Build Identity** sidecar is written next to each source-built artifact that just validated: + the hashes `packages check` and preflight later recompute to tell whether it is still current. + A source-free prebuilt Script Artifact has no local author inputs or sidecar; the authoritative + checker still parses and traces it. Each kind records its own inputs — a Script Artifact's are + its `scripts/` sources and the Runtime Script Toolchain, not the Module typings or the Base + Snapshot. A Package Delta this command emits is never committed: it is regenerated author-side on every build and installed from source, the same way `datascripts/generated/` itself is git-ignored. A @@ -457,6 +466,7 @@ datascripts/ tsconfig.json strict, noEmit src/ the Datascripts. `src/reference.ts` is the maintained reference Datascript generated/ written by step 1. NOT committed + runtime-scripts/ the Runtime Script Toolchain — a Bun workspace, so one `bun install` covers both ``` `generated/` is git-ignored on purpose. It is a ~400-file, 2 MB projection of the module wasm that @@ -481,6 +491,89 @@ Datascripts are **trusted author-time code**, run from this checkout by the pers They are not sandboxed and are not described as sandboxed. `packages build` above is what turns one into a Package Delta; `packages replay`, below, is what applies it to a Shard. +### Runtime Scripts — `packages//scripts/` + +A **Runtime Script** is the opposite kind of code: Lua the Module runs *on the realm*, inside the +Runtime Script Host, with a Fuel Budget and no access to anything the Host did not hand it. Write +one in TypeScript or in Lua: + +```text +packages/fire_nova/ + scripts/ + ember_echo.ts compiled by the Runtime Script Toolchain + bonus.lua shipped unchanged, for the author who would rather write Lua + data/.generated/ + fire_nova.script.json the Script Artifact. Committed in the Official Package Collection +``` + +The sources live **inside** the Package, unlike a Datascript. A Datascript sits outside because only +artifacts belong in a Package folder; a Runtime Script is the Package's own content. + +Every file opens with its **Script Directives**, `//` in TypeScript and `--` in Lua: + +```ts +// @event on_cast_resolved +// @id 100200 +// @priority 10 +// @enabled false +``` + +`@event` and `@id` are required. `@priority` defaults to 0 and `@enabled` to true. The directives +stop at the first line that is neither blank nor a comment, so ordinary comments below them are just +comments. `@event` must name an event this build dispatches, or a Package Event of the shipping +Package. `@id` must sit in the Package Script Range, 100,000 to 999,999. + +The identifier is **written down, not derived**. It is durable: `game_script` keys on it. Deriving +it from a sorted file index would renumber every later script the moment an author added one +alphabetically earlier, and two Packages would both start at the bottom of the band. The script +*name* is derived, because there is one obvious answer: `.`. + +A TypeScript script declares its entry point and returns its Script Answer from it: + +```ts +function script(): number | void { + const caster = event.actor; + if (!caster) return; + send_chat(caster, "the embers answer"); + return caster.level; +} +``` + +The emitted Lua ends with `return script()`, because TypeScript has no top-level return and a Script +Answer is the chunk's return value. A file without a `script` function is refused by name. Each file +compiles as its own program, so two scripts in one Package may both declare `script`. + +`datascripts/runtime-scripts/runtime-script.d.ts` is the hand-maintained Host API: the event, the +Entity Handles it carries, and the Host Operations. A name outside that file is nil inside an +Invocation. The event catalogue is deliberately **not** in it — the Module owns it, and a copy would +drift; the build reads it from `lyracore-delta-check --print-events` instead. + +The emitter rewrites one call shape. piccolo 0.3.3 passes an inline table constructor's element +count as an extra argument when the constructor is the last argument of a call, so `f({7, 8, 9})` +arrives as `f(table, 3)`. Transpiler output meets that shape constantly. Every emitted file +therefore opens with `local function ____tbl(t) return t end` and a trailing constructor is passed +through it. `module/src/runtime_script.rs` pins both the fault and the fix. A hand-written `.lua` +script ships unchanged, so its author avoids that call shape themselves. + +Syntax errors and failures raised through `error`, `assert`, or a Host Operation name a line of the +**generated** Lua, the bytes the Shard holds, never a line of the TypeScript. There is no source +map, and a number pointing into a file no Shard holds would be worse than no number. Piccolo 0.3.3 +does not expose a frame after a native VM fault unwinds or when the Host stops an unfinished fuel +step. Those diagnostics name the Runtime Script, Event Binding, failure kind, and interpreter +message, but no line. Adding generated-code instrumentation would change emitted Lua and Fuel +Budget accounting, so this Host does not guess. + +Build one Package's scripts by hand the way `packages build` does: + +```bash +bun run datascripts/runtime-scripts/build-scripts.ts fire_nova +``` + +Script Artifacts are **committed** in the Official Package Collection, so an Operator installs +prebuilt Lua and needs no Bun. In this checkout they are git-ignored output: the sources next to +them are the repository content, and a committed copy would go stale the moment the Module schema or +the pinned toolchain moved. + ## `packages replay` — reapply a Package's artifacts across the Realm A Package's artifacts are not a one-shot edit. Its catalogues live on every World Shard and @@ -536,28 +629,34 @@ The other claim families a Package may claim in (items, quests, loot, casts and replayed here. Their base import is the world dump rather than a DBC, which is a much larger and more destructive reload than this verb owns. Reapply those with the importer's own dump modes. -## `packages check` — is every Package Delta still current? +## `packages check` — is every generated artifact still current? ```bash ./lyracore packages check ``` -`packages build` writes a Build Identity next to each artifact it emits. `packages check` +`packages build` writes a Build Identity next to each source-built artifact. `packages check` recomputes every recorded input from the checkout on disk right now and refuses, naming the specific input, the moment one no longer matches. `preflight` folds the same report into its own gate on `publish`'s behalf, so a stale artifact never reaches a Shard. +A Package Delta always has a sidecar. A source-built Script Artifact has one for its `scripts/` +sources, Runtime Script Toolchain, and Bun pin. A source-free prebuilt Script Artifact has no local +author inputs or sidecar. `packages check` still sends every Script Artifact through the +authoritative parser and tracer, so a malformed or conflicting prebuilt artifact never passes. + `datascripts/generated/` is regenerated fresh, every run, with the same `spacetime generate` invocation `packages build`'s typegen step uses, so a Module schema change makes a committed -artifact stale even on a clean checkout that never ran `packages build` itself. Nothing else is -regenerated: this command never runs Bun and never re-emits a Datascript, so it needs neither Bun -nor a Base Snapshot to do its job. +artifact stale even on a clean checkout that never ran `packages build` itself. The typings are an +input of a Package Delta alone, so a checkout shipping only Runtime Scripts skips that step and +builds no module wasm. Nothing else is regenerated: this command never runs Bun and never re-emits +anything, so it needs neither Bun nor a Base Snapshot to do its job. A missing Base Snapshot is reported and does not fail the check: the snapshot is the Operator's own client-derived data, and a CI machine holding none cannot regenerate one to compare against. A Base Snapshot that is present and no longer matches its recorded hash is a real mismatch and fails like -any other input. A missing sidecar is treated as stale; it predates identity tracking, so there is -nothing to compare against. +any other input. A missing sidecar is stale for a Package Delta or a Script Artifact with sources. +For a source-free Script Artifact it is the prebuilt contract, not a skipped parser check. A checkout with no Packages at all, or none carrying a generated artifact, is a clean no-op. diff --git a/module/src/package_import/script.rs b/module/src/package_import/script.rs index 56f9cedd6..7cd698c80 100644 --- a/module/src/package_import/script.rs +++ b/module/src/package_import/script.rs @@ -120,8 +120,8 @@ impl ScriptPlan { /// Reads a payload into a plan, refusing anything that must not reach the table. /// /// The refusals, in order: an artifact that does not parse, two artifacts naming the same - /// Package, and any collision between Packages. A collision reports EVERY disagreement, not - /// just the first — the operator fixing them wants the whole list in one pass. + /// Package, and any collision between Packages. The shared tracer reports EVERY disagreement, + /// not just the first, because the operator fixing them wants the whole list in one pass. fn read(packed: &str) -> Result { let mut artifacts: Vec = Vec::new(); for (index, artifact) in packed @@ -131,12 +131,6 @@ impl ScriptPlan { { let parsed = ScriptArtifact::parse(artifact) .map_err(|e| format!("Script Artifact {index} in this plan is invalid: {e}"))?; - if let Some(seen) = artifacts.iter().find(|a| a.package() == parsed.package()) { - return Err(format!( - "package `{}` appears twice in this plan", - seen.package() - )); - } artifacts.push(parsed); } @@ -311,7 +305,10 @@ mod tests { let refusal = plan(&[once.clone(), once]).expect_err("plan is refused"); - assert!(refusal.contains("appears twice"), "{refusal}"); + assert!( + refusal.contains("more than one Script Artifact"), + "{refusal}" + ); } #[test] diff --git a/module/src/runtime_script.rs b/module/src/runtime_script.rs index 14843ba0c..5e5540632 100644 --- a/module/src/runtime_script.rs +++ b/module/src/runtime_script.rs @@ -70,8 +70,8 @@ use spacetimedb::log; use piccolo::closure::UpValueState; use piccolo::{ - Callback, CallbackReturn, Closure, Context, Error, Executor, Fuel, IntoValue, Lua, MetaMethod, - StashedClosure, Table, UserData, Value, + Callback, CallbackReturn, Closure, Context, Error, Execution, Executor, Fuel, IntoValue, Lua, + MetaMethod, StashedClosure, Table, UserData, Value, }; /// Fuel handed to one `Executor::step`. The interpreter checks its budget between operations, so a @@ -461,6 +461,12 @@ impl RuntimeScriptHost { /// On success, returns everything the script staged and the Script Answer it returned — nothing /// has touched the world yet. On any failure, returns a bounded diagnostic and both the staged /// effects and the answer are dropped unread. + /// + /// Piccolo exposes the caller's generated-Lua line while a Host callback is running, which is + /// why `error`, `assert`, and Host Verb diagnostics carry one. Native VM failures reach this + /// boundary only after Piccolo has unwound those frames, and an unfinished fuel step exposes no + /// frame. Their diagnostics therefore retain the Runtime Script, Event Binding, failure kind, + /// and interpreter message, but cannot reliably name a line with Piccolo 0.3.3. pub(crate) fn invoke( &mut self, script: RuntimeScript<'_>, @@ -600,12 +606,54 @@ fn fresh_environment<'gc>( let actor_guid = event.actor.as_ref().map(|actor| actor.guid); set(ctx, env, "event", event_table(ctx, event)); + set(ctx, env, "assert", located_assert_operation(ctx)); + set(ctx, env, "error", located_error_operation(ctx)); set(ctx, env, "heal", heal_operation(ctx, actor_guid, &staged)); set(ctx, env, "send_chat", send_chat_operation(ctx, &staged)); set(ctx, env, "grant_xp", grant_xp_operation(ctx, &staged)); env } +/// Refuse a false value and carry the generated Lua line of the call site. +fn located_assert_operation<'gc>(ctx: Context<'gc>) -> Callback<'gc> { + Callback::from_fn(&ctx, |ctx, execution, stack| { + if stack.get(0).to_bool() { + return Ok(CallbackReturn::Return); + } + let error = if stack.get(1).is_nil() { + text(ctx, "assertion failed!").into_value(ctx).into() + } else { + Error::from(stack.get(1)) + }; + Err(error_with_location(ctx, &execution, error)) + }) +} + +/// Raise a Lua error that carries the generated Lua line of the call site. +fn located_error_operation<'gc>(ctx: Context<'gc>) -> Callback<'gc> { + Callback::from_fn(&ctx, |ctx, execution, stack| { + let error = Error::from(stack.get(0)); + Err(error_with_location(ctx, &execution, error)) + }) +} + +fn error_with_location<'gc>( + ctx: Context<'gc>, + execution: &Execution<'gc, '_>, + error: Error<'gc>, +) -> Error<'gc> { + let Some(frame) = execution.upper_lua_frame() else { + return error; + }; + let message = match error { + Error::Lua(error) => error.to_string(), + Error::Runtime(error) => error.to_string(), + }; + text(ctx, &format!("line {}: {message}", frame.current_line)) + .into_value(ctx) + .into() +} + /// The environment's standard library, rebuilt name by name from [`ALLOWED_GLOBALS`] and /// [`ALLOWED_LIBRARY_MEMBERS`]. /// @@ -686,21 +734,24 @@ fn heal_operation<'gc>( staged: &Rc>>, ) -> Callback<'gc> { let staged = Rc::clone(staged); - Callback::from_fn(&ctx, move |ctx, _execution, mut stack| { - let (entity, amount): (Value, Value) = stack.consume(ctx)?; - let target = entity_argument(ctx, "heal", "target", entity)?; - let amount = amount_argument(ctx, "heal", amount)?; - stage( - ctx, - &staged, - "heal", - StagedEffect::Heal { - healer_guid: actor_guid.unwrap_or(0), - target_guid: target.guid, - amount, - }, - )?; - Ok(CallbackReturn::Return) + Callback::from_fn(&ctx, move |ctx, execution, mut stack| { + (|| { + let (entity, amount): (Value, Value) = stack.consume(ctx)?; + let target = entity_argument(ctx, "heal", "target", entity)?; + let amount = amount_argument(ctx, "heal", amount)?; + stage( + ctx, + &staged, + "heal", + StagedEffect::Heal { + healer_guid: actor_guid.unwrap_or(0), + target_guid: target.guid, + amount, + }, + )?; + Ok(CallbackReturn::Return) + })() + .map_err(|error| error_with_location(ctx, &execution, error)) }) } @@ -711,22 +762,25 @@ fn send_chat_operation<'gc>( staged: &Rc>>, ) -> Callback<'gc> { let staged = Rc::clone(staged); - Callback::from_fn(&ctx, move |ctx, _execution, mut stack| { - let (entity, text): (Value, Value) = stack.consume(ctx)?; - let recipient = player_argument(ctx, "send_chat", "recipient", entity)?; - let text = text_argument(ctx, "send_chat", text)?; - let message = crate::chat::normalized_message(&text) - .ok_or_else(|| host_error(ctx, "send_chat", "the message is empty"))?; - stage( - ctx, - &staged, - "send_chat", - StagedEffect::SendChat { - recipient_guid: recipient.guid, - message, - }, - )?; - Ok(CallbackReturn::Return) + Callback::from_fn(&ctx, move |ctx, execution, mut stack| { + (|| { + let (entity, text): (Value, Value) = stack.consume(ctx)?; + let recipient = player_argument(ctx, "send_chat", "recipient", entity)?; + let text = text_argument(ctx, "send_chat", text)?; + let message = crate::chat::normalized_message(&text) + .ok_or_else(|| host_error(ctx, "send_chat", "the message is empty"))?; + stage( + ctx, + &staged, + "send_chat", + StagedEffect::SendChat { + recipient_guid: recipient.guid, + message, + }, + )?; + Ok(CallbackReturn::Return) + })() + .map_err(|error| error_with_location(ctx, &execution, error)) }) } @@ -736,20 +790,23 @@ fn grant_xp_operation<'gc>( staged: &Rc>>, ) -> Callback<'gc> { let staged = Rc::clone(staged); - Callback::from_fn(&ctx, move |ctx, _execution, mut stack| { - let (entity, amount): (Value, Value) = stack.consume(ctx)?; - let character = player_argument(ctx, "grant_xp", "recipient", entity)?; - let amount = amount_argument(ctx, "grant_xp", amount)?; - stage( - ctx, - &staged, - "grant_xp", - StagedEffect::GrantXp { - character_guid: character.guid, - amount, - }, - )?; - Ok(CallbackReturn::Return) + Callback::from_fn(&ctx, move |ctx, execution, mut stack| { + (|| { + let (entity, amount): (Value, Value) = stack.consume(ctx)?; + let character = player_argument(ctx, "grant_xp", "recipient", entity)?; + let amount = amount_argument(ctx, "grant_xp", amount)?; + stage( + ctx, + &staged, + "grant_xp", + StagedEffect::GrantXp { + character_guid: character.guid, + amount, + }, + )?; + Ok(CallbackReturn::Return) + })() + .map_err(|error| error_with_location(ctx, &execution, error)) }) } @@ -1068,6 +1125,17 @@ local roster = table.concat(shouted, ",") if #roster > 0 then grant_xp(event.actor, 25) end "#; + /// The SAME workload, as the pinned TypeScript toolchain actually emits it. + /// + /// `datascripts/runtime-scripts/fuel-workload.ts` is the source; `packages build` compiles + /// Package scripts with exactly this toolchain, and + /// `datascripts/tests/runtime-scripts.test.ts` recompiles the workload and refuses these bytes + /// if they have gone stale. So the budget is sized against real transpiler output — the + /// `____tbl` guard, the inlined lua library, the `self` conventions and all — not against a + /// hand-written impression of it. + const GENERATED_SCRIPT: &str = + include_str!("../../datascripts/runtime-scripts/fuel-workload.lua"); + const PLAYER_GUID: u64 = 7; const CREATURE_GUID: u64 = 9; @@ -1267,6 +1335,34 @@ if #roster > 0 then grant_xp(event.actor, 25) end assert!(!failure.message.is_empty()); } + #[test] + /// Piccolo gives a caller frame only to an active callback. After `Executor::step` returns, + /// its public API exposes no native-fault or fuel-exhaustion frame. Keep the stable context + /// rather than inventing a line or instrumenting generated Lua, which would move its Fuel + /// Budget measurement. + fn native_vm_and_fuel_diagnostics_keep_the_stable_invocation_context() { + let mut host = RuntimeScriptHost::new(); + for (name, source, kind) in [ + ( + "native-fault", + "local missing = nil\nreturn missing.value", + FailureKind::Runtime, + ), + ("spin", "while true do end", FailureKind::Fuel), + ] { + let failure = host + .invoke(script(name, source), &unattended("on_damage_taken")) + .expect_err("the Runtime Script must fail"); + assert_eq!(failure.script, name); + assert_eq!(failure.event, "on_damage_taken"); + assert_eq!(failure.kind, kind); + assert!(!failure.message.is_empty()); + let report = failure.to_string(); + assert!(report.contains(name), "{report}"); + assert!(report.contains("on_damage_taken"), "{report}"); + } + } + #[test] fn a_diagnostic_message_is_capped_however_much_the_script_raises() { let mut host = RuntimeScriptHost::new(); @@ -1645,6 +1741,11 @@ if #roster > 0 then grant_xp(event.actor, 25) end "`{source}` must report `{fault}`, got `{}`", failure.message ); + assert!( + failure.message.contains("line 1"), + "`{source}` must name generated Lua line 1, got `{}`", + failure.message + ); } } @@ -1915,6 +2016,49 @@ if #roster > 0 then grant_xp(event.actor, 25) end ); } + /// The two facts `datascripts/runtime-scripts/lua-emit.cjs` rewrites generated Lua against. + /// If either stops holding, that rewrite is either unnecessary or no longer sufficient, and the + /// Package build has to be revisited before the next Runtime Script ships. + #[test] + fn only_a_trailing_table_constructor_leaks_and_a_one_parameter_passthrough_stops_it() { + let mut host = RuntimeScriptHost::new(); + let event = engagement(); + // 1 when the second parameter arrived nil, which is what a correct call gives. + let preamble = "local function seen(a, b) if b == nil then return 1 end return 2 end\n"; + for (shape, source) in [ + ( + "a table constructor that is not the last argument", + "grant_xp(event.actor, seen({7, 8, 9}, nil))", + ), + ( + "a table constructor passed through a one-parameter function", + "local function pass(t) return t end\ngrant_xp(event.actor, seen(pass({7, 8, 9})))", + ), + ] { + assert_eq!( + granted_amounts( + &committed(&mut host, &event, &format!("{preamble}{source}")).unwrap() + ), + [1], + "{shape} must not leak" + ); + } + // Parentheses do NOT stop it: the leak is in how the call site counts its arguments, not + // in adjusting a multiple-value expression, so the obvious one-token fix is not one. + assert_eq!( + granted_amounts( + &committed( + &mut host, + &event, + &format!("{preamble}grant_xp(event.actor, seen(({{7, 8, 9}})))") + ) + .unwrap() + ), + [2], + "parenthesising the constructor must still leak" + ); + } + #[test] fn the_shim_supplies_the_table_concat_piccolo_lacks() { let mut host = RuntimeScriptHost::new(); @@ -1959,6 +2103,88 @@ if #roster > 0 then grant_xp(event.actor, 25) end .expect("fifty times the representative workload must still fit"); } + /// Real transpiler output runs on this Host, answers its caller, and fits the Fuel Budget. + /// + /// The three claims are one test on purpose: a chunk that compiles but cannot reach a Host Verb + /// is not evidence that the toolchain works, and a chunk that works but does not fit the budget + /// is not evidence that the budget is sized for one. + #[test] + fn the_generated_representative_script_runs_answers_and_fits_the_fuel_budget() { + let mut host = RuntimeScriptHost::new(); + let event = engagement(); + + let invocation = host + .invoke(script("fire_nova.generated", GENERATED_SCRIPT), &event) + .expect("generated Lua must run inside the Fuel Budget"); + + // "UNIT1".."UNIT20" joined by 19 commas is 130 characters. The chunk's own answer proves + // the lua library, the string work and the join all ran, not just the Host Verb. + assert_eq!(invocation.answer, Some(130.0)); + let mut sink = FakeEffects::default(); + invocation.effects.commit(&mut sink); + assert_eq!(granted_amounts(&sink.committed), [25]); + } + + /// A diagnostic's line number is a line of the GENERATED Lua — the bytes `game_script.source` + /// holds and an Operator can read — never a line of the TypeScript it came from. There is no + /// source map, and a number that pointed into a file no Shard holds would be worse than none. + /// + #[test] + fn a_syntax_failure_names_a_line_of_the_generated_lua_not_of_its_typescript() { + let mut host = RuntimeScriptHost::new(); + let generated_lines = GENERATED_SCRIPT.lines().count(); + let broken = format!("{GENERATED_SCRIPT}local a = 1 +\n"); + + let diagnostic = host + .invoke( + script("fire_nova.generated", &broken), + &unattended("on_login"), + ) + .expect_err("the chunk does not parse"); + + assert_eq!(diagnostic.kind, FailureKind::Syntax); + assert!( + diagnostic + .message + .contains(&format!("line {}", generated_lines + 1)), + "the diagnostic must name line {} of the generated Lua: {}", + generated_lines + 1, + diagnostic.message + ); + // The lua library prologue and the guard put the generated Lua well past the length of the + // TypeScript, so this number could not be a source-mapped one. + assert!(generated_lines > 25); + } + + #[test] + fn a_runtime_failure_names_its_line_in_the_generated_lua() { + let mut host = RuntimeScriptHost::new(); + for fault in ["error('broken')", "assert(false, 'broken')"] { + let generated = format!( + "local function ____tbl(t) return t end\n\ + local function script()\n\ + local amount = 1\n\ + {fault}\n\ + end\n\ + return script()\n" + ); + + let diagnostic = host + .invoke( + script("fire_nova.generated", &generated), + &unattended("on_login"), + ) + .expect_err("the generated Lua raises an error"); + + assert_eq!(diagnostic.kind, FailureKind::Runtime); + assert!( + diagnostic.message.contains("line 4"), + "the diagnostic must name generated Lua line 4: {}", + diagnostic.message + ); + } + } + /// A Runtime Script that reaches the host again through an effect it staged must be refused, /// not panic the reducer it is running inside. Event bindings make that reachable; the guard /// is here so it is a refusal from the first day it can happen. diff --git a/packages/README.md b/packages/README.md index c20d7d26d..b8b6e4b21 100644 --- a/packages/README.md +++ b/packages/README.md @@ -14,7 +14,27 @@ path outside it, naming the Package, the file and the line. A Package may also have a DATA half: `data/.generated/*.json`, the Package Deltas a Datascript generates, which the importer reapplies after every base import. `fire_nova/` is the worked example; its Datascript lives at `datascripts/src/fire_nova/spells.ts`, because only artifacts belong inside -a Package folder. Any one half — `src/`, `client/` or `data/` — is a valid Package on its own. +a Package folder. Any one half — `src/`, `client/`, `scripts/` or `data/` — is a valid Package on +its own. + +A Package may also ship RUNTIME SCRIPTS: `scripts/*.ts` and `scripts/*.lua`, Lua the Module runs on +a gameplay event inside the Runtime Script Host. These sources DO live inside the Package, unlike a +Datascript, because they are the Package's own content rather than a description of client-derived +data. `lyracore packages build` compiles them into one Script Artifact at +`data/.generated/.script.json`. `fire_nova/scripts/ember_echo.ts` is the worked example; it +ships switched off. Each file opens with its Script Directives: + +```ts +// @event on_cast_resolved +// @id 100200 +// @priority 10 +// @enabled false +``` + +`@event` and `@id` are required, `@priority` defaults to 0 and `@enabled` to true. A TypeScript +script declares `function script(): number | void`; what it returns is the Script Answer the asking +Package reads back. See `docs/development-cli.md` for the Host API, the identifier band and how the +build checks an event name against the Module. A Package that must change a stock FrameXML or GlueXML file declares the edit in `client/ui-transforms.json` instead of shipping a whole replacement under `client/mpq/`. Each entry diff --git a/packages/fire_nova/README.md b/packages/fire_nova/README.md index fbec75837..ffd7d245c 100644 --- a/packages/fire_nova/README.md +++ b/packages/fire_nova/README.md @@ -1,18 +1,21 @@ # fire_nova -The worked example of a DATA-ONLY Package: no Rust, no client half, one Package Delta. +The worked example of a DATA-ONLY Package: no Rust, no client half, one Package Delta and one +Script Artifact. -Its content is authored in `datascripts/src/fire_nova/spells.ts`, which clones a real spell into a -five-rank ladder of Package spells. Read that file — it is the example. +Its data is authored in `datascripts/src/fire_nova/spells.ts`, which clones a real spell into a +five-rank ladder of Package spells. Its one Runtime Script is `scripts/ember_echo.ts`, which ships +switched off. Read those two files — they are the example. -Nothing under `data/` is committed. `lyracore packages build` runs the Datascript and writes -`data/.generated/spell.json`, which the importer then reapplies after every spell import. Build it -yourself: +Nothing under `data/` is committed. `lyracore packages build` runs the Datascript, compiles the +Runtime Scripts and writes both artifacts; the importer reapplies them after every spell import. +Build them yourself: ``` lyracore-importer --dbc --spell-snapshot datascripts/generated/base-snapshot.json bun run datascripts/src/fire_nova/spells.ts -lyracore-delta-check packages/fire_nova/data/.generated/spell.json +bun run datascripts/runtime-scripts/build-scripts.ts fire_nova +lyracore-delta-check packages/fire_nova/data/.generated/*.json ``` An unmodified client shows no tooltip for a Package spell: it renders spells from its own diff --git a/packages/fire_nova/scripts/ember_echo.ts b/packages/fire_nova/scripts/ember_echo.ts new file mode 100644 index 000000000..b77886db0 --- /dev/null +++ b/packages/fire_nova/scripts/ember_echo.ts @@ -0,0 +1,25 @@ +// @event on_cast_resolved +// @id 100200 +// @enabled false +// +// The worked example of a Runtime Script. It ships SWITCHED OFF: `on_cast_resolved` fires for every +// cast on a realm, and a demonstration should not talk to every player who casts anything. Switch +// it on by changing the directive above and running `lyracore packages build`. +// +// What it shows: reading the event's entities, a Host Verb, and a Script Answer. + +/// How loudly the echo lands, as a share of the caster's remaining health. +function echoShare(caster: Entity): number { + if (caster.max_health === 0) return 0; + return Math.floor((caster.health * 100) / caster.max_health); +} + +function script(): number | void { + const caster = event.actor; + if (!caster || !caster.is_player) return; + + const share = echoShare(caster); + const words = ["fire", "nova", "echo"]; + send_chat(caster, words.join(" ") + ": " + share); + return share; +}