Skip to content

refactor(core,cli): unified watch coordinator behind one seam, two adapters #212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tonyketcham merged 2 commits into from
Jul 18, 2026
Merged

refactor(core,cli): unified watch coordinator behind one seam, two adapters #212

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
995b61b
refactor(core,cli): unified watch coordinator behind one seam, two ad…
tonyketcham Jul 17, 2026
3f8efdd
Merge branch 'main' into stack/tonyketcham/feat/unified-watch-effort-…
mergify[bot] Jul 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 44 additions & 46 deletions docs/local-dev-loop.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Flatbread's local loop has four moving parts:
content paths.
2. **Schema rebuild** — `@flatbread/core` turns loaded records and refs into a
GraphQL schema after ID/ref validation.
3. **Codegen refresh** — `flatbread codegen --watch` regenerates TypeScript
artifacts when config, content, or GraphQL documents change.
3. **Codegen refresh** — the unified watcher regenerates TypeScript artifacts
when config, content, or GraphQL documents change.
4. **Framework restart / refresh** — `flatbread start -- <framework command>`
runs the GraphQL server beside your app command.

Expand All @@ -26,86 +26,84 @@ cd examples/nextjs
pnpm exec flatbread codegen --verbose
```

For development, use two terminals. This path avoids the example package's
HTTPS convenience script and keeps the Flatbread GraphQL endpoint on plain HTTP
port `5057`.
For development, use the unified watcher. This path avoids the example
package's HTTPS convenience script and keeps the Flatbread GraphQL endpoint on
plain HTTP port `5057`.

```bash
# terminal 1 — regenerate TypeScript artifacts
pnpm exec flatbread codegen --watch --verbose
```

```bash
# terminal 2 — serve GraphQL + Next.js without HTTPS for headless/dev agents
# serve GraphQL, refresh generated artifacts, and run Next.js without HTTPS
pnpm exec flatbread start --watch -- next dev --turbopack
```

Expected behavior:

- One unified watcher owns config/content/document classification, GraphQL
hot-swaps, and generated artifact refreshes.
- Editing a `.graphql` document or a content/config file refreshes
`generated/graphql.ts`.
`generated/graphql.ts`; do not run `flatbread codegen --watch` beside it.
- The generated content-model types and prototype read API are refreshed by
the same codegen command.
- The running GraphQL endpoint at `http://localhost:5057/graphql` hot-swaps
valid content and config generations without restarting the framework.

## Current reload matrix

| Change | Codegen watcher behavior | Running GraphQL server | Framework app | Action required today |
| --------------------------------------- | --------------------------------------- | ------------------------------------------------ | -------------------------------------------------------- | -------------------------------------------------------- |
| Markdown/YAML field value | Refreshes types if watched path matches | Hot-swaps after validation | Keeps rendering whatever the endpoint returns | None; framework refresh remains explicit |
| New/removed content file | Refreshes types if watched path matches | Hot-swaps after validation | Keeps rendering whatever the endpoint returns | None; framework refresh remains explicit |
| `.graphql` document | Regenerates operation types | No restart unless query text used by app changed | Framework dev server normally recompiles importing files | No Flatbread restart unless app code needs it |
| `flatbread.config.*` content/ref change | Reloads config and refreshes types | Rebuilds and hot-swaps after validation | Keeps rendering whatever the endpoint returns | None; framework refresh remains explicit |
| Transformer/source package code | Does not rebuild package code | Keeps previous imported package code | May keep previous imported package code | Rebuild/watch package separately, rerun codegen, restart |
| `generated/graphql.ts` | Output of codegen | No direct effect | Framework dev server recompiles imports | No Flatbread restart |
| Change | Unified watcher behavior | Running GraphQL server | Framework app | Action required today |
| --------------------------------------- | ------------------------------------------------------------------------------ | ------------------------------------------------ | -------------------------------------------------------- | -------------------------------------------------------- |
| Markdown/YAML field value | Atomically reindexes/hot-swaps, then refreshes codegen | Hot-swaps after validation | Keeps rendering whatever the endpoint returns | None; framework refresh remains explicit |
| New/removed content file | Atomically reindexes/hot-swaps, then refreshes codegen | Hot-swaps after validation | Keeps rendering whatever the endpoint returns | None; framework refresh remains explicit |
| `.graphql` document | Refreshes codegen only | No restart unless query text used by app changed | Framework dev server normally recompiles importing files | No Flatbread restart unless app code needs it |
| `flatbread.config.*` content/ref change | Reloads config/matchers, atomically rebuilds/hot-swaps, then refreshes codegen | Rebuilds and hot-swaps after validation | Keeps rendering whatever the endpoint returns | None; framework refresh remains explicit |
| Transformer/source package code | Does not rebuild package code | Keeps previous imported package code | May keep previous imported package code | Rebuild/watch package separately, rerun codegen, restart |
| `generated/graphql.ts` | Output of codegen | No direct effect | Framework dev server recompiles imports | No Flatbread restart |

## Failure semantics today

- If content becomes invalid while `flatbread codegen --watch` is running, the
watcher logs the validation/codegen error and keeps watching. Existing
generated files are left as-is until a later successful regeneration.
- Rejected config, content, or codegen phases are logged and keep the unified
watch loop alive. Existing generated files are left as-is until a later
successful regeneration.
- In one-shot mode (`flatbread codegen` without `--watch`), validation or
codegen errors exit non-zero and do not prove the live server changed.
- If the running GraphQL server was started before the invalid edit, it keeps
serving the schema/data it already loaded. Restarting it surfaces the
validation error at startup.
- In unified watch mode, invalid candidates are rejected atomically: generated
artifacts and the live GraphQL server remain on the previous committed graph.
A codegen failure does not undo an already committed GraphQL generation, and
edits received during an in-flight generation are queued for the next
serialized batch.

## Draft unified watch design (implemented)
## Unified watch coordinator contract

The unified loop should eventually make this one command:
The unified loop is started with:

```bash
flatbread start --watch -- next dev --turbopack
pnpm exec flatbread start --watch -- next dev --turbopack
```

Design contract:

1. Watch the same content/config/document paths that `flatbread codegen --watch`
already derives from `LoadedFlatbreadConfig`.
2. On content changes, reload records, rerun ID/ref/cardinality validation,
rebuild the schema, refresh generated TypeScript, and swap the GraphQL
server schema only if the new graph validates. If validation fails, keep the
previous schema active and log the failure.
3. On config changes, reload config, rebuild watch globs, rebuild schema,
refresh generated TypeScript, and restart only the Flatbread GraphQL server
boundary if a safe hot swap is not possible. A safe hot swap means replacing
schema/data without losing the child framework process, open port, or
in-flight request handling state.
4. On GraphQL document changes, refresh generated TypeScript only.
5. Keep framework restarts explicit. Flatbread should not assume every
framework can be restarted safely; it should document whether the app command
is left running, restarted, or expected to recompile through its own dev
server.
The coordinator contract is:

1. A single coordinator classifies config, content, and document events, then
serializes all rebuild and codegen phases.
2. Content changes reindex records and atomically hot-swap the GraphQL schema
before refreshing generated TypeScript.
3. Config changes reload the config and matchers, rebuild and atomically
hot-swap the schema, then refresh generated TypeScript.
4. GraphQL document changes refresh generated TypeScript without reindexing
content.
5. Rejected phases emit an error but do not stop the loop. Committed GraphQL
generations are not rolled back when a later codegen phase fails.
6. Events received during an in-flight generation are queued and processed
serially.
7. Framework restarts remain explicit. Flatbread keeps the framework child
process running and relies on its own dev server to recompile or refresh.

## Known limitations

- `flatbread start --watch` hot-swaps valid content/config generations; invalid
candidates leave the prior schema active.
- `flatbread codegen --watch` is a long-running process; do not use it in CI or
one-shot scripts.
- Unified watch mode is a long-running process; do not use it in CI or one-shot
scripts.
- The Next.js example `pnpm dev` includes `--https` for local convenience, but
the Flatbread GraphQL endpoint remains documented as HTTP on `5057`. In
headless environments prefer `pnpm exec flatbread start -- next dev --turbopack`.
Expand Down
184 changes: 93 additions & 91 deletions packages/codegen/src/generator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,12 @@ import { readFile, writeFile } from 'node:fs/promises';
import kleur from 'kleur';
// @ts-ignore - chokidar types will be available after npm install
import chokidar from 'chokidar';
import { generateSchema } from '@flatbread/core';
import type { LoadedFlatbreadConfig } from '@flatbread/core';
import { createWatchCoordinator, generateSchema } from '@flatbread/core';
import type {
LoadedFlatbreadConfig,
WatchCoordinatorResult,
WatchEventType,
} from '@flatbread/core';
import { generateInstallCommand } from '@flatbread/utils';
import type { CodegenOptions, CodegenResult, CodegenCache } from './types.js';
import { DEFAULT_CODEGEN_OPTIONS, PLUGIN_PRESETS } from './types.js';
Expand All @@ -40,6 +44,29 @@ function emitMissingDepsWarning(missingDeps: string[]) {
);
}

function deriveOptionsFromConfig(
loadedConfig: LoadedFlatbreadConfig,
previous: CodegenOptions
): CodegenOptions {
const cfg = loadedConfig.codegen || {};
return {
// Always enable in watch
enabled: true,
// Prefer latest config for dynamic fields changed via config
outputDir: cfg.outputDir ?? DEFAULT_CODEGEN_OPTIONS.outputDir,
outputFile: cfg.outputFile ?? DEFAULT_CODEGEN_OPTIONS.outputFile,
documents: cfg.documents ?? [],
plugins: cfg.plugins ?? previous.plugins,
pluginConfig: cfg.pluginConfig ?? previous.pluginConfig,
schema: cfg.schema ?? previous.schema,
codegenConfig: cfg.codegenConfig ?? previous.codegenConfig,
// Preserve runtime flags
watch: true,
cache: previous.cache,
preset: previous.preset,
};
}

/**
* Generate TypeScript types from a GraphQL schema using GraphQL Code Generator
*/
Expand Down Expand Up @@ -599,33 +626,12 @@ export async function watchAndGenerate(
// Maintain a mutable set of options that can be refreshed when the config changes
let currentOptions: CodegenOptions = { ...options };

// Helper to derive effective codegen options from the latest config
const deriveOptionsFromConfig = (
loadedConfig: LoadedFlatbreadConfig,
previous: CodegenOptions
): CodegenOptions => {
const cfg = loadedConfig.codegen || {};
return {
// Always enable in watch
enabled: true,
// Prefer latest config for dynamic fields changed via config
outputDir: cfg.outputDir ?? DEFAULT_CODEGEN_OPTIONS.outputDir,
outputFile: cfg.outputFile ?? DEFAULT_CODEGEN_OPTIONS.outputFile,
documents: cfg.documents ?? [],
plugins: cfg.plugins ?? previous.plugins,
pluginConfig: cfg.pluginConfig ?? previous.pluginConfig,
schema: cfg.schema ?? previous.schema,
codegenConfig: cfg.codegenConfig ?? previous.codegenConfig,
// Preserve runtime flags
watch: true,
cache: previous.cache,
preset: previous.preset,
};
};

// Initial generation using the current options
await generateTypes(schema, config, currentOptions);

let currentConfig = config;
let currentSchema = schema;

// Set up file watchers
const patterns = flattenFlatbreadWatchPatterns(
deriveFlatbreadWatchPatterns(config, currentOptions)
Expand All @@ -650,81 +656,76 @@ export async function watchAndGenerate(
persistent: true,
});

let regenerating = false;

const regenerateTypes = async (path: string, event: string) => {
if (regenerating) {
return; // Avoid concurrent regenerations
}

regenerating = true;

try {
console.log(kleur.yellow(`\n📝 ${event}: ${path}`));
console.log(kleur.blue('🔄 Regenerating schema and types...'));

let currentConfig = config;

// If a config file changed, reload the configuration
if (path.includes('flatbread.config.')) {
try {
const { loadConfig } = await import('@flatbread/config');
const configResult = await loadConfig({ cwd: process.cwd() });

if (configResult.config) {
currentConfig = configResult.config;
console.log(kleur.dim('🔧 Configuration reloaded'));
// Refresh codegen options from the updated config so changes like outputFile/outputDir/documents are applied
currentOptions = deriveOptionsFromConfig(
currentConfig,
currentOptions
);
}
} catch (error) {
console.warn(
kleur.yellow(
`⚠️ Failed to reload config, using existing: ${
error instanceof Error ? error.message : 'Unknown error'
}`
)
);
}
}

// Regenerate the schema first since the source files may have changed
const newSchema = await generateSchema({ config: currentConfig });

// Generate types with the new schema
const coordinator = createWatchCoordinator({
config: currentConfig,
documentPatterns: (cfg) => cfg.codegen?.documents ?? [],
loadConfig: async () => {
const { loadConfig } = await import('@flatbread/config');
return (await loadConfig({ cwd: process.cwd() })).config!;
},
applyConfig: async (cfg) => {
currentConfig = cfg;
currentOptions = deriveOptionsFromConfig(cfg, currentOptions);
currentSchema = await generateSchema({ config: cfg });
return { status: 'committed' };
},
reindexContent: async () => {
currentSchema = await generateSchema({ config: currentConfig });
return { status: 'committed' };
},
refreshCodegen: async () => {
const result = await generateTypes(
newSchema,
currentSchema,
currentConfig,
currentOptions
);

if (result.success) {
console.log(kleur.green('✅ Types regenerated successfully'));
} else {
console.error(
kleur.red('❌ Failed to regenerate types:'),
result.error
);
if (!result.success) {
throw new Error(result.error ?? 'Codegen failed');
}
} catch (error) {
console.error(kleur.red('❌ Error during regeneration:'), error);
} finally {
regenerating = false;
},
});

const resultMessage = (result: WatchCoordinatorResult): void => {
if (result.status === 'rejected') {
const phase =
result.kind === 'config'
? 'reload config'
: result.kind === 'content'
? 'reindex content'
: result.kind === 'documents'
? 'refresh documents'
: 'regenerate types';
console.error(kleur.red(`❌ Failed to ${phase}:`), result.error);
return;
}

if (result.kind === 'config') {
console.log(kleur.dim('🔧 Configuration reloaded'));
} else if (result.kind === 'codegen' || result.kind === 'documents') {
console.log(kleur.green('✅ Types regenerated successfully'));
}
};

coordinator.subscribe(resultMessage);

const eventLabels: Record<WatchEventType, string> = {
create: 'File added',
update: 'File changed',
delete: 'File removed',
};
const pushEvent = (path: string, type: WatchEventType) => {
console.log(kleur.yellow(`\n📝 ${eventLabels[type]}: ${path}`));
console.log(kleur.blue('🔄 Regenerating schema and types...'));
coordinator.push([{ path, type }]);
};

// Set up event handlers
watcher
.on('add', (path: string) => regenerateTypes(path, 'File added'))
.on('change', (path: string) => regenerateTypes(path, 'File changed'))
.on('unlink', (path: string) => regenerateTypes(path, 'File removed'))
.on('addDir', (path: string) => regenerateTypes(path, 'Directory added'))
.on('unlinkDir', (path: string) =>
regenerateTypes(path, 'Directory removed')
)
.on('add', (path: string) => pushEvent(path, 'create'))
.on('change', (path: string) => pushEvent(path, 'update'))
.on('unlink', (path: string) => pushEvent(path, 'delete'))
.on('addDir', (path: string) => pushEvent(path, 'create'))
.on('unlinkDir', (path: string) => pushEvent(path, 'delete'))
.on('error', (error: Error) =>
console.error(kleur.red('Watcher error:'), error)
)
Expand All @@ -733,6 +734,7 @@ export async function watchAndGenerate(
// Handle graceful shutdown
const shutdown = () => {
console.log(kleur.yellow('\n🛑 Shutting down watcher...'));
void coordinator.dispose();
watcher.close();
process.exit(0);
};
Expand Down
13 changes: 13 additions & 0 deletions packages/core/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,19 @@ export type {
PathClassification,
RecordsByCollection,
} from './records';
export {
createWatchCoordinator,
type WatchAdapterGeneration,
type WatchContentChange,
type WatchCoordinator,
type WatchCoordinatorOptions,
type WatchCoordinatorResult,
type WatchEvent,
type WatchEventType,
type WatchGenerationKind,
type WatchScheduler,
type WatchTimer,
} from './watch/coordinator';

export * from './types';
export { FlatbreadProvider } from './providers/base';
Loading
Loading