Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions typescript/vercel-ai-sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,23 @@ for model, tool, streaming and MCP configuration. That guide currently uses the
SDK-owned `@temporalio/ai-sdk` name; this repository owns the new
`@temporalio/vercel-ai-sdk` coordinate.

The plugin registers Activities named `invokeModel`, `invokeModelStreaming` and
`invokeEmbeddingModel`, plus `<server>-listTools` and `<server>-callTool` for
each MCP server. A different Activity with one of these names now fails with
`Duplicate registration of Activity '<name>'`; earlier versions kept the last
one without a warning. If it is registered before `AiSdkPlugin` (in the Worker's
`activities` or by an earlier plugin), `Worker.create` fails. If a plugin
applied after `AiSdkPlugin` replaces one, `worker.run()` fails before the Worker
polls. `Worker.create` applies the Worker's `plugins` in order, then the
`NativeConnection`'s.

To fix a collision with your own Activity or an MCP server name, rename one of
them. `StrandsPlugin` (`@temporalio/strands-agents`) also registers
`invokeModel` and `invokeModelStreaming`, and neither plugin can rename them, so
run the two integrations on separate Workers and task queues. Two `AiSdkPlugin`
instances count as the same only when they share the `modelProvider` object
and, for each MCP server name, the same factory function and idle timeout, as
when both are built from one options object. Two equal object literals are

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 (non-blocking): 'Two equal object literals are refused' says the opposite of what the code does in the common case. Equal option literals pass whenever they reference the same provider and factory objects. The PR's own test accepts new AiSdkPlugin({ modelProvider: provider, mcpClientFactories: { files } }) written out twice (test-ai-sdk.ts:994, 'the same options twice'). I chained configureWorker the way Worker.create does. Two { modelProvider: openai } literals (the @ai-sdk/openai singleton) are accepted, and so are two { modelProvider: provider, mcpClientFactories: { files } } literals. What is refused is a provider or factory created once per plugin: createOpenAI({ apiKey: 'k' }) in each literal fails on invokeModel, and an inline files: async () => ... in each fails on files-listTools. Also, 'for each MCP server name' reads as if both instances must register the same servers. In fact two instances with the same provider and disjoint servers ({ a } and { b }) are accepted, and the Worker gets both servers' Activities. Suggested: 'Two AiSdkPlugin instances count as the same only when they share the modelProvider object and, for each MCP server name both register, the same factory function and idle timeout. A provider or factory created separately for each instance is refused even when its settings are equal, for example createOpenAI({ apiKey }) called once per plugin, or an async () => ... factory written inline in each options object.' The sentence matches #202's README. There it holds for the models map, because StrandsPlugin compares that object itself.

refused.

Development details are in [typescript/README.md](../README.md).
87 changes: 85 additions & 2 deletions typescript/vercel-ai-sdk/src/__tests__/test-ai-sdk.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ import {
OpenTelemetryWorkflowClientInterceptor,
} from '@temporalio/interceptors-opentelemetry';
import { workflowInterceptorModules } from '@temporalio/testing';
import { bundleWorkflowCode, DefaultLogger, Worker as CoreWorker } from '@temporalio/worker';
import type { InjectedSinks } from '@temporalio/worker';
import { SimplePlugin } from '@temporalio/plugin';
import { bundleWorkflowCode, DefaultLogger, NativeConnection, Worker as CoreWorker } from '@temporalio/worker';
import type { InjectedSinks, WorkerOptions } from '@temporalio/worker';
import type { BaseContext } from '@temporalio/test-helpers';
import {
test as anyTest,
Expand Down Expand Up @@ -962,6 +963,88 @@ const createTestMcpClient = async () => {
return createMCPClient({ transport: clientTransport });
};

test('configureWorker refuses to replace a different Activity with the same name', (t) => {
const provider = new TestProvider(helloWorkflowGenerator());
const files = async () => ({}) as any;
const base = { taskQueue: 'tq' } as WorkerOptions;
const duplicate = (name: string) => ({
instanceOf: TypeError,
message: new RegExp(`^Duplicate registration of Activity '${name}': AiSdkPlugin would replace`),
});

// A caller Activity, another instance's provider, or another instance's MCP
// client factory under the same server name is refused.
const callerActivities = { ...base, activities: { invokeModel: async () => 'caller' } };
t.throws(() => new AiSdkPlugin({ modelProvider: provider }).configureWorker(callerActivities), duplicate('invokeModel'));
const withMcp = new AiSdkPlugin({ modelProvider: provider, mcpClientFactories: { files } }).configureWorker(base);
const otherProvider = new AiSdkPlugin({ modelProvider: new TestProvider(helloWorkflowGenerator()) });
t.throws(() => otherProvider.configureWorker(withMcp), duplicate('invokeModel'));
const otherFactory = new AiSdkPlugin({ modelProvider: provider, mcpClientFactories: { files: async () => ({}) as any } });
t.throws(() => otherFactory.configureWorker(withMcp), duplicate('files-listTools'));
// Another operation built from the same provider, registered under this name.
const aliased = { ...base, activities: { invokeModel: (createActivities(provider) as any).invokeEmbeddingModel } };
t.throws(() => new AiSdkPlugin({ modelProvider: provider }).configureWorker(aliased), duplicate('invokeModel'));

// configureWorker accepts the same instance twice (NativeConnection and
// Worker), the same options twice, and createActivities with the same
// provider. With workflowsPath, Worker.create still rejects a repeated
// AiSdkPlugin in the bundler: each one prepends its polyfill entry.
const plugin = new AiSdkPlugin({ modelProvider: provider, mcpClientFactories: { files } });
t.notThrows(() => plugin.configureWorker(plugin.configureWorker(base)));
t.notThrows(() => new AiSdkPlugin({ modelProvider: provider, mcpClientFactories: { files } }).configureWorker(withMcp));
const callerSet = { ...base, activities: createActivities(provider) };
t.notThrows(() => new AiSdkPlugin({ modelProvider: provider }).configureWorker(callerSet));
});

test('runWorker refuses an Activity that a later plugin replaced, before polling', async (t) => {
const connection = await NativeConnection.connect({ address: t.context.env.address });
// StrandsPlugin also registers invokeModel; this plugin stands in for it.
const later = new SimplePlugin({ name: 'later', activities: { invokeModel: async () => 'other' } });
const worker = await Worker.create({
connection,
namespace: t.context.env.namespace,
taskQueue: 'test-ai-later-plugin',
plugins: [new AiSdkPlugin({ modelProvider: new TestProvider(helloWorkflowGenerator()) }), later],
});
await t.throwsAsync(worker.run(), {
instanceOf: TypeError,
message: /^Duplicate registration of Activity 'invokeModel': a plugin applied after AiSdkPlugin replaced/,
});
t.is(worker.getState(), 'STOPPED');
// Throws if the Worker still holds the connection.
await connection.close();
});

test('runWorker starts the Worker at once, so runUntil stops it after its callback', async (t) => {
const connection = await NativeConnection.connect({ address: t.context.env.address });
// runUntil stops the Worker after its callback only if the Worker is already
// running. A callback that throws synchronously finishes before run() yields.
const alone = await Worker.create({
connection,
namespace: t.context.env.namespace,
taskQueue: 'test-ai-run-until-throws',
plugins: [new AiSdkPlugin({ modelProvider: new TestProvider(helloWorkflowGenerator()) })],
});
await t.throwsAsync(
alone.runUntil(() => {
throw new Error('boom');
}),
{ message: 'boom' }
);
t.is(alone.getState(), 'STOPPED');
// Each plugin's check runs before the Worker starts; none may yield.
const options = { modelProvider: new TestProvider(helloWorkflowGenerator()) };
const two = await Worker.create({
connection,
namespace: t.context.env.namespace,
taskQueue: 'test-ai-run-until-two',
plugins: [new AiSdkPlugin(options), new AiSdkPlugin(options)],
});
t.is(await two.runUntil(() => Promise.resolve('done')), 'done');
t.is(two.getState(), 'STOPPED');
await connection.close();
});

test('MCP tool schema survives activity serialization', async (t) => {
const { createWorker, executeWorkflow } = helpers(t);

Expand Down
4 changes: 3 additions & 1 deletion typescript/vercel-ai-sdk/src/activities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import { Context } from '@temporalio/activity';
import { WorkflowStreamClient } from '@temporalio/workflow-streams/client';
import { msOptionalToNumber, type Duration } from '@temporalio/common/lib/time';
import type { McpClientFactories, McpClientFactory } from './mcp';
import { withSources } from './activity-sources';

const encoder = new TextEncoder();

Expand Down Expand Up @@ -396,12 +397,13 @@ export function createActivities(
});
},
};
withSources(activities, [provider]);
if (mcpClientFactories !== undefined) {
const idleMs = msOptionalToNumber(options?.mcpConnectionIdleTimeout) ?? MCP_CONNECTION_IDLE_MS;
Object.entries(mcpClientFactories).forEach(([name, func]) => {
activities = {
...activities,
...activitiesForName(name, func, idleMs),
...withSources(activitiesForName(name, func, idleMs), [func, idleMs]),
};
});
}
Expand Down
92 changes: 92 additions & 0 deletions typescript/vercel-ai-sdk/src/activity-sources.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/**
* Lets the plugin refuse an Activity name that is already taken. The SDK's
* plugin merge (`{...existing, ...param}`) keeps the last Activity of each
* name, so without this check a plugin Activity silently replaces a different
* Activity with the same name, or a later plugin silently replaces it. @internal
*/
import type { Worker } from '@temporalio/worker';

/**
* What each plugin Activity is: the name it was built under (its operation)
* followed by what it depends on. Two Activities with the same operation and
* the same dependencies do the same thing, so one may replace the other.
*/
const activitySources = new WeakMap<object, readonly unknown[]>();

/**
* Records, for every Activity in `activities`, its own key and `sources`, and
* returns `activities`. The key keeps one operation, registered under another
* operation's name, from passing as that operation.
*/
export function withSources<T extends object>(activities: T, sources: readonly unknown[]): T {
for (const [operation, fn] of Object.entries(activities)) activitySources.set(fn, [operation, ...sources]);
return activities;
}

/**
* Throws if an Activity in `added` would replace a different Activity with the
* same name in `existing`. The same function, or one built for the same
* operation from the same sources, may replace itself: that is the same plugin
* instance, or one with the same options, registered twice (e.g. on the
* NativeConnection and the Worker).
*/
export function refuseReplacedActivities(plugin: string, existing: object | undefined, added: object): void {
// What the Worker registers: own enumerable keys with function values.
const registered = new Map(Object.entries(existing ?? {}).filter(([, fn]) => typeof fn === 'function'));
for (const [name, fn] of Object.entries(added)) {
const other = registered.get(name);
if (other === undefined || other === fn || sameSources(other, fn)) continue;
throw new TypeError(
`Duplicate registration of Activity '${name}': ${plugin} would replace a different Activity with this name. ` +
remedy(plugin)
);
}
}

/**
* For `runWorker`, which runs after every plugin's `configureWorker`: if a
* plugin applied after this one replaced an Activity in `added` with a
* different one, returns a promise that rejects with a `TypeError`. Before
* rejecting, it stops the Worker so that it never polls, and lets `next`
* release it as `run()` does, so `connection.close()` still works.
*
* Otherwise it returns `undefined` without yielding, so the caller can start
* the Worker in the same tick. `Worker.runUntil` calls its callback right
* after `run()` and stops the Worker afterwards only if it is already running;
* a Worker started later polls until something else stops it.
*/
export function refuseLaterReplacement(
plugin: string,
worker: Worker,
added: object,
next: (worker: Worker) => Promise<void>
): Promise<never> | undefined {
for (const [name, fn] of Object.entries(added)) {
const kept = worker.options.activities.get(name);
if (kept === undefined || kept === fn || sameSources(kept, fn)) continue;
const error = new TypeError(
`Duplicate registration of Activity '${name}': a plugin applied after ${plugin} replaced its Activity with this name. ` +
remedy(plugin)
);
// There is no public way to stop a Worker that has not run yet.
(worker as unknown as { nativeWorker: { initiateShutdown(): void } }).nativeWorker.initiateShutdown();
const fail = (): never => {
throw error;
};
return next(worker).then(fail, fail);
}
return undefined;
}

function remedy(plugin: string): string {
return (
'Rename or remove the other Activity, run the other integration on its own Worker and task queue, ' +
`or use one ${plugin} instance on this Worker.`
);
}

function sameSources(a: object, b: object): boolean {
const x = activitySources.get(a);
const y = activitySources.get(b);
return x !== undefined && y !== undefined && x.length === y.length && x.every((v, i) => v === y[i]);
}
32 changes: 31 additions & 1 deletion typescript/vercel-ai-sdk/src/plugin.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import type { ProviderV4 } from '@ai-sdk/provider';
import { SimplePlugin } from '@temporalio/plugin';
import type { BundleOptions } from '@temporalio/worker';
import type { BundleOptions, Worker, WorkerOptions } from '@temporalio/worker';
import type { Duration } from '@temporalio/common/lib/time';
import { createActivities, _evictMcpConnectionsForServer } from './activities';
import { refuseLaterReplacement, refuseReplacedActivities } from './activity-sources';
import type { McpClientFactories } from './mcp';

/**
Expand Down Expand Up @@ -60,6 +61,35 @@ export class AiSdkPlugin extends SimplePlugin {
});
}

/**
* Adds the model and MCP Activities to the Worker.
*
* @throws TypeError if the Worker already has a different Activity with one
* of these names: a caller Activity, another integration's Activity, or an
* `AiSdkPlugin` with a different `modelProvider`, MCP client factory or
* `mcpConnectionIdleTimeout`.
*/
override configureWorker(options: WorkerOptions): WorkerOptions {
refuseReplacedActivities('AiSdkPlugin', options.activities, this.options.activities as object);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 (non-blocking): the check only sees Activities registered before this plugin, so a plugin configured after AiSdkPlugin still silently replaces invokeModel and invokeModelStreaming. That includes every plugin on the NativeConnection, because Worker.create applies the Worker's plugins first and the connection's after them. The ordering comes from options.plugins = (options.plugins ?? []).concat(options.connection?.plugins ?? []) in @temporalio/worker 1.24.0, lib/worker.js:196. I probed this with real Worker.create calls against a dev server, pairing AiSdkPlugin with a SimplePlugin that registers invokeModel the way StrandsPlugin does on main. With AiSdkPlugin on the Worker and Strands on the connection, Worker.create succeeds and the compiled worker.options.activities.get('invokeModel') is Strands' handler. plugins: [ai, strands] on the Worker behaves the same way. With Strands on the Worker and AiSdkPlugin on the connection, you get the new TypeError. So which of two equivalent setups fails depends on list order and on where each plugin is registered. In the setup that passes, the Vercel Workflows' model calls reach Strands' handler, which is the #160 symptom. #202 closes this for Strands and #161 for Google ADK's MCP names, but only once they land, and never for third-party or caller-written plugins. The plugin can close its own side regardless of order. worker.options is public, and worker.options.activities is a Map holding the original function references (the probe above reads it). So AiSdkPlugin can override runWorker(worker, next), check that each name in this.options.activities still maps to its own function or one with the same sources, and throw before calling next(worker) if not. That catches a later replacement before polling starts. Replay Workers skip runWorker, which is fine because they run no Activities. If you'd rather keep this PR to #160's scope, say in the README that only registrations ahead of AiSdkPlugin are checked and that NativeConnection plugins count as after the Worker's own. Then add a test that pins that behaviour, so that a later change to it is deliberate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the runWorker check in 58c9ca2. runWorker runs after every plugin's configureWorker, so it reads the final worker.options.activities. It throws if a plugin applied after AiSdkPlugin (later in plugins, or on the NativeConnection) replaced one of its Activities with a different one.

Before throwing, it stops the native Worker and lets next run, so the Worker never polls and run()'s own teardown releases it. Without that step, connection.close() in the usual finally block throws "Cannot close connection while Workers hold a reference to it" and hides the TypeError. There's no public way to stop a Worker that hasn't run, so this uses the protected nativeWorker.

New test: runWorker refuses an Activity that a later plugin replaced, before polling. It fails without the check and without the release step. Against a dev server, both of your setups ([ai, strands] on the Worker, and Strands on the connection) now fail worker.run(), and the Workflow's history shows the refused Worker took no task. #202 and #166 are adding the same check, and a follow-up PR adds it to GoogleAdkPlugin, since #161 merged without it. A check in SimplePlugin would still be the better place; I'm drafting an sdk-typescript issue for it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up in 3f769db: the first version of this check awaited before starting the Worker, even when nothing was replaced. Worker.runUntil stops the Worker after its callback only if it is already running, so a callback that threw synchronously left runUntil polling forever, and so did two plugins with the check. runWorker now hands on to super.runWorker in the same tick when the check passes, and a new test covers both cases.

return super.configureWorker(options);
}

/**
* Runs the Worker.
*
* @throws TypeError, before the Worker polls, if a plugin applied after this
* one replaced one of its Activities with a different one. `Worker.create`
* applies the Worker's plugins in order, then the `NativeConnection`'s.
*/
override runWorker(worker: Worker, next: (worker: Worker) => Promise<void>): Promise<void> {
// Not async: when the check passes, the Worker starts in this tick, as
// `Worker.runUntil` needs (see `refuseLaterReplacement`).
return (
refuseLaterReplacement('AiSdkPlugin', worker, this.options.activities as object, next) ??
super.runWorker(worker, next)
);
}

override configureBundler(options: BundleOptions): BundleOptions {
// Prepend a polyfill-installer module to the webpack `entry` array so it
// evaluates before any other module in the Workflow bundle. Webpack 5
Expand Down
Loading