prebid: add pubProvidedId delivery of cached EIDs via user-id config

Author: etiennelatendresse-optable

README.md

@@ -1285,6 +1285,20 @@ window.optable.cmd = new OptableCommands(window.optable.cmd || []);
 
 For the page-side stub and behaviour details, see the [command queue addon README](lib/addons/commands.md).
 
+## Prebid pubProvidedId delivery
+
+The pubProvidedId module delivers cached EIDs to prebid through the `pubProvidedId` user-id submodule, for integrations that don't use the RTD module. EIDs from other providers are preserved, ours are replaced by source, and the work queues on the prebid global so it also runs before prebid has loaded.
+
+```typescript
+import { mergeIntoPubProvidedId } from "@optable/web-sdk/lib/dist/core/prebid/pubProvidedId";
+
+mergeIntoPubProvidedId({ instances: ["pbjs"] });
+```
+
+On Prebid versions without the fix for [prebid/Prebid.js#15562](https://github.com/prebid/Prebid.js/pull/15562), the module's filtered ID refresh can drop other vendors (LiveIntent, ID5, …) from the page's first auction. Passing `refreshAll: true` works around it with a full ID refresh: the upside is that no vendor is dropped from the first auction; the downside is that every ID vendor re-requests on that pageview (relevant under per-request quotas) and the auction can start later. Leave it off on Prebid versions that include the fix.
+
+For behavior details and options, see the [pubProvidedId README](lib/core/prebid/pubProvidedId.md).
+
 ## Demo Pages
 
 The demo pages are working examples of both `identify` and `targeting` APIs, as well as an integration with the [Google Ad Manager 360](https://admanager.google.com/home/) ad server, enabling the targeting of ads served by GAM360 to audiences activated in the [Optable](https://optable.co/) DCN.

lib/core/prebid/pubProvidedId.md

@@ -0,0 +1,42 @@
+# pubProvidedId Delivery
+
+Delivers cached EIDs to prebid through the [`pubProvidedId` user-id submodule](https://docs.prebid.org/dev-docs/modules/userid-submodules/pubprovided.html), for integrations that deliver EIDs via user-id config rather than the RTD module.
+
+## Usage
+
+```js
+import { mergeIntoPubProvidedId } from "@optable/web-sdk/lib/dist/core/prebid/pubProvidedId";
+
+mergeIntoPubProvidedId({ instances: ["pbjs"] });
+```
+
+Call it after each write to the rolling EID cache (targeting, tokenize, UID2 refresh). By default it reads EIDs from the `OPTABLE_RESOLVED` key in `localStorage`; pass `cacheKey` to read another key, or `eids` to merge an explicit list.
+
+## Behavior
+
+- Work is queued on each instance's `que` array, so it also runs when prebid hasn't loaded yet — the queue is created on the named global if needed.
+- EIDs from other providers already in `pubProvidedId` are preserved; ours are replaced by `source`.
+- Duplicate `pubProvidedId` entries in an already polluted config are collapsed back to a single entry; other user-id submodules and the rest of the `userSync` config are untouched.
+- Underscore-prefixed cache sidecars (`_ref` UID2 refresh material, `_id5` metadata) are stripped before EIDs reach prebid, so they never leak into bid requests.
+- After merging, `refreshUserIds({ submoduleNames: ["pubProvidedId"] })` propagates the change — or a full `refreshUserIds()` with `refreshAll: true` (see below).
+- Any decision to skip delivery (a split-test control group, for example) stays with the caller.
+
+## Options
+
+| Option       | Default              | Description                                                                         |
+| ------------ | -------------------- | ----------------------------------------------------------------------------------- |
+| `instances`  | `["pbjs"]`           | Names of the prebid globals to merge into.                                          |
+| `cacheKey`   | `"OPTABLE_RESOLVED"` | localStorage key of the rolling EID cache.                                          |
+| `eids`       | read from the cache  | Explicit EIDs to merge, bypassing the cache.                                        |
+| `refreshAll` | `false`              | Refresh every user-id submodule after merging, not just `pubProvidedId`. See below. |
+
+## First-auction identity and `refreshAll`
+
+Prebid versions without the fix for [prebid/Prebid.js#15562](https://github.com/prebid/Prebid.js/pull/15562) have a defect: a filtered `refreshUserIds({ submoduleNames })` issued while other ID vendors are still initializing abandons their in-flight work, so vendors like LiveIntent or ID5 are dropped from the page's first auction. The default filtered refresh this module issues at page load is exactly that trigger.
+
+`refreshAll: true` works around it by issuing an unfiltered `refreshUserIds()` instead, which starts a full refresh cycle the auction waits for — every vendor completes, and the merged EIDs are included.
+
+- Upside: no vendor is dropped from the first auction, and split-test uplift is no longer understated by treated users losing other vendors' IDs.
+- Downside: every configured ID vendor makes a fresh request on that pageview (relevant when a vendor applies per-request quotas), and the auction can start later since it waits for the slowest vendor.
+
+Leave it off on Prebid versions that include the fix — the filtered refresh is then both correct and cheaper. Neither mode can rescue an auction that fired before the merge ran at all.

lib/core/prebid/pubProvidedId.test.ts

@@ -0,0 +1,187 @@
+import { mergeIntoPubProvidedId } from "./pubProvidedId";
+
+type FakePbjs = {
+  que: Array<() => void>;
+  getConfig: jest.Mock;
+  setConfig: jest.Mock;
+  refreshUserIds: jest.Mock;
+};
+
+function makePbjs(userSync: Record<string, unknown> = {}): FakePbjs {
+  return {
+    que: [],
+    getConfig: jest.fn(() => userSync),
+    setConfig: jest.fn(),
+    refreshUserIds: jest.fn(),
+  };
+}
+
+const w = window as unknown as Record<string, any>;
+
+const EIDS = [
+  { source: "uidapi.com", uids: [{ atype: 3, id: "uid2-token" }] },
+  { source: "id5-sync.com", uids: [{ atype: 1, id: "id5-id" }] },
+];
+
+function seedCache(eids: unknown[], key = "OPTABLE_RESOLVED") {
+  localStorage.setItem(key, JSON.stringify({ ortb2: { user: { data: [], eids } } }));
+}
+
+function drain(pbjs: FakePbjs) {
+  pbjs.que.forEach((cmd) => cmd());
+}
+
+beforeEach(() => {
+  localStorage.clear();
+  delete w.pbjs;
+  delete w.owpbjs;
+});
+
+describe("mergeIntoPubProvidedId", () => {
+  it("merges cached EIDs into a single pubProvidedId entry and refreshes it", () => {
+    seedCache(EIDS);
+    const pbjs = makePbjs({});
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId();
+    drain(pbjs);
+
+    const config = pbjs.setConfig.mock.calls[0][0];
+    expect(config.userSync.userIds).toEqual([{ name: "pubProvidedId", params: { eids: EIDS } }]);
+    expect(pbjs.refreshUserIds).toHaveBeenCalledWith({ submoduleNames: ["pubProvidedId"] });
+  });
+
+  it("queues onto a stub global when prebid has not loaded yet", () => {
+    seedCache(EIDS);
+    mergeIntoPubProvidedId();
+
+    expect(w.pbjs.que).toHaveLength(1);
+    expect(() => w.pbjs.que.forEach((cmd: () => void) => cmd())).not.toThrow();
+  });
+
+  it("preserves other providers' EIDs and replaces ours by source", () => {
+    seedCache(EIDS);
+    const pbjs = makePbjs({
+      userIds: [
+        {
+          name: "pubProvidedId",
+          params: {
+            eids: [
+              { source: "uidapi.com", uids: [{ id: "old-uid2" }] },
+              { source: "publisher.com", uids: [{ id: "pub-own" }] },
+            ],
+          },
+        },
+      ],
+    });
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId();
+    drain(pbjs);
+
+    const eids = pbjs.setConfig.mock.calls[0][0].userSync.userIds[0].params.eids;
+    expect(eids.map((e: any) => e.source)).toEqual(["publisher.com", "uidapi.com", "id5-sync.com"]);
+    expect(eids.find((e: any) => e.source === "uidapi.com").uids[0].id).toBe("uid2-token");
+  });
+
+  it("collapses duplicate pubProvidedId entries and keeps other submodules", () => {
+    seedCache(EIDS);
+    const pbjs = makePbjs({
+      syncDelay: 5000,
+      userIds: [
+        { name: "sharedId" },
+        { name: "pubProvidedId", params: { eids: [{ source: "a.com", uids: [{ id: "a" }] }] } },
+        { name: "pubProvidedId", params: { eids: [{ source: "b.com", uids: [{ id: "b" }] }] } },
+      ],
+    });
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId();
+    drain(pbjs);
+
+    const config = pbjs.setConfig.mock.calls[0][0];
+    expect(config.userSync.syncDelay).toBe(5000);
+    const names = config.userSync.userIds.map((u: any) => u.name);
+    expect(names).toEqual(["sharedId", "pubProvidedId"]);
+    const eids = config.userSync.userIds[1].params.eids;
+    expect(eids.map((e: any) => e.source)).toEqual(["a.com", "b.com", "uidapi.com", "id5-sync.com"]);
+  });
+
+  it("strips underscore-prefixed cache sidecars before handing EIDs to prebid", () => {
+    seedCache([{ source: "uidapi.com", uids: [{ id: "x" }], _ref: { refresh_token: "rt" }, _id5: { t: 1 } }]);
+    const pbjs = makePbjs({});
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId();
+    drain(pbjs);
+
+    const eid = pbjs.setConfig.mock.calls[0][0].userSync.userIds[0].params.eids[0];
+    expect(eid).toEqual({ source: "uidapi.com", uids: [{ id: "x" }] });
+  });
+
+  it("does nothing when the cache has no EIDs", () => {
+    const pbjs = makePbjs({});
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId();
+
+    expect(pbjs.que).toHaveLength(0);
+  });
+
+  it("merges into every configured instance", () => {
+    seedCache(EIDS);
+    const a = makePbjs({});
+    const b = makePbjs({});
+    w.pbjs = a;
+    w.owpbjs = b;
+
+    mergeIntoPubProvidedId({ instances: ["pbjs", "owpbjs"] });
+    drain(a);
+    drain(b);
+
+    expect(a.setConfig).toHaveBeenCalled();
+    expect(b.setConfig).toHaveBeenCalled();
+  });
+
+  it("accepts explicit eids and a custom cacheKey", () => {
+    seedCache(EIDS, "MY_CACHE");
+    const pbjs = makePbjs({});
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId({ cacheKey: "MY_CACHE" });
+    drain(pbjs);
+    expect(pbjs.setConfig.mock.calls[0][0].userSync.userIds[0].params.eids).toEqual(EIDS);
+
+    const direct = makePbjs({});
+    w.pbjs = direct;
+    mergeIntoPubProvidedId({ eids: [{ source: "direct.com", uids: [{ id: "d" }] }] });
+    drain(direct);
+    expect(direct.setConfig.mock.calls[0][0].userSync.userIds[0].params.eids).toEqual([
+      { source: "direct.com", uids: [{ id: "d" }] },
+    ]);
+  });
+
+  it("refreshAll refreshes every user-id submodule instead of only pubProvidedId", () => {
+    seedCache(EIDS);
+    const pbjs = makePbjs({});
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId({ refreshAll: true });
+    drain(pbjs);
+
+    expect(pbjs.refreshUserIds).toHaveBeenCalledWith();
+  });
+
+  it("a throwing prebid config call does not break the queue", () => {
+    seedCache(EIDS);
+    const pbjs = makePbjs({});
+    pbjs.getConfig.mockImplementation(() => {
+      throw new Error("boom");
+    });
+    w.pbjs = pbjs;
+
+    mergeIntoPubProvidedId();
+    expect(() => drain(pbjs)).not.toThrow();
+    expect(pbjs.setConfig).not.toHaveBeenCalled();
+  });
+});

lib/core/prebid/pubProvidedId.ts

@@ -0,0 +1,99 @@
+import { debugLog } from "../log";
+
+// Delivers cached EIDs to prebid through the pubProvidedId user-id submodule,
+// for integrations that don't use the RTD module.
+
+type Eid = {
+  source: string;
+  uids?: unknown[];
+};
+
+type PubProvidedIdOptions = {
+  // Prebid global names to merge into. Defaults to ["pbjs"].
+  instances?: readonly string[];
+  // localStorage key of the EID cache. Defaults to OPTABLE_RESOLVED.
+  cacheKey?: string;
+  // EIDs to merge, bypassing the cache read.
+  eids?: Eid[];
+  // Refresh every user-id submodule after merging, not just pubProvidedId.
+  // Workaround for prebid/Prebid.js#15562 — see pubProvidedId.md.
+  refreshAll?: boolean;
+};
+
+const DEFAULT_CACHE_KEY = "OPTABLE_RESOLVED";
+
+function cachedEids(cacheKey: string): Eid[] {
+  try {
+    const resolved = JSON.parse(localStorage.getItem(cacheKey) || "null");
+    return resolved?.ortb2?.user?.eids || [];
+  } catch {
+    return [];
+  }
+}
+
+// Cache sidecars like _ref (UID2 refresh material) must not reach bid requests.
+function stripSidecars(eid: Eid): Eid {
+  const clean: Record<string, unknown> = {};
+  for (const key of Object.keys(eid)) {
+    if (!key.startsWith("_")) {
+      clean[key] = (eid as Record<string, unknown>)[key];
+    }
+  }
+  return clean as Eid;
+}
+
+export function mergeIntoPubProvidedId(options: PubProvidedIdOptions = {}): void {
+  const instances = options.instances ?? ["pbjs"];
+  const ourEids = (options.eids ?? cachedEids(options.cacheKey ?? DEFAULT_CACHE_KEY)).map(stripSidecars);
+
+  instances.forEach((instanceName) => {
+    if (!ourEids.length) {
+      debugLog("log", `(${instanceName}) PPID: no EIDs to merge`);
+      return;
+    }
+
+    // Queue on the named global so this also works before prebid has loaded.
+    const w = window as unknown as Record<string, { que?: Array<() => void> } & Record<string, any>>;
+    w[instanceName] = w[instanceName] || {};
+    const pbjs = w[instanceName];
+    pbjs.que = pbjs.que || [];
+    pbjs.que.push(() => {
+      try {
+        const ourSources = new Set(ourEids.map((e) => e.source));
+
+        // Collapse every pubProvidedId entry found, not just the first, so a
+        // config already polluted with duplicates heals back down to one.
+        const currentUserSync = pbjs.getConfig?.("userSync") || {};
+        const currentUserIds: Array<{ name?: string; params?: { eids?: Eid[] } }> = currentUserSync.userIds || [];
+        const existingEids = currentUserIds
+          .filter((u) => u.name === "pubProvidedId")
+          .flatMap((u) => u.params?.eids || []);
+
+        // Keep EIDs from other providers, replace ours by source.
+        const preserved = existingEids.filter((e) => !ourSources.has(e.source));
+        const mergedEids = [...preserved, ...ourEids];
+
+        const updatedUserIds = currentUserIds.filter((u) => u.name !== "pubProvidedId");
+        updatedUserIds.push({
+          name: "pubProvidedId",
+          params: { eids: mergedEids },
+        });
+
+        pbjs.setConfig?.({ userSync: { ...currentUserSync, userIds: updatedUserIds } });
+        if (options.refreshAll) {
+          pbjs.refreshUserIds?.();
+        } else {
+          pbjs.refreshUserIds?.({ submoduleNames: ["pubProvidedId"] });
+        }
+        debugLog(
+          "log",
+          `(${instanceName}) PPID: merged ${ourEids.length} EIDs (${preserved.length} preserved from others)`
+        );
+      } catch (err) {
+        debugLog("error", `(${instanceName}) PPID: merge error`, err);
+      }
+    });
+  });
+}
+
+export type { Eid, PubProvidedIdOptions };