Skip to content

Add Bing Webmaster Insights and Bing rank tracking - #341

Open
snayyar00 wants to merge 4 commits into
every-app:mainfrom
snayyar00:pr/bing-features
Open

snayyar00 wants to merge 4 commits into
every-app:mainfrom
snayyar00:pr/bing-features

Conversation

@snayyar00

@snayyar00 snayyar00 commented Sep 16, 2026

Copy link
Copy Markdown

Summary

  • Adds Bing Webmaster Insights: API-key auth, own connection card, own report page (/p/:projectId/bing-performance).
  • Adds Bing as a searchEngine option on Rank Tracking configs, alongside Google. Enforced create-only (can't switch an existing tracker's search engine) and Bing is national-only server-side (no local/city-level Bing tracking).
  • Includes fixes found in review: correct GetPageStats return type, map Bing's 400/ErrorCode:3 to a clear invalid-key error, filter to verified sites only.

Test plan

  • pnpm run types:check passes
  • drizzle-kit check clean on both SQLite and Postgres dialects
  • Exercise Bing Webmaster connection + a Bing rank tracker against a real verified site post-merge

snayyar00 and others added 4 commits September 16, 2026 14:30
Adds a Bing Insights page (/p/:projectId/bing-performance) mirroring GSC
Insights: totals, striking-distance, and Queries/Pages tables. Unlike GSC,
Bing Webmaster Tools has no OAuth app to register — auth is a per-user API
key generated at bing.com/webmasters, validated against GetUserSites and
stored AES-256-GCM encrypted (bingCrypto.ts, keyed from BETTER_AUTH_SECRET,
the same secret that protects OAuth tokens elsewhere in this app).

- New tables bing_connections (per-project site) and bing_api_keys
  (per-user key), sqlite + pg, wired through the schema barrels and
  schema-parity test. Migrations: drizzle/0047, drizzle-pg/0025.
- bingWebmasterClient.ts talks to the Bing Webmaster REST JSON API
  (ssl.bing.com/webmaster/api.svc/json), unwraps the `{"d": [...]}`
  envelope, and maps 401/403 to BingAuthError so the UI can distinguish
  "reconnect" from a real fault.
- bingPerformanceReport.ts holds the pure shaping/aggregation logic
  (unit tested): Bing gives no date-range/device/country filters and no
  server-side pagination, so the report fetches Bing's whole fixed window
  in one call and the page paginates client-side; the previous-period
  comparison splits that window in half since there's no "period before"
  to ask Bing for.
- No new env var: the API key is entered per-user in the UI, not an
  operator-level secret.

Nav: added "Bing Insights" to the My Site group. Settings: added a Bing
Webmaster Tools connection card next to Search Console / Analytics.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ed-site filter

Four bugs found in code review of the Bing Webmaster Insights feature:

- BingPageStatsRow invented a `Page` field that doesn't exist in Bing's real
  GetPageStats/GetQueryPageStats response. Bing reuses the QueryStats wire
  shape for these endpoints: `Query` holds the page URL and the row carries
  real Clicks/Impressions/AvgClickPosition/AvgImpressionPosition metrics, not
  a (query, page) pair.
- toPageRows grouped on the nonexistent row.Page; it now maps the corrected
  per-page metrics directly (sorted by impressions desc), replacing the
  distinct-query-count proxy with real click/impression data. Updated the
  Page table UI, its CSV export, and the unit test fixture to match.
- bingWebmasterClient only mapped HTTP 401/403 to BingAuthError. Bing also
  returns HTTP 400 with {"ErrorCode":3,"Message":"...InvalidApiKey..."} for a
  bad key; that body shape now maps to BingAuthError too.
- BingSite was missing IsVerified. Added it and filtered to IsVerified===true
  everywhere sites are surfaced or matched for selection (saveApiKey,
  listSitesForUser, setSite).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rank tracking configs gain a searchEngine field ("google" | "bing",
default "google") that threads end to end: schema (D1 + Postgres,
mirrored, both partial unique indexes now include search_engine so the
same domain+market can be tracked on both engines), Zod schemas,
server functions, RankTrackingService (dedupe + reactivation keyed on
engine), the rank-check workflow/DataForSEO SERP layer
(/v3/serp/{engine}/organic/...), the config modal (engine radio,
national-only for Bing in v1), table/detail UI (engine badges, "Google
volume" label on Bing configs since DataForSEO has no Bing volume
source), and the create_rank_tracker MCP tool.

Bing's stop_crawl_on_match omits find_targets_in until a live call
confirms DataForSEO accepts it there (undocumented for that endpoint);
sending it unverified risks a billed "Invalid Field" task on every
Bing rank check.

pnpm db:generate produced drizzle/0047 and drizzle-pg/0025, dropping
and recreating the two partial unique indexes in both dialects.
schema-parity.test.ts (184 cases) and the full suite (1290 tests)
pass; tsc --noEmit is clean.

Not done in this pass: Bing city/local targeting (locations endpoint
not wired), and the live-call verification of Bing SERP fields called
out above.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nal-only server-side

- find_targets_in verified DataForSEO-Bing-unsupported (Google-only param);
  existing stopCrawlOnTarget()/buildRankCheckResult() already restrict
  matches to organic items for both engines, so no change needed there.
- searchEngine removed from updateConfigSchema (RankTrackingService.updateConfig
  never applied it anyway) and from the update mutation payload; UI disables
  the engine radios when editing an existing config.
- Bing + locationName rejected at the createConfigSchema and updateConfig
  service layer (existing config's engine, since engine can't change), not
  just in the UI/MCP layer.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a2e446964

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +122 to +126
page: row.Query,
clicks: row.Clicks,
impressions: row.Impressions,
avgClickPosition: row.AvgClickPosition,
avgImpressionPosition: row.AvgImpressionPosition,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Parse the actual Bing PageStats response shape

When a real GetPageStats response is loaded, Bing returns page-stat rows with Page, Clicks, Impressions, and Date; it does not reuse QueryStats or supply the average-position fields. This mapping consequently makes every page and position undefined, and opening the Pages tab reaches formatPosition(undefined).toFixed, crashing the view. Validate the wire response and model/render the actual PageStats fields instead of asserting the QueryStats shape.

AGENTS.md reference: AGENTS.md:L11-L11

Useful? React with 👍 / 👎.

const hasKey = Boolean(connection?.currentUserHasKey);
const canManage = connection?.canManage === true;

const showPicker = picking || (!connected && hasKey && canManage);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Allow users to replace a revoked saved key

When a previously saved key is revoked or rotated, the performance request deliberately reports the integration as disconnected, but currentUserHasKey remains true. This condition forces the site picker, whose listBingSites call then fails with the same revoked key, and neither the connected nor disconnected state exposes the API-key form, leaving the user unable to replace the credential through the UI.

Useful? React with 👍 / 👎.

Comment thread src/db/bing.schema.ts
Comment on lines +17 to +19
organizationId: text("organization_id")
.notNull()
.references(() => organization.id, { onDelete: "cascade" }),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Repoint Bing connections during workspace merge

When mergeLegacyWorkspaces runs in cloudflare_access mode after a legacy project has connected Bing, it repoints projects plus GSC/GA4 rows in src/server/auth/workspace-merge.ts:124-139 and then deletes the legacy organizations at line 157. Because this new foreign key cascades on organization deletion, the Bing connection is deleted while its project survives in the shared workspace; bingConnections needs to be repointed in the same transaction.

Useful? React with 👍 / 👎.

Comment on lines +106 to +113
export function toQueryRows(rows: BingQueryStatsRow[]): BingQueryRow[] {
return rows.map((row) => ({
query: row.Query,
clicks: row.Clicks,
impressions: row.Impressions,
avgClickPosition: row.AvgClickPosition,
avgImpressionPosition: row.AvgImpressionPosition,
}));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve or aggregate weekly query records

When Bing returns multiple weekly records for the same query—as the new client contract documents—this mapping drops Date and emits every record independently. The Queries table and CSV therefore contain indistinguishable duplicate queries with duplicate React keys, while the striking-distance calculation can count the same query multiple times; group the records into a single weighted result or retain the reporting date.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant