Skip to content

Does the global catalog use context.intent for ranking? Observed near no-op vs structured filters #24

Description

@shyju-t

Summary

When searching the bundled global catalog, context.intent appears to have little to no effect on results, while structured filters are honored exactly. I'd like to confirm whether this is intended behavior (intent is a soft, possibly unused signal for the global catalog) or whether I'm using it wrong.

The CLI sends context.intent correctly (it shows up in the echoed next-page command), so this looks like a backend ranking question rather than a CLI issue.

Environment: @shopify/ucp-cli v0.6.2, global catalog (no --business), context.address_country=US.

Reproduction

All four searches use the same query (running shoes) and country (US), varying only how "cheap/expensive" is expressed.

# A. No intent (baseline)
ucp catalog search --set /query='running shoes' --set /context/address_country=US \
  --set /pagination/limit=10 --view 'result.products[*].{t: title, p: price_range.min.amount}' --format jsonl

# B. Intent = budget
ucp catalog search --set /query='running shoes' --set /context/address_country=US \
  --set /context/intent='I want the cheapest budget option, under 50 dollars, basic' \
  --set /pagination/limit=10 --view 'result.products[*].{t: title, p: price_range.min.amount}' --format jsonl

# C. Intent = premium (opposite of B)
ucp catalog search --set /query='running shoes' --set /context/address_country=US \
  --set /context/intent='premium high-end carbon plate racing shoe for marathon, money no object' \
  --set /pagination/limit=10 --view 'result.products[*].{t: title, p: price_range.min.amount}' --format jsonl

# D. Intent = explicit price ceiling (free text)
ucp catalog search --set /query='running shoes' --set /context/address_country=US \
  --set /context/intent='only show me shoes under 50 dollars' \
  --set /pagination/limit=8 --view 'result.products[*].{t: title, p: price_range.min.amount}' --format jsonl

# E. Structured price filter (control)
ucp catalog search --set /query='running shoes' --set /context/address_country=US \
  --set /context/currency=USD --set /filters/price/max=5000 \
  --set /pagination/limit=8 --view 'result.products[*].{t: title, p: price_range.min.amount}' --format jsonl

Observed results

B vs C (opposite intents): identical top 10, same order. Both the "cheapest budget under 50" intent and the "premium, money no object" intent returned the same products in the same order, led by an item around $180.

D (intent "under 50 dollars"): not reflected. Top result was $180 (ON Cloudsurfer Max), and the cheapest item in the page ($36 adidas Galaxy) sat at position 5. The price ceiling expressed in intent did not constrain or meaningfully re-rank.

E (structured filters.price.max=5000): honored exactly. Every result was under $50.00 (examples: $34.99, $36.00, $48.00, $49.99).

So structured filters work as expected, while free-text intent seems to have at most a marginal effect on the global catalog ranking, and contradictory intents are indistinguishable.

Questions

  1. Does the global catalog factor context.intent into ranking at all, or is it currently effectively a no-op for the global (cross-merchant) catalog?
  2. If it is used, is the expected effect a soft re-rank only (never a constraint), and are there input characteristics that make it more effective?
  3. Is intent handling expected to differ between the global catalog and a single --business store (where the merchant's own Search & Discovery config applies)?
  4. Is the current guidance simply "use filters for anything that must hold, and treat intent as an optional soft hint"? If so, it might be worth making that explicit in the catalog docs.

Thanks for any clarification on the intended semantics here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions