Skip to content

fix: restore evidence integrity in V2 pipeline #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
austinintelligence merged 1 commit into from
Aug 3, 2026
Merged

fix: restore evidence integrity in V2 pipeline #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,10 @@ See [`docs/public-deployment.md`](docs/public-deployment.md) for the store check

V2 builds one compact `ArticleIndex`, creates an adaptive `AnalysisPlan`, and runs a global
`RetrievalCoordinator` against a shared source ledger and evidence graph. Fast, balanced, and deep
modes change passage, mission, concurrency, deadline, and output budgets. Evidence is validated
once before it can serve any report section, and sections only project the ledger; the side panel
never reconciles sources itself.
modes change passage, mission, concurrency, deadline, and output budgets. Providers return
candidate sources only; a bounded adjudication step maps exact candidate IDs to article claims and
relationships, and centralized validation is required before an assertion can serve any report
section. The side panel only projects validated ledger assertions.

Exa uses bounded search requests with returned text/highlights. Native ChatGPT web search is one
bounded global search workflow: its URL-attributed results are `search-summary` evidence and are
Expand Down
7 changes: 5 additions & 2 deletions apps/extension/entrypoints/offscreen/main.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { createChatGPT } from "@opencoredev/loginwithchatgpt-ai";
import type { ChatGPTTokens, ReasoningEffort } from "@opencoredev/loginwithchatgpt-core";
import {
analyzeArticle,
createModelEvidenceAdjudicator,
retryArticleSections,
type AnalysisArtifacts,
type PipelineTelemetry,
Expand Down Expand Up @@ -199,7 +200,7 @@ async function testSearchProvider(
controller.signal,
);
const first = await iterator[Symbol.asyncIterator]().next();
const sourceCount = first.done ? 0 : first.value.cards.length;
const sourceCount = first.done ? 0 : first.value.candidates.length;
if (sourceCount === 0)
throw new Error("ChatGPT completed the request but did not return a web source.");
return { available: true, sourceCount };
Expand Down Expand Up @@ -236,6 +237,7 @@ async function runJob(
article: command.request.article,
retriever,
model,
adjudicator: createModelEvidenceAdjudicator(model),
modelVersion: preferences.model,
reasoningEffort: preferences.reasoningEffort,
mode: preferences.mode,
Expand Down Expand Up @@ -340,7 +342,7 @@ async function runRetryJob(
reasoningEffort: "medium" as const,
mode: "balanced" as const,
};
const { retriever } = await createRetriever(
const { model, retriever } = await createRetriever(
jobId,
preferences.model,
preferences.reasoningEffort,
Expand All @@ -349,6 +351,7 @@ async function runRetryJob(
for await (const event of retryArticleSections({
artifacts,
retriever,
adjudicator: createModelEvidenceAdjudicator(model),
sections: command.sections,
signal: controller.signal,
onTelemetry: (telemetry) => logTelemetry(jobId, telemetry),
Expand Down
1 change: 1 addition & 0 deletions apps/extension/entrypoints/sidepanel/AnalysisProgress.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ describe("analysis pipeline progress", () => {
...state,
phase: "retrieval",
research: {
candidateCount: 3,
completedMissions: 2,
totalMissions: 4,
acceptedSources: 3,
Expand Down
272 changes: 168 additions & 104 deletions apps/extension/entrypoints/sidepanel/App.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import {
memo,
useCallback,
useEffect,
useRef,
Expand Down Expand Up @@ -37,6 +38,7 @@ import {
isAnalysisActive,
} from "./report-state";
import { ReportStore } from "./report-store";
import type { ReportSectionKey } from "./report-store";
import {
extensionMode,
clearAnalysisLogs,
Expand Down Expand Up @@ -359,6 +361,136 @@ export function ProvisionalCompassWarning() {
);
}

function useReportSection<K extends ReportSectionKey>(store: ReportStore, section: K) {
const subscribe = useCallback(
(listener: () => void) => store.subscribeSection(section, listener),
[section, store],
);
const getSnapshot = useCallback(() => store.getSectionSnapshot(section), [section, store]);
return useSyncExternalStore(subscribe, getSnapshot, getSnapshot);
}

const ConnectedCompass = memo(function ConnectedCompass({ store }: { store: ReportStore }) {
const section = useReportSection(store, "compass");
return section.data ? (
<>
<Compass result={section.data} />
{section.status === "error" ? <ProvisionalCompassWarning /> : null}
</>
) : (
<div className="compass-placeholder">
<TargetIcon />
<span>
<small>Political Spectrum</small>
<strong>{section.status === "error" ? "Unavailable" : "Finding placement…"}</strong>
</span>
</div>
);
});

const ConnectedBias = memo(function ConnectedBias({ store }: { store: ReportStore }) {
const section = useReportSection(store, "bias");
const data = section.data;
return (
<Section id="bias" title="Bias" status={section.status} error={section.error}>
{data ? (
data.readerCopy ? (
<ReaderCopyBody
copy={data.readerCopy}
labels={new Map(data.findings.map((finding) => [finding.id, finding.displayName]))}
citations={
new Map(
(data.citations ?? []).map((citation: ReaderCitation) => [citation.id, citation]),
)
}
/>
) : (
<>
<p>
<ProgressiveText text={data.summary} />
</p>
{data.findings.map((finding) => (
<div className="finding" key={finding.id}>
<h3>{finding.displayName}</h3>
<blockquote>
<ProgressiveText text={`“${finding.excerpt}”`} />
</blockquote>
<p>
<ProgressiveText text={finding.explanation} />
</p>
</div>
))}
</>
)
) : null}
</Section>
);
});

const ConnectedJournalistContext = memo(function ConnectedJournalistContext({
store,
}: {
store: ReportStore;
}) {
const section = useReportSection(store, "journalistContext");
return (
<Section
id="journalist-context"
title="Journalist Context"
status={section.status}
error={section.error}
>
{section.data ? <JournalistBody result={section.data} /> : null}
</Section>
);
});

const ConnectedEvidence = memo(function ConnectedEvidence({
store,
section: sectionKey,
title,
id,
}: {
store: ReportStore;
section: "supporting" | "contradicting";
title: string;
id: string;
}) {
const section = useReportSection(store, sectionKey);
return (
<Section id={id} title={title} status={section.status} error={section.error}>
{section.data ? <EvidenceBody result={section.data} /> : null}
</Section>
);
});

const ConnectedAdditionalContext = memo(function ConnectedAdditionalContext({
store,
}: {
store: ReportStore;
}) {
const section = useReportSection(store, "additionalContext");
return (
<Section
id="additional-context"
title="Additional Context"
status={section.status}
error={section.error}
>
{section.data ? <AdditionalContextBody result={section.data} /> : null}
</Section>
);
});

const ConnectedSourceList = memo(function ConnectedSourceList({ store }: { store: ReportStore }) {
const section = useReportSection(store, "sourceList");
return (
<Section id="sources" title="Works Cited" status={section.status} error={section.error}>
{section.data ? <SourceListBody result={section.data} /> : null}
</Section>
);
});

function AnalysisReport({ preferences, onOpenSettings }: AnalysisReportProps) {
const reportStoreRef = useRef<ReportStore | null>(null);
reportStoreRef.current ??= new ReportStore();
Expand Down Expand Up @@ -558,111 +690,28 @@ function AnalysisReport({ preferences, onOpenSettings }: AnalysisReportProps) {
<PartialReportNotice onRetry={retryIncompleteSections} />
) : null}

{state.compass.data ? (
<>
<Compass result={state.compass.data} />
{state.compass.status === "error" ? <ProvisionalCompassWarning /> : null}
</>
) : (
<div className="compass-placeholder">
<TargetIcon />
<span>
<small>Political Spectrum</small>
<strong>
{state.compass.status === "error" ? "Unavailable" : "Finding placement…"}
</strong>
</span>
</div>
)}

<Section id="bias" title="Bias" status={state.bias.status} error={state.bias.error}>
{state.bias.data ? (
state.bias.data.readerCopy ? (
<ReaderCopyBody
copy={state.bias.data.readerCopy}
labels={
new Map(
state.bias.data.findings.map((finding) => [finding.id, finding.displayName]),
)
}
citations={
new Map(
(state.bias.data.citations ?? []).map((citation: ReaderCitation) => [
citation.id,
citation,
]),
)
}
/>
) : (
<>
<p>
<ProgressiveText text={state.bias.data.summary} />
</p>
{state.bias.data.findings.map((finding) => (
<div className="finding" key={finding.id}>
<h3>{finding.displayName}</h3>
<blockquote>
<ProgressiveText text={`“${finding.excerpt}”`} />
</blockquote>
<p>
<ProgressiveText text={finding.explanation} />
</p>
</div>
))}
</>
)
) : null}
</Section>

<Section
id="journalist-context"
title="Journalist Context"
status={state.journalistContext.status}
error={state.journalistContext.error}
>
{state.journalistContext.data ? (
<JournalistBody result={state.journalistContext.data} />
) : null}
</Section>

<Section
<ConnectedCompass store={reportStore} />
<ConnectedBias store={reportStore} />
<ConnectedJournalistContext store={reportStore} />
<ConnectedEvidence
store={reportStore}
section="supporting"
id="supporting"
title="Supporting Information"
status={state.supporting.status}
error={state.supporting.error}
>
{state.supporting.data ? <EvidenceBody result={state.supporting.data} /> : null}
</Section>

<Section
/>
<ConnectedEvidence
store={reportStore}
section="contradicting"
id="contradicting"
title="Contradicting Information"
status={state.contradicting.status}
error={state.contradicting.error}
>
{state.contradicting.data ? <EvidenceBody result={state.contradicting.data} /> : null}
</Section>

<Section
id="additional-context"
title="Additional Context"
status={state.additionalContext.status}
error={state.additionalContext.error}
>
{state.additionalContext.data ? (
<AdditionalContextBody result={state.additionalContext.data} />
) : null}
</Section>
/>
<ConnectedAdditionalContext store={reportStore} />
<ConnectedSourceList store={reportStore} />

<Section
id="sources"
title="Works Cited"
status={state.sourceList.status}
error={state.sourceList.error}
>
{state.sourceList.data ? <SourceListBody result={state.sourceList.data} /> : null}
</Section>
{/* Legacy section markup removed in favor of connected sections.
{state.compass.status === "error" ? "Unavailable" : "Finding placement…"}
<ProgressiveText text={`“${finding.excerpt}”`} />
*/}

<footer className="telemetry-footer">
<button
Expand Down Expand Up @@ -783,24 +832,39 @@ function ChatGptApp() {
};
}, []);

const updatePreferences = (next: SettingsPreferences) => {
if (!runtime) return;
const updatePreferences = async (next: SettingsPreferences): Promise<void> => {
if (!runtime) throw new Error("Perspectica settings are still loading.");
const previous = runtime;
const updated: ExtensionPreferences = {
...runtime.preferences,
...next,
};
setRuntime({ ...runtime, preferences: updated });
void updateExtensionPreferences(updated);
try {
const saved = await updateExtensionPreferences(updated);
setRuntime((current) => (current ? { ...current, preferences: saved } : current));
} catch (error) {
setRuntime(previous);
throw error;
}
};

const updateSearchProvider = async (provider: SearchProviderKind) => {
if (!runtime) throw new Error("Perspectica settings are still loading.");
const previous = runtime;
const test = await testSearchProvider(provider);
if (!test.available) throw new Error(`${provider} search is not available.`);
const updated = { ...runtime.preferences, searchProvider: provider };
setRuntime({ ...runtime, preferences: updated });
setProviderReady(true);
await updateExtensionPreferences(updated);
try {
const saved = await updateExtensionPreferences(updated);
setRuntime((current) => (current ? { ...current, preferences: saved } : current));
} catch (error) {
setRuntime(previous);
setProviderReady(previous.preferences.searchProvider === "chatgpt" || previous.hasExaKey);
throw error;
}
};
Comment on lines +835 to 868

@coderabbitai coderabbitai Bot Aug 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Guard updatePreferences and updateSearchProvider against overlapping calls.

Both functions capture runtime as previous at call time and later call setRuntime(previous) on failure, or setRuntime with the fetched saved value on success, with no sequence check. If two calls overlap, for example a preference toggle and a search-provider change fired close together, or one call resolving slower than a later one, the later-resolving call determines the final state regardless of which call started more recently:

  • On success: an older, slower call can overwrite the state committed by a newer, faster call with stale saved data.
  • On failure: an older call's previous rollback can undo a newer call's already-applied optimistic or committed state.

Both paths silently revert or overwrite the user's most recent settings change without any visible error. The file already has prior art for this exact problem: analyze() (Line 523) guards stale completions with a monotonic runRef counter. Apply the same pattern here, shared across both functions since they both mutate runtime.preferences.

🔒 Proposed fix using a shared sequence guard 
+  const preferencesRequestRef = useRef(0);
+
   const updatePreferences = async (next: SettingsPreferences): Promise<void> => {
     if (!runtime) throw new Error("Perspectica settings are still loading.");
+    const requestId = ++preferencesRequestRef.current;
     const previous = runtime;
     const updated: ExtensionPreferences = {
       ...runtime.preferences,
       ...next,
     };
     setRuntime({ ...runtime, preferences: updated });
     try {
       const saved = await updateExtensionPreferences(updated);
+      if (preferencesRequestRef.current !== requestId) return;
       setRuntime((current) => (current ? { ...current, preferences: saved } : current));
     } catch (error) {
-      setRuntime(previous);
+      if (preferencesRequestRef.current === requestId) setRuntime(previous);
       throw error;
     }
   };

   const updateSearchProvider = async (provider: SearchProviderKind) => {
     if (!runtime) throw new Error("Perspectica settings are still loading.");
+    const requestId = ++preferencesRequestRef.current;
     const previous = runtime;
     const test = await testSearchProvider(provider);
     if (!test.available) throw new Error(`${provider} search is not available.`);
     const updated = { ...runtime.preferences, searchProvider: provider };
     setRuntime({ ...runtime, preferences: updated });
     setProviderReady(true);
     try {
       const saved = await updateExtensionPreferences(updated);
+      if (preferencesRequestRef.current !== requestId) return;
       setRuntime((current) => (current ? { ...current, preferences: saved } : current));
     } catch (error) {
-      setRuntime(previous);
-      setProviderReady(previous.preferences.searchProvider === "chatgpt" || previous.hasExaKey);
+      if (preferencesRequestRef.current === requestId) {
+        setRuntime(previous);
+        setProviderReady(previous.preferences.searchProvider === "chatgpt" || previous.hasExaKey);
+      }
       throw error;
     }
   };
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/extension/entrypoints/sidepanel/App.tsx` around lines 835 - 868, Guard
both updatePreferences and updateSearchProvider with a shared monotonic sequence
ref, following the existing analyze() runRef pattern. Increment the shared
sequence when either function starts, and only apply optimistic updates,
successful saved preferences, rollback state, or provider readiness changes if
that call still owns the latest sequence; stale completions must not overwrite
newer preference changes.


let page: ReactNode;
Expand Down
Loading
Loading