diff --git a/docs/FRESH.md b/docs/FRESH.md new file mode 100644 index 0000000000..2319eb92be --- /dev/null +++ b/docs/FRESH.md @@ -0,0 +1,79 @@ +# Fresh discovery + +Fresh intersects release evidence from one authenticated numeric autobrr filter +with deterministic TMDB identity and downstream discovery policy. It is not +request, acquisition, download, Jellyfin availability, or permanent tracker +state. + +## Identity and admission + +- A movie identity is `(movie, tmdbId)`. Its first selected-filter observation + may qualify inside the configured movie window or the fixed 14-day + first-observation grace. Digital observations prefer TMDB Digital dates; + Blu-ray/UHD observations prefer Physical dates. Fallbacks are deterministic + and never invent a date. +- An ordinary TV identity is `(tv, tmdbId, seasonNumber)`. Admission uses the + latest episode aired by the observation time (with the bounded one-day date + tolerance), then a trustworthy season date when episode dates are absent. + Parent-series age is not admission evidence. +- An explicit special is `(tv, tmdbId, special, episodeNumber)` and uses the + exact special episode's air date. A bare or uncorroborated season zero remains + unknown. +- Public TV presentation remains one canonical series card. When more than one + season/special is currently visible, the newest immutable admission controls + ordering; the individual histories remain independently durable. +- Canonical identity is always the typed pair `(mediaType, tmdbId)`. Numeric + TMDB IDs are not globally unique across Movie and TV namespaces. + +An admitted Movie, Season, or Special has one irreversible Fresh clock. +Additional episodes, packs, PROPER/REPACK/REMUX releases, qualities, title +punctuation variants, source-generation changes, and rebuilds may add evidence +but cannot reset `firstFreshAt` or `visibleUntil`. + +## Automatic and human state + +Fresh preserves source/parsed evidence, automatic typed resolution, automatic +admission and content-policy reasons, optional typed manual resolution, optional +admission override, and effective presentation as separate concepts. + +- **Resolve** validates the administrator-selected Movie/TV namespace and TMDB + ID server-side. The durable decision is bound to versioned sanitized source + evidence and can correct both type and ID without rewriting parsed evidence. +- **Reset resolution** deactivates that human mapping and restores the current + automatic result. It does not erase source evidence or discovery history. +- **Admit to Fresh** overrides a stable automatic policy exclusion only after a + typed identity and Movie/Season/Special identity exist. It cannot override No + Match, Ambiguous, transient provider failure, unknown TV season, an already + active item, or expired history. +- **Remove override** restores automatic policy. Re-adding an override reuses + the original history and cannot create a new Fresh clock. + +All mutations are administrator-only, revision-bound, and serialized with sync, +reconciliation, and rebuild operations. Candidate Diagnostics is the durable +work queue; transient Pipeline decisions are operational telemetry. + +## Rebuild and continuity + +**Rebuild Fresh Data** clears and reconstructs only source-derived observations, +candidates, checkpoint state, and the current source projection. It preserves: + +- typed canonical media metadata; +- irreversible Movie/Season/Special histories; +- typed manual resolutions and provenance; +- admission overrides and provenance. + +Evidence already pruned by autobrr cannot be recovered. A missing authenticated +checkpoint must remain `GAP_PRESERVED`; use the explicit Fresh reconciliation +operation after backing up the isolated runtime. Reconciliation establishes a +new source generation from retained history without claiming the missing source +interval was observed. Old-generation evidence cannot activate current cards, +while matching durable histories and human decisions remain available. + +## Data minimization + +Fresh persists bounded, sanitized observation evidence and durable identity / +decision history. It must never persist or expose autobrr tokens, credentials, +download URLs, tracker/private URLs, passkeys, cookies, raw action payloads, +Axios objects, raw provider errors, or full TMDB responses. Public `/fresh` +returns the paginated effective media projection only; candidate evidence and +mutation endpoints remain administrator-only. diff --git a/docs/SEERR_DOWNSTREAM_ROADMAP.md b/docs/SEERR_DOWNSTREAM_ROADMAP.md index 95a9bca763..038b37b9c6 100644 --- a/docs/SEERR_DOWNSTREAM_ROADMAP.md +++ b/docs/SEERR_DOWNSTREAM_ROADMAP.md @@ -29,7 +29,8 @@ downstream home. - `origin/downstream-main`: `f74657aa501e1fc28bf314673a0cedde167d47e6`. - Current infrastructure branch: `chore/downstream-image-publication`. - Upstream queue-sync contribution: [seerr-team/seerr#3535](https://github.com/seerr-team/seerr/pull/3535), open against `develop`. -- Fresh discovery: merged into `downstream-main` and released as `custom-v1.0.7`. +- Fresh discovery: merged into `downstream-main`; current architecture and + operating invariants are documented in [Fresh discovery](FRESH.md). - Local integration harness: the separate `seerr-harness` workspace. ## Phase 1 — safe downstream foundation @@ -49,6 +50,10 @@ downstream home. ## Phase 2 — downstream features and repeatable upkeep - [x] Deliver persistent Fresh discovery with focused tests and downstream release validation. +- [x] Refine Fresh with irreversible Movie/Season/Special history, typed manual + correction, admission overrides, and an administrator work queue. +- [ ] Add administrator shortcuts from Fresh/Discover to Fresh Settings. +- [ ] Improve validation performance without weakening exact-tree evidence. - [ ] Evaluate direct/on-demand Download Sync refresh as a separate downstream feature. - [ ] Add automated upstream change detection. - [ ] Add reusable upstream sync-PR generation if manual sync becomes costly. diff --git a/docs/SEERR_HANDOFF.md b/docs/SEERR_HANDOFF.md index f25b2c727a..cb590e5b3b 100644 --- a/docs/SEERR_HANDOFF.md +++ b/docs/SEERR_HANDOFF.md @@ -54,9 +54,11 @@ maintenance branch. ### Fresh Fresh is merged into `downstream-main` and maintained in this repository. The -authoritative released implementation is `custom-v1.0.7`; the earlier isolated -prototype checkout has been retired. Continue Fresh maintenance from the primary -Seerr checkout and preserve its persistent incremental architecture. +earlier isolated prototype checkout has been retired. Continue Fresh maintenance +from the primary Seerr checkout and preserve its persistent incremental +architecture. The durable Movie/Season/Special identity, irreversible-history, +typed correction, override, rebuild, and GAP-recovery contract is documented in +[Fresh discovery](FRESH.md). ### Local harness diff --git a/seerr-api.yml b/seerr-api.yml index 48d8145c1a..86378f2abb 100644 --- a/seerr-api.yml +++ b/seerr-api.yml @@ -876,7 +876,7 @@ components: minimum: 1 maximum: 365 default: 90 - description: Maximum canonical-media age at the qualifying source observation. + description: Movie normal eligibility window and TV recent-season activity sanity window. freshVisibilityDays: type: integer minimum: 1 @@ -1106,21 +1106,41 @@ components: type: object required: - candidateId + - revision - displayTitle + - parsedTitle + - parsedMediaType - mediaType - resolutionStatus - displayStatus - firstObservedAt - lastObservedAt - attemptCount + - automaticReasons + - sourceTitleSamples + - observationCount - active - actionable + - actions properties: candidateId: { type: integer } + revision: { type: integer, minimum: 1 } displayTitle: { type: string } + parsedTitle: { type: string } + parsedMediaType: { type: string, enum: [movie, tv] } mediaType: { type: string, enum: [movie, tv] } matchYear: { type: integer } + seasonNumber: { type: integer, minimum: 0 } + episodeNumber: { type: integer, minimum: 0 } resolutionStatus: { type: integer } + automaticResolution: + type: object + required: [status, mediaType] + properties: + status: { type: integer } + mediaType: { type: string, enum: [movie, tv] } + tmdbId: { type: integer, minimum: 1 } + failureReason: { type: string } displayStatus: type: string enum: @@ -1136,6 +1156,9 @@ components: excluded_content_filter, visibility_expired, active_fresh, + needs_attention, + reviewable, + historical, ] tmdbId: { type: integer } firstObservedAt: { type: string, format: date-time } @@ -1146,6 +1169,54 @@ components: resolvedAt: { type: string, format: date-time } failureReason: { type: string } membershipReason: { type: string } + automaticReasons: + type: array + items: { type: string } + sourceTitleSamples: + type: array + maxItems: 5 + items: { type: string } + observationCount: { type: integer, minimum: 0 } + manualResolution: + type: object + required: [mediaType, tmdbId, canonicalTitle, updatedAt, revision] + properties: + mediaType: { type: string, enum: [movie, tv] } + tmdbId: { type: integer, minimum: 1 } + canonicalTitle: { type: string } + actorUserId: { type: integer, minimum: 1 } + updatedAt: { type: string, format: date-time } + revision: { type: integer, minimum: 1 } + admissionOverride: + type: object + required: [updatedAt, revision] + properties: + actorUserId: { type: integer, minimum: 1 } + updatedAt: { type: string, format: date-time } + revision: { type: integer, minimum: 1 } + discoveryHistory: + type: object + required: + [ + identityKind, + admitted, + legacyProjection, + admissionReason, + activitySource, + ] + properties: + identityKind: + type: string + enum: [movie, season, special, legacy_tv] + seasonNumber: { type: integer, minimum: 0 } + episodeNumber: { type: integer, minimum: 1 } + admitted: { type: boolean } + legacyProjection: { type: boolean } + admissionReason: { type: string } + activityDate: { type: string, format: date } + activitySource: { type: string } + firstFreshAt: { type: string, format: date-time } + visibleUntil: { type: string, format: date-time } firstSeenAt: { type: string, format: date-time } lastSeenAt: { type: string, format: date-time } mediaDate: { type: string } @@ -1182,6 +1253,14 @@ components: visibleUntil: { type: string, format: date-time } active: { type: boolean } actionable: { type: boolean } + actions: + type: object + required: [resolve, resetResolution, admit, removeOverride] + properties: + resolve: { type: boolean } + resetResolution: { type: boolean } + admit: { type: boolean } + removeOverride: { type: boolean } FreshCandidateDiagnosticsResponse: type: object required: [pageInfo, results, summary] @@ -1204,9 +1283,11 @@ components: type: integer FreshManualResolutionResponse: type: object - required: [candidateId, mediaType, tmdbId, title, status, active] + required: + [candidateId, parsedMediaType, mediaType, tmdbId, title, status, active] properties: candidateId: { type: integer } + parsedMediaType: { type: string, enum: [movie, tv] } mediaType: { type: string, enum: [movie, tv] } tmdbId: { type: integer } title: { type: string } @@ -6371,6 +6452,8 @@ paths: visibility_expired, active_fresh, needs_attention, + reviewable, + historical, ] default: all - in: query @@ -6379,6 +6462,7 @@ paths: type: string enum: [ + priority, title.asc, title.desc, status, @@ -6389,7 +6473,31 @@ paths: last_seen.desc, last_seen.asc, ] - default: last_seen.desc + default: priority + - in: query + name: reasonFamily + schema: + type: string + enum: [all, resolution, admission, content, history, source] + default: all + - in: query + name: seasonEvidence + schema: + type: string + enum: [all, known, unknown] + default: all + - in: query + name: manualResolution + schema: + type: string + enum: [all, present, absent] + default: all + - in: query + name: admissionOverride + schema: + type: string + enum: [all, present, absent] + default: all responses: '200': description: Persistent current-generation Fresh candidate state @@ -6413,10 +6521,12 @@ paths: application/json: schema: type: object - required: [tmdbId] + required: [mediaType, tmdbId, expectedRevision] additionalProperties: false properties: + mediaType: { type: string, enum: [movie, tv] } tmdbId: { type: integer, minimum: 1 } + expectedRevision: { type: integer, minimum: 1 } responses: '200': description: Validated canonical TMDB resolution @@ -6424,6 +6534,72 @@ paths: application/json: schema: $ref: '#/components/schemas/FreshManualResolutionResponse' + /settings/fresh/candidates/{candidateId}/reset-resolution: + post: + summary: Reset a typed Fresh manual canonical resolution + tags: [settings] + parameters: + - in: path + name: candidateId + required: true + schema: { type: integer, minimum: 1 } + requestBody: + required: true + content: + application/json: + schema: + type: object + additionalProperties: false + required: [expectedRevision] + properties: + expectedRevision: { type: integer, minimum: 1 } + responses: + '200': + description: Manual resolution reset + /settings/fresh/candidates/{candidateId}/admit: + post: + summary: Admit a stable excluded Fresh identity through an override + tags: [settings] + parameters: + - in: path + name: candidateId + required: true + schema: { type: integer, minimum: 1 } + requestBody: + required: true + content: + application/json: + schema: + type: object + additionalProperties: false + required: [expectedRevision] + properties: + expectedRevision: { type: integer, minimum: 1 } + responses: + '200': + description: Admission override applied + /settings/fresh/candidates/{candidateId}/remove-override: + post: + summary: Remove a Fresh admission override + tags: [settings] + parameters: + - in: path + name: candidateId + required: true + schema: { type: integer, minimum: 1 } + requestBody: + required: true + content: + application/json: + schema: + type: object + additionalProperties: false + required: [expectedRevision] + properties: + expectedRevision: { type: integer, minimum: 1 } + responses: + '200': + description: Admission override removed /settings/fresh/refresh: post: summary: Force and await a Fresh refresh @@ -6438,8 +6614,8 @@ paths: $ref: '#/components/schemas/FreshStatus' /settings/fresh/rebuild: post: - summary: Destructively rebuild Fresh-owned discovery data - description: Clears only persisted Fresh observations, candidates, media, and synchronization state, then awaits the normal initial synchronization path. Fresh configuration and non-Fresh Seerr state are preserved. + summary: Rebuild source-derived Fresh discovery data + description: Clears Fresh observations, automatic candidates, and synchronization state before replaying retained source history. Canonical media metadata, irreversible discovery history, typed manual resolutions, and admission overrides are preserved. tags: - settings responses: diff --git a/server/api/autobrr.test.ts b/server/api/autobrr.test.ts index fab438285e..e413d6eb8a 100644 --- a/server/api/autobrr.test.ts +++ b/server/api/autobrr.test.ts @@ -46,7 +46,12 @@ describe('autobrr Fresh boundary', () => { releaseId: '20', mediaType: 'movie', title: 'Example Movie', + sourceTitle: 'Example Movie', year: 2026, + seasonNumber: -1, + episodeNumber: -1, + explicitSeason: false, + explicitSpecial: false, observedAt: Date.parse('2026-09-26T10:00:00Z'), availabilityType: 'digital', }, @@ -54,7 +59,12 @@ describe('autobrr Fresh boundary', () => { releaseId: '19', mediaType: 'tv', title: 'Example Show', + sourceTitle: 'Example Show', year: 0, + seasonNumber: -1, + episodeNumber: -1, + explicitSeason: false, + explicitSpecial: false, observedAt: Date.parse('2026-09-26T10:00:00Z'), availabilityType: 'digital', }, @@ -85,6 +95,82 @@ describe('autobrr Fresh boundary', () => { assert.doesNotMatch(JSON.stringify(parsed.releases), /WEB-DL|BluRay|CAM/); }); + it('preserves bounded explicit TV season and special evidence without raw payloads', () => { + const parsed = parseReleasePage( + { + data: [ + row({ + id: 20, + type: 6, + title: 'Last Week Tonight with John Oliver', + torrent_name: 'Last.Week.Tonight.with.John.Oliver.S13E24.1080p', + season: 13, + episode: 24, + }), + row({ + id: 19, + type: 11, + title: 'Example Special', + torrent_name: 'Example.Special.S00E14.1080p', + season: 0, + episode: 14, + }), + row({ + id: 18, + type: 6, + title: 'Unknown Season', + torrent_name: 'Unknown.Season.1080p', + season: 0, + episode: 0, + }), + ], + next_cursor: 18, + }, + { id: 7 } + ); + + assert.deepEqual( + parsed.releases.map((release) => ({ + title: release.title, + sourceTitle: release.sourceTitle, + season: release.seasonNumber, + episode: release.episodeNumber, + explicitSeason: release.explicitSeason, + explicitSpecial: release.explicitSpecial, + })), + [ + { + title: 'Last Week Tonight with John Oliver', + sourceTitle: 'Last.Week.Tonight.with.John.Oliver.S13E24.1080p', + season: 13, + episode: 24, + explicitSeason: true, + explicitSpecial: false, + }, + { + title: 'Example Special', + sourceTitle: 'Example.Special.S00E14.1080p', + season: 0, + episode: 14, + explicitSeason: false, + explicitSpecial: true, + }, + { + title: 'Unknown Season', + sourceTitle: 'Unknown.Season.1080p', + season: 0, + episode: 0, + explicitSeason: false, + explicitSpecial: false, + }, + ] + ); + assert.doesNotMatch( + JSON.stringify(parsed.releases), + /download_url|action_status|passkey|tracker\.invalid/ + ); + }); + it('fails closed for malformed ordering and cursor boundaries', () => { assert.throws(() => parseReleasePage( diff --git a/server/api/autobrr.ts b/server/api/autobrr.ts index cda7afb095..c72ee8308a 100644 --- a/server/api/autobrr.ts +++ b/server/api/autobrr.ts @@ -16,7 +16,12 @@ export interface FreshRelease { releaseId: string; mediaType: FreshMediaType; title: string; + sourceTitle: string; year: number; + seasonNumber: number; + episodeNumber: number; + explicitSeason: boolean; + explicitSpecial: boolean; observedAt: number; availabilityType: FreshAvailabilityType; } @@ -41,6 +46,23 @@ export const safeTitle = (value: unknown): value is string => !Array.from(value).some((character) => character.charCodeAt(0) < 32) && !/:\/\/|passkey\s*=|apikey\s*=|api_key\s*=/i.test(value); +const boundedNumber = (value: unknown, maximum: number): number => + Number.isSafeInteger(value) && Number(value) >= 0 && Number(value) <= maximum + ? Number(value) + : -1; + +const sanitizedSourceTitle = (row: Record): string => { + for (const value of [ + row.torrent_name, + row.release_name, + row.name, + row.title, + ]) { + if (safeTitle(value)) return value.trim(); + } + return ''; +}; + export interface FreshReleasePage { releases: FreshRelease[]; releaseIds: string[]; @@ -154,11 +176,31 @@ export function parseReleasePage( ? Number(row.year) : 0; if (mediaType === 'movie' && !year) continue; + const sourceTitle = sanitizedSourceTitle(row); + if (!sourceTitle) continue; + const seasonNumber = boundedNumber(row.season, 10000); + const episodeNumber = boundedNumber(row.episode, 100000); + const explicitSeason = + mediaType === 'tv' && + (seasonNumber > 0 || + /(?:^|[. _-])S0*[1-9]\d{0,3}(?:E\d{1,6})?(?:[. _-]|$)/i.test( + sourceTitle + )); + const explicitSpecial = + mediaType === 'tv' && + seasonNumber === 0 && + episodeNumber > 0 && + /(?:^|[. _-])S0+E\d{1,6}(?:[. _-]|$)/i.test(sourceTitle); releases.push({ releaseId: String(row.id), mediaType, title: row.title.trim(), + sourceTitle, year, + seasonNumber, + episodeNumber, + explicitSeason, + explicitSpecial, observedAt, availabilityType: availabilityType(row.source), }); diff --git a/server/datasource.ts b/server/datasource.ts index 59ca435e81..4b53ec306c 100644 --- a/server/datasource.ts +++ b/server/datasource.ts @@ -1,6 +1,9 @@ import { Blocklist } from '@server/entity/Blocklist'; import DiscoverSlider from '@server/entity/DiscoverSlider'; +import FreshAdmissionOverride from '@server/entity/FreshAdmissionOverride'; import FreshCandidate from '@server/entity/FreshCandidate'; +import FreshDiscoveryHistory from '@server/entity/FreshDiscoveryHistory'; +import FreshManualResolution from '@server/entity/FreshManualResolution'; import FreshMedia from '@server/entity/FreshMedia'; import FreshObservation from '@server/entity/FreshObservation'; import { FreshSyncState } from '@server/entity/FreshSyncState'; @@ -31,7 +34,10 @@ const DB_SSL_PREFIX = 'DB_SSL_'; const entities = [ Blocklist, DiscoverSlider, + FreshAdmissionOverride, FreshCandidate, + FreshDiscoveryHistory, + FreshManualResolution, FreshMedia, FreshObservation, FreshSyncState, diff --git a/server/entity/FreshAdmissionOverride.ts b/server/entity/FreshAdmissionOverride.ts new file mode 100644 index 0000000000..0a13a69e29 --- /dev/null +++ b/server/entity/FreshAdmissionOverride.ts @@ -0,0 +1,45 @@ +import { DbAwareColumn, resolveDbType } from '@server/utils/DbColumnHelper'; +import { + Column, + Entity, + Index, + PrimaryGeneratedColumn, + Unique, + UpdateDateColumn, +} from 'typeorm'; + +/** Human admission policy, separate from automatic eligibility and identity. */ +@Entity() +@Unique('UQ_fresh_admission_override_identity', [ + 'mediaType', + 'tmdbId', + 'identityKind', + 'seasonKey', + 'specialEpisodeKey', +]) +@Index('IDX_fresh_admission_override_active', ['active']) +export default class FreshAdmissionOverride { + @PrimaryGeneratedColumn() public id: number; + @Column({ type: 'varchar', length: 8 }) public mediaType: 'movie' | 'tv'; + @Column({ type: 'int' }) public tmdbId: number; + @Column({ type: 'varchar', length: 16 }) + public identityKind: 'movie' | 'season' | 'special'; + @Column({ type: 'int', default: -1 }) public seasonKey = -1; + @Column({ type: 'int', default: -1 }) public specialEpisodeKey = -1; + @Column({ default: true }) public active: boolean = true; + @Column({ type: 'int', default: 1 }) public revision = 1; + @Column({ type: 'int', nullable: true }) public actorUserId?: number | null; + @DbAwareColumn({ type: 'datetime', nullable: true }) + public firstAdmittedAt?: Date | null; + @DbAwareColumn({ type: 'datetime', default: () => 'CURRENT_TIMESTAMP' }) + public createdAt: Date; + @UpdateDateColumn({ + type: resolveDbType('datetime'), + default: () => 'CURRENT_TIMESTAMP', + }) + public updatedAt: Date; + + constructor(init?: Partial) { + Object.assign(this, init); + } +} diff --git a/server/entity/FreshCandidate.ts b/server/entity/FreshCandidate.ts index acf4c19d7c..c9be5550cb 100644 --- a/server/entity/FreshCandidate.ts +++ b/server/entity/FreshCandidate.ts @@ -13,11 +13,13 @@ import { } from 'typeorm'; @Entity() -@Unique('UQ_fresh_candidate_identity', [ +@Unique('UQ_fresh_candidate_identity_v2', [ 'sourceGeneration', 'mediaType', 'normalizedTitle', 'matchYear', + 'seasonKey', + 'specialEpisodeKey', ]) @Index('IDX_fresh_candidate_due', [ 'sourceGeneration', @@ -32,9 +34,30 @@ export default class FreshCandidate { @Column({ type: 'varchar', length: 300 }) public normalizedTitle: string; @Column({ type: 'varchar', length: 300 }) public displayTitle: string; @Column({ type: 'int', default: 0 }) public matchYear = 0; + @Column({ type: 'int', default: -1 }) public seasonKey = -1; + @Column({ type: 'int', default: -1 }) public specialEpisodeKey = -1; + @Column({ default: false }) public explicitSeason: boolean = false; + @Column({ default: false }) public explicitSpecial: boolean = false; + @Column({ type: 'int', default: 1 }) public comparisonVersion = 1; + @Column({ type: 'varchar', length: 64, default: '' }) + public sourceEvidenceKey = ''; + @Column({ type: 'int', default: 1 }) public revision = 1; @Column({ type: 'int', default: FreshCandidateStatus.UNRESOLVED }) public status = FreshCandidateStatus.UNRESOLVED; @Column({ type: 'int', nullable: true }) public tmdbId?: number | null; + @Column({ type: 'int', nullable: true }) public automaticTmdbId?: + | number + | null; + @Column({ type: 'int', nullable: true }) public automaticFreshMediaId?: + | number + | null; + @Column({ type: 'int', nullable: true }) public automaticStatus?: + | number + | null; + @Column({ type: 'varchar', length: 64, nullable: true }) + public automaticFailureReason?: string | null; + @Column({ type: 'varchar', length: 8, nullable: true }) + public effectiveMediaType?: 'movie' | 'tv' | null; @Column({ type: 'int', nullable: true }) public freshMediaId?: number | null; @ManyToOne(() => FreshMedia, { nullable: true, onDelete: 'SET NULL' }) @JoinColumn({ name: 'freshMediaId' }) diff --git a/server/entity/FreshDiscoveryHistory.ts b/server/entity/FreshDiscoveryHistory.ts new file mode 100644 index 0000000000..101879ec33 --- /dev/null +++ b/server/entity/FreshDiscoveryHistory.ts @@ -0,0 +1,73 @@ +import { DbAwareColumn, resolveDbType } from '@server/utils/DbColumnHelper'; +import { + Column, + Entity, + Index, + PrimaryGeneratedColumn, + Unique, + UpdateDateColumn, +} from 'typeorm'; + +const stringArray = { + to: (value?: string[]) => JSON.stringify(value ?? []), + from: (value?: string) => (value ? (JSON.parse(value) as string[]) : []), +}; + +export type FreshHistoryIdentityKind = + | 'movie' + | 'season' + | 'special' + | 'legacy_tv'; + +/** Durable, irreversible Fresh admission history. */ +@Entity() +@Unique('UQ_fresh_discovery_history_identity', [ + 'mediaType', + 'tmdbId', + 'identityKind', + 'seasonKey', + 'specialEpisodeKey', +]) +@Index('IDX_fresh_discovery_history_projection', [ + 'mediaType', + 'tmdbId', + 'visibleUntil', +]) +export default class FreshDiscoveryHistory { + @PrimaryGeneratedColumn() public id: number; + @Column({ type: 'varchar', length: 8 }) public mediaType: 'movie' | 'tv'; + @Column({ type: 'int' }) public tmdbId: number; + @Column({ type: 'varchar', length: 16 }) + public identityKind: FreshHistoryIdentityKind; + @Column({ type: 'int', default: -1 }) public seasonKey = -1; + @Column({ type: 'int', default: -1 }) public specialEpisodeKey = -1; + @Column({ default: false }) public admitted: boolean = false; + @Column({ default: false }) public legacyProjection: boolean = false; + @DbAwareColumn({ type: 'datetime' }) public firstObservedAt: Date; + @DbAwareColumn({ type: 'datetime' }) public lastObservedAt: Date; + @DbAwareColumn({ type: 'datetime', nullable: true }) + public firstFreshAt?: Date | null; + @DbAwareColumn({ type: 'datetime', nullable: true }) + public visibleUntil?: Date | null; + @Column({ type: 'varchar', length: 10, nullable: true }) + public activityDate?: string | null; + @Column({ type: 'varchar', length: 32, default: 'unavailable' }) + public activitySource = 'unavailable'; + @Column({ type: 'varchar', length: 64, default: 'not_evaluated' }) + public admissionReason = 'not_evaluated'; + @Column({ type: 'text', transformer: stringArray }) + public automaticReasons: string[] = []; + @Column({ type: 'int', default: 0 }) public firstSeenGeneration = 0; + @Column({ type: 'int', default: 0 }) public lastSeenGeneration = 0; + @DbAwareColumn({ type: 'datetime', default: () => 'CURRENT_TIMESTAMP' }) + public createdAt: Date; + @UpdateDateColumn({ + type: resolveDbType('datetime'), + default: () => 'CURRENT_TIMESTAMP', + }) + public updatedAt: Date; + + constructor(init?: Partial) { + Object.assign(this, init); + } +} diff --git a/server/entity/FreshManualResolution.ts b/server/entity/FreshManualResolution.ts new file mode 100644 index 0000000000..6ee4ca6e64 --- /dev/null +++ b/server/entity/FreshManualResolution.ts @@ -0,0 +1,41 @@ +import { DbAwareColumn, resolveDbType } from '@server/utils/DbColumnHelper'; +import { + Column, + Entity, + Index, + PrimaryGeneratedColumn, + Unique, + UpdateDateColumn, +} from 'typeorm'; + +/** A typed human correction bound to versioned, sanitized source evidence. */ +@Entity() +@Unique('UQ_fresh_manual_resolution_source', [ + 'sourceEvidenceVersion', + 'sourceEvidenceKey', +]) +@Index('IDX_fresh_manual_resolution_identity', ['mediaType', 'tmdbId']) +export default class FreshManualResolution { + @PrimaryGeneratedColumn() public id: number; + @Column({ type: 'int', default: 1 }) public sourceEvidenceVersion = 1; + @Column({ type: 'varchar', length: 64 }) public sourceEvidenceKey: string; + @Column({ type: 'varchar', length: 8 }) public mediaType: 'movie' | 'tv'; + @Column({ type: 'int' }) public tmdbId: number; + @Column({ type: 'varchar', length: 300 }) public canonicalTitle: string; + @Column({ type: 'varchar', length: 10, nullable: true }) + public canonicalDate?: string | null; + @Column({ default: true }) public active: boolean = true; + @Column({ type: 'int', default: 1 }) public revision = 1; + @Column({ type: 'int', nullable: true }) public actorUserId?: number | null; + @DbAwareColumn({ type: 'datetime', default: () => 'CURRENT_TIMESTAMP' }) + public createdAt: Date; + @UpdateDateColumn({ + type: resolveDbType('datetime'), + default: () => 'CURRENT_TIMESTAMP', + }) + public updatedAt: Date; + + constructor(init?: Partial) { + Object.assign(this, init); + } +} diff --git a/server/entity/FreshMedia.ts b/server/entity/FreshMedia.ts index 19df21741b..f2cd80d069 100644 --- a/server/entity/FreshMedia.ts +++ b/server/entity/FreshMedia.ts @@ -14,7 +14,8 @@ const numberArray = { }; const stringArray = { to: (value?: string[]) => JSON.stringify(value ?? []), - from: (value?: string) => (value ? (JSON.parse(value) as string[]) : []), + from: (value?: string | string[]) => + Array.isArray(value) ? value : value ? (JSON.parse(value) as string[]) : [], }; @Entity() @@ -37,6 +38,8 @@ export default class FreshMedia { @Column({ default: false }) public admitted: boolean = false; @Column({ type: 'varchar', length: 64, nullable: true }) public membershipReason?: string | null; + @Column({ type: 'text', default: () => "'[]'", transformer: stringArray }) + public automaticReasons: string[] = []; @Column({ type: 'int', default: 0 }) public lastMatchedGeneration = 0; @DbAwareColumn({ type: 'datetime' }) public firstSeenAt: Date; @DbAwareColumn({ type: 'datetime' }) public lastSeenAt: Date; diff --git a/server/entity/FreshObservation.ts b/server/entity/FreshObservation.ts index afd32430bc..42b27c09fe 100644 --- a/server/entity/FreshObservation.ts +++ b/server/entity/FreshObservation.ts @@ -29,8 +29,17 @@ export default class FreshObservation { public candidate: FreshCandidate; @Column({ type: 'varchar', length: 8 }) public mediaType: 'movie' | 'tv'; @Column({ type: 'varchar', length: 300 }) public title: string; + @Column({ type: 'varchar', length: 300, default: '' }) public sourceTitle = + ''; @Column({ type: 'varchar', length: 300 }) public normalizedTitle: string; @Column({ type: 'int', default: 0 }) public year = 0; + @Column({ type: 'int', default: -1 }) public seasonNumber = -1; + @Column({ type: 'int', default: -1 }) public episodeNumber = -1; + @Column({ default: false }) public explicitSeason: boolean = false; + @Column({ default: false }) public explicitSpecial: boolean = false; + @Column({ type: 'int', default: 1 }) public comparisonVersion = 1; + @Column({ type: 'varchar', length: 64, default: '' }) + public sourceEvidenceKey = ''; @Column({ type: 'varchar', length: 16, default: 'unknown' }) public availabilityType: FreshAvailabilityType = 'unknown'; @DbAwareColumn({ type: 'datetime' }) public observedAt: Date; diff --git a/server/lib/fresh/engine.ts b/server/lib/fresh/engine.ts index a4824432a0..35c1962e44 100644 --- a/server/lib/fresh/engine.ts +++ b/server/lib/fresh/engine.ts @@ -8,6 +8,7 @@ import type { TmdbMovieResult, TmdbSearchMovieResponse, TmdbSearchTvResponse, + TmdbSeasonWithEpisodes, TmdbTvDetails, TmdbTvResult, } from '@server/api/themoviedb/interfaces'; @@ -17,10 +18,19 @@ import { FreshContinuityStatus, } from '@server/constants/fresh'; import dataSource from '@server/datasource'; +import FreshAdmissionOverride from '@server/entity/FreshAdmissionOverride'; import FreshCandidate from '@server/entity/FreshCandidate'; +import FreshDiscoveryHistory from '@server/entity/FreshDiscoveryHistory'; +import FreshManualResolution from '@server/entity/FreshManualResolution'; import FreshMedia from '@server/entity/FreshMedia'; import FreshObservation from '@server/entity/FreshObservation'; import { FreshSyncState } from '@server/entity/FreshSyncState'; +import { + evaluateMovieAdmission, + evaluateTvAdmission, + freshVisibleUntil, + recurringIdentityForEvidence, +} from '@server/lib/fresh/history'; import { applyFreshMembership, evaluateAdmissionEvidence, @@ -28,6 +38,8 @@ import { selectMovieAvailabilityDates, } from '@server/lib/fresh/membership'; import { + FRESH_COMPARISON_VERSION, + freshSourceEvidenceKey, normalizeFreshTitle, validReleaseId, } from '@server/lib/fresh/normalize'; @@ -42,7 +54,7 @@ import type { import { getSettings, type FreshSettings } from '@server/lib/settings'; import logger from '@server/logger'; import { createHash } from 'crypto'; -import type { DataSource } from 'typeorm'; +import type { DataSource, EntityManager } from 'typeorm'; import { In, LessThan } from 'typeorm'; const MAX_SOURCE_PAGES = 10000; @@ -144,7 +156,8 @@ export interface FreshEngineDependencies { tmdb: Pick< TheMovieDb, 'searchMoviesStrict' | 'searchTvShowsStrict' | 'getMovie' | 'getTvShow' - >; + > & + Partial>; now: () => Date; cancelled: () => boolean; region?: () => string; @@ -372,12 +385,7 @@ export class FreshEngine { item.lastMatchedGeneration === state.generation || state.continuityStatus === FreshContinuityStatus.GAP_PRESERVED ) { - this.applyAdmissionEvidence( - item, - await this.observationsForMedia(item.id), - settings.mediaEligibilityDays - ); - applyFreshMembership(item, settings, this.dependencies.now()); + await this.refreshMediaProjection(item, state, settings); } else { item.active = false; item.membershipReason = 'source_generation_inactive'; @@ -405,7 +413,10 @@ export class FreshEngine { async resolveManually( candidateId: number, + mediaType: 'movie' | 'tv', tmdbId: number, + expectedRevision: number, + actorUserId: number | undefined, settings: FreshSettings ): Promise<{ candidate: FreshCandidate; media: FreshMedia }> { if (!Number.isSafeInteger(candidateId) || candidateId < 1) { @@ -414,10 +425,24 @@ export class FreshEngine { if (!Number.isSafeInteger(tmdbId) || tmdbId < 1) { throw new Error('invalid_tmdb_id'); } + if (!['movie', 'tv'].includes(mediaType)) + throw new Error('invalid_media_type'); + if (!Number.isSafeInteger(expectedRevision) || expectedRevision < 1) + throw new Error('invalid_revision'); const candidate = await this.dependencies.database .getRepository(FreshCandidate) .findOneBy({ id: candidateId }); if (!candidate) throw new Error('candidate_not_found'); + if (!candidate.sourceEvidenceKey) { + candidate.sourceEvidenceKey = freshSourceEvidenceKey({ + sourceTitle: candidate.displayTitle, + year: candidate.matchYear, + seasonNumber: candidate.seasonKey, + episodeNumber: candidate.specialEpisodeKey, + }); + } + if (candidate.revision !== expectedRevision) + throw new Error('stale_candidate'); if ( ![FreshCandidateStatus.NO_MATCH, FreshCandidateStatus.AMBIGUOUS].includes( candidate.status @@ -432,7 +457,7 @@ export class FreshEngine { throw new Error('candidate_not_current'); } const details = - candidate.mediaType === 'movie' + mediaType === 'movie' ? await this.dependencies.tmdb.getMovie({ movieId: tmdbId }) : await this.dependencies.tmdb.getTvShow({ tvId: tmdbId }); if (details.id !== tmdbId) throw new Error('invalid_tmdb_response'); @@ -441,7 +466,9 @@ export class FreshEngine { details, state, settings, - this.dependencies.now() + this.dependencies.now(), + await this.seasonDetailsForCandidate(candidate, details.id, mediaType), + { mediaType, actorUserId, expectedRevision } ); return { candidate: await this.dependencies.database @@ -451,6 +478,233 @@ export class FreshEngine { }; } + async resetManualResolution( + candidateId: number, + expectedRevision: number, + settings: FreshSettings + ): Promise { + const result = await this.dependencies.database.transaction( + async (manager) => { + const state = await manager + .getRepository(FreshSyncState) + .findOneByOrFail({ id: FRESH_SYNC_STATE_ID }); + const candidates = manager.getRepository(FreshCandidate); + const candidate = await candidates.findOneBy({ id: candidateId }); + if (!candidate) throw new Error('candidate_not_found'); + if ( + candidate.revision !== expectedRevision || + candidate.sourceGeneration !== state.generation + ) + throw new Error('stale_candidate'); + const resolution = await manager + .getRepository(FreshManualResolution) + .findOneBy({ + sourceEvidenceVersion: 1, + sourceEvidenceKey: candidate.sourceEvidenceKey, + active: true, + }); + if (!resolution) throw new Error('manual_resolution_missing'); + resolution.active = false; + resolution.revision += 1; + await manager.getRepository(FreshManualResolution).save(resolution); + const manualMediaId = candidate.freshMediaId; + candidate.status = + candidate.automaticStatus ?? FreshCandidateStatus.UNRESOLVED; + candidate.tmdbId = candidate.automaticTmdbId ?? null; + candidate.freshMediaId = candidate.automaticFreshMediaId ?? null; + candidate.effectiveMediaType = candidate.mediaType; + candidate.lastFailureReason = candidate.automaticFailureReason ?? null; + candidate.revision += 1; + const saved = await candidates.save(candidate); + const mediaRepository = manager.getRepository(FreshMedia); + if (manualMediaId && manualMediaId !== candidate.freshMediaId) { + const stillReferenced = await candidates + .createQueryBuilder('candidate') + .where('candidate.sourceGeneration = :generation', { + generation: state.generation, + }) + .andWhere('candidate.freshMediaId = :mediaId', { + mediaId: manualMediaId, + }) + .getCount(); + if (!stillReferenced) { + const manualMedia = await mediaRepository.findOneBy({ + id: manualMediaId, + }); + if (manualMedia) { + manualMedia.active = false; + manualMedia.lastMatchedGeneration = 0; + manualMedia.membershipReason = 'source_generation_inactive'; + await mediaRepository.save(manualMedia); + } + } + } + const automaticMedia = candidate.freshMediaId + ? await mediaRepository.findOneBy({ id: candidate.freshMediaId }) + : undefined; + if (automaticMedia) + automaticMedia.lastMatchedGeneration = state.generation; + return { saved, automaticMedia, state }; + } + ); + if (result.automaticMedia) { + await this.refreshMediaProjection( + result.automaticMedia, + result.state, + settings + ); + await this.dependencies.database + .getRepository(FreshMedia) + .save(result.automaticMedia); + } + return result.saved; + } + + async setAdmissionOverride( + candidateId: number, + expectedRevision: number, + settings: FreshSettings, + actorUserId?: number + ): Promise { + const now = this.dependencies.now(); + const result = await this.dependencies.database.transaction( + async (manager) => { + const state = await manager + .getRepository(FreshSyncState) + .findOneByOrFail({ id: FRESH_SYNC_STATE_ID }); + const candidates = manager.getRepository(FreshCandidate); + const candidate = await candidates.findOneBy({ id: candidateId }); + if (!candidate) throw new Error('candidate_not_found'); + if ( + candidate.revision !== expectedRevision || + candidate.sourceGeneration !== state.generation + ) + throw new Error('stale_candidate'); + if (!candidate.tmdbId || !candidate.freshMediaId) + throw new Error('override_not_actionable'); + const effectiveMediaType = + candidate.effectiveMediaType ?? candidate.mediaType; + const identity = recurringIdentityForEvidence({ + ...candidate, + mediaType: effectiveMediaType, + }); + if (!identity) throw new Error('override_not_actionable'); + const histories = manager.getRepository(FreshDiscoveryHistory); + const history = await histories.findOneBy({ + mediaType: effectiveMediaType, + tmdbId: candidate.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + }); + if (!history) throw new Error('override_not_actionable'); + if ( + history.visibleUntil && + history.visibleUntil.getTime() < now.getTime() + ) + throw new Error('history_expired'); + const media = await manager + .getRepository(FreshMedia) + .findOneByOrFail({ id: candidate.freshMediaId }); + if (media.active) throw new Error('already_active'); + const repository = manager.getRepository(FreshAdmissionOverride); + let override = await repository.findOneBy({ + mediaType: effectiveMediaType, + tmdbId: candidate.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + }); + if (!history.firstFreshAt) { + history.firstFreshAt = override?.firstAdmittedAt ?? now; + history.visibleUntil = freshVisibleUntil( + history.firstFreshAt, + settings.freshVisibilityDays + ); + await histories.save(history); + } + override = new FreshAdmissionOverride({ + ...override, + mediaType: effectiveMediaType, + tmdbId: candidate.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + active: true, + revision: (override?.revision ?? 0) + 1, + actorUserId: actorUserId ?? null, + firstAdmittedAt: override?.firstAdmittedAt ?? history.firstFreshAt, + }); + await repository.save(override); + candidate.revision += 1; + await candidates.save(candidate); + return { candidate, media, state }; + } + ); + await this.refreshMediaProjection(result.media, result.state, settings); + await this.dependencies.database + .getRepository(FreshMedia) + .save(result.media); + return result.candidate; + } + + async removeAdmissionOverride( + candidateId: number, + expectedRevision: number, + settings: FreshSettings + ): Promise { + const result = await this.dependencies.database.transaction( + async (manager) => { + const state = await manager + .getRepository(FreshSyncState) + .findOneByOrFail({ id: FRESH_SYNC_STATE_ID }); + const candidates = manager.getRepository(FreshCandidate); + const candidate = await candidates.findOneBy({ id: candidateId }); + if (!candidate) throw new Error('candidate_not_found'); + if ( + candidate.revision !== expectedRevision || + candidate.sourceGeneration !== state.generation + ) + throw new Error('stale_candidate'); + if (!candidate.tmdbId || !candidate.freshMediaId) + throw new Error('override_not_actionable'); + const mediaType = candidate.effectiveMediaType ?? candidate.mediaType; + const identity = recurringIdentityForEvidence({ + ...candidate, + mediaType, + }); + if (!identity) throw new Error('override_not_actionable'); + const repository = manager.getRepository(FreshAdmissionOverride); + const override = await repository.findOneBy({ + mediaType, + tmdbId: candidate.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + active: true, + }); + if (!override) throw new Error('override_missing'); + override.active = false; + override.revision += 1; + await repository.save(override); + candidate.revision += 1; + await candidates.save(candidate); + return { + candidate, + media: await manager + .getRepository(FreshMedia) + .findOneByOrFail({ id: candidate.freshMediaId }), + state, + }; + } + ); + await this.refreshMediaProjection(result.media, result.state, settings); + await this.dependencies.database + .getRepository(FreshMedia) + .save(result.media); + return result.candidate; + } + private requireFilter( filters: AutobrrFilterOption[], filterId: number @@ -620,13 +874,31 @@ export class FreshEngine { } const normalizedTitle = normalizeFreshTitle(release.title); if (!normalizedTitle) continue; - const matchYear = release.mediaType === 'movie' ? release.year : 0; + const matchYear = release.year; + const seasonKey = + release.mediaType === 'tv' && + (release.explicitSeason || release.explicitSpecial) && + release.seasonNumber >= 0 + ? release.seasonNumber + : -1; + const specialEpisodeKey = + release.explicitSpecial && release.episodeNumber > 0 + ? release.episodeNumber + : -1; + const sourceEvidenceKey = freshSourceEvidenceKey({ + sourceTitle: release.title, + year: release.year, + seasonNumber: seasonKey, + episodeNumber: specialEpisodeKey, + }); const candidates = manager.getRepository(FreshCandidate); let candidate = await candidates.findOneBy({ sourceGeneration: generation, mediaType: release.mediaType, normalizedTitle, matchYear, + seasonKey, + specialEpisodeKey, }); const observedAt = new Date(release.observedAt); if (!candidate) { @@ -637,6 +909,13 @@ export class FreshEngine { normalizedTitle, displayTitle: release.title, matchYear, + seasonKey, + specialEpisodeKey, + explicitSeason: release.explicitSeason, + explicitSpecial: release.explicitSpecial, + comparisonVersion: FRESH_COMPARISON_VERSION, + sourceEvidenceKey, + effectiveMediaType: release.mediaType, firstObservedAt: observedAt, lastObservedAt: observedAt, }) @@ -647,6 +926,9 @@ export class FreshEngine { candidate.firstObservedAt = observedAt; if (observedAt > candidate.lastObservedAt) candidate.lastObservedAt = observedAt; + candidate.explicitSeason ||= release.explicitSeason; + candidate.explicitSpecial ||= release.explicitSpecial; + candidate.revision += 1; await candidates.save(candidate); if (candidate.freshMediaId) { const mediaRepository = manager.getRepository(FreshMedia); @@ -657,19 +939,26 @@ export class FreshEngine { if (observedAt > media.lastSeenAt) media.lastSeenAt = observedAt; media.lastMatchedGeneration = generation; - const existingEvidence = await observations.findBy({ - candidateId: candidate.id, - }); - this.applyAdmissionEvidence( - media, - [ - ...existingEvidence, - { availabilityType: release.availabilityType, observedAt }, - ], - settings.mediaEligibilityDays - ); - applyFreshMembership(media, settings, this.dependencies.now()); await mediaRepository.save(media); + const identity = recurringIdentityForEvidence(candidate); + if (identity) { + const histories = manager.getRepository( + FreshDiscoveryHistory + ); + const history = await histories.findOneBy({ + mediaType: media.mediaType, + tmdbId: media.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + }); + if (history) { + if (observedAt > history.lastObservedAt) + history.lastObservedAt = observedAt; + history.lastSeenGeneration = generation; + await histories.save(history); + } + } } } } @@ -681,8 +970,15 @@ export class FreshEngine { candidateId: candidate.id, mediaType: release.mediaType, title: release.title, + sourceTitle: release.sourceTitle, normalizedTitle, year: release.year, + seasonNumber: release.seasonNumber, + episodeNumber: release.episodeNumber, + explicitSeason: release.explicitSeason, + explicitSpecial: release.explicitSpecial, + comparisonVersion: FRESH_COMPARISON_VERSION, + sourceEvidenceKey, availabilityType: release.availabilityType, observedAt, }) @@ -837,6 +1133,7 @@ export class FreshEngine { outcome: 'rejected', reason: 'no_exact_match', }); + await this.reapplyManualResolution(candidate.id, state, settings); } else if (match.kind === 'ambiguous') { await this.finishUnmatched( candidate, @@ -854,6 +1151,7 @@ export class FreshEngine { outcome: 'rejected', reason: 'ambiguous_exact_match', }); + await this.reapplyManualResolution(candidate.id, state, settings); } else { const details = candidate.mediaType === 'movie' @@ -868,7 +1166,8 @@ export class FreshEngine { details, state, settings, - now + now, + await this.seasonDetailsForCandidate(candidate, details.id) ); if (saved.created) diagnostic.snapshot.counts.newFreshMedia!++; else diagnostic.snapshot.counts.existingFreshMediaUpdated!++; @@ -886,6 +1185,7 @@ export class FreshEngine { ? 'canonical_identity_resolved' : 'outside_eligibility_window', }); + await this.reapplyManualResolution(candidate.id, state, settings); } } catch (error) { if (error instanceof FreshRunError) throw error; @@ -945,6 +1245,51 @@ export class FreshEngine { | { kind: 'none' } | { kind: 'ambiguous' } | { kind: 'match'; result: TmdbMovieResult | TmdbTvResult } + > { + const year = candidate.matchYear || undefined; + let primary = await this.searchExactIdentity( + candidate, + candidate.displayTitle, + candidate.normalizedTitle, + candidate.mediaType === 'tv' && candidate.seasonKey >= 0 + ? undefined + : year + ); + if ( + candidate.mediaType === 'tv' && + candidate.seasonKey >= 0 && + year && + primary.kind === 'ambiguous' + ) { + primary = await this.searchExactIdentity( + candidate, + candidate.displayTitle, + candidate.normalizedTitle, + year + ); + } + if (primary.kind !== 'none' || candidate.mediaType !== 'tv') return primary; + const volumeFallback = candidate.displayTitle.match( + /^(.*?)\s+Volume\s+\d+\s*$/i + ); + if (!volumeFallback?.[1]?.trim()) return primary; + return this.searchExactIdentity( + candidate, + volumeFallback[1].trim(), + normalizeFreshTitle(volumeFallback[1]), + undefined + ); + } + + private async searchExactIdentity( + candidate: FreshCandidate, + query: string, + expectedNormalizedTitle: string, + year?: number + ): Promise< + | { kind: 'none' } + | { kind: 'ambiguous' } + | { kind: 'match'; result: TmdbMovieResult | TmdbTvResult } > { const found = new Map(); let page = 1; @@ -954,11 +1299,12 @@ export class FreshEngine { candidate.mediaType === 'movie' ? await this.dependencies.tmdb.searchMoviesStrict({ query: candidate.displayTitle, - year: candidate.matchYear, + year, page, }) : await this.dependencies.tmdb.searchTvShowsStrict({ - query: candidate.displayTitle, + query, + ...(year ? { year } : {}), page, }); this.validateSearchResponse(response, page); @@ -976,14 +1322,20 @@ export class FreshEngine { ]; if ( !titles.some( - (title) => normalizeFreshTitle(title) === candidate.normalizedTitle + (title) => normalizeFreshTitle(title) === expectedNormalizedTitle ) ) continue; if ( candidate.mediaType === 'movie' && - (result as TmdbMovieResult).release_date?.slice(0, 4) !== - String(candidate.matchYear) + year && + (result as TmdbMovieResult).release_date?.slice(0, 4) !== String(year) + ) + continue; + if ( + candidate.mediaType === 'tv' && + year && + (result as TmdbTvResult).first_air_date?.slice(0, 4) !== String(year) ) continue; found.set(result.id, result); @@ -1025,29 +1377,229 @@ export class FreshEngine { const repository = manager.getRepository(FreshCandidate); const current = await repository.findOneByOrFail({ id: candidate.id }); current.status = status; + current.automaticStatus = status; + current.automaticTmdbId = null; + current.automaticFreshMediaId = null; + current.automaticFailureReason = reason; current.nextAttemptAt = retryAt(status, current.attemptCount, now); current.lastFailureReason = reason; current.resolutionStartedAt = null; + current.effectiveMediaType = current.mediaType; + current.revision += 1; await repository.save(current); }); } catch (error) { if (error instanceof FreshRunError) throw error; + if ( + error instanceof Error && + ['stale_candidate', 'source_evidence_collision'].includes(error.message) + ) + throw error; throw new FreshRunError('resolution_search', 'persistence_failed'); } } + private async seasonDetailsForCandidate( + candidate: FreshCandidate, + tmdbId: number, + mediaType: 'movie' | 'tv' = candidate.mediaType + ): Promise { + const identity = recurringIdentityForEvidence({ ...candidate, mediaType }); + if (!identity || identity.identityKind === 'movie') return undefined; + return this.dependencies.tmdb.getTvSeason?.({ + tvId: tmdbId, + seasonNumber: identity.seasonKey, + }); + } + + private async reapplyManualResolution( + candidateId: number, + state: FreshSyncState, + settings: FreshSettings + ): Promise { + const candidates = this.dependencies.database.getRepository(FreshCandidate); + const candidate = await candidates.findOneByOrFail({ id: candidateId }); + if (!candidate.sourceEvidenceKey) return; + const currentMatches = await candidates.countBy({ + sourceGeneration: state.generation, + sourceEvidenceKey: candidate.sourceEvidenceKey, + }); + if (currentMatches !== 1) { + candidate.lastFailureReason = 'source_evidence_collision'; + candidate.revision += 1; + await candidates.save(candidate); + return; + } + const resolution = await this.dependencies.database + .getRepository(FreshManualResolution) + .findOneBy({ + sourceEvidenceVersion: 1, + sourceEvidenceKey: candidate.sourceEvidenceKey, + active: true, + }); + if (!resolution) return; + const details = + resolution.mediaType === 'movie' + ? await this.dependencies.tmdb.getMovie({ movieId: resolution.tmdbId }) + : await this.dependencies.tmdb.getTvShow({ tvId: resolution.tmdbId }); + if (details.id !== resolution.tmdbId) + throw new Error('invalid_tmdb_response'); + await this.finishResolved( + candidate, + details, + state, + settings, + this.dependencies.now(), + await this.seasonDetailsForCandidate( + candidate, + details.id, + resolution.mediaType + ), + { + mediaType: resolution.mediaType, + actorUserId: resolution.actorUserId ?? undefined, + expectedRevision: candidate.revision, + reapply: true, + } + ); + } + + private async upsertDiscoveryHistory( + manager: EntityManager, + candidate: FreshCandidate, + media: FreshMedia, + observations: FreshObservation[], + state: FreshSyncState, + settings: FreshSettings, + seasonDetails?: TmdbSeasonWithEpisodes + ): Promise { + const identity = recurringIdentityForEvidence(candidate); + if (!identity) return undefined; + const ordered = [...observations].sort( + (left, right) => left.observedAt.getTime() - right.observedAt.getTime() + ); + const first = ordered[0]?.observedAt ?? candidate.firstObservedAt; + const last = ordered.at(-1)?.observedAt ?? candidate.lastObservedAt; + const repository = manager.getRepository(FreshDiscoveryHistory); + let history = await repository.findOneBy({ + mediaType: media.mediaType, + tmdbId: media.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + }); + + if (!history) { + const decision = + identity.identityKind === 'movie' + ? evaluateMovieAdmission( + media, + ordered, + settings.mediaEligibilityDays + ) + : seasonDetails + ? evaluateTvAdmission( + seasonDetails, + first, + identity, + settings.mediaEligibilityDays + ) + : { + eligible: false, + reason: 'tv_activity_unknown', + firstObservedAt: first, + activitySource: 'unavailable', + }; + history = new FreshDiscoveryHistory({ + mediaType: media.mediaType, + tmdbId: media.tmdbId, + identityKind: identity.identityKind, + seasonKey: identity.seasonKey, + specialEpisodeKey: identity.specialEpisodeKey, + admitted: decision?.eligible ?? false, + firstObservedAt: first, + lastObservedAt: last, + firstFreshAt: decision?.eligible ? first : null, + visibleUntil: decision?.eligible + ? freshVisibleUntil(first, settings.freshVisibilityDays) + : null, + activityDate: decision?.activityDate ?? null, + activitySource: decision?.activitySource ?? 'unavailable', + admissionReason: decision?.reason ?? 'not_evaluated', + automaticReasons: [decision?.reason ?? 'not_evaluated'], + firstSeenGeneration: state.generation, + lastSeenGeneration: state.generation, + }); + } else { + // firstObservedAt/firstFreshAt form the irreversible Fresh clock. A + // later replay, source generation, PROPER, or quality variant may add + // evidence, but can never move that clock in either direction. + if (last > history.lastObservedAt) history.lastObservedAt = last; + history.lastSeenGeneration = state.generation; + // Provider/date absence is not a stable exclusion. Permit a later + // reconciliation to complete it, while stable admission/exclusion is + // irreversible for this recurring identity. + if ( + !history.admitted && + ['tv_activity_unknown', 'not_evaluated'].includes( + history.admissionReason + ) && + identity.identityKind !== 'movie' && + seasonDetails + ) { + const decision = evaluateTvAdmission( + seasonDetails, + history.firstObservedAt, + identity, + settings.mediaEligibilityDays + ); + history.activityDate = decision.activityDate ?? null; + history.activitySource = decision.activitySource; + history.admissionReason = decision.reason; + history.automaticReasons = [decision.reason]; + if (decision.eligible) { + history.admitted = true; + history.firstFreshAt = history.firstObservedAt; + history.visibleUntil = freshVisibleUntil( + history.firstFreshAt, + settings.freshVisibilityDays + ); + } + } + } + return repository.save(history); + } + private async finishResolved( candidate: FreshCandidate, details: TmdbMovieDetails | TmdbTvDetails, state: FreshSyncState, settings: FreshSettings, - now: Date + now: Date, + seasonDetails?: TmdbSeasonWithEpisodes, + manual?: { + mediaType: 'movie' | 'tv'; + actorUserId?: number; + expectedRevision: number; + reapply?: boolean; + } ): Promise<{ created: boolean; admitted: boolean; media: FreshMedia }> { try { return await this.dependencies.database.transaction(async (manager) => { const mediaRepository = manager.getRepository(FreshMedia); + const effectiveMediaType = manual?.mediaType ?? candidate.mediaType; + const currentCandidate = await manager + .getRepository(FreshCandidate) + .findOneByOrFail({ id: candidate.id }); + if ( + manual && + (currentCandidate.revision !== manual.expectedRevision || + currentCandidate.sourceGeneration !== state.generation) + ) { + throw new Error('stale_candidate'); + } let media = await mediaRepository.findOneBy({ - mediaType: candidate.mediaType, + mediaType: effectiveMediaType, tmdbId: details.id, }); const created = !media; @@ -1066,7 +1618,7 @@ export class FreshEngine { ); media = new FreshMedia({ ...media, - mediaType: candidate.mediaType, + mediaType: effectiveMediaType, tmdbId: details.id, active: media?.active ?? false, admitted: media?.admitted ?? false, @@ -1084,27 +1636,126 @@ export class FreshEngine { originalTitle: media?.originalTitle ?? candidate.displayTitle, }); this.applyMetadata(media, details, now); - this.applyAdmissionEvidence( + const history = await this.upsertDiscoveryHistory( + manager, + new FreshCandidate({ ...candidate, mediaType: effectiveMediaType }), media, observations, - settings.mediaEligibilityDays + state, + settings, + seasonDetails ); - applyFreshMembership(media, settings, now); + if (history?.admitted && history.firstFreshAt) { + media.admitted = true; + media.firstSeenAt = history.firstFreshAt; + media.membershipReason = history.admissionReason; + } else { + media.admitted = false; + media.membershipReason = + history?.admissionReason ?? + (effectiveMediaType === 'tv' + ? 'season_unknown' + : 'outside_eligibility_window'); + } + if (history || effectiveMediaType === 'movie') { + applyFreshMembership(media, settings, now); + } else { + media.active = false; + media.automaticReasons = ['season_unknown']; + } media = await mediaRepository.save(media); const candidates = manager.getRepository(FreshCandidate); - const current = await candidates.findOneByOrFail({ id: candidate.id }); + const current = currentCandidate; + if (!current.sourceEvidenceKey && candidate.sourceEvidenceKey) { + current.sourceEvidenceKey = candidate.sourceEvidenceKey; + } current.status = FreshCandidateStatus.RESOLVED; current.tmdbId = details.id; + if (!manual) { + current.automaticStatus = FreshCandidateStatus.RESOLVED; + current.automaticTmdbId = details.id; + current.automaticFreshMediaId = media.id; + current.automaticFailureReason = null; + } + current.effectiveMediaType = candidate.mediaType; current.freshMediaId = media.id; current.resolvedAt = current.resolvedAt ?? now; current.resolutionStartedAt = null; current.nextAttemptAt = null; current.lastFailureReason = null; + current.revision += 1; await candidates.save(current); + if (manual) { + current.effectiveMediaType = manual.mediaType; + await candidates.save(current); + const automaticMediaId = current.automaticFreshMediaId; + if (automaticMediaId && automaticMediaId !== media.id) { + const stillReferenced = await candidates + .createQueryBuilder('candidate') + .where('candidate.sourceGeneration = :generation', { + generation: state.generation, + }) + .andWhere('candidate.id != :candidateId', { + candidateId: current.id, + }) + .andWhere('candidate.freshMediaId = :mediaId', { + mediaId: automaticMediaId, + }) + .getCount(); + if (!stillReferenced) { + const automaticMedia = await mediaRepository.findOneBy({ + id: automaticMediaId, + }); + if (automaticMedia) { + automaticMedia.active = false; + automaticMedia.lastMatchedGeneration = 0; + automaticMedia.membershipReason = 'source_generation_inactive'; + await mediaRepository.save(automaticMedia); + } + } + } + } + if (manual && !manual.reapply) { + const resolutions = manager.getRepository(FreshManualResolution); + const collisionCount = await candidates.countBy({ + sourceEvidenceKey: current.sourceEvidenceKey, + sourceGeneration: state.generation, + }); + if (!current.sourceEvidenceKey || collisionCount !== 1) + throw new Error('source_evidence_collision'); + let resolution = await resolutions.findOneBy({ + sourceEvidenceVersion: 1, + sourceEvidenceKey: current.sourceEvidenceKey, + }); + resolution = new FreshManualResolution({ + ...resolution, + sourceEvidenceVersion: 1, + sourceEvidenceKey: current.sourceEvidenceKey, + mediaType: manual.mediaType, + tmdbId: details.id, + canonicalTitle: + manual.mediaType === 'movie' + ? (details as TmdbMovieDetails).title + : (details as TmdbTvDetails).name, + canonicalDate: + manual.mediaType === 'movie' + ? (details as TmdbMovieDetails).release_date || null + : (details as TmdbTvDetails).first_air_date || null, + active: true, + revision: (resolution?.revision ?? 0) + 1, + actorUserId: manual.actorUserId ?? null, + }); + await resolutions.save(resolution); + } return { created, admitted: media.admitted, media }; }); } catch (error) { if (error instanceof FreshRunError) throw error; + if ( + error instanceof Error && + ['stale_candidate', 'source_evidence_collision'].includes(error.message) + ) + throw error; throw new FreshRunError('resolution_search', 'persistence_failed'); } } @@ -1132,39 +1783,126 @@ export class FreshEngine { const mediaRepository = this.dependencies.database.getRepository(FreshMedia); const before = await mediaRepository.countBy({ active: true }); - if (state.continuityStatus === FreshContinuityStatus.GAP_PRESERVED) { - const current = await mediaRepository.findBy({ - lastMatchedGeneration: state.generation, - }); - for (const media of current) { - applyFreshMembership(media, settings, this.dependencies.now()); - } - await mediaRepository.save(current); - } else if (!unresolved) { - const all = await mediaRepository.find(); - for (const media of all) { - if (media.lastMatchedGeneration === state.generation) { - applyFreshMembership(media, settings, this.dependencies.now()); - } else { - media.active = false; - media.membershipReason = 'source_generation_inactive'; - } - } - await mediaRepository.save(all); - } else { - const current = await mediaRepository.findBy({ - lastMatchedGeneration: state.generation, - }); - for (const media of current) { - applyFreshMembership(media, settings, this.dependencies.now()); - } - await mediaRepository.save(current); + const mediaToEvaluate = + state.continuityStatus !== FreshContinuityStatus.GAP_PRESERVED && + !unresolved + ? await mediaRepository.find() + : await mediaRepository.findBy({ + lastMatchedGeneration: state.generation, + }); + for (const media of mediaToEvaluate) { + await this.refreshMediaProjection(media, state, settings); } + if (mediaToEvaluate.length) await mediaRepository.save(mediaToEvaluate); const after = await mediaRepository.countBy({ active: true }); diagnostic.snapshot.counts.expiredFreshMedia = Math.max(0, before - after); diagnostic.snapshot.counts.currentFreshMedia = after; } + private async refreshMediaProjection( + media: FreshMedia, + state: FreshSyncState, + settings: FreshSettings + ): Promise { + const histories = await this.dependencies.database + .getRepository(FreshDiscoveryHistory) + .findBy({ mediaType: media.mediaType, tmdbId: media.tmdbId }); + if (media.lastMatchedGeneration !== state.generation) { + media.active = false; + media.membershipReason = 'source_generation_inactive'; + media.automaticReasons = ['source_generation_inactive']; + return; + } + if (!histories.length) { + if (!media.admitted && media.mediaType === 'movie') { + this.applyAdmissionEvidence( + media, + await this.observationsForMedia(media.id), + settings.mediaEligibilityDays + ); + } + if (media.admitted && media.lastMatchedGeneration === state.generation) { + applyFreshMembership(media, settings, this.dependencies.now()); + } else if ( + !media.admitted && + media.membershipReason === 'eligibility_unknown' + ) { + media.active = false; + media.automaticReasons = ['eligibility_unknown']; + } else { + media.active = false; + media.membershipReason = 'source_generation_inactive'; + media.automaticReasons = ['source_generation_inactive']; + } + return; + } + const overrides = await this.dependencies.database + .getRepository(FreshAdmissionOverride) + .findBy({ + mediaType: media.mediaType, + tmdbId: media.tmdbId, + active: true, + }); + const overridden = new Set( + overrides.map( + (value) => + `${value.identityKind}:${value.seasonKey}:${value.specialEpisodeKey}` + ) + ); + const now = this.dependencies.now(); + const effective = histories + .filter((history) => history.lastSeenGeneration === state.generation) + .filter((history) => history.firstFreshAt && history.visibleUntil) + .filter( + (history) => (history.visibleUntil as Date).getTime() >= now.getTime() + ) + .filter( + (history) => + history.admitted || + overridden.has( + `${history.identityKind}:${history.seasonKey}:${history.specialEpisodeKey}` + ) + ) + .sort((left, right) => { + const firstFresh = + (right.firstFreshAt?.getTime() ?? 0) - + (left.firstFreshAt?.getTime() ?? 0); + if (firstFresh) return firstFresh; + if (right.seasonKey !== left.seasonKey) + return right.seasonKey - left.seasonKey; + return right.id - left.id; + }); + const selected = effective[0]; + if (!selected?.firstFreshAt) { + media.active = false; + media.admitted = histories.some((history) => history.admitted); + media.membershipReason = histories.some( + (history) => history.lastSeenGeneration === state.generation + ) + ? histories.some((history) => history.admitted) + ? 'visibility_expired' + : (histories[0]?.admissionReason ?? 'outside_eligibility_window') + : 'source_generation_inactive'; + media.automaticReasons = [media.membershipReason]; + return; + } + media.admitted = true; + media.firstSeenAt = selected.firstFreshAt; + media.lastSeenAt = histories.reduce( + (maximum, history) => + history.lastObservedAt > maximum ? history.lastObservedAt : maximum, + selected.lastObservedAt + ); + const automatic = applyFreshMembership(media, settings, now); + const overrideActive = overridden.has( + `${selected.identityKind}:${selected.seasonKey}:${selected.specialEpisodeKey}` + ); + if (overrideActive && !automatic.active) { + media.active = true; + media.membershipReason = 'override_active'; + } + } + private async refreshStaleMetadata( state: FreshSyncState, settings: FreshSettings @@ -1196,12 +1934,7 @@ export class FreshEngine { continue; } this.applyMetadata(media, details, now); - this.applyAdmissionEvidence( - media, - await this.observationsForMedia(media.id), - settings.mediaEligibilityDays - ); - applyFreshMembership(media, settings, now); + await this.refreshMediaProjection(media, state, settings); try { await repository.save(media); } catch { diff --git a/server/lib/fresh/fresh.test.ts b/server/lib/fresh/fresh.test.ts index b8e0bb51e3..3b54029c09 100644 --- a/server/lib/fresh/fresh.test.ts +++ b/server/lib/fresh/fresh.test.ts @@ -6,6 +6,7 @@ import type { import type { TmdbMovieDetails, TmdbMovieResult, + TmdbSeasonWithEpisodes, TmdbTvDetails, TmdbTvResult, } from '@server/api/themoviedb/interfaces'; @@ -16,6 +17,8 @@ import { } from '@server/constants/fresh'; import dataSource from '@server/datasource'; import FreshCandidate from '@server/entity/FreshCandidate'; +import FreshDiscoveryHistory from '@server/entity/FreshDiscoveryHistory'; +import FreshManualResolution from '@server/entity/FreshManualResolution'; import FreshMedia from '@server/entity/FreshMedia'; import FreshObservation from '@server/entity/FreshObservation'; import { FreshSyncState } from '@server/entity/FreshSyncState'; @@ -84,7 +87,12 @@ const release = ( releaseId: String(releaseId), mediaType, title, + sourceTitle: title, year, + seasonNumber: -1, + episodeNumber: -1, + explicitSeason: false, + explicitSpecial: false, observedAt: Date.parse(observedAt), availabilityType, }); @@ -191,7 +199,7 @@ describe('persistent Fresh engine', () => { assert.equal(await dataSource.getRepository(FreshMedia).count(), 2); assert.equal( await dataSource.getRepository(FreshMedia).countBy({ active: true }), - 2 + 1 ); assert.equal(movieSearches, 1); assert.equal(tvSearches, 1); @@ -261,7 +269,7 @@ describe('persistent Fresh engine', () => { .findOneByOrFail({ mediaType: 'movie', tmdbId: 101 }); assert.equal( canonical.firstSeenAt.toISOString(), - '2026-09-20T10:00:00.000Z' + '2026-09-24T12:00:00.000Z' ); assert.equal( canonical.lastSeenAt.toISOString(), @@ -722,8 +730,8 @@ describe('persistent Fresh engine', () => { }); assert.equal(repairedMovie.admitted, false); assert.equal(repairedMovie.membershipReason, 'eligibility_unknown'); - assert.equal(repairedTv.admitted, true); - assert.equal(repairedTv.membershipReason, 'active_fresh'); + assert.equal(repairedTv.admitted, false); + assert.equal(repairedTv.membershipReason, 'source_generation_inactive'); assert.equal( await candidateRepository.countBy({ status: FreshCandidateStatus.OUTSIDE_WINDOW, @@ -758,6 +766,8 @@ describe('persistent Fresh engine', () => { displayTitle: 'Manual Movie', matchYear: 2026, status: FreshCandidateStatus.NO_MATCH, + automaticStatus: FreshCandidateStatus.NO_MATCH, + automaticFailureReason: 'no_exact_match', firstObservedAt: new Date('2026-09-25T00:00:00Z'), lastObservedAt: new Date('2026-09-25T00:00:00Z'), }) @@ -801,14 +811,674 @@ describe('persistent Fresh engine', () => { now: () => new Date(now), cancelled: () => false, }); - const resolved = await manual.resolveManually(candidate.id, 9001, settings); + const resolved = await manual.resolveManually( + candidate.id, + 'movie', + 9001, + candidate.revision, + undefined, + settings + ); assert.equal(resolved.candidate.status, FreshCandidateStatus.RESOLVED); assert.equal(resolved.candidate.tmdbId, 9001); assert.equal(resolved.media.displayTitle, 'Canonical Manual Movie'); assert.equal(resolved.media.admitted, true); assert.equal(resolved.media.active, true); await assert.rejects(() => - manual.resolveManually(candidate.id, 9002, settings) + manual.resolveManually( + candidate.id, + 'movie', + 9002, + candidate.revision, + undefined, + settings + ) + ); + const reset = await manual.resetManualResolution( + candidate.id, + resolved.candidate.revision, + settings + ); + assert.equal(reset.status, FreshCandidateStatus.NO_MATCH); + assert.equal(reset.effectiveMediaType, 'movie'); + assert.equal(reset.tmdbId, null); + assert.equal( + ( + await dataSource + .getRepository(FreshMedia) + .findOneByOrFail({ id: resolved.media.id }) + ).active, + false + ); + assert.equal( + ( + await dataSource + .getRepository(FreshManualResolution) + .findOneByOrFail({ sourceEvidenceKey: reset.sourceEvidenceKey }) + ).active, + false + ); + }); + + it('uses retained TV year evidence for Youth and Apocalypse without rank guessing', async () => { + await dataSource.synchronize(true); + releases = [ + { + ...release(4002, 'tv', 'Apocalypse', 2026, '2026-09-25T00:00:00Z'), + seasonNumber: 1, + explicitSeason: true, + }, + { + ...release(4001, 'tv', 'Youth', 2026, '2026-09-24T00:00:00Z'), + seasonNumber: 1, + explicitSeason: true, + }, + ]; + const queries: { query: string; year?: number; page?: number }[] = []; + const resolver = new FreshEngine({ + database: dataSource, + autobrr: () => ({ + filters: async () => filters, + page: async (_filter, cursor = 0) => page(releases, cursor), + }), + tmdb: { + getMovie: async () => { + throw new Error('wrong namespace'); + }, + getTvShow: async ({ tvId }) => + tv( + tvId, + tvId === 285418 ? 'Youth' : 'Apocalypse', + '2026-01-01' + ) as unknown as TmdbTvDetails, + searchMoviesStrict: async () => { + throw new Error('wrong namespace'); + }, + searchTvShowsStrict: async ({ query, year, page = 1 }) => { + queries.push({ query, year, page }); + const result = + query === 'Youth' && year === 2026 + ? [tv(285418, 'Youth', '2026-01-01')] + : query === 'Apocalypse' && year === 2026 + ? [tv(315479, 'Apocalypse', '2026-01-02')] + : query === 'Youth' + ? [ + tv(1, 'Youth', '2013-01-01'), + tv(285418, 'Youth', '2026-01-01'), + ] + : []; + return { + page, + total_pages: query === 'Apocalypse' && !year ? 4 : 1, + total_results: result.length, + results: result, + }; + }, + }, + now: () => new Date(now), + cancelled: () => false, + }); + await resolver.run(settings, true); + const candidates = await dataSource.getRepository(FreshCandidate).find({ + order: { displayTitle: 'ASC' }, + }); + assert.deepEqual( + candidates.map((candidate) => [ + candidate.displayTitle, + candidate.matchYear, + candidate.tmdbId, + ]), + [ + ['Apocalypse', 2026, 315479], + ['Youth', 2026, 285418], + ] + ); + for (const title of ['Apocalypse', 'Youth']) { + const titleQueries = queries.filter(({ query }) => query === title); + assert.equal(titleQueries[0].year, undefined); + assert.equal(titleQueries.at(-1)?.year, 2026); + } + assert.deepEqual( + queries + .filter(({ query, year }) => query === 'Apocalypse' && !year) + .map(({ page }) => page), + [1, 2, 3] + ); + }); + + it('uses only the anchored terminal Volume N fallback for TV', async () => { + await dataSource.synchronize(true); + releases = [ + release(5002, 'movie', 'Movie Volume 4', 2026, '2026-09-25T00:00:00Z'), + release(5001, 'tv', 'Chopped Volume 4', 0, '2026-09-24T00:00:00Z'), + ]; + const queries: { namespace: 'movie' | 'tv'; query: string }[] = []; + const resolver = new FreshEngine({ + database: dataSource, + autobrr: () => ({ + filters: async () => filters, + page: async (_filter, cursor = 0) => page(releases, cursor), + }), + tmdb: { + getMovie: async () => { + throw new Error('movie must remain unresolved'); + }, + getTvShow: async ({ tvId }) => + tv(tvId, 'Chopped', '2009-01-13') as unknown as TmdbTvDetails, + searchMoviesStrict: async ({ query, page = 1 }) => { + queries.push({ namespace: 'movie', query }); + return { page, total_pages: 1, total_results: 0, results: [] }; + }, + searchTvShowsStrict: async ({ query, page = 1 }) => { + queries.push({ namespace: 'tv', query }); + const results = + query === 'Chopped' ? [tv(17404, 'Chopped', '2009-01-13')] : []; + return { + page, + total_pages: 1, + total_results: results.length, + results, + }; + }, + }, + now: () => new Date(now), + cancelled: () => false, + }); + await resolver.run(settings, true); + const chopped = await dataSource + .getRepository(FreshCandidate) + .findOneByOrFail({ + displayTitle: 'Chopped Volume 4', + }); + const movieVolume = await dataSource + .getRepository(FreshCandidate) + .findOneByOrFail({ displayTitle: 'Movie Volume 4' }); + assert.equal(chopped.tmdbId, 17404); + assert.equal(movieVolume.status, FreshCandidateStatus.NO_MATCH); + assert.deepEqual( + queries.sort((left, right) => left.query.localeCompare(right.query)), + [ + { namespace: 'tv', query: 'Chopped' }, + { namespace: 'tv', query: 'Chopped Volume 4' }, + { namespace: 'movie', query: 'Movie Volume 4' }, + ] + ); + }); + + it('keeps FIA parsed Movie evidence while allowing typed TV manual correction', async () => { + await dataSource.synchronize(true); + releases = [ + release( + 6001, + 'movie', + 'FIA WEC 2026 6 Hours Of Fuji', + 2026, + '2026-09-25T00:00:00Z' + ), + ]; + const resolver = new FreshEngine({ + database: dataSource, + autobrr: () => ({ + filters: async () => filters, + page: async (_filter, cursor = 0) => page(releases, cursor), + }), + tmdb: { + getMovie: async () => { + throw new Error('wrong namespace'); + }, + getTvShow: async ({ tvId }) => + tv(tvId, 'FIA WEC', '2012-01-01') as unknown as TmdbTvDetails, + searchMoviesStrict: async ({ page = 1 }) => ({ + page, + total_pages: 1, + total_results: 0, + results: [], + }), + searchTvShowsStrict: async () => { + throw new Error('automatic cross-type lookup is forbidden'); + }, + }, + now: () => new Date(now), + cancelled: () => false, + }); + await resolver.run(settings, true); + const parsed = await dataSource + .getRepository(FreshCandidate) + .findOneByOrFail({ + displayTitle: 'FIA WEC 2026 6 Hours Of Fuji', + }); + assert.equal(parsed.mediaType, 'movie'); + assert.equal(parsed.status, FreshCandidateStatus.NO_MATCH); + const corrected = await resolver.resolveManually( + parsed.id, + 'tv', + 305251, + parsed.revision, + 42, + settings + ); + assert.equal(corrected.candidate.mediaType, 'movie'); + assert.equal(corrected.candidate.effectiveMediaType, 'tv'); + assert.equal(corrected.candidate.tmdbId, 305251); + assert.equal(corrected.media.mediaType, 'tv'); + assert.equal(corrected.media.displayTitle, 'FIA WEC'); + assert.equal( + await dataSource.getRepository(FreshMedia).countBy({ + mediaType: 'movie', + tmdbId: 305251, + }), + 0 + ); + + await resolver.run( + { ...settings, baseUrl: 'https://autobrr-generation-2.test' }, + false + ); + const reapplied = await dataSource + .getRepository(FreshCandidate) + .findOneOrFail({ + where: { sourceGeneration: 2 }, + }); + assert.equal(reapplied.mediaType, 'movie'); + assert.equal(reapplied.effectiveMediaType, 'tv'); + assert.equal(reapplied.tmdbId, 305251); + assert.equal( + await dataSource.getRepository(FreshManualResolution).countBy({ + sourceEvidenceKey: reapplied.sourceEvidenceKey, + active: true, + }), + 1 + ); + }); + + it('keeps one immutable Last Week Tonight history per season and projects the newest active season', async () => { + await dataSource.synchronize(true); + let clock = new Date('2026-09-28T12:00:00.000Z'); + const lastWeekRelease = ( + id: number, + seasonNumber: number, + episode: number, + at: string + ) => ({ + ...release(id, 'tv', 'Last Week Tonight with John Oliver', 2026, at), + sourceTitle: `Last.Week.Tonight.with.John.Oliver.S${seasonNumber}E${episode}.1080p`, + seasonNumber, + episodeNumber: episode, + explicitSeason: true, + }); + releases = [lastWeekRelease(7001, 13, 24, '2026-09-28T01:00:00Z')]; + const resolver = new FreshEngine({ + database: dataSource, + autobrr: () => ({ + filters: async () => filters, + page: async (_filter, cursor = 0) => page(releases, cursor), + }), + tmdb: { + getMovie: async () => { + throw new Error('wrong namespace'); + }, + getTvShow: async ({ tvId }) => + tv( + tvId, + 'Last Week Tonight with John Oliver', + '2014-04-27' + ) as unknown as TmdbTvDetails, + getTvSeason: async ({ seasonNumber }) => + ({ + id: seasonNumber, + name: `Season ${seasonNumber}`, + overview: '', + air_date: seasonNumber === 13 ? '2026-02-15' : '2027-02-14', + season_number: seasonNumber, + poster_path: null, + episodes: [ + { + id: seasonNumber * 100 + 1, + name: 'Episode 1', + overview: '', + air_date: seasonNumber === 13 ? '2026-09-27' : '2027-02-14', + episode_number: seasonNumber === 13 ? 24 : 1, + season_number: seasonNumber, + production_code: '', + runtime: 30, + still_path: null, + vote_average: 0, + vote_count: 0, + crew: [], + guest_stars: [], + }, + ], + }) as unknown as TmdbSeasonWithEpisodes, + searchMoviesStrict: async () => { + throw new Error('wrong namespace'); + }, + searchTvShowsStrict: async ({ page = 1 }) => ({ + page, + total_pages: 1, + total_results: 1, + results: [ + tv(60694, 'Last Week Tonight with John Oliver', '2014-04-27'), + ], + }), + }, + now: () => new Date(clock), + cancelled: () => false, + }); + + await resolver.run(settings, true); + let histories = await dataSource.getRepository(FreshDiscoveryHistory).find({ + order: { seasonKey: 'ASC' }, + }); + assert.equal(histories.length, 1); + assert.equal(histories[0].seasonKey, 13); + assert.equal(histories[0].admitted, true); + const season13FirstFresh = histories[0].firstFreshAt?.toISOString(); + const season13VisibleUntil = histories[0].visibleUntil?.toISOString(); + + releases = [ + lastWeekRelease(7002, 13, 25, '2026-09-29T01:00:00Z'), + ...releases, + ]; + clock = new Date('2026-09-29T12:00:00.000Z'); + await resolver.run(settings, false); + histories = await dataSource.getRepository(FreshDiscoveryHistory).find(); + assert.equal(histories.length, 1); + assert.equal(histories[0].firstFreshAt?.toISOString(), season13FirstFresh); + assert.equal( + histories[0].visibleUntil?.toISOString(), + season13VisibleUntil + ); + + const generationSettings = { + ...settings, + baseUrl: 'https://autobrr-generation-2.test', + }; + await resolver.run(generationSettings, false); + histories = await dataSource.getRepository(FreshDiscoveryHistory).find(); + assert.equal(histories.length, 1); + assert.equal(histories[0].firstFreshAt?.toISOString(), season13FirstFresh); + assert.equal( + histories[0].visibleUntil?.toISOString(), + season13VisibleUntil + ); + + releases = [ + lastWeekRelease(7004, 13, 27, '2026-10-10T01:00:00Z'), + ...releases, + ]; + clock = new Date('2026-10-10T12:00:00.000Z'); + await resolver.run(generationSettings, false); + histories = await dataSource.getRepository(FreshDiscoveryHistory).find(); + assert.equal(histories.length, 1); + assert.equal(histories[0].firstFreshAt?.toISOString(), season13FirstFresh); + assert.equal( + histories[0].visibleUntil?.toISOString(), + season13VisibleUntil + ); + assert.equal( + ( + await dataSource.getRepository(FreshMedia).findOneByOrFail({ + mediaType: 'tv', + tmdbId: 60694, + }) + ).active, + false + ); + + releases = [ + lastWeekRelease(7005, 14, 1, '2027-02-15T01:00:00Z'), + ...releases, + ]; + clock = new Date('2027-02-15T12:00:00.000Z'); + await resolver.run(generationSettings, false); + histories = await dataSource.getRepository(FreshDiscoveryHistory).find({ + order: { seasonKey: 'ASC' }, + }); + assert.deepEqual( + histories.map((history) => history.seasonKey), + [13, 14] + ); + assert.equal(histories[0].firstFreshAt?.toISOString(), season13FirstFresh); + const projection = await dataSource + .getRepository(FreshMedia) + .findOneByOrFail({ + mediaType: 'tv', + tmdbId: 60694, + }); + assert.equal( + projection.firstSeenAt.toISOString(), + '2027-02-15T01:00:00.000Z' + ); + assert.equal(projection.active, true); + }); + + it('keeps explicit Specials as independent irreversible identities without inventing season zero', async () => { + await dataSource.synchronize(true); + let clock = new Date('2026-09-28T12:00:00.000Z'); + const special = ( + id: number, + episodeNumber: number, + at: string, + explicitSpecial = true + ): FreshRelease => ({ + ...release(id, 'tv', 'Example Series', 2026, at), + sourceTitle: explicitSpecial + ? `Example.Series.S00E${String(episodeNumber).padStart(2, '0')}.1080p` + : 'Example.Series.1080p', + seasonNumber: 0, + episodeNumber, + explicitSpecial, + }); + releases = [ + special(7103, 0, '2026-09-28T03:00:00Z', false), + special(7102, 14, '2026-09-28T02:00:00Z'), + special(7101, 1, '2026-09-28T01:00:00Z'), + ]; + const resolver = new FreshEngine({ + database: dataSource, + autobrr: () => ({ + filters: async () => filters, + page: async (_filter, cursor = 0) => page(releases, cursor), + }), + tmdb: { + getMovie: async () => { + throw new Error('wrong namespace'); + }, + getTvShow: async ({ tvId }) => + tv(tvId, 'Example Series', '2020-01-01') as unknown as TmdbTvDetails, + getTvSeason: async () => + ({ + id: 0, + name: 'Specials', + overview: '', + air_date: '2020-01-01', + season_number: 0, + poster_path: null, + episodes: [ + { + id: 1, + name: 'Recent Special', + overview: '', + air_date: '2026-09-27', + episode_number: 1, + season_number: 0, + production_code: '', + runtime: 30, + still_path: null, + vote_average: 0, + vote_count: 0, + crew: [], + guest_stars: [], + }, + { + id: 14, + name: 'Historical Special', + overview: '', + air_date: '2020-01-01', + episode_number: 14, + season_number: 0, + production_code: '', + runtime: 30, + still_path: null, + vote_average: 0, + vote_count: 0, + crew: [], + guest_stars: [], + }, + ], + }) as unknown as TmdbSeasonWithEpisodes, + searchMoviesStrict: async () => { + throw new Error('wrong namespace'); + }, + searchTvShowsStrict: async ({ page = 1 }) => ({ + page, + total_pages: 1, + total_results: 1, + results: [tv(7100, 'Example Series', '2020-01-01')], + }), + }, + now: () => new Date(clock), + cancelled: () => false, + }); + + await resolver.run(settings, true); + let histories = await dataSource.getRepository(FreshDiscoveryHistory).find({ + order: { specialEpisodeKey: 'ASC' }, + }); + assert.deepEqual( + histories.map((history) => [ + history.identityKind, + history.specialEpisodeKey, + history.admitted, + ]), + [ + ['special', 1, true], + ['special', 14, false], + ] + ); + assert.equal( + await dataSource.getRepository(FreshMedia).countBy({ + mediaType: 'tv', + tmdbId: 7100, + }), + 1 + ); + const firstFreshAt = histories[0].firstFreshAt?.toISOString(); + + releases = [special(7104, 1, '2026-09-29T01:00:00Z'), ...releases]; + clock = new Date('2026-09-29T12:00:00.000Z'); + await resolver.run(settings, false); + histories = await dataSource.getRepository(FreshDiscoveryHistory).find({ + order: { specialEpisodeKey: 'ASC' }, + }); + assert.equal(histories.length, 2); + assert.equal(histories[0].firstFreshAt?.toISOString(), firstFreshAt); + }); + + it('applies and removes a durable admission override without resetting its Fresh clock', async () => { + await dataSource.synchronize(true); + let clock = new Date('2026-09-28T12:00:00.000Z'); + releases = [ + release(8001, 'movie', 'Historical Movie', 2020, '2026-09-28T01:00:00Z'), + ]; + const resolver = new FreshEngine({ + database: dataSource, + autobrr: () => ({ + filters: async () => filters, + page: async (_filter, cursor = 0) => page(releases, cursor), + }), + tmdb: { + getMovie: async ({ movieId }) => + movie( + movieId, + 'Historical Movie', + '2020-01-01' + ) as unknown as TmdbMovieDetails, + getTvShow: async () => { + throw new Error('wrong namespace'); + }, + searchMoviesStrict: async ({ page = 1 }) => ({ + page, + total_pages: 1, + total_results: 1, + results: [movie(8080, 'Historical Movie', '2020-01-01')], + }), + searchTvShowsStrict: async () => { + throw new Error('wrong namespace'); + }, + }, + now: () => new Date(clock), + cancelled: () => false, + }); + await resolver.run(settings, true); + let candidate = await dataSource + .getRepository(FreshCandidate) + .findOneByOrFail({ + displayTitle: 'Historical Movie', + }); + let history = await dataSource + .getRepository(FreshDiscoveryHistory) + .findOneByOrFail({ mediaType: 'movie', tmdbId: 8080 }); + assert.equal(history.admitted, false); + assert.equal(history.firstFreshAt, null); + + candidate = await resolver.setAdmissionOverride( + candidate.id, + candidate.revision, + settings, + 42 + ); + history = await dataSource + .getRepository(FreshDiscoveryHistory) + .findOneByOrFail({ mediaType: 'movie', tmdbId: 8080 }); + const firstFreshAt = history.firstFreshAt?.toISOString(); + assert.equal(firstFreshAt, clock.toISOString()); + assert.equal( + ( + await dataSource + .getRepository(FreshMedia) + .findOneByOrFail({ mediaType: 'movie', tmdbId: 8080 }) + ).active, + true + ); + await assert.rejects( + () => + resolver.setAdmissionOverride( + candidate.id, + candidate.revision - 1, + settings, + 42 + ), + /stale_candidate/ + ); + + candidate = await resolver.removeAdmissionOverride( + candidate.id, + candidate.revision, + settings + ); + assert.equal( + ( + await dataSource + .getRepository(FreshMedia) + .findOneByOrFail({ mediaType: 'movie', tmdbId: 8080 }) + ).active, + false + ); + clock = new Date('2026-09-29T12:00:00.000Z'); + candidate = await resolver.setAdmissionOverride( + candidate.id, + candidate.revision, + settings, + 42 + ); + history = await dataSource + .getRepository(FreshDiscoveryHistory) + .findOneByOrFail({ mediaType: 'movie', tmdbId: 8080 }); + assert.equal(history.firstFreshAt?.toISOString(), firstFreshAt); + assert.equal( + history.visibleUntil?.toISOString(), + '2026-10-05T12:00:00.000Z' ); }); }); diff --git a/server/lib/fresh/history.test.ts b/server/lib/fresh/history.test.ts new file mode 100644 index 0000000000..2063e24531 --- /dev/null +++ b/server/lib/fresh/history.test.ts @@ -0,0 +1,225 @@ +import type { TmdbSeasonWithEpisodes } from '@server/api/themoviedb/interfaces'; +import FreshMedia from '@server/entity/FreshMedia'; +import { + evaluateMovieAdmission, + evaluateTvAdmission, + recurringIdentityForEvidence, + selectTvActivityDate, +} from '@server/lib/fresh/history'; +import { normalizeFreshTitle } from '@server/lib/fresh/normalize'; +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +const day = 86_400_000; +const observedAt = new Date('2026-09-28T00:00:00.000Z'); + +const movie = (date: string) => + new FreshMedia({ + mediaType: 'movie', + tmdbId: 1, + mediaDate: date, + firstSeenAt: observedAt, + lastSeenAt: observedAt, + resolvedAt: observedAt, + metadataRefreshedAt: observedAt, + displayTitle: 'Movie', + sortTitle: 'movie', + originalTitle: 'Movie', + }); + +const observation = (ageDays: number) => ({ + availabilityType: 'unknown' as const, + observedAt: new Date(Date.parse('2026-01-01T00:00:00.000Z') + ageDays * day), +}); + +const season = ( + airDate: string | null, + episodes: { number: number; date: string | null }[] +) => + ({ + id: 1, + name: 'Season 13', + overview: '', + air_date: airDate, + season_number: 13, + poster_path: null, + episodes: episodes.map(({ number, date }) => ({ + id: number, + name: `Episode ${number}`, + overview: '', + air_date: date, + episode_number: number, + season_number: 13, + production_code: '', + runtime: 30, + still_path: null, + vote_average: 0, + vote_count: 0, + crew: [], + guest_stars: [], + })), + }) as unknown as TmdbSeasonWithEpisodes; + +describe('Fresh discovery history policy', () => { + it('uses inclusive Movie normal and 14-day first-observation grace boundaries', () => { + const reference = movie('2026-01-01'); + assert.equal( + evaluateMovieAdmission(reference, [observation(0)], 90)?.eligible, + true + ); + assert.equal( + evaluateMovieAdmission(reference, [observation(90)], 90)?.reason, + 'eligible_movie' + ); + assert.equal( + evaluateMovieAdmission(reference, [observation(91)], 90)?.reason, + 'eligible_movie_first_observation_grace' + ); + assert.equal( + evaluateMovieAdmission(reference, [observation(104)], 90)?.eligible, + true + ); + assert.equal( + evaluateMovieAdmission(reference, [observation(105)], 90)?.eligible, + false + ); + assert.equal( + evaluateMovieAdmission(reference, [observation(-1)], 90)?.reason, + 'movie_not_yet_available' + ); + }); + + it('admits the audited Virginia Woolf first observation through grace and normalizes aliases', () => { + const virginia = movie('2026-06-19'); + virginia.tmdbId = 1291375; + const decision = evaluateMovieAdmission( + virginia, + [ + { + availabilityType: 'unknown', + observedAt: new Date('2026-09-28T01:45:34.000Z'), + }, + ], + 90 + ); + assert.equal(decision?.reason, 'eligible_movie_first_observation_grace'); + assert.equal(decision?.eligible, true); + assert.equal( + normalizeFreshTitle("Virginia.Woolf's.Night.&.Day"), + 'virginia woolfs night and day' + ); + assert.equal( + normalizeFreshTitle('virginia.woolfs.night.and.day'), + 'virginia woolfs night and day' + ); + assert.equal( + normalizeFreshTitle('Amélie'), + normalizeFreshTitle('Ame\u0301lie') + ); + }); + + it('requires explicit ordinary-season or special identity evidence', () => { + assert.deepEqual( + recurringIdentityForEvidence({ + mediaType: 'tv', + seasonKey: 13, + specialEpisodeKey: -1, + explicitSeason: true, + explicitSpecial: false, + }), + { identityKind: 'season', seasonKey: 13, specialEpisodeKey: -1 } + ); + assert.deepEqual( + recurringIdentityForEvidence({ + mediaType: 'tv', + seasonKey: 0, + specialEpisodeKey: 14, + explicitSeason: false, + explicitSpecial: true, + }), + { identityKind: 'special', seasonKey: 0, specialEpisodeKey: 14 } + ); + assert.equal( + recurringIdentityForEvidence({ + mediaType: 'tv', + seasonKey: 0, + specialEpisodeKey: 0, + explicitSeason: false, + explicitSpecial: false, + }), + undefined + ); + }); + + it('uses the latest aired episode for recurring TV season activity', () => { + const details = season('2026-01-01', [ + { number: 1, date: '2026-01-01' }, + { number: 23, date: '2026-09-20' }, + { number: 24, date: '2026-09-27' }, + { number: 25, date: '2026-10-10' }, + ]); + const identity = { + identityKind: 'season' as const, + seasonKey: 13, + specialEpisodeKey: -1 as const, + }; + assert.deepEqual(selectTvActivityDate(details, observedAt, identity), { + date: '2026-09-27', + source: 'tmdb_latest_aired_episode', + }); + assert.equal( + evaluateTvAdmission(details, observedAt, identity, 90).reason, + 'eligible_tv_recent_activity' + ); + }); + + it('falls back to a trustworthy season date and refuses future-only or missing activity', () => { + const identity = { + identityKind: 'season' as const, + seasonKey: 13, + specialEpisodeKey: -1 as const, + }; + assert.deepEqual( + selectTvActivityDate(season('2026-09-20', []), observedAt, identity), + { date: '2026-09-20', source: 'tmdb_season_air_date' } + ); + assert.equal( + evaluateTvAdmission( + season('2027-01-01', [{ number: 1, date: '2027-01-01' }]), + observedAt, + identity, + 90 + ).reason, + 'tv_not_yet_aired' + ); + assert.equal( + evaluateTvAdmission(season(null, []), observedAt, identity, 90).reason, + 'tv_activity_unknown' + ); + }); + + it('evaluates explicit specials by exact episode rather than by season activity', () => { + const details = season('2020-01-01', [ + { number: 1, date: '2026-09-27' }, + { number: 14, date: '2020-01-01' }, + ]); + assert.equal( + evaluateTvAdmission( + details, + observedAt, + { identityKind: 'special', seasonKey: 0, specialEpisodeKey: 14 }, + 90 + ).reason, + 'historical_tv_season' + ); + assert.equal( + evaluateTvAdmission( + details, + observedAt, + { identityKind: 'special', seasonKey: 0, specialEpisodeKey: 1 }, + 90 + ).eligible, + true + ); + }); +}); diff --git a/server/lib/fresh/history.ts b/server/lib/fresh/history.ts new file mode 100644 index 0000000000..278fc07f66 --- /dev/null +++ b/server/lib/fresh/history.ts @@ -0,0 +1,193 @@ +import type { TmdbSeasonWithEpisodes } from '@server/api/themoviedb/interfaces'; +import type FreshMedia from '@server/entity/FreshMedia'; +import type FreshObservation from '@server/entity/FreshObservation'; +import { eligibilityForObservation } from '@server/lib/fresh/membership'; + +const DAY = 86_400_000; +export const FRESH_MOVIE_GRACE_DAYS = 14; +export const FRESH_TV_AIR_DATE_TOLERANCE_DAYS = 1; + +export type FreshRecurringIdentity = + | { identityKind: 'movie'; seasonKey: -1; specialEpisodeKey: -1 } + | { identityKind: 'season'; seasonKey: number; specialEpisodeKey: -1 } + | { identityKind: 'special'; seasonKey: 0; specialEpisodeKey: number }; + +export type FreshAdmissionDecision = { + eligible: boolean; + reason: string; + firstObservedAt: Date; + activityDate?: string; + activitySource: string; + ageDays?: number; +}; + +const dateOnly = (value: string | null | undefined): Date | undefined => { + if (!value || !/^\d{4}-\d{2}-\d{2}$/.test(value)) return undefined; + const result = new Date(`${value}T00:00:00.000Z`); + return Number.isNaN(result.getTime()) ? undefined : result; +}; + +export const recurringIdentityForEvidence = (value: { + mediaType: 'movie' | 'tv'; + seasonKey: number; + specialEpisodeKey: number; + explicitSeason: boolean; + explicitSpecial: boolean; +}): FreshRecurringIdentity | undefined => { + if (value.mediaType === 'movie') { + return { identityKind: 'movie', seasonKey: -1, specialEpisodeKey: -1 }; + } + if ( + value.explicitSpecial && + value.seasonKey === 0 && + value.specialEpisodeKey > 0 + ) { + return { + identityKind: 'special', + seasonKey: 0, + specialEpisodeKey: value.specialEpisodeKey, + }; + } + if (value.explicitSeason && value.seasonKey > 0) { + return { + identityKind: 'season', + seasonKey: value.seasonKey, + specialEpisodeKey: -1, + }; + } + return undefined; +}; + +export const evaluateMovieAdmission = ( + media: FreshMedia, + observations: Pick[], + eligibilityDays: number +): FreshAdmissionDecision | undefined => { + const first = [...observations].sort( + (left, right) => left.observedAt.getTime() - right.observedAt.getTime() + )[0]; + if (!first) return undefined; + const evaluation = eligibilityForObservation(media, first, eligibilityDays); + if (evaluation.ageDays === undefined) { + return { + eligible: false, + reason: 'movie_availability_unknown', + firstObservedAt: first.observedAt, + activitySource: evaluation.eligibilityDateSource, + }; + } + const inNormalWindow = + evaluation.ageDays >= 0 && evaluation.ageDays <= eligibilityDays; + const inGrace = + evaluation.ageDays > eligibilityDays && + evaluation.ageDays <= eligibilityDays + FRESH_MOVIE_GRACE_DAYS; + return { + eligible: inNormalWindow || inGrace, + reason: inNormalWindow + ? 'eligible_movie' + : inGrace + ? 'eligible_movie_first_observation_grace' + : evaluation.ageDays < 0 + ? 'movie_not_yet_available' + : 'outside_movie_eligibility', + firstObservedAt: first.observedAt, + ...(evaluation.eligibilityDate + ? { activityDate: evaluation.eligibilityDate } + : {}), + activitySource: evaluation.eligibilityDateSource, + ageDays: evaluation.ageDays, + }; +}; + +export const selectTvActivityDate = ( + season: TmdbSeasonWithEpisodes, + observationAt: Date, + identity: Exclude +): { date?: string; source: string; reason?: string } => { + const tolerance = + observationAt.getTime() + FRESH_TV_AIR_DATE_TOLERANCE_DAYS * DAY; + if (identity.identityKind === 'special') { + const episode = season.episodes.find( + (value) => value.episode_number === identity.specialEpisodeKey + ); + const airDate = dateOnly(episode?.air_date); + if (!airDate) + return { source: 'tmdb_special_episode', reason: 'tv_activity_unknown' }; + if (airDate.getTime() > tolerance) + return { + date: episode?.air_date ?? undefined, + source: 'tmdb_special_episode', + reason: 'tv_not_yet_aired', + }; + return { + date: episode?.air_date ?? undefined, + source: 'tmdb_special_episode', + }; + } + + const latestEpisode = season.episodes + .map((episode) => ({ episode, airDate: dateOnly(episode.air_date) })) + .filter( + (value): value is { episode: typeof value.episode; airDate: Date } => + !!value.airDate && value.airDate.getTime() <= tolerance + ) + .sort((left, right) => right.airDate.getTime() - left.airDate.getTime())[0]; + if (latestEpisode) { + return { + date: latestEpisode.episode.air_date ?? undefined, + source: 'tmdb_latest_aired_episode', + }; + } + const seasonDate = dateOnly(season.air_date); + if (seasonDate && seasonDate.getTime() <= tolerance) { + return { date: season.air_date, source: 'tmdb_season_air_date' }; + } + if ( + season.episodes.some((episode) => { + const value = dateOnly(episode.air_date); + return value && value.getTime() > tolerance; + }) || + (seasonDate && seasonDate.getTime() > tolerance) + ) { + return { source: 'tmdb_season', reason: 'tv_not_yet_aired' }; + } + return { source: 'tmdb_season', reason: 'tv_activity_unknown' }; +}; + +export const evaluateTvAdmission = ( + season: TmdbSeasonWithEpisodes, + observationAt: Date, + identity: Exclude, + eligibilityDays: number +): FreshAdmissionDecision => { + const activity = selectTvActivityDate(season, observationAt, identity); + if (!activity.date) { + return { + eligible: false, + reason: activity.reason ?? 'tv_activity_unknown', + firstObservedAt: observationAt, + activitySource: activity.source, + }; + } + const aired = dateOnly(activity.date); + const ageDays = aired + ? (observationAt.getTime() - aired.getTime()) / DAY + : undefined; + const eligible = + ageDays !== undefined && + ageDays >= -FRESH_TV_AIR_DATE_TOLERANCE_DAYS && + ageDays <= eligibilityDays; + return { + eligible, + reason: eligible ? 'eligible_tv_recent_activity' : 'historical_tv_season', + firstObservedAt: observationAt, + activityDate: activity.date, + activitySource: activity.source, + ...(ageDays === undefined ? {} : { ageDays }), + }; +}; + +export const freshVisibleUntil = ( + firstFreshAt: Date, + visibilityDays: number +): Date => new Date(firstFreshAt.getTime() + visibilityDays * DAY); diff --git a/server/lib/fresh/index.ts b/server/lib/fresh/index.ts index 0d6dab0504..0e902692ca 100644 --- a/server/lib/fresh/index.ts +++ b/server/lib/fresh/index.ts @@ -4,6 +4,15 @@ export { validReleaseId, } from '@server/lib/fresh/normalize'; export type { + FreshCandidateDiagnosticQuery, + FreshCandidateDiagnosticResponse, + FreshCandidateDiagnosticRow, + FreshCandidateDiagnosticSort, + FreshCandidateDiagnosticStatus, + FreshCandidateDiagnosticSummary, + FreshCandidatePresenceFilter, + FreshCandidateReasonFamily, + FreshCandidateSeasonEvidence, FreshDiagnosticCounts, FreshDiagnosticDecision, FreshDiagnosticsSnapshot, diff --git a/server/lib/fresh/membership.test.ts b/server/lib/fresh/membership.test.ts index 0ca8fc62d7..e1ada6f3bf 100644 --- a/server/lib/fresh/membership.test.ts +++ b/server/lib/fresh/membership.test.ts @@ -98,7 +98,7 @@ describe('Fresh membership', () => { new Date('2026-03-20T00:00:00Z'), 90 ), - true + false ); }); @@ -258,7 +258,7 @@ describe('Fresh membership', () => { { ...settings, includeOriginalLanguages: ['fr'] }, new Date('2026-03-21T00:00:00Z') ).reason, - 'excluded_original_language' + 'missing_required_original_language' ); assert.equal( evaluateFreshMembership( @@ -284,6 +284,28 @@ describe('Fresh membership', () => { ).reason, 'below_tmdb_vote_count' ); + + assert.deepEqual( + evaluateFreshMembership( + value, + { + ...settings, + excludeGenreIds: [53], + excludeOriginalLanguages: ['en'], + excludeContentRatings: ['movie:R'], + minimumTmdbScore: 8.5, + minimumTmdbVotes: 501, + }, + new Date('2026-03-21T00:00:00Z') + ).reasons, + [ + 'excluded_genre', + 'excluded_original_language', + 'excluded_content_rating', + 'below_tmdb_score', + 'below_tmdb_vote_count', + ] + ); }); it('allows a removed exclusion only before visibility expires', () => { diff --git a/server/lib/fresh/membership.ts b/server/lib/fresh/membership.ts index c4ba36bcb4..e320c05d68 100644 --- a/server/lib/fresh/membership.ts +++ b/server/lib/fresh/membership.ts @@ -15,15 +15,20 @@ export type FreshMembershipReason = | 'eligibility_unknown' | 'visibility_expired' | 'excluded_genre' + | 'missing_required_genre' | 'excluded_original_language' + | 'missing_required_original_language' | 'excluded_content_rating' + | 'missing_required_content_rating' | 'below_tmdb_score' | 'below_tmdb_vote_count' + | 'season_unknown' | 'source_generation_inactive'; export interface FreshMembershipEvaluation { active: boolean; reason: FreshMembershipReason; + reasons: FreshMembershipReason[]; } export type FreshEligibilityDateSource = @@ -60,8 +65,8 @@ export const qualifiesAtObservation = ( ): boolean => { const released = validDate(mediaDate); if (!released) return false; - const age = Math.max(0, observationTime.getTime() - released.getTime()); - return age <= mediaEligibilityDays * DAY; + const age = observationTime.getTime() - released.getTime(); + return age >= 0 && age <= mediaEligibilityDays * DAY; }; const earliestReleaseDate = ( @@ -212,56 +217,66 @@ export const evaluateFreshMembership = ( now: Date ): FreshMembershipEvaluation => { if (!media.admitted) { + const reason = ['eligibility_unknown', 'season_unknown'].includes( + media.membershipReason ?? '' + ) + ? (media.membershipReason as 'eligibility_unknown' | 'season_unknown') + : 'outside_eligibility_window'; return { active: false, - reason: - media.membershipReason === 'eligibility_unknown' - ? 'eligibility_unknown' - : 'outside_eligibility_window', + reason, + reasons: [reason], }; } if ( now.getTime() > media.firstSeenAt.getTime() + settings.freshVisibilityDays * DAY ) { - return { active: false, reason: 'visibility_expired' }; + return { + active: false, + reason: 'visibility_expired', + reasons: ['visibility_expired'], + }; } + const reasons: FreshMembershipReason[] = []; + if (intersects(media.genreIds, settings.excludeGenreIds)) + reasons.push('excluded_genre'); if ( - intersects(media.genreIds, settings.excludeGenreIds) || - (settings.includeGenreIds.length > 0 && - !intersects(media.genreIds, settings.includeGenreIds)) - ) { - return { active: false, reason: 'excluded_genre' }; - } + settings.includeGenreIds.length > 0 && + !intersects(media.genreIds, settings.includeGenreIds) + ) + reasons.push('missing_required_genre'); + if (settings.excludeOriginalLanguages.includes(media.originalLanguage)) + reasons.push('excluded_original_language'); if ( - settings.excludeOriginalLanguages.includes(media.originalLanguage) || - (settings.includeOriginalLanguages.length > 0 && - !settings.includeOriginalLanguages.includes(media.originalLanguage)) - ) { - return { active: false, reason: 'excluded_original_language' }; - } + settings.includeOriginalLanguages.length > 0 && + !settings.includeOriginalLanguages.includes(media.originalLanguage) + ) + reasons.push('missing_required_original_language'); const contentRating = ratingKey(media); + if (contentRating && settings.excludeContentRatings.includes(contentRating)) + reasons.push('excluded_content_rating'); if ( - (contentRating && settings.excludeContentRatings.includes(contentRating)) || - (settings.includeContentRatings.length > 0 && - !settings.includeContentRatings.includes(contentRating)) - ) { - return { active: false, reason: 'excluded_content_rating' }; - } + settings.includeContentRatings.length > 0 && + !settings.includeContentRatings.includes(contentRating) + ) + reasons.push('missing_required_content_rating'); if ( settings.minimumTmdbScore > 0 && media.voteAverage < settings.minimumTmdbScore ) { - return { active: false, reason: 'below_tmdb_score' }; + reasons.push('below_tmdb_score'); } if ( settings.minimumTmdbVotes > 0 && media.voteCount < settings.minimumTmdbVotes ) { - return { active: false, reason: 'below_tmdb_vote_count' }; + reasons.push('below_tmdb_vote_count'); } - return { active: true, reason: 'active_fresh' }; + return reasons.length + ? { active: false, reason: reasons[0], reasons } + : { active: true, reason: 'active_fresh', reasons: ['active_fresh'] }; }; export const applyFreshMembership = ( @@ -272,6 +287,7 @@ export const applyFreshMembership = ( const result = evaluateFreshMembership(media, settings, now); media.active = result.active; media.membershipReason = result.reason; + media.automaticReasons = result.reasons; return result; }; diff --git a/server/lib/fresh/migration.test.ts b/server/lib/fresh/migration.test.ts index 0b8ec3006e..4b5aa78b04 100644 --- a/server/lib/fresh/migration.test.ts +++ b/server/lib/fresh/migration.test.ts @@ -1,5 +1,8 @@ import { FreshCandidateStatus } from '@server/constants/fresh'; +import FreshAdmissionOverride from '@server/entity/FreshAdmissionOverride'; import FreshCandidate from '@server/entity/FreshCandidate'; +import FreshDiscoveryHistory from '@server/entity/FreshDiscoveryHistory'; +import FreshManualResolution from '@server/entity/FreshManualResolution'; import FreshMedia from '@server/entity/FreshMedia'; import FreshObservation from '@server/entity/FreshObservation'; import { FreshSyncState } from '@server/entity/FreshSyncState'; @@ -7,21 +10,96 @@ import { AddFreshPersistence1790000000000 } from '@server/migration/sqlite/17900 import { UpgradeFreshMembership1790000000002 } from '@server/migration/sqlite/1790000000002-UpgradeFreshMembership'; import { AddFreshAvailabilityDates1790000000004 } from '@server/migration/sqlite/1790000000004-AddFreshAvailabilityDates'; import { AddFreshEligibilitySchemaVersion1790000000006 } from '@server/migration/sqlite/1790000000006-AddFreshEligibilitySchemaVersion'; +import { RefineFreshDiscovery1790000000010 } from '@server/migration/sqlite/1790000000010-RefineFreshDiscovery'; import assert from 'node:assert/strict'; +import { mkdtempSync, readFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; import { describe, it } from 'node:test'; import { DataSource } from 'typeorm'; describe('Fresh persistence migration', () => { - it('creates the four-entity SQLite schema and singleton state through migrations', async () => { + it('preserves existing Fresh media when development synchronization adds automatic reasons', async () => { + const directory = mkdtempSync(path.join(tmpdir(), 'seerr-fresh-sync-')); + const databasePath = path.join(directory, 'db.sqlite3'); + const legacy = new DataSource({ + type: 'sqlite', + database: databasePath, + migrations: [ + AddFreshPersistence1790000000000, + UpgradeFreshMembership1790000000002, + AddFreshAvailabilityDates1790000000004, + AddFreshEligibilitySchemaVersion1790000000006, + ], + synchronize: false, + }); + const synchronized = new DataSource({ + type: 'sqlite', + database: databasePath, + entities: [FreshMedia], + synchronize: true, + }); + try { + await legacy.initialize(); + await legacy.runMigrations(); + await legacy.query(` + INSERT INTO "fresh_media" + ("mediaType", "tmdbId", "active", "admitted", "lastMatchedGeneration", + "firstSeenAt", "lastSeenAt", "resolvedAt", "metadataRefreshedAt", + "displayTitle", "sortTitle", "originalTitle", "membershipReason", + "genreIds", "originCountries", "originalLanguage", "voteAverage", "voteCount") + VALUES + ('movie', 101, 1, 1, 4, '2026-09-20 00:00:00', '2026-09-21 00:00:00', + '2026-09-20 00:00:00', '2026-09-20 00:00:00', 'Existing Movie', + 'existing movie', 'Existing Movie', 'active_fresh', '[]', '[]', '', 0, 0) + `); + await legacy.destroy(); + + await synchronized.initialize(); + const [media] = (await synchronized.query( + `SELECT "tmdbId", "automaticReasons" FROM "fresh_media"` + )) as { tmdbId: number; automaticReasons: string }[]; + assert.deepEqual(media, { tmdbId: 101, automaticReasons: '[]' }); + } finally { + if (legacy.isInitialized) await legacy.destroy(); + if (synchronized.isInitialized) await synchronized.destroy(); + rmSync(directory, { recursive: true, force: true }); + } + }); + + it('keeps the PostgreSQL refinement schema timezone-aware and sentinel-keyed', () => { + const source = readFileSync( + path.join( + process.cwd(), + 'server/migration/postgres/1790000000011-RefineFreshDiscovery.ts' + ), + 'utf8' + ); + assert.match(source, /const time = 'timestamp with time zone'/); + assert.match(source, /UQ_fresh_discovery_history_identity/); + assert.match(source, /'seasonKey',\s*'specialEpisodeKey'/s); + assert.doesNotMatch(source, /timestamp without time zone/); + }); + + it('creates the refined Fresh SQLite schema and singleton state through migrations', async () => { const database = new DataSource({ type: 'sqlite', database: ':memory:', - entities: [FreshSyncState, FreshCandidate, FreshObservation, FreshMedia], + entities: [ + FreshSyncState, + FreshCandidate, + FreshObservation, + FreshMedia, + FreshDiscoveryHistory, + FreshManualResolution, + FreshAdmissionOverride, + ], migrations: [ AddFreshPersistence1790000000000, UpgradeFreshMembership1790000000002, AddFreshAvailabilityDates1790000000004, AddFreshEligibilitySchemaVersion1790000000006, + RefineFreshDiscovery1790000000010, ], synchronize: false, }); @@ -38,6 +116,9 @@ describe('Fresh persistence migration', () => { 'fresh_observation', 'fresh_candidate', 'fresh_media', + 'fresh_discovery_history', + 'fresh_manual_resolution', + 'fresh_admission_override', ]) { assert.ok(tables.includes(name)); } @@ -111,21 +192,9 @@ describe('Fresh persistence migration', () => { ); await database.getRepository(FreshCandidate).delete(candidate.id); assert.equal(await database.getRepository(FreshObservation).count(), 0); - await database.undoLastMigration(); - await database.undoLastMigration(); - await database.undoLastMigration(); - await database.undoLastMigration(); - const remaining = ( - (await database.query( - `SELECT name FROM sqlite_master WHERE type = 'table'` - )) as { name: string }[] - ).map(({ name }) => name); - assert.equal(remaining.includes('fresh_media'), false); - await database.runMigrations(); - assert.equal( - await database.getRepository(FreshSyncState).countBy({ id: 1 }), - 1 - ); + // The refinement migration is deliberately forward-only: rolling it + // back would erase irreversible discovery history and human decisions. + assert.equal(await database.showMigrations(), false); } finally { await database.destroy(); } @@ -183,4 +252,79 @@ describe('Fresh persistence migration', () => { await database.destroy(); } }); + + it('seeds only admitted legacy media into irreversible history', async () => { + const database = new DataSource({ + type: 'sqlite', + database: ':memory:', + migrations: [ + AddFreshPersistence1790000000000, + UpgradeFreshMembership1790000000002, + AddFreshAvailabilityDates1790000000004, + AddFreshEligibilitySchemaVersion1790000000006, + ], + synchronize: false, + }); + await database.initialize(); + try { + await database.runMigrations(); + await database.query( + `UPDATE "fresh_sync_state" SET "generation" = 4, "checkpointReleaseId" = '4126', "freshVisibilityDays" = 10 WHERE "id" = 1` + ); + await database.query(` + INSERT INTO "fresh_media" + ("mediaType", "tmdbId", "active", "admitted", "lastMatchedGeneration", + "firstSeenAt", "lastSeenAt", "resolvedAt", "metadataRefreshedAt", + "displayTitle", "sortTitle", "originalTitle", "mediaDate", "membershipReason", + "genreIds", "originCountries", "originalLanguage", "voteAverage", "voteCount") + VALUES + ('movie', 101, 1, 1, 4, '2026-09-20 00:00:00', '2026-09-21 00:00:00', + '2026-09-20 00:00:00', '2026-09-20 00:00:00', 'Admitted Movie', + 'admitted movie', 'Admitted Movie', '2026-09-01', 'active_fresh', '[]', '[]', '', 0, 0), + ('movie', 102, 0, 0, 4, '2026-09-20 00:00:00', '2026-09-21 00:00:00', + '2026-09-20 00:00:00', '2026-09-20 00:00:00', 'Outside Movie', + 'outside movie', 'Outside Movie', '2020-01-01', 'outside_eligibility_window', '[]', '[]', '', 0, 0), + ('tv', 201, 1, 1, 4, '2026-09-22 00:00:00', '2026-09-23 00:00:00', + '2026-09-22 00:00:00', '2026-09-22 00:00:00', 'Legacy Series', + 'legacy series', 'Legacy Series', '2020-01-01', 'active_fresh', '[]', '[]', '', 0, 0) + `); + + const runner = database.createQueryRunner(); + await new RefineFreshDiscovery1790000000010().up(runner); + await runner.release(); + const histories = (await database.query( + `SELECT "mediaType", "tmdbId", "identityKind", "seasonKey", "legacyProjection", "firstFreshAt", "visibleUntil" FROM "fresh_discovery_history" ORDER BY "tmdbId"` + )) as { + mediaType: string; + tmdbId: number; + identityKind: string; + seasonKey: number; + legacyProjection: number; + firstFreshAt: string; + visibleUntil: string; + }[]; + assert.equal(histories.length, 2); + assert.deepEqual( + histories.map((history) => ({ + type: history.mediaType, + id: history.tmdbId, + kind: history.identityKind, + season: history.seasonKey, + legacy: Boolean(history.legacyProjection), + })), + [ + { type: 'movie', id: 101, kind: 'movie', season: -1, legacy: false }, + { type: 'tv', id: 201, kind: 'legacy_tv', season: -1, legacy: true }, + ] + ); + assert.match(histories[0].firstFreshAt, /2026-09-20/); + assert.match(histories[0].visibleUntil, /2026-09-30/); + const [state] = (await database.query( + `SELECT "generation", "checkpointReleaseId" FROM "fresh_sync_state" WHERE "id" = 1` + )) as { generation: number; checkpointReleaseId: string }[]; + assert.deepEqual(state, { generation: 4, checkpointReleaseId: '4126' }); + } finally { + await database.destroy(); + } + }); }); diff --git a/server/lib/fresh/normalize.ts b/server/lib/fresh/normalize.ts index d77a014031..a70abddfac 100644 --- a/server/lib/fresh/normalize.ts +++ b/server/lib/fresh/normalize.ts @@ -1,10 +1,43 @@ +import { createHash } from 'crypto'; + +export const FRESH_COMPARISON_VERSION = 2; +export const FRESH_SOURCE_EVIDENCE_VERSION = 1; + +/** + * A deterministic comparison key, never a display title. Keep this deliberately + * narrower than fuzzy matching: equivalent punctuation/Unicode forms collapse, + * but words are neither reordered nor approximately matched. + */ export const normalizeFreshTitle = (title: string): string => title .normalize('NFKC') - .toLowerCase() - .replace(/['’]/g, '') + .toLocaleLowerCase('und') + .replace(/[\u2018\u2019]/g, "'") + .replace(/'/g, '') + .replace(/(^|[^\p{L}\p{N}])&(?=[^\p{L}\p{N}]|$)/gu, '$1and') + .normalize('NFKD') + .replace(/\p{M}+/gu, '') .replace(/[^\p{L}\p{N}]+/gu, ' ') - .trim(); + .trim() + .replace(/\s+/g, ' '); export const validReleaseId = (value: string | undefined): value is string => typeof value === 'string' && /^[1-9]\d{0,19}$/.test(value); + +export const freshSourceEvidenceKey = (value: { + sourceTitle: string; + year: number; + seasonNumber: number; + episodeNumber: number; +}): string => + createHash('sha256') + .update( + JSON.stringify([ + FRESH_SOURCE_EVIDENCE_VERSION, + normalizeFreshTitle(value.sourceTitle), + value.year || 0, + value.seasonNumber, + value.episodeNumber, + ]) + ) + .digest('hex'); diff --git a/server/lib/fresh/service.test.ts b/server/lib/fresh/service.test.ts index f92bcc32c0..82ea3aa6c1 100644 --- a/server/lib/fresh/service.test.ts +++ b/server/lib/fresh/service.test.ts @@ -166,6 +166,41 @@ describe('Fresh application service', () => { assert.equal(service.running(), false); }); + it('serializes administrator mutations behind an active synchronization', async () => { + const events: string[] = []; + let finishSync: (() => void) | undefined; + const service = new FreshService({ + ...fixture().deps, + engine: { + cancel: () => undefined, + run: async () => { + events.push('sync:start'); + await new Promise((resolve) => { + finishSync = resolve; + }); + events.push('sync:end'); + return { diagnostics }; + }, + resolveManually: async () => { + events.push('manual'); + return { + candidate: new FreshCandidate({ id: 1 }), + media: new FreshMedia({ mediaType: 'tv', tmdbId: 305251 }), + }; + }, + }, + }); + service.configure(settings); + + const synchronization = service.sync(); + const mutation = service.resolveCandidate(1, 'tv', 305251, 7, 42); + await new Promise((resolve) => setImmediate(resolve)); + assert.deepEqual(events, ['sync:start']); + finishSync?.(); + await Promise.all([synchronization, mutation]); + assert.deepEqual(events, ['sync:start', 'sync:end', 'manual']); + }); + it('returns sanitized browser-safe filter options', async () => { const f = fixture(); const service = new FreshService(f.deps); @@ -224,7 +259,6 @@ describe('Fresh application service', () => { assert.deepEqual(cleared, [ FreshObservation, FreshCandidate, - FreshMedia, FreshSyncState, ]); return { diagnostics }; @@ -238,7 +272,6 @@ describe('Fresh application service', () => { assert.deepEqual(cleared, [ FreshObservation, FreshCandidate, - FreshMedia, FreshSyncState, ]); assert.equal((cleared as unknown[]).includes(Media), false); diff --git a/server/lib/fresh/service.ts b/server/lib/fresh/service.ts index 02c75cb575..9506893026 100644 --- a/server/lib/fresh/service.ts +++ b/server/lib/fresh/service.ts @@ -10,7 +10,10 @@ import { MIN_MEDIA_ELIGIBILITY_DAYS, } from '@server/constants/fresh'; import dataSource from '@server/datasource'; +import FreshAdmissionOverride from '@server/entity/FreshAdmissionOverride'; import FreshCandidate from '@server/entity/FreshCandidate'; +import FreshDiscoveryHistory from '@server/entity/FreshDiscoveryHistory'; +import FreshManualResolution from '@server/entity/FreshManualResolution'; import FreshMedia from '@server/entity/FreshMedia'; import FreshObservation from '@server/entity/FreshObservation'; import { FreshSyncState } from '@server/entity/FreshSyncState'; @@ -27,14 +30,25 @@ import { In, type DataSource, type SelectQueryBuilder } from 'typeorm'; const contentFilterReasons = [ 'excluded_genre', + 'missing_required_genre', 'excluded_original_language', + 'missing_required_original_language', 'excluded_content_rating', + 'missing_required_content_rating', 'below_tmdb_score', 'below_tmdb_vote_count', ]; +const technicalIdentityReasons = [ + 'source_evidence_collision', + 'parsed_type_suspect', + 'special_identity_ambiguous', +]; + const diagnosticStatus = ( - candidate: FreshCandidate + candidate: FreshCandidate, + history?: FreshDiscoveryHistory, + now = new Date() ): FreshCandidateDiagnosticStatus => { if (candidate.status === FreshCandidateStatus.NO_MATCH) return 'no_match'; if (candidate.status === FreshCandidateStatus.AMBIGUOUS) return 'ambiguous'; @@ -42,7 +56,24 @@ const diagnosticStatus = ( return 'temporary_failure'; if (candidate.status === FreshCandidateStatus.UNRESOLVED) return 'pending'; if (candidate.status === FreshCandidateStatus.RESOLVING) return 'resolving'; + if ( + candidate.lastFailureReason && + technicalIdentityReasons.includes(candidate.lastFailureReason) + ) + return 'needs_attention'; const media = candidate.freshMedia; + if ( + (candidate.effectiveMediaType ?? candidate.mediaType) === 'tv' && + !history + ) + return 'needs_attention'; + if ( + history?.firstFreshAt && + history.visibleUntil && + history.visibleUntil.getTime() < now.getTime() + ) + return 'historical'; + if (history && !history.admitted) return 'reviewable'; if (media?.membershipReason === 'eligibility_unknown') return 'eligibility_unknown'; if ( @@ -50,8 +81,7 @@ const diagnosticStatus = ( (media && !media.admitted) ) return 'outside_eligibility_window'; - if (media?.membershipReason === 'visibility_expired') - return 'visibility_expired'; + if (media?.membershipReason === 'visibility_expired') return 'historical'; if ( media?.membershipReason && contentFilterReasons.includes(media.membershipReason) @@ -111,12 +141,66 @@ const applyDiagnosticStatus = ( membershipReason: 'eligibility_unknown', }); else if (status === 'needs_attention') - query.andWhere('candidate.status IN (:...attentionStatuses)', { - attentionStatuses: [ - FreshCandidateStatus.NO_MATCH, - FreshCandidateStatus.AMBIGUOUS, - ], - }); + query.andWhere( + `(candidate.status IN (:...attentionStatuses) OR + candidate.lastFailureReason IN (:...technicalIdentityReasons) OR + (candidate.status = :attentionResolved + AND COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'tv' + AND history.id IS NULL))`, + { + attentionStatuses: [ + FreshCandidateStatus.NO_MATCH, + FreshCandidateStatus.AMBIGUOUS, + FreshCandidateStatus.TRANSIENT_FAILURE, + FreshCandidateStatus.UNRESOLVED, + FreshCandidateStatus.RESOLVING, + ], + technicalIdentityReasons, + attentionResolved: FreshCandidateStatus.RESOLVED, + } + ); + else if (status === 'reviewable') + query + .andWhere('candidate.status = :resolvedStatus', { + resolvedStatus: FreshCandidateStatus.RESOLVED, + }) + .andWhere('media.active = :reviewableActive', { reviewableActive: false }) + .andWhere( + '(history."visibleUntil" IS NULL OR history."visibleUntil" >= :diagnosticNow)' + ) + .andWhere( + '(media.membershipReason IS NULL OR media.membershipReason NOT IN (:...historicalReasons))', + { + historicalReasons: [ + 'visibility_expired', + 'source_generation_inactive', + ], + } + ) + .andWhere( + '(history.id IS NOT NULL OR media.membershipReason IN (:...reviewableReasons))', + { + reviewableReasons: [ + 'outside_eligibility_window', + 'eligibility_unknown', + ...contentFilterReasons, + ], + } + ); + else if (status === 'historical') + query + .andWhere('candidate.status = :resolvedStatus', { + resolvedStatus: FreshCandidateStatus.RESOLVED, + }) + .andWhere( + `(history."visibleUntil" < :diagnosticNow OR media.membershipReason IN (:...historicalReasons))`, + { + historicalReasons: [ + 'visibility_expired', + 'source_generation_inactive', + ], + } + ); else if (status === 'excluded_content_filter') query .andWhere('candidate.status = :resolvedStatus', { @@ -141,13 +225,89 @@ const applyDiagnosticStatus = ( .andWhere('media.active = :active', { active: true }); }; +const applyDiagnosticFacets = ( + query: SelectQueryBuilder, + input: FreshCandidateDiagnosticQuery +) => { + if (input.reasonFamily === 'resolution') { + query.andWhere( + '(candidate.status != :facetResolved OR candidate.lastFailureReason IN (:...facetTechnicalReasons))', + { + facetResolved: FreshCandidateStatus.RESOLVED, + facetTechnicalReasons: technicalIdentityReasons, + } + ); + } else if (input.reasonFamily === 'admission') { + query + .andWhere('candidate.status = :facetResolved', { + facetResolved: FreshCandidateStatus.RESOLVED, + }) + .andWhere('history.id IS NOT NULL') + .andWhere('history.admitted = :facetNotAdmitted', { + facetNotAdmitted: false, + }); + } else if (input.reasonFamily === 'content') { + query.andWhere('media.membershipReason IN (:...facetContentReasons)', { + facetContentReasons: contentFilterReasons, + }); + } else if (input.reasonFamily === 'history') { + query.andWhere('history.id IS NOT NULL'); + } else if (input.reasonFamily === 'source') { + query.andWhere('media.membershipReason = :facetSourceReason', { + facetSourceReason: 'source_generation_inactive', + }); + } + + const knownSeason = `(candidate."seasonKey" > 0 OR + (candidate."explicitSpecial" = :facetTrue + AND candidate."seasonKey" = 0 + AND candidate."specialEpisodeKey" > 0))`; + if (input.seasonEvidence === 'known') { + query + .andWhere( + `COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'tv'` + ) + .andWhere(knownSeason, { facetTrue: true }); + } else if (input.seasonEvidence === 'unknown') { + query + .andWhere( + `COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'tv'` + ) + .andWhere(`NOT ${knownSeason}`, { facetTrue: true }); + } + + if (input.manualResolution !== 'all') { + query.andWhere( + input.manualResolution === 'present' + ? 'manual.id IS NOT NULL' + : 'manual.id IS NULL' + ); + } + if (input.admissionOverride !== 'all') { + query.andWhere( + input.admissionOverride === 'present' + ? 'admissionOverride.id IS NOT NULL' + : 'admissionOverride.id IS NULL' + ); + } +}; + export interface PublicFreshSettings extends Omit { apiTokenConfigured: boolean; } export interface FreshServiceDependencies { engine: Pick & - Partial>; + Partial< + Pick< + typeof freshEngine, + | 'reevaluate' + | 'resolveManually' + | 'resetManualResolution' + | 'setAdmissionOverride' + | 'removeAdmissionOverride' + > + >; database: DataSource; createAutobrr: ( baseUrl: string, @@ -295,8 +455,14 @@ export class FreshService { | 'sync' | 'reconciliation' | 'reevaluation' - | 'rebuild'; + | 'rebuild' + | 'manual_resolution' + | 'resolution_reset' + | 'admission_override' + | 'override_removal'; private latestAttempt?: FreshDiagnosticsSnapshot; + private coordinator: Promise = Promise.resolve(); + private coordinatorBusy = false; constructor( private readonly dependencies: FreshServiceDependencies = { @@ -316,21 +482,58 @@ export class FreshService { return this.settings; } + private coordinate( + operation: NonNullable, + action: () => Promise, + joinSameOperation = false + ): Promise { + if ( + joinSameOperation && + this.inFlight && + this.activeOperation === operation + ) + return this.inFlight as unknown as Promise; + const execute = async () => { + this.activeOperation = operation; + const actionPromise = action(); + this.inFlight = actionPromise.then(() => undefined); + void this.inFlight.catch(() => undefined); + try { + return await actionPromise; + } finally { + this.inFlight = undefined; + this.activeOperation = undefined; + } + }; + const result = this.coordinatorBusy + ? this.coordinator.catch(() => undefined).then(execute) + : execute(); + this.coordinatorBusy = true; + const tail = result.catch(() => undefined); + this.coordinator = tail; + void tail + .finally(() => { + if (this.coordinator === tail) this.coordinatorBusy = false; + }) + .catch(() => undefined); + return result; + } + async sync(reconcile = false): Promise { const settings = this.configured(); if (!settings.enabled) return; - if (this.inFlight) return this.inFlight; - this.activeOperation = reconcile ? 'reconciliation' : 'sync'; - this.inFlight = this.dependencies.engine - .run(settings, reconcile) - .then(({ diagnostics }) => { - this.latestAttempt = diagnostics; - }) - .finally(() => { - this.inFlight = undefined; - this.activeOperation = undefined; - }); - return this.inFlight; + if (this.inFlight && this.activeOperation === 'rebuild') + return this.inFlight; + return this.coordinate( + reconcile ? 'reconciliation' : 'sync', + () => + this.dependencies.engine + .run(settings, reconcile) + .then(({ diagnostics }) => { + this.latestAttempt = diagnostics; + }), + true + ); } async refresh(): Promise { @@ -343,73 +546,111 @@ export class FreshService { async reevaluate(): Promise { const settings = this.configured(); - if (this.inFlight) await this.inFlight; - if (this.inFlight) return this.inFlight; if (!this.dependencies.engine.reevaluate) { throw new Error('Fresh reevaluation is unavailable'); } - this.activeOperation = 'reevaluation'; - this.inFlight = this.dependencies.engine - .reevaluate(settings) - .then(() => undefined) - .finally(() => { - this.inFlight = undefined; - this.activeOperation = undefined; - }); - return this.inFlight; + return this.coordinate('reevaluation', () => + this.dependencies.engine.reevaluate!(settings).then(() => undefined) + ); } async rebuild(): Promise { const settings = this.configured(); if (!settings.enabled) throw new Error('fresh_disabled'); - if (this.inFlight && this.activeOperation === 'rebuild') { - return this.inFlight; - } - if (this.inFlight) await this.inFlight; - if (this.inFlight) return this.inFlight; - - this.activeOperation = 'rebuild'; - this.inFlight = (async () => { - await this.dependencies.database.transaction(async (manager) => { - for (const entity of [ - FreshObservation, - FreshCandidate, - FreshMedia, - FreshSyncState, - ]) { - await manager - .getRepository(entity) - .createQueryBuilder() - .delete() - .execute(); + return this.coordinate( + 'rebuild', + async () => { + await this.dependencies.database.transaction(async (manager) => { + for (const entity of [ + FreshObservation, + FreshCandidate, + FreshSyncState, + ]) { + await manager + .getRepository(entity) + .createQueryBuilder() + .delete() + .execute(); + } + }); + this.latestAttempt = undefined; + const { diagnostics } = await this.dependencies.engine.run( + settings, + false + ); + this.latestAttempt = diagnostics; + if (diagnostics.outcome !== 'succeeded') { + throw new Error('fresh_rebuild_failed'); } - }); - this.latestAttempt = undefined; - const { diagnostics } = await this.dependencies.engine.run( - settings, - false - ); - this.latestAttempt = diagnostics; - if (diagnostics.outcome !== 'succeeded') { - throw new Error('fresh_rebuild_failed'); - } - })().finally(() => { - this.inFlight = undefined; - this.activeOperation = undefined; - }); - return this.inFlight; + }, + true + ); } - async resolveCandidate(candidateId: number, tmdbId: number) { + async resolveCandidate( + candidateId: number, + mediaType: 'movie' | 'tv', + tmdbId: number, + expectedRevision: number, + actorUserId?: number + ) { const settings = this.configured(); - if (this.inFlight) await this.inFlight; if (!this.dependencies.engine.resolveManually) { throw new Error('Fresh manual resolution is unavailable'); } - return this.dependencies.engine.resolveManually( - candidateId, - tmdbId, - settings + return this.coordinate('manual_resolution', () => + this.dependencies.engine.resolveManually!( + candidateId, + mediaType, + tmdbId, + expectedRevision, + actorUserId, + settings + ) + ); + } + + async resetCandidateResolution( + candidateId: number, + expectedRevision: number + ) { + if (!this.dependencies.engine.resetManualResolution) + throw new Error('Fresh manual resolution reset is unavailable'); + return this.coordinate('resolution_reset', () => + this.dependencies.engine.resetManualResolution!( + candidateId, + expectedRevision, + this.configured() + ) + ); + } + + async admitCandidate( + candidateId: number, + expectedRevision: number, + actorUserId?: number + ) { + if (!this.dependencies.engine.setAdmissionOverride) + throw new Error('Fresh admission override is unavailable'); + return this.coordinate('admission_override', () => + this.dependencies.engine.setAdmissionOverride!( + candidateId, + expectedRevision, + this.configured(), + actorUserId + ) + ); + } + + async removeCandidateOverride(candidateId: number, expectedRevision: number) { + if (!this.dependencies.engine.removeAdmissionOverride) + throw new Error('Fresh admission override removal is unavailable'); + return this.coordinate('override_removal', () => + this.dependencies.engine.removeAdmissionOverride!( + candidateId, + expectedRevision, + this.configured() + ) ); } @@ -515,6 +756,8 @@ export class FreshService { excludedContentFilter: 0, visibilityExpired: 0, needsAttention: 0, + reviewable: 0, + historical: 0, }, }; } @@ -523,22 +766,111 @@ export class FreshService { repository .createQueryBuilder('candidate') .leftJoinAndSelect('candidate.freshMedia', 'media') + .leftJoin( + FreshDiscoveryHistory, + 'history', + `history."mediaType" = COALESCE(candidate."effectiveMediaType", candidate."mediaType") + AND history."tmdbId" = candidate."tmdbId" + AND history."seasonKey" = CASE + WHEN COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'movie' THEN -1 + ELSE candidate."seasonKey" END + AND history."specialEpisodeKey" = CASE + WHEN COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'movie' THEN -1 + ELSE candidate."specialEpisodeKey" END` + ) + .leftJoin( + FreshManualResolution, + 'manual', + `manual."sourceEvidenceVersion" = 1 + AND manual."sourceEvidenceKey" = candidate."sourceEvidenceKey" + AND manual.active = true` + ) + .leftJoin( + FreshAdmissionOverride, + 'admissionOverride', + `admissionOverride."mediaType" = COALESCE(candidate."effectiveMediaType", candidate."mediaType") + AND admissionOverride."tmdbId" = candidate."tmdbId" + AND admissionOverride."seasonKey" = CASE + WHEN COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'movie' THEN -1 + ELSE candidate."seasonKey" END + AND admissionOverride."specialEpisodeKey" = CASE + WHEN COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'movie' THEN -1 + ELSE candidate."specialEpisodeKey" END + AND admissionOverride.active = true` + ) .where('candidate.sourceGeneration = :generation', { generation: state.generation, - }); + }) + .setParameter('diagnosticNow', this.dependencies.now()); const query = base(); if (input.search?.trim()) { - query.andWhere('LOWER(candidate.displayTitle) LIKE :search', { - search: `%${input.search.trim().toLowerCase()}%`, - }); + const search = input.search.trim().toLowerCase(); + const numeric = /^\d+$/.test(search) ? Number(search) : undefined; + const typed = search.match(/^(movie|tv)\s*[:/]\s*(\d+)$/); + query.andWhere( + `(LOWER(candidate.displayTitle) LIKE :search OR + LOWER(media.displayTitle) LIKE :search OR + EXISTS (SELECT 1 FROM fresh_observation observation + WHERE observation."candidateId" = candidate.id + AND LOWER(observation."sourceTitle") LIKE :search) + ${numeric ? 'OR candidate.tmdbId = :numericTmdbId' : ''} + ${typed ? `OR (COALESCE(candidate.effectiveMediaType, candidate.mediaType) = :typedMediaType AND candidate.tmdbId = :typedTmdbId)` : ''})`, + { + search: `%${search}%`, + ...(numeric ? { numericTmdbId: numeric } : {}), + ...(typed + ? { + typedMediaType: typed[1], + typedTmdbId: Number(typed[2]), + } + : {}), + } + ); } if (input.mediaType !== 'all') { - query.andWhere('candidate.mediaType = :mediaType', { - mediaType: input.mediaType, - }); + query.andWhere( + 'COALESCE(candidate.effectiveMediaType, candidate.mediaType) = :mediaType', + { + mediaType: input.mediaType, + } + ); } applyDiagnosticStatus(query, input.status); - if (input.sort === 'title.asc') + applyDiagnosticFacets(query, input); + if (input.sort === 'priority') + query + .addSelect( + `CASE + WHEN candidate.status = ${FreshCandidateStatus.NO_MATCH} THEN 0 + WHEN candidate.status = ${FreshCandidateStatus.AMBIGUOUS} THEN 1 + WHEN candidate.status IN (${FreshCandidateStatus.TRANSIENT_FAILURE}, ${FreshCandidateStatus.UNRESOLVED}, ${FreshCandidateStatus.RESOLVING}) THEN 2 + WHEN candidate.lastFailureReason IN (:...technicalIdentityReasons) THEN 2 + WHEN candidate.status = ${FreshCandidateStatus.RESOLVED} + AND COALESCE(candidate."effectiveMediaType", candidate."mediaType") = 'tv' + AND history.id IS NULL THEN 2 + WHEN candidate.status = ${FreshCandidateStatus.RESOLVED} AND media.active = false + AND (history."visibleUntil" IS NULL OR history."visibleUntil" >= :diagnosticNow) + AND (media.membershipReason IS NULL OR media.membershipReason NOT IN ('visibility_expired', 'source_generation_inactive')) THEN 3 + WHEN media.active = true THEN 4 + ELSE 5 + END`, + 'diagnostic_priority' + ) + .setParameter('technicalIdentityReasons', technicalIdentityReasons) + .addSelect( + 'LOWER(COALESCE(media.displayTitle, candidate.displayTitle))', + 'diagnostic_title' + ) + .addSelect( + 'COALESCE(candidate.effectiveMediaType, candidate.mediaType)', + 'diagnostic_media_type' + ) + .orderBy('diagnostic_priority', 'ASC') + .addOrderBy('diagnostic_title', 'ASC') + .addOrderBy('diagnostic_media_type', 'ASC') + .addOrderBy('candidate.tmdbId', 'ASC') + .addOrderBy('candidate.id', 'ASC'); + else if (input.sort === 'title.asc') query.orderBy('candidate.displayTitle', 'ASC'); else if (input.sort === 'title.desc') query.orderBy('candidate.displayTitle', 'DESC'); @@ -557,7 +889,7 @@ export class FreshService { else if (input.sort === 'last_seen.asc') query.orderBy('candidate.lastObservedAt', 'ASC'); else query.orderBy('candidate.lastObservedAt', 'DESC'); - query.addOrderBy('candidate.id', 'ASC'); + if (input.sort !== 'priority') query.addOrderBy('candidate.id', 'ASC'); const [candidates, total] = await query .skip((input.page - 1) * FRESH_CANDIDATE_PAGE_SIZE) .take(FRESH_CANDIDATE_PAGE_SIZE) @@ -576,6 +908,55 @@ export class FreshService { values.push(observation); observationsByCandidate.set(observation.candidateId, values); }); + const sourceKeys = [ + ...new Set( + candidates + .map((candidate) => candidate.sourceEvidenceKey) + .filter(Boolean) + ), + ]; + const manualResolutions = sourceKeys.length + ? await this.dependencies.database + .getRepository(FreshManualResolution) + .findBy({ sourceEvidenceKey: In(sourceKeys), active: true }) + : []; + const manualByKey = new Map( + manualResolutions.map((resolution) => [ + resolution.sourceEvidenceKey, + resolution, + ]) + ); + const mediaIdentities = candidates + .filter((candidate) => candidate.tmdbId) + .map((candidate) => ({ + mediaType: candidate.effectiveMediaType ?? candidate.mediaType, + tmdbId: candidate.tmdbId as number, + })); + const histories = mediaIdentities.length + ? await this.dependencies.database + .getRepository(FreshDiscoveryHistory) + .createQueryBuilder('history') + .where( + mediaIdentities + .map( + (_, index) => + `(history.mediaType = :historyType${index} AND history.tmdbId = :historyId${index})` + ) + .join(' OR '), + Object.fromEntries( + mediaIdentities.flatMap((identity, index) => [ + [`historyType${index}`, identity.mediaType], + [`historyId${index}`, identity.tmdbId], + ]) + ) + ) + .getMany() + : []; + const overrides = mediaIdentities.length + ? await this.dependencies.database + .getRepository(FreshAdmissionOverride) + .findBy({ active: true }) + : []; const settings = this.configured(); const count = async (status: FreshCandidateDiagnosticStatus) => { @@ -593,6 +974,9 @@ export class FreshService { eligibilityUnknown, excludedContentFilter, visibilityExpired, + needsAttention, + reviewable, + historical, ] = await Promise.all([ count('all'), count('active_fresh'), @@ -603,6 +987,9 @@ export class FreshService { count('eligibility_unknown'), count('excluded_content_filter'), count('visibility_expired'), + count('needs_attention'), + count('reviewable'), + count('historical'), ]); return { pageInfo: { @@ -613,21 +1000,95 @@ export class FreshService { }, results: candidates.map((candidate) => { const media = candidate.freshMedia; - const evidence = media - ? evaluateAdmissionEvidence( - media, - observationsByCandidate.get(candidate.id) ?? [], - settings.mediaEligibilityDays - ) - : undefined; + const manual = manualByKey.get(candidate.sourceEvidenceKey); + const effectiveMediaType = + candidate.effectiveMediaType ?? candidate.mediaType; + const identitySeasonKey = + effectiveMediaType === 'movie' ? -1 : candidate.seasonKey; + const identitySpecialEpisodeKey = + effectiveMediaType === 'movie' ? -1 : candidate.specialEpisodeKey; + const candidateHistories = histories.filter( + (history) => + history.mediaType === effectiveMediaType && + history.tmdbId === candidate.tmdbId + ); + const history = candidateHistories + .filter( + (value) => + value.seasonKey === identitySeasonKey && + value.specialEpisodeKey === identitySpecialEpisodeKey + ) + .sort((left, right) => right.id - left.id)[0]; + const override = overrides.find( + (value) => + value.mediaType === effectiveMediaType && + value.tmdbId === candidate.tmdbId && + value.seasonKey === identitySeasonKey && + value.specialEpisodeKey === identitySpecialEpisodeKey + ); + const evidence = + media?.mediaType === 'movie' + ? evaluateAdmissionEvidence( + media, + observationsByCandidate.get(candidate.id) ?? [], + settings.mediaEligibilityDays + ) + : undefined; const selected = evidence?.selected; + const automaticStatus = candidate.automaticStatus ?? candidate.status; + const canResolve = + !manual && + [ + FreshCandidateStatus.NO_MATCH, + FreshCandidateStatus.AMBIGUOUS, + ].includes(automaticStatus); + const canAdmit = + !override && + !!candidate.tmdbId && + !!history && + !media?.active && + (!history.visibleUntil || + history.visibleUntil.getTime() >= + this.dependencies.now().getTime()) && + ![ + FreshCandidateStatus.NO_MATCH, + FreshCandidateStatus.AMBIGUOUS, + FreshCandidateStatus.TRANSIENT_FAILURE, + FreshCandidateStatus.UNRESOLVED, + FreshCandidateStatus.RESOLVING, + ].includes(automaticStatus); + const actions = { + resolve: canResolve, + resetResolution: !!manual, + admit: canAdmit, + removeOverride: !!override, + }; return { candidateId: candidate.id, - displayTitle: candidate.displayTitle, - mediaType: candidate.mediaType, + revision: candidate.revision, + displayTitle: media?.displayTitle ?? candidate.displayTitle, + parsedTitle: candidate.displayTitle, + parsedMediaType: candidate.mediaType, + mediaType: effectiveMediaType, matchYear: candidate.matchYear || undefined, + seasonNumber: + candidate.seasonKey >= 0 ? candidate.seasonKey : undefined, + episodeNumber: + candidate.specialEpisodeKey > 0 + ? candidate.specialEpisodeKey + : undefined, resolutionStatus: candidate.status, - displayStatus: diagnosticStatus(candidate), + automaticResolution: { + status: automaticStatus, + mediaType: candidate.mediaType, + tmdbId: candidate.automaticTmdbId ?? undefined, + failureReason: candidate.automaticFailureReason ?? undefined, + }, + displayStatus: diagnosticStatus( + candidate, + history, + this.dependencies.now() + ), tmdbId: candidate.tmdbId ?? undefined, firstObservedAt: candidate.firstObservedAt.toISOString(), lastObservedAt: candidate.lastObservedAt.toISOString(), @@ -637,7 +1098,71 @@ export class FreshService { resolvedAt: candidate.resolvedAt?.toISOString(), failureReason: candidate.lastFailureReason ?? undefined, membershipReason: media?.membershipReason ?? undefined, - firstSeenAt: media?.firstSeenAt?.toISOString(), + automaticReasons: [ + candidate.automaticFailureReason ?? + candidate.lastFailureReason ?? + '', + ...(history?.automaticReasons ?? []), + ...(media?.automaticReasons ?? []), + ].filter( + (value, index, all) => + Boolean(value) && all.indexOf(value) === index + ), + sourceTitleSamples: [ + ...new Set( + (observationsByCandidate.get(candidate.id) ?? []) + .map( + (observation) => observation.sourceTitle || observation.title + ) + .filter(Boolean) + ), + ].slice(0, 5), + observationCount: + observationsByCandidate.get(candidate.id)?.length ?? 0, + ...(manual + ? { + manualResolution: { + mediaType: manual.mediaType, + tmdbId: manual.tmdbId, + actorUserId: manual.actorUserId ?? undefined, + updatedAt: manual.updatedAt.toISOString(), + revision: manual.revision, + canonicalTitle: manual.canonicalTitle, + }, + } + : {}), + ...(override + ? { + admissionOverride: { + actorUserId: override.actorUserId ?? undefined, + updatedAt: override.updatedAt.toISOString(), + revision: override.revision, + }, + } + : {}), + ...(history + ? { + discoveryHistory: { + identityKind: history.identityKind, + seasonNumber: + history.seasonKey >= 0 ? history.seasonKey : undefined, + episodeNumber: + history.specialEpisodeKey > 0 + ? history.specialEpisodeKey + : undefined, + admitted: history.admitted, + legacyProjection: history.legacyProjection, + admissionReason: history.admissionReason, + activityDate: history.activityDate ?? undefined, + activitySource: history.activitySource, + firstFreshAt: history.firstFreshAt?.toISOString(), + visibleUntil: history.visibleUntil?.toISOString(), + }, + } + : {}), + firstSeenAt: + history?.firstFreshAt?.toISOString() ?? + media?.firstSeenAt?.toISOString(), lastSeenAt: media?.lastSeenAt?.toISOString(), mediaDate: media?.mediaDate ?? undefined, ...(selected @@ -657,17 +1182,16 @@ export class FreshService { } : {}), visibleUntil: - media?.admitted && media.firstSeenAt + history?.visibleUntil?.toISOString() ?? + (media?.admitted && media.firstSeenAt ? new Date( media.firstSeenAt.getTime() + settings.freshVisibilityDays * 86_400_000 ).toISOString() - : undefined, + : undefined), active: media?.active ?? false, - actionable: [ - FreshCandidateStatus.NO_MATCH, - FreshCandidateStatus.AMBIGUOUS, - ].includes(candidate.status), + actionable: Object.values(actions).some(Boolean), + actions, }; }), summary: { @@ -680,7 +1204,9 @@ export class FreshService { eligibilityUnknown, excludedContentFilter, visibilityExpired, - needsAttention: noMatch + ambiguous, + needsAttention, + reviewable, + historical, }, }; } diff --git a/server/lib/fresh/types.ts b/server/lib/fresh/types.ts index 2921ec44f3..26ddab2a92 100644 --- a/server/lib/fresh/types.ts +++ b/server/lib/fresh/types.ts @@ -104,9 +104,12 @@ export type FreshCandidateDiagnosticStatus = | 'excluded_content_filter' | 'visibility_expired' | 'active_fresh' - | 'needs_attention'; + | 'needs_attention' + | 'reviewable' + | 'historical'; export type FreshCandidateDiagnosticSort = + | 'priority' | 'title.asc' | 'title.desc' | 'status' @@ -117,21 +120,47 @@ export type FreshCandidateDiagnosticSort = | 'last_seen.desc' | 'last_seen.asc'; +export type FreshCandidateReasonFamily = + | 'all' + | 'resolution' + | 'admission' + | 'content' + | 'history' + | 'source'; + +export type FreshCandidateSeasonEvidence = 'all' | 'known' | 'unknown'; +export type FreshCandidatePresenceFilter = 'all' | 'present' | 'absent'; + export interface FreshCandidateDiagnosticQuery { page: number; search?: string; mediaType: 'all' | 'movie' | 'tv'; status: FreshCandidateDiagnosticStatus; sort: FreshCandidateDiagnosticSort; + reasonFamily: FreshCandidateReasonFamily; + seasonEvidence: FreshCandidateSeasonEvidence; + manualResolution: FreshCandidatePresenceFilter; + admissionOverride: FreshCandidatePresenceFilter; } export interface FreshCandidateDiagnosticRow { candidateId: number; + revision: number; displayTitle: string; + parsedTitle: string; + parsedMediaType: 'movie' | 'tv'; mediaType: 'movie' | 'tv'; matchYear?: number; + seasonNumber?: number; + episodeNumber?: number; resolutionStatus: number; displayStatus: FreshCandidateDiagnosticStatus; + automaticResolution?: { + status: number; + mediaType: 'movie' | 'tv'; + tmdbId?: number; + failureReason?: string; + }; tmdbId?: number; firstObservedAt: string; lastObservedAt: string; @@ -141,6 +170,34 @@ export interface FreshCandidateDiagnosticRow { resolvedAt?: string; failureReason?: string; membershipReason?: string; + automaticReasons: string[]; + sourceTitleSamples: string[]; + observationCount: number; + manualResolution?: { + mediaType: 'movie' | 'tv'; + tmdbId: number; + actorUserId?: number; + updatedAt: string; + revision: number; + canonicalTitle: string; + }; + admissionOverride?: { + actorUserId?: number; + updatedAt: string; + revision: number; + }; + discoveryHistory?: { + identityKind: 'movie' | 'season' | 'special' | 'legacy_tv'; + seasonNumber?: number; + episodeNumber?: number; + admitted: boolean; + legacyProjection: boolean; + admissionReason: string; + activityDate?: string; + activitySource: string; + firstFreshAt?: string; + visibleUntil?: string; + }; firstSeenAt?: string; lastSeenAt?: string; mediaDate?: string; @@ -157,6 +214,12 @@ export interface FreshCandidateDiagnosticRow { visibleUntil?: string; active: boolean; actionable: boolean; + actions: { + resolve: boolean; + resetResolution: boolean; + admit: boolean; + removeOverride: boolean; + }; } export interface FreshCandidateDiagnosticSummary { @@ -170,6 +233,8 @@ export interface FreshCandidateDiagnosticSummary { excludedContentFilter: number; visibilityExpired: number; needsAttention: number; + reviewable: number; + historical: number; } export interface FreshCandidateDiagnosticResponse { diff --git a/server/migration/postgres/1790000000011-RefineFreshDiscovery.ts b/server/migration/postgres/1790000000011-RefineFreshDiscovery.ts new file mode 100644 index 0000000000..ea93276900 --- /dev/null +++ b/server/migration/postgres/1790000000011-RefineFreshDiscovery.ts @@ -0,0 +1,364 @@ +import type { MigrationInterface, QueryRunner } from 'typeorm'; +import { Table, TableColumn, TableIndex, TableUnique } from 'typeorm'; + +const addColumn = async ( + queryRunner: QueryRunner, + table: string, + column: TableColumn +) => { + if (!(await queryRunner.hasColumn(table, column.name))) { + await queryRunner.addColumn(table, column); + } +}; + +export class RefineFreshDiscovery1790000000011 implements MigrationInterface { + name = 'RefineFreshDiscovery1790000000011'; + + public async up(queryRunner: QueryRunner): Promise { + const booleanDefault = false; + for (const [table, column] of [ + [ + 'fresh_observation', + new TableColumn({ + name: 'sourceTitle', + type: 'varchar', + length: '300', + default: "''", + }), + ], + [ + 'fresh_observation', + new TableColumn({ name: 'seasonNumber', type: 'integer', default: -1 }), + ], + [ + 'fresh_observation', + new TableColumn({ + name: 'episodeNumber', + type: 'integer', + default: -1, + }), + ], + [ + 'fresh_observation', + new TableColumn({ + name: 'explicitSeason', + type: 'boolean', + default: booleanDefault, + }), + ], + [ + 'fresh_observation', + new TableColumn({ + name: 'explicitSpecial', + type: 'boolean', + default: booleanDefault, + }), + ], + [ + 'fresh_observation', + new TableColumn({ + name: 'comparisonVersion', + type: 'integer', + default: 1, + }), + ], + [ + 'fresh_observation', + new TableColumn({ + name: 'sourceEvidenceKey', + type: 'varchar', + length: '64', + default: "''", + }), + ], + [ + 'fresh_candidate', + new TableColumn({ name: 'seasonKey', type: 'integer', default: -1 }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'specialEpisodeKey', + type: 'integer', + default: -1, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'explicitSeason', + type: 'boolean', + default: booleanDefault, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'explicitSpecial', + type: 'boolean', + default: booleanDefault, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'comparisonVersion', + type: 'integer', + default: 1, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'sourceEvidenceKey', + type: 'varchar', + length: '64', + default: "''", + }), + ], + [ + 'fresh_candidate', + new TableColumn({ name: 'revision', type: 'integer', default: 1 }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'effectiveMediaType', + type: 'varchar', + length: '8', + isNullable: true, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'automaticTmdbId', + type: 'integer', + isNullable: true, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'automaticFreshMediaId', + type: 'integer', + isNullable: true, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'automaticStatus', + type: 'integer', + isNullable: true, + }), + ], + [ + 'fresh_candidate', + new TableColumn({ + name: 'automaticFailureReason', + type: 'varchar', + length: '64', + isNullable: true, + }), + ], + [ + 'fresh_media', + new TableColumn({ + name: 'automaticReasons', + type: 'text', + default: "'[]'", + }), + ], + ] as const) + await addColumn(queryRunner, table, column); + await queryRunner.query( + `UPDATE "fresh_candidate" SET "automaticTmdbId" = "tmdbId", "automaticFreshMediaId" = "freshMediaId", "automaticStatus" = "status", "automaticFailureReason" = "lastFailureReason", "effectiveMediaType" = "mediaType" WHERE "automaticStatus" IS NULL` + ); + + const candidateTable = await queryRunner.getTable('fresh_candidate'); + const oldIdentity = candidateTable?.uniques.find( + (unique) => unique.name === 'UQ_fresh_candidate_identity' + ); + if (oldIdentity) + await queryRunner.dropUniqueConstraint('fresh_candidate', oldIdentity); + await queryRunner.createUniqueConstraint( + 'fresh_candidate', + new TableUnique({ + name: 'UQ_fresh_candidate_identity_v2', + columnNames: [ + 'sourceGeneration', + 'mediaType', + 'normalizedTitle', + 'matchYear', + 'seasonKey', + 'specialEpisodeKey', + ], + }) + ); + + const time = 'timestamp with time zone'; + await queryRunner.createTable( + new Table({ + name: 'fresh_discovery_history', + columns: [ + { name: 'id', type: 'serial', isPrimary: true }, + { name: 'mediaType', type: 'varchar', length: '8' }, + { name: 'tmdbId', type: 'integer' }, + { name: 'identityKind', type: 'varchar', length: '16' }, + { name: 'seasonKey', type: 'integer', default: -1 }, + { name: 'specialEpisodeKey', type: 'integer', default: -1 }, + { name: 'admitted', type: 'boolean', default: false }, + { name: 'legacyProjection', type: 'boolean', default: false }, + { name: 'firstObservedAt', type: time }, + { name: 'lastObservedAt', type: time }, + { name: 'firstFreshAt', type: time, isNullable: true }, + { name: 'visibleUntil', type: time, isNullable: true }, + { + name: 'activityDate', + type: 'varchar', + length: '10', + isNullable: true, + }, + { + name: 'activitySource', + type: 'varchar', + length: '32', + default: "'unavailable'", + }, + { + name: 'admissionReason', + type: 'varchar', + length: '64', + default: "'not_evaluated'", + }, + { name: 'automaticReasons', type: 'text', default: "'[]'" }, + { name: 'firstSeenGeneration', type: 'integer', default: 0 }, + { name: 'lastSeenGeneration', type: 'integer', default: 0 }, + { name: 'createdAt', type: time, default: 'now()' }, + { name: 'updatedAt', type: time, default: 'now()' }, + ], + uniques: [ + { + name: 'UQ_fresh_discovery_history_identity', + columnNames: [ + 'mediaType', + 'tmdbId', + 'identityKind', + 'seasonKey', + 'specialEpisodeKey', + ], + }, + ], + }), + true + ); + await queryRunner.createIndex( + 'fresh_discovery_history', + new TableIndex({ + name: 'IDX_fresh_discovery_history_projection', + columnNames: ['mediaType', 'tmdbId', 'visibleUntil'], + }) + ); + + await queryRunner.createTable( + new Table({ + name: 'fresh_manual_resolution', + columns: [ + { name: 'id', type: 'serial', isPrimary: true }, + { name: 'sourceEvidenceVersion', type: 'integer', default: 1 }, + { name: 'sourceEvidenceKey', type: 'varchar', length: '64' }, + { name: 'mediaType', type: 'varchar', length: '8' }, + { name: 'tmdbId', type: 'integer' }, + { name: 'canonicalTitle', type: 'varchar', length: '300' }, + { + name: 'canonicalDate', + type: 'varchar', + length: '10', + isNullable: true, + }, + { name: 'active', type: 'boolean', default: true }, + { name: 'revision', type: 'integer', default: 1 }, + { name: 'actorUserId', type: 'integer', isNullable: true }, + { name: 'createdAt', type: time, default: 'now()' }, + { name: 'updatedAt', type: time, default: 'now()' }, + ], + uniques: [ + { + name: 'UQ_fresh_manual_resolution_source', + columnNames: ['sourceEvidenceVersion', 'sourceEvidenceKey'], + }, + ], + }), + true + ); + await queryRunner.createIndex( + 'fresh_manual_resolution', + new TableIndex({ + name: 'IDX_fresh_manual_resolution_identity', + columnNames: ['mediaType', 'tmdbId'], + }) + ); + + await queryRunner.createTable( + new Table({ + name: 'fresh_admission_override', + columns: [ + { name: 'id', type: 'serial', isPrimary: true }, + { name: 'mediaType', type: 'varchar', length: '8' }, + { name: 'tmdbId', type: 'integer' }, + { name: 'identityKind', type: 'varchar', length: '16' }, + { name: 'seasonKey', type: 'integer', default: -1 }, + { name: 'specialEpisodeKey', type: 'integer', default: -1 }, + { name: 'active', type: 'boolean', default: true }, + { name: 'revision', type: 'integer', default: 1 }, + { name: 'actorUserId', type: 'integer', isNullable: true }, + { name: 'firstAdmittedAt', type: time, isNullable: true }, + { name: 'createdAt', type: time, default: 'now()' }, + { name: 'updatedAt', type: time, default: 'now()' }, + ], + uniques: [ + { + name: 'UQ_fresh_admission_override_identity', + columnNames: [ + 'mediaType', + 'tmdbId', + 'identityKind', + 'seasonKey', + 'specialEpisodeKey', + ], + }, + ], + }), + true + ); + await queryRunner.createIndex( + 'fresh_admission_override', + new TableIndex({ + name: 'IDX_fresh_admission_override_active', + columnNames: ['active'], + }) + ); + + await queryRunner.query(` + INSERT INTO "fresh_discovery_history" + ("mediaType", "tmdbId", "identityKind", "seasonKey", "specialEpisodeKey", + "admitted", "legacyProjection", "firstObservedAt", "lastObservedAt", + "firstFreshAt", "visibleUntil", "activityDate", "activitySource", + "admissionReason", "automaticReasons", "firstSeenGeneration", "lastSeenGeneration") + SELECT "mediaType", "tmdbId", CASE WHEN "mediaType" = 'movie' THEN 'movie' ELSE 'legacy_tv' END, + -1, -1, true, CASE WHEN "mediaType" = 'tv' THEN true ELSE false END, + "firstSeenAt", "lastSeenAt", "firstSeenAt", + "firstSeenAt" + make_interval(days => COALESCE((SELECT "freshVisibilityDays" FROM "fresh_sync_state" WHERE "id" = 1), 7)), + "mediaDate", CASE WHEN "mediaType" = 'movie' THEN 'legacy_movie' ELSE 'legacy_tv' END, + CASE WHEN "mediaType" = 'movie' THEN 'legacy_movie_admission' ELSE 'legacy_tv_projection' END, + '[]', "lastMatchedGeneration", "lastMatchedGeneration" + FROM "fresh_media" WHERE "admitted" = true + ON CONFLICT DO NOTHING + `); + } + + public async down(): Promise { + // Fresh downstream history is intentionally irreversible. + } +} diff --git a/server/migration/sqlite/1790000000010-RefineFreshDiscovery.ts b/server/migration/sqlite/1790000000010-RefineFreshDiscovery.ts new file mode 100644 index 0000000000..d593ebe23f --- /dev/null +++ b/server/migration/sqlite/1790000000010-RefineFreshDiscovery.ts @@ -0,0 +1,384 @@ +import type { MigrationInterface, QueryRunner } from 'typeorm'; +import { Table, TableColumn, TableIndex, TableUnique } from 'typeorm'; + +const addColumn = async ( + queryRunner: QueryRunner, + table: string, + column: TableColumn +) => { + if (!(await queryRunner.hasColumn(table, column.name))) { + await queryRunner.addColumn(table, column); + } +}; + +export class RefineFreshDiscovery1790000000010 implements MigrationInterface { + name = 'RefineFreshDiscovery1790000000010'; + + public async up(queryRunner: QueryRunner): Promise { + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ + name: 'sourceTitle', + type: 'varchar', + length: '300', + default: "''", + }) + ); + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ name: 'seasonNumber', type: 'integer', default: -1 }) + ); + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ name: 'episodeNumber', type: 'integer', default: -1 }) + ); + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ name: 'explicitSeason', type: 'boolean', default: 0 }) + ); + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ name: 'explicitSpecial', type: 'boolean', default: 0 }) + ); + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ + name: 'comparisonVersion', + type: 'integer', + default: 1, + }) + ); + await addColumn( + queryRunner, + 'fresh_observation', + new TableColumn({ + name: 'sourceEvidenceKey', + type: 'varchar', + length: '64', + default: "''", + }) + ); + + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ name: 'seasonKey', type: 'integer', default: -1 }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'specialEpisodeKey', + type: 'integer', + default: -1, + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ name: 'explicitSeason', type: 'boolean', default: 0 }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ name: 'explicitSpecial', type: 'boolean', default: 0 }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'comparisonVersion', + type: 'integer', + default: 1, + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'sourceEvidenceKey', + type: 'varchar', + length: '64', + default: "''", + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ name: 'revision', type: 'integer', default: 1 }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'effectiveMediaType', + type: 'varchar', + length: '8', + isNullable: true, + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'automaticTmdbId', + type: 'integer', + isNullable: true, + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'automaticFreshMediaId', + type: 'integer', + isNullable: true, + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'automaticStatus', + type: 'integer', + isNullable: true, + }) + ); + await addColumn( + queryRunner, + 'fresh_candidate', + new TableColumn({ + name: 'automaticFailureReason', + type: 'varchar', + length: '64', + isNullable: true, + }) + ); + await addColumn( + queryRunner, + 'fresh_media', + new TableColumn({ + name: 'automaticReasons', + type: 'text', + default: "'[]'", + }) + ); + await queryRunner.query( + `UPDATE "fresh_candidate" SET "automaticTmdbId" = "tmdbId", "automaticFreshMediaId" = "freshMediaId", "automaticStatus" = "status", "automaticFailureReason" = "lastFailureReason", "effectiveMediaType" = "mediaType" WHERE "automaticStatus" IS NULL` + ); + + const candidateTable = await queryRunner.getTable('fresh_candidate'); + const oldIdentity = candidateTable?.uniques.find( + (unique) => unique.name === 'UQ_fresh_candidate_identity' + ); + if (oldIdentity) + await queryRunner.dropUniqueConstraint('fresh_candidate', oldIdentity); + if ( + !candidateTable?.uniques.some( + (unique) => unique.name === 'UQ_fresh_candidate_identity_v2' + ) + ) { + await queryRunner.createUniqueConstraint( + 'fresh_candidate', + new TableUnique({ + name: 'UQ_fresh_candidate_identity_v2', + columnNames: [ + 'sourceGeneration', + 'mediaType', + 'normalizedTitle', + 'matchYear', + 'seasonKey', + 'specialEpisodeKey', + ], + }) + ); + } + + await queryRunner.createTable( + new Table({ + name: 'fresh_discovery_history', + columns: [ + { + name: 'id', + type: 'integer', + isPrimary: true, + isGenerated: true, + generationStrategy: 'increment', + }, + { name: 'mediaType', type: 'varchar', length: '8' }, + { name: 'tmdbId', type: 'integer' }, + { name: 'identityKind', type: 'varchar', length: '16' }, + { name: 'seasonKey', type: 'integer', default: -1 }, + { name: 'specialEpisodeKey', type: 'integer', default: -1 }, + { name: 'admitted', type: 'boolean', default: 0 }, + { name: 'legacyProjection', type: 'boolean', default: 0 }, + { name: 'firstObservedAt', type: 'datetime' }, + { name: 'lastObservedAt', type: 'datetime' }, + { name: 'firstFreshAt', type: 'datetime', isNullable: true }, + { name: 'visibleUntil', type: 'datetime', isNullable: true }, + { + name: 'activityDate', + type: 'varchar', + length: '10', + isNullable: true, + }, + { + name: 'activitySource', + type: 'varchar', + length: '32', + default: "'unavailable'", + }, + { + name: 'admissionReason', + type: 'varchar', + length: '64', + default: "'not_evaluated'", + }, + { name: 'automaticReasons', type: 'text', default: "'[]'" }, + { name: 'firstSeenGeneration', type: 'integer', default: 0 }, + { name: 'lastSeenGeneration', type: 'integer', default: 0 }, + { name: 'createdAt', type: 'datetime', default: 'CURRENT_TIMESTAMP' }, + { name: 'updatedAt', type: 'datetime', default: 'CURRENT_TIMESTAMP' }, + ], + uniques: [ + { + name: 'UQ_fresh_discovery_history_identity', + columnNames: [ + 'mediaType', + 'tmdbId', + 'identityKind', + 'seasonKey', + 'specialEpisodeKey', + ], + }, + ], + }), + true + ); + await queryRunner.createIndex( + 'fresh_discovery_history', + new TableIndex({ + name: 'IDX_fresh_discovery_history_projection', + columnNames: ['mediaType', 'tmdbId', 'visibleUntil'], + }) + ); + + await queryRunner.createTable( + new Table({ + name: 'fresh_manual_resolution', + columns: [ + { + name: 'id', + type: 'integer', + isPrimary: true, + isGenerated: true, + generationStrategy: 'increment', + }, + { name: 'sourceEvidenceVersion', type: 'integer', default: 1 }, + { name: 'sourceEvidenceKey', type: 'varchar', length: '64' }, + { name: 'mediaType', type: 'varchar', length: '8' }, + { name: 'tmdbId', type: 'integer' }, + { name: 'canonicalTitle', type: 'varchar', length: '300' }, + { + name: 'canonicalDate', + type: 'varchar', + length: '10', + isNullable: true, + }, + { name: 'active', type: 'boolean', default: 1 }, + { name: 'revision', type: 'integer', default: 1 }, + { name: 'actorUserId', type: 'integer', isNullable: true }, + { name: 'createdAt', type: 'datetime', default: 'CURRENT_TIMESTAMP' }, + { name: 'updatedAt', type: 'datetime', default: 'CURRENT_TIMESTAMP' }, + ], + uniques: [ + { + name: 'UQ_fresh_manual_resolution_source', + columnNames: ['sourceEvidenceVersion', 'sourceEvidenceKey'], + }, + ], + }), + true + ); + await queryRunner.createIndex( + 'fresh_manual_resolution', + new TableIndex({ + name: 'IDX_fresh_manual_resolution_identity', + columnNames: ['mediaType', 'tmdbId'], + }) + ); + + await queryRunner.createTable( + new Table({ + name: 'fresh_admission_override', + columns: [ + { + name: 'id', + type: 'integer', + isPrimary: true, + isGenerated: true, + generationStrategy: 'increment', + }, + { name: 'mediaType', type: 'varchar', length: '8' }, + { name: 'tmdbId', type: 'integer' }, + { name: 'identityKind', type: 'varchar', length: '16' }, + { name: 'seasonKey', type: 'integer', default: -1 }, + { name: 'specialEpisodeKey', type: 'integer', default: -1 }, + { name: 'active', type: 'boolean', default: 1 }, + { name: 'revision', type: 'integer', default: 1 }, + { name: 'actorUserId', type: 'integer', isNullable: true }, + { name: 'firstAdmittedAt', type: 'datetime', isNullable: true }, + { name: 'createdAt', type: 'datetime', default: 'CURRENT_TIMESTAMP' }, + { name: 'updatedAt', type: 'datetime', default: 'CURRENT_TIMESTAMP' }, + ], + uniques: [ + { + name: 'UQ_fresh_admission_override_identity', + columnNames: [ + 'mediaType', + 'tmdbId', + 'identityKind', + 'seasonKey', + 'specialEpisodeKey', + ], + }, + ], + }), + true + ); + await queryRunner.createIndex( + 'fresh_admission_override', + new TableIndex({ + name: 'IDX_fresh_admission_override_active', + columnNames: ['active'], + }) + ); + + await queryRunner.query(` + INSERT OR IGNORE INTO "fresh_discovery_history" + ("mediaType", "tmdbId", "identityKind", "seasonKey", "specialEpisodeKey", + "admitted", "legacyProjection", "firstObservedAt", "lastObservedAt", + "firstFreshAt", "visibleUntil", "activityDate", "activitySource", + "admissionReason", "automaticReasons", "firstSeenGeneration", "lastSeenGeneration") + SELECT "mediaType", "tmdbId", + CASE WHEN "mediaType" = 'movie' THEN 'movie' ELSE 'legacy_tv' END, + -1, -1, 1, CASE WHEN "mediaType" = 'tv' THEN 1 ELSE 0 END, + "firstSeenAt", "lastSeenAt", "firstSeenAt", + datetime("firstSeenAt", '+' || COALESCE((SELECT "freshVisibilityDays" FROM "fresh_sync_state" WHERE "id" = 1), 7) || ' days'), + "mediaDate", CASE WHEN "mediaType" = 'movie' THEN 'legacy_movie' ELSE 'legacy_tv' END, + CASE WHEN "mediaType" = 'movie' THEN 'legacy_movie_admission' ELSE 'legacy_tv_projection' END, + '[]', "lastMatchedGeneration", "lastMatchedGeneration" + FROM "fresh_media" WHERE "admitted" = 1 + `); + } + + public async down(): Promise { + // Fresh downstream history is intentionally irreversible. + } +} diff --git a/server/routes/fresh.test.ts b/server/routes/fresh.test.ts index 1f985c882f..9204a2f9cc 100644 --- a/server/routes/fresh.test.ts +++ b/server/routes/fresh.test.ts @@ -1,7 +1,9 @@ import { FreshCandidateStatus } from '@server/constants/fresh'; import { MediaStatus, MediaType } from '@server/constants/media'; import dataSource from '@server/datasource'; +import FreshAdmissionOverride from '@server/entity/FreshAdmissionOverride'; import FreshCandidate from '@server/entity/FreshCandidate'; +import FreshManualResolution from '@server/entity/FreshManualResolution'; import FreshMedia from '@server/entity/FreshMedia'; import FreshObservation from '@server/entity/FreshObservation'; import { FreshSyncState } from '@server/entity/FreshSyncState'; @@ -208,6 +210,95 @@ describe('Fresh route authorization and safe responses', () => { } }); + it('requires typed, revision-bound administrator candidate mutations', async () => { + const originalResolve = freshService.resolveCandidate; + const originalReset = freshService.resetCandidateResolution; + const originalAdmit = freshService.admitCandidate; + const originalRemove = freshService.removeCandidateOverride; + const calls: string[] = []; + freshService.resolveCandidate = async ( + candidateId, + mediaType, + tmdbId, + expectedRevision + ) => { + calls.push( + `resolve:${candidateId}:${mediaType}:${tmdbId}:${expectedRevision}` + ); + return { + candidate: new FreshCandidate({ + id: candidateId, + mediaType: 'movie', + effectiveMediaType: mediaType, + status: FreshCandidateStatus.RESOLVED, + }), + media: new FreshMedia({ + mediaType, + tmdbId, + displayTitle: 'Canonical Title', + active: false, + }), + }; + }; + freshService.resetCandidateResolution = async (candidateId, revision) => { + calls.push(`reset:${candidateId}:${revision}`); + return new FreshCandidate({ id: candidateId, revision: revision + 1 }); + }; + freshService.admitCandidate = async (candidateId, revision) => { + calls.push(`admit:${candidateId}:${revision}`); + return new FreshCandidate({ id: candidateId, revision: revision + 1 }); + }; + freshService.removeCandidateOverride = async (candidateId, revision) => { + calls.push(`remove:${candidateId}:${revision}`); + return new FreshCandidate({ id: candidateId, revision: revision + 1 }); + }; + try { + assert.equal( + ( + await request(app) + .post('/settings/fresh/candidates/10/resolve') + .set('x-test-role', 'user') + .send({ mediaType: 'tv', tmdbId: 305251, expectedRevision: 7 }) + ).status, + 403 + ); + const resolved = await request(app) + .post('/settings/fresh/candidates/10/resolve') + .set('x-test-role', 'admin') + .send({ mediaType: 'tv', tmdbId: 305251, expectedRevision: 7 }); + assert.equal(resolved.status, 200); + assert.equal(resolved.body.parsedMediaType, 'movie'); + assert.equal(resolved.body.mediaType, 'tv'); + assert.equal( + ( + await request(app) + .post('/settings/fresh/candidates/10/resolve') + .set('x-test-role', 'admin') + .send({ tmdbId: 305251 }) + ).status, + 400 + ); + for (const [endpoint, expected] of [ + ['reset-resolution', 'reset:10:8'], + ['admit', 'admit:10:8'], + ['remove-override', 'remove:10:8'], + ] as const) { + const response = await request(app) + .post(`/settings/fresh/candidates/10/${endpoint}`) + .set('x-test-role', 'admin') + .send({ expectedRevision: 8 }); + assert.equal(response.status, 200); + assert.ok(calls.includes(expected)); + } + assert.ok(calls.includes('resolve:10:tv:305251:7')); + } finally { + freshService.resolveCandidate = originalResolve; + freshService.resetCandidateResolution = originalReset; + freshService.admitCandidate = originalAdmit; + freshService.removeCandidateOverride = originalRemove; + } + }); + it('paginates the prepared projection after server-side filtering and sorting', async () => { const settings = getSettings(); settings.fresh = { @@ -418,6 +509,8 @@ describe('Fresh route authorization and safe responses', () => { }); it('serves persistent candidate diagnostics with admin-only filtering and summary counts', async () => { + await dataSource.getRepository(FreshManualResolution).clear(); + await dataSource.getRepository(FreshAdmissionOverride).clear(); await dataSource.getRepository(FreshCandidate).clear(); await dataSource.getRepository(FreshSyncState).save( new FreshSyncState({ @@ -456,6 +549,18 @@ describe('Fresh route authorization and safe responses', () => { firstObservedAt: new Date('2026-09-20T00:00:00Z'), lastObservedAt: new Date('2026-09-25T00:00:00Z'), }), + new FreshCandidate({ + sourceGeneration: 3, + mediaType: 'movie', + normalizedTitle: 'identity collision', + displayTitle: 'Identity Collision', + matchYear: 2026, + status: FreshCandidateStatus.RESOLVED, + automaticStatus: FreshCandidateStatus.RESOLVED, + lastFailureReason: 'source_evidence_collision', + firstObservedAt: new Date('2026-09-22T00:00:00Z'), + lastObservedAt: new Date('2026-09-23T00:00:00Z'), + }), new FreshCandidate({ sourceGeneration: 3, mediaType: 'tv', @@ -472,6 +577,7 @@ describe('Fresh route authorization and safe responses', () => { displayTitle: 'Legacy Evidence', matchYear: 2025, status: FreshCandidateStatus.RESOLVED, + sourceEvidenceKey: 'diagnostic-source-key', tmdbId: eligibilityUnknownMedia.tmdbId, freshMediaId: eligibilityUnknownMedia.id, firstObservedAt: new Date('2026-09-20T00:00:00Z'), @@ -483,7 +589,7 @@ describe('Fresh route authorization and safe responses', () => { sourceGeneration: 3, releaseId: 'diagnostic-legacy', filterId: 7, - candidateId: candidates[2].id, + candidateId: candidates[3].id, mediaType: 'movie', title: 'Legacy Evidence', normalizedTitle: 'legacy evidence', @@ -492,6 +598,26 @@ describe('Fresh route authorization and safe responses', () => { observedAt: new Date('2026-09-20T00:00:00Z'), }) ); + await dataSource.getRepository(FreshManualResolution).save( + new FreshManualResolution({ + sourceEvidenceVersion: 1, + sourceEvidenceKey: 'diagnostic-source-key', + mediaType: 'movie', + tmdbId: 9001, + canonicalTitle: 'Legacy Evidence', + active: true, + }) + ); + await dataSource.getRepository(FreshAdmissionOverride).save( + new FreshAdmissionOverride({ + mediaType: 'movie', + tmdbId: 9001, + identityKind: 'movie', + seasonKey: -1, + specialEpisodeKey: -1, + active: true, + }) + ); const forbidden = await request(app) .get('/settings/fresh/candidates') .set('x-test-role', 'user'); @@ -508,17 +634,30 @@ describe('Fresh route authorization and safe responses', () => { assert.equal(response.body.results[0].actionable, true); assert.equal(response.body.summary.noMatch, 1); assert.equal(response.body.summary.ambiguous, 1); - assert.equal(response.body.summary.needsAttention, 2); + assert.equal(response.body.summary.needsAttention, 3); assert.equal(response.body.summary.eligibilityUnknown, 1); assert.equal(response.body.summary.outsideEligibilityWindow, 0); + const prioritized = await request(app) + .get( + '/settings/fresh/candidates?page=1&mediaType=all&status=all&sort=priority' + ) + .set('x-test-role', 'admin'); + assert.equal(prioritized.status, 200); + assert.deepEqual( + prioritized.body.results + .slice(0, 3) + .map((row: { displayStatus: string }) => row.displayStatus), + ['no_match', 'ambiguous', 'needs_attention'] + ); + const attention = await request(app) .get( '/settings/fresh/candidates?page=1&mediaType=all&status=needs_attention&sort=status' ) .set('x-test-role', 'admin'); assert.equal(attention.status, 200); - assert.equal(attention.body.results.length, 2); + assert.equal(attention.body.results.length, 3); const unknown = await request(app) .get( @@ -537,5 +676,51 @@ describe('Fresh route authorization and safe responses', () => { eligibilityLimitDays: 90, legacyEvidence: true, }); + + const unknownSeason = await request(app) + .get( + '/settings/fresh/candidates?page=1&mediaType=tv&status=all&sort=priority&seasonEvidence=unknown' + ) + .set('x-test-role', 'admin'); + assert.equal(unknownSeason.status, 200); + assert.equal(unknownSeason.body.results.length, 1); + assert.equal( + unknownSeason.body.results[0].displayTitle, + 'Ambiguous Series' + ); + + const resolutionFamily = await request(app) + .get( + '/settings/fresh/candidates?page=1&mediaType=all&status=all&sort=priority&reasonFamily=resolution' + ) + .set('x-test-role', 'admin'); + assert.equal(resolutionFamily.status, 200); + assert.equal(resolutionFamily.body.results.length, 3); + + const manuallyResolved = await request(app) + .get( + '/settings/fresh/candidates?page=1&search=movie%3A9001&mediaType=all&status=all&sort=priority&manualResolution=present' + ) + .set('x-test-role', 'admin'); + assert.equal(manuallyResolved.status, 200); + assert.equal(manuallyResolved.body.results.length, 1); + assert.equal( + manuallyResolved.body.results[0].manualResolution.mediaType, + 'movie' + ); + + const overridden = await request(app) + .get( + '/settings/fresh/candidates?page=1&mediaType=all&status=all&sort=priority&admissionOverride=present' + ) + .set('x-test-role', 'admin'); + assert.equal(overridden.status, 200); + assert.equal(overridden.body.results.length, 1); + assert.equal(overridden.body.results[0].admissionOverride.revision, 1); + + const invalidFacet = await request(app) + .get('/settings/fresh/candidates?manualResolution=sometimes') + .set('x-test-role', 'admin'); + assert.equal(invalidFacet.status, 400); }); }); diff --git a/server/routes/settings/fresh.ts b/server/routes/settings/fresh.ts index f9ad385f81..d4ba8e268a 100644 --- a/server/routes/settings/fresh.ts +++ b/server/routes/settings/fresh.ts @@ -134,10 +134,13 @@ const CandidateQuerySchema = z 'visibility_expired', 'active_fresh', 'needs_attention', + 'reviewable', + 'historical', ]) .default('all'), sort: z .enum([ + 'priority', 'title.asc', 'title.desc', 'status', @@ -148,7 +151,13 @@ const CandidateQuerySchema = z 'last_seen.desc', 'last_seen.asc', ]) - .default('last_seen.desc'), + .default('priority'), + reasonFamily: z + .enum(['all', 'resolution', 'admission', 'content', 'history', 'source']) + .default('all'), + seasonEvidence: z.enum(['all', 'known', 'unknown']).default('all'), + manualResolution: z.enum(['all', 'present', 'absent']).default('all'), + admissionOverride: z.enum(['all', 'present', 'absent']).default('all'), }) .strict(); @@ -169,13 +178,24 @@ freshSettingsRoutes.get('/candidates', async (req, res, next) => { freshSettingsRoutes.post('/candidates/:id/resolve', async (req, res, next) => { try { const candidateId = z.coerce.number().int().positive().parse(req.params.id); - const { tmdbId } = z - .object({ tmdbId: z.number().int().positive() }) + const { mediaType, tmdbId, expectedRevision } = z + .object({ + mediaType: z.enum(['movie', 'tv']), + tmdbId: z.number().int().positive(), + expectedRevision: z.number().int().positive(), + }) .strict() .parse(req.body); - const result = await freshService.resolveCandidate(candidateId, tmdbId); + const result = await freshService.resolveCandidate( + candidateId, + mediaType, + tmdbId, + expectedRevision, + req.user?.id + ); return res.status(200).json({ candidateId: result.candidate.id, + parsedMediaType: result.candidate.mediaType, mediaType: result.media.mediaType, tmdbId: result.media.tmdbId, title: result.media.displayTitle, @@ -184,12 +204,17 @@ freshSettingsRoutes.post('/candidates/:id/resolve', async (req, res, next) => { active: result.media.active, }); } catch (error) { + if (error instanceof z.ZodError) { + return next({ status: 400, message: 'Invalid TMDB resolution.' }); + } const code = error instanceof Error ? error.message : ''; if ( [ 'candidate_not_found', 'candidate_not_actionable', 'candidate_not_current', + 'stale_candidate', + 'source_evidence_collision', ].includes(code) ) { return next({ @@ -202,6 +227,8 @@ freshSettingsRoutes.post('/candidates/:id/resolve', async (req, res, next) => { 'invalid_candidate', 'invalid_tmdb_id', 'invalid_tmdb_response', + 'invalid_media_type', + 'invalid_revision', ].includes(code) ) { return next({ status: 400, message: 'Invalid TMDB resolution.' }); @@ -213,6 +240,101 @@ freshSettingsRoutes.post('/candidates/:id/resolve', async (req, res, next) => { } }); +const CandidateMutationSchema = z + .object({ expectedRevision: z.number().int().positive() }) + .strict(); + +freshSettingsRoutes.post( + '/candidates/:id/reset-resolution', + async (req, res, next) => { + try { + const candidateId = z.coerce + .number() + .int() + .positive() + .parse(req.params.id); + const { expectedRevision } = CandidateMutationSchema.parse(req.body); + const candidate = await freshService.resetCandidateResolution( + candidateId, + expectedRevision + ); + return res + .status(200) + .json({ candidateId, revision: candidate.revision }); + } catch (error) { + if (error instanceof z.ZodError) { + return next({ + status: 400, + message: 'Invalid candidate mutation request.', + }); + } + const code = error instanceof Error ? error.message : ''; + return next({ + status: code === 'candidate_not_found' ? 404 : 409, + message: 'Candidate resolution could not be reset.', + }); + } + } +); + +freshSettingsRoutes.post('/candidates/:id/admit', async (req, res, next) => { + try { + const candidateId = z.coerce.number().int().positive().parse(req.params.id); + const { expectedRevision } = CandidateMutationSchema.parse(req.body); + const candidate = await freshService.admitCandidate( + candidateId, + expectedRevision, + req.user?.id + ); + return res.status(200).json({ candidateId, revision: candidate.revision }); + } catch (error) { + if (error instanceof z.ZodError) { + return next({ + status: 400, + message: 'Invalid candidate mutation request.', + }); + } + const code = error instanceof Error ? error.message : ''; + return next({ + status: code === 'candidate_not_found' ? 404 : 409, + message: 'Candidate cannot be admitted to Fresh.', + }); + } +}); + +freshSettingsRoutes.post( + '/candidates/:id/remove-override', + async (req, res, next) => { + try { + const candidateId = z.coerce + .number() + .int() + .positive() + .parse(req.params.id); + const { expectedRevision } = CandidateMutationSchema.parse(req.body); + const candidate = await freshService.removeCandidateOverride( + candidateId, + expectedRevision + ); + return res + .status(200) + .json({ candidateId, revision: candidate.revision }); + } catch (error) { + if (error instanceof z.ZodError) { + return next({ + status: 400, + message: 'Invalid candidate mutation request.', + }); + } + const code = error instanceof Error ? error.message : ''; + return next({ + status: code === 'candidate_not_found' ? 404 : 409, + message: 'Fresh admission override could not be removed.', + }); + } + } +); + freshSettingsRoutes.post('/refresh', async (_req, res, next) => { try { await freshService.refresh(); diff --git a/src/components/Settings/SettingsFresh/CandidateDiagnostics.tsx b/src/components/Settings/SettingsFresh/CandidateDiagnostics.tsx index 60c861c39b..23cd6f1309 100644 --- a/src/components/Settings/SettingsFresh/CandidateDiagnostics.tsx +++ b/src/components/Settings/SettingsFresh/CandidateDiagnostics.tsx @@ -10,6 +10,9 @@ import type { FreshCandidateDiagnosticRow, FreshCandidateDiagnosticSort, FreshCandidateDiagnosticStatus, + FreshCandidatePresenceFilter, + FreshCandidateReasonFamily, + FreshCandidateSeasonEvidence, } from '@server/lib/fresh/types'; import axios from 'axios'; import Link from 'next/link'; @@ -31,9 +34,12 @@ const statusLabels: Record = { visibility_expired: 'Visibility Expired', active_fresh: 'Active Fresh', needs_attention: 'Needs Attention', + reviewable: 'Reviewable', + historical: 'Historical', }; const sortLabels: Record = { + priority: 'Needs Attention first', 'title.asc': 'Title A–Z', 'title.desc': 'Title Z–A', status: 'Status', @@ -45,6 +51,32 @@ const sortLabels: Record = { 'last_seen.asc': 'Last Seen oldest', }; +const reasonFamilyLabels: Record = { + all: 'All reason families', + resolution: 'Resolution', + admission: 'Admission policy', + content: 'Content policy', + history: 'Discovery history', + source: 'Source continuity', +}; + +const seasonEvidenceLabels: Record = { + all: 'All season evidence', + known: 'Season known', + unknown: 'Season unknown', +}; + +const presenceLabels: Record = { + all: 'Any', + present: 'Present', + absent: 'Absent', +}; + +const diagnosticReasonLabel = (value: string) => + value + .replaceAll('_', ' ') + .replace(/\b\w/g, (character) => character.toUpperCase()); + const ManualResolution = ({ candidate, onResolved, @@ -53,10 +85,21 @@ const ManualResolution = ({ onResolved: () => Promise; }) => { const [tmdbId, setTmdbId] = useState(''); + const [mediaType, setMediaType] = useState<'movie' | 'tv'>( + candidate.mediaType + ); const [busy, setBusy] = useState(false); const [message, setMessage] = useState(''); return (
+ ( `/api/v1/settings/fresh/candidates/${candidate.candidateId}/resolve`, - { tmdbId: Number(tmdbId) } + { + mediaType, + tmdbId: Number(tmdbId), + expectedRevision: candidate.revision, + } ); setMessage( `Resolved as ${response.data.title} (${response.data.mediaType}:${response.data.tmdbId}).` ); await onResolved(); - } catch { - setMessage('The TMDB identity could not be validated.'); + } catch (error) { + if (axios.isAxiosError(error) && error.response?.status === 409) { + setMessage('Candidate changed. Reloaded the current state.'); + await onResolved(); + } else { + setMessage('The TMDB identity could not be validated.'); + } } finally { setBusy(false); } @@ -98,6 +150,56 @@ const ManualResolution = ({ ); }; +const CandidateMutation = ({ + candidate, + endpoint, + label, + buttonType, + onChanged, +}: { + candidate: FreshCandidateDiagnosticRow; + endpoint: 'reset-resolution' | 'admit' | 'remove-override'; + label: string; + buttonType?: 'default' | 'primary' | 'danger' | 'warning' | 'success'; + onChanged: () => Promise; +}) => { + const [busy, setBusy] = useState(false); + const [message, setMessage] = useState(''); + return ( + <> + + {message && {message}} + + ); +}; + const EligibilityDetails = ({ candidate, id, @@ -160,6 +262,39 @@ const EligibilityDetails = ({ 'Membership', statusLabels[candidate.displayStatus], ], + ['Parsed title', candidate.parsedTitle], + ['Parsed type', candidate.parsedMediaType === 'movie' ? 'Movie' : 'TV'], + candidate.seasonNumber !== undefined && [ + 'Parsed season', + `S${String(candidate.seasonNumber).padStart(2, '0')}`, + ], + candidate.episodeNumber !== undefined && [ + 'Parsed episode', + `E${String(candidate.episodeNumber).padStart(2, '0')}`, + ], + candidate.automaticResolution && [ + 'Automatic resolution', + candidate.automaticResolution.tmdbId + ? `${candidate.automaticResolution.mediaType}:${candidate.automaticResolution.tmdbId}` + : (candidate.automaticResolution.failureReason ?? 'Unresolved'), + ], + candidate.manualResolution && [ + 'Manual resolution', + `${candidate.manualResolution.mediaType}:${candidate.manualResolution.tmdbId} · ${candidate.manualResolution.canonicalTitle}`, + ], + ['Observations', String(candidate.observationCount)], + candidate.sourceTitleSamples.length > 0 && [ + 'Source samples', + candidate.sourceTitleSamples.join(' · '), + ], + candidate.discoveryHistory?.activityDate && [ + 'Activity date', + `${candidate.discoveryHistory.activityDate} (${candidate.discoveryHistory.activitySource})`, + ], + candidate.automaticReasons.length > 0 && [ + 'Automatic reasons', + candidate.automaticReasons.map(diagnosticReasonLabel).join(', '), + ], ].filter(Boolean) as [string, string][]; return (
· {candidate.matchYear} ) : null} + {candidate.seasonNumber !== undefined ? ( + + · S{String(candidate.seasonNumber).padStart(2, '0')} + {candidate.episodeNumber !== undefined + ? `E${String(candidate.episodeNumber).padStart(2, '0')}` + : ''} + + ) : null} · {candidate.mediaType === 'movie' ? 'Movie' : 'Series'} @@ -241,9 +384,50 @@ const CandidateRow = ({
{statusLabels[candidate.displayStatus]} - {candidate.actionable && ( + {!candidate.active && candidate.automaticReasons[0] && ( + + {diagnosticReasonLabel(candidate.automaticReasons[0])} + + )} + {candidate.manualResolution && ( + + Manual resolution + + )} + {candidate.admissionOverride && ( + + Admission override + + )} + {candidate.actions.resolve && ( )} + {candidate.actions.resetResolution && ( + + )} + {candidate.actions.admit && ( + + )} + {candidate.actions.removeOverride && ( + + )}
{expanded && } @@ -251,18 +435,33 @@ const CandidateRow = ({ ); }; -const CandidateDiagnostics = () => { +const CandidateDiagnostics = ({ + showHeader = true, +}: { + showHeader?: boolean; +}) => { const [page, setPage] = useState(1); const [search, setSearch] = useState(''); const [mediaType, setMediaType] = useState<'all' | 'movie' | 'tv'>('all'); const [status, setStatus] = useState('all'); - const [sort, setSort] = - useState('last_seen.desc'); + const [sort, setSort] = useState('priority'); + const [reasonFamily, setReasonFamily] = + useState('all'); + const [seasonEvidence, setSeasonEvidence] = + useState('all'); + const [manualResolution, setManualResolution] = + useState('all'); + const [admissionOverride, setAdmissionOverride] = + useState('all'); const params = new URLSearchParams({ page: String(page), mediaType, status, sort, + reasonFamily, + seasonEvidence, + manualResolution, + admissionOverride, }); if (search.trim()) params.set('search', search.trim()); const { data, error, mutate } = useSWR( @@ -273,45 +472,23 @@ const CandidateDiagnostics = () => { callback(); }; return ( -
-

Candidate Diagnostics

-

- Durable current candidate and membership state. This is separate from - the latest synchronization attempt and survives restarts. -

+
+ {showHeader && ( + <> +

Candidate Diagnostics

+

+ Durable current candidate and membership state. This is separate + from the latest synchronization attempt and survives restarts. +

+ + )} {data && (
{[ - ['Total candidates', data.summary.totalCandidates, 'all'], - ['Active Fresh', data.summary.activeFresh, 'active_fresh'], - ['No Match', data.summary.noMatch, 'no_match'], - ['Ambiguous', data.summary.ambiguous, 'ambiguous'], - [ - 'Temporary Failure', - data.summary.temporaryFailure, - 'temporary_failure', - ], - [ - 'Outside Eligibility Window', - data.summary.outsideEligibilityWindow, - 'outside_eligibility_window', - ], - [ - 'Eligibility Unknown', - data.summary.eligibilityUnknown, - 'eligibility_unknown', - ], - [ - 'Excluded by Content Filter', - data.summary.excludedContentFilter, - 'excluded_content_filter', - ], - [ - 'Visibility Expired', - data.summary.visibilityExpired, - 'visibility_expired', - ], ['Needs Attention', data.summary.needsAttention, 'needs_attention'], + ['Reviewable', data.summary.reviewable, 'reviewable'], + ['Active Fresh', data.summary.activeFresh, 'active_fresh'], + ['Historical', data.summary.historical, 'historical'], ].map(([label, value, cardStatus]) => (
+
+ ({ value, label: seasonEvidenceLabels[value] }))} + onChange={(option) => + change(() => setSeasonEvidence(option?.value ?? 'all')) + } + /> + ({ + value, + label: `Admission override: ${presenceLabels[value]}`, + }))} + onChange={(option) => + change(() => setAdmissionOverride(option?.value ?? 'all')) } />
diff --git a/src/components/Settings/SettingsFresh/index.tsx b/src/components/Settings/SettingsFresh/index.tsx index 11b14c94d1..157474edcc 100644 --- a/src/components/Settings/SettingsFresh/index.tsx +++ b/src/components/Settings/SettingsFresh/index.tsx @@ -5,23 +5,35 @@ import SensitiveInput from '@app/components/Common/SensitiveInput'; import CandidateDiagnostics from '@app/components/Settings/SettingsFresh/CandidateDiagnostics'; import FreshContentFilters from '@app/components/Settings/SettingsFresh/FreshContentFilters'; import { + FRESH_SECTION_STATE_KEY, + defaultFreshSectionState, loadFreshFilters, + parseFreshSectionState, selectedFreshFilter, toFreshConnectionUpdate, toFreshFilterSelectOptions, toFreshSettingsFormValues, toFreshSettingsUpdate, type FreshFilterOption, + type FreshSectionId, + type FreshSectionState, type FreshSettingsResponse, } from '@app/components/Settings/SettingsFresh/settingsFresh'; import useToasts from '@app/hooks/useToasts'; import defineMessages from '@app/utils/defineMessages'; import { Transition } from '@headlessui/react'; -import { ArrowPathIcon, BeakerIcon } from '@heroicons/react/24/solid'; -import type { FreshDiagnosticsSnapshot } from '@server/lib/fresh'; +import { + ArrowPathIcon, + BeakerIcon, + ChevronDownIcon, +} from '@heroicons/react/24/solid'; +import type { + FreshCandidateDiagnosticResponse, + FreshDiagnosticsSnapshot, +} from '@server/lib/fresh'; import axios from 'axios'; import { Field, Form, Formik } from 'formik'; -import { Fragment, useEffect, useState } from 'react'; +import { Fragment, useEffect, useState, type ReactNode } from 'react'; import { useIntl } from 'react-intl'; import Select from 'react-select'; import useSWR, { mutate } from 'swr'; @@ -57,7 +69,7 @@ const messages = defineMessages('components.Settings.SettingsFresh', { unavailableFilter: 'Configured filter unavailable (ID: {id})', mediaEligibilityDays: 'Media eligibility window (days)', mediaEligibilityHelp: - 'At the first qualifying observation, how old the canonical movie release or series first-air date may be.', + 'Normal Movie eligibility window and TV recent-season activity sanity window. Movies also receive a fixed 14-day first-observation grace.', freshVisibilityDays: 'Keep items in Fresh for (days)', freshVisibilityHelp: 'How long admitted media remains visible after its first qualifying observation.', @@ -77,11 +89,11 @@ const messages = defineMessages('components.Settings.SettingsFresh', { refreshDisabled: 'Enable and save Fresh settings before refreshing.', maintenance: 'Maintenance', maintenanceHelp: - 'Rebuild persistent Fresh discovery data only when a clean source reconstruction is required.', + 'Rebuild source-derived observations and automatic candidates only when a clean source reconstruction is required.', rebuild: 'Rebuild Fresh Data', rebuildTitle: 'Rebuild Fresh Data?', rebuildConfirm: - 'This deletes all persisted Fresh discovery history and rebuilds it from releases currently retained by autobrr. Fresh observations no longer present in autobrr Release History cannot be recovered. Watchlist, requests, media-library state, Radarr, Sonarr, and autobrr are unaffected.', + 'This clears source-derived observations, automatic candidates, and synchronization state, then rebuilds from releases currently retained by autobrr. Irreversible Fresh history, canonical media metadata, typed manual resolutions, and admission overrides are preserved. Releases no longer retained by autobrr cannot be recovered.', rebuildSuccess: 'Fresh data was rebuilt successfully.', rebuildFailed: 'Fresh data could not be rebuilt. Check the pipeline status for the failed stage.', @@ -171,6 +183,52 @@ const diagnosticLabel = (value: string) => value .replaceAll('_', ' ') .replace(/\b\w/g, (character) => character.toUpperCase()); + +const FreshSettingsSection = ({ + id, + title, + summary, + open, + order, + onToggle, + children, +}: { + id: FreshSectionId; + title: string; + summary: string; + open: boolean; + order: 1 | 2 | 3; + onToggle: () => void; + children: ReactNode; +}) => { + const contentId = `fresh-settings-${id}`; + const orderClass = { 1: 'order-1', 2: 'order-2', 3: 'order-3' }[order]; + return ( +
+ + {open && ( +
+ {children} +
+ )} +
+ ); +}; + const SettingsFresh = () => { const intl = useIntl(); const { addToast } = useToasts(); @@ -181,6 +239,9 @@ const SettingsFresh = () => { const [refreshing, setRefreshing] = useState(false); const [rebuilding, setRebuilding] = useState(false); const [showRebuildConfirmation, setShowRebuildConfirmation] = useState(false); + const [sections, setSections] = useState( + defaultFreshSectionState + ); const { data, error, @@ -188,6 +249,9 @@ const SettingsFresh = () => { } = useSWR('/api/v1/settings/fresh'); const { data: diagnostics, mutate: revalidateDiagnostics } = useSWR('/api/v1/settings/fresh/diagnostics'); + const { data: candidateSummary } = useSWR( + '/api/v1/settings/fresh/candidates?page=1&mediaType=all&status=all&sort=priority' + ); const apiTokenConfigured = data?.apiTokenConfigured; const baseUrl = data?.baseUrl; useEffect(() => { @@ -213,6 +277,23 @@ const SettingsFresh = () => { active = false; }; }, [apiTokenConfigured, baseUrl]); + useEffect(() => { + setSections( + parseFreshSectionState( + window.localStorage.getItem(FRESH_SECTION_STATE_KEY) + ) + ); + }, []); + const toggleSection = (id: FreshSectionId) => { + setSections((current) => { + const next = { ...current, [id]: !current[id] }; + window.localStorage.setItem( + FRESH_SECTION_STATE_KEY, + JSON.stringify(next) + ); + return next; + }); + }; if (!data && !error) return ; if (!data) return null; const schema = Yup.object({ @@ -303,6 +384,25 @@ const SettingsFresh = () => { [messages.currentMedia, counts.currentFreshMedia], ] as const) : []; + const candidateSectionSummary = candidateSummary + ? `Needs Attention ${candidateSummary.summary.needsAttention} · Reviewable ${candidateSummary.summary.reviewable} · Active ${candidateSummary.summary.activeFresh}` + : 'Loading durable candidate state…'; + const pipelineFailure = + diagnostics?.latestAttempt?.outcome === 'failed' + ? `Failed at ${diagnosticLabel( + diagnostics.latestAttempt.failingStage ?? 'reconciliation' + )}: ${diagnosticLabel( + diagnostics.latestAttempt.failureReason ?? 'unexpected_failure' + )}` + : undefined; + const pipelineSectionSummary = pipelineFailure + ? `${pipelineFailure} · ${continuity} · Last success ${formatTimestamp( + projection?.lastRefresh + )}` + : projection + ? `${projection.status} · Last sync ${formatTimestamp(projection.lastRefresh)} · ${continuity} · ${projection.itemCount} Fresh` + : 'Pipeline state unavailable'; + const configurationSectionSummary = `${data.enabled ? 'Enabled' : 'Disabled'} · ${data.cachedFilterName || `Filter ${data.filterId || 'not selected'}`} · ${data.mediaEligibilityDays}-day Movie/TV-activity window · ${data.freshVisibilityDays}-day visibility`; const rebuildFresh = async () => { setShowRebuildConfirmation(false); setRebuilding(true); @@ -351,483 +451,537 @@ const SettingsFresh = () => { {intl.formatMessage(messages.description)}

- { - try { - await axios.put( - '/api/v1/settings/fresh', - toFreshSettingsUpdate(values) - ); - await Promise.all([ - revalidate(), - mutate('/api/v1/settings/public'), - ]); - addToast(intl.formatMessage(messages.saved), { - appearance: 'success', - autoDismiss: true, - }); - } catch { - addToast(intl.formatMessage(messages.saveFailed), { - appearance: 'error', - autoDismiss: true, - }); - } - }} - > - {({ - values, - setFieldValue, - isSubmitting, - isValid, - errors, - touched, - }) => { - const loadFilters = async () => { - setTesting(true); - const result = await loadFreshFilters(async () => { - const response = await axios.post( - '/api/v1/settings/fresh/filters', - toFreshConnectionUpdate(values) - ); - return response.data; - }); - if (result.ok) { - setFilters(result.filters); - setFiltersLoaded(true); - addToast(intl.formatMessage(messages.testSuccess), { - appearance: 'success', - autoDismiss: true, - }); - } else { - addToast(intl.formatMessage(messages.testFailed), { - appearance: 'error', - autoDismiss: true, - }); - } - setTesting(false); - }; - const filterOptions = toFreshFilterSelectOptions(filters); - return ( -
-
- -
- -
-
-
- -
-
- - {values.protocol}:// - - -
- {errors.hostname && touched.hostname && ( -
{errors.hostname}
- )} -
-
-
- -
- - {errors.port && touched.port && ( -
{errors.port}
- )} -
-
-
- -
- - setFieldValue( - 'protocol', - values.protocol === 'https' ? 'http' : 'https' - ) - } - /> -
-
-
- -
-
- ) => - setFieldValue('apiToken', e.target.value) - } - /> -
- {errors.apiToken && - touched.apiToken && - typeof errors.apiToken === 'string' && ( -
{errors.apiToken}
- )} -
-
-
-
-
- -
-
-
- -
- { + setFieldValue('filterId', option?.value ?? 0); + setFieldValue( + 'cachedFilterName', + option?.label ?? '' + ); + }} + /> + {errors.filterId && touched.filterId && ( +
{errors.filterId}
+ )} +
+
+
+ +
+
+ +
+ {errors.mediaEligibilityDays && + touched.mediaEligibilityDays && ( +
+ {errors.mediaEligibilityDays} +
+ )} +
+
+
+ +
+
+ +
+ {errors.freshVisibilityDays && + touched.freshVisibilityDays && ( +
+ {errors.freshVisibilityDays} +
+ )} +
+
+ +
+
+ +
+
+ + ); }} - > - - {intl.formatMessage(messages.refresh)} - -
- {!data.enabled && ( -

- {intl.formatMessage(messages.refreshDisabled)} -

- )} -
- {projectionRows.map(([label, value]) => ( -
-
- {intl.formatMessage(label)} + +
+

+ {intl.formatMessage(messages.maintenance)} +

+

+ {intl.formatMessage(messages.maintenanceHelp)} +

+ +
+ + toggleSection('pipeline')} + > +
+
+
+

+ {intl.formatMessage(messages.latestAttempt)}:{' '} + {diagnostics?.latestAttempt?.outcome ?? '—'} ·{' '} + {intl.formatMessage(messages.currentProjection)}:{' '} + {diagnostics?.currentProjection.status ?? '—'} ( + {diagnostics?.currentProjection.itemCount ?? 0}) +

+ {diagnostics?.latestAttempt?.failingStage && + diagnostics.latestAttempt.failureReason && ( +

+ {intl.formatMessage(messages.failedAt, { + stage: diagnosticLabel( + diagnostics.latestAttempt.failingStage + ), + reason: diagnosticLabel( + diagnostics.latestAttempt.failureReason + ), + })} +

+ )} + {['failed', 'cancelled'].includes( + diagnostics?.latestAttempt?.outcome ?? '' + ) && ( +

+ {diagnostics?.latestAttempt?.lastGood + ? intl.formatMessage(messages.retainedProjection, { + timestamp: + diagnostics.latestAttempt.lastGood.timestamp, + count: diagnostics.latestAttempt.lastGood.itemCount, + }) + : intl.formatMessage(messages.noProjection)} +

+ )}
-
{value}
+
- ))} -
- {!diagnostics?.latestAttempt ? ( -

- {intl.formatMessage(messages.noDiagnostics)} -

- ) : ( - <> -
- {countRows.map(([label, value]) => ( + {!data.enabled && ( +

+ {intl.formatMessage(messages.refreshDisabled)} +

+ )} +
+ {projectionRows.map(([label, value]) => (
{intl.formatMessage(label)}
-
- {value === null ? '—' : value.toLocaleString()} +
+ {value}
))}
-

- {intl.formatMessage(messages.stages)} -

-
- {Object.entries(diagnostics.latestAttempt.stages).map( - ([stage, status]) => ( -
-
- {diagnosticLabel(stage)} -
-
- {diagnosticLabel(status)} -
+ {!diagnostics?.latestAttempt ? ( +

+ {intl.formatMessage(messages.noDiagnostics)} +

+ ) : ( +
+ + Detailed diagnostics + +
+
+ {countRows.map(([label, value]) => ( +
+
+ {intl.formatMessage(label)} +
+
+ {value === null ? '—' : value.toLocaleString()} +
+
+ ))}
- ) - )} -
- {diagnostics.latestAttempt.decisions.length > 0 && ( - <> -

- {intl.formatMessage(messages.decisions)} -

-
- - - - - - - - - - - - - - {diagnostics.latestAttempt.decisions.map( - (decision, index) => ( - - - - - - - - - - ) - )} - -
{intl.formatMessage(messages.mediaTitle)}{intl.formatMessage(messages.type)}{intl.formatMessage(messages.year)}{intl.formatMessage(messages.gate)}{intl.formatMessage(messages.outcome)}{intl.formatMessage(messages.reason)}{intl.formatMessage(messages.matched)}
{decision.title}{decision.mediaType}{decision.year ?? '—'}{diagnosticLabel(decision.stage)}{diagnosticLabel(decision.outcome)}{diagnosticLabel(decision.reason)} - {decision.tmdbId - ? `${decision.mediaType}:${decision.tmdbId}` - : '—'} -
+

+ {intl.formatMessage(messages.stages)} +

+
+ {Object.entries(diagnostics.latestAttempt.stages).map( + ([stage, status]) => ( +
+
+ {diagnosticLabel(stage)} +
+
+ {diagnosticLabel(status)} +
+
+ ) + )} +
+ {diagnostics.latestAttempt.decisions.length > 0 && ( + <> +

+ {intl.formatMessage(messages.decisions)} +

+
+ + + + + + + + + + + + + + {diagnostics.latestAttempt.decisions.map( + (decision, index) => ( + + + + + + + + + + ) + )} + +
{intl.formatMessage(messages.mediaTitle)}{intl.formatMessage(messages.type)}{intl.formatMessage(messages.year)}{intl.formatMessage(messages.gate)}{intl.formatMessage(messages.outcome)}{intl.formatMessage(messages.reason)}{intl.formatMessage(messages.matched)}
{decision.title}{decision.mediaType}{decision.year ?? '—'}{diagnosticLabel(decision.stage)}{diagnosticLabel(decision.outcome)}{diagnosticLabel(decision.reason)} + {decision.tmdbId + ? `${decision.mediaType}:${decision.tmdbId}` + : '—'} +
+
+ + )}
- +
)} - - )} +
+
- ); }; diff --git a/src/components/Settings/SettingsFresh/settingsFresh.test.ts b/src/components/Settings/SettingsFresh/settingsFresh.test.ts index 150399001e..c2a7faa853 100644 --- a/src/components/Settings/SettingsFresh/settingsFresh.test.ts +++ b/src/components/Settings/SettingsFresh/settingsFresh.test.ts @@ -10,7 +10,9 @@ import MediaTypeFilter from '../../Common/MediaTypeFilter'; import { CONFIGURED_TOKEN_MASK, composeAutobrrBaseUrl, + defaultFreshSectionState, loadFreshFilters, + parseFreshSectionState, selectedFreshFilter, splitAutobrrBaseUrl, toFreshFilterSelectOptions, @@ -143,7 +145,7 @@ describe('Fresh settings client boundary', () => { assert.match(source, /setFieldValue\('filterId', option\?\.value \?\? 0\)/); assert.match( source, - /setFieldValue\('cachedFilterName', option\?\.label \?\? ''\)/ + /setFieldValue\(\s*'cachedFilterName',\s*option\?\.label \?\? ''\s*\)/ ); }); @@ -193,7 +195,17 @@ describe('Fresh settings client boundary', () => { assert.match(source, /value: 'tv'/); assert.match(source, /status/); assert.match(source, /sort/); + assert.match(source, /reasonFamily/); + assert.match(source, /seasonEvidence/); + assert.match(source, /manualResolution/); + assert.match(source, /admissionOverride/); assert.match(source, /Needs Attention/); + assert.match(source, /Reviewable/); + assert.match(source, /Historical/); + assert.match( + source, + /useState\('priority'\)/ + ); assert.match( source, /setStatus\(cardStatus as FreshCandidateDiagnosticStatus\)/ @@ -215,11 +227,48 @@ describe('Fresh settings client boundary', () => { assert.ok( source.indexOf("['First observed'") < source.indexOf('const CandidateRow') ); - assert.match(source, /candidate\.actionable/); + assert.match(source, /candidate\.actions\.resolve/); + assert.match(source, /candidate\.actions\.resetResolution/); + assert.match(source, /candidate\.actions\.admit/); + assert.match(source, /candidate\.actions\.removeOverride/); + assert.match(source, /expectedRevision: candidate\.revision/); + assert.match(source, /error\.response\?\.status === 409/); + assert.match(source, /mediaType,/); assert.match(source, /\/resolve`/); assert.doesNotMatch(source, /latestAttempt/); }); + it('uses versioned, independent first-visit disclosure state safely', () => { + assert.deepEqual(defaultFreshSectionState, { + candidates: true, + pipeline: false, + configuration: false, + }); + assert.deepEqual(parseFreshSectionState(null), defaultFreshSectionState); + assert.deepEqual( + parseFreshSectionState('{not json'), + defaultFreshSectionState + ); + assert.deepEqual( + parseFreshSectionState( + JSON.stringify({ + candidates: true, + pipeline: true, + configuration: true, + }) + ), + { candidates: true, pipeline: true, configuration: true } + ); + const source = readFileSync(path.join(__dirname, 'index.tsx'), 'utf8'); + assert.match(source, /FRESH_SECTION_STATE_KEY/); + assert.match(source, /aria-expanded={open}/); + assert.match(source, /aria-controls={contentId}/); + assert.match(source, /focus:ring-2 focus:ring-indigo-500/); + assert.match(source, /id="candidates"[\s\S]*?order=\{1\}/); + assert.match(source, /id="pipeline"[\s\S]*?order=\{2\}/); + assert.match(source, /id="configuration"[\s\S]*?order=\{3\}/); + }); + it('renders the native media icon segment adjacent to the media-type select', () => { const markup = renderToStaticMarkup( createElement(MediaTypeFilter, { @@ -245,8 +294,9 @@ describe('Fresh settings client boundary', () => { it('uses native destructive confirmation for a completed Fresh rebuild', () => { const source = readFileSync(path.join(__dirname, 'index.tsx'), 'utf8'); assert.match(source, /rebuildTitle: 'Rebuild Fresh Data\?'/); - assert.match(source, /persisted Fresh discovery history/); - assert.match(source, /Watchlist, requests, media-library state/); + assert.match(source, /Irreversible Fresh history/); + assert.match(source, /typed manual resolutions/); + assert.match(source, /admission overrides are preserved/); assert.match(source, /okButtonType="danger"/); assert.match(source, /\/api\/v1\/settings\/fresh\/rebuild/); assert.match(source, /await axios\.post/); diff --git a/src/components/Settings/SettingsFresh/settingsFresh.ts b/src/components/Settings/SettingsFresh/settingsFresh.ts index 4237f3f984..ad36942bec 100644 --- a/src/components/Settings/SettingsFresh/settingsFresh.ts +++ b/src/components/Settings/SettingsFresh/settingsFresh.ts @@ -66,6 +66,38 @@ export interface FreshFilterSelectOption { label: string; } +export const FRESH_SECTION_STATE_KEY = 'seerr:fresh-settings-sections:v1'; +export type FreshSectionId = 'candidates' | 'pipeline' | 'configuration'; +export type FreshSectionState = Record; +export const defaultFreshSectionState: FreshSectionState = { + candidates: true, + pipeline: false, + configuration: false, +}; + +export const parseFreshSectionState = ( + value: string | null +): FreshSectionState => { + if (!value) return { ...defaultFreshSectionState }; + try { + const parsed = JSON.parse(value) as Partial; + if ( + typeof parsed.candidates === 'boolean' && + typeof parsed.pipeline === 'boolean' && + typeof parsed.configuration === 'boolean' + ) { + return { + candidates: parsed.candidates, + pipeline: parsed.pipeline, + configuration: parsed.configuration, + }; + } + } catch { + // Stale or malformed client preferences safely use first-visit defaults. + } + return { ...defaultFreshSectionState }; +}; + const DEFAULT_AUTOBRR_PORT = 7474; export const CONFIGURED_TOKEN_MASK = '•'.repeat(28); diff --git a/src/i18n/locale/en.json b/src/i18n/locale/en.json index 65148361d3..47421ae110 100644 --- a/src/i18n/locale/en.json +++ b/src/i18n/locale/en.json @@ -1006,10 +1006,10 @@ "components.Settings.SettingsFresh.latestAttempt": "Latest refresh attempt", "components.Settings.SettingsFresh.loadingFilter": "Loading configured filter…", "components.Settings.SettingsFresh.maintenance": "Maintenance", - "components.Settings.SettingsFresh.maintenanceHelp": "Rebuild persistent Fresh discovery data only when a clean source reconstruction is required.", + "components.Settings.SettingsFresh.maintenanceHelp": "Rebuild source-derived observations and automatic candidates only when a clean source reconstruction is required.", "components.Settings.SettingsFresh.matched": "Matched media", "components.Settings.SettingsFresh.mediaEligibilityDays": "Media eligibility window (days)", - "components.Settings.SettingsFresh.mediaEligibilityHelp": "At the first qualifying observation, how old the canonical movie release or series first-air date may be.", + "components.Settings.SettingsFresh.mediaEligibilityHelp": "Normal Movie eligibility window and TV recent-season activity sanity window. Movies also receive a fixed 14-day first-observation grace.", "components.Settings.SettingsFresh.mediaTitle": "Title", "components.Settings.SettingsFresh.newMedia": "New Fresh media", "components.Settings.SettingsFresh.newReleases": "New autobrr releases", @@ -1022,7 +1022,7 @@ "components.Settings.SettingsFresh.port": "Port", "components.Settings.SettingsFresh.reason": "Reason", "components.Settings.SettingsFresh.rebuild": "Rebuild Fresh Data", - "components.Settings.SettingsFresh.rebuildConfirm": "This deletes all persisted Fresh discovery history and rebuilds it from releases currently retained by autobrr. Fresh observations no longer present in autobrr Release History cannot be recovered. Watchlist, requests, media-library state, Radarr, Sonarr, and autobrr are unaffected.", + "components.Settings.SettingsFresh.rebuildConfirm": "This clears source-derived observations, automatic candidates, and synchronization state, then rebuilds from releases currently retained by autobrr. Irreversible Fresh history, canonical media metadata, typed manual resolutions, and admission overrides are preserved. Releases no longer retained by autobrr cannot be recovered.", "components.Settings.SettingsFresh.rebuildFailed": "Fresh data could not be rebuilt. Check the pipeline status for the failed stage.", "components.Settings.SettingsFresh.rebuildSuccess": "Fresh data was rebuilt successfully.", "components.Settings.SettingsFresh.rebuildTitle": "Rebuild Fresh Data?",