diff --git a/docs/contact-attribution.md b/docs/contact-attribution.md index 41138363..de2653cb 100644 --- a/docs/contact-attribution.md +++ b/docs/contact-attribution.md @@ -1,93 +1,92 @@ -# Contact import attribution - -Contacts retain separate records of creation, submissions, declared import sources, -provider ingestion and subsequent contact or relationship edits. Relationship owner -remains the person responsible for the work. Changing ownership does not rewrite -who submitted the contact. - -For example, if one member uploads another member's list, the authenticated uploader -is the submitter and the other member is an explicitly **declared source**. The source -declaration is not proof of original authorship. When a member's assistant submits -contacts through verified OAuth MCP, the member, transport, verified client ID and -grant ID come from authentication. Caller-supplied actor claims cannot override them. - -Open **Details → Import attribution** in a contact record or inspector to read the original -recorded creator visible in the selected product, most recent recorded editor and -paginated contribution history. -**Record import source** opens an inline form for a reviewed declaration on a -known contact. People -filters distinguish **Submitted by**, **Declared source**, coverage and multiple -contributors independently of current ownership. Multiple contributors means more -than one distinct recorded actor or declared source across visible submissions. -Edits do not count as imports. Filters respect the selected product. Archived -contact records retain readable attribution, with new declarations disabled. - -Existing contacts without recorded provenance remain **Unknown**. The migrations -add tables and constraints without rewriting people, relationships, notes, owners -or versions. Historical owner and creation timestamps are insufficient to infer an -importer. Deactivated members retain their recorded names; they lose permission to -submit new data. Names come from the stored member profile; immutable member IDs -retain identity if a profile is renamed. This is recorded history from this feature -onward, not a complete reconstruction of older activity. - -## MCP and HTTP import workflow - -1. Call `create_contact_import_batch` with `organizationId`, `productId`, a readable - `label`, `sourceKind` (`file`, `agent` or `manual`), and a stable `submissionKey`. - Optionally pass `sourceMemberId` for a current member with access to the product. -2. Create each new contact with `create_person` and - `importSource: {batchId, sourceRecordId}`. Keep each source row ID stable. Exact - retries return the same contact and relationship; changed payloads for an already - recorded source row return `IMPORT_SOURCE_CONFLICT`. -3. For a confirmed existing contact already related to the product, call - `record_contact_import` with its current `version`, `batchId` and `sourceRecordId`. - This appends provenance without replacing its original creator or owner. Exact - retries remain safe after later edits. Do not automatically merge contacts by name. -4. Inspect `get_contact_attribution` with a person ID and optional product ID. It - returns `creator`, `lastEditor`, `items`, `nextOffset` and coverage. Pass the next - offset to read more history. `list_records` for people supports `submittedBy`, - `sourceMemberId`, and `attribution: recorded | unknown | shared`. - -The shared operation registry maps these to `/api/crm`: POST `import-batch`, POST -`contact-import`, GET `contact-attribution`. Both transports enforce the same -membership, grants, product, version and retry rules. A batch belongs to its -authenticated submitter; another member cannot reuse it. Stable keys identify -retries, not authorization. The source form retains retry keys for unchanged -declarations after a partial failure. Correcting the batch label, kind or declared source creates a new batch; -existing batches remain immutable. No general CSV parser or new bulk upload UI is -included; agents can submit parsed rows using these operations. - -## Provider sources and privacy - -Materialized provider messages and meetings record the source connection owner and -provider. Background sync records a system event rather than attributing each event -to a human submitter. An explicit link records the authenticated linking actor, -including MCP identity when applicable. Duplicate provider receipts add no duplicate -contribution. Pending unmatched items become contributions when linked to a contact. - -Provider source details and their filter identifiers remain private to the account -owner, even when a conversation has separately been shared. Attribution records -contain no message body, credential or external account handle. Product-shared -manual declarations are visible to authorized product members. - -This feature does not change sequence approval, provider consent or sending gates. -Sender and approver remain recorded by existing outreach and delivery operations; -this contact panel is not a unified outbound activity log. Attribution is append-only -through the business API; database administration remains able to alter stored data. - -## Migration and validation - -Apply generated migrations `0023` and `0024` before running this application version. -They introduce import batches, contributions and same-tenant/product constraints. -Production rollout still requires reviewing the target and backup policy. Tests cover -populated upgrades, retry collisions, actor spoofing, shared submissions, revoked -membership, private accounts, product isolation, MCP provider links and the UI flow. - -## Fictional demo screenshots - -The local review uses fictional members and contacts. It shows a declared source -without replacing an unknown historical creator, plus the matching People filter. - -![Contact history](assets/screenshots/contact-attribution-history.jpg) - -![Declared-source filter](assets/screenshots/contact-attribution-filters.jpg) +# Contact import attribution + +Contacts retain separate records of creation, submissions, declared import sources, +provider ingestion and subsequent contact or relationship edits. Relationship owner +remains the person responsible for the work. Changing ownership does not rewrite +who submitted the contact. + +For example, if one member uploads another member's list, the authenticated uploader +is the submitter and the other member is an explicitly **declared source**. The source +declaration is not proof of original authorship. When a member's assistant submits +contacts through verified OAuth MCP, the member, transport, verified client ID and +grant ID come from authentication. Caller-supplied actor claims cannot override them. + +Open **Details → Import attribution** in a contact record or inspector to read the original +recorded creator visible in the selected product, most recent recorded editor and +paginated contribution history. +**Record import source** opens an inline form for a reviewed declaration on a +known contact. People +filters distinguish **Submitted by**, **Declared source**, coverage and multiple +contributors independently of current ownership. Multiple contributors means more +than one distinct recorded actor or declared source across visible submissions. +Edits do not count as imports. Filters respect the selected product. Archived +contact records retain readable attribution, with new declarations disabled. + +Existing contacts without recorded provenance remain **Unknown**. The migrations +add tables and constraints without rewriting people, relationships, notes, owners +or versions. Historical owner and creation timestamps are insufficient to infer an +importer. Deactivated members retain their recorded names; they lose permission to +submit new data. Names come from the stored member profile; immutable member IDs +retain identity if a profile is renamed. This is recorded history from this feature +onward, not a complete reconstruction of older activity. + +## MCP and HTTP import workflow + +1. Call `create_contact_import_batch` with `organizationId`, `productId`, a readable + `label`, `sourceKind` (`file`, `agent` or `manual`), and a stable `submissionKey`. + Optionally pass `sourceMemberId` for a current member with access to the product. +2. Create each new contact with `create_person` and + `importSource: {batchId, sourceRecordId}`. Keep each source row ID stable. Exact + retries return the same contact and relationship; changed payloads for an already + recorded source row return `IMPORT_SOURCE_CONFLICT`. +3. For a confirmed existing contact already related to the product, call + `record_contact_import` with its current `version`, `batchId` and `sourceRecordId`. + This appends provenance without replacing its original creator or owner. Exact + retries remain safe after later edits. Do not automatically merge contacts by name. +4. Inspect `get_contact_attribution` with a person ID and optional product ID. It + returns `creator`, `lastEditor`, `items`, `nextOffset` and coverage. Pass the next + offset to read more history. `list_records` for people supports `submittedBy`, + `sourceMemberId`, and `attribution: recorded | unknown | shared`. + +The shared operation registry maps these to `/api/crm`: POST `import-batch`, POST +`contact-import`, GET `contact-attribution`. Both transports enforce the same +membership, grants, product, version and retry rules. A batch belongs to its +authenticated submitter; another member cannot reuse it. Stable keys identify +retries, not authorization. The source form retains retry keys for unchanged +declarations after a partial failure. Correcting the batch label, kind or declared source creates a new batch; +existing batches remain immutable. Reviewed CSV imports with preview, column mapping, and duplicate review are documented in [CSV imports](csv-imports.md); agents can also submit parsed rows using these operations. + +## Provider sources and privacy + +Materialized provider messages and meetings record the source connection owner and +provider. Background sync records a system event rather than attributing each event +to a human submitter. An explicit link records the authenticated linking actor, +including MCP identity when applicable. Duplicate provider receipts add no duplicate +contribution. Pending unmatched items become contributions when linked to a contact. + +Provider source details and their filter identifiers remain private to the account +owner, even when a conversation has separately been shared. Attribution records +contain no message body, credential or external account handle. Product-shared +manual declarations are visible to authorized product members. + +This feature does not change sequence approval, provider consent or sending gates. +Sender and approver remain recorded by existing outreach and delivery operations; +this contact panel is not a unified outbound activity log. Attribution is append-only +through the business API; database administration remains able to alter stored data. + +## Migration and validation + +Apply generated migrations `0023` and `0024` before running this application version. +They introduce import batches, contributions and same-tenant/product constraints. +Production rollout still requires reviewing the target and backup policy. Tests cover +populated upgrades, retry collisions, actor spoofing, shared submissions, revoked +membership, private accounts, product isolation, MCP provider links and the UI flow. + +## Fictional demo screenshots + +The local review uses fictional members and contacts. It shows a declared source +without replacing an unknown historical creator, plus the matching People filter. + +![Contact history](assets/screenshots/contact-attribution-history.jpg) + +![Declared-source filter](assets/screenshots/contact-attribution-filters.jpg) diff --git a/docs/csv-imports.md b/docs/csv-imports.md new file mode 100644 index 00000000..6b8ab145 --- /dev/null +++ b/docs/csv-imports.md @@ -0,0 +1,44 @@ +# Reviewed CSV Imports + +Gravity supports reviewed CSV imports for contacts and companies with column mapping, duplicate detection, and final summaries, matching Slice 2 of the roadmap. + +## Overview & Policy + +- **No silent merges**: Rows whose person email or company domain already exists within the organization (or duplicates within the same file) are highlighted for review. They are never silently overwritten or merged. Attempting an import without skipping duplicates explicitly rejects with `PERSON_EXISTS` or `COMPANY_EXISTS`. +- **Product relationship**: Every imported contact must be bound to a selected product with a defined purpose (`buyer` or `partner`). +- **Attribution & Provenance**: Imports create a batch in `contact_import_batches` with source kind `file` (or `agent` via MCP) and record submission attribution. +- **Atomic transactions**: Imports run inside a database transaction with organization locks. If `skipDuplicates` is false and a duplicate is encountered, the entire batch rolls back. + +## UI Workflow + +1. Navigate to **People** (`/people`) or **Companies** (`/companies`). +2. Click **Import CSV** in the view toolbar, or use the command menu (Ctrl+K / ⌘+K) $\rightarrow$ **Import contacts and companies from CSV**. +3. Select the target **Product relationship** and **Relationship type** (Buyer or Partner). +4. Upload or drop a `.csv` file: + - Headers are parsed in the client using RFC 4180 parsing. + - Column mappings are auto-detected (e.g. `Name`, `Email`, `Title`, `Phone`, `LinkedIn`, `Company`, `Domain`, `Summary / Notes`). You can adjust mappings or set unwanted columns to "Do not import". +5. Click **Preview import**: + - The preview runs `preview_csv_import` against the server without writing any database records. + - Summarizes total rows, valid rows, duplicates for review, and invalid rows. + - Review duplicate conflicts: displays the existing record name, email/domain, and reason why it cannot be merged silently. + - Review validation errors (e.g. missing required person name, malformed emails or domains). +6. Click **Import valid contacts**: + - Executes `import_contacts_csv` with `skipDuplicates: true`. + - Displays the **Import summary** card with counts of people created, companies created, and duplicates skipped. + +## MCP & Operations Catalog + +Both browser clients and AI assistants execute the exact same reviewed flow via `packages/operations/catalog.ts`: + +- **`preview_csv_import`** (POST `/api/crm`, operation `csv-import-preview`): + - Parameters: `organizationId`, `productId`, `csvText`, `columnMapping` (optional), `purpose` (`buyer` | `partner`). + - Returns: summary counts, detected mapping, valid row previews, duplicate rows with collision reasons, and invalid rows with error messages. +- **`import_contacts_csv`** (POST `/api/crm`, operation `csv-import`): + - Parameters: `organizationId`, `productId`, `csvText`, `columnMapping` (optional), `purpose`, `skipDuplicates` (`boolean`), `rowIndices` (optional), `label` (optional). + - Returns: summary with `importedCount`, `peopleCreated`, `companiesCreated`, `relationshipsCreated`, `duplicateCount`, and `invalidCount`. + +## Fictional Fixture & Test Suite + +The test suite exercises several hundred rows from `tests/fixtures/fictional-contacts.csv` containing exclusively fictional data: +- `tests/csv-import.test.ts`: Validates RFC 4180 parsing, column detection, 271-row preview, duplicate collision detection, rejection on duplicates, atomic transactional rollback, provenance attribution recording, product permissions, and cross-organization isolation. +- `tests/csv-import-ui.test.tsx`: Exercises toolbar launch, file selection, column mapping, preview tabs, duplicate review, and import completion in jsdom. diff --git a/packages/core/csv-import.ts b/packages/core/csv-import.ts new file mode 100644 index 00000000..2e01609e --- /dev/null +++ b/packages/core/csv-import.ts @@ -0,0 +1,599 @@ +import { randomUUID } from "node:crypto"; +import { and, asc, eq, isNull } from "drizzle-orm"; +import { z } from "zod"; +import type { Database } from "../database/client"; +import * as s from "../database/schema"; +import { recordContactSubmission } from "./contact-attribution"; +import { authorize, DomainError, type Principal } from "./policy"; +import { assertProductActive } from "./products"; +import { companyDomain, linkedinUrl } from "./records"; +import { lockOrganization } from "./visibility"; + +export * from "./csv"; + +import { detectColumnMapping, parseCsv } from "./csv"; + +export const csvImportPreviewSchema = z.object({ + organizationId: z.uuid(), + productId: z.uuid(), + csvText: z.string().min(1).max(5000000), + columnMapping: z.record(z.string(), z.string()).optional(), + purpose: z.enum(["buyer", "partner"]).default("buyer"), +}); + +export const csvImportSchema = z.object({ + organizationId: z.uuid(), + productId: z.uuid(), + csvText: z.string().min(1).max(5000000), + columnMapping: z.record(z.string(), z.string()).optional(), + purpose: z.enum(["buyer", "partner"]).default("buyer"), + skipDuplicates: z.boolean().default(false), + rowIndices: z.array(z.number().int().positive()).optional(), + label: z.string().trim().min(1).max(150).default("CSV Import"), +}); + +export interface ValidatedPerson { + name: string; + email?: string; + title: string; + phone: string; + linkedinUrl: string; + summary: string; +} + +export interface ValidatedCompany { + name?: string; + domain?: string; + description: string; +} + +export interface ValidatedRow { + rowIndex: number; + person: ValidatedPerson; + company?: ValidatedCompany; +} + +export interface DuplicateReason { + kind: "person_email" | "company_domain" | "csv_duplicate"; + message: string; + existingId?: string; + existingName?: string; +} + +export interface DuplicateRow { + rowIndex: number; + person: ValidatedPerson; + company?: ValidatedCompany; + reasons: DuplicateReason[]; +} + +export interface InvalidRow { + rowIndex: number; + row: Record; + errors: string[]; +} + +export interface PreviewResult { + summary: { + totalRows: number; + validCount: number; + duplicateCount: number; + invalidCount: number; + headers: string[]; + detectedMapping: Record; + }; + validRows: ValidatedRow[]; + duplicateRows: DuplicateRow[]; + invalidRows: InvalidRow[]; +} + +export interface ImportResult { + totalRows: number; + importedCount: number; + peopleCreated: number; + companiesCreated: number; + relationshipsCreated: number; + duplicateCount: number; + invalidCount: number; + imported: Array<{ + rowIndex: number; + personId: string; + companyId?: string; + relationshipId: string; + }>; + duplicates: DuplicateRow[]; + invalid: InvalidRow[]; +} + +const phonePattern = /^[+\d\s().-]*$/; + +type Transaction = Parameters[0]>[0]; +type Reader = Database | Transaction; + +export class CsvImportService { + constructor(private db: Database) {} + + private prepareRows( + headers: string[], + rawRows: string[][], + columnMapping?: Record, + ) { + const mapping: Record = { + ...detectColumnMapping(headers), + ...(columnMapping ?? {}), + }; + + const parsedRows: Array<{ + rowIndex: number; + raw: Record; + values: Record; + }> = []; + + for (let i = 0; i < rawRows.length; i++) { + const row = rawRows[i]; + const raw: Record = {}; + const values: Record = {}; + + for (let c = 0; c < headers.length; c++) { + const header = headers[c]; + const val = row[c] ?? ""; + raw[header] = val; + const target = mapping[header]; + if (target && target !== "ignore") { + values[target] = val.trim(); + } + } + parsedRows.push({ rowIndex: i + 1, raw, values }); + } + + return { mapping, parsedRows }; + } + + /** + * Preview a CSV import without writing changes. + * Identifies valid rows, invalid rows, and rows that collide with existing people/companies + * or duplicate other rows in the file for duplicate review. + */ + async preview( + principal: Principal, + rawInput: z.input, + executor: Reader = this.db, + ): Promise { + const input = csvImportPreviewSchema.parse(rawInput); + await authorize(executor, principal, input.organizationId); + await assertProductActive(executor, input.organizationId, input.productId); + if ( + principal.productIds && + !principal.productIds.includes(input.productId) + ) { + throw new DomainError("FORBIDDEN", 403); + } + + const { headers, rows: rawRows } = parseCsv(input.csvText); + const { mapping, parsedRows } = this.prepareRows( + headers, + rawRows, + input.columnMapping, + ); + + const [existingPeople, existingCompanies] = await Promise.all([ + executor + .select({ id: s.people.id, name: s.people.name, email: s.people.email }) + .from(s.people) + .where( + and( + eq(s.people.organizationId, input.organizationId), + isNull(s.people.archivedAt), + ), + ), + executor + .select({ + id: s.companies.id, + name: s.companies.name, + domain: s.companies.domain, + }) + .from(s.companies) + .where( + and( + eq(s.companies.organizationId, input.organizationId), + isNull(s.companies.archivedAt), + ), + ), + ]); + + const peopleByEmail = new Map(); + for (const p of existingPeople) { + if (p.email) peopleByEmail.set(p.email.toLowerCase(), p); + } + + const companiesByDomain = new Map(); + for (const c of existingCompanies) { + if (c.domain) companiesByDomain.set(c.domain.toLowerCase(), c); + } + + const validRows: ValidatedRow[] = []; + const duplicateRows: DuplicateRow[] = []; + const invalidRows: InvalidRow[] = []; + const seenEmailsInCsv = new Set(); + + const emailCounts = new Map(); + for (const { values } of parsedRows) { + if (values.email) { + const parsedEmail = z + .string() + .trim() + .toLowerCase() + .pipe(z.email().max(254)) + .safeParse(values.email); + if (parsedEmail.success) { + const e = parsedEmail.data; + emailCounts.set(e, (emailCounts.get(e) ?? 0) + 1); + } + } + } + + for (const { rowIndex, raw, values } of parsedRows) { + const errors: string[] = []; + + const rawName = values.name ?? ""; + if (!rawName) { + errors.push("Person name is required"); + } else if (rawName.length > 100) { + errors.push("Person name must be 100 characters or fewer"); + } + + let email: string | undefined; + if (values.email) { + const parsedEmail = z + .string() + .trim() + .toLowerCase() + .pipe(z.email().max(254)) + .safeParse(values.email); + if (!parsedEmail.success) { + errors.push(`Invalid email address: ${values.email}`); + } else { + email = parsedEmail.data; + } + } + + let compDomain: string | undefined; + if (values.companyDomain) { + const parsedDomain = companyDomain.safeParse(values.companyDomain); + if (!parsedDomain.success || !parsedDomain.data) { + errors.push(`Invalid company domain: ${values.companyDomain}`); + } else { + compDomain = parsedDomain.data; + } + } + + let linkedIn: string = ""; + if (values.linkedinUrl) { + const parsedLinkedin = linkedinUrl.safeParse(values.linkedinUrl); + if (!parsedLinkedin.success) { + errors.push(`Invalid LinkedIn URL: ${values.linkedinUrl}`); + } else { + linkedIn = parsedLinkedin.data; + } + } + + const phone = values.phone ?? ""; + if (phone && (!phonePattern.test(phone) || phone.length > 40)) { + errors.push(`Invalid phone number: ${phone}`); + } + + const compName = values.companyName + ? values.companyName.slice(0, 100) + : undefined; + const compDesc = values.companyDescription + ? values.companyDescription.slice(0, 2000) + : ""; + const title = values.title ? values.title.slice(0, 150) : ""; + const summary = values.summary ? values.summary.slice(0, 10000) : ""; + + if (errors.length > 0) { + invalidRows.push({ rowIndex, row: raw, errors }); + continue; + } + + const person: ValidatedPerson = { + name: rawName, + email, + title, + phone, + linkedinUrl: linkedIn, + summary, + }; + + const company: ValidatedCompany | undefined = + compName || compDomain || compDesc + ? { + name: compName, + domain: compDomain, + description: compDesc, + } + : undefined; + + const reasons: DuplicateReason[] = []; + + if (email) { + const existingPerson = peopleByEmail.get(email); + if (existingPerson) { + reasons.push({ + kind: "person_email", + message: `Email "${email}" already belongs to ${existingPerson.name} in this organization.`, + existingId: existingPerson.id, + existingName: existingPerson.name, + }); + } else if ((emailCounts.get(email) ?? 0) > 1) { + reasons.push({ + kind: "csv_duplicate", + message: `Email "${email}" appears multiple times in this CSV.`, + }); + } + } + + if (compDomain) { + const existingComp = companiesByDomain.get(compDomain); + if (existingComp) { + reasons.push({ + kind: "company_domain", + message: `Company domain "${compDomain}" already belongs to ${existingComp.name} in this organization.`, + existingId: existingComp.id, + existingName: existingComp.name, + }); + } + } + + if (reasons.length > 0) { + duplicateRows.push({ rowIndex, person, company, reasons }); + if (email) seenEmailsInCsv.add(email); + } else { + validRows.push({ rowIndex, person, company }); + if (email) seenEmailsInCsv.add(email); + } + } + + return { + summary: { + totalRows: parsedRows.length, + validCount: validRows.length, + duplicateCount: duplicateRows.length, + invalidCount: invalidRows.length, + headers, + detectedMapping: mapping, + }, + validRows, + duplicateRows, + invalidRows, + }; + } + + /** + * Execute CSV import. + * If skipDuplicates is false and duplicates exist, rejects with PERSON_EXISTS / COMPANY_EXISTS + * so records are never silently merged. + * If skipDuplicates is true, imports valid non-duplicate rows while recording provenance. + */ + async importCsv( + principal: Principal, + rawInput: z.input, + ): Promise { + const input = csvImportSchema.parse(rawInput); + await authorize(this.db, principal, input.organizationId); + await assertProductActive(this.db, input.organizationId, input.productId); + if ( + principal.productIds && + !principal.productIds.includes(input.productId) + ) { + throw new DomainError("FORBIDDEN", 403); + } + + return this.db.transaction(async (tx) => { + await lockOrganization(tx, input.organizationId); + + const preview = await this.preview( + principal, + { + organizationId: input.organizationId, + productId: input.productId, + csvText: input.csvText, + columnMapping: input.columnMapping, + purpose: input.purpose, + }, + tx, + ); + + const allowedIndices = input.rowIndices + ? new Set(input.rowIndices) + : null; + + const candidateValid = allowedIndices + ? preview.validRows.filter((r) => allowedIndices.has(r.rowIndex)) + : preview.validRows; + const candidateDuplicates = allowedIndices + ? preview.duplicateRows.filter((r) => allowedIndices.has(r.rowIndex)) + : preview.duplicateRows; + const candidateInvalid = allowedIndices + ? preview.invalidRows.filter((r) => allowedIndices.has(r.rowIndex)) + : preview.invalidRows; + + if (!input.skipDuplicates && candidateDuplicates.length > 0) { + const firstReason = candidateDuplicates[0].reasons[0]; + if (firstReason?.kind === "company_domain") { + throw new DomainError("COMPANY_EXISTS", 409); + } + throw new DomainError("PERSON_EXISTS", 409); + } + + if (candidateValid.length === 0) { + return { + totalRows: preview.summary.totalRows, + importedCount: 0, + peopleCreated: 0, + companiesCreated: 0, + relationshipsCreated: 0, + duplicateCount: candidateDuplicates.length, + invalidCount: candidateInvalid.length, + imported: [], + duplicates: candidateDuplicates, + invalid: candidateInvalid, + }; + } + + const [firstStage] = await tx + .select({ id: s.stages.id }) + .from(s.stages) + .where( + and( + eq(s.stages.organizationId, input.organizationId), + eq(s.stages.productId, input.productId), + eq(s.stages.pipeline, "outreach"), + eq(s.stages.category, "open"), + isNull(s.stages.archivedAt), + ), + ) + .orderBy(asc(s.stages.position)) + .limit(1); + + await tx + .insert(s.contactImportBatches) + .values({ + organizationId: input.organizationId, + productId: input.productId, + submittedBy: principal.userId, + sourceKind: "file", + label: input.label || "CSV Import", + submissionKey: `csv-import-${randomUUID()}`, + transport: principal.source === "mcp" ? "mcp" : "session", + }) + .returning(); + + const createdCompaniesByKey = new Map(); + let companiesCreatedCount = 0; + + for (const row of candidateValid) { + if (row.company && (row.company.domain || row.company.name)) { + const key = row.company.domain + ? `domain:${row.company.domain.toLowerCase()}` + : `name:${row.company.name?.toLowerCase()}`; + + if (!createdCompaniesByKey.has(key)) { + const companyName = + row.company.name || + (row.company.domain + ? row.company.domain.split(".")[0].toUpperCase() + : "Company"); + + const [createdComp] = await tx + .insert(s.companies) + .values({ + organizationId: input.organizationId, + name: companyName, + domain: row.company.domain || null, + description: row.company.description || "", + }) + .returning(); + + createdCompaniesByKey.set(key, createdComp.id); + companiesCreatedCount++; + + await tx.insert(s.changeEvents).values({ + organizationId: input.organizationId, + actorId: principal.userId, + type: "company.created", + entityId: createdComp.id, + }); + } + } + } + + const importedList: Array<{ + rowIndex: number; + personId: string; + companyId?: string; + relationshipId: string; + }> = []; + + for (const row of candidateValid) { + let companyId: string | null = null; + if (row.company && (row.company.domain || row.company.name)) { + const key = row.company.domain + ? `domain:${row.company.domain.toLowerCase()}` + : `name:${row.company.name?.toLowerCase()}`; + companyId = createdCompaniesByKey.get(key) ?? null; + } + + const [person] = await tx + .insert(s.people) + .values({ + organizationId: input.organizationId, + name: row.person.name, + title: row.person.title || "", + email: row.person.email || null, + phone: row.person.phone || "", + linkedinUrl: row.person.linkedinUrl || "", + summary: row.person.summary || "", + companyId, + }) + .returning(); + + const [rel] = await tx + .insert(s.relationships) + .values({ + organizationId: input.organizationId, + productId: input.productId, + personId: person.id, + ownerId: principal.userId, + purpose: input.purpose, + context: row.person.summary || "", + stageId: firstStage?.id ?? null, + }) + .returning(); + + await recordContactSubmission(tx, principal, { + organizationId: input.organizationId, + personId: person.id, + productId: input.productId, + kind: "created", + sourceRecordId: String(row.rowIndex), + }); + + await tx.insert(s.changeEvents).values([ + { + organizationId: input.organizationId, + actorId: principal.userId, + type: "person.created", + entityId: person.id, + }, + { + organizationId: input.organizationId, + actorId: principal.userId, + type: "relationship.created", + entityId: rel.id, + }, + ]); + + importedList.push({ + rowIndex: row.rowIndex, + personId: person.id, + companyId: companyId ?? undefined, + relationshipId: rel.id, + }); + } + + return { + totalRows: preview.summary.totalRows, + importedCount: importedList.length, + peopleCreated: importedList.length, + companiesCreated: companiesCreatedCount, + relationshipsCreated: importedList.length, + duplicateCount: candidateDuplicates.length, + invalidCount: candidateInvalid.length, + imported: importedList, + duplicates: candidateDuplicates, + invalid: candidateInvalid, + }; + }); + } +} diff --git a/packages/core/csv.ts b/packages/core/csv.ts new file mode 100644 index 00000000..3722fd71 --- /dev/null +++ b/packages/core/csv.ts @@ -0,0 +1,159 @@ +export const allowedFields = [ + "name", + "email", + "title", + "phone", + "linkedinUrl", + "summary", + "companyName", + "companyDomain", + "companyDescription", + "ignore", +] as const; + +export type ImportField = (typeof allowedFields)[number]; + +export interface ParsedCsv { + headers: string[]; + rows: string[][]; +} + +/** RFC 4180 compliant CSV parser with multiline & quote escaping support. */ +export function parseCsv(text: string): ParsedCsv { + const clean = text.replace(/^\uFEFF/, ""); + const lines: string[][] = []; + let currentRow: string[] = []; + let currentField = ""; + let inQuotes = false; + let i = 0; + + while (i < clean.length) { + const char = clean[i]; + if (inQuotes) { + if (char === '"') { + if (i + 1 < clean.length && clean[i + 1] === '"') { + currentField += '"'; + i += 2; + continue; + } + inQuotes = false; + i++; + continue; + } + currentField += char; + i++; + continue; + } + + if (char === '"') { + inQuotes = true; + i++; + continue; + } + if (char === ",") { + currentRow.push(currentField.trim()); + currentField = ""; + i++; + continue; + } + if (char === "\r" || char === "\n") { + if (char === "\r" && i + 1 < clean.length && clean[i + 1] === "\n") { + i++; + } + currentRow.push(currentField.trim()); + currentField = ""; + if (currentRow.some((field) => field.length > 0)) { + lines.push(currentRow); + } + currentRow = []; + i++; + continue; + } + currentField += char; + i++; + } + + if (currentField.length > 0 || currentRow.length > 0) { + currentRow.push(currentField.trim()); + if (currentRow.some((field) => field.length > 0)) { + lines.push(currentRow); + } + } + + if (lines.length === 0) { + return { headers: [], rows: [] }; + } + const headers = lines[0].map((h) => h.trim()); + const rows = lines.slice(1); + return { headers, rows }; +} + +/** Auto-detect column mapping by matching common English header variants. */ +export function detectColumnMapping( + headers: string[], +): Record { + const mapping: Record = {}; + for (const header of headers) { + const norm = header.toLowerCase().replace(/[^a-z0-9]/g, ""); + if ( + ["name", "fullname", "contactname", "personname", "contact"].includes( + norm, + ) + ) { + mapping[header] = "name"; + } else if ( + ["email", "emailaddress", "workemail", "contactemail", "mail"].includes( + norm, + ) + ) { + mapping[header] = "email"; + } else if ( + ["title", "jobtitle", "role", "position", "job"].includes(norm) + ) { + mapping[header] = "title"; + } else if ( + ["phone", "phonenumber", "telephone", "mobile", "cell"].includes(norm) + ) { + mapping[header] = "phone"; + } else if ( + ["linkedin", "linkedinurl", "profile", "linkedinprofile"].includes(norm) + ) { + mapping[header] = "linkedinUrl"; + } else if ( + [ + "company", + "companyname", + "organization", + "organisation", + "account", + "business", + ].includes(norm) + ) { + mapping[header] = "companyName"; + } else if ( + [ + "domain", + "companydomain", + "website", + "companywebsite", + "url", + "web", + ].includes(norm) + ) { + mapping[header] = "companyDomain"; + } else if ( + ["description", "companydescription", "about", "aboutcompany"].includes( + norm, + ) + ) { + mapping[header] = "companyDescription"; + } else if ( + ["summary", "notes", "context", "background", "comments"].includes(norm) + ) { + mapping[header] = "summary"; + } else { + mapping[header] = "ignore"; + } + } + return mapping; +} diff --git a/packages/core/products.ts b/packages/core/products.ts index e898697f..30500fa8 100644 --- a/packages/core/products.ts +++ b/packages/core/products.ts @@ -29,7 +29,7 @@ export const archiveProductSchema = productScope; export const restoreProductSchema = productScope; export async function assertProductActive( - tx: Transaction, + tx: Reader, organizationId: string, productId: string, ) { diff --git a/packages/i18n/translations/en.json b/packages/i18n/translations/en.json index a0c73c84..c2c65dfa 100644 --- a/packages/i18n/translations/en.json +++ b/packages/i18n/translations/en.json @@ -2452,5 +2452,55 @@ "edit": "Edit action reason", "original": "Original action reason", "hint": "Replace imported JSON with readable notes after reviewing the source. The original is kept, and editing clears any message approval." - } + }, + "importCsv": "Import CSV", + "importContacts": "Import contacts", + "csvImportTitle": "Import contacts and companies from CSV", + "csvImportDescription": "Upload a CSV, map columns to person and company fields, choose the product relationship to create, and preview the result before anything is written.", + "uploadCsv": "Upload CSV file", + "chooseFile": "Choose CSV file", + "dragCsvHere": "Drag and drop a CSV file here, or click to browse", + "csvFileSelected": "Selected file", + "changeFile": "Change file", + "mapColumns": "Map columns", + "csvColumn": "CSV column", + "mapsToField": "Target field", + "doNotImport": "Don't import", + "fieldPersonName": "Person name (required)", + "fieldEmail": "Email address", + "fieldTitle": "Job title", + "fieldPhone": "Phone number", + "fieldLinkedin": "LinkedIn profile", + "fieldSummary": "Notes / Summary", + "fieldCompanyName": "Company name", + "fieldCompanyDomain": "Company domain", + "fieldCompanyDescription": "Company description", + "previewCsvImport": "Preview import", + "readyToImport": "Ready to import", + "duplicatesForReview": "Duplicates for review", + "invalidRows": "Invalid rows", + "noDuplicatesFound": "No duplicates found. All rows are new.", + "noInvalidRows": "No invalid rows found.", + "importValidContacts": "Import valid contacts", + "importSummary": "Import summary", + "importedSuccess": "Successfully imported {count} contacts.", + "duplicateReviewNote": "These rows match existing records or appear multiple times in the file. Gravity does not silently merge records; review each duplicate below.", + "csvRequiredNameMissing": "At least one column must be mapped to Person name.", + "totalRows": "Total rows", + "validRowsCount": "Valid", + "duplicatesCount": "Duplicates", + "invalidCount": "Invalid", + "rowIndex": "Row", + "importReason": "Reason", + "importedPeople": "People created", + "importedCompanies": "Companies created", + "skippedDuplicates": "Duplicates skipped", + "purposeBuyer": "Buyer", + "purposePartner": "Partner", + "selectProduct": "Product relationship", + "relationshipPurpose": "Relationship type", + "selectProductPrompt": "Select a product", + "selectFilePrompt": "Select a CSV file to preview", + "title": "Title", + "back": "Back" } diff --git a/packages/operations/catalog.ts b/packages/operations/catalog.ts index 954e5de7..2537bfde 100644 --- a/packages/operations/catalog.ts +++ b/packages/operations/catalog.ts @@ -61,6 +61,11 @@ import { scopeSchema, workspaceSchema, } from "../core/crm"; +import { + CsvImportService, + csvImportPreviewSchema, + csvImportSchema, +} from "../core/csv-import"; import { changeInternalTaskSchema, createInternalTaskSchema, @@ -950,6 +955,31 @@ export const operations: Operation[] = [ run: (c, input) => new ContactAttributionService(c.db).list(c.principal, input), }), + operation({ + api: "crm", + method: "POST", + operation: "csv-import-preview", + name: "preview_csv_import", + idempotent: true, + destructive: false, + publish: false, + description: + "Preview a CSV import of people and companies. Detects column mappings, validates fields, and checks for existing duplicate emails or company domains for explicit review without silent merging.", + schema: csvImportPreviewSchema, + run: (c, input) => new CsvImportService(c.db).preview(c.principal, input), + }), + operation({ + api: "crm", + method: "POST", + operation: "csv-import", + name: "import_contacts_csv", + idempotent: false, + destructive: true, + description: + "Import people and companies from CSV with product relationships. Rows with duplicate emails or company domains must be explicitly reviewed; by default duplicates reject with PERSON_EXISTS or COMPANY_EXISTS rather than silently merging. Pass skipDuplicates to import valid rows while skipping reviewed duplicates.", + schema: csvImportSchema, + run: (c, input) => new CsvImportService(c.db).importCsv(c.principal, input), + }), operation({ api: "crm", method: "GET", diff --git a/src/app/api/crm/route.ts b/src/app/api/crm/route.ts index a94e2524..e4a98a55 100644 --- a/src/app/api/crm/route.ts +++ b/src/app/api/crm/route.ts @@ -23,7 +23,7 @@ export async function POST(request: Request) { assertMutationOrigin(request); const principal = await currentPrincipal(request.headers); const input = JSON.parse( - new TextDecoder().decode(await limitedBody(request, 100000)), + new TextDecoder().decode(await limitedBody(request, 2000000)), ); const operation = apiOperation("crm", "POST", input.operation); return Response.json( diff --git a/src/app/globals.css b/src/app/globals.css index 6b11d518..ac6daf03 100644 --- a/src/app/globals.css +++ b/src/app/globals.css @@ -6279,3 +6279,343 @@ nav.list-pagination > label:has(select) { flex: 1 1 200px; } } + +/* CSV Import Dialog */ +.import-dialog { + width: 720px; + max-width: min(720px, calc(100vw - 32px)); + padding: 24px; +} +.import-dialog .dialog-header { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 16px; + margin-bottom: 20px; +} +.import-dialog .dialog-header h2 { + font-size: var(--text-lg); + font-weight: 600; + margin: 0 0 6px; + color: var(--gravity-text-strong); +} +.import-dialog .dialog-header p { + margin: 0; + font-size: var(--text-sm); + color: var(--gravity-muted); + line-height: 1.45; +} +.import-dialog .dialog-header .icon-button { + flex-shrink: 0; + color: var(--gravity-muted); + border-radius: var(--gravity-radius); + padding: 4px; +} +.import-dialog .dialog-header .icon-button:hover { + color: var(--gravity-text); + background: var(--gravity-hover); +} +.import-settings-grid { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 16px; + margin-bottom: 20px; +} +@media (max-width: 600px) { + .import-settings-grid { + grid-template-columns: 1fr; + } +} +.import-dialog .field { + display: flex; + flex-direction: column; + gap: 6px; +} +.import-dialog .field > span { + font-size: var(--text-xs); + font-weight: 500; + color: var(--gravity-secondary); +} +.import-dialog .file-drop-zone { + display: flex; + flex-direction: column; + align-items: center; + justify-content: center; + width: 100%; + padding: 32px 24px; + border: 1.5px dashed var(--gravity-border); + border-radius: var(--gravity-radius-lg); + background: var(--gravity-surface-2); + transition: + border-color var(--duration-fast) var(--ease-standard), + background var(--duration-fast) var(--ease-standard); + cursor: pointer; + text-align: center; + margin-bottom: 20px; +} +.import-dialog button.file-drop-zone:hover, +.import-dialog button.file-drop-zone:focus-visible { + border-color: var(--gravity-accent); + background: var(--gravity-surface-3); + outline: none; +} +.import-dialog .file-drop-zone.has-file { + padding: 14px 18px; + border-style: solid; + border-color: var(--gravity-border); + background: var(--gravity-surface); + cursor: default; +} +.import-dialog .upload-prompt { + display: flex; + flex-direction: column; + align-items: center; + gap: 8px; + background: transparent; + border: none; + cursor: pointer; + width: 100%; +} +.import-dialog .upload-prompt .muted-icon { + color: var(--gravity-muted); + margin-bottom: 4px; +} +.import-dialog .upload-prompt p { + margin: 0; + font-size: var(--text-sm); + color: var(--gravity-text); +} +.import-dialog .upload-prompt p.muted { + color: var(--gravity-muted); + font-size: var(--text-xs); +} +.import-dialog .file-info { + display: flex; + align-items: center; + width: 100%; + gap: 14px; +} +.import-dialog .file-info .accent-icon { + color: var(--gravity-accent); + flex-shrink: 0; +} +.import-dialog .file-info > div { + display: flex; + flex-direction: column; + gap: 2px; + flex: 1; + min-width: 0; + text-align: left; +} +.import-dialog .file-info strong { + font-size: var(--text-sm); + color: var(--gravity-text-strong); + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.import-dialog .file-info .muted { + font-size: var(--text-xs); + color: var(--gravity-muted); +} +.import-dialog .mapping-section { + margin-bottom: 20px; +} +.import-dialog .mapping-section h3 { + font-size: var(--text-dense); + font-weight: 600; + margin: 0 0 10px; + color: var(--gravity-text-strong); +} +.import-dialog .table-scroll { + border: 1px solid var(--gravity-border); + border-radius: var(--gravity-radius); + background: var(--gravity-surface); + overflow: auto; +} +.import-dialog table { + width: 100%; + border-collapse: collapse; + font-size: var(--text-xs); +} +.import-dialog th { + position: sticky; + top: 0; + background: var(--gravity-surface-2); + padding: 8px 12px; + text-align: left; + font-weight: 600; + color: var(--gravity-secondary); + border-bottom: 1px solid var(--gravity-border); + z-index: 1; +} +.import-dialog td { + padding: 8px 12px; + border-bottom: 1px solid var(--gravity-border-hairline); + color: var(--gravity-text); + vertical-align: middle; +} +.import-dialog tr:last-child td { + border-bottom: none; +} +.import-dialog td code { + background: var(--gravity-code); + padding: 2px 6px; + border-radius: 4px; + font-size: var(--text-2xs); + font-family: monospace; +} +.preview-stat-cards { + display: grid; + grid-template-columns: repeat(4, 1fr); + gap: 12px; + margin-bottom: 20px; +} +.preview-stat-cards .stat-card { + display: flex; + flex-direction: column; + gap: 4px; + padding: 12px 14px; + border-radius: var(--gravity-radius); + background: var(--gravity-surface); + border: 1px solid var(--gravity-border); +} +.preview-stat-cards .stat-value { + font-size: var(--text-xl); + font-weight: 700; + color: var(--gravity-text-strong); + line-height: 1; +} +.preview-stat-cards .stat-label { + font-size: var(--text-2xs); + color: var(--gravity-muted); + text-transform: uppercase; + letter-spacing: 0.04em; + font-weight: 600; +} +.preview-stat-cards .stat-card.success .stat-value { + color: var(--gravity-chart-positive); +} +.preview-stat-cards .stat-card.warning .stat-value { + color: var(--gravity-warning); +} +.preview-stat-cards .stat-card.danger .stat-value { + color: var(--gravity-danger); +} +.preview-tabs { + display: flex; + gap: 4px; + border-bottom: 1px solid var(--gravity-border); + margin-bottom: 16px; +} +.preview-tabs .tab { + padding: 8px 14px; + font-size: var(--text-xs); + font-weight: 500; + color: var(--gravity-muted); + background: transparent; + border: none; + border-bottom: 2px solid transparent; + cursor: pointer; + transition: + color var(--duration-fast), + border-color var(--duration-fast); + border-radius: 0; +} +.preview-tabs .tab:hover { + color: var(--gravity-text); +} +.preview-tabs .tab.active { + color: var(--gravity-accent); + border-bottom-color: var(--gravity-accent); + font-weight: 600; +} +.reasons-list { + list-style: none; + padding: 0; + margin: 0; + display: flex; + flex-direction: column; + gap: 4px; +} +.import-badge { + display: inline-flex; + align-items: center; + padding: 2px 8px; + border-radius: 9999px; + font-size: var(--text-2xs); + font-weight: 500; + line-height: 1.4; +} +.import-badge.warning { + background: rgba(150, 87, 12, 0.15); + color: #d97706; + border: 1px solid rgba(150, 87, 12, 0.3); +} +.import-badge.danger { + background: var(--gravity-danger-soft); + color: var(--gravity-danger); + border: 1px solid rgba(186, 50, 55, 0.3); +} +.banner.warning { + display: flex; + align-items: flex-start; + gap: 10px; + padding: 10px 14px; + border-radius: var(--gravity-radius); + background: rgba(150, 87, 12, 0.1); + border: 1px solid rgba(150, 87, 12, 0.3); + color: var(--gravity-warning); + font-size: var(--text-xs); + line-height: 1.45; + margin-bottom: 14px; +} +.banner.warning p { + margin: 0; +} +.summary-card { + display: flex; + flex-direction: column; + align-items: center; + text-align: center; + padding: 24px 16px; + gap: 12px; +} +.summary-card .success-icon { + color: var(--gravity-chart-positive); +} +.summary-headline { + font-size: var(--text-dense); + color: var(--gravity-text); + margin: 0; +} +.summary-breakdown { + width: 100%; + max-width: 320px; + display: flex; + flex-direction: column; + gap: 8px; + margin-top: 12px; + padding: 14px 18px; + background: var(--gravity-surface-2); + border-radius: var(--gravity-radius-lg); + border: 1px solid var(--gravity-border); +} +.summary-row { + display: flex; + justify-content: space-between; + align-items: center; + font-size: var(--text-xs); + color: var(--gravity-secondary); +} +.import-dialog .summary-row strong { + color: var(--gravity-text-strong); +} +.import-dialog :where(.dialog-actions) { + margin-top: 20px; + padding-top: 16px; + border-top: 1px solid var(--gravity-border-hairline); + display: flex; + justify-content: flex-end; + gap: 10px; +} diff --git a/src/components/crm-app.tsx b/src/components/crm-app.tsx index 9b01a218..ada7ce08 100644 --- a/src/components/crm-app.tsx +++ b/src/components/crm-app.tsx @@ -21,6 +21,7 @@ import { focusedRecord, navigableRecords } from "./keyboard-navigation"; import { EnrollDialog } from "./outreach/enroll-dialog"; import { ResizeHandle, usePanelLayout } from "./panel-layout"; import { ProductDialog } from "./product-dialog"; +import { CsvImportDialog } from "./records/csv-import-dialog"; import { PeekPanel } from "./records/peek-panel"; import { actionFilters, @@ -666,6 +667,13 @@ function CrmShell({ children }: { children: ReactNode }) { crm.setPersonDialog(true); }, }, + { + id: "import-csv", + title: t.importCsv, + shortcut: "", + disabled: !data?.products.length, + run: () => crm.setImportDialog(true), + }, ...(section === "sequences" || outreachTab === "sequences" ? [ { @@ -713,6 +721,18 @@ function CrmShell({ children }: { children: ReactNode }) { onClose={() => setProductDialog("")} /> )} + {crm.importDialog && data && ( + crm.setImportDialog(false)} + onImported={async () => { + await crm.refresh(); + crm.notify(t.updated, "success"); + }} + /> + )} {!data ? (
{organizationId ? ( diff --git a/src/components/crm/crm-context.tsx b/src/components/crm/crm-context.tsx index ee51d495..6239fe1d 100644 --- a/src/components/crm/crm-context.tsx +++ b/src/components/crm/crm-context.tsx @@ -139,6 +139,7 @@ function useCrmState({ const [expanded, setExpanded] = useState(false); const [searchState, setSearchState] = useState({ path: pathname, text: "" }); const [personDialog, setPersonDialogState] = useState(false); + const [importDialog, setImportDialogState] = useState(false); const [recordDialog, setRecordDialog] = useState( null, ); @@ -208,6 +209,10 @@ function useCrmState({ if (open && !canLeaveEditor()) return; setPersonDialogState(open); }; + const setImportDialog = (open: boolean) => { + if (open && !canLeaveEditor()) return; + setImportDialogState(open); + }; const setSearch = (text: string) => setSearchState({ path: pathname, text }); const currentOrg = sourceData?.organization?.id === organizationId @@ -762,6 +767,8 @@ function useCrmState({ setExpanded, personDialog, setPersonDialog, + importDialog, + setImportDialog, recordDialog, openRecordDialog, closeRecordDialog: () => setRecordDialog(null), diff --git a/src/components/records/csv-import-dialog.tsx b/src/components/records/csv-import-dialog.tsx new file mode 100644 index 00000000..28f46eae --- /dev/null +++ b/src/components/records/csv-import-dialog.tsx @@ -0,0 +1,684 @@ +"use client"; +import { detectColumnMapping, parseCsv } from "@crm/core/csv"; +import type { ClientSnapshot } from "@crm/core/dto"; +import t from "@crm/i18n/translations/en.json"; +import { + AlertTriangle, + ArrowRight, + CheckCircle2, + FileSpreadsheet, + Upload, + X, + XCircle, +} from "lucide-react"; +import { useRef, useState } from "react"; +import { errorText, requestJson } from "../client-api"; +import { useModalLifecycle } from "../modal-lifecycle"; +import { Select } from "../ui/select"; +import { LoadingState } from "../ui/states"; + +export interface PreviewResponse { + summary: { + totalRows: number; + validCount: number; + duplicateCount: number; + invalidCount: number; + headers: string[]; + detectedMapping: Record; + }; + validRows: Array<{ + rowIndex: number; + person: { + name: string; + email?: string; + title?: string; + phone?: string; + linkedinUrl?: string; + summary?: string; + }; + company?: { + name?: string; + domain?: string; + description?: string; + }; + }>; + duplicateRows: Array<{ + rowIndex: number; + person: { + name: string; + email?: string; + title?: string; + phone?: string; + linkedinUrl?: string; + summary?: string; + }; + company?: { + name?: string; + domain?: string; + description?: string; + }; + reasons: Array<{ + kind: string; + message: string; + existingId?: string; + existingName?: string; + }>; + }>; + invalidRows: Array<{ + rowIndex: number; + row: Record; + errors: string[]; + }>; +} + +export interface ImportResponse { + totalRows: number; + importedCount: number; + peopleCreated: number; + companiesCreated: number; + relationshipsCreated: number; + duplicateCount: number; + invalidCount: number; + duplicates: unknown[]; + invalid: unknown[]; +} + +export const mappingFieldOptions = [ + { value: "ignore", label: t.doNotImport }, + { value: "name", label: t.fieldPersonName }, + { value: "email", label: t.fieldEmail }, + { value: "title", label: t.fieldTitle }, + { value: "phone", label: t.fieldPhone }, + { value: "linkedinUrl", label: t.fieldLinkedin }, + { value: "summary", label: t.fieldSummary }, + { value: "companyName", label: t.fieldCompanyName }, + { value: "companyDomain", label: t.fieldCompanyDomain }, + { value: "companyDescription", label: t.fieldCompanyDescription }, +]; + +export function CsvImportDialog({ + data, + organizationId, + productId: defaultProductId, + onClose, + onImported, +}: { + data: ClientSnapshot; + organizationId: string; + productId: string; + onClose: () => void; + onImported: () => Promise; +}) { + const modal = useRef(null); + useModalLifecycle(modal); + + const fileInputRef = useRef(null); + const [file, setFile] = useState(null); + const [csvText, setCsvText] = useState(""); + const [productId, setProductId] = useState( + defaultProductId || data.products[0]?.id || "", + ); + const [purpose, setPurpose] = useState<"buyer" | "partner">("buyer"); + const [headers, setHeaders] = useState([]); + const [totalRows, setTotalRows] = useState(0); + const [columnMapping, setColumnMapping] = useState>( + {}, + ); + const [step, setStep] = useState<"upload" | "preview" | "summary">("upload"); + const [previewTab, setPreviewTab] = useState< + "valid" | "duplicates" | "invalid" + >("valid"); + + const [loading, setLoading] = useState(false); + const [error, setError] = useState(""); + const [preview, setPreview] = useState(null); + const [summary, setSummary] = useState(null); + + function readFileText(f: File): Promise { + return new Promise((resolve, reject) => { + if (typeof f.text === "function") { + try { + const res = f.text(); + if (res && typeof res.then === "function") { + return resolve(res); + } + } catch {} + } + const reader = new FileReader(); + reader.onload = () => resolve(String(reader.result || "")); + reader.onerror = () => + reject(reader.error || new Error("FILE_READ_ERROR")); + reader.readAsText(f); + }); + } + + async function handleFileSelect(selectedFile: File) { + setError(""); + setFile(selectedFile); + try { + const text = await readFileText(selectedFile); + setCsvText(text); + const parsed = parseCsv(text); + setHeaders(parsed.headers); + setTotalRows(parsed.rows.length); + const detected = detectColumnMapping(parsed.headers); + setColumnMapping(detected); + } catch (err) { + setError(errorText(err)); + } + } + + async function runPreview(currentMapping?: Record) { + if (!csvText) { + setError(t.selectFilePrompt); + return; + } + if (!productId) { + setError(t.selectProductPrompt); + return; + } + const mapping = currentMapping || columnMapping; + const hasNameMapping = Object.values(mapping).includes("name"); + if (!hasNameMapping) { + setError(t.csvRequiredNameMissing); + return; + } + + setLoading(true); + setError(""); + try { + const res = await requestJson("/api/crm", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + operation: "csv-import-preview", + organizationId, + productId, + csvText, + columnMapping: mapping, + purpose, + }), + }); + setPreview(res); + setStep("preview"); + if (res.summary.validCount > 0) { + setPreviewTab("valid"); + } else if (res.summary.duplicateCount > 0) { + setPreviewTab("duplicates"); + } else { + setPreviewTab("invalid"); + } + } catch (err) { + setError(errorText(err)); + } finally { + setLoading(false); + } + } + + async function executeImport(skipDuplicates: boolean) { + if (!csvText || !productId) return; + setLoading(true); + setError(""); + try { + const res = await requestJson("/api/crm", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + operation: "csv-import", + organizationId, + productId, + csvText, + columnMapping, + purpose, + skipDuplicates, + label: file?.name || "CSV Import", + }), + }); + setSummary(res); + setStep("summary"); + await onImported(); + } catch (err) { + setError(errorText(err)); + } finally { + setLoading(false); + } + } + + return ( + { + e.preventDefault(); + onClose(); + }} + > +
+
+

{t.csvImportTitle}

+

{t.csvImportDescription}

+
+ +
+ + {error && ( +
+ + {error} +
+ )} + + {loading && } + + {!loading && step === "upload" && ( +
+
+
+ {t.selectProduct} + setPurpose(val as "buyer" | "partner")} + /> +
+
+ + {file ? ( +
+ { + const picked = e.target.files?.[0]; + if (picked) void handleFileSelect(picked); + }} + /> +
+ +
+ {file.name} + + {(file.size / 1024).toFixed(1)} KB · {totalRows} rows + +
+ +
+
+ ) : ( + + )} + + {headers.length > 0 && ( +
+

{t.mapColumns}

+
+ + + + + + + + + {headers.map((header) => ( + + + + + ))} + +
{t.csvColumn}{t.mapsToField}
+ {header} + +
+
+
+ )} + +
+ + +
+
+ )} + + {!loading && step === "preview" && preview && ( +
+
+
+ {preview.summary.totalRows} + {t.totalRows} +
+
+ {preview.summary.validCount} + {t.validRowsCount} +
+
0 ? "warning" : ""}`} + > + + {preview.summary.duplicateCount} + + {t.duplicatesCount} +
+
0 ? "danger" : ""}`} + > + {preview.summary.invalidCount} + {t.invalidCount} +
+
+ +
+ + + +
+ +
+ {previewTab === "valid" && ( +
+ {preview.validRows.length === 0 ? ( +

+ No valid rows ready to import. +

+ ) : ( + + + + + + + + + + + + {preview.validRows.slice(0, 100).map((row) => ( + + + + + + + + ))} + +
{t.rowIndex}{t.name}{t.email}{t.company}{t.title}
{row.rowIndex} + {row.person.name} + {row.person.email || "—"} + {row.company?.name || row.company?.domain || "—"} + {row.person.title || "—"}
+ )} + {preview.validRows.length > 100 && ( +

+ Showing first 100 of {preview.validRows.length} valid rows. +

+ )} +
+ )} + + {previewTab === "duplicates" && ( +
+
+ + {t.duplicateReviewNote} +
+
+ {preview.duplicateRows.length === 0 ? ( +

{t.noDuplicatesFound}

+ ) : ( + + + + + + + + + + + + {preview.duplicateRows.map((row) => ( + + + + + + + + ))} + +
{t.rowIndex}{t.name}{t.email}{t.company}{t.reason}
{row.rowIndex}{row.person.name}{row.person.email || "—"} + {row.company?.name || row.company?.domain || "—"} + +
    + {row.reasons.map((reason) => ( +
  • + + {reason.message} + +
    • + ))} +
+
+ )} +
+
+ )} + + {previewTab === "invalid" && ( +
+
+ {preview.invalidRows.length === 0 ? ( +

{t.noInvalidRows}

+ ) : ( + + + + + + + + + {preview.invalidRows.map((row) => ( + + + + + ))} + +
{t.rowIndex}{t.importReason}
{row.rowIndex} +
    + {row.errors.map((err) => ( +
  • + + {err} + +
    • + ))} +
+
+ )} +
+
+ )} +
+ +
+ + {preview.summary.duplicateCount > 0 ? ( + + ) : ( + + )} +
+
+ )} + + {!loading && step === "summary" && summary && ( +
+
+ +

{t.importSummary}

+

+ {t.importedSuccess.replace( + "{count}", + String(summary.importedCount), + )} +

+ +
+
+ {t.importedPeople} + {summary.peopleCreated} +
+
+ {t.importedCompanies} + {summary.companiesCreated} +
+
+ {t.skippedDuplicates} + {summary.duplicateCount} +
+
+ {t.invalidCount} + {summary.invalidCount} +
+
+
+ +
+ +
+
+ )} + + ); +} diff --git a/src/components/shell/view-toolbar.tsx b/src/components/shell/view-toolbar.tsx index 7c6a88b1..86da6d42 100644 --- a/src/components/shell/view-toolbar.tsx +++ b/src/components/shell/view-toolbar.tsx @@ -1,7 +1,7 @@ "use client"; import { shortcutLabel } from "@crm/core/shortcuts"; import t from "@crm/i18n/translations/en.json"; -import { Plus, Search, Send, X } from "lucide-react"; +import { Plus, Search, Send, Upload, X } from "lucide-react"; import { useSearchParams } from "next/navigation"; import type { RefObject } from "react"; import { label } from "../client-api"; @@ -172,32 +172,56 @@ export function ViewToolbar({ )} {section === "people" && ( - + <> + + + )} {section === "companies" && ( - + <> + + + )} {section === "meetings" && (