Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,27 @@ GOOGLE_CLIENT_SECRET=""
# MICROSOFT_CLIENT_ID=""
# MICROSOFT_CLIENT_SECRET=""

# Zoho Mail — the third mailbox, for a company whose domain is on Zoho rather
# than Google or Microsoft. Set both or neither, like the pairs above.
#
# Create the client at https://api-console.zoho.com as a "Server-based
# Application", with the redirect URI
# <API_URL>/api/auth/oauth2/callback/zoho. Note the /oauth2/ segment: Zoho goes
# through the generic OAuth route, not the social one Google and Microsoft use.
# The README has the full walkthrough.
#
# Zoho can be a sign-in method on its own, but the common case is signing in
# with Google or Microsoft and attaching a Zoho mailbox on Settings >
# Connections.
# ZOHO_CLIENT_ID=""
# ZOHO_CLIENT_SECRET=""

# Which Zoho data centre the account lives in. An account belongs to exactly
# one, and a token minted in one is refused by the others, so this has to match
# the domain you log in to Zoho on. One of: com, eu, in, com.au, jp, ca, sa,
# com.cn. Defaults to com.
# ZOHO_REGION="com"

# Optional. Enables Slack account linking on Settings > Connections.
# Add APP_URL + /api/auth/oauth2/callback/slack as the Slack OAuth redirect URL.
# SLACK_CLIENT_ID=""
Expand Down
40 changes: 37 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,10 +221,14 @@ Open `.env` and set these. Everything else in the file is optional and commented
| `ALLOWED_SIGN_IN` | Your email domain, e.g. `acme.com`. Or one address, e.g. `you@gmail.com`. |
| `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET`| A Google OAuth client — 2 minutes, below. Both or neither. |
| `MICROSOFT_CLIENT_ID` / `MICROSOFT_CLIENT_SECRET` | A Microsoft Entra app registration — below. Both or neither. |
| `ZOHO_CLIENT_ID` / `ZOHO_CLIENT_SECRET` | A Zoho API console client, if your mail is on Zoho — below. Both or neither. |

**Pick at least one of Google and Microsoft**, or add your own identity provider on
**Settings → SSO** once you are in. Setting both is fine and common: the sign-in page
offers both buttons, and each rep's mail is read from whichever they signed in with.
**Pick at least one of Google, Microsoft and Zoho**, or add your own identity provider
on **Settings → SSO** once you are in. Setting several is fine and common: the sign-in
page offers each button, and a rep's mail is read from whichever they signed in with.
Zoho can also be attached to an existing Google or Microsoft account from **Settings →
Connections**, which is what you want if the company signs in on one domain and keeps
its sales mailbox on another.

`DATABASE_URL` already matches the `docker compose` Postgres, so leave it alone unless
you brought your own.
Expand Down Expand Up @@ -287,6 +291,36 @@ failing to sync.

</details>

<details>
<summary><strong>Getting the Zoho OAuth client</strong></summary>

1. [Zoho API console](https://api-console.zoho.com) → **Add Client** → **Server-based
Applications**. Sign in as an admin of the Zoho org that owns the mailbox.
2. **Homepage URL** is your app's origin, e.g. `http://localhost:3000`.
3. **Authorized Redirect URIs** →
`http://localhost:3001/api/auth/oauth2/callback/zoho`. In production this is
`https://<your-api-host>/api/auth/oauth2/callback/zoho` — the API's origin, not the
app's. Note the `/oauth2/` segment: Zoho goes through the generic OAuth route, so
this path differs from the Google and Microsoft ones above.
4. Copy the **Client ID** and **Client Secret** into `.env` as `ZOHO_CLIENT_ID` and
`ZOHO_CLIENT_SECRET`.
5. If your Zoho account is not on `zoho.com`, set `ZOHO_REGION` to the suffix you log
in on — `eu`, `in`, `com.au`, `jp`, `ca`, `sa` or `com.cn`. An account lives in
exactly one data centre and a token minted in one is refused by the others, so a
wrong value here looks like an account that will not connect.

The scopes are requested at sign-in and need no entry in the console:
`ZohoMail.accounts.READ`, `ZohoMail.folders.READ`, `ZohoMail.messages.READ` and
`AaaServer.profile.READ`. All four are read-only — the CRM can list and read mail and
can never send, reply, move or delete. Reading is forward-only, exactly like the other
two: the first check records the current time and imports nothing.

Zoho only issues a refresh token while it is showing the consent screen, so a
connection that comes back without one has to be disconnected and reconnected rather
than repaired. The connection card says so when it happens.

</details>

`ALLOWED_SIGN_IN` is the entire authorisation model — an unset value means nobody can
sign in, which is the safe direction to fail. It takes whole domains, individual
addresses, or a mix:
Expand Down
67 changes: 67 additions & 0 deletions adrs/zoho-mail.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Read Zoho Mail as a third mailbox provider

We run this CRM against a business that hosts its mail on Zoho, not on Google
Workspace or Microsoft 365. Today that means the agent has nothing to read: our
sales mailbox is `@elitesystemsdesign.com` on Zoho, and the only mail the CRM
can see belongs to a personal Gmail account that isn't where the business
actually happens. Every company, contact and thread the product is supposed to
fill in by itself has to be typed in by hand instead.

Zoho Mail is not a niche choice — it's the usual answer for a small company that
wanted its own domain without paying per seat for Workspace. So this is less
"support my setup" than "the second-tier mail host that the CRM's whole premise
quietly excludes".

## Why not the two easy answers

**Generic IMAP** would cover Zoho and everything else in one go, and it's the
obvious suggestion. We didn't do it, for two reasons. It would be the only
provider with no OAuth story — a password in the database, or an app-specific
password the user has to mint and re-mint, against a codebase where every other
credential is a refreshable token in the `account` table. And IMAP gives no
usable incremental cursor without holding a connection and a UID validity map,
which is a different shape of sync service from the two that already exist. It
is a bigger change that fits the codebase worse.

**A side script** that pushes mail in through an intake API was the other
option. It puts the mail in the CRM but leaves it outside everything that makes
the mailbox layer worth having: no connection card, no scope checking, no
purge, no reconnect flow, no `syncedByUserId` to purge against.

## What we did instead

Followed #73. The provider union in `packages/auth/src/scopes.ts` and the
`satisfies Record<…>` maps in `mailbox.constants.ts` are already the shape a
third provider slots into — the compiler names every arm that needs filling,
which is exactly the property you want when adding one. `ThreadWriterService`,
`MailboxMatchService`, `SyncStateService` and the `EmailThread`/`EmailMessage`
models needed no changes at all; `zohoMessageId` and `zohoWebLink` sit beside
the Gmail and Outlook columns.

Four things about Zoho are genuinely different from Graph, and they are where
the code is not a copy of the Outlook adapter:

- **It is a generic OAuth provider, not a better-auth social one.** It goes
through the same `genericOAuth` plugin Slack already uses, so it links onto an
existing account rather than only being a sign-in. That matters for the case
above: sign in with Google, attach a Zoho mailbox on a different domain.
- **`Authorization: Zoho-oauthtoken <token>`, not `Bearer`.** `MailboxApiClient`
grew one optional scheme argument.
- **There is no "changed since" filter.** The list endpoint pages with
`start`/`limit` over a date-sorted list, so the incremental sync reads
newest-first and stops at the first message the last tick already saw. The
cursor is epoch milliseconds rather than an ISO string.
- **The list carries no RFC `Message-ID`.** Without one, the same mail seen
through Gmail and through Zoho would be stored twice, so each new message
costs a second call to the header endpoint. That is the main cost of this
adapter and the reason its per-tick ceiling is lower than Outlook's.

## What it breaks

`hasSyncScopes`, `mailboxGrantsNeeded` and the `SYNC_SOURCES` maps gain a third
arm, which is a compile error everywhere until filled in — that is the union
doing its job, and every site is in this diff. `rebuildThreads` was duplicated
verbatim in the Google and Microsoft connection services; rather than add a
third copy it moved to `mailbox/thread-rebuild.ts`. Nothing else changes for an
install that never sets `ZOHO_CLIENT_ID`: unset, the provider is not registered,
the connection card says so, and no new query runs.
2 changes: 2 additions & 0 deletions apps/api/src/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import { TrackingModule } from "./tracking/tracking.module";
import { TrpcModule } from "./trpc/trpc.module";
import { UsersModule } from "./users/users.module";
import { WorkspaceModule } from "./workspace/workspace.module";
import { ZohoModule } from "./zoho/zoho.module";

@Module({
imports: [
Expand Down Expand Up @@ -69,6 +70,7 @@ import { WorkspaceModule } from "./workspace/workspace.module";
MailboxModule,
GoogleModule,
MicrosoftModule,
ZohoModule,
SyncModule,
SettingsModule,
WorkspaceModule,
Expand Down
13 changes: 13 additions & 0 deletions apps/api/src/config/env.validation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,19 @@ export class EnvironmentVariables {
@IsString()
MICROSOFT_TENANT_ID?: string;

@IsOptional()
@IsString()
ZOHO_CLIENT_ID?: string;

@IsOptional()
@IsString()
ZOHO_CLIENT_SECRET?: string;

/** Zoho data-centre suffix: com, eu, in, com.au, jp, ca, sa, com.cn. */
@IsOptional()
@IsString()
ZOHO_REGION?: string;

@IsOptional()
@IsString()
SLACK_CLIENT_ID?: string;
Expand Down
19 changes: 19 additions & 0 deletions apps/api/src/generated/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import { slackStatusOutput, slackMatchesOutput, slackChannelsInput, slackChannel
import { ssoSignInOptionsOutput, ssoSettingsOutput, ssoProviderListInput, ssoProviderListOutput, registerSsoProviderInput, ssoProviderOutput, deleteSsoProviderInput, deleteSsoProviderOutput } from "../sso/sso.contracts";
import { trackingSettingsOutput, trackingFlagInput, cookieLifetimeInput, addDomainInput, trackedDomainOutput, removeDomainInput, rotateSiteIdOutput, verifyInput, verifyOutput, sourcesOutput, companyActivityInput, websiteActivityOutput, contactActivityInput } from "../tracking/tracking.contracts";
import { workspaceOutput, memberListInput, memberListOutput, updateWorkspaceInput, setMemberRoleInput, workspaceMemberOutput } from "../workspace/workspace.contracts";
import { zohoConnectionStatusOutput, zohoPurgeSyncedDataOutput, zohoRevokeAccessOutput, setZohoAutoCreateInput } from "../zoho/zoho.contracts";
import type { UsersRouter } from "../users/users.router";

const appRouter = t.router({
Expand Down Expand Up @@ -750,6 +751,24 @@ const appRouter = t.router({
.input(setMemberRoleInput)
.output(workspaceMemberOutput)
.mutation(async () => "PLACEHOLDER_DO_NOT_REMOVE" as any)
}),
zoho: t.router({
status: publicProcedure
.output(zohoConnectionStatusOutput)
.query(async () => "PLACEHOLDER_DO_NOT_REMOVE" as any),
purgeSyncedData: publicProcedure
.output(zohoPurgeSyncedDataOutput)
.mutation(async () => "PLACEHOLDER_DO_NOT_REMOVE" as any),
revokeAccess: publicProcedure
.output(zohoRevokeAccessOutput)
.mutation(async () => "PLACEHOLDER_DO_NOT_REMOVE" as any),
syncNow: publicProcedure
.output(zohoConnectionStatusOutput)
.mutation(async () => "PLACEHOLDER_DO_NOT_REMOVE" as any),
setAutoCreate: publicProcedure
.input(setZohoAutoCreateInput)
.output(zohoConnectionStatusOutput)
.mutation(async () => "PLACEHOLDER_DO_NOT_REMOVE" as any)
})
});

Expand Down
7 changes: 5 additions & 2 deletions apps/api/src/google/conversation.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ export class ConversationService {
sentAt: true,
gmailMessageId: true,
outlookWebLink: true,
zohoWebLink: true,
},
},
},
Expand All @@ -66,12 +67,14 @@ export class ConversationService {
fromImageUrl: faces.get(message.fromEmail.toLowerCase()) ?? null,
mailboxUrl: message.gmailMessageId
? `https://mail.google.com/mail/u/0/#all/${message.gmailMessageId}`
: message.outlookWebLink,
: (message.outlookWebLink ?? message.zohoWebLink),
mailboxName: message.gmailMessageId
? "Gmail"
: message.outlookWebLink
? "Outlook"
: null,
: message.zohoWebLink
? "Zoho Mail"
: null,
})),
};
}
Expand Down
43 changes: 1 addition & 42 deletions apps/api/src/google/google-connection.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { InjectDatabase } from "../database/database.constants";
import { MailboxMatchService } from "../mailbox/mailbox-match.service";
import { MailboxTokenService } from "../mailbox/mailbox-token.service";
import { SyncStateService } from "../mailbox/sync-state.service";
import { rebuildThreads } from "../mailbox/thread-rebuild";
import {
GOOGLE_PROVIDER_ID,
GOOGLE_SYNC_SOURCES,
Expand Down Expand Up @@ -222,45 +223,3 @@ export class GoogleConnectionService {
return { domain: normalised, purged: threads.count + events.count };
}
}

async function rebuildThreads(
tx: Prisma.TransactionClient,
threadIds: string[],
): Promise<void> {
if (threadIds.length === 0) return;

const remaining = await tx.emailMessage.findMany({
where: { threadId: { in: threadIds } },
select: { threadId: true, sentAt: true, subject: true, snippet: true },
orderBy: { sentAt: "asc" },
});

const byThread = new Map<string, typeof remaining>();

for (const message of remaining) {
const group = byThread.get(message.threadId);
if (group) group.push(message);
else byThread.set(message.threadId, [message]);
}

for (const [threadId, messages] of byThread) {
const first = messages.at(0);
const last = messages.at(-1);
if (!first || !last) continue;

await tx.emailThread.update({
where: { id: threadId },
data: {
messageCount: messages.length,
firstMessageAt: first.sentAt,
lastMessageAt: last.sentAt,
subject: first.subject,
},
});

await tx.activity.updateMany({
where: { emailThreadId: threadId },
data: { body: last.snippet, occurredAt: last.sentAt },
});
}
}
1 change: 1 addition & 0 deletions apps/api/src/google/google.contracts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ const emailThreadMessageOutput = z.object({
sentAt: z.string(),
gmailMessageId: z.string().nullable(),
outlookWebLink: z.string().nullable(),
zohoWebLink: z.string().nullable(),
fromImageUrl: z.string().nullable(),
mailboxUrl: z.string().nullable(),
mailboxName: z.string().nullable(),
Expand Down
23 changes: 22 additions & 1 deletion apps/api/src/mailbox/mailbox-api.client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ export type MailboxResult<T> =

const DEFAULT_TIMEOUT_MS = 20_000;

// Google and Microsoft both take `Authorization: Bearer <token>`. Zoho does
// not — it wants its own `Zoho-oauthtoken` scheme and answers 401 to a Bearer.
const DEFAULT_AUTH_SCHEME = "Bearer";

export type MailboxRequestOptions = {
/** OAuth authorization scheme, e.g. `Bearer` or `Zoho-oauthtoken`. */
scheme?: string;
};

const MIN_BACKOFF_MS = 30_000;
const MAX_BACKOFF_MS = 15 * 60_000;

Expand All @@ -20,6 +29,7 @@ export class MailboxApiClient {
url: string,
accessToken: string,
params: Record<string, string | number | boolean | undefined> = {},
options: MailboxRequestOptions = {},
): Promise<MailboxResult<T>> {
const target = new URL(url);
for (const [key, value] of Object.entries(params)) {
Expand All @@ -30,8 +40,13 @@ export class MailboxApiClient {
const timeout = setTimeout(() => controller.abort(), DEFAULT_TIMEOUT_MS);

try {
const scheme = options.scheme ?? DEFAULT_AUTH_SCHEME;

const response = await fetch(target, {
headers: { authorization: `Bearer ${accessToken}` },
headers: {
authorization: `${scheme} ${accessToken}`,
accept: "application/json",
},
signal: controller.signal,
});

Expand Down Expand Up @@ -114,11 +129,17 @@ export class MailboxApiClient {
try {
const body = (await response.json()) as {
error?: { message?: string; status?: string; code?: string };
// Zoho puts the human-readable reason in the envelope instead.
status?: { description?: string };
data?: { errorCode?: string; moreInfo?: string };
};
return (
body.error?.message ??
body.error?.status ??
body.error?.code ??
body.data?.moreInfo ??
body.data?.errorCode ??
body.status?.description ??
`HTTP ${response.status}`
);
} catch {
Expand Down
10 changes: 9 additions & 1 deletion apps/api/src/mailbox/mailbox-token.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ import { Injectable, Logger } from "@nestjs/common";
import { InjectDatabase } from "../database/database.constants";
import {
GOOGLE_PROVIDER_ID,
MICROSOFT_PROVIDER_ID,
PROVIDER_FOR_SOURCE,
SCOPE_FOR_SOURCE,
type SyncSource,
ZOHO_PROVIDER_ID,
} from "./mailbox.constants";

export type TokenFailure =
Expand Down Expand Up @@ -164,6 +166,12 @@ export class MailboxTokenService {
}
}

const PROVIDER_LABELS = {
[GOOGLE_PROVIDER_ID]: "Google",
[MICROSOFT_PROVIDER_ID]: "Microsoft",
[ZOHO_PROVIDER_ID]: "Zoho",
} satisfies Record<MailboxProviderId, string>;

function label(providerId: MailboxProviderId): string {
return providerId === GOOGLE_PROVIDER_ID ? "Google" : "Microsoft";
return PROVIDER_LABELS[providerId];
}
Loading