Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .supervision/creative-studio-private-sql/concept.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Concept: Creative Studio Private SQL

## Intent

Deploy the internal Creative Studio trial in `iconic-ds-creative-studio-dev` in `us-central1` without a public Cloud SQL address. Preserve the existing Cloud Run Cloud SQL Auth Proxy socket integration while making the proxy reach the instance over private networking.

## Constraints

- The deployment must use `feature/KN-DATAX-15064-deploy-creative-studio` in the organisation-owned fork.
- The inherited `constraints/sql.restrictPublicIp` policy prohibits public Cloud SQL IP addresses.
- Do not apply or destroy infrastructure until Terraform plan review confirms the private design.
- Do not store credentials, Terraform state, generated environment directories, bootstrap state, or tfvars containing real values in Git.
- Keep project resources and the VPC connector in `us-central1`; use no shared development project resources.
- The existing administrative Shared VPC arrangement may require network-user or service-agent permissions in the host project. Terraform must accept an approved existing network rather than assume it can create a new one there.

## Design

- Create a reusable Terraform networking module that either creates a dedicated VPC or consumes an explicitly supplied approved VPC network.
- Reserve a global internal range and create Private Services Access peering for `servicenetworking.googleapis.com`.
- Configure Cloud SQL PostgreSQL with `ipv4_enabled = false` and the selected VPC network. The instance receives only a private address.
- Create a regional Serverless VPC Access connector with a non-overlapping `/28` CIDR range in `us-central1`.
- Attach only the backend Cloud Run service to that connector with `PRIVATE_RANGES_ONLY` egress. The backend's Cloud SQL Python Connector is explicitly configured for `PRIVATE` IP; this corrects the former implicit `PUBLIC` connector path.
- Enable the Compute Engine, Service Networking, and Serverless VPC Access APIs. Retain existing Cloud SQL client and Secret Manager IAM bindings.
- Expose non-sensitive outputs needed to validate the connection and document manual host-project permissions where a Shared VPC is selected.

## Trade-offs

- A Serverless VPC Access connector has ongoing cost and throughput/scale limits, but is supported by established Cloud Run Terraform fields and avoids requiring a newer provider for Direct VPC egress.
- A dedicated VPC is self-contained for a time-boxed trial, while an existing Shared VPC can meet organisational networking requirements but needs host-project coordination and preallocated non-overlapping ranges.
- `PRIVATE_RANGES_ONLY` routes database traffic privately without forcing all Internet-bound backend traffic through a NAT gateway. This is the smallest change compatible with the Cloud SQL Python Connector path.
31 changes: 31 additions & 0 deletions .supervision/creative-studio-private-sql/progress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Progress

## Status

- **State**: Executing
- **Current Objective**: Correct Alembic private-IP selection and complete private-network seeding.

## Plan

- [x] Phase 1: Check branch, repository state, deployment wiring, and provider constraints.
- [x] Phase 2: Establish the private Cloud SQL concept and risks.
- [x] Phase 3: Implement Terraform networking and Cloud Run connectivity.
- [x] Phase 4: Add deployment, validation, and teardown documentation.
- [x] Phase 5: Run available validation and commit.
- [ ] Phase 6: Validate and commit bootstrap branch-resume correction.
- [ ] Phase 7: Validate and commit Alembic private-IP migration correction.

## Journal

### 2026-08-20 Private Cloud SQL Deployment

- Validated deployed backend — **Finding**: Cloud SQL is `RUNNABLE` with only private IP `172.30.0.3`, but Alembic's connector was hard-coded to `IPTypes.PUBLIC`, producing `CloudSQLIPTypeError` with preference `PRIMARY` — **Decision**: make Alembic honor `CLOUD_SQL_IP_TYPE`; Cloud Shell remains unsuitable for private-only seeding, so use a VPC-attached Cloud Run Job after the corrected image deploys.
- Validated fresh Cloud Shell checkout — **Finding**: Terraform initialized the private-networking module, then Google provider v7.45.0 rejected `project` on `google_service_networking_connection` — **Decision**: remove the unsupported attribute; the fully qualified VPC self link identifies the network-owning project.
- Resumed deployment investigation — **Finding**: Cloud SQL apply attempted a public IP despite the feature branch setting `ipv4_enabled = false`; bootstrap reuses an existing checkout without previously switching it to the branch entered by the user — **Decision**: update a clean existing checkout to the selected branch before reuse and document the recovery commands.
- Committed implementation — **Finding**: static diagnostics and Git whitespace checks pass, while `uv`, `terraform`, and `gcloud` are unavailable locally — **Decision**: commit the reviewed change without apply; require Cloud Shell Terraform formatting, validation, plan review, and runtime validation before deployment.
- Implemented private connectivity — **Finding**: the application selected `IPTypes.PUBLIC` when `USE_CLOUD_SQL_AUTH_PROXY` was unset, despite mounting the Cloud SQL socket — **Decision**: configure the Python Connector with an explicit validated IP type and set `CLOUD_SQL_IP_TYPE=PRIVATE` in the deployment.
- Added VPC resources — **Finding**: the provider configuration has no version constraints or lock file — **Decision**: use the broadly supported Serverless VPC Access connector rather than Direct VPC egress; use `PRIVATE_RANGES_ONLY` to avoid requiring Cloud NAT for Internet traffic.
- Validation environment — **Finding**: neither `uv` nor `terraform` is installed in the local execution environment — **Decision**: record focused test and Terraform validation as pending Cloud Shell or a configured developer environment.
- Inspected the organisation fork on `feature/KN-DATAX-15064-deploy-creative-studio` — **Finding**: existing untracked `AGENTS.md` and `README_monorepo.md` are user files and will remain untouched — **Decision**: make implementation changes only in tracked Creative Studio deployment files.
- Traced the database path — **Finding**: the backend Cloud Run service mounts the Cloud SQL Auth Proxy socket at `/cloudsql/<instance-connection-name>` but has no VPC connectivity; PostgreSQL enables public IPv4 — **Decision**: retain the socket path and add private VPC connectivity for the backend only.
- Reviewed provider declarations — **Finding**: the example environment does not constrain Google provider versions — **Decision**: use Serverless VPC Access connector fields rather than require Direct VPC egress support from an unknown provider version.
26 changes: 26 additions & 0 deletions .supervision/trial-warning-banner/concept.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Concept: Trial Warning Banner

## Intent

Warn Creative Studio trial users not to enter, upload, or generate sensitive, confidential, or personal information.

## Constraints

- Implement on branch `KN-DATAX-15227-Put-a-warning-banner-onto-Creative-Studio-Frontend-for-trial`.
- Show the warning across authenticated Creative Studio and admin routes.
- Exclude login, password-reset, and support-ticket routes, which do not render the standard application shell.
- Keep navigation and primary content usable on desktop and mobile viewports.
- Use the existing Angular and Tailwind styling conventions; do not add dependencies.
- Do not modify existing untracked deployment handoff, log, or reference files.

## Design

- Add a persistent red warning banner in the root Angular application shell, above `app-header`.
- Use the copy: `Trial environment: Do not enter, upload, or generate sensitive, confidential, or personal information.`
- Reuse `AppComponent` route state so the banner renders only when the authenticated application shell is shown.
- Add focused styles in `AppComponent` and a focused test asserting the banner and warning copy render.

## Trade-offs

- A root-shell banner gives all authenticated routes one consistent warning with minimal maintenance, but it remains visible while users work and consumes a small amount of vertical space.
- Excluding authentication and support flows limits warning coverage before sign-in, but preserves the existing simplified layouts and focuses the notice where users provide application content.
24 changes: 24 additions & 0 deletions .supervision/trial-warning-banner/progress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Progress

## Status

- **State**: Executing
- **Current Objective**: Resolve local frontend dependency installation, then run focused banner validation.

## Plan

- [x] Phase 1: Confirm branch, repository guidance, and the root application shell.
- [x] Phase 2: Establish intent, constraints, design, and trade-offs.
- [x] Phase 3: Add the authenticated-route warning banner and focused styling.
- [ ] Phase 4: Add focused test coverage and run frontend validation.
- [ ] Phase 5: Update status and prepare the pull request summary.

## Journal

### 2026-08-24 Trial Warning Banner

- Validated the pushed banner commit in Cloud Shell — **Finding**: TypeScript compilation passed, the focused Karma test remains blocked because Cloud Shell has no ChromeHeadless binary, and lint reported four Prettier errors only in the new warning assertion — **Decision**: reformat the assertion locally, then push a follow-up commit and rerun lint in Cloud Shell; the remaining 335 lint warnings predate this change.
- Implemented root-shell warning — **Finding**: `showHeader` controls the authenticated application shell — **Decision**: render an accessible red alert above `app-header`, with a focused spec that verifies the alert is shown only when that shell is visible.
- Attempted focused validation — **Finding**: `npm test -- --include='src/app/app.component.spec.ts' --watch=false` did not run because `tsc` is absent; `npm ci` then failed to validate the package registry certificate under Node `v24.19.0`, which also produces an unsupported-engine warning — **Decision**: make no source changes until dependencies can be installed with the organisation's trusted certificate configuration and a supported Node version.
- Reviewed the root Angular shell and route visibility logic — **Finding**: `AppComponent` already decides whether the standard header is shown, covering authenticated application and admin routes while excluding login-related routes — **Decision**: render the warning alongside the existing standard shell above `app-header`.
- Reviewed repository workflow — **Finding**: this is a non-trivial user-facing frontend change on a non-main feature branch — **Decision**: create this task-specific supervision record before implementation.
2 changes: 1 addition & 1 deletion backend/alembic/env.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ async def alembic_get_connection():
user=config_service.DB_USER,
password=config_service.DB_PASS,
db=config_service.DB_NAME,
ip_type=IPTypes.PUBLIC,
ip_type=IPTypes[config_service.CLOUD_SQL_IP_TYPE],
)
return conn

Expand Down
3 changes: 2 additions & 1 deletion backend/src/config/config_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

from typing import Any
from typing import Any, Literal

import google.auth
from google.auth.exceptions import DefaultCredentialsError
Expand Down Expand Up @@ -67,6 +67,7 @@ class ConfigService(BaseSettings):
DB_PASS: str = "password"
DB_NAME: str = "creative_studio"
USE_CLOUD_SQL_AUTH_PROXY: bool = False
CLOUD_SQL_IP_TYPE: Literal["PUBLIC", "PRIVATE"] = "PUBLIC"
DB_HOST: str = "localhost"
DB_PORT: str = "5432"

Expand Down
4 changes: 2 additions & 2 deletions backend/src/database.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ async def get_connection():
user=config_service.DB_USER,
password=config_service.DB_PASS,
db=config_service.DB_NAME,
ip_type=IPTypes.PUBLIC, # Adjust if using Private IP
ip_type=IPTypes[config_service.CLOUD_SQL_IP_TYPE],
)

return conn
Expand Down Expand Up @@ -196,7 +196,7 @@ async def get_conn():
user=config_service.DB_USER,
password=config_service.DB_PASS,
db=config_service.DB_NAME,
ip_type=IPTypes.PUBLIC,
ip_type=IPTypes[config_service.CLOUD_SQL_IP_TYPE],
)

self.engine = create_async_engine(
Expand Down
10 changes: 10 additions & 0 deletions backend/tests/test_database.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from unittest.mock import AsyncMock, MagicMock, patch

import pytest
from google.cloud.sql.connector import IPTypes

from src.config.config_service import config_service
from src.database import (
Expand Down Expand Up @@ -131,6 +132,7 @@ async def test_get_connection_cloud_sql():
"INSTANCE_CONNECTION_NAME",
"projects/p/locations/l/instances/i",
),
patch.object(config_service, "CLOUD_SQL_IP_TYPE", "PRIVATE"),
):
mock_connector = AsyncMock()
mock_connector.connect_async = AsyncMock(return_value="cloud_conn")
Expand All @@ -143,6 +145,14 @@ async def test_get_connection_cloud_sql():

res = await get_connection()
assert res == "cloud_conn"
mock_connector.connect_async.assert_awaited_once_with(
"projects/p/locations/l/instances/i",
"asyncpg",
user=config_service.DB_USER,
password=config_service.DB_PASS,
db=config_service.DB_NAME,
ip_type=IPTypes.PRIVATE,
)


@pytest.mark.anyio
Expand Down
20 changes: 19 additions & 1 deletion bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -349,14 +349,32 @@ setup_repo() {
if [[ -d "$REPO_CLONE_DIR" ]]; then
warn "Directory '$REPO_CLONE_DIR' already exists."; prompt "Do you want to use this existing directory? (y/n)"; read -r REPLY < /dev/tty
if [[ ! $REPLY =~ ^[Yy]$ ]]; then fail "Please remove the directory or run the script from a different location."; fi

cd "$REPO_CLONE_DIR"
git rev-parse --is-inside-work-tree > /dev/null 2>&1 || fail "Existing directory '$REPO_CLONE_DIR' is not a Git repository."
if ! git diff --quiet || ! git diff --cached --quiet; then
fail "Existing repository has uncommitted changes. Commit, stash, or use a new directory before continuing."
fi
if [[ -n "$(git ls-files --others --exclude-standard)" ]]; then
fail "Existing repository has untracked files. Move them or use a new directory before continuing."
fi
info "Updating existing checkout to branch '$SELECTED_BRANCH'..."
git fetch origin "$SELECTED_BRANCH"
if git show-ref --verify --quiet "refs/heads/$SELECTED_BRANCH"; then
git checkout "$SELECTED_BRANCH"
else
git checkout --track "origin/$SELECTED_BRANCH"
fi
git pull --ff-only origin "$SELECTED_BRANCH"
cd ..
else
info "Performing a sparse checkout of '$REPO_CLONE_DIR' (Branch: $SELECTED_BRANCH)..."

# 1. Clone with -b branch_name
git clone --filter=blob:none --no-checkout --depth 1 --sparse -b "$SELECTED_BRANCH" "$GITHUB_REPO_URL" "$REPO_CLONE_DIR"

cd "$REPO_CLONE_DIR"

# 2. Sparse checkout for ROOT folders only
git sparse-checkout set "infra" "backend" "frontend" "bootstrap.sh"

Expand Down
4 changes: 4 additions & 0 deletions frontend/src/app/app.component.html
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@
mode="indeterminate"
></mat-progress-bar>
<app-notification-container></app-notification-container>
<div *ngIf="showHeader" class="trial-warning" role="alert">
Trial environment: Do not enter, upload, or generate sensitive,
confidential, or personal information.
</div>
<app-header *ngIf="showHeader"></app-header>
<div
[ngClass]="showHeader ? 'content' : 'login'"
Expand Down
12 changes: 12 additions & 0 deletions frontend/src/app/app.component.scss
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,18 @@
position: relative;
}

.trial-warning {
background-color: #b91c1c;
color: #fff;
font-size: 0.875rem;
font-weight: 600;
line-height: 1.4;
padding: 0.5rem 1rem;
position: relative;
text-align: center;
z-index: 40;
}

::ng-deep .mat-mdc-snack-bar-container {
&.green-snackbar {
--mdc-snackbar-container-color: #0f9d58;
Expand Down
19 changes: 18 additions & 1 deletion frontend/src/app/app.component.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

import {TestBed} from '@angular/core/testing';
import {RouterTestingModule} from '@angular/router/testing';
import {NoopAnimationsModule} from '@angular/platform-browser/animations';
import {AppComponent} from './app.component';
import {LoadingService} from './common/services/loading.service';
import {of} from 'rxjs';
Expand All @@ -30,7 +31,7 @@ describe('AppComponent', () => {
};

await TestBed.configureTestingModule({
imports: [RouterTestingModule],
imports: [RouterTestingModule, NoopAnimationsModule],
declarations: [AppComponent],
providers: [{provide: LoadingService, useValue: loadingServiceMock}],
schemas: [NO_ERRORS_SCHEMA],
Expand All @@ -48,4 +49,20 @@ describe('AppComponent', () => {
const app = fixture.componentInstance;
expect(app.title).toEqual('creative-studio');
});

it('should show the trial warning only with the application header', () => {
const fixture = TestBed.createComponent(AppComponent);
const app = fixture.componentInstance;

fixture.detectChanges();
expect(
fixture.nativeElement.querySelector('.trial-warning')?.textContent,
).toContain(
'Do not enter, upload, or generate sensitive, confidential, or personal information.',
);

app.showHeader = false;
fixture.detectChanges();
expect(fixture.nativeElement.querySelector('.trial-warning')).toBeNull();
});
});
Loading