feat(creative-studio): deployed Creative Studio to a dedicated project on GCP

.supervision/creative-studio-private-sql/concept.md

@@ -0,0 +1,30 @@
+# Concept: Creative Studio Private SQL
+
+## Intent
+
+Deploy the internal Creative Studio trial in `iconic-ds-creative-studio-dev` in `us-central1` without a public Cloud SQL address. Preserve the existing Cloud Run Cloud SQL Auth Proxy socket integration while making the proxy reach the instance over private networking.
+
+## Constraints
+
+- The deployment must use `feature/KN-DATAX-15064-deploy-creative-studio` in the organisation-owned fork.
+- The inherited `constraints/sql.restrictPublicIp` policy prohibits public Cloud SQL IP addresses.
+- Do not apply or destroy infrastructure until Terraform plan review confirms the private design.
+- Do not store credentials, Terraform state, generated environment directories, bootstrap state, or tfvars containing real values in Git.
+- Keep project resources and the VPC connector in `us-central1`; use no shared development project resources.
+- The existing administrative Shared VPC arrangement may require network-user or service-agent permissions in the host project. Terraform must accept an approved existing network rather than assume it can create a new one there.
+
+## Design
+
+- Create a reusable Terraform networking module that either creates a dedicated VPC or consumes an explicitly supplied approved VPC network.
+- Reserve a global internal range and create Private Services Access peering for `servicenetworking.googleapis.com`.
+- Configure Cloud SQL PostgreSQL with `ipv4_enabled = false` and the selected VPC network. The instance receives only a private address.
+- Create a regional Serverless VPC Access connector with a non-overlapping `/28` CIDR range in `us-central1`.
+- Attach only the backend Cloud Run service to that connector with `PRIVATE_RANGES_ONLY` egress. The backend's Cloud SQL Python Connector is explicitly configured for `PRIVATE` IP; this corrects the former implicit `PUBLIC` connector path.
+- Enable the Compute Engine, Service Networking, and Serverless VPC Access APIs. Retain existing Cloud SQL client and Secret Manager IAM bindings.
+- Expose non-sensitive outputs needed to validate the connection and document manual host-project permissions where a Shared VPC is selected.
+
+## Trade-offs
+
+- A Serverless VPC Access connector has ongoing cost and throughput/scale limits, but is supported by established Cloud Run Terraform fields and avoids requiring a newer provider for Direct VPC egress.
+- A dedicated VPC is self-contained for a time-boxed trial, while an existing Shared VPC can meet organisational networking requirements but needs host-project coordination and preallocated non-overlapping ranges.
+- `PRIVATE_RANGES_ONLY` routes database traffic privately without forcing all Internet-bound backend traffic through a NAT gateway. This is the smallest change compatible with the Cloud SQL Python Connector path.

.supervision/creative-studio-private-sql/progress.md

@@ -0,0 +1,31 @@
+# Progress
+
+## Status
+
+- **State**: Executing
+- **Current Objective**: Correct Alembic private-IP selection and complete private-network seeding.
+
+## Plan
+
+- [x] Phase 1: Check branch, repository state, deployment wiring, and provider constraints.
+- [x] Phase 2: Establish the private Cloud SQL concept and risks.
+- [x] Phase 3: Implement Terraform networking and Cloud Run connectivity.
+- [x] Phase 4: Add deployment, validation, and teardown documentation.
+- [x] Phase 5: Run available validation and commit.
+- [ ] Phase 6: Validate and commit bootstrap branch-resume correction.
+- [ ] Phase 7: Validate and commit Alembic private-IP migration correction.
+
+## Journal
+
+### 2026-08-20 Private Cloud SQL Deployment
+
+- Validated deployed backend — **Finding**: Cloud SQL is `RUNNABLE` with only private IP `172.30.0.3`, but Alembic's connector was hard-coded to `IPTypes.PUBLIC`, producing `CloudSQLIPTypeError` with preference `PRIMARY` — **Decision**: make Alembic honor `CLOUD_SQL_IP_TYPE`; Cloud Shell remains unsuitable for private-only seeding, so use a VPC-attached Cloud Run Job after the corrected image deploys.
+- Validated fresh Cloud Shell checkout — **Finding**: Terraform initialized the private-networking module, then Google provider v7.45.0 rejected `project` on `google_service_networking_connection` — **Decision**: remove the unsupported attribute; the fully qualified VPC self link identifies the network-owning project.
+- Resumed deployment investigation — **Finding**: Cloud SQL apply attempted a public IP despite the feature branch setting `ipv4_enabled = false`; bootstrap reuses an existing checkout without previously switching it to the branch entered by the user — **Decision**: update a clean existing checkout to the selected branch before reuse and document the recovery commands.
+- Committed implementation — **Finding**: static diagnostics and Git whitespace checks pass, while `uv`, `terraform`, and `gcloud` are unavailable locally — **Decision**: commit the reviewed change without apply; require Cloud Shell Terraform formatting, validation, plan review, and runtime validation before deployment.
+- Implemented private connectivity — **Finding**: the application selected `IPTypes.PUBLIC` when `USE_CLOUD_SQL_AUTH_PROXY` was unset, despite mounting the Cloud SQL socket — **Decision**: configure the Python Connector with an explicit validated IP type and set `CLOUD_SQL_IP_TYPE=PRIVATE` in the deployment.
+- Added VPC resources — **Finding**: the provider configuration has no version constraints or lock file — **Decision**: use the broadly supported Serverless VPC Access connector rather than Direct VPC egress; use `PRIVATE_RANGES_ONLY` to avoid requiring Cloud NAT for Internet traffic.
+- Validation environment — **Finding**: neither `uv` nor `terraform` is installed in the local execution environment — **Decision**: record focused test and Terraform validation as pending Cloud Shell or a configured developer environment.
+- Inspected the organisation fork on `feature/KN-DATAX-15064-deploy-creative-studio` — **Finding**: existing untracked `AGENTS.md` and `README_monorepo.md` are user files and will remain untouched — **Decision**: make implementation changes only in tracked Creative Studio deployment files.
+- Traced the database path — **Finding**: the backend Cloud Run service mounts the Cloud SQL Auth Proxy socket at `/cloudsql/<instance-connection-name>` but has no VPC connectivity; PostgreSQL enables public IPv4 — **Decision**: retain the socket path and add private VPC connectivity for the backend only.
+- Reviewed provider declarations — **Finding**: the example environment does not constrain Google provider versions — **Decision**: use Serverless VPC Access connector fields rather than require Direct VPC egress support from an unknown provider version.

backend/alembic/env.py

@@ -115,7 +115,7 @@ async def alembic_get_connection():
         user=config_service.DB_USER,
         password=config_service.DB_PASS,
         db=config_service.DB_NAME,
-        ip_type=IPTypes.PUBLIC,
+        ip_type=IPTypes[config_service.CLOUD_SQL_IP_TYPE],
     )
     return conn
 

backend/src/config/config_service.py

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from typing import Any
+from typing import Any, Literal
 
 import google.auth
 from google.auth.exceptions import DefaultCredentialsError
@@ -67,6 +67,7 @@ class ConfigService(BaseSettings):
     DB_PASS: str = "password"
     DB_NAME: str = "creative_studio"
     USE_CLOUD_SQL_AUTH_PROXY: bool = False
+    CLOUD_SQL_IP_TYPE: Literal["PUBLIC", "PRIVATE"] = "PUBLIC"
     DB_HOST: str = "localhost"
     DB_PORT: str = "5432"
 

backend/src/database.py

@@ -131,7 +131,7 @@ async def get_connection():
         user=config_service.DB_USER,
         password=config_service.DB_PASS,
         db=config_service.DB_NAME,
-        ip_type=IPTypes.PUBLIC,  # Adjust if using Private IP
+        ip_type=IPTypes[config_service.CLOUD_SQL_IP_TYPE],
     )
 
     return conn
@@ -196,7 +196,7 @@ async def get_conn():
                     user=config_service.DB_USER,
                     password=config_service.DB_PASS,
                     db=config_service.DB_NAME,
-                    ip_type=IPTypes.PUBLIC,
+                    ip_type=IPTypes[config_service.CLOUD_SQL_IP_TYPE],
                 )
 
             self.engine = create_async_engine(

backend/tests/test_database.py

@@ -17,6 +17,7 @@
 from unittest.mock import AsyncMock, MagicMock, patch
 
 import pytest
+from google.cloud.sql.connector import IPTypes
 
 from src.config.config_service import config_service
 from src.database import (
@@ -131,6 +132,7 @@ async def test_get_connection_cloud_sql():
             "INSTANCE_CONNECTION_NAME",
             "projects/p/locations/l/instances/i",
         ),
+        patch.object(config_service, "CLOUD_SQL_IP_TYPE", "PRIVATE"),
     ):
         mock_connector = AsyncMock()
         mock_connector.connect_async = AsyncMock(return_value="cloud_conn")
@@ -143,6 +145,14 @@ async def test_get_connection_cloud_sql():
 
             res = await get_connection()
             assert res == "cloud_conn"
+            mock_connector.connect_async.assert_awaited_once_with(
+                "projects/p/locations/l/instances/i",
+                "asyncpg",
+                user=config_service.DB_USER,
+                password=config_service.DB_PASS,
+                db=config_service.DB_NAME,
+                ip_type=IPTypes.PRIVATE,
+            )
 
 
 @pytest.mark.anyio

bootstrap.sh

@@ -349,14 +349,31 @@ setup_repo() {
     if [[ -d "$REPO_CLONE_DIR" ]]; then
         warn "Directory '$REPO_CLONE_DIR' already exists."; prompt "Do you want to use this existing directory? (y/n)"; read -r REPLY < /dev/tty
         if [[ ! $REPLY =~ ^[Yy]$ ]]; then fail "Please remove the directory or run the script from a different location."; fi
+
+        cd "$REPO_CLONE_DIR"
+        git rev-parse --is-inside-work-tree > /dev/null 2>&1 || fail "Existing directory '$REPO_CLONE_DIR' is not a Git repository."
+        if ! git diff --quiet || ! git diff --cached --quiet; then
+            fail "Existing repository has uncommitted changes. Commit, stash, or use a new directory before continuing."
+        fi
+        if [[ -n "$(git ls-files --others --exclude-standard)" ]]; then
+            fail "Existing repository has untracked files. Move them or use a new directory before continuing."
+        fi
+        info "Updating existing checkout to branch '$SELECTED_BRANCH'..."
+        git fetch origin "$SELECTED_BRANCH"
+        if git show-ref --verify --quiet "refs/heads/$SELECTED_BRANCH"; then
+            git checkout "$SELECTED_BRANCH"
+        else
+            git checkout --track "origin/$SELECTED_BRANCH"
+        fi
+        git pull --ff-only origin "$SELECTED_BRANCH"
     else
         info "Performing a sparse checkout of '$REPO_CLONE_DIR' (Branch: $SELECTED_BRANCH)..."
 
         # 1. Clone with -b branch_name
         git clone --filter=blob:none --no-checkout --depth 1 --sparse -b "$SELECTED_BRANCH" "$GITHUB_REPO_URL" "$REPO_CLONE_DIR"
 
         cd "$REPO_CLONE_DIR"
-        
+
         # 2. Sparse checkout for ROOT folders only
         git sparse-checkout set "infra" "backend" "frontend" "bootstrap.sh"