Private ip cloudsql

PRIVATE_IP_SETUP.md

@@ -0,0 +1,160 @@
+# Google Cloud Creative Studio: Private IP Deployment Guide
+
+This guide details how to deploy Creative Studio with a **Private IP only** Cloud SQL database and configure the **Cloud Run** backend to access it securely using **Direct VPC egress**. 
+
+Because the database does not expose a public IP, the initial setup is split into a **two-step lifecycle** to comply with network security boundaries.
+
+---
+
+## Architecture Overview
+1.  **Private Database:** The Cloud SQL Postgres database has public access disabled (`ipv4_enabled = false`) and uses VPC Peering (`servicenetworking.googleapis.com`) to receive connections.
+2.  **Serverless Egress:** Cloud Run backend service is connected directly to the VPC subnet using Direct VPC egress (`egress = "ALL_TRAFFIC"`), allowing it to query the database internally.
+3.  **Secure Seeding:** Database seeding is performed using a temporary jump-box VM residing inside the VPC to securely load templates and assets.
+
+---
+
+## Setup Lifecycle (Step-by-Step)
+
+### Phase 1: Deploy Infrastructure
+Run the bootstrap script from your standard **Google Cloud Shell** session to deploy the network, database, secrets, and Cloud Build triggers:
+
+```bash
+curl -sSL https://raw.githubusercontent.com/PKAgarwal157/gcc-creative-studio/private-ip-cloudsql/bootstrap.sh | bash
+```
+*   When prompted for your fork URL, enter: `https://github.com/PKAgarwal157/gcc-creative-studio.git`
+*   When prompted for the branch, enter: `private-ip-cloudsql`
+*   *Note: This script will complete successfully but will skip the database seeding step, which must be run internally inside the VPC.*
+
+---
+
+### Phase 2: Grant Seeding Permissions
+The temporary setup VM will run as the project's **Default Compute Engine Service Account**. Before creating the VM, run these commands in your **Cloud Shell** to grant the service account access to Secret Manager, Cloud SQL, and the Asset Storage bucket:
+
+```bash
+# Get your project details
+export PROJECT_ID=$(gcloud config get project)
+export PROJECT_NUM=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
+export COMPUTE_SA="${PROJECT_NUM}-compute@developer.gserviceaccount.com"
+
+# 1. Grant Secret Manager Access (to retrieve DB password)
+gcloud secrets add-iam-policy-binding "creative-studio-db-password" \
+    --role="roles/secretmanager.secretAccessor" \
+    --member="serviceAccount:$COMPUTE_SA" \
+    --project="$PROJECT_ID"
+
+# 2. Grant Cloud SQL Client Access (to run the DB proxy)
+gcloud projects add-iam-policy-binding "$PROJECT_ID" \
+    --role="roles/cloudsql.client" \
+    --member="serviceAccount:$COMPUTE_SA"
+
+# 3. Grant Cloud SQL Viewer Access (to locate the DB instance connection name)
+gcloud projects add-iam-policy-binding "$PROJECT_ID" \
+    --role="roles/cloudsql.viewer" \
+    --member="serviceAccount:$COMPUTE_SA"
+
+# 4. Grant GCS Object Admin Access (to upload asset templates)
+gcloud projects add-iam-policy-binding "$PROJECT_ID" \
+    --role="roles/storage.objectAdmin" \
+    --member="serviceAccount:$COMPUTE_SA"
+```
+
+---
+
+### Phase 3: Run Database Seeding inside the VPC
+Since your local terminal cannot reach the private IP of the database, you must run the seeding script from a temporary VM inside the VPC.
+
+#### 1. Setup Temporary Networking & Firewall (in Cloud Shell)
+Create a temporary Cloud Router, NAT Gateway (so the VM can pull dependencies from GitHub/npm), and a firewall rule to allow IAP SSH connections:
+
+```bash
+# Create Router
+gcloud compute routers create temp-router \
+    --network="cs-vpc-development" \
+    --region="us-central1"
+
+# Create NAT Gateway
+gcloud compute routers nats create temp-nat \
+    --router="temp-router" \
+    --region="us-central1" \
+    --nat-custom-subnet-ip-ranges="cs-subnet-development" \
+    --auto-allocate-nat-external-ips
+
+# Allow Ingress from Google IAP range to port 22
+gcloud compute firewall-rules create temp-allow-iap-ssh \
+    --network="cs-vpc-development" \
+    --allow=tcp:22 \
+    --source-ranges="35.235.240.0/20"
+```
+
+#### 2. Create the Temporary VM (in Cloud Shell)
+Create a private VM (using `--no-address` to comply with external IP blocks, and Shielded VM configurations to comply with secure boot org policies):
+
+```bash
+gcloud compute instances create temp-seed-vm \
+    --zone="us-central1-a" \
+    --machine-type="e2-micro" \
+    --network="cs-vpc-development" \
+    --subnet="cs-subnet-development" \
+    --no-address \
+    --scopes="https://www.googleapis.com/auth/cloud-platform" \
+    --shielded-secure-boot \
+    --shielded-vtpm \
+    --shielded-integrity-monitoring \
+    --metadata="startup-script=sudo apt-get update && sudo apt-get install -y git"
+```
+
+#### 3. Tunnel SSH and Run the Seeding Script
+SSH into the private VM, clone the repository, and run the standalone seeding script:
+
+```bash
+# SSH into the VM (IAP tunnels securely to the private IP)
+gcloud compute ssh temp-seed-vm --zone="us-central1-a" --tunnel-through-iap
+
+# --- Inside the VM Session ---
+# Clone the repository
+git clone -b private-ip-cloudsql https://github.com/PKAgarwal157/gcc-creative-studio.git ~/gcc-creative-studio
+
+# Navigate and execute the seeding script
+cd ~/gcc-creative-studio
+chmod +x seed_only.sh
+./seed_only.sh
+
+# Exit VM when complete
+exit
+```
+
+---
+
+### Phase 4: Clean Up & Trigger Deployments
+
+#### 1. Delete Temporary Resources (in Cloud Shell)
+Once seeding is complete, delete the temporary VM and network pathways to avoid costs:
+
+```bash
+# Delete VM
+gcloud compute instances delete temp-seed-vm --zone="us-central1-a" --quiet
+
+# Delete Firewall Rule
+gcloud compute firewall-rules delete temp-allow-iap-ssh --quiet
+
+# Delete NAT Gateway and Router
+gcloud compute routers nats delete temp-nat --router="temp-router" --region="us-central1" --quiet
+gcloud compute routers delete temp-router --region="us-central1" --quiet
+```
+
+#### 2. Optional: Manually Trigger Initial Builds (in Cloud Shell)
+If you did not trigger the builds during the `bootstrap.sh` script execution (or if you need to redeploy the containers), you can trigger the Cloud Build pipelines manually using these commands:
+
+```bash
+# Trigger Backend
+gcloud builds triggers run "cstudio-be-trigger" \
+    --branch="private-ip-cloudsql" \
+    --project="$PROJECT_ID" \
+    --region="us-central1"
+
+# Trigger Frontend
+gcloud builds triggers run "${PROJECT_ID}-trigger" \
+    --branch="private-ip-cloudsql" \
+    --project="$PROJECT_ID" \
+    --region="us-central1"
+```

backend/alembic/env.py

@@ -101,7 +101,7 @@ async def alembic_get_connection():
         user=config_service.DB_USER,
         password=config_service.DB_PASS,
         db=config_service.DB_NAME,
-        ip_type=IPTypes.PUBLIC,
+        ip_type=IPTypes.PRIVATE if config_service.DB_IP_TYPE == "PRIVATE" else IPTypes.PUBLIC,
     )
     return conn
 

backend/src/config/config_service.py

@@ -69,6 +69,7 @@ class ConfigService(BaseSettings):
     USE_CLOUD_SQL_AUTH_PROXY: bool = False
     DB_HOST: str = "localhost"
     DB_PORT: str = "5432"
+    DB_IP_TYPE: str = "PUBLIC"
 
     # --- Veo ---
     VEO_MODEL_ID: str = "veo-2.0-generate-001"

backend/src/database.py

@@ -131,7 +131,7 @@ async def get_connection():
         user=config_service.DB_USER,
         password=config_service.DB_PASS,
         db=config_service.DB_NAME,
-        ip_type=IPTypes.PUBLIC,  # Adjust if using Private IP
+        ip_type=IPTypes.PRIVATE if config_service.DB_IP_TYPE == "PRIVATE" else IPTypes.PUBLIC,
     )
 
     return conn
@@ -196,7 +196,7 @@ async def get_conn():
                     user=config_service.DB_USER,
                     password=config_service.DB_PASS,
                     db=config_service.DB_NAME,
-                    ip_type=IPTypes.PUBLIC,
+                    ip_type=IPTypes.PRIVATE if config_service.DB_IP_TYPE == "PRIVATE" else IPTypes.PUBLIC,
                 )
 
             self.engine = create_async_engine(

bootstrap.sh

@@ -157,7 +157,7 @@ start_sql_proxy() {
     fi
 
     # 3. Start Proxy in Background (Port 5432)
-    ./cloud-sql-proxy --address 0.0.0.0 --port 5432 "$DB_INSTANCE_NAME" > /dev/null 2>&1 &
+    ./cloud-sql-proxy --address 0.0.0.0 --port 5432 --private-ip "$DB_INSTANCE_NAME" > /dev/null 2>&1 &
     PROXY_PID=$!
     export PROXY_PID
 
@@ -449,7 +449,7 @@ configure_environment() {
 
 handle_manual_steps() {
     step 6 "Manual Steps Required"; cd "$REPO_ROOT/infra"; TFVARS_FILE_PATH="$ENV_DIR/$ENV_NAME.tfvars"
-    info "Enabling required Google Cloud APIs..."; gcloud services enable cloudbuild.googleapis.com secretmanager.googleapis.com firebase.googleapis.com iap.googleapis.com identitytoolkit.googleapis.com texttospeech.googleapis.com workflows.googleapis.com --project="$GCP_PROJECT_ID"
+    info "Enabling required Google Cloud APIs..."; gcloud services enable cloudbuild.googleapis.com secretmanager.googleapis.com firebase.googleapis.com iap.googleapis.com identitytoolkit.googleapis.com texttospeech.googleapis.com workflows.googleapis.com servicenetworking.googleapis.com --project="$GCP_PROJECT_ID"
     if [ -z "$GITHUB_CONN_NAME" ]; then
         prompt "\nDo you already have a Cloud Build Host Connection for GitHub in this project? (y/n)"; read -r REPLY < /dev/tty
         if [[ $REPLY =~ ^[Yy]$ ]]; then prompt "Please enter the existing connection name:"; read -p "   Connection Name: " GITHUB_CONN_NAME < /dev/tty
@@ -793,7 +793,6 @@ main() {
         "populate_oauth_secrets"
         "update_oauth_client"
         "update_secrets"
-        "seed_data"
         "trigger_builds" 
     )
     for i in "${!steps_to_run[@]}"; do

infra/environments/dev-infra-example/dev.tfvars

@@ -21,6 +21,7 @@ frontend_custom_audiences = ["YOUR_OAUTH_WEB_CLIENT_ID_HERE", "YOUR_GCP_PROJECT_
 be_env_vars = {
   common = {
     LOG_LEVEL = "INFO"
+    DB_IP_TYPE = "PRIVATE"
   }
   development = {
     ENVIRONMENT  = "development"

infra/modules/cloud-run-service/main.tf

@@ -39,6 +39,14 @@ resource "google_cloud_run_v2_service" "this" {
 
   template {
     service_account = google_service_account.run_sa.email
+
+    vpc_access {
+      network_interfaces {
+        network    = var.vpc_network_id
+        subnetwork = var.vpc_subnetwork_id
+      }
+      egress = "ALL_TRAFFIC"
+    }
     volumes {
       name = "cloudsql"
       cloud_sql_instance {

infra/modules/cloud-run-service/variables.tf

@@ -129,3 +129,14 @@ variable "db_secret_id" {
 }
 variable "db_name" { type = string }
 variable "db_user" { type = string }
+
+variable "vpc_network_id" {
+  description = "The ID of the VPC network"
+  type        = string
+}
+
+variable "vpc_subnetwork_id" {
+  description = "The ID of the subnetwork"
+  type        = string
+}
+

infra/modules/network/main.tf

@@ -0,0 +1,49 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+resource "google_compute_network" "vpc" {
+  name                    = "cs-vpc-${var.environment}"
+  auto_create_subnetworks = false
+  project                 = var.project_id
+}
+
+resource "google_compute_subnetwork" "subnet" {
+  name          = "cs-subnet-${var.environment}"
+  ip_cidr_range = "10.0.0.0/24"
+  region        = var.region
+  network       = google_compute_network.vpc.id
+  project       = var.project_id
+  private_ip_google_access = true
+
+  log_config {
+    aggregation_interval = "INTERVAL_5_SEC"
+    flow_sampling        = 0.5
+    metadata             = "INCLUDE_ALL_METADATA"
+  }
+}
+
+resource "google_compute_global_address" "private_ip_address" {
+  name          = "cs-private-ip-address"
+  purpose       = "VPC_PEERING"
+  address_type  = "INTERNAL"
+  prefix_length = 16
+  network       = google_compute_network.vpc.id
+  project       = var.project_id
+}
+
+resource "google_service_networking_connection" "private_vpc_connection" {
+  network                 = google_compute_network.vpc.id
+  service                 = "servicenetworking.googleapis.com"
+  reserved_peering_ranges = [google_compute_global_address.private_ip_address.name]
+}

infra/modules/network/outputs.tf

@@ -0,0 +1,33 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+output "network_id" {
+  description = "The ID of the VPC network"
+  value       = google_compute_network.vpc.id
+}
+
+output "network_name" {
+  description = "The name of the VPC network"
+  value       = google_compute_network.vpc.name
+}
+
+output "subnetwork_id" {
+  description = "The ID of the subnetwork"
+  value       = google_compute_subnetwork.subnet.id
+}
+
+output "private_vpc_connection_id" {
+  description = "The ID of the private VPC connection"
+  value       = google_service_networking_connection.private_vpc_connection.id
+}

infra/modules/network/variables.tf

@@ -0,0 +1,28 @@
+# Copyright 2025 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+variable "project_id" {
+  type        = string
+  description = "The GCP Project ID."
+}
+
+variable "region" {
+  type        = string
+  description = "The region to deploy GKE and network resources."
+}
+
+variable "environment" {
+  type        = string
+  description = "The deployment environment (e.g., dev, prod)."
+}