Google Cloud Platform Deployment: Lessons Learned and Best Practices

Status: Complete
Generated: 2025-02-01
Purpose: Comprehensive guide for deploying projects to Google Cloud Platform based on real-world experience

Executive Summary

This document captures all lessons learned, tips, and best practices from successfully deploying a Flask/Python application to Google Cloud Run. The deployment achieved 100% cost savings compared to traditional hosting while maintaining high performance and reliability.

Table of Contents

1. Architecture Decisions
2. Pre-Deployment Preparation
3. Container Configuration
4. Deployment Automation
5. Secret Management
6. Database Strategy
7. Cost Optimization
8. Monitoring and Health Checks
9. Common Pitfalls and Solutions
10. Performance Optimization
11. Security Best Practices
12. Multi-Cloud Strategy

Architecture Decisions

Key Decision: Choose Cloud Run Over Other GCP Services

Why Cloud Run Won:

• Serverless: No infrastructure management
• Container-native: Full control over runtime environment
• Auto-scaling: From 0 to 1000+ instances automatically
• Cost-effective: Pay only for actual usage
• Global reach: 35+ regions available

Architecture Pattern: Database Abstraction

Lesson Learned: Design for database flexibility from the start.

# Good: Database abstraction layer
class DatabaseManager:
    def __init__(self):
        self.active_db = os.environ.get('ACTIVE_DATABASE', 'postgres')
        
    def get_connection(self):
        if self.active_db == 'postgres':
            return self.postgres_connection
        elif self.active_db == 'mysql':
            return self.mysql_connection

Benefits:

• Switch databases without code changes
• Support multiple cloud providers
• Enable gradual migration

Pre-Deployment Preparation

Essential Checklist

Before deploying to GCP, ensure:

1. Google Cloud Setup

   # Install Google Cloud SDK
   curl https://sdk.cloud.google.com | bash
   
   # Authenticate
   gcloud auth login
   
   # Set project
   gcloud config set project YOUR_PROJECT_ID
   

2. Enable Required APIs

   gcloud services enable \
     run.googleapis.com \
     cloudbuild.googleapis.com \
     secretmanager.googleapis.com \
     logging.googleapis.com \
     monitoring.googleapis.com
   

3. Service Account Creation

   gcloud iam service-accounts create cloud-run-sa \
     --display-name="Cloud Run Service Account"
   

Environment Detection

Critical Lesson: Implement platform detection for environment-specific configuration.

def detect_platform():
    """Detect deployment platform from environment"""
    if os.environ.get('K_SERVICE'):
        return 'google-cloud-run'
    elif os.environ.get('DIGITALOCEAN_APP_ID'):
        return 'digitalocean'
    elif os.environ.get('RAILWAY_ENVIRONMENT'):
        return 'railway'
    else:
        return 'local'

Container Configuration

Dockerfile Best Practices

Lesson: Optimize for Cloud Run's specific requirements.

# Dockerfile.cloudrun
FROM python:3.10-slim

# Cloud Run specific environment variables
ENV PORT=8080
ENV PYTHONUNBUFFERED=1
ENV DEPLOYMENT_PLATFORM=google-cloud-run

# Use /tmp for writable storage (Cloud Run has read-only filesystem)
ENV UPLOAD_FOLDER=/tmp/uploads

# Install system dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    postgresql-client \
    && rm -rf /var/lib/apt/lists/*

# Copy and install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . /app
WORKDIR /app

# Health check for Cloud Run
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8080/health || exit 1

# Use PORT environment variable (required by Cloud Run)
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 app:app

Key Container Lessons

1. Always use PORT environment variable - Cloud Run assigns dynamic ports
2. Use /tmp for file storage - Only /tmp is writable in Cloud Run
3. Set PYTHONUNBUFFERED=1 - Ensures logs appear immediately
4. Include health checks - Improves deployment reliability
5. Use slim base images - Reduces cold start time

Deployment Automation

Enhanced Deployment Script

Major Lesson: Invest in robust deployment automation early.

#!/bin/bash
# deploy-gcp.sh

set -euo pipefail  # Exit on error, undefined variables

# Color-coded logging
log_info() { echo -e "\033[0;36m[INFO]\033[0m $1"; }
log_success() { echo -e "\033[0;32m[SUCCESS]\033[0m $1"; }
log_error() { echo -e "\033[0;31m[ERROR]\033[0m $1"; }

# Error handling
trap 'log_error "Deployment failed on line $LINENO"' ERR

# Build metadata
BUILD_TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
BUILD_VERSION="v$(date +%Y%m%d-%H%M%S)"
GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")

# Validate environment
validate_environment() {
    log_info "Validating environment..."
    
    # Check Google Cloud auth
    if ! gcloud auth list --filter=status:ACTIVE --format="value(account)" &>/dev/null; then
        log_error "Not authenticated with Google Cloud"
        exit 1
    fi
    
    # Verify secrets exist
    for secret in supabase-url supabase-anon-key flask-secret-key; do
        if ! gcloud secrets describe $secret &>/dev/null; then
            log_error "Secret '$secret' not found. Run setup-gcp-secrets.sh first"
            exit 1
        fi
    done
    
    log_success "Environment validated"
}

# Build and deploy
deploy() {
    log_info "Starting deployment..."
    
    # Build with Cloud Build
    gcloud builds submit \
        --config=cloudbuild.yaml \
        --substitutions=_BUILD_TIMESTAMP="$BUILD_TIMESTAMP",_GIT_COMMIT="$GIT_COMMIT"
    
    # Deploy to Cloud Run
    gcloud run deploy my-app \
        --image gcr.io/$PROJECT_ID/my-app:latest \
        --platform managed \
        --region us-central1 \
        --allow-unauthenticated \
        --set-env-vars="BUILD_VERSION=$BUILD_VERSION,GIT_COMMIT=$GIT_COMMIT"
    
    # Verify deployment
    SERVICE_URL=$(gcloud run services describe my-app --region=us-central1 --format='value(status.url)')
    
    log_info "Testing health endpoint..."
    if curl -s "$SERVICE_URL/health" | jq -e '.status == "healthy"' > /dev/null; then
        log_success "Deployment successful!"
        log_info "Service URL: $SERVICE_URL"
    else
        log_error "Health check failed"
        exit 1
    fi
}

# Main execution
validate_environment
deploy

Cloud Build Configuration

# cloudbuild.yaml
steps:
  # Build the container image
  - name: 'gcr.io/cloud-builders/docker'
    args: [
      'build',
      '--build-arg', 'BUILD_TIMESTAMP=${_BUILD_TIMESTAMP}',
      '--build-arg', 'GIT_COMMIT=${_GIT_COMMIT}',
      '-t', 'gcr.io/$PROJECT_ID/$REPO_NAME:latest',
      '-t', 'gcr.io/$PROJECT_ID/$REPO_NAME:${SHORT_SHA}',
      '-f', 'Dockerfile.cloudrun',
      '.'
    ]
  
  # Push to Container Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', '--all-tags', 'gcr.io/$PROJECT_ID/$REPO_NAME']

options:
  machineType: 'E2_HIGHCPU_8'
  logging: CLOUD_LOGGING_ONLY
  
substitutions:
  _BUILD_TIMESTAMP: '${BUILD_TIMESTAMP}'
  _GIT_COMMIT: '${SHORT_SHA}'

Secret Management

Secret Manager Best Practices

Critical Lesson: Never hardcode credentials, even in setup scripts.

#!/bin/bash
# setup-gcp-secrets.sh

# Prompt for secrets instead of hardcoding
read_secret() {
    local secret_name=$1
    local prompt=$2
    echo -n "$prompt: "
    read -s value
    echo
    echo "$value" | gcloud secrets create $secret_name --data-file=- 2>/dev/null || \
    echo "$value" | gcloud secrets versions add $secret_name --data-file=-
}

# Create secrets interactively
read_secret "supabase-url" "Enter Supabase URL"
read_secret "supabase-anon-key" "Enter Supabase Anonymous Key"
read_secret "flask-secret-key" "Enter Flask Secret Key"

# Grant permissions to Cloud Run service account
PROJECT_ID=$(gcloud config get-value project)
SERVICE_ACCOUNT="cloud-run-sa@${PROJECT_ID}.iam.gserviceaccount.com"

for secret in supabase-url supabase-anon-key flask-secret-key; do
    gcloud secrets add-iam-policy-binding $secret \
        --member="serviceAccount:${SERVICE_ACCOUNT}" \
        --role="roles/secretmanager.secretAccessor"
done

Service Configuration with Secrets

# cloudrun-service.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: my-app
spec:
  template:
    metadata:
      annotations:
        run.googleapis.com/execution-environment: gen2
    spec:
      serviceAccountName: cloud-run-sa@PROJECT_ID.iam.gserviceaccount.com
      containers:
      - image: gcr.io/PROJECT_ID/my-app:latest
        env:
        - name: SUPABASE_URL
          valueFrom:
            secretKeyRef:
              name: supabase-url
              key: latest
        - name: FLASK_SECRET_KEY
          valueFrom:
            secretKeyRef:
              name: flask-secret-key
              key: latest

Database Strategy

🗄️ PostgreSQL-Only Architecture

Major Lesson: Simplify by using a single database technology.

Benefits Realized:

• Eliminated MySQL dependency and costs
• Simplified connection management
• Leveraged Supabase's built-in features
• Reduced operational complexity

# Database connection with pooling
import psycopg2
from psycopg2 import pool

class DatabasePool:
    def __init__(self):
        self.pool = psycopg2.pool.SimpleConnectionPool(
            1, 20,  # min and max connections
            host=os.environ.get('PG_HOST'),
            database=os.environ.get('PG_DATABASE'),
            user=os.environ.get('PG_USER'),
            password=os.environ.get('PG_PASSWORD'),
            port=6543  # Supabase pooler port
        )
    
    def get_connection(self):
        return self.pool.getconn()
    
    def return_connection(self, conn):
        self.pool.putconn(conn)

Connection Resilience

import time
from functools import wraps

def retry_database(max_attempts=3, delay=1):
    """Decorator for database operation retry logic"""
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except (psycopg2.OperationalError, psycopg2.InterfaceError) as e:
                    if attempt < max_attempts - 1:
                        time.sleep(delay * (2 ** attempt))  # Exponential backoff
                        continue
                    raise
            return None
        return wrapper
    return decorator

@retry_database()
def execute_query(query, params=None):
    """Execute database query with automatic retry"""
    conn = db_pool.get_connection()
    try:
        with conn.cursor() as cursor:
            cursor.execute(query, params)
            return cursor.fetchall()
    finally:
        db_pool.return_connection(conn)

Cost Optimization

Achieving Zero-Cost Deployment

Key Strategies That Worked:

1. Leverage Free Tier Fully

   • 2 million requests/month FREE
   • 180,000 vCPU-seconds FREE
   • 360,000 GiB-seconds FREE

2. Optimize Resource Allocation

   resources:
     limits:
       cpu: "2"
       memory: "2Gi"
   

3. Scale to Zero When Idle

   autoscaling:
     minInstances: 0  # Scale to zero when no traffic
     maxInstances: 10  # Cap maximum spending
   

4. Use Existing Infrastructure

   • Leverage existing Supabase database
   • No additional database costs
   • Use Cloud Build free tier

Cost Monitoring

# Set up budget alerts
gcloud billing budgets create \
  --billing-account=BILLING_ACCOUNT_ID \
  --display-name="Cloud Run Budget" \
  --budget-amount=10 \
  --threshold-rule=percent=50 \
  --threshold-rule=percent=90

Monitoring and Health Checks

Comprehensive Health Endpoint

Lesson: Implement detailed health checks for better observability.

@app.route('/health')
def health_check():
    """Comprehensive health check endpoint"""
    health_status = {
        'status': 'healthy',
        'timestamp': datetime.utcnow().isoformat(),
        'platform': detect_platform(),
        'version': os.environ.get('BUILD_VERSION', 'unknown'),
        'checks': {}
    }
    
    # Database check
    try:
        db_manager.test_connection()
        health_status['checks']['database'] = 'healthy'
    except Exception as e:
        health_status['checks']['database'] = f'unhealthy: {str(e)}'
        health_status['status'] = 'degraded'
    
    # Memory check
    import psutil
    memory = psutil.virtual_memory()
    health_status['checks']['memory'] = {
        'percent_used': memory.percent,
        'available_mb': memory.available / (1024 * 1024)
    }
    
    # Disk check (for /tmp)
    disk = psutil.disk_usage('/tmp')
    health_status['checks']['tmp_storage'] = {
        'percent_used': disk.percent,
        'available_mb': disk.free / (1024 * 1024)
    }
    
    # Return appropriate status code
    status_code = 200 if health_status['status'] == 'healthy' else 503
    return jsonify(health_status), status_code

Monitoring Setup

# monitoring.yaml - Cloud Monitoring alert policy
displayName: "Cloud Run High Error Rate"
conditions:
  - displayName: "Error rate above 1%"
    conditionThreshold:
      filter: |
        resource.type="cloud_run_revision"
        resource.labels.service_name="my-app"
        metric.type="run.googleapis.com/request_count"
        metric.labels.response_code_class!="2xx"
      comparison: COMPARISON_GT
      thresholdValue: 0.01
      duration: 60s

Common Pitfalls and Solutions

Critical Issues We Encountered

Filesystem Handling

Problem: Cloud Run has read-only filesystem except /tmp

def get_safe_upload_path(filename):
    """Get writable path for file uploads"""
    if detect_platform() == 'google-cloud-run':
        # Use /tmp for Cloud Run
        upload_dir = '/tmp/uploads'
    else:
        # Use local directory for other platforms
        upload_dir = 'uploads'
    
    os.makedirs(upload_dir, exist_ok=True)
    return os.path.join(upload_dir, filename)

Graceful Degradation

def get_config_value(key, default=None, required=False):
    """Safely get configuration values"""
    value = os.environ.get(key, default)
    
    if required and value is None:
        # Log error but don't crash
        app.logger.error(f"Required config '{key}' is missing")
        # Use fallback or feature flag
        if key.startswith('FEATURE_'):
            return False  # Disable feature
        else:
            raise ValueError(f"Required configuration '{key}' not set")
    
    return value

Performance Optimization

Cold Start Optimization

Lessons for Reducing Cold Starts:

1. Use slim base images

   # Good: 124MB
   FROM python:3.10-slim
   
   # Bad: 884MB
   FROM python:3.10
   

2. Minimize dependencies

   # requirements.txt - only essentials
   flask==2.3.0
   gunicorn==20.1.0
   psycopg2-binary==2.9.6  # Use binary version
   

3. Lazy loading

   # Load heavy libraries only when needed
   def process_pdf():
       import PyPDF2  # Import here, not at module level
       # Process PDF
   

4. Keep containers warm

   autoscaling:
     minInstances: 1  # Always keep one warm
   

Response Time Optimization

# Enable response compression
from flask_compress import Compress

app = Flask(__name__)
Compress(app)

# Cache static responses
from functools import lru_cache

@lru_cache(maxsize=128)
def get_cached_data(key):
    """Cache frequently accessed data"""
    return expensive_database_query(key)

# Use connection pooling
from werkzeug.middleware.proxy_fix import ProxyFix
app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1)

Security Best Practices

Security Hardening

1. Environment Variable Security

   # Never log sensitive values
   def log_config():
       config = {}
       for key in os.environ:
           if any(secret in key.lower() for secret in ['key', 'password', 'token']):
               config[key] = '***REDACTED***'
           else:
               config[key] = os.environ[key]
       app.logger.info(f"Configuration: {config}")
   

2. Request Validation

   from flask_limiter import Limiter
   
   limiter = Limiter(
       app,
       key_func=lambda: request.remote_addr,
       default_limits=["200 per day", "50 per hour"]
   )
   
   @app.route('/api/sensitive')
   @limiter.limit("5 per minute")
   def sensitive_endpoint():
       pass
   

3. Security Headers

   @app.after_request
   def set_security_headers(response):
       response.headers['X-Content-Type-Options'] = 'nosniff'
       response.headers['X-Frame-Options'] = 'DENY'
       response.headers['X-XSS-Protection'] = '1; mode=block'
       response.headers['Strict-Transport-Security'] = 'max-age=31536000'
       return response
   

Multi-Cloud Strategy

Platform Agnostic Design

Key Lesson: Design for portability from the start.

class CloudAdapter:
    """Adapter pattern for multi-cloud support"""
    
    @staticmethod
    def get_adapter():
        platform = detect_platform()
        
        if platform == 'google-cloud-run':
            return GoogleCloudAdapter()
        elif platform == 'aws-lambda':
            return AWSAdapter()
        elif platform == 'azure-functions':
            return AzureAdapter()
        else:
            return LocalAdapter()
    
    def get_storage_path(self):
        raise NotImplementedError
    
    def get_secret(self, key):
        raise NotImplementedError
    
    def log_metric(self, name, value):
        raise NotImplementedError

class GoogleCloudAdapter(CloudAdapter):
    def get_storage_path(self):
        return '/tmp'
    
    def get_secret(self, key):
        from google.cloud import secretmanager
        client = secretmanager.SecretManagerServiceClient()
        name = f"projects/{PROJECT_ID}/secrets/{key}/versions/latest"
        return client.access_secret_version(request={"name": name}).payload.data.decode()

Quick Reference Checklist

Pre-Deployment

• Google Cloud SDK installed and authenticated
• Required APIs enabled
• Service account created with proper permissions
• Secrets configured in Secret Manager
• Dockerfile optimized for Cloud Run
• Health endpoints implemented
• Environment detection in place
• Database abstraction layer ready

Deployment

• Build metadata generation
• Automated deployment script tested
• Cloud Build configuration ready
• Service YAML configured
• Resource limits set appropriately
• Autoscaling configured
• Health checks passing

Post-Deployment

• Monitoring alerts configured
• Budget alerts set up
• Logs aggregation working
• Performance metrics baseline established
• Security headers verified
• Load testing completed
• Rollback procedure documented

Conclusion

Deploying to Google Cloud Run can achieve dramatic cost savings (95-100% reduction) while improving scalability and reliability. The key is proper preparation, robust automation, and learning from common pitfalls.

Top 5 Most Important Lessons:

1. Automate everything - Invest in deployment scripts early
2. Abstract database access - Enable multi-cloud flexibility
3. Handle secrets properly - Never hardcode, always use Secret Manager
4. Design for serverless - Understand platform limitations (read-only filesystem, dynamic ports)
5. Monitor comprehensively - Implement detailed health checks and logging

With these lessons and best practices, you can confidently deploy any application to Google Cloud Platform and achieve similar success.

Document History

Related Documentation

• Google Cloud Deployment Guide
• Deployment Success Report
• Docker Local Development Guide