Save $12K Monthly by Migrating AI from Google to Claude

Google Vertex AI works. However, when you see three startups in one week moving their models to Claude—all citing more predictable inference costs, more generous rate limits, and better contextual quality responses—you wonder if there's a broader trend. By 2026, switching from one cloud AI provider to another is no longer a six-month project with entire teams dedicated to it. Terraform, a somewhat overlooked infrastructure as code tool deemed "too DevOps," has become an essential shortcut. It lets you switch your model stack in under 72 hours without disrupting production.

Photo: Growtika on Unsplash

This tutorial doesn't promise magic. But it will show you how three technical founders I know rewrote their infrastructure from Google Cloud to Anthropic using Terraform. They managed to save between $8,000 and $15,000 monthly on API costs and reduce average latency from 2.4 seconds to 1.7 seconds. The key isn't that Claude is objectively "better"—it's that Terraform allows you to test both providers in parallel, measure the real impact, and execute the switch with a single command.

Why Terraform and Not Manual Migration Scripts

Migrating your AI infrastructure again might tempt you to use a Python script to translate configurations from Google Cloud to Anthropic. Three startups tried this in Q3 of 2025. All ended up with inconsistent states between production and staging, poorly managed secrets, and manual rollbacks that took days.

Terraform solves this with declarative state management. Instead of saying "execute these steps," you describe the final infrastructure you want. Terraform automatically figures out what needs to be created, updated, or deleted. Migrating from Vertex AI to Claude means you can keep both providers active simultaneously, direct 10% of traffic to the new model, measure real results, and instantly roll back if something fails—without custom scripts only you understand.

The second critical benefit is reproducibility. A fintech startup in Barcelona moved its models from Google to Anthropic in staging using Terraform. It worked seamlessly. Two weeks later, they applied the exact same Terraform plan to production—without surprises or rewrites. The Terraform state file ensures that what you applied in one environment replicates exactly in another.

Previous Architecture: How Your Models Connect to Google Now

Photo: Hazel Z on Unsplash

Before touching a line of Terraform, you need to map how your application consumes Google models. Most startups using Vertex AI follow this structure:

Frontend → API Gateway → Backend (Node/Python) → Google Cloud Client Library → Vertex AI Endpoint

Your backend utilizes credentials from a GCP Service Account, stored as secrets in Cloud Secret Manager or environment variables. Each request to the model involves:

1. Authentication with Google (OAuth 2.0 with service account).
2. HTTP call to the Vertex AI API specifying the model (like PaLM 2, Gemini, etc.).
3. Receiving a JSON response with the model's output.
4. Processing tokens and logging usage for billing.

The challenge in migrating is that Anthropic uses a different authentication architecture. Claude utilizes static API keys (instead of dynamic OAuth), different REST endpoints, structures prompts with an optional XML format, and charges for tokens in a differentiated way (input/output separated versus total tokens). If you rewrite this manually, you're touching application code, secrets, logging, and monitoring—all at once.

Terraform changes the approach. First, you define the new infrastructure (Anthropic resources, secrets, configurations), then you update only the references in your application code, and finally, you apply the change in a controlled manner.

Step 1: Structure Your Terraform Directory for Multi-Provider

Create this file structure in your project's root:

terraform/
├── main.tf
├── variables.tf
├── providers.tf
├── google.tf
├── anthropic.tf
├── secrets.tf
└── outputs.tf

In providers.tf, declare both providers:

terraform {
  required_version = ">= 1.6"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.0"
    }
    anthropic = {
      source  = "jianyuan/anthropic"
      version = "~> 0.2"
    }
  }
}

provider "google" {
  project = var.gcp_project_id
  region  = var.gcp_region
}

provider "anthropic" {
  api_key = var.anthropic_api_key
}

The Anthropic provider for Terraform isn’t official (community-maintained) but has been stable in production for at least 40 startups I've tracked over the past 8 months. Note, the API key is managed as a variable—never hardcoded.

In variables.tf:

variable "gcp_project_id" {
  description = "Current GCP Project ID"
  type        = string
}

variable "anthropic_api_key" {
  description = "Anthropic API Key"
  type        = string
  sensitive   = true
}

variable "migration_percentage" {
  description = "Traffic percentage to Claude (0-100)"
  type        = number
  default     = 0
}

The migration_percentage variable will control your traffic during the gradual migration.

Step 2: Map Your Current Google Resources to Terraform Code

If you already have infrastructure on Google and have never used Terraform, you need to import the current state. Terraform has an import command that connects existing resources with your code.

First, list your Vertex AI resources:

gcloud ai endpoints list --region=us-central1

Identify the endpoint ID your model uses. Then, in google.tf, create the resource:

resource "google_vertex_ai_endpoint" "current_model" {
  name         = "production-model-endpoint"
  display_name = "Production PaLM 2 Endpoint"
  location     = var.gcp_region
}

Import the existing resource into Terraform's state:

terraform import google_vertex_ai_endpoint.current_model \
  projects/your-project/locations/us-central1/endpoints/1234567890

Now Terraform knows that endpoint already exists. It won’t recreate it—it will manage it. This is crucial because you don’t want Terraform to destroy your production model on the first apply.

Step 3: Define Anthropic Resources in Parallel

In anthropic.tf, you don’t define endpoints (Anthropic doesn’t use that concept)—you define API configurations and secrets. Claude's architecture is stateless: you don't deploy an endpoint, you just call the API.

resource "google_secret_manager_secret" "anthropic_key" {
  secret_id = "anthropic-api-key"

  replication {
    automatic = true
  }
}

resource "google_secret_manager_secret_version" "anthropic_key_version" {
  secret      = google_secret_manager_secret.anthropic_key.id
  secret_data = var.anthropic_api_key
}

Here you store the Anthropic API key in Google Secret Manager—why not? You’re already paying for that service, and it works well. You don’t need to migrate everything immediately.

Next, define a configuration resource that your application will read:

resource "google_storage_bucket_object" "model_config" {
  name   = "model-config.json"
  bucket = var.config_bucket_name
  content = jsonencode({
    provider         = var.migration_percentage > 50 ? "anthropic" : "google"
    anthropic_model  = "claude-3-5-sonnet-20241022"
    google_model     = "gemini-1.5-pro"
    traffic_split    = var.migration_percentage
  })
}

This JSON file in Cloud Storage acts as a distributed feature flag. Your application reads it at startup and decides which provider to use.

Step 4: Implement Traffic Splitting in Your Application Code

Terraform manages infrastructure, but the logical routing is done by your application. Here’s an example in Node.js:

const { GoogleAuth } = require('google-auth-library');
const Anthropic = require('@anthropic-ai/sdk');
const { Storage } = require('@google-cloud/storage');

const storage = new Storage();
let modelConfig = null;

async function loadConfig() {
  const [file] = await storage
    .bucket('your-config-bucket')
    .file('model-config.json')
    .download();
  modelConfig = JSON.parse(file.toString());
}

async function generateResponse(prompt) {
  if (!modelConfig) await loadConfig();
  
  const useAnthropic = Math.random() * 100 < modelConfig.traffic_split;
  
  if (useAnthropic) {
    const anthropic = new Anthropic({
      apiKey: process.env.ANTHROPIC_API_KEY
    });
    const message = await anthropic.messages.create({
      model: modelConfig.anthropic_model,
      max_tokens: 1024,
      messages: [{ role: 'user', content: prompt }]
    });
    return message.content[0].text;
  } else {
    // Your current call to Vertex AI
    return await callVertexAI(prompt);
  }
}

The traffic_split allows sending 10%, 25%, or 50% of traffic to Claude while the rest stays on Google. You measure latency, costs, and quality on both, and adjust the percentage.

Step 5: Execute the Gradual Migration with Terraform

With everything defined, the migration workflow is:

Week 1: 10% Traffic to Claude

terraform apply -var="migration_percentage=10"

Terraform updates the configuration file in Cloud Storage. Your application reads it automatically (or restarts pods if using Kubernetes). Now 10% of requests go to Claude.

Monitor for a week:

• Average latency (Datadog, Prometheus, or Cloud Monitoring)
• Cost per 1M tokens (extracted from billing dashboards)
• API errors (4xx/5xx rate)
• Response quality (custom metrics, user feedback)

An edtech startup found that Claude generated responses 23% longer for the same prompt—impacting costs more than expected. They adjusted max_tokens before proceeding.

Week 2: 50% Traffic

If the data looks good:

terraform apply -var="migration_percentage=50"

At half load, you start seeing real cost patterns. This is where many founders discover Claude is more expensive in raw volume but cheaper for "useful response quality"—you need fewer regenerations.

Week 3: Full Migration to 100%

terraform apply -var="migration_percentage=100"

All traffic goes to Claude. If something blows up, you do an immediate rollback:

terraform apply -var="migration_percentage=0"

In less than 2 minutes, all traffic returns to Google.

Step 6: Decommission Google Resources Without Breaking Anything

Once you're two full weeks on Claude without incidents, you can start shutting down Google resources to stop paying for unused capacity.

In google.tf, change the resources to lifecycle with prevent_destroy:

resource "google_vertex_ai_endpoint" "current_model" {
  name         = "production-model-endpoint"
  display_name = "Production PaLM 2 Endpoint - DEPRECATED"
  location     = var.gcp_region
  
  lifecycle {
    prevent_destroy = true
  }
}

This prevents Terraform from accidentally destroying the resource. Then, manually from the Google Cloud console, reduce scaling to zero or pause the endpoint. Don’t destroy it immediately—keep it on standby for 30 days as a safety net.

After 30 days without incidents, remove the Terraform block:

terraform state rm google_vertex_ai_endpoint.current_model
terraform apply

Now the resource is no longer managed by Terraform. You can delete it manually from GCP when ready.

The Mistakes I've Seen Made (and How to Avoid Them)

Mistake 1: Not Managing API Keys as Secrets

A founder hardcoded the Anthropic API key directly in the providers.tf file and uploaded it to GitHub. In 4 hours, he received a GitHub Secret Scanning alert. He had to rotate the key, review all access logs, and rewrite the Terraform. Always use terraform.tfvars in .gitignore or secret managers.

Mistake 2: Not Measuring End-to-End Latency Before Migrating

Another startup assumed Claude would be "as fast as Vertex AI" because that's what public benchmarks said. In production, their requests went through an API Gateway with custom transformations that added 800ms of overhead. The migration exposed the problem—not because of Claude, but because they never measured real latency. Establish baselines before switching providers.

Mistake 3: Migrating 100% in One Go "Because Staging Worked"

Staging with 300 requests/day doesn’t predict production with 40K requests/day. A founder did a full migration on a Friday, and Claude started returning 429 (rate limit exceeded) because they hadn’t negotiated enterprise limits with Anthropic. Always use gradual traffic splitting.

How Much This Migration Really Costs (Beyond API Bills)

The obvious cost is engineering time: between 20 and 40 hours of technical work for a small team. But the hidden costs are:

• Duplicating bills for 2-3 weeks: You pay both Google and Anthropic while testing.
• Monitoring overhead: You need dashboards comparing metrics side-by-side.
• Risk of contract change: If you had enterprise discounts with Google, you lose them upon migrating.

A legal tech startup in Madrid calculated that the entire migration cost them €3,200 in engineering time + €1,800 in double billing during testing. But they saved €12,400 in the first three months because Claude needed fewer tokens to generate legal summaries of equivalent quality.

The math doesn’t always come out positive. If your use case is massive embedding generation or simple text classification, smaller models on Google can still be cheaper. Claude shines in complex reasoning, long context, and tasks where "nearly correct" isn't good enough.

Final Reflection: Infrastructure as a Product Decision

Migrating from Google to Anthropic isn’t a DevOps project—it’s a product decision disguised as infrastructure. Terraform only gives you the technical ability to make the change without breaking everything. But the strategic question remains: does your product improve enough with Claude to justify the cost of switching?

I’ve seen founders migrate because "Claude is trendy" and end up paying 40% more without a measurable improvement in their star metric. I’ve also seen others double their conversion rate because Claude understood user intent in cases Gemini misinterpreted. The AI provider isn’t neutral—it changes what your product can do.

Is your startup already considering a provider switch, or are you staying on the same stack because "it works well enough"?