Back to all posts

GCP Professional Cloud Architect Certification - Study Notes Part 3: VPC Architecture & Core Routing

Tue, June 2, 2026
#google-cloud#professional-cloud-architect#notes#vpc-networking

View all study notes here

Fundamental VPC Architecture

Google Cloud Virtual Private Cloud (VPC)

Global by design. Once created, the network infrastructure exists globally across all Google data centres.

  • Built entirely on top of Google’s private, global fibre optic network.
  • Most other cloud providers tie VPCs to a single geographical region, requiring manual routing, VPN tunnels, or peering to connect resources across continents.
  • Always create a Custom Mode VPC instead of an Auto Mode VPC for production enterprises.
    • Auto mode creates a subnet in every single Google Cloud region automatically, wasting IP addresses and risking CIDR overlaps with on-premises infrastructure.

Global Load Balancing

  • External Application Load Balancers can use a single, global Anycast IP address (users in the EU and US hit the exact same public IP).
  • Traffic routes to the nearest Google Edge Location (facilities near major cities that sit closer to users than standard Regions or Availability Zones).
  • User traffic moves across Google’s private backbone to the healthiest, closest backend server, regardless of its region.

Global VPC Firewall Rules

  • Firewall rules are defined at the VPC level but applied globally.
  • Administrators use Network Tags or Service Accounts to target specific VMs regardless of their regional placement.

Cross-Region Egress Costs

  • !! Important: Organizations are billed for data transferring out of one region and into another, even though the traffic never leaves Google’s private network backbone.

VPC Flow Logs

  • Record history of network traffic passing through GCP network interfaces.
  • Sample IP packet data in 5 second intervals and aggregate by standard 5 tuple (source IP, destination IP, source port, destination port, and protocol) to give near real time visibility.
  • Enabled per subnet.
  • Supports
    • Compute resources: VM instances, GKE nodes.
    • Serverless endpoints: Cloud Run resources configured with Direct VPC egress.
    • Hybrid & cross cloud links: Cloud VPN tunnels and VLAN attachments for Cloud Interconnect.
  • Used for network monitoring, forensics, and cost attribution.
  • Firewall Interaction
    • Egress (Outbound)
      • Sampled before egress firewall rules. Packets blocked by an outbound rule are still logged.
    • Ingress (Inbound)
      • Sampled after ingress firewall rules. Packets blocked by an inbound rule are dropped immediately and not logged.
  • Has a sampling rate config to reduce data volume and manage Google Cloud Logging storage costs.

Regional Subnets

Subnets are created in specific regions inside the global VPC.

  • !! A single subnet cannot span across multiple regions, but it can span multiple Availability Zones within its home region.
  • A subnet’s IP range (/8 to /29) can be expanded dynamically without deleting the subnet or destroying workloads, provided the new range does not overlap with existing subnets in the same VPC.
    • Subnet size can only be increased (decrease the prefix length, e.g., /24 to /22).
      • Cannot shrink a subnet CIDR range once it is created.
      • To shrink it, it must deleted then recreated.

Shared VPC (Single VPC Pattern)

An enterprise architecture pattern where an organization designates a central project as the Host Project to manage shared network infrastructure.

  • Service Projects attach directly to these centralized subnets.
  • Enforces a clean separation of duties between central network administrators and application developers.
  • Shared VPC Constraints
    • A project can only be a host project OR a service project, not both. Service projects can attach to only one host project.
  • Shared VPC IAM Roles
    • Shared VPC Admin (org-level, enables host projects).
    • Network User (grants service project dev access to specific subnets).
    • Network Viewer (read-only)
  • Centralized Network Governance
    • Central NetOps or SecOps teams retain complete control over firewall rules, Cloud NATs, Cloud VPNs, and Interconnects. Developers spin up VMs but cannot modify underlying infrastructure or internet access.
  • Low-Latency Microservices
    • Keeping internal microservices inside the same VPC removes the operational overhead of managing VPC Peering connections.
  • Shared Common Services
    • Simplifies connections to enterprise Active Directory (AD), shared build farms, and centralized monitoring engines.

Multiple VPC Pattern

Splits the organization into distinct, isolated VPC networks to prioritize team autonomy and independent deployment velocity.

  • Overlapping IP Address Spaces
    • If two independent teams or merging companies use the exact same subnet CIDR blocks, they cannot live in the same VPC.
    • They require Private Service Connect (PSC) or a proxy layer to communicate.
  • Strict Regulatory Requirements
    • Isolates blast radius to comply with rigorous security standards.
  • Quota Limits
    • Bypasses project-level GCP resource quotas.

Instance Groups

Managed Instance Groups (MIGs)

Uses an Instance Template to create a cluster of identical VM instances, who’s lifecycle can be automated. Standard for deploying scalable, highly available, and resilient applications.

  • Can be deployed as Zonal (All VMs in one zone) or Regional (VMs are automatically distributed across multiple zones within single region).
    • Regional MIGs protect application against localized zonal outage. (high availability)
  • Automatically add or remove VM instances based on demand.
    • Can autoscale based on:
      • CPU utilization.
      • Cloud Monitoring metrics.
      • Pub/Sub queue depth.
      • Load balancing capacity (e.g. target requests per second).
    • Connection Draining
      • Allow existing in flight requests to finish before destroying the VM when autoscaler decides to scale down MIG.
      • 300 seconds default.
  • Health check can be attached for autohealing.
    • If an instance fails health check (e.g. web server crashes), MIG automatically deletes that VM and provisions a new one to replace it.
  • Update Instance Template when updating application code or OS image. Orchestrate Rolling Update, replacing VMs one by one or in batches to ensure no downtime.
    • Can also perform Canary Deployments by rolling out new template to small percentage of instances first.
  • Stateless vs stateful
    • Stateless MIGs auto scale and heal. Ideal for frontend web scaling.
    • Stateful MIGs preserves underlying persistent data disks, fixed machines metadata, static network IP addresses, even when instance undergo auto healing restart or rolling deployment.

Unmanaged Instance Groups

A collection of different VM instances that are manually added to a group.

  • No automation. Does not support autoscaling, autohealing, templates and rolling updates.
  • Used to group existing, mismatched VMs so they can be added together as a backend service for Google Cloud Load Balancer.
  • Avoid using Unmanaged Instance Groups unless prompt explicitly states VMs must have different OS, configurations, or cannot be configured via template.

Alias IP Ranges

  • VMs can be assigned secondary IP ranges from their subnet (alias IPs).
  • Alias IPs used heavily by GKE pods so each pod gets its own IP without a separate VM.
  • A single VM or Pod can have a primary IP for the node and a secondary/alias IP range for the containers.

Core Routing & Control Planes (How Traffic Moves Internally)

In traditional on prem data centre, a router is a physical appliance that performs two jobs: calculating paths (control plane) and physically moving packets across cables (data plane). In GCP, they are separate.

Andromeda SDN Layer (data plane)

  • Proprietary Software Defined Networking (SDN) stack by Google.
  • Google Cloud networks do not rely on physical hardware routers inside data centres to handle internal VPC traffic.
  • Every VM contains a local software controller pre-programmed with the locations of all other global VPC resources.
  • Once Cloud Router learns a route, it programs that route directly into Andromeda SDN.
  • When a VM sends a packet, Andromeda handles the physical routing across Google’s global fibre network.

Cloud Router (control plane)

  • Fully managed, software defined control plane tool.
  • Talk to external BGP peers, learn their IP prefixes, and advertise our VPC’s IP prefixes.
  • Use BGP to dynamically exchange routing paths between VPC and on prem routers.
    • Does not sit in data path of packets.
      • No throughput limit.
    • Manages control plane (BGP routing tables), while Google Andromeda SDN handles data plane (forwarding actual packets).
  • If Cloud Router instance fails or restarts, traffic continues flowing over existing paths uninterrupted.
  • Used in: HA VPN, Cloud Interconnect (Dedicated & Partner), Cloud NAT, Network Connectivity Center.

Regional vs Global Dynamic Routing Mode

  • VPC Dynamic Routing Mode must be configured as either Regional or Global. This setting determines how the Cloud Router handles Border Gateway Protocol (BGP) routes discovered from hybrid environments:
    • Regional
      • A Cloud Router in region-a only advertises on-premises routes to resources located within that same region.
    • Global
      • A Cloud Router in region-a dynamically discovers and advertises on-premises routes to VMs located in all regions across the global VPC.

Custom Rout Advertisements (CRA)

Cloud Router by default automatically advertises our VPC’s subnet CIDR blocks. This can be changed to advertise Custom Routes.

  • Private Google Access for on prem
    • How to allow on prem servers to securely access Google APIs (e.g. Cloud Storage, BigQuery) without using public internet?
    • Set up private DNS zone on prem pointing to private.googleapis.com (199.36.153.8/30). Configure Cloud Router to use Custom Route Advertisement to broadcast that exact /30 range back to on prem routers so they know to route Google API traffic through VPN/Interconnect.

Custom Learned Routes

  • Inject Custom Learned Routes into Cloud Router.
  • Manually add static network destinations to Cloud Router’s BGP session, which it will then distribute inside VPC as a dynamic route.

BGP Route Policies

  • Define BGP route policies.
  • Construct granular filter rules or modify route attributes (e.g. altering MED or AS-Path) as routes are imported or exported.

Quotas & Hard Limits

  • BGP Session Reset (5000 prefix limit)
    • Cloud Router will accept a maximum of 5000 prefixes from a single BGP peer.
    • If on prem router attempts to advertise 5001 routes, Cloud Router will automatically drop and reset the BGP session.
    • Any question that states connectivity is flapping for failing because an enterprise is sending too many routes, the correct architecture choice is to configure route summarization (aggregation) on the on prem router (e.g. combining dozens of /24 subnets into a single /16 block).
  • Deterministic Route Dropping
    • Predictable rule used to manage and filter routes.
    • Certain routes are dropped when a Cloud Router learns more routes than VPC allows (exceeding quota limits).
    • The rule is deterministic, so the same routes are dropped every time.
    • GCP drops unique destination prefixes using this exact sorting order:
      1. IPv6 before IPv4: IPv6 prefixes are discarded first.
      2. Longest Prefix Match (LPM): If mask lengths are equal, it sorts the prefixes.
      3. Sorting Limit: Prefixes that “sort beyond the limit” for the quota are permanently dropped.
  • Max Custom Advertisements
    • Can configure maximum of 200 custom advertised routes per BGP session.

Network Failure Scenarios

  • Cloud Router continuously exchange “Keepalive” messages with its on prem BGP peer.
    • If a physical link fails (e.g. a fibre cut on an Interconnect) or an intermediate device drops traffic, Cloud Router stops receiving Keepalives.
    • By Default, Cloud Router uses a Hold time of 30 sec (with Keepalive sent every 10 sec).
    • If no response from on prem router for 30 sec, Cloud Router declares BGP session dead.
    • Once session declared dead, Cloud Router immediately purges all routes learned from that specific peer from VPC’s dynamic routing table.
Examples
  • High Availability (HA) VPN (Active-Active)
    • HA VPN requires two tunnels from a single Cloud Router to your on-premises architecture, ensuring a 99.99% SLA.
    • Normal Operation
      • If both tunnels are configured to advertise identical routes with identical priorities, Google Cloud distributes egress traffic across both tunnels using ECMP (Equal-Cost Multi-Pathing).
    • During a Failure
      • If Tunnel 1 fails, Cloud Router detects missing Keepalives on that specific BGP session. It tears down the session and removes Tunnel 1’s routes. Because Tunnel 2’s session is still active, traffic automatically and instantly shifts entirely to Tunnel 2.
  • Cloud Interconnect with HA VPN Backup
    • Classic enterprise architecture uses a high-speed Cloud Interconnect as primary path and an HA VPN over public internet as a lower-speed backup path.
    • Normal Operation (Path preference)
      • Must configure Cloud Router to prefer Interconnect over VPN. This is done by assigning a lower Multi-exit Discriminator (MED) to Interconnect, and higher MED to VPN routes.
        • In BGP, lower MED indicates higher preference/lower cost.
      • During Failure
        • If Interconnect circuit goes down, its BGP session times out. Cloud Router pulls Interconnect’s low MED routes out of the VPC routing table. The VPC immediately falls back to the next best available rout (high MED HA VPN routes).
      • When Failure is Fixed (Failback)
        • Once Interconnect link is physically restored and BGP session re-establishes, Cloud Router learns the low MED routes again, and injects them back into the VPC. Traffic automatically shifts back away from VPN to Interconnect.
Graceful Restart (Minimizing downtime)

Cloud Router supports BGP Graceful Restart. This is a critical feature designed to prevent traffic disruption during brief control plane hiccups.

  • If Cloud Router service itself undergoes a brief restart or maintenance update, it signals to peer on prem router not to immediately flush routing table.
  • On prem router will continue forwarding packets along known “stale” data paths for a grace period (up to 120 sec), assuming data plane is still intact.
  • Prevents routing flaps and packet drops during minor cloud infrastructure updates.
Asymmetric Routing

Cloud Router only controls how traffic leaves VPC (Egress). It influences how traffic returns to VPC (Ingress) by advertising routes to on prem router, but cannot force on prem router to listen.

  • If using an Interconnect and a backup VPN, and we fail to configure on prem router correctly, GCP VMs might send traffic out via the Interconnect (Egress), but on prem network might try to send response traffic back via VPN (Ingress).
    • This is asymmetric routing, and stateful firewalls will instantly drop this traffic.
  • Solution
    • For GCP Egress
      • Adjust MED lower on Cloud Router’s BGP sessions so GCP prefers Interconnect.
    • For GCP Ingress (On-Premises Egress)
      • Use AS-Path Prepending on on prem router when advertising routes over backup VPN.
      • By making VPN path look artificially “longer” (e.g., prepending Autonomous System number multiple times), Cloud Router will naturally prefer the shorter Interconnect path for traffic returning to GCP.
      • If the Interconnect fails, Cloud Router will automatically switch to the longer AS-Path via the VPN.

Private Google Access (PGA)

  • All subnets live inside the same global VPC, enabling native cross region routing.
  • VMs in different regions or continents communicate using internal private IP addresses.
  • Allows secure access to Google APIs & services (e.g. Cloud Storage, BigQuery) without needing a public IP address
  • Lower latency, higher throughput, and tighter security.
  • Must be enabled per subnet (off by default).