View all study notes here

---

Object Storage (Cloud Storage)

A fully managed, RESTful, serverless object storage service, Designed for unstructured data and scales infinitely. Accessed via HTTPS; not a mounted file system or block device.

Storage Classes & Technical Retrieval Limits

Cloud Storage offers 4 primary storage classes, all providing the same sub-second latency (millisecond time-to-first-byte) and 99.999999999% (11 nines) durability, but differing in availability, minimum storage duration, and retrieval cost.

Storage Class	Intended Use Case	Minimum Duration	Retrieval Fees	Availability SLA (Multi-Region / Regional)
Standard	High-frequency access, active data, website assets, streaming.	None	None	99.99% / 99.9%
Nearline	Fast backup, data accessed less than once a month.	30 days	Per GB fee	99.95% / 99.9%
Coldline	Disaster recovery, data accessed less than once a quarter.	90 days	Higher per GB fee	99.95% / 99.9%
Archive	Long-term digital preservation, regulatory compliance, accessed less than once a year.	365 days	Highest per GB fee	99.95% / 99.9%

GCS vs. AWS S3

• No Tape Delay
  • Unlike AWS Glacier, Archive and Coldline classes in Google Cloud provide immediate, millisecond retrieval.
  • Do not choose an architecture that introduces application timeouts waiting for data to hydrate (AWS Glacier pattern, not needed for GCS).
• Early Deletion Fees
  • If you modify or delete an object before its minimum duration is met, you are billed for the remaining days at that class’s storage rate.
  • Nearline = 30 days, Coldline = 90 days, Archive = 365 days.

Location Types

• Multi-Region (e.g., us, eu, asia)
  • Data redundantly stored across geographically separated data centres within the continent.
  • Highest availability.
  • Best for globally accessed data.
• Dual-Region
  • Redundant storage across exactly two specified regions (e.g., nam4 = Iowa + South Carolina).
  • Satisfies data residency requirements while maintaining redundancy.
• Region (e.g., us-central1)
  • Stored in a single region.
  • Lowest latency for co-located compute.
  • Lower cost.
  • Used when data residency mandates a single region.

Object Lifecycle Management

Automate data transitions to cheaper storage tiers or enforce data deletion based on specific conditions, eliminating manual housekeeping.

• Supported Actions: Delete and SetStorageClass
• Common Conditions:
  • Age: Object age in days.
  • CreatedBefore: A specific date timestamp.
  • MatchesStorageClass: Targets specific tiers (e.g. move only Standard to Nearline)
  • IsLive / NumNewerVersions: Used with Object Versioning to retain historical copies while automatically purging or archiving old versions.
• !! Lifecycle rules are evaluated once daily
  • They do not trigger in real time.
  • Do not design architectures expecting immediate transitions.
• Typical Lifecycle Pattern:
  • Day 0–30: Standard (active access)
  • Day 30–90: Nearline (warm backup)
  • Day 90–365: Coldline (cold archive)
  • Day 365+: Archive (long-term retention) or Delete

Object Versioning

• When enabled, GCS retains every previous version of an object rather than overwriting on upload.
• Protects against accidental deletion and overwrites.
• Non-current (historical) versions still incur storage costs.
  • Combine with Lifecycle Management using NumNewerVersions to automatically archive or delete old versions and control costs.
• !! Enabling versioning does not protect against bucket deletion itself. Use Soft Delete (bucket-level recycle bin) or Object Holds for additional protection.

Object Holds & Retention Policies

• Object Hold
  • Places a lock on a specific object, preventing deletion or modification until released.
  • Two types: eventBasedHold (released manually) and temporaryHold (released manually for short-term locks).
• Retention Policy
  • Applied at the bucket level.
  • Sets a minimum time period that all objects in the bucket must be retained before deletion is permitted.
  • Used for regulatory compliance (e.g., FINRA 7-year retention, HIPAA audit logs).
• Bucket Lock
  • After a retention policy is confirmed, applying a Bucket Lock makes the policy permanent and immutable.
    • Not even the bucket owner can reduce or remove it.
  • Required for WORM (Write Once, Read Many) compliance.

Access Patterns & Security

• Uniform Bucket-Level Access
  • Disables Access Control Lists (ACLs) entirely.
  • IAM permissions apply uniformly to all objects within the bucket.
    • No per-object exceptions.
  • Use by default for enterprise compliance and simplified security auditing.
  • Once enabled and locked, ACLs cannot be re-enabled.
• Fine-Grained Access
  • Enables both IAM and per-object ACLs simultaneously.
  • Use only when individual objects within the same bucket require radically different access permissions.
  • Harder to audit, avoid unless there is a specific, justified use case.
• Signed URLs
  • A cryptographically signed URL providing time-limited read or write access to a specific object for unauthenticated users.
  • URL includes the resource path, expiration time, and a cryptographic signature generated by a service account.
  • Ideal for: secure direct-to-bucket mobile uploads, temporary file download links, sharing assets with external partners without granting IAM roles.
  • !! Signed URLs bypass IAM
    • They grant access based on the signing service account’s permissions, not the caller’s identity.
• Signed Policy Documents
  • Controls what data an unauthenticated user can upload via an HTML form POST.
    • Specifying allowed content types, file size limits, and expiration.

Cloud Storage FUSE

• A POSIX-compliant file system interface that mounts a GCS bucket as a local file system on Linux/macOS.
• Enables legacy applications expecting a traditional file system path to read/write from GCS without code changes.
• !! Not a high-performance solution.
  • Latency is significantly higher than local disk.
  • Use for convenience/compatibility, not for performance critical workloads.

---

Relational Databases (SQL)

Choose Cloud SQL, AlloyDB, or Cloud Spanner when structured data requires strict transactional integrity (ACID compliance) and relational joins. The decision then hinges on scale, geography, and cost.

ACID Compliance

• Atomicity
  • Every operation in a transaction either fully commits or fully rolls back; no partial writes.
• Consistency
  • Every transaction brings the database from one valid state to another.
• Isolation
  • Concurrent transactions execute as if they were sequential; no dirty reads.
• Durability
  • Committed transactions are permanently persisted even after a system crash.

Cloud SQL

Fully managed relational database service for MySQL, PostgreSQL, and SQL Server. Scales vertically up to 96 vCPUs, 624 GB RAM, and 64 TB of storage depending on the engine.

• High Availability (HA)
  • Configured at instance creation time (can be enabled later but requires downtime).
  • Uses a Primary Instance in Zone A and a synchronized Standby Instance in Zone B within the same region.
  • Data is synchronously replicated via shared regional persistent disks.
  • If Zone A fails, a failover automatically reroutes traffic to Zone B via a shared floating IP address (typically < 60 seconds).
  • !! HA protects against zonal failure only, not regional failure.
• Read Replicas
  • Offload read heavy query traffic from the primary instance to horizontally scale read operations.
  • Replication is asynchronous, which introduces potential read lag.
  • Can be cross-zonal (same region, different zone) or cross-regional (different GCP region; useful for disaster recovery and serving local reads to users in a distant geography).
  • !! Read replicas cannot serve as automatic HA failover targets.
    • A cross-regional read replica must be manually promoted to a standalone primary if the primary region fails. This is not automatic.
• Cascade Replicas
  • A read replica of a read replica. Reduces load on the primary by chaining replication.
• Database Migration Service (DMS)
  • A managed, low-downtime migration tool for moving homogeneous (MySQL → Cloud SQL for MySQL) or heterogeneous (Oracle → AlloyDB) database workloads to GCP.
  • Uses continuous change data capture (CDC) to replicate ongoing changes until the cutover, minimizing downtime.

AlloyDB for PostgreSQL

High performance, enterprise grade, fully managed PostgreSQL compatible database designed for the most demanding relational workloads.

• Architecture
  • Decouples compute from storage.
  • Utilizes a distributed, intelligent, log-structured storage layer built on Google’s internal infrastructure.
  • In-Memory Columnar Engine
    • A machine-learning-driven cache that automatically identifies hot analytical columns and maintains them in an in-memory columnar format without any schema changes.
• Performance metrics
  • Up to 4× faster transactional (OLTP) performance than standard PostgreSQL.
  • Up to 100× faster analytical (OLAP) query performance due to the columnar engine.
  • Designed to serve as a hybrid HTAP (Hybrid Transactional/Analytical Processing) database.
• AlloyDB Omni
  • A downloadable, self managed version of AlloyDB that runs on any infrastructure (on-premises, other clouds).
    • Useful for development/testing or hybrid environments.
• When to Choose AlloyDB:
  • Modernizing massive legacy commercial databases (Oracle, SQL Server) to an open source PostgreSQL compatible engine without sacrificing enterprise performance.
  • Workloads requiring both OLTP and ad-hoc analytical queries on the same dataset (HTAP).
  • When Cloud SQL’s performance ceiling has been reached.

Cloud Spanner

Enterprise grade, globally distributed, horizontally scalable database that provides relational capabilities with global strong consistency.

• Scalability Paradox Solved
  • Traditional relational databases scale vertically; NoSQL databases scale horizontally but lose strong consistency.
  • Spanner achieves both horizontal scaling (across regions) and ACID transactions.
• TrueTime API
  • Google’s specialized infrastructure incorporating synchronized GPS receivers and atomic clocks in every data centre.
  • Bounds clock drift to a strict epsilon window (ε), allowing Spanner to assign monotonically increasing global timestamps to transactions.
  • Guarantees external consistency (strict serializability).
    • The highest consistency level possible in distributed systems.
• Instance Configuration Types:
  • Regional
    • Data replicated across 3 zones in one region.
    • Lower cost, lower latency for single-region workloads.
  • Multi-Region
    • Data replicated across regions globally (e.g., nam6 covers North America).
    • Higher cost, global consistency and availability.
• Pricing Model
  • Charged by processing unit (compute) + storage. Minimum 1 processing unit.
  • Can be expensive at scale.
• !! Only select Spanner when the scenario explicitly requires BOTH horizontal write scalability AND cross-regional relational strong consistency.
  • Classic exam triggers: global financial ledger, global inventory with strict consistency, global gaming leaderboard with real time accuracy.
• For localized transactional apps: use Cloud SQL.
• For PostgreSQL performance at scale: use AlloyDB.

Relational Database Decision Matrix

	Cloud SQL	AlloyDB	Cloud Spanner
Engine	MySQL / PostgreSQL / SQL Server	PostgreSQL-compatible	Proprietary (SQL compatible)
Scale Model	Vertical (single region)	Vertical + columnar cache	Horizontal (global)
Consistency	Strong (single region)	Strong (single region)	External consistency (global)
HA	Zonal failover (same region)	Regional failover (<60s)	Multi-region by design
Best For	Standard OLTP apps	Performance critical PostgreSQL / HTAP	Global distributed OLTP at scale
Cost	Low–Medium	Medium–High	High