Branches, tags and write-audit-publish

Write, audit, publish

1. Write

   Commit the new data where production readers cannot see it

2. →

   Audit

   Run checks on it: row counts, nulls, duplicates, reconciliation with the source

3. →

   Publish

   Make it visible atomically, or discard it if checks fail

Without WAP, quality checks run after the data is already live, and a failed check means a rollback after people may have read bad numbers.

Iceberg branches and tags

Every Iceberg table already has a history of snapshotssnapshot: A complete, consistent version of a table at one moment. Readers always see one snapshot, never a half-finished write. Learn more →. A branch is a named, movable pointer to a snapshot that can receive its own commits; main is the branch normal reads and writes use. A tag is a named, fixed pointer to one snapshot, useful for "the data as of quarter close". Both can carry retention settings, and snapshots they reference are kept when old snapshots expire.

# a tag: keep the quarter-end state for a year
spark.sql("ALTER TABLE prod.db.sales CREATE TAG `q4_2024` RETAIN 365 DAYS")
spark.sql("SELECT * FROM prod.db.sales VERSION AS OF 'q4_2024'")

# a branch: an isolated line of commits
spark.sql("ALTER TABLE prod.db.sales CREATE BRANCH audit")
spark.sql("SELECT * FROM prod.db.sales VERSION AS OF 'audit'")
# or: SELECT * FROM prod.db.sales.branch_audit;

-- a tag: keep the quarter-end state for a year
ALTER TABLE prod.db.sales CREATE TAG `q4_2024` RETAIN 365 DAYS;
SELECT * FROM prod.db.sales VERSION AS OF 'q4_2024';

-- a branch: an isolated line of commits
ALTER TABLE prod.db.sales CREATE BRANCH audit;
SELECT * FROM prod.db.sales VERSION AS OF 'audit';
-- or: SELECT * FROM prod.db.sales.branch_audit;

WAP with a branch, step by step

spark.sql("ALTER TABLE prod.db.sales SET TBLPROPERTIES ('write.wap.enabled' = 'true')")
spark.sql("ALTER TABLE prod.db.sales CREATE BRANCH etl_20250301 RETAIN 7 DAYS")

ALTER TABLE prod.db.sales SET TBLPROPERTIES ('write.wap.enabled' = 'true');
ALTER TABLE prod.db.sales CREATE BRANCH etl_20250301 RETAIN 7 DAYS;

spark.conf.set("spark.wap.branch", "etl_20250301")
daily.writeTo("prod.db.sales").append()

spark.sql("""
    SELECT COUNT(*), COUNT_IF(amount IS NULL)
    FROM prod.db.sales VERSION AS OF 'etl_20250301'
    WHERE sale_date = DATE'2025-03-01'
""")

SELECT COUNT(*), COUNT_IF(amount IS NULL)
FROM prod.db.sales VERSION AS OF 'etl_20250301'
WHERE sale_date = DATE'2025-03-01';

spark.sql("CALL prod.system.fast_forward('db.sales', 'main', 'etl_20250301')")
spark.sql("ALTER TABLE prod.db.sales DROP BRANCH etl_20250301")

CALL prod.system.fast_forward('db.sales', 'main', 'etl_20250301');
ALTER TABLE prod.db.sales DROP BRANCH etl_20250301;

Fast-forward only works if main has not moved since the branch was created; otherwise publish by cherry-picking the branch's snapshot, or redo the write. Before branches existed, Iceberg did WAP with write.wap.enabled and a spark.wap.id on staged snapshots, published with the cherrypick_snapshot procedure; you may still see it.

On Delta Lake

Delta has no table branches. The same guarantee comes from staging:

• Staging table: write the batch to a staging table, audit it, then publish with one MERGE or INSERT OVERWRITE ... REPLACE WHERE into production. The publish step is a single atomic commit.
• Shallow clone: CREATE TABLE sales_wap SHALLOW CLONE sales copies only metadata. Run the job against the clone, audit, then apply the same change to production. Useful for testing a whole pipeline against production data without copying it.
• Rollback as a safety net: if bad data does get published, RESTORE to the previous version, but readers may already have seen it, which is what WAP avoids.

Branching the whole catalog

Iceberg branches are per table. A pipeline that updates five tables needs all five to publish together. Catalog-level tools give Git-like branches across many tables: Project Nessie (an Iceberg catalog with branches, tags and multi-table commits) and lakeFS (branches over object storage for any format). The pipeline writes all tables on a branch, audits, and merges the branch: every table changes at once.

Common mistakes

• ✗Running quality checks after publishing — Readers see bad data before the check fails.
• ✗Forgetting to drop failed branches — Their snapshots are retained and storage grows.
• ✗Assuming fast-forward always works — If main moved, fast-forward fails; cherry-pick or rewrite.
• ✗Publishing related tables one by one — Readers see a mix of old and new across tables. Use catalog-level branches or publish in one transaction where supported.