Lake, warehouse, lakehouse
What data lakes and warehouses each get right, and what the lakehouse borrows from both.
On this page
You will learn
- What a data warehouse and a data lake each do well, and badly
- What a lakehouse is, concretely
- Which pieces make up a lakehouse stack
- When a lakehouse is the wrong choice
Read first
Nothing. This lesson starts from scratch.
The warehouse
A data warehouse (Teradata, Redshift, Snowflake, BigQuery, Synapse) stores data in its own managed format and gives you SQL with transactions, a schema, indexes or clustering, access control and good performance. The costs: data must be loaded into it, storage and compute are typically priced per vendor, machine learning and Python workloads need the data exported, and the format belongs to the vendor.
The lake
A data lake is object storage (S3, ADLS, GCS) holding files in open formats: CSV, JSON, Parquet, images, logs. Storage is very cheap and any engine can read it: Spark, Trino, pandas, ML frameworks. But a folder of files is not a table:
- No transactions: a failed job leaves half-written files that readers see.
- No safe updates or deletes: changing one row means rewriting files, with readers seeing in-between states.
- No schema enforcement: a job writing a wrong type silently corrupts the table.
- Slow listing and no statistics, so queries scan far more than needed.
Many lakes turned into "data swamps": nobody trusted the data enough to use it.
The lakehouse
The lakehouse keeps the lake's storage and formats and adds a metadata layer that records exactly which files form each version of a table. That one idea brings back warehouse features:
| Warehouse feature | How the lakehouse gets it |
|---|---|
| ACID transactions | Atomic commits to a transaction log or metadata tree |
| Updates, deletes, MERGE | Rewrite or mark files, then commit the new file list atomically |
| Schema enforcement and evolution | The schema is stored in table metadata and checked on write |
| Fast queries | File statistics, clustering and compaction for data skipping |
| Time travel and audit | Old versions remain until cleaned up |
| Governance | A catalog (Unity Catalog, Polaris, Glue, Hive Metastore) for names and permissions |
The stack
-
Storage
S3, ADLS, GCS
-
File format
Parquet
-
Table format
Delta, Iceberg, Hudi
-
Catalog
Names, permissions
-
Engines
Spark, Trino, Flink, warehouses
Because every layer is open, several engines can work on the same tables: Spark for ETL, Trino for interactive SQL, a warehouse reading Iceberg directly, Python for ML, without copying data.
When it is not the answer
- Small data with simple reporting: a managed warehouse or even PostgreSQL is less to operate.
- Very high-frequency single-row updates and lookups: that is an OLTP database's job.
- Teams without the skills to run maintenance (compaction, cleanup, statistics): managed platforms do this for you; self-managed lakehouses need it done.
Common mistakes
Thinking Parquet alone makes a lakehouse
Treating lakehouse tables like OLTP tables
Skipping table maintenance
Key takeaways
- Warehouses are reliable but closed; lakes are open and cheap but unreliable.
- A lakehouse adds a table format over lake files to get transactions and schemas.
- The stack: object storage, Parquet, a table format, a catalog, many engines.
- It needs maintenance and is not a replacement for OLTP databases.
Check yourself
3 questions1. What turns a folder of Parquet files into a transactional table?
Show the answer
A table format such as Delta Lake or Iceberg. The table format's metadata defines which files are in each version and commits atomically.
2. Which is a typical weakness of a plain data lake?
Show the answer
No atomic writes: readers can see half-written data. Without transactions, partial writes are visible.
3. Which workload is a poor fit for a lakehouse?
Show the answer
Thousands of single-row updates per second. That is OLTP; each tiny commit adds files and log entries.
Go deeper
Primary sources: Delta Lake documentation · Apache Iceberg documentation