Sign in
ToolData Pipeline

Delta Lake

Delta Lake is an open-source storage layer that brings ACID transactions, schema enforcement, and time travel to data lakes built on top of Apache Spark and cloud object stores. It enables reliable, scalable data pipelines and lakehouse architectures by ensuring data quality and consistency across batch and streaming workloads.

by Linux Foundation / Delta Lake ProjectOpenSource

Key Features

  • •ACID transactions on data lakes using optimistic concurrency control
  • •Schema enforcement and evolution for structured and semi-structured data
  • •Time travel with versioned data for reproducibility and rollback
  • •Unified batch and streaming processing on the same tables
  • •Scalable metadata handling using transaction logs instead of file listing

Pricing

OpenSource

Delta Lake is open source under the Apache 2.0 license. Commercial support and managed offerings are available via Databricks and some cloud providers.

Alternatives

Apache HudiApache IcebergSnowflakeBigQueryAmazon Redshift

Links

Website →

Industries Using Delta Lake

insurance1aerospace defense1entertainment1

Often Used With

MLflow (2)Weights & Biases (2)Kubernetes (2)Databricks (1)LLM (1)Vector DB (1)AWS S3 + AWS DataSync (1)Azure Data Lake (1)Google Cloud Storage Transfer (1)GDAL (1)Rasterio (1)Orfeo Toolbox (1)

Quick Stats

3
Total Use Cases
3
Industries
12
Related Technologies