SnapLogic extends alliance with Databricks to boost data lake reliability; analyze big data workloads in the cloud

SnapLogic has expanded its partnership with Databricks with new support for Delta Lake, the open source storage layer created by Databricks that brings reliability to traditional data lakes. 

Together, the joint solution helps customers accelerate the integration, transformation, and processing of big data workloads into Delta Lake, increasing data quality and accelerating the time to value of advanced analytics and machine learning initiatives.

Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake runs on top of existing data lake and is fully compatible with Apache Spark APIs. Delta Lake on Databricks allows users to configure Delta Lake based on their workload patterns and provides optimized layouts and indexes for fast interactive queries.

Delta Lake sits on top of Apache Spark. The format and the compute layer helps to simplify building big data pipelines and increase the overall efficiency of the pipelines.

Organizations are increasingly investing in data lakes to gain actionable insights into their growing data assets. However, the high volume and complexity of data often results in data quality, reliability and performance issues. 

Together, SnapLogic and Databricks are removing these roadblocks by providing a low-code, visual paradigm for data engineers to create and process data pipelines that leverage the full power of Delta Lake — including features like ACID transactions, scalable metadata handling, schema enforcement, and batch and streaming support.

“Databricks and SnapLogic are committed to delivering product innovations that help organizations reduce the time, effort, and skills needed to manage their big data initiatives so they can quickly turn their data into meaningful insights that drive the business forward,” said Craig Stewart, chief technology officer, SnapLogic. “By teaming up with Databricks, we aim to remove the key technical barriers to data lake and big data management so our customers can accelerate their analytics and machine learning initiatives and focus on delivering real business value.”