Data Engineering 
Data Strategy + Governance 
Data Migrations 
Concierge + Support 
Artificial Intelligence 
Training + Enablement 
Data Analytics + Insights 
Financial Services 
Communications, Media + Entertainment 
Healthcare + Life Sciences 
Technology + Digital Natives 
Manufacturing + Automotive 
Public Sector 
Retail + Consumer Goods 
Race to the Lakehouse 
AI + Data Maturity Assessment 
Unity Catalog 
Rapid GenAI 
Modern Data Connectivity 
Gatehouse Security 
Health Check 
Sample Use Case Library 
Blog 
Events 
About Us 
Careers 
Founding Team 
The Challenge
A leading Sales Automation software platform was facing challenges in maximizing the performance of its existing Databricks pipelines. The client’s goal was to improve data processing efficiency and evaluate the potential for migrating its EMR streaming jobs. This is where Entrada stepped in to help optimize data infrastructure and support growth objectives.
The Solution
Entrada collaborated closely with the client to deliver a comprehensive assessment of their current data infrastructure and provide actionable recommendations for enhancing performance and return on investment (ROI). The engagement focused on the following key areas:
- Performance Optimization of Databricks Pipelines: Entrada utilized the latest capabilities of Databricks, including Liquid Clustering, Z-Ordering, and advanced partitioning techniques, to significantly improve the performance of lakehouse pipelines. The enhancements were particularly focused on jobs responsible for managing over 300 data pipelines, ensuring a more efficient and responsive data processing environment.
- Migration to Unity Catalog: As a value-added service, Entrada migrated the client’s data pipelines to Unity Catalog. This strategic move not only improved data governance and security, but also laid the foundation for future AI and machine learning initiatives, enhancing the client’s ability to innovate and scale.
- Optimization for DLT Compliance: To further streamline operations, Entrada optimized pipelines to be compliant with Delta Live Tables (DLT). This adjustment reduced operational complexities and ensured a more robust and reliable data processing framework.
The Results
Through strategic collaboration with Entrada, the client successfully enhanced the performance and cost-efficiency of its data infrastructure. The optimizations not only delivered immediate value in terms of cost savings and performance improvements, but also positioned the client for future growth, enabling the company to better leverage its data assets for innovation and competitive advantage:
- 72% Cost Savings: The largest data pipeline saw a 72% reduction in run costs, translating into significant financial savings.
- 82% Reduction in Initial Load Time: The initial load time for data processing was reduced by more than 82%, accelerating data availability and enhancing operational efficiency.
- 50% Reduction in Run Time: The runtime for the largest tables was cut by over 50%, leading to faster data processing and improved overall system performance.
About Entrada
Entrada is a Databricks-focused consulting and implementation partner backed by Databricks Ventures. Entrada harnesses the power of Databricks to help customers accelerate their AI + data initiatives. Our expertise in AI/ML, Databricks, and analytics is centered around industry-centric solutions. Our mission is to simplify complex data + AI challenges and support end-to-end transformations, delivering future-ready solutions fast.
