Lead Data Engineer
At a glance
Senior Data Engineering role at Nxp. Bangalore · full-time.
Pay not stated Lead Data Engineer salary
dataroles summary, based on the employer's posting.
What you'll do
- Build scalable Databricks pipelines and Lakehouse solutions, then tune clusters, queries and Delta Lake tables for cost and speed
- Set standards for partitioning, clustering and workload isolation, and investigate expensive queries or slow loads with source teams
- Configure Unity Catalog governance for access, lineage and security across the data platform
- Create Databricks monitoring and automated recovery, and guide CI/CD practices for deployment and releases
- Lead design reviews, mentor data engineers, and coordinate vendor cases, escalations and release adoption
What you bring
- At least six years in data engineering, including two or more years using Databricks in enterprise environments
- Strong knowledge of Databricks Lakehouse architecture, Delta Lake, Unity Catalog and workflow orchestration
- Production-scale experience improving Spark workload cost and performance, with advanced Python or PySpark and SQL skills
- Working knowledge of CI/CD and DevOps for data workloads, plus experience with Databricks observability tools
Who this fits
This senior role suits a hands-on Databricks engineer ready to set platform standards, mentor colleagues and influence partner teams without direct authority. The role is based in Bangalore, India.
From the employer
Position Summary:
We are seeking an experienced Senior Data Engineer to drive the performance, governance, and AI-native maturity of our enterprise Data Platform in Databricks. This is a Databricks-focused Data Engineering role with a working understanding of DevOps practices — designing scalable pipelines, tuning workloads for performance and cost, and operationalizing modern data and AI capabilities on Lakehouse.
The ideal candidate has deep, hands-on Databricks expertise, a strong performance-engineering instinct, and a builder's mindset for AI-assisted operations. You'll own the Databricks performance and governance standards for the platform, mentor engineers, and shape the direction for AI-native operations.
Key Responsibilities:
- Design and develop scalable data pipelines and Lakehouse solutions on Databricks.
- Tune Databricks workloads for performance and cost, including cluster sizing, query optimization, and Delta Lake table design.
- Establish and enforce best practices for partitioning, clustering, and workload isolation.
- Track performance trends, identify high-cost queries, and partner with source teams and end users to resolve long-running loads.
- Design and operationalize Unity Catalog for data governance — access control, lineage, and security.
- Build monitoring and self-healing automation using Databricks-native AI and agentic capabilities.
- Drive CI/CD workflows for Databricks assets, setting DevOps best practices for deployment and release management.
- Lead design reviews and mentor Data Engineers on Databricks best practices and AI-native features.
- Own Databricks vendor coordination — case management, escalations, and release adoption strategy.
What Success Looks Like (First 6–12 Months)
- Within 6–12 months, you'll define the platform's tuning and governance standards, lead design reviews, mentor junior engineers, and shape the AI-native operations roadmap.
Required Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent relevant experience.
- 6+ years of data engineering experience with 2+ years hands-on Databricks in enterprise settings.
- Deep understanding of Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and Workflow orchestration.
- Proven ability to tune Spark workloads for cost and performance at production scale.
- Advanced Python (PySpark) and SQL skills.
- Working knowledge of CI/CD practices and DevOps principles applied to data workloads.
- Experience with observability tooling for Databricks.
Preferred Qualifications:
- Experience with Databricks-native AI capabilities and agentic frameworks.
- Familiarity with Databricks Serverless Compute and DBSQL performance tuning.
- A Databricks Certified Professional.
- Exposure to Infrastructure-as-Code is a plus.
Competencies:
- Performance-engineering mindset — measures, tunes, and re-measures.
- Curiosity for AI-native operations and continuous automation.
- Strong sense of platform ownership — quality, cost, and reliability.
- Effective communication with engineering peers, vendors, and business stakeholders.
- Influence outcomes across source teams, vendors, and business stakeholders without direct authority.
Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.
Each morning you get everything that fits, strongest first.
Data Engineering
Find me workBy the team behind dataroles.