Opportunity Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list. Find me work
dataroles

AVP - Lead Databricks Data Engineer

Fa Ewjt Saasfaprod1 Open
Hybrid New York Full-time Senior Data Engineering

At a glance

Senior Data Engineering role at Fa Ewjt Saasfaprod1. New York · full-time · no salary stated · we estimate $150,500–$208,708.

Estimated pay

Fa Ewjt Saasfaprod1 has not published a salary for this role. The figure below is our estimate, not theirs.

$150,500 – $208,708
What the middle half of senior data engineering roles in New York paid, among postings that did state a salary.
Worked out from 57 postings that stated pay, median $187,500. Last updated Nov 21, 2026.
RoleData Engineering
SenioritySenior
LocationNew York
WorkplaceHybrid
EmploymentFull-time
PostedNov 20, 2026
Role brief

dataroles summary, based on the employer's posting.

What you'll do

  • Design Databricks Lakehouse platforms and build batch and streaming pipelines, data models, and ingestion approaches
  • Lead Delta Lake, Unity Catalog, Spark and PySpark delivery, while setting engineering practices and reviewing architecture and code
  • Modernize cloud data platforms and improve workload cost, speed, resilience, and operational monitoring
  • Set governance, lineage, security, and data quality controls in partnership with security and compliance teams
  • Turn business data needs into technical plans, coordinate delivery risks and resources, and explain design choices to leaders

What you bring

  • At least twelve years in IT, including six or more years working directly with Databricks
  • Seven or more years building data engineering solutions with Spark or PySpark
  • Experience designing enterprise data lake and lakehouse platforms and leading major modernization efforts
  • A track record of leading technical teams and making architecture decisions

Who this fits

This role suits a senior, hands-on technical lead who can shape platform architecture, guide engineers, and work with business and technical stakeholders. The posting lists the work location as the US.

From the employer

Job Description: Lead Databricks Data Engineer

Job Title: Lead Databricks Data Engineer
Experience: 10-12 Years
Location: US
Employment Type: Full-Time

Job Summary

We are looking for a highly experienced Lead Databricks Data Engineer with 10-12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms. The ideal candidate will lead the design, development, modernization, and optimization of enterprise-scale data platforms using Databricks Lakehouse architecture. This role requires strong technical leadership, solution architecture expertise, stakeholder management skills, and hands-on experience building scalable cloud-native data ecosystems.

The candidate will be responsible for defining data engineering standards, mentoring teams, driving modernization initiatives, and delivering high-performance data solutions that support analytics, AI/ML, and business intelligence initiatives.

Key Responsibilities

Data Engineering & Architecture

  • Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
  • Architect and develop scalable ETL/ELT data pipelines for batch and real-time processing.
  • Define enterprise data models, ingestion frameworks, and transformation strategies.
  • Lead the implementation of Delta Lake, Unity Catalog, and data governance frameworks.
  • Build and optimize highly scalable Spark and PySpark solutions handling large-volume datasets.
  • Drive cloud data platform modernization and migration initiatives.

Technical Leadership

  • Serve as the technical lead for Databricks and Data Engineering programs.
  • Establish engineering best practices, coding standards, and architecture guidelines.
  • Conduct architecture reviews, code reviews, and solution assessments.
  • Mentor and guide Data Engineers, Technical Leads, and junior team members.
  • Collaborate with Enterprise Architects, Product Owners, Data Scientists, and Business Stakeholders.

Cloud & Platform Engineering

  • Design cloud-native data solutions on Azure, AWS, or GCP.
  • Implement secure, scalable, and resilient data architectures.
  • Optimize costs, performance, reliability, and availability of data platforms.
  • Lead CI/CD and Infrastructure-as-Code implementation for data workloads.
  • Define and implement monitoring, observability, and operational excellence practices.

Governance & Data Quality

  • Establish data governance, lineage, cataloging, and security controls.
  • Ensure compliance with organizational and regulatory requirements.
  • Implement data quality frameworks and monitoring solutions.
  • Collaborate with security and compliance teams to enforce data protection standards.

Stakeholder & Delivery Management

  • Partner with business leaders to understand strategic data requirements.
  • Translate business requirements into scalable technical solutions.
  • Drive delivery planning, risk management, and resource coordination.
  • Present architecture decisions and technical recommendations to leadership teams.

Required Technical Skills

Databricks

  • Databricks Lakehouse Platform
  • Databricks Workflows
  • Delta Lake
  • Unity Catalog
  • Structured Streaming
  • Databricks SQL
  • Delta Live Tables (DLT)
  • Databricks Asset Bundles

Big Data Technologies

  • Apache Spark
  • PySpark
  • Spark SQL
  • Kafka
  • Real-Time Streaming Architectures
  • Data Lake & Lakehouse Architectures

Cloud Platforms (Strong Experience in One or More)

Azure

  • Azure Databricks
  • Azure Data Factory (ADF)
  • ADLS Gen2
  • Azure Synapse Analytics
  • Azure Key Vault
  • Event Hub

AWS

  • S3
  • EMR
  • Glue
  • Redshift
  • Lambda
  • IAM

GCP

  • BigQuery
  • Cloud Storage
  • Dataflow
  • Pub/Sub
  • Composer

Programming

  • Python
  • SQL
  • PySpark
  • Scala (Preferred)
  • Shell Scripting

DevOps & Automation

  • Git
  • Azure DevOps
  • GitHub Actions
  • Jenkins
  • Terraform
  • Docker
  • Kubernetes

Data Warehousing

  • Dimensional Modeling
  • Star & Snowflake Schema
  • Data Vault
  • Master Data Management
  • Data Governance Frameworks

Required Experience

  • 12-15 years of overall IT experience.
  • Minimum 6+ years of hands-on Databricks experience.
  • 7+ years of experience in Spark/PySpark-based data engineering.
  • Experience designing enterprise-scale Data Lake and Lakehouse solutions.
  • Strong experience leading large data transformation and modernization programs.
  • Experience handling structured, semi-structured, and unstructured data processing.
  • Experience supporting analytics, AI/ML, and reporting ecosystems.
  • Proven experience managing technical teams and driving architectural decisions.

Preferred Qualifications

  • Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related field.
  • Databricks Certified Data Engineer Professional.
  • Azure Data Engineer Associate.
  • AWS Data Analytics Specialty.
  • Google Professional Data Engineer Certification.
  • TOGAF or Cloud Architecture certifications are a plus.

Nice to Have

  • MLOps and AI Engineering exposure.
  • Snowflake and Databricks integration experience.
  • Microsoft Fabric exposure.
  • Data Mesh implementation experience.
  • Data Governance tools such as Collibra, Alation, or Informatica.
  • Expertise in Data Quality frameworks such as Great Expectations or Deequ.

Key Competencies

  • Solution Architecture
  • Technical Leadership
  • Stakeholder Management
  • Program Delivery
  • Data Strategy & Governance
  • Team Mentoring & Coaching
  • Problem Solving & Decision Making
  • Communication & Executive Presentation Skills
  • Performance Optimization
  • Innovation & Continuous Improvement

Technology Classification

Technology Tower: Data & AI
Practice Area: Data Engineering
Primary Skill: Databricks Lakehouse Engineering
Secondary Skills: Apache Spark, PySpark, Cloud Data Platforms (Azure/AWS/GCP), Data Architecture, Data Governance, MLOps.

Job Description: Lead Databricks Data Engineer

Job Title: Lead Databricks Data Engineer
Experience: 10-12 Years
Location: US
Employment Type: Full-Time

Job Summary

We are looking for a highly experienced Lead Databricks Data Engineer with 10-12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms. The ideal candidate will lead the design, development, modernization, and optimization of enterprise-scale data platforms using Databricks Lakehouse architecture. This role requires strong technical leadership, solution architecture expertise, stakeholder management skills, and hands-on experience building scalable cloud-native data ecosystems.

The candidate will be responsible for defining data engineering standards, mentoring teams, driving modernization initiatives, and delivering high-performance data solutions that support analytics, AI/ML, and business intelligence initiatives.

Key Responsibilities

Data Engineering & Architecture

  • Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
  • Architect and develop scalable ETL/ELT data pipelines for batch and real-time processing.
  • Define enterprise data models, ingestion frameworks, and transformation strategies.
  • Lead the implementation of Delta Lake, Unity Catalog, and data governance frameworks.
  • Build and optimize highly scalable Spark and PySpark solutions handling large-volume datasets.
  • Drive cloud data platform modernization and migration initiatives.

Technical Leadership

  • Serve as the technical lead for Databricks and Data Engineering programs.
  • Establish engineering best practices, coding standards, and architecture guidelines.
  • Conduct architecture reviews, code reviews, and solution assessments.
  • Mentor and guide Data Engineers, Technical Leads, and junior team members.
  • Collaborate with Enterprise Architects, Product Owners, Data Scientists, and Business Stakeholders.

Cloud & Platform Engineering

  • Design cloud-native data solutions on Azure, AWS, or GCP.
  • Implement secure, scalable, and resilient data architectures.
  • Optimize costs, performance, reliability, and availability of data platforms.
  • Lead CI/CD and Infrastructure-as-Code implementation for data workloads.
  • Define and implement monitoring, observability, and operational excellence practices.

Governance & Data Quality

  • Establish data governance, lineage, cataloging, and security controls.
  • Ensure compliance with organizational and regulatory requirements.
  • Implement data quality frameworks and monitoring solutions.
  • Collaborate with security and compliance teams to enforce data protection standards.

Stakeholder & Delivery Management

  • Partner with business leaders to understand strategic data requirements.
  • Translate business requirements into scalable technical solutions.
  • Drive delivery planning, risk management, and resource coordination.
  • Present architecture decisions and technical recommendations to leadership teams.

Required Technical Skills

Databricks

  • Databricks Lakehouse Platform
  • Databricks Workflows
  • Delta Lake
  • Unity Catalog
  • Structured Streaming
  • Databricks SQL
  • Delta Live Tables (DLT)
  • Databricks Asset Bundles

Big Data Technologies

  • Apache Spark
  • PySpark
  • Spark SQL
  • Kafka
  • Real-Time Streaming Architectures
  • Data Lake & Lakehouse Architectures

Cloud Platforms (Strong Experience in One or More)

Azure

  • Azure Databricks
  • Azure Data Factory (ADF)
  • ADLS Gen2
  • Azure Synapse Analytics
  • Azure Key Vault
  • Event Hub

AWS

  • S3
  • EMR
  • Glue
  • Redshift
  • Lambda
  • IAM

GCP

  • BigQuery
  • Cloud Storage
  • Dataflow
  • Pub/Sub
  • Composer

Programming

  • Python
  • SQL
  • PySpark
  • Scala (Preferred)
  • Shell Scripting

DevOps & Automation

  • Git
  • Azure DevOps
  • GitHub Actions
  • Jenkins
  • Terraform
  • Docker
  • Kubernetes

Data Warehousing

  • Dimensional Modeling
  • Star & Snowflake Schema
  • Data Vault
  • Master Data Management
  • Data Governance Frameworks

Required Experience

  • 12-15 years of overall IT experience.
  • Minimum 6+ years of hands-on Databricks experience.
  • 7+ years of experience in Spark/PySpark-based data engineering.
  • Experience designing enterprise-scale Data Lake and Lakehouse solutions.
  • Strong experience leading large data transformation and modernization programs.
  • Experience handling structured, semi-structured, and unstructured data processing.
  • Experience supporting analytics, AI/ML, and reporting ecosystems.
  • Proven experience managing technical teams and driving architectural decisions.

Preferred Qualifications

  • Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related field.
  • Databricks Certified Data Engineer Professional.
  • Azure Data Engineer Associate.
  • AWS Data Analytics Specialty.
  • Google Professional Data Engineer Certification.
  • TOGAF or Cloud Architecture certifications are a plus.

Nice to Have

  • MLOps and AI Engineering exposure.
  • Snowflake and Databricks integration experience.
  • Microsoft Fabric exposure.
  • Data Mesh implementation experience.
  • Data Governance tools such as Collibra, Alation, or Informatica.
  • Expertise in Data Quality frameworks such as Great Expectations or Deequ.

Key Competencies

  • Solution Architecture
  • Technical Leadership
  • Stakeholder Management
  • Program Delivery
  • Data Strategy & Governance
  • Team Mentoring & Coaching
  • Problem Solving & Decision Making
  • Communication & Executive Presentation Skills
  • Performance Optimization
  • Innovation & Continuous Improvement

Technology Classification

Technology Tower: Data & AI
Practice Area: Data Engineering
Primary Skill: Databricks Lakehouse Engineering
Secondary Skills: Apache Spark, PySpark, Cloud Data Platforms (Azure/AWS/GCP), Data Architecture, Data Governance, MLOps.

Job Description: Lead Databricks Data Engineer

Job Title: Lead Databricks Data Engineer
Experience: 10-12 Years
Location: US
Employment Type: Full-Time

Job Summary

We are looking for a highly experienced Lead Databricks Data Engineer with 10-12 years of experience in Data Engineering, Big Data, and Cloud Data Platforms. The ideal candidate will lead the design, development, modernization, and optimization of enterprise-scale data platforms using Databricks Lakehouse architecture. This role requires strong technical leadership, solution architecture expertise, stakeholder management skills, and hands-on experience building scalable cloud-native data ecosystems.

The candidate will be responsible for defining data engineering standards, mentoring teams, driving modernization initiatives, and delivering high-performance data solutions that support analytics, AI/ML, and business intelligence initiatives.

Key Responsibilities

Data Engineering & Architecture

  • Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
  • Architect and develop scalable ETL/ELT data pipelines for batch and real-time processing.
  • Define enterprise data models, ingestion frameworks, and transformation strategies.
  • Lead the implementation of Delta Lake, Unity Catalog, and data governance frameworks.
  • Build and optimize highly scalable Spark and PySpark solutions handling large-volume datasets.
  • Drive cloud data platform modernization and migration initiatives.

Technical Leadership

  • Serve as the technical lead for Databricks and Data Engineering programs.
  • Establish engineering best practices, coding standards, and architecture guidelines.
  • Conduct architecture reviews, code reviews, and solution assessments.
  • Mentor and guide Data Engineers, Technical Leads, and junior team members.
  • Collaborate with Enterprise Architects, Product Owners, Data Scientists, and Business Stakeholders.

Cloud & Platform Engineering

  • Design cloud-native data solutions on Azure, AWS, or GCP.
  • Implement secure, scalable, and resilient data architectures.
  • Optimize costs, performance, reliability, and availability of data platforms.
  • Lead CI/CD and Infrastructure-as-Code implementation for data workloads.
  • Define and implement monitoring, observability, and operational excellence practices.

Governance & Data Quality

  • Establish data governance, lineage, cataloging, and security controls.
  • Ensure compliance with organizational and regulatory requirements.
  • Implement data quality frameworks and monitoring solutions.
  • Collaborate with security and compliance teams to enforce data protection standards.

Stakeholder & Delivery Management

  • Partner with business leaders to understand strategic data requirements.
  • Translate business requirements into scalable technical solutions.
  • Drive delivery planning, risk management, and resource coordination.
  • Present architecture decisions and technical recommendations to leadership teams.

Required Technical Skills

Databricks

  • Databricks Lakehouse Platform
  • Databricks Workflows
  • Delta Lake
  • Unity Catalog
  • Structured Streaming
  • Databricks SQL
  • Delta Live Tables (DLT)
  • Databricks Asset Bundles

Big Data Technologies

  • Apache Spark
  • PySpark
  • Spark SQL
  • Kafka
  • Real-Time Streaming Architectures
  • Data Lake & Lakehouse Architectures

Cloud Platforms (Strong Experience in One or More)

Azure

  • Azure Databricks
  • Azure Data Factory (ADF)
  • ADLS Gen2
  • Azure Synapse Analytics
  • Azure Key Vault
  • Event Hub

AWS

  • S3
  • EMR
  • Glue
  • Redshift
  • Lambda
  • IAM

GCP

  • BigQuery
  • Cloud Storage
  • Dataflow
  • Pub/Sub
  • Composer

Programming

  • Python
  • SQL
  • PySpark
  • Scala (Preferred)
  • Shell Scripting

DevOps & Automation

  • Git
  • Azure DevOps
  • GitHub Actions
  • Jenkins
  • Terraform
  • Docker
  • Kubernetes

Data Warehousing

  • Dimensional Modeling
  • Star & Snowflake Schema
  • Data Vault
  • Master Data Management
  • Data Governance Frameworks

Required Experience

  • 12-15 years of overall IT experience.
  • Minimum 6+ years of hands-on Databricks experience.
  • 7+ years of experience in Spark/PySpark-based data engineering.
  • Experience designing enterprise-scale Data Lake and Lakehouse solutions.
  • Strong experience leading large data transformation and modernization programs.
  • Experience handling structured, semi-structured, and unstructured data processing.
  • Experience supporting analytics, AI/ML, and reporting ecosystems.
  • Proven experience managing technical teams and driving architectural decisions.

Preferred Qualifications

  • Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related field.
  • Databricks Certified Data Engineer Professional.
  • Azure Data Engineer Associate.
  • AWS Data Analytics Specialty.
  • Google Professional Data Engineer Certification.
  • TOGAF or Cloud Architecture certifications are a plus.

Nice to Have

  • MLOps and AI Engineering exposure.
  • Snowflake and Databricks integration experience.
  • Microsoft Fabric exposure.
  • Data Mesh implementation experience.
  • Data Governance tools such as Collibra, Alation, or Informatica.
  • Expertise in Data Quality frameworks such as Great Expectations or Deequ.

Key Competencies

  • Solution Architecture
  • Technical Leadership
  • Stakeholder Management
  • Program Delivery
  • Data Strategy & Governance
  • Team Mentoring & Coaching
  • Problem Solving & Decision Making
  • Communication & Executive Presentation Skills
  • Performance Optimization
  • Innovation & Continuous Improvement

Technology Classification

Technology Tower: Data & AI
Practice Area: Data Engineering
Primary Skill: Databricks Lakehouse Engineering
Secondary Skills: Apache Spark, PySpark, Cloud Data Platforms (Azure/AWS/GCP), Data Architecture, Data Governance, MLOps.

Opportunity

Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.

Each morning you get everything that fits, strongest first.

Data Engineering

Find me work

By the team behind dataroles.

Find me work
Opportunity

Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.

Each morning you get everything that fits, strongest first.

Find me work

By the team behind dataroles.