Senior Data Engineer - Gurugram
At a glance
Senior Data Engineering role at ClanX. Gurgaon · full-time.
Pay not stated Data Engineer salary
dataroles summary, based on the employer's posting.
What you'll do
- Build and run batch, incremental and CDC pipelines that ingest sensor and operational data into the lakehouse
- Map client schemas and tags, then normalize time-series data and handle resampling and gaps
- Shape raw data into documented tables and maintain datasets used for ML features and self-serve analytics
- Improve Spark or PySpark and SQL performance by tuning partitions, storage formats and query costs
- Set up data checks, freshness monitoring, lineage and alerts, while improving PostgreSQL schemas, indexes and queries
What you bring
- At least four years building production pipelines used by real data consumers
- Strong Python and SQL skills, including query tuning, window functions and production-quality code
- Production PostgreSQL schema design experience, with knowledge of indexing, transactions, locking and query plans
- Hands-on distributed processing with Spark, PySpark or an equivalent engine, plus experience with an orchestrator and AWS or Azure
- A firm grasp of pipeline reliability, data modeling and distributed-system behavior, including idempotency, retries and failure recovery
Who this fits
This suits a senior engineer who can take ownership of pipelines and a lakehouse data domain, from ingestion through modeling and quality. The role is hybrid in Gurugram, with two to three days on-site.
From the employer
Mechademy is hiring a Senior Data Engineer to build and scale reliable data platforms, pipelines, and models that power enterprise AI, machine learning, and analytics for industrial asset monitoring and predictive maintenance.
Company Details
Mechademy is an enterprise AI company building real-time monitoring, diagnostics, and predictive maintenance solutions for industrial equipment. The company serves clients across oil & gas, power generation, and LNG sectors through production-grade AI and physics-informed machine learning systems.
Website: https://mechademy.com/
Responsibilities
What You’ll Own
Lakehouse Pipelines & Ingestion (35%)
Design and own batch ETL/ELT and CDC pipelines that bring sensor and operational data into the lakehouse, orchestrated in Dagster
Build for reliability: idempotent, incremental, backfill-safe pipelines with sane retry and failure handling, that still produce correct output when a worker is killed mid-run or a message is delivered twice
Onboard new client data sources: schema and tag mapping, time-series normalization, resampling, gap handling at scale
2. Modeling & Serving (25%)
Model raw data into well-structured, documented tables that downstream ML and analytics can trust
Build and maintain the datasets behind ML feature pipelines and the lakehouse layer powering self-serve analytics
Write performant Spark/PySpark and SQL; optimize partitioning, storage formats, and query cost
3. Data Quality & Reliability (10%)
Own data quality: validation, freshness/SLA monitoring, and observability so bad data is caught before it reaches consumers
Make the data layer debuggable: lineage, tests, and alerting that tell you what broke and where
Reason about failure modes across the whole path (queue, worker, orchestrator, database, object store) and design so that a partial failure leaves the system in a state you can recover from
4. Relational & Operational Data (30%)
Contribute to the schema, indexing, and query performance of the relational database the product runs on
Design tables and constraints so that correctness is enforced at the database layer, and diagnose slow queries from their plans
Own retention and the boundary between the operational database and the lakehouse: what stays, what moves, and how it gets there
What Success Looks Like
First 30 days: Productive in the codebase and orchestration layer. First pipeline change merged.
First 90 days: Independently shipping and owning pipelines. Onboarded at least one new data source end-to-end.
First 6 months: Owning a lakehouse data domain, its ingestion, models, and quality, that ML and analytics teams rely on you to drive.
Requirement
Must-Have
4+ years building production data pipelines: real systems with real consumers, not just one-off scripts
Strong data engineering fundamentals: data modeling, batch vs. streaming, idempotency, incremental processing, partitioning.
Expert SQL and strong Python: query optimization, window functions, clean production-quality code
Relational database depth: you’ve designed schemas for a production PostgreSQL (or equivalent) system and understand normalization and when to break it, indexing strategies, transactions and isolation levels, locking, and how to read a query plan and fix the query
Distributed systems fundamentals: at-least-once delivery and idempotent consumers, partitioning and its effect on ordering, consistency and durability trade-offs, retries, timeouts, and backpressure. You can explain what happens to in-flight work when a worker or a database node dies
Hands-on with a distributed processing engine (Spark/PySpark or equivalent) on non-trivial data volumes
Experience with an orchestrator (Dagster, Airflow, Prefect, or equivalent) and a cloud platform (AWS/Azure)
Data-quality mindset: you build validation and monitoring into pipelines, not after something breaks
Strong Signals (Nice-to-Have)
Time-series or high-frequency sensor data at scale
TimescaleDB or another time-series database (hypertables, continuous aggregates, compression, retention policies)
Warehouse/lakehouse modeling (Delta/Iceberg/Snowflake/Redshift or equivalent) and file-format/partition tuning (Parquet)
CDC / database-replication pipelines
Message brokers or task queues in production (Kafka, RabbitMQ, or equivalent)
Building data for ML: feature pipelines, training datasets, serving consistency
dbt or similar transformation/modeling frameworks
Docker, Terraform/IaC, CI/CD for data
IoT, energy, or industrial sector experience. Not required, but it compresses your ramp
Job Details
Gurugram - Hybrid (2–3 days on-site)
Interview Process
Technical Round (Python & SQL)
System Design Round
Culture Fit Round
Important Note
ClanX is a recruitment partner, helping Mechademy hire a Senior Data Engineer.
Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.
Each morning you get everything that fits, strongest first.
Data Engineering
Find me workBy the team behind dataroles.