Senior Big Data Engineer
At a glance
Senior Data Engineering role at Qualys. Pune · full-time.
Pay not stated Data Engineering salary
dataroles summary, based on the employer's posting.
What you'll do
- Build and operate high-throughput platform components and data pipelines with Spark and Kafka
- Investigate Spark jobs and Kafka flows, then improve performance as transaction volumes increase
- Shape architecture decisions with product, professional services and sales engineering teams
- Set engineering practices and system documentation, while guiding colleagues through difficult pipeline issues
What you bring
- At least five years developing data platforms or engineering data systems
- A bachelor's degree in computer science, computer engineering or another related technical field
- You have built and optimized distributed pipelines with Apache Spark and Apache Kafka
- You have scaled data infrastructure to handle heavy transaction workloads
Who this fits
This senior role suits an engineer who can tackle distributed pipeline performance and guide less experienced teammates. The position is based in Pune, India.
From the employer
Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!
Senior Big Data Engineer
Job Description: We are seeking a Senior Big Data Engineer to design, build, and operate core components of the data platform and pipeline architecture behind Qualys' Enterprise TruRisk Platform, processing billions of daily transactions.
Responsibilities:
- Design, build, and maintain high-throughput data platform and pipeline components using Spark and Kafka.
- Debug, tune, and optimize Spark jobs and Kafka pipelines to manage growing data volumes.
- Collaborate with product, professional services, and sales engineering teams on architecture design and technical trade-offs.
- Establish and refine technical standards, system documentation, and best practices across the engineering team.
- Mentor junior and mid-level engineers on complex technical issues and pipeline performance.
Required Qualifications:
- Minimum 5 years of experience in data engineering or big data platform development.
- Bachelor's degree in Computer Science, Computer Engineering, or a related technical field.
- Demonstrated experience designing, building, and optimizing distributed pipelines using Apache Spark and Apache Kafka.
- Hands-on experience scaling data platform infrastructure to process high-volume/high-throughput transaction workloads.
Preferred Qualifications:
- Experience working with high-volume, multi-tenant enterprise platform environments.
- Proficiency in Scala, Java, or Python for data processing applications.
- Proven experience mentoring junior or mid-level engineering team members.
Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.
Each morning you get everything that fits, strongest first.
Data Engineering
Find me workBy the team behind dataroles.