Data Engineer
At a glance
Mid-level Data Engineering role at Malomatia. India.
Pay not stated Data Engineer salary
dataroles summary, based on the employer's posting.
What you'll do
- Build and maintain ETL pipelines that move data from different sources into their destinations
- Automate data processing and add checks that keep information reliable and accurate
- Create secure storage and analytics solutions with Azure services, Azure SQL Database or Databricks
- Track and improve pipeline, database and query performance while resolving processing issues
- Protect sensitive information through encryption and controlled access
What you bring
- At least five years of relevant work in data engineering, including pipeline development on Azure
- Five or more years building solutions with Databricks
- A bachelor's degree in computer science, information technology, data engineering or a related area
- Working ability in one scripting language, such as Python, PowerShell or another listed language
Who this fits
This mid-level role suits someone who has hands-on experience designing data pipelines on Azure and developing with Databricks. You'll work with data architects, analysts and business system owners to deliver data ingestion solutions.
From the employer
The Data Engineer is responsible for building, managing, and optimizing data pipelines and ensuring data flow from source to destination. This role requires expertise in data architecture, data lakehouse, data warehousing, and data integration. You will work closely with data architects, data analysts, business system owners to deliver high-quality data ingestion solutions that meet business needs.
- Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading (ETL) data from various sources.
- Automate data workflows to ensure data is consistently and reliably processed.
- Design and implement scalable and secure data solutions using Azure analytics services, Azure SQL Database and/or Databricks.
- Develop and maintain data storage solutions, including data warehouses, data lakes, and databases, ensuring optimal performance and cost efficiency.
- Implement and maintain data quality checks to ensure data accuracy & consistency.
- Implement security measures to protect sensitive data, including encryption &, access control.
- Identify, troubleshoot, and resolve issues related to data pipelines, storage, and processing.
- Monitor and optimize the performance of data pipelines, databases & queries.
Education: Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
Must Have:
- 5+ years of relevant work experience in data engineering.
- Proven hands-on experience data pipeline design & development on Azure cloud platform.
- 5+ years experience developing solutions using Databricks,
- Proficiency in at least one scripting languages such as Python, PowerShell, JavaScript or Scala.
Nice to have:
- Previous experience with Informatica IICS data governance or IDQ a strong plus.
- MS Power BI Development experience is good to have.
- Experience with Oracle Analytics Cloud a strong plus.