Senior Data Engineer (Databricks, Azure)
At a glance
Senior Data Engineering role at Acuity Analytics. Ref: 26508, Malta, Bulgaria, Portugal, Malta · full-time.
Pay not stated Data Engineer salary
dataroles summary, based on the employer's posting.
What you'll do
- Build Databricks Workflows to coordinate large-scale data processing
- Create PyFunc models and connect them to pipelines running in production
- Secure credentials and settings with Azure Key Vault, and automate data flows through Azure Logic Apps
- Design and tune ETL and ELT pipelines, then govern assets with Unity Catalog and Data Lakehouse patterns
- Set up event-driven messaging with Azure Service Bus and manage CI/CD pipelines in Azure DevOps
What you bring
- Strong Python skills for data engineering, automation and integration, with experience producing modular, production-ready code
- Hands-on capability with Databricks Workflows, Unity Catalog, Delta Lake and ETL/ELT implementation
- Strong SQL knowledge, including query tuning and data modeling
- Experience with Azure services including Key Vault, Logic Apps and Service Bus, or another listed service
Who this fits
This senior data engineering role suits you if you can build cloud-based data solutions and work with architects, analysts and data scientists. You’ll collaborate on end-to-end delivery, with a focus on automation and performance.
From the employer
If you like providing efficient, effective solutions to complex challenges, join us at Acuity! Be at the heart of helping achieve extraordinary things with software and data for our customers!
About Us
Ascent has recently been acquired by Acuity Analytics. This is both a significant milestone for us and a tremendous opportunity for you. Acuity Analytics is a business with a strong global reputation, an impressive client base and ambitious growth plans. We delivers deep insights and domain-led digital transformation to high-growth and heavily regulated organisations. To our customers, we bring a partnership that provides the talent, technology and capability to enhance performance and operational efficiency.
About the role
We are looking for a skilled Data Engineer with strong experience in Databricks, Azure Cloud, and data orchestration to join our team. You will be responsible for designing, building, and managing scalable data solutions in a cloud-native environment, with a focus on automation, performance, and collaboration.
Key Responsibilities
- Build and manage Databricks Workflows for large-scale data orchestration.
- Develop and deploy PyFunc models and integrate them into production pipelines.
- Implement secure secrets and configuration management using Azure Key Vault.
- Automate data flows using Azure Logic Apps.
- Design and optimize ETL / ELT pipelines in line with best practices.
- Govern and manage data assets using Unity Catalog and Data Lakehouse patterns.
- Implement messaging and event-driven workflows with Azure Service Bus.
- Collaborate with architects, analysts, and data scientists to deliver end-to-end solutions.
- Utilize Azure DevOps for CI/CD pipeline management
Skills Required
Data Engineering & Orchestration
- Databricks Workflows, Unity Catalog, Delta Lake
- ETL/ELT design and implementation
- Data Lakehouse architecture
Programming
- Strong Python skills for data engineering, automation, and integration
- Experience writing modular, production-ready code
- CI/CD pipelines for Databricks and Azure
Databases
- Strong knowledge of SQL (including query optimization and data modeling)
- Exposure to NoSQL databases (e.g., Cosmos DB)
Azure Cloud Services
- Azure Key Vault
- Azure Logic Apps
- Azure Service Bus
- API Management (APIM)
Nice to Haves
AI & Cognitive Services
- Experience with Microsoft Cognitive Services (OCR, Layout, Form Recognizer, etc.)
- Integration of cognitive services into data pipelines
Architecture & Monitoring
- Familiarity with event-driven and microservices architecture patterns
- Experience with monitoring tools like Datadog, Application Insights, or similar
Collaboration & Methodologies
- Agile team collaboration
- DevOps practices
Why join us
People are at the Heart of our Business. By investing in people, we achieve exceptional results for our clients and create new opportunities for our teams to thrive. Check out this page for more details.
If you have any questions contact our Talent Acquisition team on ta.admin@acuityanalytics.
For more details about Acuity Analytics please see here: Read here.
Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.
Each morning you get everything that fits, strongest first.
Data Engineering
Find me workBy the team behind dataroles.