Data Integration System Engineer
At a glance
Mid-level Data Engineering role at City of Philadelphia. Philadelphia · full-time · $87,550 base.
dataroles summary, based on the employer's posting.
What you'll do
- Build cloud deployment and automation approaches for Linux and Windows server stacks, using infrastructure as code and related practices
- Set up system monitoring with tools such as Grafana, Prometheus and AWS CloudWatch
- Help data engineers maintain DataBridge, Airflow, MuleSoft and AIS for integrations and open-data work
- Improve data pipelines with Airflow, custom Python software and other tools
- Document systems clearly and explain technical details to data stewards and nontechnical partners
What you bring
- You have production experience in DevOps or a related role, including testing and deployment practices
- You bring a computer science, information systems or related degree, or equivalent work experience
- You can administer Linux systems, use command-line tools with Bash and Python, and work with Kubernetes and Docker
- You understand Git-based version control and CI/CD, and have experience with cloud platforms and container orchestration
Who this fits
This mid-level role suits someone who can support production infrastructure while working with data engineers on pipelines and integrations. The position is onsite in Philadelphia, and you must live in the city within six months of starting. A cover letter is required with your application.
From the employer
With a workforce of over 25,000 people, and opportunities in more than 1,000 different job categories, the City of Philadelphia is the fifth largest city in the United States and one of the largest employers in Southeastern Pennsylvania. As an employer, the City of Philadelphia operates through the guiding principles of service, integrity, respect, accountability, collaboration, diversity and inclusion. Through these principles, we strive to effectively deliver services, to resolve the challenges facing our city, and to make Philadelphia a place where all our residents have the opportunity to reach their potential.
The Office of Innovation & Technology (OIT) is the central IT agency for the City of Philadelphia headed by the Chief Information Officer (CIO). OIT oversees all major information and communications technology initiatives for the City of Philadelphia - increasing the effectiveness of the information technology infrastructure, where the services provided are advanced, optimized, and responsive to the needs of the City of Philadelphia’s businesses, residents, and visitors. OIT responsibilities include: identifying the most effective approach for implementing new information technology directions throughout city government; improving the value of the city’s technology assets and the return on the city’s technology investments; ensuring data security continuity; planning for continuing operations in the event of disruption of information technology or communications services; and supporting accountable, efficient and effective government across every city department, board, commission and agency.
Dev-ops / data integration systems administration at the City offers a unique opportunity for impactful, dynamic work and professional growth, leveraging cutting-edge technologies. This is a position on a diverse team of engineers, analysts, and GIS specialists, contributing to developing, maintaining, and optimizing our data pipelines and integration strategies, assisting departments and agencies to innovate and enhance the City’s service to its residents.
This role will work closely with a senior Systems Engineer to support and enhance existing infrastructure. The dev-ops / data integration systems engineer will help deploy and streamline the team’s various server stacks and automations that host the City’s data engineering infrastructure, pipelines, models, and integrations. The team (CityGeo) uses a blend of open source, custom developed and off-the-shelf tools, including Kubernetes, Docker, OpenTofu (Terraform), Python, Bash, DBT, GIS, Apache Airflow, Jenkins, Postgres, PostGIS, AWS, GitHub, GitHub actions, MuleSoft as an iPaaS and SAS API provides like ArcGIS Online and CARTO.
Some prominent services the team is responsible for include DataBridge, OIT’s enterprise data warehouse, AIS, the City’s custom address information systems which supports public address search and internal geocoding, and Airflow, a custom workflow generator and orchestrator for departmental integrations (https://metadata.phila.gov/) and open data publishing (https://opendataphilly.org/). Some examples of OIT’s commitment to serving the public through the effective and innovative integration of City open data include:
https://streetsmartphl.phila.gov/
We are looking for a candidate who is not only technically skilled but also passionate about leveraging data to improve City operations and community life. This team is fundamental to the innovation of City services that directly affect residents. If you are eager to contribute to a team that values innovation, efficiency, service-oriented work and data-driven decision-making, we encourage you to apply.
Essential Functions
- Design and implement new deployment and automation strategies for maintaining and updating various server stacks in the cloud (AWS(primary)/Azure) on Linux and Windows servers, using Infrastructure as Code (IaC) and other best practices
- Configuring various system health monitors using Grafana, AlertManager, and Prometheus, AWS CloudWatch, SNS
- Work closely with data engineers to support integration and open data initiatives, developing and maintaining critical systems like the City’s DataBridge, Airflow, MuleSoft and AIS.
- Design, develop, expand and refine existing data pipelines using Apache Airflow, custom Python-based software, and other tools.
- Write clear, descriptive technical documentation for systems/applications, partner with data stewards of various technical levels and communicate technical details to non-technical stakeholders.
- Previous work experience in a dev-ops or related role that included managing systems/software stacks in a production environment with proper testing/deployment practices.
- Bachelor’s/advanced degree in Computer Science, Information Systems, a related field or equivalent work experience.
- Dedication to making Philadelphia municipal government more robust, efficient, and equitable.
- Previous work in a public sector or large organization setting is a plus.
Competencies:
- Clear and concise writing and communication skills – documentation driven.
- Creative problem-solving and critical thinking.
- Efficient time management and the ability to manage multiple workflows simultaneously.
Knowledge and Skills:
- Proficient in Systems Administration (Linux preferred, but we work with Windows too).
- Proficient with CLI (Linux and PowerShell), Bash and Python.
- Proficient with Kubernetes and Docker
- Automation Mindset: Leverages automation for maintainability/security with CI/CD pipelines.
- Familiar or ability to learn tools like GitHub Actions, Terraform, Jenkins, Airflow.
- Understanding of Git/GitHub for version control and CI/CD pipelines.
- Experience or ability to learn databases administration (Postgres preferred) and SQL (proficiency a plus).
- Experience working with cloud technologies, containers, and orchestration tools (AWS: Batch, ECS, EKS, CloudWatch).
- Experience working with spatial datasets a plus.
Abilities:
- Troubleshoot system health and understand the basics of deploying maintainable systems.
- Maintain clean and secure data environments.
- Work effectively in a hybrid (on-premises and cloud) environment.
- To communicate complex technical concepts in understandable terms.
Salary cannot exceed $87,550
We’re interested in hiring the best possible candidate for the role. We recognize that experience, education, and qualifications can be attained in a variety of ways and that many skillsets are transferable. If you feel you’re a good fit, please don’t hesitate to apply.
Please include:
- Resume
- A cover letter
Please note: Applications will not be considered without a cover letter.
Did you know?
● We are a Public Service Loan Forgiveness Program qualified employer : 25% tuition discount program for City employees (and sometimes spouses and dependents as well) in partnership with area colleges and universities
● We offer Comprehensive health coverage for employees and their eligible dependents
● Our wellness program offers eligibility into the discounted medical plan
● Employees receive paid vacation, sick leave, and holidays
● Generous retirement savings options are available
*The successful candidate must be a city of Philadelphia resident within six months of hire
Effective May 22, 2023, vaccinations are no longer required for new employees that work in non-medical, non-emergency or patient facing positions with the City of Philadelphia. As a result, only employees in positions providing services that are patient-facing medical care (ex: Nurses, doctors, emergency medical personnel), must be fully vaccinated.
The City of Philadelphia is an Equal Opportunity employer and does not permit discrimination based on race, ethnicity, color, sex, sexual orientation, gender identity, religion, national origin, ancestry, age, disability, marital status, source of income, familial status, genetic information or domestic or sexual violence victim status. If you believe you were discriminated against, call the Philadelphia Commission on Human Relations at 215-686-4670 or send an email to faqpchr @phila.gov. For more information, go to: Human Relations Website:
http://www.phila.gov/humanrelations/Pages/default.aspx
Data Engineering work from LinkedIn, X, Reddit and Upwork, in one list.
Each morning you get everything that fits, strongest first.
Data Engineering$80k+
Find me workBy the team behind dataroles.