Associate Data Engineer, Enterprise Data Engineering

Job Posting Organization:
The World Bank Group is a prominent global partnership consisting of five institutions dedicated to addressing the most pressing development challenges faced by countries around the world. Established with the mission to end extreme poverty and promote shared prosperity, the organization has grown to include 189 member countries and operates more than 130 offices globally. The World Bank Group is recognized as one of the largest sources of funding and knowledge for developing nations, leveraging its extensive resources to invest in innovative projects that utilize data, research, and technology to create sustainable solutions. The organization is committed to fostering a culture of responsible innovation and collaboration, working closely with both public and private sector partners to achieve its goals. For more information about the World Bank Group, visit their official website at www.worldbank.org.

Job Overview:
The Associate Data Engineer position within the World Bank Group's Information and Technology Solutions (ITS) Vice Presidency is a critical role that focuses on the development and maintenance of data pipelines essential for the organization's operations. This position is designed for individuals who are passionate about leveraging data to solve complex problems and drive impactful change. The role involves working closely with various teams to design, develop, and optimize data pipelines that support both batch and streaming workloads. The Associate Data Engineer will be responsible for building ETL/ELT workflows, integrating data from diverse sources, and ensuring that data products are compliant with enterprise standards. This position also emphasizes the importance of collaboration, as the Associate Data Engineer will partner with data architects, business analysts, and data scientists to align pipeline development with organizational needs. The role is pivotal in advancing the Bank's digital transformation efforts and fostering a culture of data-driven decision-making across the institution.

Duties and Responsibilities:
The Associate Data Engineer will have a comprehensive set of responsibilities that include:

• Data Pipeline Development: Designing, developing, and maintaining data pipelines for ingestion, transformation, and serving across batch and streaming workloads. Building ETL/ELT workflows to integrate data from various sources into enterprise data platforms. Developing document ingestion and processing pipelines that incorporate OCR, metadata extraction, content enrichment, classification, and indexing capabilities. Implementing change data capture (CDC) patterns for real-time data synchronization and optimizing pipeline performance for efficiency.
• Federated Data Pipelines & Domain Enablement: Supporting federated data pipeline architecture that allows Line of Business (LOB) teams to manage their domain data. Contributing to self-serve data infrastructure and developing standardized deployment patterns for LOB teams.
• Templates, Blueprints & Patterns: Creating reusable pipeline templates and Infrastructure as Code (IaC) patterns for common data product types. Documenting best practices and reference architectures to guide LOB teams in building compliant pipelines.
• Data Integration: Integrating data from multiple internal and external sources into unified data assets and ensuring data consistency across integrated sources.
• Data Modeling & Transformation: Implementing medallion architecture patterns for data organization and quality progression, developing dimensional models, and optimizing data models for query performance.
• Data Quality & Testing: Implementing data quality checks, validation rules, and automated testing within pipelines. Monitoring data quality metrics and documenting data quality rules.
• Data Observability & Operations: Implementing logging, monitoring, and alerting for pipeline health and performance. Supporting incident response and troubleshooting for pipeline failures.
• Analytics & AI Enablement: Building data pipelines that enable analytics and reporting use cases, and preparing data for machine learning and AI workloads.
• Collaboration & Enablement: Partnering with data architects and collaborating with cross-functional teams to understand data requirements and contribute to technical documentation.

Required Qualifications:
Candidates for the Associate Data Engineer position are expected to possess a Master's degree with at least 5 years of relevant experience, or a Bachelor's degree with a minimum of 7 years of experience in data platform operations, cloud operations, or related fields. Additionally, candidates should have 5-10 years of experience in data engineering, including expertise in designing and optimizing scalable data pipelines and data processing solutions. Relevant certifications such as SAFe or other Agile certifications, as well as Databricks Certified Data Engineer Associate, are highly desirable. Candidates should demonstrate strong knowledge of data modeling, data structures, algorithms, and data integration techniques, along with proven experience applying DevOps principles and practices.

Educational Background:
The educational background required for this position includes a Master's degree in a relevant field such as Computer Science, Information Technology, Data Engineering, or a related discipline. Alternatively, a Bachelor's degree in a similar field combined with significant relevant experience may also be considered. The emphasis is on candidates who have a strong foundation in data engineering principles and practices, as well as the ability to apply this knowledge in a practical setting.

Experience:
The position requires candidates to have a substantial level of experience, typically ranging from 5 to 10 years in data engineering or related fields. This experience should encompass a variety of roles that involve data platform operations, cloud operations, and the development of data processing solutions. Candidates should have hands-on experience with modern data architectures, data integration techniques, and the implementation of data pipelines that support analytics and AI use cases. Experience working within Agile environments and familiarity with the Scaled Agile Framework (SAFe) is also beneficial.

Languages:
The mandatory language requirement for this position is English, as it is the primary language of communication within the World Bank Group. Proficiency in additional languages may be considered a plus, particularly if they align with the organization's global operations and partnerships.

Additional Notes:
This position is a local recruitment opportunity with a term duration of 3 years. The World Bank Group offers a comprehensive benefits package, including a retirement plan, medical, life and disability insurance, and paid leave, including parental leave. The organization is committed to diversity and inclusion, ensuring that all individuals are treated fairly and equitably, regardless of gender, religion, race, ethnicity, sexual orientation, or disability. The role is full-time and is based in Sofia, Bulgaria.