Data Lakehouse Consultant

Job Posting Organization:
The International Atomic Energy Agency (IAEA) is an international organization that works to promote the peaceful use of nuclear energy and to inhibit its use for any military purpose, including nuclear weapons. Established in 1957, the IAEA has a mission to ensure that nuclear energy is used safely and securely. The organization operates in over 180 countries and employs a diverse workforce of professionals dedicated to nuclear safety and security" style="border-bottom: 1px dotted #007bff !important;">security. The Department of Safeguards, which is part of the IAEA, plays a crucial role in monitoring nuclear activities worldwide to prevent the proliferation of nuclear weapons.

Job Overview:
The position of Data Lakehouse Consultant is situated within the Development Section of the Office for Information and Communication Systems (SGIS) at the IAEA. The primary function of this role is to contribute to the establishment of a new platform known as the Data Lakehouse, which is essential for the Department of Safeguards. The consultant will provide expert guidance on new technologies and platforms, lead the implementation of tasks, support project delivery, and promote best practices related to the Data Lakehouse. This role requires collaboration with various stakeholders, including business users, data scientists, and development teams, to ensure that the data services and products meet the needs of the organization. The consultant will also be responsible for designing and implementing data products, ensuring data quality and integrity, and optimizing data flows and performance.

Duties and Responsibilities:
The Data Lakehouse Consultant will have several key responsibilities, including:

• Designing and implementing data products using the Data Lakehouse architecture.
• Advising and validating architectural components in collaboration with the IT Solution Architect.
• Building and maintaining data pipelines and architectures that integrate large volumes of data from various sources into the Data Lakehouse.
• Ensuring data quality, integrity, and consistency throughout the project.
• Collaborating with business users, data scientists, analysts, and development teams to provide relevant data services and products.
• Optimizing data flows, database queries, and performance tuning to enhance efficiency and effectiveness.

Required Qualifications:
Candidates must possess a university degree in Computer Science, Data Science, or a related field. Additionally, they should have at least 5 years of relevant professional experience working with complex data integration platforms that support data analysis and services. In-depth experience in data engineering and familiarity with tools such as Spark, Flink, Trino, Airflow, Iceberg, Jupyter, dbt core, meltano, Kafka, Debezium, S3, and elastic is essential. Proficiency in programming languages such as Python and SQL is also required to effectively perform the duties of this role.

Educational Background:
The educational background required for this position includes a university degree in Computer Science, Data Science, or a similar field. This educational foundation is critical for understanding the complexities of data engineering and integration, as well as for effectively utilizing the various tools and technologies associated with the Data Lakehouse.

Experience:
The position requires a minimum of 5 years of relevant professional experience. This experience should involve working with complex data integration platforms that support data analysis and services. Candidates should have a strong background in data engineering, which includes hands-on experience with various data processing tools and technologies.

Languages:
While the job description does not specify mandatory languages, proficiency in English is likely essential given the international nature of the IAEA. Additional languages may be beneficial but are not explicitly required for this position.

Additional Notes:
The consultancy offers a daily fee of up to a maximum of €350, which is determined based on the candidate's qualifications and experience. If travel is required as part of the assignment, a daily subsistence allowance (DSA) and travel costs will be covered. It is important to note that health coverage and pension fund responsibilities fall on the incumbent, indicating that this is a consultancy role rather than a permanent position.