SENIOR DATA ENGINEER

Sobre o nosso cliente Our client is investing in a major technology transformation to harness and enrich our technology capabilities in Data Engineering, Data Science & Artificial Intelligence space, and their commitment to delivery excellence across our engineering function is critical to underpin this.
This is an exciting time to join our client's team on this journey and play a pivotal role in building great solutions for their customers and clients.
Reporting into our client's Technology division, this role will be instrumental for the end-to-end delivery of their data product roadmap.
Descrição Play a key role in defining Data Engineering design patterns for data acquisition, transformation and serving.
Provide technical guidance and assurance to engineers, ensuring consistency and alignment to agreed standards and quality.
Collaborate with team members to translate functional and non-functional requirements into technical features for the data pipeline/product/application.
Implementation of data applications, data pipelines, and involvement in code reviews, prototyping and spikes Actively be responsible for the implementation of common data models, data acquisition & transformation pipelines from design to delivery.
Drive continuous improvement of our internal data acquisition and transformation frameworks, our central function library and SDKs.
Conform to the DRY programming principle and build re-usable templates through parameterization and configurations.
Build data models that are optimal for various access patterns - both real-time & batch.
Support change management and data governance activities.
Assess the impact on data models resulting from upstream changes or downstream consumption changes.
Verify requirements and produce Data Engineering implementation designs; support the creation of delivery and QA plans.
Engage regularly with stakeholders and platform support teams to verify future capacity planning considerations.
Identify and implement internal process improvements.
Alignment with compliance and info-security principles, ensuring risk registers are maintained for all data pipelines and products.
Represent delivery changes at Change Advisory Board.
Support the architects in shaping the future of the data platform and aid in landing new capabilities.
Identify relevant emerging trends and build compelling cases for adoption.
Involved in PoCs, prototypes and innovation spikes to seek directional outcomes.
Serve as the technical expert and encourage standard methodologies within the product engineering teams.
Perfil ideal Demonstrated ability in a similar senior role.
Experience in operationalizing innovative Data Engineering solutions.
Experience using DevOps toolchains for managing CI/CD.
Experience with modern cloud data technology stacks.
Proficient in Python, Apache Spark, or any other distributed data programming frameworks.
In-depth understanding of data concepts, data structures, modelling techniques and how best to provision data to support varying consumption needs.
Accomplished ETL/ELT engineer - with commercial experience in building and architecting scalable data.
Experience with Azure Data Factory and Databricks (preferably python API).
Shown experience in putting a strong emphasis on high-quality application design - with a microservice mentality.
Exposure to working with large complex and diverse datasets/types.
Experience with orchestration implementations.
Possess an "automation-first" mindset when building solutions; considerations for self-healing and fault-tolerant methods.
Proficient in using Test or Business Driven Development (TDD / BDD) approaches.
Fluency in English.
Vantagens A good opportunity to evolve in the Data development environment.