We are currently seeking a highly skilled Data Engineer to join our team and play a key role in designing and implementing a data platform for our pricing actuaries.
Summary:
You'll be responsible for assisting in gathering requirements, designing a scalable and efficient data platform, and leading a third-party team to build and deploy the system. Our platform will need to support an array of highly specific domains, each represented in its fullest detail, but tied together to be meaningful in the aggregate.
Additionally, you will support and maintain existing systems built in SQL Server and investigate data anomalies alongside the business with minimal handover. Your primary focus will be on designing data models and pipelines, integrating data from various sources, ensuring data quality and reliability, and providing technical leadership.
Job Duties:
- Collaborate with pricing actuaries, data analysts, and other stakeholders to understand their data needs and requirements.
- Identify and document data sources, data integration requirements, and data transformation processes.
- Architect and design a robust and scalable data platform that meets the requirements of pricing actuaries.
- Determine the appropriate technologies, tools, and frameworks for data integration, storage, and processing.
- Ensure data security, privacy, and compliance standards are met.
- Develop and maintain data pipelines to extract, transform, and load (ETL) data from various sources into the data platform.
- Integrate external data sources, APIs, and third-party systems with the data platform.
- Implement data validation and quality checks to ensure accuracy and reliability.
- Maintain and support existing systems built in SQL Server.
- Investigate data anomalies reported by the business, proactively identifying and resolving issues.
- Conduct root cause analysis and implement corrective measures.
- Collaborate with external vendors or third-party development teams to build and deploy the data system.
- Provide technical guidance, support, and oversight to the third-party team.
- Ensure timely delivery of milestones and adherence to project time lines.
Skills:
- Proven experience working as a Data Engineer or a similar role in designing and implementing data systems.
- Strong understanding of data engineering principles, data modelling, ETL processes, and data integration techniques.
- Experience with cloud platform - Azure and familiarity with relevant services (e.g., Data Factory, Azure Blob Storage).
- Proficient in programming languages such as SQL, Python, and experience with relevant data processing frameworks Apache Spark.
- Strong expertise in SQL Server, including performance optimisation, query tuning, and troubleshooting.
- Curiosity and a willingness to learn and support existing systems built in SQL Server.
- Worked within Financial Services, an Insurance background would be more desired.
