Senior Data Platform Engineer (Contract)

Sydney

6 Months (Initially)

$970pd Inc Super



About Us:

Leading Insurance business in Australia are seeking a highly skilled Senior Data Engineer to join their Sydney team on a 6-month contract basis.



Key Responsibilities:

Infrastructure Setup: Utilize Terraform or Azure Resource Manager to automate Azure infrastructure setup, ensuring efficiency and scalability.





Data Architecture: Design and implement a robust data lakehouse for diverse data storage and analytics, ensuring optimal performance and scalability.



Data Pipelines: Implement efficient data pipelines to ingest and process data into the data lakehouse, enabling timely and accurate insights generation.



Terraform Scripts: Develop Terraform scripts for application infrastructure provisioning, automating deployment tasks and enhancing efficiency.





Scalability Planning: Plan for capacity and scale up infrastructure as needed, anticipating future growth and ensuring seamless scalability of the platform.



CSIO Team Collaboration: Collaborate with the CSIO team to ensure cybersecurity and secure design principles are integrated into the GenAI platform, mitigating risks and ensuring compliance.



Requirements:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience in Azure Databricks
• Strong expertise in designing and implementing data architectures, including data lakehouses.
• Experience in developing and maintaining data pipelines for large-scale data processing.
• Hands-on experience with Azure DevOps for CI/CD implementation.
• Proficiency in scripting languages such as Python (Pyspark)
• Experience in monitoring and maintaining AI model performance (Desirable)

If you feel you possess the relevant skills to the above, apply now with most updated CV!