Senior/Lead Data Engineer-Canberra, Brisbane, Melbourne, Adelaide,Sydney

One of our leading federal govt clients is seeking to engage Senior /Lead Data Engineer for Canberra, Brisbane, Melbourne, Adelaide, Sydney locations

Please note Candidates must have a NV1 security clearance

Location of work- Canberra, Brisbane, Melbourne, Adelaide, Sydney

Senior Data Engineer

• Negative Vetting Level 1
• Canberra, Brisbane, Melbourne, Adelaide
• 12 Months + 24 months

Job details

Technical Skills include

• Data modelling techniques (relational, dimensional, messaging and big data), data analysis and profiling using modern analytical tools and programming languages, and visualisation methods and tools.
• Experience with one or more of the following - Teradata, Cloudera Data Platform.
• Experience in implementing solutions on Red Hat Open shift platforms is highly desirable.
• Best-practice data cleansing, validation, quality assurance and feature preparation techniques for complex datasets.
• Programming & Software Engineering – one or more of following:
• Modern engineering practices supporting data products (API integration, Data Replication, Data Acquisition, Data Reconciliation, ETL/ELT using Batch and near Real Time)
• Programming in one or more of the following - Python, SQL, Apache Spark, Java, Apache NiFi.
• Experience in utilising Source control (Git) repositories, utilising and building CI/CD pipelines
• Automated testing, Secure coding practices, Code review processes

Essential criteria

• 1.Demonstrated experience engaging stakeholders to define business requirements and design, develop, implement, maintain and support data engineering, enterprise data warehouse, data lake, streaming and software solutions that meet complex agency needs.
• 2.Demonstrated ability to perform complex data extraction, manipulation, analysis, querying, programming, testing and troubleshooting, while applying data quality, integrity, security, risk controls and endorsed engineering standards to deliver reliable business outcomes.
• 3.Demonstrated technical capability across data modelling, analysis, profiling, cleansing, validation, API integration, data replication, data acquisition, data reconciliation, ETL/ELT, batch and near real-time pipeline delivery.

Desirable criteria

• 1.Experience with one or more of the following platforms: Teradata, Cloudera Data Platform, Red Hat OpenShift
• 2.Experience in one or more of the following: Python, SQL, Apache Spark, Java, Apache NiFi, Hadoop
• 3.Experience building and supporting data products using automated testing, secure coding practices, Git repositories, DevOps pipelines, code review processes and contemporary software engineering practices.
• 4.Relevant qualification or equivalent practical experience in computer science, software engineering, data engineering, enterprise data warehousing or software development.

Lead Data Engineer

• Initial contract duration- 12 Months
• Extension term-24 Months
• Canberra, Brisbane, Melbourne, Adelaide.Sydney

Requirements

Overview: Data Engineers prepare raw data for analytical or operational uses. They make data accessible so that it can be used for optimisation, delivery of services and enhanced analytics solutions. The Data Engineer operates under limited direction to support and maintain enterprise data assets across environments. The position will work closely with business and technical teams to deliver continuous improvement of data pipelines and processes.

• Develop, maintain and optimise enterprise data pipelines, Extract, Transform and Load (ETL) processes and analytical code.
• Define and maintain data modelling, peer review, development and release standards.
• Lead technical implementation activities from requirements analysis through to deployment and warranty support.
• Develop and maintain Structured Query Language (SQL) analytical and ETL code supporting enterprise reporting and analytics.
• Optimise and tune SQL code to improve performance and efficiency.
• Produce and maintain technical documentation, data dictionaries and implementation artefacts.
• Conduct peer reviews of code, data models, and technical designs.
• Analyse and translate business requirements into technical solutions and data assets.
• Collaborate with business stakeholders, developers, testers and support teams to deliver solutions.
• Facilitate knowledge transfer and contribute to team capability uplift.

• Support backlog prioritisation, work queue and management, and delivery planning activities. Knowledge/Qualifications:

• Cyber security knowledge and experience.
• Protective Security Policy Framework (PSPF).
• Information Security Manual (ISM).
• Essential 8 Strategies.
• National Institute of Standards and Technology (NIST).
• Business intelligence BINT - Level 4.
• Data engineering DENG - Level 6.
• Data management DATM - Level 6.
• Data modelling and design DTAN - Level 5.
• Data visualisation VISL - Level 5.
• Database design DBDS - Level 5.
• Information management IRMG - Level 6.
• Measurement MEAS - Level 5.
• Quality assurance QUAS - Level 5.
• Storage management STMG - Level 5.

• Structured Query Language (SQL)
• Cloud Platforms: 1. Amazon Web Services 2. Google Cloud Platform 3. Microsoft Azure

If you would like to apply for the above role, then please send y our updated resume with Cover Let ter to Sign in to view email m.au