AWS ETL Cloud Data Engineer
Job Description
We are looking to hire a AWS ETL Data Engineer. This is an exciting opportunity to use and further expand your AWS cloud skill set, and join Data Team to build Hybrid Datalake as part of a key IFRS project..
Responsibilities:
Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties Spark/Python on Glue, Redshift, S3, Athena, RDS-PostgreSQL, Airflow, Lambda, DMS, Code Commit, Code Pipeline, Code build, etc.
Design and build production ETL data pipelines from ingestion to consumption within a big data architecture, using DMS, DataSync & Glue.
Understand the existing applications(including on-premise Cloudera Datalake) and infrastructure architecture.
Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
Collaborate with development, infrastructure and data center teams to define Continuous Integration and Continuous Delivery processes in accordance with industry standards.
Work on hybrid Datalake.
Work closely with multiple stakeholders to ensure high standards are maintained.
Key Requirements:
Bachelor's Degree in Computer Science, Information Technology or other relevant fields
5+ years of work experience with ETL, Data Modelling, and Data Architecture to build Datalake. Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark.
3+ years of extensive experience in working on the AWS platform using core services like AWS Athena, Glue Pyspark, Redshift, RDS-PostgreSQL, S3 & Airflow(for orchestration).
Thanks,
Rajkumar
Sr. Team Lead-Recruitment | Quess Selection & Services, Singapore
EA Licence Number: 23C2060 Registration ID is R1983662
Job Description
We are looking to hire a AWS ETL Data Engineer. This is an exciting opportunity to use and further expand your AWS cloud skill set, and join Data Team to build Hybrid Datalake as part of a key IFRS project..
Responsibilities:
Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties Spark/Python on Glue, Redshift, S3, Athena, RDS-PostgreSQL, Airflow, Lambda, DMS, Code Commit, Code Pipeline, Code build, etc.
Design and build production ETL data pipelines from ingestion to consumption within a big data architecture, using DMS, DataSync & Glue.
Understand the existing applications(including on-premise Cloudera Datalake) and infrastructure architecture.
Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
Collaborate with development, infrastructure and data center teams to define Continuous Integration and Continuous Delivery processes in accordance with industry standards.
Work on hybrid Datalake.
Work closely with multiple stakeholders to ensure high standards are maintained.
Key Requirements:
Bachelor's Degree in Computer Science, Information Technology or other relevant fields
5+ years of work experience with ETL, Data Modelling, and Data Architecture to build Datalake. Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark.
3+ years of extensive experience in working on the AWS platform using core services like AWS Athena, Glue Pyspark, Redshift, RDS-PostgreSQL, S3 & Airflow(for orchestration).
Thanks,
Rajkumar
Sr. Team Lead-Recruitment | Quess Selection & Services, Singapore
EA Licence Number: 23C2060 Registration ID is R1983662