Epicareer Might not Working Properly
Learn More

Senior Data Engineer

Salary undisclosed

Checking job availability...

Original
Simplified

YOUR ROLE:

The Senior Data Engineer will work on acquiring, storing, governing and processing huge sets of structured / unstructured data. You will bring a vision of the big data solutions landscape for choosing optimal architecture components to use for these purposes as well as a set of data engineering skills for implementing enterprise data foundations such as data lake. Finally, you will work closely with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact to them.

YOUR RESPONSIBILITIES:

  • Design, implement, and maintain robust and scalable data pipelines to ingest, transform, and process structured and unstructured data from various sources.
  • Build and manage data warehouses and data lakes, implementing efficient data storage and retrieval mechanisms. Design data models that support business requirements and analytics use cases.
  • Leverage cloud platforms (e.g., AWS, Azure, GCP) to design and deploy scalable data infrastructure, optimizing for performance, cost-effectiveness, and reliability.
  • Implement data quality checks, validation processes, and data governance frameworks to ensure data accuracy, integrity, and compliance with security standards.
  • Continuously monitor and optimize data pipelines and infrastructure to improve processing speed, reduce latency, and enhance overall system performance.
  • Work collaboratively with data scientists, analysts, and other stakeholders to understand their data needs, provide technical expertise, and deliver solutions that meet business objectives.
  • Support business users by implementing data presentation layer including data visualisation using tools like Tableau, PowerBI etc.
  • Stay up-to-date with emerging technologies and industry trends in the big data and data engineering space, evaluating and implementing innovative solutions to improve data processing capabilities.
  • Support the definition and optimization of underlying data infrastructure

WHO YOU ARE:

  • You hold a Bachelor's, Master's, or Ph.D. degree in IT, Information Management, and/or Computer Science with at least 6 years of experience.
  • Good knowledge of the big data technology landscape and concepts related to distributed storage/computing
  • Experience with big data frameworks (e.g. Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR)
  • Experience with batch & ETL jobs to ingest and process data from multiple data sources
  • Experience with NoSQL databases (e.g. Cassandra, MongoDB, Neo4J, ElasticSearch)
  • Experience with querying tools (e.g Hive, Spark SQL, Impala)
  • Experience with PowerBI
  • Experience or willingness to go in real-time stream processing, using solutions such as Kafka, AWS Kinesis, Flume, and/or Spark Streaming
  • Experience or willingness to learn about DevOps and DataOps principles (e.g Infrastructure as Code, automating different parts of the data pipeline)
  • High-level understanding of Data Science concepts and methodologies (how models are built, trained, and deployed)
  • You are passionate about technology and continuous learning comes naturally to you

MORE ABOUT PALO IT:

We’re eager to adapt to change, learn from our experiences and move to meet our planet’s urgent needs. We are continuously taking action to:

  • Become a climate net-zero company
  • Attain 50% of revenue from projects with a positive impact
  • Train 100% of our workforce on impact
  • Achieve B Corp certification among all our offices across the globe
  • Continuously measure & improve employee happiness

Our clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.

What We Offer

  • Stimulating working environments
  • Unique career path
  • International mobility
  • Internal R&D projects
  • Knowledge sharing
  • Personalized training

For more on our team culture and benefits, check out our careers page.

PALO IT Singapore is an equal opportunity employer. Employment decisions will be based on merit, qualifications and abilities. Palo IT SG does not discriminate in employment opportunities or practices on the basis of race, colour, religion, sex, sexuality, national origin, age, disability, marital status or any other characteristics protected by law.

Protecting your privacy and the security of your data are longstanding top priorities for Palo-IT.

Your personal data will be processed for the purposes of managing Palo-IT’s recruitment related activities, which include setting up and conducting interviews and tests for applicants, evaluating and assessing the results, and as is otherwise needed in the recruitment and hiring processes.

Please consult our Privacy Notice to know more about how we collect, use, and transfer the personal data of our candidates. Here you can find how you can request for access, correction and/or withdrawal of your Personal Data.

Job Type: Full-time

Pay: $7,000.00 - $12,000.00 per month

Benefits:

  • Dental insurance
  • Health insurance
  • Professional development
  • Work from home

Schedule:

  • Monday to Friday

Work Location: Hybrid remote in Singapore 189675

YOUR ROLE:

The Senior Data Engineer will work on acquiring, storing, governing and processing huge sets of structured / unstructured data. You will bring a vision of the big data solutions landscape for choosing optimal architecture components to use for these purposes as well as a set of data engineering skills for implementing enterprise data foundations such as data lake. Finally, you will work closely with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact to them.

YOUR RESPONSIBILITIES:

  • Design, implement, and maintain robust and scalable data pipelines to ingest, transform, and process structured and unstructured data from various sources.
  • Build and manage data warehouses and data lakes, implementing efficient data storage and retrieval mechanisms. Design data models that support business requirements and analytics use cases.
  • Leverage cloud platforms (e.g., AWS, Azure, GCP) to design and deploy scalable data infrastructure, optimizing for performance, cost-effectiveness, and reliability.
  • Implement data quality checks, validation processes, and data governance frameworks to ensure data accuracy, integrity, and compliance with security standards.
  • Continuously monitor and optimize data pipelines and infrastructure to improve processing speed, reduce latency, and enhance overall system performance.
  • Work collaboratively with data scientists, analysts, and other stakeholders to understand their data needs, provide technical expertise, and deliver solutions that meet business objectives.
  • Support business users by implementing data presentation layer including data visualisation using tools like Tableau, PowerBI etc.
  • Stay up-to-date with emerging technologies and industry trends in the big data and data engineering space, evaluating and implementing innovative solutions to improve data processing capabilities.
  • Support the definition and optimization of underlying data infrastructure

WHO YOU ARE:

  • You hold a Bachelor's, Master's, or Ph.D. degree in IT, Information Management, and/or Computer Science with at least 6 years of experience.
  • Good knowledge of the big data technology landscape and concepts related to distributed storage/computing
  • Experience with big data frameworks (e.g. Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR)
  • Experience with batch & ETL jobs to ingest and process data from multiple data sources
  • Experience with NoSQL databases (e.g. Cassandra, MongoDB, Neo4J, ElasticSearch)
  • Experience with querying tools (e.g Hive, Spark SQL, Impala)
  • Experience with PowerBI
  • Experience or willingness to go in real-time stream processing, using solutions such as Kafka, AWS Kinesis, Flume, and/or Spark Streaming
  • Experience or willingness to learn about DevOps and DataOps principles (e.g Infrastructure as Code, automating different parts of the data pipeline)
  • High-level understanding of Data Science concepts and methodologies (how models are built, trained, and deployed)
  • You are passionate about technology and continuous learning comes naturally to you

MORE ABOUT PALO IT:

We’re eager to adapt to change, learn from our experiences and move to meet our planet’s urgent needs. We are continuously taking action to:

  • Become a climate net-zero company
  • Attain 50% of revenue from projects with a positive impact
  • Train 100% of our workforce on impact
  • Achieve B Corp certification among all our offices across the globe
  • Continuously measure & improve employee happiness

Our clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.

What We Offer

  • Stimulating working environments
  • Unique career path
  • International mobility
  • Internal R&D projects
  • Knowledge sharing
  • Personalized training

For more on our team culture and benefits, check out our careers page.

PALO IT Singapore is an equal opportunity employer. Employment decisions will be based on merit, qualifications and abilities. Palo IT SG does not discriminate in employment opportunities or practices on the basis of race, colour, religion, sex, sexuality, national origin, age, disability, marital status or any other characteristics protected by law.

Protecting your privacy and the security of your data are longstanding top priorities for Palo-IT.

Your personal data will be processed for the purposes of managing Palo-IT’s recruitment related activities, which include setting up and conducting interviews and tests for applicants, evaluating and assessing the results, and as is otherwise needed in the recruitment and hiring processes.

Please consult our Privacy Notice to know more about how we collect, use, and transfer the personal data of our candidates. Here you can find how you can request for access, correction and/or withdrawal of your Personal Data.

Job Type: Full-time

Pay: $7,000.00 - $12,000.00 per month

Benefits:

  • Dental insurance
  • Health insurance
  • Professional development
  • Work from home

Schedule:

  • Monday to Friday

Work Location: Hybrid remote in Singapore 189675