Epicareer Might not Working Properly
Learn More

Data Engineer

Salary undisclosed

Apply on


Original
Simplified


Responsibilities:

  • Design and build data pipelines: Create the infrastructure to collect, process, and store data from various sources.
  • Develop data assets platform: This platform provides a centralized repository for managing and accessing data assets.
  • Implement automated SQL generation: Create tools and processes to automatically generate SQL queries based on user-defined criteria or business requirements. This can involve using techniques like query templates, metadata-driven generation, or machine learning algorithms.
  • Create new IPs in Singapore in area of expertise
  • Manage collaboration projects with Institute of Higher Learning or Research Institutes on new technology development
  • Overseas process development support for short term assignments


Key Qualifications/Requirements:

  • Bachelors (w/ min. 3 yrs of relevant working exp.)/ Masters (w/ min. 2 yrs of relevant working exp./ PhD (w/ min. 1 yr of relevant working exp.
    • Computer Science,
    • Data Science,
    • Statistics Applied Sciences, or related field.


Work experience/Skillset:

  • Programming skills : Proficiency in programming like Python, SQL, Java, or Scala is essential for data manipulation and automation.
  • Data Warehousing and ETL: Understanding of data warehousing concepts, ETL (Extract, Transform, Load) processes, and tools like Informatica, Talend, or Airflow.
  • Database Management: Knowledge of relational databases (MySQL, PostgreSQL, Oracle) and NoSQL databases (MongoDB, Cassandra) is crucial.
  • Cloud Platforms: Experience with deploying and managing data infrastructure.
  • Data Modelling: Ability to design and implement data models that effectively represent business requirements.
  • Data Quality: Understanding of data quality concepts, validation techniques, and tools.
  • Automation Tools: Familiarity with tools like Apache Airflow, Kubernetes, or Ansible for automating data pipelines and infrastructure.
  • Machine Learning: Basic understanding of machine learning algorithms can be beneficial for certain automation tasks, such as query optimization or anomaly detection.