Epicareer Might not Working Properly
Learn More
H

Quantexa Cloud Data Engineer

$ 4,000 - $ 5,000 / month

Checking job availability...

Original
Simplified

Job Summary

We are looking for a Data Engineer with strong expertise in data pipeline development, database management, and AI/ML integration. The ideal candidate has hands-on experience in big data systems, cloud infrastructure, and data-driven automation. This role involves building scalable data solutions, optimizing ETL processes, and working closely with analysts and data scientists to develop data-intensive applications.

Key Responsibilities

  • Design, build, and maintain scalable ETL/ELT pipelines to process large volumes of structured and unstructured data.
  • Develop automated data ingestion and transformation workflows using Python, SQL, and cloud services (AWS, GCP, Alibaba Cloud).
  • Integrate real-time and batch data sources from APIs, databases, and third-party platforms.
  • Design and optimize relational and NoSQL databases (PostgreSQL, MySQL, MongoDB).
  • Manage cloud-based data infrastructure using AWS, Alibaba Cloud, or Google Cloud.
  • Implement CI/CD pipelines for data applications using GitHub Actions, Docker, and Kubernetes.
  • Work with big data technologies (Hadoop, Spark) for large-scale data processing.
  • Build data models, warehouses, and lakes to support business intelligence and AI/ML projects.
  • Ensure data integrity, quality, and governance across multiple platforms.
  • Collaborate with data scientists to deploy ML models into production environments.
  • Optimize model performance and inference pipelines using Python frameworks (TensorFlow, PyTorch, Scikit-learn).
  • Develop tools for feature engineering, data labeling, and model monitoring.

Qualifications

  • Education: Bachelor's degree in Data Science, Computer Science, or a related field.
  • Technical Skills:
    • Strong experience with Quantexa, Python, SQL, Java, and shell scripting.
    • Knowledge of big data tools (Hadoop, Spark, Kafka).
    • Hands-on experience with cloud platforms (AWS, Alibaba Cloud, GCP).
    • Proficiency in database management (PostgreSQL, MySQL, MongoDB).
    • Experience with APIs, ETL pipelines, and data streaming.
  • Preferred Experience:
    • Exposure to risk analytics, financial data modeling, or trading systems.
    • Experience with data visualization tools (Tableau, Power BI, Streamlit).

Understanding of DevOps practices, CI/CD, and containerization