H
Quantexa Cloud Data Engineer
$ 4,000 - $ 5,000 / month
Checking job availability...
Original
Simplified
Job Summary
We are looking for a Data Engineer with strong expertise in data pipeline development, database management, and AI/ML integration. The ideal candidate has hands-on experience in big data systems, cloud infrastructure, and data-driven automation. This role involves building scalable data solutions, optimizing ETL processes, and working closely with analysts and data scientists to develop data-intensive applications.
Key Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines to process large volumes of structured and unstructured data.
- Develop automated data ingestion and transformation workflows using Python, SQL, and cloud services (AWS, GCP, Alibaba Cloud).
- Integrate real-time and batch data sources from APIs, databases, and third-party platforms.
- Design and optimize relational and NoSQL databases (PostgreSQL, MySQL, MongoDB).
- Manage cloud-based data infrastructure using AWS, Alibaba Cloud, or Google Cloud.
- Implement CI/CD pipelines for data applications using GitHub Actions, Docker, and Kubernetes.
- Work with big data technologies (Hadoop, Spark) for large-scale data processing.
- Build data models, warehouses, and lakes to support business intelligence and AI/ML projects.
- Ensure data integrity, quality, and governance across multiple platforms.
- Collaborate with data scientists to deploy ML models into production environments.
- Optimize model performance and inference pipelines using Python frameworks (TensorFlow, PyTorch, Scikit-learn).
- Develop tools for feature engineering, data labeling, and model monitoring.
Qualifications
- Education: Bachelor's degree in Data Science, Computer Science, or a related field.
- Technical Skills:
- Strong experience with Quantexa, Python, SQL, Java, and shell scripting.
- Knowledge of big data tools (Hadoop, Spark, Kafka).
- Hands-on experience with cloud platforms (AWS, Alibaba Cloud, GCP).
- Proficiency in database management (PostgreSQL, MySQL, MongoDB).
- Experience with APIs, ETL pipelines, and data streaming.
- Preferred Experience:
- Exposure to risk analytics, financial data modeling, or trading systems.
- Experience with data visualization tools (Tableau, Power BI, Streamlit).
Understanding of DevOps practices, CI/CD, and containerization