Big Data Engineer
$ 7,500 - $ 8,500 / month
Checking job availability...
Original
Simplified
Job Description
Big Data Engineer
Job Requirement:
- 5 years of hands on work experience in Hadoop, Scala, Pyspark and Python.
- Work with Hadoop Distributed File System (HDFS) to manage and store large datasets
- Utilize tools like Hive, HBase, and Pig to process data in Hadoop
- Set up and manage data pipelines to extract, transform, and load (ETL) large datasets
- Develop and optimize PySpark jobs and Scala-based Spark applications to process and analyze large-scale data
- Integrate data from various sources (relational databases, flat files, APIs, etc.) using Python and Spark.
- Build data models and perform advanced analytics using PySpark, Python libraries and Scala.
Job Description
Big Data Engineer
Job Requirement:
- 5 years of hands on work experience in Hadoop, Scala, Pyspark and Python.
- Work with Hadoop Distributed File System (HDFS) to manage and store large datasets
- Utilize tools like Hive, HBase, and Pig to process data in Hadoop
- Set up and manage data pipelines to extract, transform, and load (ETL) large datasets
- Develop and optimize PySpark jobs and Scala-based Spark applications to process and analyze large-scale data
- Integrate data from various sources (relational databases, flat files, APIs, etc.) using Python and Spark.
- Build data models and perform advanced analytics using PySpark, Python libraries and Scala.