Epicareer Might not Working Properly
Learn More

Big Data Engineer

$ 7,500 - $ 8,500 / month

Checking job availability...

Original
Simplified

Job Description

Big Data Engineer

Job Requirement:

  • 5 years of hands on work experience in Hadoop, Scala, Pyspark and Python.
  • Work with Hadoop Distributed File System (HDFS) to manage and store large datasets
  • Utilize tools like Hive, HBase, and Pig to process data in Hadoop
  • Set up and manage data pipelines to extract, transform, and load (ETL) large datasets
  • Develop and optimize PySpark jobs and Scala-based Spark applications to process and analyze large-scale data
  • Integrate data from various sources (relational databases, flat files, APIs, etc.) using Python and Spark.
  • Build data models and perform advanced analytics using PySpark, Python libraries and Scala.

Job Description

Big Data Engineer

Job Requirement:

  • 5 years of hands on work experience in Hadoop, Scala, Pyspark and Python.
  • Work with Hadoop Distributed File System (HDFS) to manage and store large datasets
  • Utilize tools like Hive, HBase, and Pig to process data in Hadoop
  • Set up and manage data pipelines to extract, transform, and load (ETL) large datasets
  • Develop and optimize PySpark jobs and Scala-based Spark applications to process and analyze large-scale data
  • Integrate data from various sources (relational databases, flat files, APIs, etc.) using Python and Spark.
  • Build data models and perform advanced analytics using PySpark, Python libraries and Scala.