Job Overview:
We are seeking a highly skilled Data Engineer to oversee the operations, monitoring, and maintenance of our Cloudera Data Platform. This role requires hands-on expertise in managing big data infrastructure, ensuring high availability, performance, and security of the platform. The ideal candidate will work closely with technical teams to support data ingestion, processing workflows, automation, and troubleshooting.
Key Responsibilities:
Big Data Platform Operations & Maintenance
- Oversee day-to-day operations of the big data platform, ensuring high availability, reliability, and performance.
- Proactively monitor big data platform services, clusters, and components to identify and resolve potential issues.
- Manage configurations, upgrades, and patching of the big data platform to keep all services up to date.
- Implement security best practices, continuously monitoring for vulnerabilities and applying patches as needed.
- Work closely with technical teams to facilitate the deployment and adoption of new solutions for data ingestion, processing, and workflows.
- Maintain detailed documentation of platform configurations, troubleshooting steps, and incident resolutions.
Monitoring & Troubleshooting
- Utilize monitoring tools (e.g., Cloudera Manager, Zabbix, Grafana, Splunk, SyslogNG) to track system health and performance.
- Troubleshoot complex infrastructure and application issues involving system resources, middleware, and application stack traces.
- Ensure the smooth operation and service levels of IT solutions and provide support for production issues.
- Develop and implement automation scripts (e.g., Bash, Python, Shell) to streamline administrative tasks and maintain operational consistency.
Security & Compliance
- Implement and enforce security best practices across the big data environment.
- Work with security teams to ensure compliance with data governance and industry regulations.
- Address security vulnerabilities by deploying updates and patches in a timely manner.
Job Requirements:
Hands-on experience managing and troubleshooting Cloudera Data Platform components such as HDFS, YARN, HIVE, Spark, Impala, Ranger.
Strong knowledge of operating systems, security, and networking related to big data environments.
Experience with monitoring tools like Cloudera Manager, Zabbix, Grafana, Splunk, and SyslogNG.
Familiarity with middleware applications (e.g., Informatica, Denodo).
Proficiency in scripting languages (e.g., Bash, Python, Shell) for automation.
Experience with cloud technologies (AWS, Azure) is a plus.
Ability to troubleshoot complex technical issues across infrastructure and application layers.
Proven track record of implementing high-availability, high-performance, and high-security systems in data centers or hybrid cloud environments.
Cloudera Certified Administrator or similar certification is a plus.
Job Overview:
We are seeking a highly skilled Data Engineer to oversee the operations, monitoring, and maintenance of our Cloudera Data Platform. This role requires hands-on expertise in managing big data infrastructure, ensuring high availability, performance, and security of the platform. The ideal candidate will work closely with technical teams to support data ingestion, processing workflows, automation, and troubleshooting.
Key Responsibilities:
Big Data Platform Operations & Maintenance
- Oversee day-to-day operations of the big data platform, ensuring high availability, reliability, and performance.
- Proactively monitor big data platform services, clusters, and components to identify and resolve potential issues.
- Manage configurations, upgrades, and patching of the big data platform to keep all services up to date.
- Implement security best practices, continuously monitoring for vulnerabilities and applying patches as needed.
- Work closely with technical teams to facilitate the deployment and adoption of new solutions for data ingestion, processing, and workflows.
- Maintain detailed documentation of platform configurations, troubleshooting steps, and incident resolutions.
Monitoring & Troubleshooting
- Utilize monitoring tools (e.g., Cloudera Manager, Zabbix, Grafana, Splunk, SyslogNG) to track system health and performance.
- Troubleshoot complex infrastructure and application issues involving system resources, middleware, and application stack traces.
- Ensure the smooth operation and service levels of IT solutions and provide support for production issues.
- Develop and implement automation scripts (e.g., Bash, Python, Shell) to streamline administrative tasks and maintain operational consistency.
Security & Compliance
- Implement and enforce security best practices across the big data environment.
- Work with security teams to ensure compliance with data governance and industry regulations.
- Address security vulnerabilities by deploying updates and patches in a timely manner.
Job Requirements:
✅ Hands-on experience managing and troubleshooting Cloudera Data Platform components such as HDFS, YARN, HIVE, Spark, Impala, Ranger.
✅ Strong knowledge of operating systems, security, and networking related to big data environments.
✅ Experience with monitoring tools like Cloudera Manager, Zabbix, Grafana, Splunk, and SyslogNG.
✅ Familiarity with middleware applications (e.g., Informatica, Denodo).
✅ Proficiency in scripting languages (e.g., Bash, Python, Shell) for automation.
✅ Experience with cloud technologies (AWS, Azure) is a plus.
✅ Ability to troubleshoot complex technical issues across infrastructure and application layers.
✅ Proven track record of implementing high-availability, high-performance, and high-security systems in data centers or hybrid cloud environments.
✅ Cloudera Certified Administrator or similar certification is a plus.