Epicareer Might not Working Properly
Learn More

Data Engineer

Salary undisclosed

Checking job availability...

Original
Simplified

Job Description:

Data Platform Construction

  • Design, develop, and optimize the company-wide data platform to support large-scale data collection, storage, computation, and analysis needs.
  • Lead technical selection, secondary development, and performance optimization of data computation, storage, and scheduling frameworks to ensure stable platform operation.
  • Continuously drive the evolution of the data platform architecture to meet the fast-changing business requirements.

Data Development Tools Construction

  • Design and develop efficient and user-friendly data development tools to enhance the overall development efficiency of the data team.
  • Support data integration and business requirements implementation, and build automated tool-based solutions tailored to business scenarios.
  • Build a data development and operations toolchain to improve platform development and management experience.

Data Security and Compliance

  • Design, implement, and optimize the data security system to ensure data confidentiality, integrity, and availability.
  • Implement security mechanisms such as data encryption, access control, and permission management to ensure platform security.
  • Promote data compliance implementation to ensure the platform meets relevant regulatory requirements (e.g., SOC2, GDPR).
  • Design and implement data auditing, tracking, and monitoring mechanisms to ensure transparent and compliant data usage.

Data Lifecycle Management

  • Lead the management of the entire data lifecycle, including data source access, extraction, cleaning, modeling, metadata management, and optimization.
  • Develop data quality assurance mechanisms, provide technical support, and ensure data reliability, accuracy, and consistency.
  • Drive data governance implementation and build an efficient data management system.

Job Requirements:

Education and Experience

  • Bachelor's degree or above in Computer Science, Software Engineering, Data Science, or related fields, with 5+ years of experience in data platform development, including architectural design and implementation capabilities.

Technical Skills

  • Strong computer science foundation, familiar with operating system principles, networks, and distributed systems.
  • Proficient in Java and Python, and skilled in common data structures and algorithms.
  • In-depth understanding of mainstream big data technology stacks, including but not limited to: Hadoop, Flink, Spark, Hive, Kafka, Paimon, etc.
  • Strong technical sensitivity and optimization experience in distributed computing, storage, and scheduling frameworks.

Data Security and Compliance Expertise

  • Familiar with data security technologies, including but not limited to: data encryption, access control, permission management, data masking, etc.
  • Practical experience in data compliance, familiar with relevant laws and regulations (e.g., GDPR, SOC2), and able to implement compliance requirements.
  • Experience designing and implementing data security auditing, traceability, and monitoring systems.

Platform and Tool Development Experience

  • Experience in designing and developing large-scale data platforms or tools, familiar with technical selection, architectural design, and performance optimization.
  • Able to quickly understand complex business requirements and translate them into efficient technical solutions.
  • Capable of building systems from scratch or optimizing platforms from one to a hundred.

Preferred:

  • Strong communication and teamwork skills, capable of driving cross-departmental projects.
  • Highly self-driven, adept at solving complex problems, with a results-oriented mindset.
  • Continuous passion for technology, with a willingness to explore cutting-edge technologies and apply them in practical scenarios.