[VNG]Senior Data Engineer

Mã vị trí: 21-GDS-0856
Nơi làm việc: Tp.Hồ Chí Minh
Lương: Thỏa thuận

Mô tả công việc

As a Hadoop big data engineer, you will develop, operate and drive scalable and resilient data platform based on Hadoop ecosystem to address the business requirements:
  • Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy
  • Design and implement business-specific large-scale data processing pipelines
  • Work with complex data structures, manipulate, cleanse data, and perform transformations to make insights from data.
  • Responsible to Ingest data from files, streams, and databases. Process the data with Spark Scala, Kafka…
  • Develop efficient software code for multiple use cases leveraging Spark and Big Data Technologies for various use cases built on the platform
  • Provide high operational excellence guaranteeing high availability and platform stability.

Yêu cầu

Technical expertise:
  • 3+ years experience in professional software development (e.g. Data Engineering)
  • Hands on and proven technical experience in Big Data environments and projects.
  • Experience with integration of data from multiple data sources and of multiple formats/types.
Must requirements:
  • Experience in Hadoop ecosystem including HDFS, MapReduce, YARN…
  • Experience in building large-scale data processing (batch-processing, stream processing)
  • Experiences in (most of) these technologies:
  • AWS Cloud native development
  • AWS S3, Lambda
  • Apache Spark
  • SQL, Scala, Python, Java
  • Big Data tools, such as MongoDB / NoSQL technologies
  • Understanding of SLA and meeting Timelines for support activities
Nice to have:
  • Experience in Apache Kafka
  • Experience with Data warehouse
  • Experience in ETL
  • Experience in Data management: (Governance, Architecture, Modelling, Quality, Integration)
  • Experience in SQL and NoSQL Database
  • Experience in Backend development using Java
  • Experience in Data API