[VNG]Senior Data Engineer

Mã vị trí: 21-PCD-0829
Nơi làm việc: Tp.Hồ Chí Minh
Lương: Thỏa thuận

Mô tả công việc

As a Big Data Platform in ZaloPay, we are developing a high quality data warehouse that can address business problems, provide data insight to each business aspect, transform product to Data-driven decision making manner. We are looking for an Engineer who will join our team to be responsible for building out the data pipelines (data ingestion framework and ETL processing) that are scalable and reusable. This role will work with other cross functional teams to help build and support the next generation of data platform.

Responsibilities:

  • Possess an understanding of the data structures and governance.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
  • Design and maintain optimal data pipeline architecture.
  • Assemble large, complex data sets to build data mart that meet functional/non-functional business requirements.
  • Communicate effectively with Developers and Business users to understand and clarify requirements.
  • Support internal training and proper documentation ensuring the successful onboarding of new team members
  • Implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Develop efficient software code for multiple use cases leveraging Spark and Big Data Technologies for various use cases built on the platform

Yêu cầu

  • Bachelor’s degree in computer science, engineering, mathematics, or a related technical discipline.
  • 3+ years experience working as a Data Engineer role, ETL with large amounts of data.
  • Experience with data warehousing architecture and data modeling, ETL data pipeline.
  • Experience with using the following software/tool in big data field.
  • Big data tool: Hadoop, Apache Spark, Presto, Kafka, etc.
  • NoSQL and OLAP database: MongoDB, Clickhouse, Elastic search, etc.
  • Data pipeline and workflow management tools: Luigi, Airflow, etc.
  • Good at programming languages: Scala, Python, Java.
  • Data governance: Apache Atlas.
  • Experience with SRE, Patching & Automation: Kubernetes or Docker & Containerization
  • Experience in data testing process.
  • Experience in performing root cause analysis, Identify problems and propose recommendations for improvement.
  • Strong organizational and multitasking skills with ability to balance competing priorities
  • Fundamental knowledge of modern cloud computing platforms and concepts is a plus.