Dewei Zhai

Senior Data Engineer & Applied AI Engineer

Skills

Technology experience, project responsibilities and related certifications.

Data platform architecture

PVH: 500+TB AWS platform. ENYQUANT: cloud and electricity-market data platform.

Project responsibilities

  • PVH (2020–2022, 2023–2025): platform architecture responsibilities in a core team of about ten; Adobe, Salesforce and SAP integrations, config-driven ETL, data-quality checks and cross-timezone scheduling.
  • ENYQUANT (2026): responsibility for cloud and data-platform architecture, code, deployment and operations using Airflow, DuckDB, Polars, DuckLake and object storage.
  • Azure
  • Alibaba Cloud
  • Airflow
  • DuckDB
  • Polars
  • DuckLake

Idempotent ETL & reliability

PVH: config-driven, idempotent ETL, data quality, observability and scheduling.

Project responsibilities

  • PVH (2020–2022): developed config-driven, idempotent production ETL with consistent target data when processing the same input again.
  • Responsibilities included pipeline data-quality checks, operational observability and cross-timezone scheduling.
  • Python
  • SQL
  • Spark
  • Airflow
  • AWS
  • Idempotent ETL

SQL & data pipelines

PVH: SQL, YAML contracts, validation and an Airflow DAG factory.

Project responsibilities

  • PVH (2023–2025): integrated analyst SQL into production delivery and developed YAML contracts, static validation, data-quality checks and a DAG factory.
  • Compared old and new outputs before adoption; the platform supported 60+ dashboards.
  • SQL
  • YAML
  • Airflow
  • Data contracts

Spark & performance

Spark / PySpark and Python since 2016; projects on HDP, EMR and GKE.

Experience dates

Python · Spark / PySpark: since 2016

Project responsibilities

  • KPN: Hive/Spark ETL development and Hortonworks HDP cluster installation and operations.
  • ABN AMRO: Python/Spark/Hive pipelines and Hive-to-PySpark workload migration.
  • FedEx: Spark/EMR ETL development and operations; deployed, migrated and ran Spark on an existing GKE cluster with Kubernetes YAML and kubectl.
  • PVH: Spark production ETL development. VodafoneZiggo: evolution of the existing Scala/Spark ETL framework.

Related certifications

  • Databricks Certified Associate Developer for Apache Spark 2.4 · 2020-11
  • PCAP – Certified Associate in Python Programming · 2021-09
  • Spark
  • PySpark
  • EMR
  • GKE

Cloud & infrastructure

AWS, Azure, GCP and Alibaba Cloud: experience dates, platform responsibilities and certifications.

AWS

Since 2018

FedEx · PVH · VodafoneZiggo

  • FedEx: AWS infrastructure management with Terraform and Spark/EMR ETL development.
  • PVH: architecture and development on a 500+TB data platform, idempotent ETL and event-driven serverless workflows.
  • VodafoneZiggo: built and operated DMS full-load and CDC tasks. Terraform use started in 2018.

Certification (issued → expiry) · AWS Certified Solutions Architect — Associate · 2020-10 → 2023-10

Azure

2024 · 2026

PVH (2024) · ENYQUANT (2026)

  • PVH: Azure Databricks solution reviews and data-platform migration support.
  • ENYQUANT: owned platform design and implementation from zero, covering Azure Databricks, Microsoft Foundry (formerly Azure AI Foundry), Azure Data Factory, ADLS, relational databases and Azure Functions.
  • Responsible for networking, Key Vault, Microsoft Entra ID and permissions, Azure Monitor observability, Terraform infrastructure as code and CI/CD.

Certification (issued → expiry) · Databricks Certified Data Engineer Associate · 2024-12 → 2026-12

GCP

2019–2020 · 2024

FedEx (2019–2020) · PVH (2024)

  • FedEx: managed GCP infrastructure with Terraform and developed BigQuery workloads; deployed, migrated and ran Spark on an existing GKE cluster with Kubernetes YAML and kubectl.
  • PVH: used BigQuery and Cloud Storage in the data-platform project.

Alibaba Cloud

Since 2026

ENYQUANT (2026)

  • Designed, built, deployed and operated the Alibaba Cloud platform from zero, with full implementation and operational responsibility.
  • Scope covered storage, RDS relational databases, VMs, networking and serverless functions.

Quality & delivery automation

VodafoneZiggo: PowerCenter conversion and GitLab CI/CD checks for 40+ developers.

Project responsibilities

  • VodafoneZiggo (2022–2024): integrated PowerCenter workflow conversion, preflight validation and migration checks into GitLab CI/CD for at least 40 developers.
  • KPN (2016–2018): developed system-health checks with Robot Framework and automated infrastructure and delivery tasks with Python, Ansible and Jenkins.
  • GitLab CI/CD
  • PowerCenter
  • ETL validation

CDC & platform migration

VodafoneZiggo: AWS DMS Full Load + CDC; hundreds of TB from Oracle to Snowflake.

Project responsibilities

  • VodafoneZiggo (2022–2024): built and operated AWS DMS full-load and CDC tasks as a cloud administrator and data engineer, supporting a hundreds-of-terabytes Oracle-to-Snowflake migration.
  • Contributed to the existing Scala/Spark ETL framework and migration technical support.
  • AWS DMS
  • CDC
  • Oracle
  • Snowflake
  • Scala
  • Spark

MLOps enablement

JupyterHub, MLflow, production Spark jobs and training-data time semantics.

Project responsibilities

  • VodafoneZiggo: built JupyterHub + MLflow services for 4–5 data scientists.
  • FedEx: operated JupyterHub and productionised and tuned Spark analytical code written by data scientists.
  • ENYQUANT: defined point-in-time visibility for historical training data and implemented snapshot-bound readback and data checks.
  • JupyterHub
  • MLflow
  • Python
  • Spark
  • Point-in-time data

Agent architecture & governance

ENYQUANT: team contracts, shared capability interfaces, permissions and resource coordination.

Experience dates

ENYQUANT · 2026

Project responsibilities

  • Designed team contracts, shared capability interfaces and discovery, adoption, contribution and release workflows for the company AI collaboration repository.
  • Maintained agent permissions, shared-resource coordination and delivery-evidence rules; consolidated duplicate instructions and recorded capability ownership.
  • AI Agents
  • Skills
  • Plugins
  • Context Design
  • Resource Coordination

Agent validation & tooling

ENYQUANT: Python validation CLI, Docker distribution, contract checks and structured reports.

Experience dates

ENYQUANT · 2026

Project responsibilities

  • Developed a Docker-distributed Python validation CLI connecting development contracts, test selection, source-sampling plans and structured reports.
  • Checked code versions, contracts and sources, with rejection paths for incorrect intervals, units, members and missing evidence.
  • Designed dual-architecture image checks and release gates; design and execution status are recorded separately.
  • Python CLI
  • Docker
  • Contract Validation
  • Structured Reports
  • CI/CD

Agent workflows & shared delivery

ENYQUANT: three shared Skills for data development and source understanding.

Experience dates

ENYQUANT · 2026

Project responsibilities

  • Packaged data development and source understanding into three shared Skills with merged source; defined inputs, tool versions, permissions, acceptance evidence and stop conditions.
  • Updated environment discovery, UTF-8 reading and blocked-execution handoffs using feedback from another member’s Windows/WSL agent trials.
  • AI Agents
  • Skills
  • Plugins
  • Workflow Engineering
  • Behavioral Validation