Skip to content
OPSWAT

Senior Data Engineer

Ho Chi Minh City, Ho Chi Minh City, Vietnam; VietnamVietnam
Senior · Enterprise Products & Intelligence
POSTED 14 MAY 2025
Apply Ask
Role
557 days opentrue age
Past the 32d median close
Same title, closed before
×1
Posted 14 May 2025 · n=11
CompanyOPSWAT →
101open roles
Opened · 30d
+5
Closed · 30d
−3
13d of snapshots · 30d window
Market
1,196open · Senior Data engineering
Opened · 7d
+1018% of open
All data engineering
3,779open
All tracked boards · live 22 Nov · n=1,196
PostgreSQLAzureTerraformAirflowdbt

OPSWAT, a global leader in IT, OT, and ICS critical infrastructure cybersecurity, delivers an end-to-end platform that gives public and private sector organizations and enterprises the critical advantage needed to protect their complex networks, secure their devices, and ensure compliance. Over the last 20 years our commitment to innovative technology has earned the trust of more than 1,700 organizations, governments, and institutions globally, solidifying our role in protecting the world’s critical infrastructure and securing our way of life.

The Position

OPSWAT's data platform ingests from more than forty business systems such as CRM, finance, HR, marketing, support, product telemetry, lands the raw output in a Delta lakehouse, refines it through bronze and silver, and publishes marts that downstream consuming applications read. Apache Airflow on AKS orchestrates it, dbt Core models it, and the warehouse is moving from Microsoft Fabric Warehouse to Azure Database for PostgreSQL while all of that stays running.

You own that path end to end, not a link of it: a source nobody has ingested before, carried all the way to a table a business user can open in a dashboard, and everything in between that decides whether the number is right and arrives on time. The extraction that survives pagination, throttling and a schema that changed without telling anyone; the schedule that fires when the data lands rather than when the clock says; the silver table that refuses bad rows instead of passing them downstream; the mart whose columns are documented and whose keys are tested; the semantic-layer definition that makes it usable; and the freshness and the cost of the whole arrangement.

Senior Data Engineering owns the modelling standards and the parity proof that a dbt mart matches the legacy procedure it replaces. You own the path those models sit on; the pipeline that feeds them and the serving layer that consumes them and you build marts to that standard rather than to your own. The seam is real and it is worked together: you bring the source contract, the schedule and the serving behaviour; they bring the layer rules and the value comparison.

You will work in an AI-assisted engineering workflow, where agentic tooling carries the mechanical parts and your judgement stays on the design: what a source's contract is, where a pipeline should fail loudly, and what a dashboard can afford to query.

What you will be doing

  • Onboard sources end to end: authentication, pagination, rate limits, incremental watermarks, landing, bronze, silver with quality gates, warehouse sync as one DAG family per data source, not a bespoke script per table.
  • Own orchestration: DAGs triggered on data arrival, idempotent re-runs, backfills that do not double-count, pod sizing that fits the work, and alerts that page a human only when a human is needed.
  • Build the last mile: the marts and the semantic-layer definitions on top of them, so a metric in Zenlytic or a chart in Superset resolves to a column that means what its name says.
  • Keep the lakehouse healthy: compaction, Z-ordering chosen from real query patterns, vacuum only inside retention, and schema evolution that does not break a consumer.
  • Make performance a measured number: before and after on the pipeline you tuned, capacity spend on the warehouse, caching or pre-aggregation where a dashboard is paying for the same scan every morning.
  • Set and defend freshness and completeness expectations per source; when one breaks, diagnose to root cause and write it down.
  • Run the platform underneath: AKS workloads, Helm or Kustomize, Terraform, image builds and CI/CD, Key Vault and managed identity, least-privilege access to every store you touch.
  • Review pull requests on pipelines and models, and keep runbooks and standards current enough that the next engineer does not need to ask you.
  • Work with the Analytics Engineers and Product Analysts who own the business definitions your marts serve, so a definition change and a pipeline change land in the right order.
Source · OPSWATFirst seen 15 NovOriginal ↗