Skip to content
Brillio

Lead Data Engineer - R01553055

Bangalore, Karnataka, IndiaHybrid · India
Staff · Employee · AI & Data Engineering
POSTED 17 JUL 2025VERIFIED 6H AGO
Apply Ask
Role
491 days opentrue age
Past the 43d median close
Posted 17 Jul 2025 · n=63
Pay
₹17.8L–₹37.9Lno pay posted
Market p25–p75
₹17.8L–₹37.9L
H-1B median · Data engineering
$137k
India · n=4 · 365d to 19 Nov
CompanyBrillio →
217open roles
Opened · 30d
+71
Closed · 30d
−60
Signal
● Steady+11 net
19d of snapshots · 30d window
Market
470open · Staff Data engineering
Opened · 7d
+6113% of open
All data engineering
3,796open
All tracked boards · live 20 Nov · n=470
Visa
138H-1B LCAs · FY26
PERM · FY26
3
Median wage
$131k
LCAs FY26 · n=138 · to 2026-06-30
PythonJavaKafkaGCPTerraformSparkAirflowdbtSQL

Lead Data Engineer

Primary Skills

  • GCP, Big Query, Python, Airflow, SQL, DBT

Job requirements

About the Role

  • We are seeking a Senior Data Engineer with deep expertise in Google Cloud Platform (GCP) and BigQuery to lead cloud modernization initiatives, develop scalable data pipelines, and enable real-time data processing for enterprise-level systems. This is a high-impact role focused on driving the transformation of legacy infrastructure into a robust, cloud-native data ecosystem.

Key Responsibilities

  • 1. Data Migration & Cloud Modernization
  • Analyze legacy on-premises and hybrid cloud data warehouse environments (e.g., SQL Server).
  • Lead the migration of large-scale datasets to Google BigQuery.
  • Design and implement data migration strategies ensuring data quality, integrity, and performance.
  • 2. Data Integration & Streaming
  • Integrate data from various structured and unstructured sources, including APIs, relational databases, and IoT devices.
  • Build real-time streaming pipelines for large-scale ingestion and processing of IoT and telemetry data.
  • 3. ETL / Data Pipeline Development
  • Modernize and refactor legacy SSIS packages into cloud-native ETL pipelines.
  • Develop scalable, reliable workflows using Apache Airflow, Python, Spark, and GCP-native tools.
  • Ensure high-performance data transformation and loading into BigQuery for analytical use cases.
  • 4. Programming & Query Optimization
  • Write and optimize complex SQL queries, stored procedures, and scheduled jobs within BigQuery.
  • Develop modular, reusable transformation scripts using Python, Java, Spark, and SQL.
  • Continuously monitor and optimize query performance and cost efficiency in the cloud data environment.

Required Skills & Experience

  • 5+ years in Data Engineering with a strong focus on cloud and big data technologies.
  • Minimum 2+ years of hands-on experience with GCP, specifically BigQuery.
  • Proven experience migrating on-premise data systems to the cloud.
  • Strong development experience with Apache Airflow, Python, and Apache Spark.
  • Expertise in streaming data ingestion, particularly in IoT or sensor data environments.
  • Strong SQL development skills; experience with BigQuery performance tuning.
  • Solid understanding of cloud architecture, data modeling, and data warehouse design.
  • Familiarity with Git and CI/CD practices for managing data pipelines.

Preferred Qualifications

  • GCP Professional Data Engineer certification.
  • Experience with modern data stack tools like dbt, Kafka, or Terraform.
  • Exposure to ML pipelines, analytics engineering, or DataOps/DevOps methodologies.

Why Join Us?

  • Work with cutting-edge technologies in a fast-paced, collaborative environment.
  • Lead cloud transformation initiatives at scale.
  • Competitive compensation and benefits.
  • Remote flexibility and growth opportunities.
Source · BrillioFirst seen 2 NovOriginal ↗