Senior Data Engineer

About this role

Not quite right? These jobs in India match the same skills — apply to a few to improve your chances.

Data Engineer(Python/AWS/SQL)

LSEG

In OfficeIndia

Apply

Cloud PaaS-Operations Support-Azure & AWS Certified

Zensar Technologies

In OfficeIndia

Apply

PMO Analyst

WPP

In OfficeIndia

Apply

Data Steward - Quality Excellence

AlphaSense

RemoteIndia

Apply

Skills you'll use

What you'll do

• Design, build, and deploy robust ETL/ELT pipelines within the lakehouse platform (Google BigQuery or Databricks) using SQL, Python, PySpark, and Spark SQL.
• Implement and manage the Medallion Architecture (Bronze, Silver, Gold layers) using Delta Lake or BigQuery datasets to ensure data quality and progressive data refinement.
• Leverage native ingestion tooling – such as BigQuery Data Transfer Service, Pub/Sub streaming, or Databricks Auto Loader – for efficient, scalable, and incremental ingestion of data from sources like GA4 into the Bronze layer.
• Develop, schedule, and monitor complex, multi-task data workflows using Cloud Composer (Airflow), BigQuery scheduled queries, or Databricks Workflows.
• Optimize BigQuery tables (partitioning, clustering, materialised views) and Spark jobs / Delta Lake tables (using techniques like OPTIMIZE, Z-ORDER, and partitioning) for high performance and cost efficiency.
• Implement data governance, security, and discovery using Dataplex / BigQuery policy tags or Unity Catalog, including managing access controls and data lineage.
• Write complex, customized SQL queries to manipulate data and support ad-hoc analytical requests from business teams.
• Develop strategies for data ingestion from multiple sources, using various techniques including streaming, API consumption, and replication.

Who can apply

• Education: Minimum of a bachelor’s degree in computer science, Engineering, Mathematics, or a related technical field preferred.
• Experience: 8+ years of relevant experience in data engineering, with a significant focus on building data pipelines on distributed systems.
• Engineer's Core Skills:
• Lakehouse Platform Expertise (Google BigQuery and/or Databricks)
• BigQuery: Deep, hands-on experience with BigQuery architecture, including partitioning, clustering, materialised views, slot/cost optimisation, and diagnosing query performance using query plans and INFORMATION_SCHEMA.
• Apache Spark / Delta Lake: Strong experience with Spark architecture, writing and optimising PySpark and Spark SQL jobs, and building reliable pipelines on Delta Lake. Proficient with ACID transactions, time travel, schema evolution, and DML operations (MERGE, UPDATE, DELETE).
• Data Ingestion: Experience with modern ingestion tools, such as BigQuery Data Transfer Service, Pub/Sub / Dataflow streaming, Databricks Auto Loader, and COPY INTO for scalable file processing.
• Data Governance: Strong understanding of data governance concepts and practical experience implementing security, lineage, and discovery using Dataplex, BigQuery IAM and policy tags, or Unity Catalog.
• Core Engineering & Cloud Skills:
• Programming: 5+ years of strong, hands-on experience in Python, with an emphasis on PySpark for large-scale data transformation.

About WPP

Searching for “WPP jobs” or “jobs at WPP”? You’re in the right place.