Lead Data Engineer

Job Description:

Role: Lead the design, development, and optimization of data pipelines and data platform components for an e-commerce business. Work across ingestion, transformation, modeling, governance, and performance to support large-scale Data & AI use cases.

Key Responsibilities

• Build and maintain scalable batch and streaming data pipelines from transactional systems, clickstream, and third-party sources.

• Implement and optimize data lake/lakehouse/warehouse environments on cloud platforms (GCP/AWS/Azure).

• Develop data models and support medallion-layer architecture (bronze/silver/gold) for analytics, BI, and AI workloads.

• Ensure end-to-end data quality, reliability, lineage, and performance.

• Apply best practices in data governance, privacy, PII protection, and security standards.

• Collaborate with architects, analysts, ML teams, and product stakeholders to translate business needs into scalable data solutions.

• Optimize SQL, ETL/ELT jobs, orchestration workflows, and cloud infrastructure for performance and cost efficiency.

• Contribute to defining data engineering standards, coding practices, and technology improvements.

Required Skills – Lead Data Engineer

• 6 -7+ years of experience in data engineering, with strong exposure to cloud-based data platforms.

• Hands-on expertise with data pipelines and streaming technologies (Kafka/PubSub, Spark, Flink).

• Strong SQL and Python programming skills.

• Experience building and optimizing data lake/lakehouse/warehouse platforms on GCP/AWS/Azure.

• Solid understanding of data modeling: dimensional modeling, data vault (or similar), and medallion architecture.

• Familiarity with lakehouse/warehouse tools (Databricks, BigQuery, Snowflake, Redshift, ClickHouse).

• Experience with orchestration tools (Airflow/Cloud Composer, Databricks Workflows, etc.).

• Strong background in data governance, data quality frameworks, cataloging, lineage, RBAC/ABAC, and compliance (GDPR/CCPA).

• Experience in e-commerce, digital, or consumer-facing data environments is a strong advantage (clickstream, customer 360, orders, marketing).

• Good communication skills; able to work with cross-functional teams and support architectural decisions.
  
  
  

  keyword skill 

  • AWS

  • Azure

  • Google Cloud Platform (GCP)

  • Data Pipelines

  • Streaming (Kafka / Spark)

  • SQL & Python

  • Data Lake / Lakehouse

  • Medallion Architecture

  • Databricks

  • Snowflake