PySpark Developer

Job Description:

Job Title: PySpark Developer
Location: Navi Mumbai (Work From Office)
Experience Required: 3–4 Years
Salary: ₹7–9 LPA

Role Summary

We are seeking a highly skilled PySpark Developer with strong experience in Python, SQL, and AWS cloud services. The ideal candidate will be responsible for developing scalable data pipelines, optimizing data processing workflows, and ensuring efficient data integration across multiple platforms and formats.

Key Responsibilities

• Develop modular, reusable Python/PySpark scripts to integrate data from APIs, databases, SFTP, and S3 into systems such as Data bricks and S3 storage.

• Work with diverse data file formats including CSV, Excel, JSON, XML, and Parquet.

• Write and optimize complex SQL queries focused on performance and scalability.

• Work within AWS environments to manage data pipelines and related infrastructure.

• Build scripts for data cleansing, validation, audit logging, and automated testing.

• Automate recurring data and reporting activities to enhance operational efficiency.

• Apply Data Lake and Data Warehouse best practices during development and implementation cycles.

Mandatory Skills

• PySpark / Python scripting

• Advanced SQL (complex query writing & performance tuning)

• AWS Cloud experience

• Hands-on exposure to AWS services such as Glue, S3, Cloud Watch, CLI, etc.

Preferred / Good-to-Have Skills

• Basic understanding of data modeling

• Knowledge of ETL/ELT processes

• Familiarity with Data Lake and Data Warehouse concepts

• Shell scripting

• Experience working with APIs