Senior Python & Spark Developer

Job Description:

Role Overview:

We are looking for an experienced Senior Python & Spark Developer with strong expertise in Python (Django, Flask) and PySpark for large-scale data processing. The ideal candidate will be a self-starter who can independently own and deliver end-to-end tasks, while also guiding junior team members.

Key Responsibilities:

• Design, develop, and maintain scalable data-driven and event-driven applications.

• Develop backend services using Python (Django, Flask) and integrate them with distributed data processing pipelines using PySpark.

• Work independently on assigned tasks and take complete ownership from design to deployment.

• Collaborate with cross-functional teams for requirements gathering, design discussions, and delivery planning.

• Guide and mentor junior resources in coding best practices and problem-solving.

• Work with RDBMS and NoSQL databases to design and optimize storage solutions.

• Integrate applications with messaging services like Kafka and MQ for event-driven architectures.

• Ensure smooth deployments through CI/CD pipelines using Jenkins and Docker.

Core Skills & Technologies:

• Languages: Python 3

• Frameworks: Apache Spark (PySpark), Django, Flask

• Databases: PostgreSQL (RDBMS), Cassandra, MongoDB (NoSQL)

• Messaging: Kafka, MQ

• Architecture: Event-Driven, Data-Driven

• CI/CD Tools: Jenkins, Docker

• Monitoring tools: ELK, Prometheus, Grafana

Good to Have:

• Knowledge of Data Lake and Data Warehouse concepts.

• Exposure to large-scale distributed systems.

• Exposure to any of cloud provider [GCP, AWS, Azure]

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.

• 5+ years of professional software development experience.

• At least 2+ years of hands-on experience with PySpark for big data processing.

• Strong backend development experience with Django and Flask.

• Proven ability to work independently and take end-to-end ownership of tasks.

• Strong problem-solving, communication, and mentoring skills.

In the absence of accountable person the reporting manager can take the decision