BIG Data Engineer

Job Description:

BIG Data Engineer

Exp – 6 Yrs to 10 Yrs  

Salary Grid – Up to 25 lpa

Job Code - TechM160753

Work Location: Hyderabad / Chennai

Hybrid – Work from Office – 3 Days a week/ 12 days a month

Key Skills – Pyspark, Spark, Python, Hive, SQL

Job Descriptions:

-  6+ Year experience as Data Engineer with experience in building ETL/Data pipelines with cloud technologies
-  4+ years extensive experience in designing, developing, and implementing scalable data pipelines using Databricks to ingest, transform, and store structured and unstructured data.
-  4+ years' experience programming languages such as Python/Pyspark and query languages like SQL
-  4+ years' experience in building metadata driven data ingestion pipelines using ADF
-  4+ years' experience in analysing, optimizing, and tuning existing data pipelines for performance, reliability, and efficiency
-  2+ years' experience in implementing ML Ops practices to streamline the deployment and management of machine learning models. 
-  2+ years' experience in utilizing Apache Airflow for job orchestration and workflow management
-  Familiarity with CI/CD tools and practices for automating the deployment of data engineering solutions.
-  Experience in collaborating with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions
-  Knowledge/Experience in implementing security measures and standard processes to ensure data privacy and compliance with regulatory standards
-  In-depth knowledge of data engineering concepts, ETL processes, and data architecture principles.