74 Job openings found

Job Summary Hands-on experience with writing Spark or Map-Reduce jobs and proficient understanding of distributed computing principles. Proficient in more than one of Python, Java, Scala & Shell-Scripting. Job Responsibilities: Understand & provide innovative solutions to business and product requirements using Big Data Architecture Take ownership of end-to-end data-pipeline including system design and integrating ...

Python, SQL, Spark, Pandas, Airflow, Azure Data Factory, or equivalent, Azure (preferred), AWS or GCP, PostgreSQL, MS SQL Server, NoSQL (MongoDB, etc.), S3, Delta Lake, Snowflake, Redshift, BigQuery, Prometheus, Grafana, ELK, etc Cloud Technologies: Familiarity with cloud platforms such as AWS, Azure, or GCP. Programming Languages: Proficiency in Python Problem-Solving: Strong analytical and problem-solving skills. Communication: Excellent communication and collaboration skills.

Responsibilities: • Responsible for the architecture planning, design, and core code implementation of the company's monitoring system platform, including product technology pre-research, technology architecture selection, core code writing, design document writing, etc.• Participate in the R&D of the company's big data computing platform, responsible for data quality and stability management• Responsible ...

Data Engineer   Job Description: - Having experience in Analysis, Design, Development, Testing, Customisation, Bug fixes, Enhancement, Support and Implementation using Python, spark programming. - Worked on AWS environments such as lambda, server-less applications, EMR, Athena, Glue, IAM policies, roles, S3,CFT and Ec2. - Developed Python and Pyspark programs for data analysis. ...

Experience – 3+ Shift - 2-10 pm Package - based on the exp and Current CTC If Required Need to Visit the client Office . Requirement 1: GCP Data Enginer (With ETL  Experience : 4+ Years  Tech stack: BigQuery , any ETL tool (Informatica, Talend, DataStage), Dataflow, Dataproc •            3-5 years Experience in Data ...

Scala, Spark, Apache Sqoop, Cloudera, Unix Shell scripting, CAAS. Experience: 4-6 years At least four years of data engineering experience working with sophisticated data pipelines using Spark , Scala , Apache Sqoop, Cloudera , Unix Shell scripting and CAAS.   Scala, Spark, Apache Sqoop, Cloudera, Unix Shell scripting, CAAS. Experience: 4-6 years At least four years of ...

Must to Have • We are looking for good resource who has very good experience on Azure Data Lake, Azure Data Factory, Azure Data Bricks (Pyspark) and Azure Synapse • Resource should have knowledge about source systems like Sales Force/ Oracle ERP/SAP Ariba etc and should have worked on the integrations with ...

Data Engineer with Airflow Wilmington DE - Look for Nearby candidates W2 Candidates only F2F is required for this role.   Must Have: 7+ years.Need Pyhton - PySpark or Java SparkAirflowAWS, Data Lake or DatabricksEMR / MapreduceRelational DB Preferred:NOSQLTerraformScala Responsibilities:Software engineering concepts with over 7+ years of applied experienceDegree in Computer Science, Computer Engineering, Mathematics, or a related technical fieldExtensive ...

Role: Azure Data Engineer Work Mode: Onsite/Full-time Location:  Sholinganallur - Chennai   Year of Experience: 5 - 10 Years No of Positions: 1 Notice Period: Immediate  Budget: 8 - 12 LPA Good To Have Skills: Good Communication Must Have Skills: Databricks (Strong), Azure Data Engineering, PySpark, Python, SQL   External Description:  Azure Data Engineering + Databricks (Strong) + Python + SQL  

Analyse and organize raw data Build data systems and pipelines Combine raw information from different sources Explore ways to enhance data quality and reliability Identify opportunities for data acquisition Develop, implement and optimize stored procedures and functions using T-SQL Analyse existing SQL queries for performance improvements Develop procedures and scripts for data migration Provide timely scheduled management reporting Research ...