Data Engineer

Responsibilities include:

• Act as the liaison between Product Owners and technical developers, translating new requirements into designs, development strategies, and implementation plans.
• Build, and implement a Data Lake by ingesting and integrating commercial data from diverse sources to support both descriptive and predictive analytics.
• Lead a cross-functional team of AWS developers, Snowflake developers, UI developers, and Testers, working with technologies like Snowflake PL/SQL, StreamSets, AWS Glue, Lambda, Airflow, shell scripting, and React on EBS.
• Ensure data integrity and reporting consistency by establishing optimal processes and procedures for teams to follow.
• Conduct code reviews for Python Lambda and Snowflake stored procedures, optimizing them to adhere to industry best practices.
• Design and develop solutions that align with both business needs and central technology requirements.
• Lead estimation efforts, collaborating with architects for scoping and with business teams for prioritization.
• Validate and approve technical direction and implementation as part of the formal review process.
• Provide issue resolution support before, during, and after deployment phases.
• Balance the requirements of individual product teams with the overarching Salesforce program objectives.
• Facilitate team meetings, lead Scrum ceremonies, and engage with business stakeholders.
• Enhance and expand the technical knowledge base within the BSC Salesforce community.

Basic Qualifications:

• Bachelor’s degree with 5-7 years of experience leading Data Engineering teams.
• Over 6 years of expertise in writing advanced SQL and PL/SQL, specifically in Snowflake or Oracle.
• 5+ years of hands-on experience with various AWS services, particularly Lambda and CloudFormation.
• Proficient in core Python, including advanced techniques such as slicing and working with dictionaries.
• Expertise in Snowflake, focusing on query optimization, automation, data governance, security (encryption, access control), compliance (HIPAA, GDPR), and workflow automation (Streams, Tasks, and Materialized Views).
• Experience in at least three big data projects utilizing Snowflake and Snowpark.
• Skilled in data warehouse and data mart modeling.
• Proven ability to design efficient data models that optimize storage and retrieval, with experience securely sourcing external data from multiple sources.
• Strong background in addressing data scaling and product challenges, designing, building, and launching reliable data pipelines of varying complexity.
• Proficient in ETL tools such as IICS, StreamSets, Data Factory, etc.
• Familiar with major platforms like Salesforce, SAP, ServiceNow, Hana, and Zoom.
• Expertise in release management, with experience using GIT, Jenkins, or other CI/CD tools.
• Experience in consuming data from REST APIs using Python or Java, with the ability to parse JSON and XML data.
• Hands-on experience with reporting tools like Tableau, PowerBI, or SSRS.
• Strong written, verbal, and presentation skills, with the ability to collaborate effectively with team members.
• Familiar with Jira and other Atlassian tools for agile lifecycle management.
• Knowledge of quality assurance and documentation best practices in software development.

Preferred Qualifications:

• AWS Platform Certifications (Developer, Solution Architect)
• SnowPro Core Certification
• Knowledge on ML & AI, Snowflake Cortex, Document AI
• Experience in Airflow, AWS Glue, Streamsets,
• Expertise in Jupyter notebook
• Experienced with front-end web development frameworks and hands-on development experience with react JS.
• Expertise with Agile software development methods and processes