Job Description:

The right candidate enjoys working collaboratively to solve complex data engineering challenges that support our Business needs

Responsibilities

• Design, implement and maintain large-scale data pipelines using Python, Databricks
• Provisioning, developing, maintaining and tuning Amazon Oracle RDS
• Work with cross-functional teams to integrate solutions that meet our data processing and analysis needs
• Ensure performance, reliability, security and maintainability of data pipelines
• Develop monitoring and alerting mechanisms to proactively identify and resolve issues
• Continuously look for opportunities to improve data engineering processes and practices

Requirements

• Minimum of 3+ years with expertise in designing and maintaining large-scale data pipelines
• Strong proficiency in building and maintaining stored procedures, triggers and functions in Oracle DB
• Strong proficiency in using DMS to sync data from RDS to Databricks
• Strong proficiency in Python and Databricks(AWS) in production environment
• Excellent understanding of Big Data technologies, primarily Spark
• Strong understanding of Data Architecture and modelling concepts
• Highly knowledgeable about data warehousing principles and practices

Good to have skills

• Knowledge in State Street internal Datawarehouse ESP
• Certification on AWS, Databricks

Educational qualifications

BS in Computer Science with 5 years of experience or MS in Computer Science with 3 years of experience

Salary Range:

The range quoted above applies to the role in the primary location specified. If the candidate would ultimately work outside of the primary location above, the applicable range could differ.