Data Engineer 5

Capital One is building large-scale, cloud-first data capabilities for Enterprise Data Tech (EDT), and the company is seeking a Data Engineer 5 (Senior Manager, IC) to lead a major data transformation. In this role, you will design and deliver resilient data pipelines and platforms, applying modern engineering patterns and distributed systems to support AI-era workloads.

Based in New York, NY (onsite), this position is backed by a compensation range of USD 250,800 - 286,200 per year. The role calls for deep hands-on engineering experience across data platforms, cloud tooling, and end-to-end pipeline delivery.

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions using full-stack development tools and technologies
• Influence a team of developers, data analysts, and data scientists with strong backgrounds in machine learning, distributed microservices, lakehouse architecture, and full-stack systems
• Use Python and Spark along with open-source relational and NoSQL databases and cloud data warehousing platforms including Databricks and Snowflake
• Stay current with data engineering trends by experimenting with new technologies, participating in internal and external communities, and mentoring others
• Work with product managers and software engineers to deliver robust cloud-first data solutions that help drive experiences for millions of Americans focused on financial empowerment
• Independently design, build, and deliver cloud data solutions and applications with little or no support from supervisors or managers
• Architect and enforce common data engineering design patterns to improve code quality, maintainability, and reusability across pipelines and platforms
• Deliver pipelines and platforms focused on scalability, resilience, and operational efficiency, maintaining strong performance as data volume and demand increase
• Serve as a force-multiplier by balancing deep technical contribution and innovation with mentoring and elevating peers and junior engineers
• Lead large-scale transformative initiatives end to end, including critical architecture and platform evaluations such as Snowflake versus Databricks based on technical and business requirements
• Act as an ambassador for the team by clearly communicating technical concepts and data outcomes to internal and external stakeholders to align on shared understanding

Requirements

• Bachelor’s degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• At least 6 years of experience in application development (internship experience does not apply)
• At least 4 years of experience in distributed data
• At least 4 years of experience with SQL
• At least 4 years programming with at least one of: Python, Java, or Scala
• At least 4 years designing and developing data pipelines
• At least 2 years in data modeling and designing end-to-end data solutions using both relational and non-relational database systems

Technologies

• Python, Spark, SQL, Databricks, Snowflake
• NoSQL and relational databases
• AWS services and tooling including EMR, Glue, and Redshift
• Lakehouse architecture, microservices, and real-time or streaming data pipelines
• Orchestration and observability tools including Airflow, Dagster, Monte Carlo, and Splunk
• Datastores and platforms referenced in the role: MongoDB, Cassandra, DynamoDB

Benefits

• Health, financial, and other benefits that support total well-being
• Performance-based incentive compensation, including cash bonus(es) and/or long-term incentives (LTI)

Preferred Qualifications

• Master’s degree in Computer Science or a related field
• 8+ years of experience in data engineering
• 5+ years hands-on experience designing, deploying, and operating data workloads in AWS
• 5+ years experience building or supporting distributed data or compute workloads using tools such as EMR, Spark, Glue, or Databricks
• 5+ years experience designing, implementing, and operating real-time or streaming data pipelines
• 5+ years experience with unstructured or semistructured data using NoSQL databases (MongoDB, Cassandra, DynamoDB)
• 5+ years experience designing and supporting data warehousing solutions (Snowflake, Redshift)
• 4+ years of data modeling experience
• 3+ years working in an Agile development environment
• 3+ years developing user-centric reusable data products
• 3+ years working on data observability (Monte Carlo, Splunk) or data orchestration tools (Airflow, Dagster)