Senior Lead Data Engineer

Capital One is hiring a Senior Lead Data Engineer in Plano, TX (onsite) to help design and deliver an enterprise-scale streaming data platform built for low latency and fault tolerance. In this role, you will lead technical delivery across distributed streaming pipelines, while partnering with agile teams, including Cyber & Security, to strengthen how data solutions are built, tested, and supported.

The position focuses on building next-generation streaming pipelines and analytics data stores, using modern distributed processing frameworks and cloud-native data services. You will also guide engineering practices for data pipeline architecture and processing across structured and unstructured data, supporting teams that build at medium to large dataset scale.

What you’ll do

• Lead the design and build of an enterprise scalable, low-latency, fault-tolerant streaming data platform that delivers meaningful and timely insights
• Develop distributed streaming data pipelines and analytics data stores using streaming frameworks such as Flink and Spark Streaming
• Build streaming and analytics solutions using programming languages including Java, Scala, and Python
• Lead a group of engineers using big data technologies including Spark, Flink, Kafka, Snowflake, AWS Big Data Services, and Redshift on medium to large scale datasets
• Influence best practices for data pipeline design, data architecture, and processing of structured and unstructured data
• Collaborate across agile teams to design, develop, test, implement, and support technical solutions using full-stack development tools and technologies
• Work in an agile environment with emphasis on CI/CD and Application Resiliency Standards, partnering with Cyber & Security teams
• Lead a team of developers with deep experience in machine learning, distributed microservices, and full stack systems
• Use Open Source RDBMS and NoSQL databases, along with cloud data warehousing services such as Redshift and Snowflake
• Perform unit tests and participate in code reviews to ensure code is rigorously designed, elegantly coded, and tuned for performance
• Share a commitment to learning and mentoring, experimenting with emerging technologies, and participating in internal or external technology communities
• Collaborate with digital product managers to deliver robust cloud-based solutions that help drive experiences supporting financial empowerment for millions of Americans

Required qualifications

• Bachelor’s Degree
• 6+ years of experience in application development (internship experience does not apply)
• 2+ years of experience in big data technologies
• 1+ year of experience with cloud computing (AWS, Microsoft Azure, or Google Cloud)

Technologies

• Flink
• Spark Streaming
• Java, Scala, Python
• Spark, Kafka
• Snowflake
• AWS Big Data Services
• Redshift
• Open Source RDBMS, NoSQL
• AWS, Microsoft Azure, Google Cloud
• CI/CD

Compensation and benefits

Plano, TX: $209,000 - $238,500 per year for a Senior Lead Data Engineer. The role is also eligible to earn performance-based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI). Capital One provides a comprehensive, competitive, and inclusive set of health, financial, and other benefits to support overall well-being.

Preferred qualifications

• Master’s Degree
• 9+ years of experience in application development including Python, SQL, Scala, or Java
• 4+ years of experience in a public cloud (AWS, Microsoft Azure, or Google Cloud)
• 5+ years experience with distributed data and computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
• 4+ years working on real-time data and streaming applications
• 4+ years experience with NoSQL implementation (Mongo, Cassandra)
• 4+ years of data warehousing experience (Redshift or Snowflake)
• 4+ years of experience with UNIX/Linux including basic commands and shell scripting
• 2+ years of experience with Agile engineering practices
• Experience leveraging interactive AI tooling to accelerate productivity, using capabilities beyond basic code completion

Additional notes

• Capital One will consider sponsoring a new qualified applicant for employment authorization for this position
• This role is expected to accept applications for a minimum of 5 business days
• No agencies please; Capital One is an equal opportunity employer (EOE, including disability/vet) committed to non-discrimination in compliance with applicable federal, state, and local laws
• Capital One promotes a drug-free workplace
• Accommodation requests can be directed to Capital One Recruiting at 1-800-304-9102 or [email protected]
• For technical support or questions about Capital One’s recruiting process, email [email protected]
• Capital One does not provide, endorse, or guarantee third-party products, services, educational tools, or other information available through this site