Lead Data Engineer
Manager
Application Security
Aws Data Engineer
Azure Data Engineer
Big Data
Bigdata
CI/CD
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Infrastructure
Cloud Platform
Cloud Platforms
Cloud Technology
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Engineering Lead
Data Governance
Data Integration
Data Lake
Data Lakehouse
Data Lineage
Data Management
Data Observability
Data Pipeline
Data Platform
Data Processing
Data Security
Data Warehouse
Data Warehousing
Database
Databases
Databricks
Databricks Lakehouse
Delta Lake
DevOps
Devops Tools
DevSecOps
Identity and Access Management
Infrastructure As Code
Lead Data Engineering
Metadata Management
Multi Cloud
Project Management
Security Automation
Security Devops
Snowflake
Software Development
Software Security
Spark
SQL
Job Description
Mastercard’s Data Commercialization Platform (DCP) team is building cloud-native data and Business Intelligence capabilities across a modern multi-cloud data ecosystem. This role focuses on designing, developing, and scaling data platforms and analytics solutions spanning Databricks, Snowflake, AWS, and multi-cloud interoperability across AWS and Azure. The position is based in Arlington, VA and works in a hybrid schedule with three days per week onsite.
In this Lead Data Engineer role, you will help shape enterprise data products by delivering scalable pipelines, governed access patterns, and reusable platform capabilities. You will also lead technical design reviews, influence architecture decisions, and mentor engineers through initiatives from design through implementation.
Responsibilities
- Design and develop cloud-native data and BI solutions using Databricks and Snowflake on AWS.
- Build scalable, high-performance data pipelines and analytics frameworks for enterprise data products.
- Drive multi-cloud architecture implementation across AWS and Azure, including Azure Databricks and ADLS Gen2.
- Enable seamless integration and interoperability across Databricks, Snowflake, AWS services, and Azure services.
- Build and maintain CI/CD pipelines, automation frameworks, and Infrastructure as Code for reliable data platform provisioning and deployment.
- Implement secure, governed access patterns using RBAC, IAM, service principals, and enterprise identity integration.
- Design reusable platform capabilities that support data interoperability across multiple compute engines and cloud providers.
- Establish engineering standards for development, testing, deployment, monitoring, documentation, and operational excellence.
- Optimize platforms and workloads for performance, scalability, reliability, resiliency, and cost efficiency.
- Lead technical design reviews and influence architecture and platform engineering decisions.
- Partner with Product, Architecture, Engineering, Data, and Business teams to translate complex requirements into scalable technical solutions.
- Provide technical leadership and mentorship to engineers while driving initiatives from design through implementation.
Requirements
- Deep hands-on experience as a Lead or Senior Data Engineer designing and building enterprise-scale cloud data, analytics, and BI platforms.
- Strong expertise with Databricks, Snowflake, and AWS, including AWS S3, AWS Glue, AWS EMR, and AWS IAM.
- Advanced proficiency with Apache Spark/PySpark, Delta Lake, SQL, and Python for large-scale data processing and analytics.
- Experience building scalable ETL/ELT pipelines, data integration frameworks, and distributed data solutions.
- Experience designing multi-cloud data architectures and integrations across AWS and Azure; experience with Azure Databricks and ADLS Gen2 is a plus.
- Strong experience with CI/CD, Infrastructure as Code, automation, Git/GitHub, and DevOps practices for data platforms.
- Experience implementing enterprise security, governance, and access controls, including RBAC, IAM, service principals, data lineage, metadata management, and observability.
- Knowledge of modern and open data architectures and formats, including Delta Lake and Apache Iceberg.
- Understanding of FinOps, cloud cost optimization, performance, and scalability for large-scale data workloads.
- Proven ability to lead technical design and architecture decisions, drive complex engineering initiatives across teams, and mentor engineers.
- Strong communication and stakeholder management skills, including the ability to articulate technical decisions, trade-offs, risks, and recommendations.
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related STEM discipline (equivalent practical experience considered).
Technologies
- Databricks, Snowflake, AWS, Azure
- Azure Databricks, ADLS Gen2
- AWS S3, AWS Glue, AWS EMR, AWS IAM
- Apache Spark, PySpark, Delta Lake, SQL, Python
- CI/CD, Infrastructure as Code, Git, GitHub, RBAC, service principals
- data lineage, metadata management, observability, Apache Iceberg
Benefits
- Competitive base salary
- Annual bonus or commissions (depending on the role)
- Insurance including medical, prescription drug, dental, vision, disability, and life insurance
- Flexible spending account and health savings account
- Paid leaves including 16 weeks of new parent leave and up to 20 days of bereavement leave
- 80 hours of Paid Sick and Safe Time
- 25 days of vacation time and 5 personal days (pro-rated based on date of hire)
- 10 annual paid U.S. observed holidays
- 401k with a best-in-class company match
- Deferred compensation for eligible roles
- Fitness reimbursement or on-site fitness facilities
- Eligibility for tuition reimbursement
Corporate Security Responsibility
- Abide by Mastercard’s security policies and practices
- Ensure the confidentiality and integrity of the information being accessed
- Report any suspected information security violation or breach
- Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines
Pay Range
- Arlington, Virginia: $161,000 - $266,000 USD
Similar Jobs
S
