Senior Software Engineer
Job Description
Northwestern Mutual is seeking a Senior Software Engineer to lead the end-to-end design and delivery of data platforms and pipelines on AWS, enabling analytics, reporting, ML, and operational use cases. This onsite role in Milwaukee, WI combines ownership of data architecture, governance, security, cost optimization, and production reliability with mentoring responsibilities for engineers. The position offers a salary range of USD 118,960 to 205,200 per year and calls for at least seven years of hands-on experience in data engineering.
Responsibilities
- Oversee the full lifecycle of data platforms and pipelines, including batch and streaming workloads, leveraging AWS services to support analytics, reporting, ML, and operations.
- Own data architecture decisions, establish standards and best practices, and produce Architecture Decision Records, runbooks, and high-quality diagrams.
- Design and implement data lake and data warehouse solutions covering ingestion, storage, transformation, and serving using S3, Glue, Redshift, Athena, and related services.
- Develop and optimize ETL/ELT workflows with Spark, Glue, Lambda, Step Functions, and orchestration tools such as Airflow or Dagster.
- Design and manage streaming ingestion and processing using Kinesis, MSK, and Kafka ecosystems; participate in re-architecture efforts to reduce cost and improve resilience.
- Lead data modeling, schema design, partitioning, and query optimization for large datasets to meet performance and cost targets.
- Establish and operationalize data quality, validation, observability, lineage, and metadata practices (Glue Data Catalog, Data Hub/Amundsen or equivalent).
- Implement robust security, governance, and compliance controls (IAM, Lake Formation, KMS, VPC, data masking and security scanning) in collaboration with security and privacy teams.
- Champion cost optimization across the data stack, employing serverless architectures, right-sizing, and Graviton/spot/commitment strategies with ongoing governance.
- Own production reliability through monitoring, SLIs/SLOs, incident response, postmortems, and continuous improvements.
- Lead and mentor engineers, conduct design reviews, and foster cross-team collaboration with product, analytics, ML, and platform groups.
- Influence the data engineering roadmap, capacity planning, and hiring; represent the team in senior technical forums.
Requirements
- More than seven years of hands-on data engineering experience, including designing and operating scalable data platforms.
- Proven leadership on large, complex data initiatives with the ability to influence technical and product stakeholders.
- Extensive practical experience with AWS data services such as S3, Glue, Redshift (Spectrum/RA3), Athena, EMR, Lambda, Kinesis or MSK, and DMS.
- Proficiency in Python and/or Scala (or Java) with a track record of delivering production-quality, testable code.
- Deep SQL expertise with a history of designing and optimizing queries for large-scale data.
- Hands-on experience with distributed processing frameworks such as Spark and modern ETL/ELT patterns.
- Familiarity with orchestration and workflow tools like Airflow, Step Functions, Dagster, and CI/CD practices for data pipelines.
- Strong infrastructure-as-code skills using Terraform, CloudFormation, or CDK, plus GitOps practices.
- Experience implementing data security, governance, and access controls at scale.
- Expertise in observability and incident management using CloudWatch, Prometheus, Grafana, structured logging, and tracing.
- Excellent communication skills with proven mentoring and leadership abilities.
Technologies
- S3
- Glue
- Redshift
- Athena
- EMR
- Lambda
- Kinesis
- MSK
- DMS
- IAM
- Lake Formation
- KMS
- VPC
- Glue Data Catalog
- Data Hub
- Amundsen
- OpenLineage
- Delta Lake
- Iceberg
- Databricks Unity Catalog
- Kafka
- Spark
- Airflow
- Dagster
- Step Functions
- Terraform
- CloudFormation
- CDK
- GitOps
- Python
- Scala
- Java
- SQL
- CloudWatch
- Prometheus
- Grafana
Benefits
- Flexible work schedules
- Concierge service
- Comprehensive benefits
- Employee resource groups
Similar Jobs
