Aurora is building the BatchAPI high-performance, distributed batch compute engine that powers the self-driving truck Driver. This Software Engineer II role focuses on designing schedulers, APIs, workflow abstractions, and distributed systems solutions to improve performance, scalability, and resilience for critical compute workloads.
Role Overview
In this position, you will design, implement, and maintain core components of BatchAPI, including the scheduler, resource allocator, and execution engine. You will also develop low-latency interfaces and resilient communication between the Python SDK and the Go-based core engine, while creating workflow abstractions that enable teams to define, deploy, and manage complex pipelines.
Responsibilities
- Architect, optimize, and maintain the high-performance large-scale distributed batch compute engine (BatchAPI), with a focus on minimizing overhead for bursty, heterogeneous workloads.
- Build low-latency APIs and resilient communication protocols bridging the Python SDK with the Go core engine.
- Create high-level workflow abstractions that allow engineers to programmatically define, deploy, and manage data processing, simulation, and ML training pipelines.
- Address distributed computing challenges including distributed locking, throttling, and fair-share scheduling to support multi-tenant stability.
- Improve performance, scalability, and resilience across the compute infrastructure by implementing robust monitoring and alerting for critical workflows.
- Collaborate with infrastructure and product engineering teams (Autonomy, Data, Simulation, Machine Learning) to gather requirements, provide compute expertise, and integrate compute workflows with key systems.
Required Qualifications
- 5+ years of professional software engineering experience.
- Deep expertise in Golang (for core systems) and some knowledge of Python (for SDK/API layering).
- Strong distributed systems fundamentals, including topics such as the CAP theorem, consensus algorithms, or gossip protocols.
- Experience profiling and tuning performance, including memory management, I/O bottlenecks, or network latency optimization.
- Experience with container orchestration systems, including Kubernetes.
- Proven ability to drive continuous performance, scalability, and resilience improvements in production environments supporting critical data workloads.
- Familiarity with cloud compute and data services, including AWS EKS, S3, and RDS.
Technologies
- Golang
- Python
- Kubernetes
- AWS EKS
- S3
- RDS
- CAP theorem
Compensation and Location
Location: Louisville, CO (onsite)
Salary: USD 126,000 - 201,000 per yearly
For roles based in Louisville, CO: The salary range is $126,000 - $201,000 per year.
Benefits
- Annual bonus
- Equity compensation
- Benefits
Desirable Qualifications
- Experience with computational workloads in autonomous vehicle, robotics, or large-scale machine learning domains, such as data processing for perception, simulation, or model training.
- Ability to create and refine user-facing tools, incorporating user feedback, managing expectations, and prioritizing development based on user needs.
- Web UI development experience with TypeScript and React.
Working at Aurora
Aurorans are expected to work from the office at least 3 days per week, with some roles requiring more frequent in-office presence, up to 5 days per week, based on operational needs.
Commitment to Safety
Aurora expects commitment to its safety policies from every employee, seeking candidates who take active responsibility, help build an atmosphere of trust, and invest in the organization’s long-term success by prioritizing working safely.