Senior Software Engineer, Crusoe Managed Data Transfer
Backend Developer
Senior
API
APIs
Application Security
Cloud
Cloud Infrastructure
Cloud Platform
Cloud Platforms
Cloud Technology
Data Integration
Database
Databases
Engineering
Go
Grpc
Infrastructure
Infrastructure As Code
Kubernetes
Platform Engineering
REST APIs
Software Development
Software Engineer
Software Engineering
Software Security
Job Description
Crusoe is hiring a Senior Software Engineer to build the Managed Data Transfer (MDT) control plane for petabyte-scale, resumable transfers into Crusoe storage.
Responsibilities
- Design and build the data-transfer API with provider-neutral source and destination models (create, list, get, cancel) to support new clouds and storage types without breaking customers.
- Develop the scheduler and state machine to split bucket listings into units of work, dispatch them to regional worker pools, and track progress, retries, and resume behavior after failures.
- Enable safe retry and cancel for petabyte-scale jobs, including end-to-end checksum verification and clear errors for permission issues or invalid destinations.
- Implement secure access to customer buckets using short-lived, least-privilege credentials (AWS role assumption first, with other clouds next), integrated with Crusoe IAM and audit logging.
- Ship the CLI and collaborate with partner teams on Terraform and Console experiences for creating and monitoring transfers.
- Instrument MDT with metrics, logs, and alerts; write runbooks; participate in the team on-call rotation.
- Align on interfaces MDT depends on with Storage, Network, IAM, and Console; work with Product and Solutions Architects on early customer onboarding.
Requirements
- Production experience designing and shipping backend services and APIs, ideally in Go, using gRPC or REST, with relational databases and Kubernetes.
- 5+ years building cloud infrastructure or platform services, including taking a service from zero to general availability.
- Comfort working with long-running, failure-prone systems: idempotency, retries with backoff, leases, checkpointing, and handling partial failure.
- Ability to design public APIs that customers can automate against and that can evolve without breaking changes.
- Working knowledge of authentication, authorization, and least privilege, with careful handling of customer credentials.
- Experience running services in production, participating in on-call, and building dashboards and alerts that reduce incident duration.
- Cross-team collaboration skills; design docs that drive clear decisions.
- Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent professional experience).
Technologies
- Go, gRPC, REST
- Relational databases, Kubernetes
- AWS role assumption, AWS IAM, STS
- Terraform, CLI, Cloud Console
- NFS, object storage, parallel file systems
Benefits
- Competitive compensation and equity packages
- Restricted Stock Units
- Paid time off, paid holidays & leave of absence programs
- Comprehensive health, dental & vision insurance
- Employer contributions to HSA account
- Paid parental leave
- Paid life insurance, short-term and long-term disability
- Professional development & tuition reimbursement
- Mental health & wellness support
- Commuter benefits (parking & transit)
- Cell phone stipend
- 401(k) Retirement plan with company match up to 4% of salary
- Volunteer time off
- Global travel insurance & emergency assistance
- Daily meals allowance
- Additional perks & programs specific to location
Bonus Points
- Experience with large-scale data movement such as S3 multipart uploads, rclone, s5cmd, AWS DataSync, or Google Storage Transfer Service.
- Hands-on cross-cloud identity experience (AWS IAM and STS, GCP service-account impersonation, or Azure workload identity).
- Experience with storage systems (object storage, NFS, parallel file systems) or high-throughput networking.
- Experience with job orchestration systems like Temporal, Argo, or Kubernetes operators.
- Experience building on GPU or AI infrastructure.
Compensation Range
- Up to $170,000 - $205,000 per year + Bonus
- Restricted Stock Units included in all offers
- Compensation determined by knowledge, education, and abilities, plus internal equity and market alignment
Location: San Francisco, CA (onsite)
Experience: 5+ years
Education: Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent professional experience)