DeveloperJobs.io
← Back to all jobs

Job Description

Crusoe is hiring a Senior Software Engineer to build the Managed Data Transfer (MDT) control plane for petabyte-scale, resumable transfers into Crusoe storage.

Responsibilities

  • Design and build the data-transfer API with provider-neutral source and destination models (create, list, get, cancel) to support new clouds and storage types without breaking customers.
  • Develop the scheduler and state machine to split bucket listings into units of work, dispatch them to regional worker pools, and track progress, retries, and resume behavior after failures.
  • Enable safe retry and cancel for petabyte-scale jobs, including end-to-end checksum verification and clear errors for permission issues or invalid destinations.
  • Implement secure access to customer buckets using short-lived, least-privilege credentials (AWS role assumption first, with other clouds next), integrated with Crusoe IAM and audit logging.
  • Ship the CLI and collaborate with partner teams on Terraform and Console experiences for creating and monitoring transfers.
  • Instrument MDT with metrics, logs, and alerts; write runbooks; participate in the team on-call rotation.
  • Align on interfaces MDT depends on with Storage, Network, IAM, and Console; work with Product and Solutions Architects on early customer onboarding.

Requirements

  • Production experience designing and shipping backend services and APIs, ideally in Go, using gRPC or REST, with relational databases and Kubernetes.
  • 5+ years building cloud infrastructure or platform services, including taking a service from zero to general availability.
  • Comfort working with long-running, failure-prone systems: idempotency, retries with backoff, leases, checkpointing, and handling partial failure.
  • Ability to design public APIs that customers can automate against and that can evolve without breaking changes.
  • Working knowledge of authentication, authorization, and least privilege, with careful handling of customer credentials.
  • Experience running services in production, participating in on-call, and building dashboards and alerts that reduce incident duration.
  • Cross-team collaboration skills; design docs that drive clear decisions.
  • Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent professional experience).

Technologies

  • Go, gRPC, REST
  • Relational databases, Kubernetes
  • AWS role assumption, AWS IAM, STS
  • Terraform, CLI, Cloud Console
  • NFS, object storage, parallel file systems

Benefits

  • Competitive compensation and equity packages
  • Restricted Stock Units
  • Paid time off, paid holidays & leave of absence programs
  • Comprehensive health, dental & vision insurance
  • Employer contributions to HSA account
  • Paid parental leave
  • Paid life insurance, short-term and long-term disability
  • Professional development & tuition reimbursement
  • Mental health & wellness support
  • Commuter benefits (parking & transit)
  • Cell phone stipend
  • 401(k) Retirement plan with company match up to 4% of salary
  • Volunteer time off
  • Global travel insurance & emergency assistance
  • Daily meals allowance
  • Additional perks & programs specific to location

Bonus Points

  • Experience with large-scale data movement such as S3 multipart uploads, rclone, s5cmd, AWS DataSync, or Google Storage Transfer Service.
  • Hands-on cross-cloud identity experience (AWS IAM and STS, GCP service-account impersonation, or Azure workload identity).
  • Experience with storage systems (object storage, NFS, parallel file systems) or high-throughput networking.
  • Experience with job orchestration systems like Temporal, Argo, or Kubernetes operators.
  • Experience building on GPU or AI infrastructure.

Compensation Range

  • Up to $170,000 - $205,000 per year + Bonus
  • Restricted Stock Units included in all offers
  • Compensation determined by knowledge, education, and abilities, plus internal equity and market alignment

Location: San Francisco, CA (onsite)

Experience: 5+ years

Education: Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent professional experience)

Similar Jobs