DeveloperJobs.io
← Back to all jobs

Job Description

AuraOne Human Data is hiring a remote contractor engineer for an engineering review role focused on evaluating C++ code generated by LLMs. You will validate model outputs in a sandbox, reproduce failures, and provide written, rubric-aligned feedback so the modeling team can improve correctness, style, and edge-case handling.

This is an independent specialist contractor position with US-authorized remote work. The role includes grading paired solutions, writing minimal failing tests, and documenting recurring failure modes.

What you’ll do

  • Run and reproduce C and C++ outputs in a sandboxed environment for LLM code evaluation assignments.
  • Grade LLM C/C++ developer engineering review submissions for correctness, style, and edge-case handling.
  • Create minimal failing tests that capture the specific bug a model missed.
  • Compare paired solutions and rank them, including a written rationale tied to the rubric.
  • Tag failure modes such as compile errors, runtime crashes, off-by-one, and security issues using severity scores.
  • Document recurring code-generation failures so the modeling team can target them.
  • Calibrate against gold-standard reviews to keep inter-rater agreement above target.

Example tasks

  • Reproduce a generated C++ solution, run the test suite, and grade the result.
  • Write the smallest failing test that demonstrates an edge-case bug.
  • Compare two paired solutions and rank them with rubric-based reasoning.
  • Triage a security issue raised by a model and document the patch the model should have produced.

Required skills and experience

  • Strong day-to-day C++ experience, including the ability to read, run, and debug unfamiliar C/C++ code.
  • Comfort writing concise unit tests that capture a single failure mode.
  • Familiarity with a testing framework such as pytest, Jest, or JUnit (or equivalent frameworks like Go test and RSpec).
  • Clear written reasoning that can convince another senior engineer.
  • Reliable availability for at least 10 hours per week, with async responsiveness.
  • Prior code-review or technical-interview grading experience is a plus.

Technologies

  • C++
  • pytest
  • Jest
  • JUnit
  • Go test
  • RSpec
  • C

Nice to have

  • Open-source contributions or a public portfolio showing production-quality code.
  • Experience with C++ standard tooling, linters, and idiomatic style guidance.
  • Familiarity with security review checklists and AppSec patterns, including OWASP and CWE.

Work model, location, and contract details

  • Remote work (US-eligible).
  • Remote independent specialist contractor.
  • Employment type: CONTRACTOR.
  • Applicants must be authorized to work from the US.

Compensation

Hourly rate is confirmed after the interview process.

How to apply

Apply through AuraOne’s specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.

Similar Jobs