C and C++ Developer for LLM Code Evaluation
Job Description
AuraOne Human Data is hiring a remote contractor engineer for an engineering review role focused on evaluating C++ code generated by LLMs. You will validate model outputs in a sandbox, reproduce failures, and provide written, rubric-aligned feedback so the modeling team can improve correctness, style, and edge-case handling.
This is an independent specialist contractor position with US-authorized remote work. The role includes grading paired solutions, writing minimal failing tests, and documenting recurring failure modes.
What you’ll do
- Run and reproduce C and C++ outputs in a sandboxed environment for LLM code evaluation assignments.
- Grade LLM C/C++ developer engineering review submissions for correctness, style, and edge-case handling.
- Create minimal failing tests that capture the specific bug a model missed.
- Compare paired solutions and rank them, including a written rationale tied to the rubric.
- Tag failure modes such as compile errors, runtime crashes, off-by-one, and security issues using severity scores.
- Document recurring code-generation failures so the modeling team can target them.
- Calibrate against gold-standard reviews to keep inter-rater agreement above target.
Example tasks
- Reproduce a generated C++ solution, run the test suite, and grade the result.
- Write the smallest failing test that demonstrates an edge-case bug.
- Compare two paired solutions and rank them with rubric-based reasoning.
- Triage a security issue raised by a model and document the patch the model should have produced.
Required skills and experience
- Strong day-to-day C++ experience, including the ability to read, run, and debug unfamiliar C/C++ code.
- Comfort writing concise unit tests that capture a single failure mode.
- Familiarity with a testing framework such as pytest, Jest, or JUnit (or equivalent frameworks like Go test and RSpec).
- Clear written reasoning that can convince another senior engineer.
- Reliable availability for at least 10 hours per week, with async responsiveness.
- Prior code-review or technical-interview grading experience is a plus.
Technologies
- C++
- pytest
- Jest
- JUnit
- Go test
- RSpec
- C
Nice to have
- Open-source contributions or a public portfolio showing production-quality code.
- Experience with C++ standard tooling, linters, and idiomatic style guidance.
- Familiarity with security review checklists and AppSec patterns, including OWASP and CWE.
Work model, location, and contract details
- Remote work (US-eligible).
- Remote independent specialist contractor.
- Employment type: CONTRACTOR.
- Applicants must be authorized to work from the US.
Compensation
Hourly rate is confirmed after the interview process.
How to apply
Apply through AuraOne’s specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.