Data Annotation Specialist - Software Engineer (OpenClaw Trajectory Evaluation)
Careerflow.ai · 139 days ago
Role Overview:
We are looking for highly analytical professionals to join as S3 Annotators, responsible for producing and validating advanced OpenClaw trajectories for research-oriented tasks involving ambiguous goals, multi-step reasoning, and long-context workflows. This role requires strong problem-solving ability, research thinking, and the capability to justify decisions while executing complex, multi-tool workflows used to train sophisticated AI agents.
What does day-to-day look like
-
Create and validate OpenClaw trajectories for complex, research-oriented tasks
-
Work on tasks such as data analysis, web research, and document synthesis
-
Operate under underspecified or ambiguous goals, making logical and defensible decisions
-
Use Python (notebooks, scripts) for data processing, automation, and analysis
-
Perform workflows spanning multiple tools, environments, and applications
-
Clearly document both what actions were taken and why they were taken
-
Review and improve trajectory quality for reasoning depth and correctness
-
Collaborate with teams to refine evaluation standards and annotation frameworks.
Requirements
-
Background in research, data science, engineering, or related fields (academic or industry)
-
5+ years of experience in Python / Bash, including experience with notebooks and data wrangling
-
Experience with OpenClaw or similar advanced environments
-
Experience with web research, scraping tools, or data analysis workflows
-
Ability to articulate reasoning behind decisions and steps
-
Experience handling multi-step workflows across tools or platforms
-
Exposure to LLM-based tools, AI systems, or agentic workflows
Nice to have
-
Academic research experience (thesis, publications, lab work)
-
Familiarity with evaluation methodologies or experimental design
-
Experience in AI evaluation, RLHF, or advanced annotation workflow
Offer Details:
-
Engagement type: Contractor assignment/freelancer (no medical/paid leave)
-
Duration: 5 weeks
Evaluation Process:
-
Resume screening
-
Take home assessment (60 mins)