Core responsibilities
- Select a model and output format
- Build safeguards and validation
- Evaluate regressions and instrument calls
Technical skills
- Structured outputs and context budgets
- LLM evaluation sets and sampling
- Inference, usage monitoring and error handling
Behavioural skills
- Agree thresholds with domain teams
- Make answer quality measurable
Typical deliverable
Structured clause extraction with abstention, regression tests and a token budget.
Possible assessment methods
Improve a fragile pipeline, validate its outputs and measure the effect on annotated samples.
