A set of Language Model Risk cards for assessing a language model use case. To use these:
- Choose what use-case, model, and interface is to be assessed
- Select which of these risk cards is relevant in the given use-case scenario
- Recruit people to do the assessment
- For each risk card,
- Design how one will probe the model, and for how long
- Try to provoke the described behaviour from the language model, using your own prompts
- Record all inputs and outputs
- Compile an assessment report
Full details are given in the paper: Assessing Language Model Deployment with Risk Cards (2023), Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, Saif Mohammad