Fewer unsupported claims, but fewer useful answers.
- What happened
- We applied the same training approach to three model families. They made fewer unsupported claims, but also gave fewer correct answers to questions they should have been able to answer. None passed all the study’s checks.
- What you can use
- When a score improves, ask what got worse and how often the model still gives a correct answer.
- Keep in mind
- Two other AI models scored answers in an early test. The separate final test was not run, broader abilities were not evaluated, and no trained version was recommended for release.