For anyone planning a first experiment
How to try an AI model on a small task
Choose a task whose answers you can check, set the rules first, and keep a record of what worked and what did not.
Updated
Try sorting a small list into categories
Hypothetical example — this is not a tested Vinci workflow or a performance claim. Make a list of 12 imaginary purchases. Ask a model to put each into a category using rules you supply. Use invented entries so no personal records are needed. For Vinci MLE 1.0, a valid test on ML-engineering tasks set aside for assessment was not completed; this example is not evidence of its capabilities.
| Part of the plan | A concrete example |
|---|---|
| The task | Use three categories: Food, Travel and Supplies. Your rules say “bread” is Food, “bus ticket” is Travel and “notebook” is Supplies. Include a few practice examples, then ask for one category for each of 12 new entries. |
| The answer sheet | Write the expected categories yourself before the trial and keep them out of the model’s input. Choose entries with one clear answer under your rules, so you can check the result fairly. |
| What counts as success | For this example, require all 12 categories to match your answer sheet, every entry to appear exactly once, and the original descriptions to remain unchanged. Choose a time limit before starting. |
| What counts as failure | A wrong category, missing or repeated entry, changed description, or exceeded time limit fails this particular test. Save the answer and record which rule failed. |
| What you can conclude | Report the number correct out of 12 and whether all the rules passed. A good result here tells you about this small task; it does not establish performance on real financial records or a whole engineering project. |
More detail and a worksheet
Ready to try this yourself? These extra checks and a worksheet help you keep track of what you used and what happened.
Set the question and answer rules first
Write down what you want back and how you will check it. For an engineering model such as Vinci MLE, this same planning principle applies, but the test must match the engineering work you want to assess.
- Keep practice examples separate from the final questions and answer sheet. Do not include the answer sheet in the model’s input.
- Save the starting files and decide what counts as success, failure or an invalid attempt before beginning.
- Vinci MLE 1.0 did not complete a valid test on reserved ML-engineering tasks — tasks kept aside for assessment. A small personal trial does not fill that gap or prove autonomous engineering ability.
Decide what the model is allowed to do
A downloaded model needs software around it to receive questions and return answers. Access to files or other tools depends on that software; it is not supplied by the weights alone.
- Use copied or invented material. Keep original work and passwords outside the trial.
- If your software lets the model edit files or use tools, allow only the actions needed for your task and set a time limit.
- Save the model name, software version, exact instructions and all returned answers so you can compare attempts.
Check the answer yourself
The model’s statement that it finished is not your score. Compare its answer with the rules you wrote, using your answer sheet or a separate checking program.
- Keep failed attempts as well as successful ones. Decide in advance how many attempts you will allow.
- Record correct answers, missing items, unwanted changes and time taken separately.
- Use what you learned to choose the next small test. Do not treat success on one example as proof that the model can handle every similar task.
Copy a small-test plan
Fill in the plan before asking the model. Add the result afterward without changing the original rules.
MY PLAN Task I want to try: Input files or invented examples: What the model should return: Where I keep the answer sheet (not given to the model): How I will check the answer: What counts as success: What counts as failure: What would make an attempt invalid, such as a software interruption: Allowed actions and time limit: Number of attempts I will allow: Model and software versions: Exact instructions I will give: EACH RESULT Attempt number: Saved answer: Number correct / number checked: Missing items or unwanted changes: Success / failure / invalid, and why: Time taken: WHAT I LEARNED Successful attempts / all attempts: Invalid attempts and reasons: Unanswered questions: Next small test, or reason to stop:
Sources and next steps
A model card is the publisher’s information page for a model. Follow it for the exact download instructions, licence and known limits. External technical sources are in English.