Fine-tuning is attractive for a small AI lab.
It is fast. Relatively inexpensive. Easy to explain.
Take a strong open foundation model, show it examples of the behaviour you want, and train a small set of additional weights.
That is close to how we built the first Vinci models.
It was enough to change their voice.
It was not enough to build the models we now want to release.
The short version: we are moving from light behaviour tuning to full-weight model redevelopment.
We are not training a foundation model from random initialization. We will not pretend otherwise.
We are starting with open foundations built by other labs, disclosing where they came from, and continuing the training much more deeply — across knowledge, capability, verifiable behaviour, safety, and character.
Open foundations.
Rebuilt by Vinci.
What fine-tuning gave us
The first Vinci models proved something useful.
Behaviour can move.
A model can become more willing to admit uncertainty.
It can develop a recognizable voice.
It can learn to challenge a bad premise instead of agreeing automatically.
It can become less likely to invent a citation.
Those are not superficial product details.
A highly capable model that fabricates confidently or cannot hold a consistent point of view is still a bad model to think with.
But our larger unreleased experiment also showed the limit of trying to produce capability and character through the same preference stage.
One checkpoint barely changed from its foundation.
Another changed substantially, but lost coding, instruction-following, tool discipline, and safety in exchange.
We had trained the personality without building a stronger underlying model.
Worse, the personality stage could undo capability the foundation already had.
So the architecture changes.
What "full training" means here
There are five parts.
Each one has a different job.
1. Continued pretraining: change what the model represents
Continued pretraining exposes the complete model to a large, carefully constructed body of raw text and code.
This is where a model can absorb a new distribution more deeply than it would through a few thousand instruction examples.
We will begin with a controlled pilot of roughly 500 million to one billion tokens.
Not because that is enough to finish the job.
Because it is enough to determine whether the data and objective are moving the model in the intended direction before we spend several days on the full run.
If the pilot works, the first main stage is approximately five billion high-quality tokens.
We extend beyond that only while the measured improvement per billion tokens remains worth the compute.
A planned maximum is not an instruction to consume it.
The data must also have a thesis.
"More internet" is not a thesis.
If we are improving engineering knowledge, multilingual depth, business reasoning, or some other distribution, we have to name it, measure it, and compare it against an equal-compute control.
2. Supervised training: show complete, verified work
The next stage teaches the model how strong work looks.
Not merely how a Vinci answer sounds.
- Complete code that passes its tests.
- Reasoning examples whose conclusions can be checked.
- Tool calls with valid schemas and real execution.
- Agent trajectories that actually finish the task.
- Instructions followed literally when literal compliance matters.
The target is tens or hundreds of thousands of verified examples, built through rejection sampling and approved teacher models.
Generate many candidates.
Keep the ones that are correct.
Recheck them after any voice transformation.
If making an answer sound more like Vinci breaks the code, weakens the reasoning, or removes a required section, the example is rejected.
Voice does not get to overrule correctness.
3. Reinforcement learning: reward what can be proven
For coding, the reward is not whether another model says the answer looks good.
The code runs or it doesn't.
For math, the answer checks or it doesn't.
For a tool call, the schema is valid and the tool executes — or it isn't.
For instruction-following, the requested constraints are present — or they aren't.
This is reinforcement learning with verifiable rewards.
It is where we expect much of the capability improvement to come from.
Preference data cannot contain a solution better than the solutions it was given.
Reinforcement learning can generate many attempts, test them, and strengthen the behaviours that actually succeed.
The verifier is the reward function.
That means building the tests is part of training the model.
Weak tests create a different shortcut: code that collects reward without solving the real problem.
So the highest-reward outputs have to be inspected, especially early in a run.
The model will optimize what we measure.
Our job is to make the measurement difficult to game.
4. Preference training: add the Vinci layer without taxing capability
Preference training still matters.
It moves the parts that programmatic rewards cannot fully describe:
- Honesty
- Judgment
- Restraint
- Safety
- Identity
- Voice
- Knowing when to challenge the user
- Knowing when thoroughness is warranted
But it comes later, after the capability stages.
And the dataset changes.
Restraint can no longer mean shorter.
Every behavioural direction needs its counterweight.
- Concision and completeness.
- Decisiveness and clarification.
- Scope discipline and persistence.
- Refusal and useful redirection.
- Tool use and tool abstention.
Response length is measured in every pair. If length predicts the preferred answer materially better than chance, the batch fails.
The run also checkpoints densely.
We are not training toward "the last step." We are mapping the point where character improves and the point where capability starts to degrade.
The checkpoint between those points is the candidate.
5. Selection: the run does not decide what ships
Training produces checkpoints.
Evaluation selects a model.
The distinction sounds obvious. We did not enforce it strongly enough before.
Every stage now preserves an immutable parent:
Foundation
→ continued-pretraining candidate
→ supervised candidate
→ reinforcement-learning candidate
→ preference checkpoint sweep
→ selected model
If continued pretraining damages the model, we return to the foundation.
If supervised training damages it, we return to the continued-pretraining checkpoint.
If the preference pass undoes what reinforcement learning bought, we release neither the final step nor a story explaining the regression away. We return to the RL parent or select an earlier preference checkpoint.
Finishing a stage does not earn the next one.
Passing its gate does.
The model is not the only thing we're building
The durable asset is not one checkpoint.
It is the system that can produce better checkpoints repeatedly:
- A licensed and versioned data factory
- Rejection sampling
- Teacher distillation
- Code, math, tool, and instruction verifiers
- A trainer we control
- Dense checkpoint evaluation
- Frozen private tests
- External reproducibility
- Model bills of materials
- Signed, deployable weights
If the method only works once, on one foundation, we have not built much of a lab.
It has to transfer.
One model line may prioritize maximum open capability. Another can begin from a foundation selected for stricter provenance or procurement requirements. The data, reward systems, evaluations, and operating discipline should improve both.
That is a much more durable capability than "we fine-tuned this month's best checkpoint."
The economics force clarity
We train on multi-region Canadian sovereign infrastructure.
The fleet costs approximately $45,000 each month before funding support.
Roughly $1,500 every day.
That number creates two temptations.
The first is to keep every GPU busy, even when the job has no credible path to a better model.
The second is to hesitate so long that an expensive machine sits idle while we debate the perfect plan.
Both are waste.
Our new operating rule is not "always running."
It is: every run must improve a model, produce a reusable research asset, or resolve a decision.
We maintain a queue of ready work. Small pilots fail cheaply. The next job is prepared before the current one ends. Data, verifiers, evaluations, and training infrastructure move in parallel.
Aggressive does not mean skipping gates.
It means moving immediately when a gate passes — and killing the run immediately when it doesn't.
What the next model has to do
Our own models get no home-field advantage.
The next larger Vinci model must beat its properly elicited foundation across multiple product capabilities.
It cannot buy personality by giving back coding.
It cannot buy honesty by weakening safety.
It cannot claim better tool use by improving one tool track while losing overall function-calling discipline.
And it cannot receive credit for a judgment gain that disappears when response length is controlled.
Before release, it has to clear:
- Coding and reasoning
- Strict instruction-following
- Tool selection, execution, and abstention
- Multi-turn agentic work
- Safety and jailbreak resistance
- Grounding and anti-fabrication
- Quantized deployment
- Real Vinci product tasks
Then independent or externally reproducible evaluation.
If it loses, we do not release it because the GPUs were expensive or the announcement was scheduled.
We keep the checkpoint private and continue.
What Vinci is becoming
Vinci began with a Constitution and a character thesis.
That remains.
But character cannot be the layer that makes a model less capable.
We are becoming a foundation-independent post-training lab: taking strong open foundations, continuing their training across the full model, building capability through verifiable work, and adding a judgment layer that survives contact with the real tasks people use AI for.
Still open weight.
Still transparent about the foundation.
Still something you can own.
But trained much more deeply than what we released before.
The next Vinci model will not ship because the run finished.
It will not ship because it has our name.
It will ship when it earns it.
