Try a simple example
Where does my information go?
Some AI runs on your device. Some uses a service elsewhere. Switch between the two examples, then see what changes when you add search or sync.
Your proposed data flow
No selected outgoing route
Task constraints
These two numbers test a simple memory assumption. Passing does not verify hardware support, speed, model quality or all runtime overhead.
What this setup needs
- Data boundaryFits this assumption
No external route is selected in this sketch. Inspect the real application before treating it as local-only.
- ConnectionFits this assumption
No selected route needs a connection once the local model and software are installed.
- Local setupFits this assumption
Local inference needs a compatible model and runtime installed on the device.
- Memory assumptionFits this assumption
Compare the supplied runtime-memory estimate with the memory available. These are assumptions, not verified requirements.
- Current evidenceNot applicable here
The task asks for current information; this sketch has no selected source for it.
- Quality on your taskNeeds a real-world check
Use representative inputs and check the outputs. Location alone does not establish model quality.
- Recipient policiesNot applicable here
Inspect every recipient’s actual data handling and retention terms. Hosted services and integrations differ.
- Operation and upkeepNeeds a real-world check
Plan installation, updates, device resources and ongoing maintenance. Local does not mean zero operating cost.
Planned routes, in words
- Your device → Local modelPrompt and selected input
- Local model → Your deviceAnswer
This sketch shows only the routes you selected. Other app connections, logs and backups may matter.
A planning sketch only. No model is loaded, no settings are changed and no data is transferred. Memory figures are editable assumptions, not measurements or product requirements.
Read the data-flow comparison
With a model already installed, local inference and no selected search or sync route can operate without the external routes shown here. Moving inference to a hosted service adds an outgoing prompt route and requires a connection. Local inference with search still sends a query outside the device; sync sends a conversation copy. If all task data must remain on-device, any of these selected outgoing routes conflicts with that constraint. An 8 GB available-memory assumption cannot fit a 12 GB runtime-memory assumption. Neither location alone proves task quality, privacy, speed or lower cost. All routes are illustrative; no data is sent.
Follow every outgoing route
Local inference can keep the prompt on the device in this sketch. Turning on web search or conversation sync adds an outside recipient. A hosted model adds its own prompt route. Compare these routes with the actual task constraints.
Check the task, then the whole application
Before choosing, identify what data may leave the device, whether current information or offline use is required, and who maintains the system. Test the same task on the candidate setup. Check the actual app’s integrations, logging, backups and provider policies; this sketch cannot verify them.
Go deeper
Inference location is one choice
Inference is running a model on an input. An application can combine local inference with remote features. Follow the whole data path, including search and sync.
Memory is a workload constraint
Weights are only part of runtime memory. Context and concurrent work also matter. Quantization can reduce memory needs, with method-dependent trade-offs. Check the actual model and runtime.
Control includes responsibilities
Local operation brings setup and maintenance work. Hosted operation depends on a provider and a connection. Compare actual quality, availability, data handling and whole operating costs for the same task; this activity does not rank providers.
Explore the original sources
Primary references. These explain the mechanisms; the examples here are authored teaching simulations.
What would you like to try next?
Choose another small example, or return to this one whenever you need it.
See all the examples