Piccolo on a 6 GB Android phone and the wait for an answer

Research by Anamika Rawat and Tulsi Patel, full-time co-op student researchers at Conestoga College, supervised by Dushyant Puri. This work is part of SimpleDirect®’s collaboration with Conestoga’s Centre for Commercialization; the benchmarking comes through the SMART Centre.

A Samsung S20 FE with 6 GB of RAM generated text from Piccolo. In the shorter of the two recorded runs, the app reported 96.7 seconds of generation, about a minute and 37 seconds.

That would be a long wait for a quick question. It could be acceptable for something I leave running while doing other work, provided the answer is useful. That last part still needs checking.

I’m interpreting the Conestoga team’s observations here. We haven’t independently repeated the hardware test.

What happened on this phone

The team ran Vinci Piccolo 1.0, our roughly four-billion-parameter model, in Off Grid AI. The download was identified as Vinci-Piccolo-1.0-GGUF. A separate app monitored the device.

There was one phone, one prompt and two sequential observations. The prompt asked for a 200-word summary of aquatic life. Thinking was enabled for the first run and disabled for the second. [1]

Tokens are pieces of generated text, so those counts don’t tell us whether the final summaries had 200 words. The first run generated tokens faster and produced many more of them. The app reported about three minutes and 22 seconds of generation. The records do not provide a separate end-to-end measurement covering loading, prompt processing and the final displayed answer. [1, slides 6–11]

These observations don’t isolate the effect of thinking mode. The runs happened in a fixed order, with starting CPU readings of 31°C and 42°C. Both prompts appear in the same conversation, so the second may have included additional context.

The 1,024-token result also warrants checking against a possible output limit. Without a confirmed cap or finish reason, I can’t call it truncated. Neither answer’s quality or compliance with the requested length has been established.

Check the warning before drawing a conclusion

A memory warning appeared during setup. We need the exact model file, compression settings, available memory and app configuration to understand it. The records don’t identify the engine running the model inside the app. The warning cannot establish a minimum RAM requirement for Piccolo. [1, slide 7]

The monitor also showed 84°C CPU readings during or after both responses. These are internal sensor readings. They don’t measure surface temperature or establish battery safety. Two snapshots cannot show sustained thermal behaviour, prove throttling or explain the speed difference. [1, slides 8–11]

Would I use this setup

Before relying on it, I’d choose a few phone tasks with answers I can check. For each, I’d save the full response, measure when useful information first appears and record total waiting time. I’d repeat them in fresh conversations with the settings and starting conditions written down.

Then I could decide whether the wait is acceptable, whether a shorter task helps, or whether I should try another setup. This first observation gives us a reason to investigate the phone we already have. It doesn’t yet support an everyday-use recommendation.

Read the Mac comparison, the Jetson and Raspberry Pi results, and the collaboration overview for the other early findings.

Sources

[1] SimpleDirect Vinci-Piccolo Benchmarking.pptx, slides 2–11. Device and applications: slides 2–3. Prompt and setup: slides 6–7. Generation figures and monitor readings: slides 8–11. The deck credits Conestoga SMART Centre.

The supplied SimpleDirect Vinci-Piccolo Benchmarking (1).pptx contains the same substantive phone observations. It is not an independent replication. There are no repeated-trial averages or uncertainty estimates.

Model reference: Vinci Piccolo 1.0 release, model card and Apache-2.0 download.

Benchmarking by Anamika Rawat and Tulsi Patel, supervised by Dushyant Puri, through Conestoga College’s SMART Centre. SimpleDirect®’s collaboration is with Conestoga’s Centre for Commercialization. Article and interpretation by George Pu.