how to use this with Google 's litertlm on Android
I am using com.google.ai.edge.litertlm with my Android projects for local AI.
But they use litertlm files. This is a tflite file model. Does it work directly with litertlm ?
if not why is it here on litert-community ?
thanks.
Good question — this one is not a LiteRT-LM model, so it won't load with com.google.ai.edge.litertlm. .litertlm bundles are for LiteRT-LM's LLM pipeline; Bonsai Image is a text-to-image diffusion pipeline shipped as three plain .tflite graphs (Qwen3 text encoder, DiT, VAE decoder) that you run directly with the LiteRT runtime (CompiledModel / Interpreter API), with your app driving the diffusion loop. The litert-community org hosts models for the LiteRT runtime in general, not only LiteRT-LM bundles — that's why it lives here.
If you want to run it on Android, there's a complete Kotlin demo app showing the whole pipeline (on-device Qwen3 tokenizer, the three graphs, the FlowMatch-Euler loop, memory-safe sequential loading): https://github.com/john-rocky/hf-to-litertlm/tree/main/bonsai_image_work/device/BonsaiAppAndroid — and the official samples repo has the Python host sample plus iOS/macOS apps under https://github.com/google-ai-edge/litert-samples/tree/main/samples/litert/image_generation. One honest caveat on speed: a 4-step 512×512 image takes about 7 minutes on a Pixel 8a (vs ~62 s on an iPhone 17 Pro), so on today's mid-range Android hardware treat it as a proof-of-run rather than an interactive experience.
ok. thanks for the very detailed explanation. I would really like to see how good it is on better phone like samsung s26 ultra.
I'll try the Kotlin demo and then try to convert it to Java.
Hi — just letting you know this model is now available to download in Box, an open-source, fully offline on-device AI app for Android (Apache-2.0, forked from Google's AI Edge Gallery). Box runs LiteRT / LiteRT-LM alongside llama.cpp, so .litertlm models run natively with GPU/NPU acceleration where the hardware supports it.
It appears in the in-app model browser with attribution and its original licence, and downloads directly from this repo — nothing is mirrored or re-hosted.
Thanks for publishing it. If you'd prefer it not be included, or want the description or credits changed, just say and I'll sort it.
@mekabe Thanks — and please do share what you see on the S26 Ultra; my Android reference device is a mid-range Pixel 8a, so I'd genuinely like flagship numbers. The pipeline portion is plain LiteRT runtime calls, so the Java port should be straightforward — if anything resists, open an issue on the demo repo and I'll help.
@jegly Thanks for the heads-up, and for asking first — inclusion is very welcome, and the attribution/licence handling you describe sounds exactly right. Glad to see the diffusion pipeline actually implemented in a Gallery-style app — as the thread above shows, this model isn't a .litertlm bundle and needs a host app to drive the three graphs, so Box giving it a real home is good news for users landing here. If anything about the graphs behaves oddly on some device, open a discussion and I'll take a look.
image quality is insane ! i get a img output in about 4 mins on my p10profold, Box has a 4x upscale built into it which then outputs a 2048x2048 image, i might build it in to the pipeline, thanks again for the great work! 😃
It's shipped: Bonsai went out in Box v3.3.3 as the recommended image generator, running the three graphs from the host app as you describe. Quality is genuinely the best on-device image gen we've had — better than the FLUX.2 klein and Z-Image pipelines we already ship.
Two things from getting it working that might be useful to you or anyone else landing here.
- All three graphs abort on the LiteRT GPU path (Tensor G5, Pixel 10 Pro Fold).
LiteRtCreateCompiledModel raises bad_any_cast inside libLiteRtClGlAccelerator.so, and because that library is built -fno-exceptions it aborts the process rather than returning an error — so no host-side try/catch can fall back to CPU. We had to make CPU the default rather than attempt GPU first.
textenc_int4 (1.8 GB): 79/2252 nodes delegated → abort
dit_gpu_int4b32 (2.27 GB): 2266/2266 nodes delegated → abort
The second one is the interesting case: full delegation, so it isn't unsupported ops. Tested and ruled out: sharing a single Environment, and passing Options(GPU, CPU) as an accelerator list — identical abort both ways.
The distinguishing factor against FLUX.2 klein and Z-Image, which both run fine on the GPU on the same device, looks like graph size: those feed the CL delegate chunks of ≤912 MB, whereas Bonsai's DiT is a single 2.27 GB graph. If a chunked DiT export were ever on the table it would likely unlock the GPU path.
CPU works well regardless — roughly 4 minutes for a 512×512 at 4 steps on a Tensor G5, ~2.9 GiB peak.
- Small interop note on the tokenizer. tokenizer/merges.txt here ships with a leading #version: 0.2 line. klein's pre-converted merges file doesn't have one, so an implementer reusing that loader silently shifts every merge rank by one and gets subtly corrupted tokenization with no error. Cost us a while to spot — might be worth a line in the model card.
you have some really great models ! alot of them are in Box ! check it out if you have not :)if you have suggestions let me know ! :D