REALBITS/MOSS-TTS-Nano-100M-ONNX-fp16

MOSS-TTS-Nano-100M — FP16 (shared external data)

Half-precision-quantized version of OpenMOSS-Team/MOSS-TTS-Nano-100M-ONNX, re-exported with shared external weight data so prefill and decode_step mmap the same 220 MB blob instead of duplicating weights. Tuned for ARM CPUs with native fp16 SIMD (Snapdragon 8 Gen 2/3, Apple A14+, etc).

Audio codec is unchanged from upstream. Use this repo for the language model graphs and pull the codec from OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano-ONNX.

What's here

File	Size	Purpose
moss_tts_prefill.onnx	476 KB	global LM, full-context prefill
moss_tts_decode_step.onnx	503 KB	global LM, autoregressive single step (KV cache)
moss_tts_local_fixed_sampled_frame.onnx	819 KB	local LM with baked-in sampling
moss_tts_global_shared_fp16.data	220 MB	fp16 weights shared by prefill + decode_step
moss_tts_local_fixed_sampled_frame_fp16.data	113 MB	fp16 weights for the local sampler
tokenizer.model	471 KB	SentencePiece (unchanged from upstream)
browser_poc_manifest.json	503 KB	prompt templates + 18 builtin voices
tts_browser_onnx_meta.json	4 KB	I/O metadata
Total	~336 MB	(vs 640 MB upstream fp32 LM, 196 MB int8)

Variant comparison (this repo's role)

Variant	LM size	Quality vs FP32	Use when
FP32 (upstream)	640 MB	reference	offline rendering, high-end devices
FP16 (this repo)	336 MB	indistinguishable from FP32	mobile devices with native fp16
INT8 (sister repo)	196 MB	~3 dB envelope shift, slightly more monotone delivery	size-constrained, fastest

Triple A/B objective analysis (Adam voice, same long demo, seed=42):

Metric	FP32	FP16	INT8
RMS dBFS	-20.79	-20.49	-20.18
F0 mean (Hz)	110.0	110.7	114.0
F0 std (Hz)	32.7	35.1	21.9 ← more monotone
Voice-band envelope MAE vs FP32	0 dB	9.7 dB ← content-length variance	13.4 dB
Silent-region noise floor RMS	0.0014	0.0016	0.0017

Notes on fp16 numerics

The conversion uses onnxruntime.transformers.float16.convert_float_to_float16(keep_io_types=True) with these ops kept at fp32 (precision-sensitive):

• LayerNormalization
• GroupNormalization
• Softmax
• Cast

This preserves the standard "fp16 forward pass with fp32 norm/softmax" recipe that's robust across most transformer architectures. The audio output is visually indistinguishable from fp32 in spectral envelope (within the noise floor of run-to-run sampling variance — the model has stochastic decoding).

Speed expectations

Hardware	FP16 RTF	Notes
x86 desktop CPU (no AVX-512 BF16)	~0.75 (slower than fp32 0.38)	ORT-CPU has no fp16 GEMM; expands to fp32 + downconverts
Snapdragon 8 Gen 2/3 ARM cores	TBD	Native fp16 SIMD; expect close to int8
Apple A14+	TBD	Native fp16 SIMD
ARM Cortex-X1 / older	varies	Less native support

If you're targeting x86 CPU, use the int8 variant instead — fp16 is a loss there.

Quick test (Python ORT)

from onnxruntime import InferenceSession, SessionOptions
from huggingface_hub import snapshot_download

lm_dir = snapshot_download("REALBITS/MOSS-TTS-Nano-100M-ONNX-fp16")
codec_dir = snapshot_download(
    "OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano-ONNX",
    allow_patterns=["moss_audio_tokenizer_decode_*", "*.json"],
)
# Drive prefill → loop {sampler → decode_step} → codec.decode_full
# (or codec.decode_step for streaming) per upstream onnx_tts_runtime.py.

On-device example

This repo is a v2 candidate for the Fictures MOSS-TTS-Nano Android prototype. Switch to it via:

adb shell setprop debug.mosstts.variant FP16

then restart the app and tap Download.

How it was quantized

Repro: see _dryrun/13_quantize_fp16.py. Three steps:

1. onnx.load(load_external_data=True) for each LM graph
2. convert_float_to_float16(keep_io_types=True, op_block_list=[LayerNorm, GroupNorm, Softmax, Cast])
3. Save with save_as_external_data=True; hash the resulting .data files; collapse byte-identical pairs (prefill + decode_step share the underlying transformer weights) into one shared moss_tts_global_shared_fp16.data and rewire both graphs' external_data location attributes.

License + attribution

Apache 2.0, inherited from upstream OpenMOSS/MOSS-TTS-Nano.

Known limitations

Same as the upstream and int8 variant: English / Chinese / Japanese only; no microphone voice cloning out of the box (use the 18 builtin voices in browser_poc_manifest.json); codec stays fp32.