Tencent publishes Hy4 Flash weights for developers to run and further train
The release includes standard, FP8 and base checkpoints. Its architecture distinguishes a roughly 220-billion-parameter model from the much smaller portion active for each token.
Tencent’s weight release gives developers a route to run Hy4 Flash outside Tencent’s hosted apps and API, and to continue training from its base checkpoint. The package includes standard and FP8 checkpoints, deployment materials under the Tencent Hy License, and official serving recipes for vLLM and SGLang. The model’s roughly 220-billion total parameters should not be confused with the roughly 12 billion active per token; another release report rounds the total to 224 billion.
01
Hy4 Flash handles text and images and supports a context window of up to one million tokens.
02
The three checkpoints are standard weights, an FP8 version, and a base model intended for further training.
03
Tencent describes DFly2, a 0.85-billion-parameter component, as a speculative-decoding drafter that shares attention resources with the main model.
Tencent’s Hy4 Flash is no longer limited to hosted access. Developers can now download its model weights—the trained numerical values used to run it—and deploy or further train the model themselves. The release includes three checkpoint versions, alongside deployment materials under the Tencent Hy License.
This is a new way to obtain an existing model, not Hy4 Flash’s first appearance. Tencent previously introduced it through its applications and cloud API. Publishing the weights follows an earlier commitment to make them available.
Three checkpoints, one model
Standard model weights.
An FP8 version.
A base model intended for further training.
Hy4 Flash accepts text and images and supports a context window of up to one million tokens. That window describes how much material the model can work with in a request; it is a different measure from the parameter counts describing the model itself.
Tencent positions the model for everyday productivity, including research, document preparation and spreadsheet operations. Those are intended uses, rather than an independent assessment of how reliably it completes the work.
The active count describes the portion used for each token, not the entire model.
A large model with selective computation
The language backbone has 42 layers: one dense feed-forward layer and 41 mixture-of-experts layers. The architecture therefore relies overwhelmingly on expert layers, rather than treating every layer as the same kind of processing block.
Beinsure’s release coverage gives a slightly different rounded total of approximately 224 billion parameters, while also reporting around 12 billion active per token. Tencent’s model card uses about 220 billion overall. Both descriptions distinguish the full parameter count from the smaller active portion.
A built-in drafter shares resources
Hy4 Flash also contains DFly2, a built-in component with 0.85 billion parameters. Tencent describes it as a drafter for speculative decoding: its role concerns drafting output during generation, separate from the model’s headline context limit and active-parameter count.
The DFly2 description shows how closely the drafter is connected to the main model. Tencent says each draft layer shares the corresponding sparse-attention layer’s KV cache—the stored attention information from earlier context. It also shares the indexer’s selection of up to 2,048 positions.
For production serving, Tencent recommends vLLM or SGLang and points developers to deployment recipes. These accompany the downloadable model, giving developers an official starting point for operating it beyond Tencent’s hosted services.
Reader comments
Newest comments first. Replies stay oldest first.