You can use K3 as a teacher model (with logits!) to train up smaller dense models. It’s actually feasible even to use it to create a trillion token full pre and post training dataset for even better models. https://x.com/teortaxestex/status/2078306041721245771

@emostaque·Jul 18, 2026·2 sources·neutral

AI Summary

K3 can be used as a teacher model to train smaller dense models and create a trillion token dataset.

All Sources

You can use K3 as a teacher model (with logits!) to train up smaller dense models. It’s actually feasible even to use it to create a trillion token full pre and post training dataset for even better models.