Around this time the majority of compute at Stability AI was being used on building open LLMs but: 1. We didn’t do the easy thing of training GPT-J / Neo-X more (pre chinchilla!) 2. Couldn’t make it safe / tried to do too many changes to formula (Pile 2, Reddit crawls etc) We succeeded in buildi
@emostaque·Jul 20, 2026·2 sources·neutral
Read articleAI Summary
A reflection on Stability AI's compute allocation challenges in building open LLMs, noting that Mistral and Meta later led the wave.
Related Projects
All Sources
Around this time the majority of compute at Stability AI was being used on building open LLMs but: 1. We didn’t do the easy thing of training GPT-J / Neo-X more (pre chinchilla!) 2. Couldn’t make it safe / tried to do too many changes to formula (Pile 2, Reddit crawls etc) We succeeded in buildi
@emostaque
Jul 20, 2026
Around this time the majority of compute at Stability AI was being used on building open LLMs but: 1. We didn’t do the easy thing of training GPT-J / Neo-X more (pre chinchilla!) 2. Couldn’t make it safe / tried to do too many changes to formula (Pile 2, Reddit crawls etc) We succeeded in buildi
@emostaque
Jul 20, 2026