The AI startup humans& has introduced a socially intelligent application called Boba and an open-source tool to accelerate the speed of text generation in large language models. Boba, which has already exchanged millions of messages, is built on the company’s collaborator AI model. The new FTL decode pipeline increases text generation speed by up to 10% by utilizing higher precision, a move the company says is counterintuitive to standard precision reductions.
The underlying Ampersand model contains over 700B parameters and was trained using a proprietary simulation tool, Persimmon, to help groups pursue shared goals. humans& has also contributed its large-scale reinforcement learning and serving work to the Miles, SGLang, and FlashInfer projects to enable other developers to build on the technology.
