The company unveiled Together Compiler on Wednesday, a system that generates, tests, and improves GPU kernels without an engineer directing each experiment, and Together AI says it can deliver gains overnight. The process runs without a person steering each step, but it doesn't ship anything on its own.
Asked how much people remain in the loop, Together AI tells The New Stack, "The optimization loop is autonomous," adding that "the final evaluations and deployment process are automatic but must be kicked off by a human, who reviews the changes before deploying to production." The final evaluations and deployment process … must be kicked off by a human, who reviews the changes before deploying to production." Why GPU kernels fall behind New AI models arrive so frequently that keeping inference infrastructure optimized has become a challenge.
