Recursive self-improvement is officially making its way into open-source artificial intelligence. Together AI, an Nvidia-backed startup specializing in model inference, has introduced a groundbreaking compiler designed to automate GPU tuning for freshly launched open-source models.

This new compiler serves as the inaugural project within a broader initiative dubbed Polymorphic Inference, which establishes a framework for recursive self-improvement inside inference workflows. Through this system, inference operations are built to autonomously evolve, utilizing AI agents to continuously refine the infrastructure executing open-source models.

The rollout arrives amid persistent hardware scarcity. Recent findings from technology research firm Information Service Group (ISG) indicate that two-thirds of enterprise organizations anticipate moderate to severe deficits in accelerated computing capacity, particularly regarding GPUs.

Mitigating the Severe GPU Shortage

Traditionally, whenever developers unveil new open-source models, engineering teams invest anywhere from several days to multiple weeks manually optimizing code prior to production deployment.

Together AI—fresh off securing an $800 million Series C funding round that pegs its valuation at $8.3 billion—intends to eliminate this bottleneck. Its newly developed compiler targets Nvidia and AMD processors, deploying AI agents to write accelerated, highly efficient GPU code. Consequently, rather than forcing human teams to spend weeks preparing an architecture, emerging models can be deployed on their official release day.

During a video interview, Vipul Ved Prakash, cofounder and CEO of Together AI, explained that open models have matured significantly over recent years, effectively “closed the gap in terms of quality with the frontier labs.”

Simultaneously, Prakash noted a surge in both the overall volume of open-source models and their release frequency. Major laboratories now push updates roughly every six weeks, bringing continuous enhancements to model weights and structural architectures.

Prakash highlighted that since last year, the Together team has engineered “self-optimizing systems that can adapt to model architectures,” along with new hardware variants to achieve “frontier performance.” This falls under their “polymorphic inference” strategy, tailoring technology stacks for diverse classes of Nvidia and AMD chips.

“I think you know we are sort of headed into this, you know, shortage of GPUs and and power, and the being able to maximize the value that you can extract from GPUs that are already in the ground is like becoming more important than ever,” Prakash stated.

Initial benchmarks for Together’s compiler have yielded remarkable outcomes. The startup asserts that its tool has surpassed hand-tuned megakernels, delivering speeds up to 1.3 times faster than manual alternatives on the Llama 1B model, securing top performance metrics across every evaluated output length on both Nvidia H100 and B200 hardware.

When handling the MiniMax M3 model, Together leveraged parallel agents to synthesize optimal strategies from multiple concurrent workers, rendering a MiniMax M3 layer 1.5 times faster. In a similar vein, deployed agents overhauled how Kimi K3 processes lengthy requests, accelerating throughput by a factor of 1.7 to 2.5.

Simran Arora, SVP of Research for AI Systems at Together AI, shared in an interview that the enterprise has spent roughly two years developing these capabilities across various iterations, with a dedicated squad of five engineers focusing squarely on the compiler. She pointed out that recent recursive self-improvement inference systems introduced over the past month have achieved an astonishing 250-fold boost in throughput.

“The fundamental language that has propelled most of AI is PyTorch, which was released by Meta 10 years ago, and it was really designed for a world with a lot of modularity of software components, really easy to read. But given how many people demand tokens right now, we really, really need to get much more utilization out of the chips that we use,” Arora remarked.

Arora detailed how Together initially partnered with Stanford to forge a fresh set of GPU programming primitives named Thunder Kittens. This foundation empowered the firm to craft novel AI primitives directly at the raw GPU level, extracting maximum computational utility. The group subsequently began stringing these primitives together into mega kernels—large-scale streams and SPU programs capable of executing numerous kernels simultaneously. Because hand-coding mega kernels historically demands multiple months of labor, the company pivoted toward automated generation.

Optimizing Every Layer of the Tech Stack

As software engineering utilities grow increasingly sophisticated, developers encounter expanded avenues for automated self-enhancement. Ori Goshen, cofounder and CEO of AI21 Labs, emphasized via email that Together’s latest release mirrors a broader paradigm shift across the tech sector, where “AI is now improving the systems that run AI.”

“Every AI workload runs on three layers: the harness, the model weights and the inference engine. Each one calls for automated optimization, and each potentially delivers significant gains,” Goshen noted. From this vantage point, Together’s technology automates the inference layer to render models faster and more cost-effective to deploy.

As AI21 Labs and peer organizations work to automate harnesses and model weights alongside inference engines, Goshen predicts that “over time each layer will recursively improve itself, and eventually the layers will improve jointly, since a gain in one shapes what’s possible in the others.”

Human-driven software development is no longer the ultimate constraint; autonomous AI agents have unlocked continuous evolution and automated fine-tuning. Illustrating this trend, AI21 recently unveiled an initiative training Nvidia’s Nemotron architecture, slashing operational costs to roughly 50 times cheaper than comparable frontier models while retaining the vast majority of its baseline quality.