Join our each day and weekly newsletters to receive the latest updates and exclusive content on industry-leading AI coverage. Learn more
Google continues aggressive Gemini updates on the option to the 2.0 model.
The company today announced a smaller version of the Gemini 1.5, the Gemini 1.5 Flash-8B, alongside the “significantly improved” Gemini 1.5 Flash and the “more powerful” Gemini 1.5 Pro. They show improved performance in a variety of internal benchmarks, the company says, with “huge gains” for the 1.5 Flash across the board and the 1.5 Pro significantly higher at math, coding, and complex commands.
“Gemini 1.5 Flash is currently the best… experience in the world for developers,” boasted Logan Kilpatrick, product manager at Google AI Studio, in a post on X.
‘Latest experimental iteration’ of ‘unprecedented’ Gemini models
Google has introduced Gemini 1.5 Flash — light version Gemini 1.5 — in May. The Gemini 1.5 family of models is built to handle long contexts and can reason from detailed information from 10 million or more tokens. This allows the models to process large amounts of multimodal input, including documents, video, and audio.
Today, Google is rolling out an “improved version” of the smaller Gemini 1.5 Flash variant with 8 billion parameters. Meanwhile, the latest Gemini 1.5 Pro shows performance gains in encoding and complex prompts and serves as a “drop-in replacement” for the previous model released in early August.
Kilpatrick didn’t provide many details, saying Google will release a future production-ready version in the coming weeks that can “hopefully include evaluation features!”
He explained in the X thread that experimental models are a option to gather feedback and provide developers with the latest, live updates as quickly as possible. “What we learn from experimental runs informs how we release models more broadly,” he wrote.
The “latest experimental version” of Gemini 1.5 Flash and Pro comes with a limit of 1 million tokens and could be tested for free via Google AI Studio AND API Geminiand soon through an experimental Vertex AI endpoint. There is a free tier for each, and the company will release a future version for production use in the coming weeks, in accordance with Kilpatrick.
Starting Sept. 3, Google will routinely redirect requests to the latest model and remove the older model from Google AI Studio and the API to “avoid the confusion of having too many versions active at once,” Kilpatrick said.
“We’re excited to hear your feedback and learn how this model can enable even more new multimodal applications,” he wrote on X.
Google DeepMind researchers call the Gemini 1.5 scale “unprecedented”among contemporary LLMs.
“We’re blown away by the enthusiasm for our first experimental model we released earlier this month,” Kilpatrick wrote on X. “At Google, we’ve been putting in a ton of hard work behind the scenes to bring these models to the world. We can’t wait to see what you build!”
“Solid Improvements” Still Suffer from “Lazy Coding Disease”
Just a few hours after today’s premiere Organization of enormous model systems (LMSO) has released an update to the leaderboard for its chatbot arena based on 20,000 community votes. Gemini 1.5-Flash made a “huge leap,” rising from twenty third to sixth place, matching Llama and surpassing Google’s open Gemma models.
Gemini 1.5-Pro also showed “significant progress” in coding and mathematics and “improvement[d] much.”
LMSO praised the models, writing: “Huge congratulations to the Google DeepMind Gemini team for an incredible launch!”
As is normally the case with model releases, initial reviews ranged from glowing praise to derision and confusion.
Some X users wondered why so many updates were coming back to back and not 2.0. One wrote, “Dude, this is not enough anymore 😐 we need Gemini 2.0, a real update.”
On the other hand, many self-proclaimed fanboys praised the fast updates and fast shipping, reporting “solid improvements” in image evaluation. “The speed is amazing,” wrote one, while one other noted that Google continues to ship while OpenAI effectively stays silent. One even went so far as to say that “the Google team is quietly, diligently, and relentlessly shipping.”
Some critics, nevertheless, have called it “terrible” and “lazy” for tasks that require longer-term results, saying Google “lags far behind” Claude, OpenAI, and Anthropic.
As one X user lamented, the update “unfortunately suffers from lazy coding disease,” much like GPT-4 Turbo.
Another called the updated version “definitely not as good” and said that it “frequently goes crazy and starts repeating things over and over again, as small models tend to do.” Another agreed that they were excited to try it out, but that Gemini “was definitely the worst at coding.”
Some also mocked Google’s unconvincing naming capabilities and recalled its massive gaffe from earlier this 12 months.
“You guys have completely lost the ability to name things,” one user joked, while one other agreed: “You really need someone to help you with nomenclature.”
And another person asked a dry query: “Does Gemini 1.5 still hate white people?”
