Georgian is a four million speaker language with its own alphabet, seven cases, and a verb system that makes linguists sweat. It is also a language the AI industry has never bothered to measure. The big multilingual benchmarks skip it, and the one standard translation test that includes Georgian has a known problem: its test sentences are public, and researchers keep finding them inside model training data.
So we spent a week building the measurement ourselves. 230 original test items nobody could have memorized, nine skill areas, eight AI models plus Google Translate, grammar answers verified against printed reference grammars and the Georgian National Corpus, and every output published. It is called the Georgian AI Benchmark, it is free, and it will re-run every time the model companies ship something new.
Here is what we found.
The verdict in one chart
If you use AI for Georgian, use Gemini or Claude’s top model. Nothing else is close on the hard stuff. Three clearly separated groups came out of the data. At the top, Gemini 3.1 Pro and Claude Fable 5 tied outright, with Gemini 3.5 Flash close enough to call even. In the middle, Claude Opus 4.8, GPT-5.4, Grok 4.3, and GPT-5.5 form a pack where the differences are statistical noise. And then there is a cliff, with the budget tier far below.
Two surprises inside that. First, Gemini 3.5 Flash is a budget-priced model, and it beat every OpenAI and xAI flagship on Georgian. Which company you pick matters more than how much you pay. Second, this benchmark was built and operated by Claude, and we published the methodology, the bias checks, and every judgment precisely so you do not have to take a tied first place on faith. The skeptical reader can re-run our scoring from the raw files.
Reading Georgian is nearly solved. Writing it is not.
Every model we tested, even the weakest, scored above 96 out of 100 on translation. If you want AI to tell you what a Georgian news article or a text from your mother-in-law says, any frontier model will do it well. That part of the story is over, and we report it honestly rather than pretending otherwise.
Producing Georgian is a different world. Ask models to conjugate verbs and the field spreads from 97.5 down to 57.5. The failures are the exact traps every learner knows. Nominative subjects where transitive verbs in the aorist demand გიორგიმ (giorgim). მაქვს (makvs) for owning a cat, when cats need მყავს (mq’avs). And the official romanization is a disaster area: most models silently drop the apostrophes that mark ejective consonants, which is why AI-generated Georgian learning content so often teaches pronunciation no Georgian would recognize. Only two systems scored 100 there.
The part that should worry app users
Most language tools route their AI features to small, cheap models. In Georgian, the budget model we tested scored 57.5 on verb morphology, 35 on romanization, and confidently invented a grammar fact, twice. If your study tool lets you pick the model behind it, pick deliberately.
Nobody has actually solved Georgian
The scores above measure everyday Georgian. So we built a second tier from the hard tail of the language: fifty rare-morphology tasks pulled from real Georgian text. Pluperfects. Inverted constructions. The old plural. On those, the best model on Earth scores 90, and most flagships collapse to between 36 and 60. That is the mountain future models will be measured against, and we will re-run the whole benchmark when they arrive.
What you should actually do
- Reading Georgian: trust it. Any frontier model translates Georgian reliably now, colloquial speech included.
- Writing Georgian: verify it. Case endings and verb forms still go wrong, more the further you get from the top tier.
- Romanized Georgian: distrust it by default. Most models drop the apostrophes that carry real pronunciation.
- Cheap AI tiers: not for Georgian. The gap between the best and worst model here is the gap between a competent assistant and a confident beginner.
Everything is published on the Georgian AI Benchmark page: the full scoreboard with confidence intervals, per-dimension charts, every error example, the complete methodology including how we handled the awkward fact that our own operator tied for first, and the whole dataset as a free download. Its sibling report, The State of Georgian Learning 2026, covers the human side of the story.
And if you are Georgian and spot an error in our reference answers, we genuinely want to hear it: [email protected]. Native speakers outrank every model we tested.
