Loading page…
The result puts usability and practical application alongside benchmark results as the government decides which domestic models receive more compute and a shot at final selection.
Listen to this story
South Korea’s AI foundation-model program is rewarding broader readiness over benchmark leadership: Motif Technologies’ Motif 3 led Artificial Analysis’ Intelligence Index by more than 10 points over Upstage, but was cut after weaker usability and deployment evaluations. LG AI Research, SK Telecom and Upstage advance to a third review expected early next year, with two winners planned. All four models were judged technologically independent.
Benchmark results counted for 40% of the second-stage score; experts contributed 35% and users 25%.
Motif 3 was rated world-class technically but weaker on usability and practical application.
The field began with five teams; Naver Cloud and NC AI exited before Motif joined through an additional competition.
South Korea’s national effort to build an independent AI foundation model is down to LG AI Research, SK Telecom and Upstage. Motif Technologies was eliminated even though it ranked first in the reported benchmark evaluation, making the contest’s broader selection criteria central to who gets the next round of state support.
Motif 3 scored more than 10 points ahead of Upstage on Artificial Analysis’ Intelligence Index, according to a background assessment. But Second Vice Minister Ryu Je Myung said Motif received lower evaluations than the other models for usability and practical application, despite demonstrating a world-class technological level.
Benchmark performance carried 40 points in the second-stage evaluation.
Expert evaluations carried 35 points.
User evaluations carried 25 points.
The second-stage score assigned 40 points to benchmarks, 35 to expert evaluations and 25 to user evaluations. Ryu said different teams led each category. The ministry also described this round as more sophisticated than the first, adding difficult international benchmarks, public usability testing and a broader assessment of potential industrial AI deployment.
The project is intended to reduce reliance on foreign AI models, and the ministry determined that all four second-round models had achieved technological independence. The competition began with five teams; Naver Cloud and NC AI were eliminated in the first evaluation, and Motif joined through an additional competition before this second round.
The surviving teams will keep developing their models until a third evaluation early next year, when two teams are expected to be selected. The ministry plans to raise participating teams’ Nvidia B200 GPU support from about 768 to roughly 1,000, is reviewing a new support system, and has discussed with participants whether the program’s scale and competitive structure can keep pace with frontier-model progress.
Editorial analysis
South Korea’s tournament design is testing a difficult policy choice: whether a national model program should reward the strongest measured model, the most usable model, or the team best positioned to build an ecosystem. Motif’s exit makes that trade-off concrete. The next evaluation will show whether larger GPU allocations help LG AI Research, SK Telecom and Upstage close performance gaps while improving deployment prospects. The ministry’s review of the program’s scale is equally important: a contest can concentrate resources, but only if its structure remains credible to the teams it asks to compete.
Citation desk / original work
South Korea’s tournament design is testing a difficult policy choice: whether a national model program should reward the strongest measured model, the most usable model, or the team best positioned to build an ecosystem.
/posts/south-korea-cuts-ai-model-contest-to-3-eliminating-benchmark-leader-motif#finding-1
Loading discussion...
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.