NYU secures three-year backing for AI to reassemble archaeological fragments

NYU researchers and University of Tübingen collaborators have secured three-year support from Schmidt Sciences to develop AI that reconnects broken archaeological objects and suggests missing pieces. Selected for the funder’s October 14 Humanities and AI Virtual Institute awards, the project aims to turn unmatched stone tools, pottery and bones into evidence about past lives.

The bottleneck inside museum drawers

The project, called Fragmented Legacy, targets refitting: finding which fragments belong together and reassembling the original object. That work is slow enough that most archaeological collections are never systematically refitted, according to NYU. Radu Iovita, an NYU anthropologist on the team, describes a practical obstacle: museum researchers often see only one drawer at a time, rather than an entire collection.

A successful match can answer questions beyond an object’s appearance. A stone flake that fits a core elsewhere on a site can help reveal how people moved, how they made tools, or whether the site’s layers were later disturbed. Matching animal bones can illuminate butchery and meat sharing. Human fossil refits can expose important anatomical features.

The response grew from a collaboration between Chen Feng’s AI4CE Lab and Iovita’s lab that began about four years ago. Their earlier LUWA work used AI to classify microscopic wear on stone tools. Jing Zhang, then a postdoctoral researcher working across both labs, connected the groups.

In 2024, Zhang asked Iovita which technical problem archaeology most needed solved. He pointed to refitting. That led to GARF, which reassembles broken objects, and CRAG, which also fills in missing pieces. Those projects laid the groundwork for the new award. Feng and Zhang lead the effort, joined by Iovita, NYU anthropologist Scott Williams and Tübingen prehistory professor Raiko Krauß.

Broken archaeological material is one of the hardest versions of that problem. The pieces are worn, many are missing, and there's no picture on the box.

Chen Feng, NYU project lead

Teaching the models how materials break

The team’s first planned step is a collection of at least 100,000 real and digitally simulated broken objects. Each will pair three-dimensional models with photographs, descriptions and catalog data. The simulated breaks are intended to follow how stone, clay and bone actually fracture, rather than treating every material as the same geometric puzzle.

Next, the researchers plan to develop models that locate each fragment while generating a plausible version of the complete object. The systems will combine shape, images and text, allowing archaeological knowledge to rule out unlikely reconstructions. The goal is therefore more than finding edges that appear to fit: the proposed whole must also make archaeological sense.

Archaeologists will compare competing reconstructions and teach the system which ones hold up. That review is part of the planned development process, not a step left until after the models are finished. Feng describes the division of labor as building AI that reasons through uncertainty while archaeological colleagues judge whether its answers make sense.

An open release is the next commitment

The team plans to release all project datasets, model weights—the learned parameters behind the models—and benchmarks under permissive open-source licenses. The intended users include university researchers and cultural resource management firms, which conduct much of the archaeological work required before construction in the United States and abroad.

For now, the commitment is a funded development program: building the collection, developing the reconstruction models and evaluating their suggestions with archaeologists. Zhang describes the intended role as an assistant that matches fragments and proposes possibilities, leaving researchers more time to interpret evidence and ask new questions.