Meta adds website-crawling rights to business terms, raising AI-training concerns
The October 30 update offers no opt-out for businesses that keep using Meta services. Meta says website owners can still block crawling through robots.txt.
Listen to this story
The audio brief
Story brief
3 key pointsEffective October 30, 2026, Meta’s revised commercial terms let the company and its agents crawl business websites, cache and analyze pages, retrieve updates, and use the material across Meta products—including for training—rather than limiting use to advertising. Meta says it is consolidating existing language and will continue honoring robots.txt, but businesses cannot reject the agreement while remaining on its services; blocking crawlers is a separate option. New “test credentials” wording has raised concerns about paywall access, while whether the terms authorize AI training or competitive reuse remains legally unresolved.
- 01
The previous ad-related terms covered URLs supplied with ads and tied use to advertising or that agreement’s purposes.
- 02
Meta says news organizations can opt out of scraping and that it will not use the revised terms as a basis to ignore robots.txt.
- 03
Cornell professor James Grimmelmann said the language could permit using publishers’ content in products that compete with them; three legal experts said enforceability is unclear.
Meta’s new commercial terms reach beyond business accounts’ social posts to websites associated with their products and services. Taking effect October 30, 2026, the terms authorize crawling and using that content to train Meta products. They do not explicitly mention AI, but legal experts warn the language could give Meta permission to train its AI systems.
From advertising permissions to wider website access
The updated language authorizes Meta and entities acting on its behalf to collect content through automated tools, including scrapers and crawlers. It covers domains, pages and URLs associated with a business’s products or services, as well as those made available through its own or users’ use of Meta services.
The permission extends to indexing, caching and analyzing content, and periodically retrieving updates. Meta may use the material to provide, develop, improve, research and train its products, services and features, or maintain their integrity. The wording does not confine that use to creating advertisements.
That is a departure from the narrower advertising language Ars Technica identified in its comparison of the policies. The earlier Ad Creative Generative AI Terms permitted crawling URLs supplied in connection with ads, with the resulting use tied to advertising or that agreement’s purposes. The updated commercial terms instead describe uses across Meta products.
Meta says crawler blocks still apply
Meta describes the revision as streamlining rather than a new claim on publishers’ work. A spokesperson told Nieman Lab that crawling language had been scattered across several agreements, including terms for generative AI in ads, and was being consolidated. The company’s blog said the changes reduce overlap and clarify which terms apply.
The spokesperson also said Meta has no plans to use the terms as a legal basis to ignore robots.txt, the website instructions used to block crawling. Meta continues to let news organizations opt out of scraping and provides instructions for blocking its crawlers.
Another addition has drawn scrutiny: permission to create “test credentials.” Ars found that phrase absent from the previous commercial terms and the product-catalog and generative-ad agreements being combined with them. Legal experts speaking to Nieman Lab warned it could allow paywall access and automated scraping. Meta did not immediately answer Ars’s request to clarify the language.
this is a remarkable provision to bury in [the] terms of service, given that this would give Meta access to material that would likely cost many millions of dollars in a licensing market.
Matt Topic, media and intellectual property attorney at Loevy & Loevy, speaking to Nieman Lab, as quoted by Ars Technica
Publishers face an acceptance decision
Businesses cannot reject the updated terms while continuing to use Meta services, according to Ars. That makes accepting the agreement a separate issue from using technical controls to block a crawler: Meta says those controls remain available, but the commercial agreement itself offers no opt-out.
For at least one publisher, leaving is already under consideration. Stephen Beale, founder and editor of The Steampunk Explorer, told Nieman Lab he depends on Facebook for traffic but is considering quitting over the update. His objection concerns the permission Meta is demanding, even though he believes his site has already been heavily scraped.
The potential consequences extend beyond collecting training material. Cornell Law School professor James Grimmelmann told Nieman Lab that the terms could permit training on publishers’ website content and potentially deploying some of it in products that compete with those publishers.
Whether that interpretation would hold in court remains unresolved. Three legal experts consulted by Nieman Lab said enforceability was unclear, while warning that Meta could invoke the terms as permission for AI training. That is a possible legal argument, not a finding that the update overrides website restrictions.
Sources
- arstechnica.comMeta may use sweeping new commercial TOS to avoid millions in AI licensing fees
Reader comments
Newest comments first. Replies stay oldest first.