By continuing, you agree · when terms of use bind a crawler

19 August 20267 minute readLaw cited as of 19 August 2026

Two articles in, the pattern is clear: the statutes everyone worries about keep not applying. The instrument that keeps deciding cases is the one nobody assigns an engineer to, because it is not a header, a file or a status code. It is a page of prose your crawler has never opened.

We meet this in an unglamorous place. UnGovr files public records requests by email, each carrying its own reply address so the correspondence threads. Some agencies offer no email at all, only a vendor's web portal, and we do not use it: we go looking for an address, and failing that we ask them for one, saying plainly that we do not submit through portals. That costs us time on exactly the bodies that are hardest to reach.

It is worth the time for reasons this article is about. Filling in that form would hand the correspondence to a vendor, create an identity for us inside their system, and require accepting their terms, which is the one act in the whole process that would make us a party to anything. Nobody would experience it as a legal decision. It is a checkbox on a page a person is trying to get past.

The case that teaches the whole lesson

hiQ v. LinkedIn is cited constantly for a proposition it does support: in 2022 the Ninth Circuit held that scraping public profiles is not access "without authorization" under the Computer Fraud and Abuse Act. hiQ won that argument. It is cited far less often for how the case actually ended.

In November 2022 the district court granted LinkedIn summary judgment on breach of contract. hiQ had a corporate account, so it had expressly agreed to the User Agreement, and it had breached that agreement by scraping and by paying crowdsourced workers to create false profiles. In December 2022 the parties filed a stipulated consent judgment: 500,000 dollars, plus a permanent injunction requiring hiQ to stop scraping and to destroy the source code, data and algorithms it had built.

So the company that won the landmark computer-misuse ruling was nonetheless enjoined out of the business, on a contract. That is the whole article in one case, and it is why the rest of this piece is about formation.

What forms a contract with something that cannot read?

Consumer contract law splits on how the terms were presented. Clickwrap, where the user actively agrees, is routinely enforced. Browsewrap, where terms sit behind a link and assent is inferred from use, is enforced only where notice was conspicuous enough that a reasonable user would know. A crawler is neither user. It does not read the footer, it does not click, and it forms no belief about anything.

Courts have mostly answered this by looking past the software to the operator, and the answers sort into three shapes.

• An account is assent, and it is the strongest shape by far

  Where the operator registered, agreed, and then crawled, the analysis is ordinary contract law and the crawler's illiteracy is irrelevant. This is what made LinkedIn's claim straightforward. If any part of your pipeline authenticates, your terms question is settled and the answer is yes.

• Repeated access with knowledge can bind an anonymous crawler

  In Register.com v. Verio (2004) the Second Circuit held that an operator who queried a service repeatedly, and who by then knew what its terms said, could be bound by them even though the terms arrived with the response rather than before the request. Knowledge from prior dealings does the work that a click would otherwise do.

• A single anonymous fetch of a public page is the weakest shape

  Ticketmaster v. Tickets.com is the early marker of judicial scepticism here, and modern browsewrap decisions turn on how conspicuous the notice was. There is no general rule, and anyone who offers you one is overstating. It is fact-specific and it varies by circuit.

The engineering consequence is unglamorous and worth stating plainly. The moment a crawl acquires an account, a session, or a history of being told what the terms are, it moves up this list. Nothing about the fetch changes. The legal position changes anyway.

Where intellectual property stops, contract carries on

The instinct that a page with no copyright in it is free to take is wrong in a specific and well-documented way. In Ryanair v. PR Aviation (Case C-30/14, 2015) the EU's Court of Justice held that where a database enjoys neither copyright nor the sui generis database right, the Database Directive's mandatory user protections simply do not apply to it, so nothing in that directive stops the owner restricting use by contract instead.

That is a genuinely counterintuitive result: the weaker the owner's intellectual property, the freer their hand to write restrictive terms. Public sector data, factual compilations and thin databases are exactly the material this catches, which is why it matters to anyone crawling government pages.

The two places contract cannot reach

Contract is not unlimited, and where a legislature has decided to limit it, the limit is usually explicit and narrow.

In the EU, Article 7(1) of the copyright directive makes any contractual term contrary to the exceptions in Articles 3, 5 and 6 unenforceable. Article 3 is text and data mining by research organisations, so a research exception cannot be written away. Article 4, the general text and data mining exception that commercial crawlers rely on, is not in that list. It was left out deliberately, and the reservation mechanism in Article 4(3) is the whole point of leaving it out. Commercial mining is contractually defeasible in the EU, and research mining is not.

Singapore went the other way. Its computational data analysis exception in the Copyright Act 2021 covers commercial as well as non-commercial use, and section 187 makes any contractual term excluding or restricting it void. That is a real design difference rather than a drafting accident, and it is the clearest example of a jurisdiction deciding that mining rights should not be negotiable.

What breach is worth in 2026

The live test of how far contract reaches is Reddit v. Anthropic, and its most useful moment so far is procedural. In March 2026 Judge Trina L. Thompson of the Northern District of California remanded the case to California state court, holding that Reddit's claims for breach of contract, unjust enrichment, trespass to chattels, tortious interference and unfair competition are not preempted by the Copyright Act.

Anthropic allegedly misappropriated Reddit's content and data for its own use, in violation of contractual rights under Reddit's User Agreement, which are distinct from the rights granted by copyright law.

Reddit v. Anthropic, N.D. Cal., March 2026

Preemption is the doctrine that would have folded these claims into copyright and disposed of them there. Holding that it does not apply keeps contract alive as an independent route, which is exactly the structure that beat hiQ. As of August 2026 the case is proceeding in state court.

What to do about it

If you crawl, read the terms of the sites you crawl at volume, before you crawl them rather than after a letter arrives. Keep authenticated access strictly separate from bulk collection, because mixing them converts the hardest formation question into the easiest one. And treat a crawl that has been individually put on notice as a different legal object from the same crawl the week before.

If you publish, terms are the cheapest instrument on the shelf and the one most likely to be enforced. They cost no visitors, no accessibility and no search traffic, which is more than can be said for any rung of the barrier ladder. Make them conspicuous, say specifically what automated collection is and is not permitted, and understand that in the EU they will not touch research mining and will not, on their own, satisfy Article 4(3).

Which raises the obvious question: if prose in your terms does not reserve your rights, what does? That is the next article. Meanwhile the Law pages carry the per-jurisdiction posture we hold today, and LexLint itself will read your app's declared profile and report which of these attach to it.

Sources and further reading

• hiQ v. LinkedIn wrapped up: web scraping lessons learned The November 2022 contract ruling and the December 2022 consent judgment, set out together. The two halves are usually reported apart, which is how the case acquired its misleading reputation.
• hiQ Labs v. LinkedIn, 9th Cir. (2022) The computer-misuse holding hiQ won, and the one everyone remembers.
• Register.com v. Verio, 2d Cir. (2004) How repeated access plus knowledge of the terms can bind an automated user with no click anywhere in the story.
• Ryanair v. PR Aviation, Case C-30/14 The Court of Justice on unprotected databases: no copyright, no database right, and therefore no statutory limit on restricting use by contract.
• Directive (EU) 2019/790, Articles 3, 4 and 7 Read Article 7(1) against Article 4 directly. Which exceptions are contract-proof, and which one conspicuously is not, is decided by a list of three numbers.
• Singapore Copyright Act 2021 Sections 243 to 245 on computational data analysis, and section 187, which voids contractual terms excluding it.
• Reddit v. Anthropic: state contract and tort claims not preempted The March 2026 remand, and why keeping these claims outside copyright matters more than where the case is heard.
• Consent in Crisis: The Rapid Decline of the AI Data Commons Measures how often a site's robots.txt and its terms of service contradict each other across about 14,000 domains. If you only read one of the two, you are reading the wrong one about a third of the time.

This article is a research summary of published law, with every source linked above. It is not legal advice, it creates no attorney-client relationship, and nothing here is a substitute for reading the terms of a site you intend to crawl. Law cited as of 19 August 2026; Reddit v. Anthropic is live litigation and will move.