In the context of large language models, a token is the basic unit of text the model processes — roughly a word fragment, word, or punctuation mark — used to measure both input length and output length, with practical limits imposed by the model's context window.
Last reviewed: 2026/05/19
The context window is the maximum amount of text — measured in tokens — that a large language model can process at one time, determining how much document content, conversation history, and instructions the model can consider when generating a response.
Tech / ModelIn AI, inference is the process of running a trained model to generate outputs from new inputs — as distinct from training, which creates the model. Every time a lawyer submits a query to a legal AI tool, inference occurs.
Tech / ModelA large language model (LLM) is an AI system trained on large volumes of text data to predict and generate human-like text; it serves as the core engine underlying most legal AI tools for research, drafting, and document analysis.
The most expensive legal AI in the market — Am Law 100 firms only.
Thomson Reuters' GPT-backed legal research and drafting with Westlaw integration (relaunched as CoCounsel Legal, 2025).
AI-powered legal research with citation-validated answers from Westlaw.
Conversational legal research with real-time Shepard's citation validation.
AI contract drafting and review inside Microsoft Word for transactional lawyers.
Move from this definition to role-based legal AI shortlists and the selection criteria that matter for each type of legal team.
Am Law 200 and global firm workflows: accuracy at scale, security compliance, and matter-level auditability.
Reviews built for 2–20 attorney firms: collaborative workflows, mid-range budgets, limited IT overhead.
60 legal AI tools vetted for the solo lawyer: tight budget, no IT team, billable-hours pressure.
Last reviewed: 2026/05/19. Definitions are written by the LawyerAI Editorial team. Commercial relationships are disclosed and do not determine editorial scores or conclusions. See our Sponsorship & Affiliate Disclosure.
In the context of large language models, a token is the basic unit of text the model processes — roughly a word fragment, word, or punctuation mark — used to measure both input length and output length, with practical limits imposed by the model's context window.
Tokens define the practical limits of what a legal AI tool can process in a single session. A typical English word is approximately 1.3 tokens. A 100-page contract is roughly 25,000–35,000 tokens. A model with a 128,000-token context window can process the entire contract in one pass; a model with a 4,000-token limit would need the document to be chunked into sections, potentially losing context across chunks.
For lawyers using AI tools on large documents — lengthy contracts, voluminous discovery productions, multi-volume deposition transcripts — the token capacity of the underlying model directly affects whether the tool can process the entire document or must operate on fragments. Processing fragments can cause the AI to miss connections across document sections, such as a defined term introduced in one section that modifies an obligation in another.
Token limits also affect cost: most AI APIs charge by the number of tokens processed. Tools that maintain conversation history accumulate tokens with each exchange, eventually hitting context limits that force the session to restart.
Understanding token constraints helps lawyers evaluate whether a tool is suitable for large-document tasks and whether a tool's behavior on long documents (such as apparent failure to "remember" earlier document content) is a context limit issue.
Legal AI tools manage token limits through different approaches. Tools designed for document-length analysis use chunking strategies — dividing large documents into overlapping segments for processing — with the trade-off that cross-segment context can be lost. More capable tools use longer context models to reduce the need for chunking.
Recent advances in context window size have significantly expanded what's practical. Models with context windows of 128,000 or 200,000 tokens can process substantial legal documents in full — a 200-page acquisition agreement, a multi-day deposition transcript — without chunking.
Harvey AI and similar enterprise legal tools typically build on models with large context windows specifically to address the document-length requirements of legal work. Tool documentation or vendor materials should specify the effective context limit for document analysis tasks.
For e-discovery workflows involving very large document collections, individual document-level token limits are less relevant than the platform's batch processing architecture, which handles collection-level analysis outside a single context window.