Document Processing · Enterprise Knowledge Search
Wordsmith needed to observe, evaluate, and continuously optimize increasingly complex legal-assistant workflows that spanned multiple data sources, model providers, and experiment branches without slowing development or losing production visibility.
Reported impact
Wordsmith can compare new models in under an hour, debug production problems in seconds instead of minutes, and cut costs on some tasks by up to 10x while keeping legal-assistant quality grounded in repeatable evaluations.
Wordsmith began with a configurable RAG pipeline for Slack and expanded into multi-stage inference chains over Slack messages, Zendesk tickets, pull requests, and legal documents. LangSmith became the control plane for this stack: hierarchical traces show what each nested step sends to and receives from the model, letting engineers debug bad intermediate behavior such as an invalid Dynamo query without trawling general logs. LangSmith datasets provide reproducible evaluation sets for RAG, agentic workloads, attribute extraction, and XML changeset targeting, so the team can compare models like Claude 3.5 and GPT-4o quickly and keep accuracy stable while pushing down cost and latency. Statsig experiment exposures are written into LangSmith tags, which makes online experiments queryable, exportable, and analyzable alongside development and production traces.
Messages and documents from Slack, Zendesk, pull requests, and legal files enter Wordsmith’s inference chains, LangSmith records each nested step and evaluates datasets against model or prompt changes, and Statsig experiment tags let the team export and compare production cohorts for downstream analysis.
These workflows can contain up to 100 nested inferences
Wordsmith’s first feature was a configurable RAG pipeline for Slack.
compare its performance to GPT-4o within an hour
reduced cost on particular tasks by up to 10x
reducing time to debug an inference from minutes to seconds
Review how this reference was scored and trace each normalized claim back to its documented source.
Scoring methodology
Customer Reference Value measures usefulness as a customer reference. Evidence Confidence measures the quality and traceability of available evidence, not independent verification.
Traceability
Field coverage shows which facts each source supports. Dates are shown only when documented by the source.
The homepage JSON-LD lists addressLocality London, postalCode EC2A 4NE, and addressCountry GB.
The LinkedIn Organization schema lists numberOfEmployees value 204.
Related
MongoDB Sample Sep 9, 2026 Squary AI Squary AI needed to eliminate slow, outage-prone legal search infrastructure that combined MongoDB with OpenSearch and hurt document review speed, cost, and trust. At Quatrini Law Group, Squary AI analyzed more than 460,000 pages and saved over 400 attorney and paralegal hours. Amazon Bedrock · Claude · MongoDB Atlas 
In-house Sample Sep 1, 2026 Thomson Reuters General-purpose LLMs can produce subtle but consequential errors in legal and compliance work, where professionals need trustworthy, citation-backed analysis across very large document sets. As of Q1 2026, CoCounsel has crossed one million users CoCounsel · Practical Law · Tabular Analysis 
OpenAI Sample Aug 20, 2026 Stampli Stampli needed to launch complex new products and keep product knowledge current without waiting on scarce design, contractor, and operations bandwidth. 243 modeled active role-hours compressed to about 77 for the Deep Finance launch ChatGPT Work · Codex · Deep Finance 
