Customer Service Automation · IT Operations Automation
monday.com needed a reliable way to validate autonomous service agents before release because small prompt or tool-call changes could cascade into incorrect outcomes across IT, HR, and legal workflows.
Reported impact
monday.com shortened the evaluation loop for customer-facing agents, increased test coverage to hundreds of examples in minutes, and added real-time production observability so AI quality regressions are caught before users see them.
monday.com embedded evaluation into monday Service from the start instead of treating it as a final QA step. Its AI service workforce runs as a customizable LangGraph-based ReAct agent that uses company knowledge-base articles and tools to resolve service requests. For offline testing, the team built datasets of sanitized IT tickets and ran deterministic runtime, schema, persistence, and tool-sanity checks alongside OpenEvals, AgentEvals, and LangSmith quality evaluators through the LangSmith Vitest integration, which logs every CI run as an experiment. In production, LangSmith Multi-Turn Evaluator scores complete conversation threads after custom inactivity windows, while TypeScript-based judges are deployed through a GitOps-style CLI that syncs prompts, reconciles rules, and prunes stale evaluations from the platform.
Sanitized ticket examples and production conversations flow through the LangGraph-based service agent, offline suites run in Vitest and log traces to LangSmith, and online threads are later scored by Multi-Turn Evaluator before prompts and rules are redeployed from TypeScript definitions through CI/CD.
8.7x faster evaluation feedback loops (from 162 seconds to 18 seconds).
AI service workforce is a customizable, LangGraph-based, ReAct agent
Many teams treat evaluation as a last-mile check, but we made it a Day 0 requirement.
every CI run is automatically logged as a distinct experiment in the LangSmith platform
we built a custom CLI command, yarn eval deploy, that runs in our CI/CD pipeline
Review how this reference was scored and trace each normalized claim back to its documented source.
Scoring methodology
Customer Reference Value measures usefulness as a customer reference. Evidence Confidence measures the quality and traceability of available evidence, not independent verification.
Traceability
Field coverage shows which facts each source supports. Dates are shown only when documented by the source.
The filing lists the principal executive offices at "6 Yitzhak Sadeh Street, Tel Aviv, 6777506" and states the company headquarters are in Tel Aviv, Israel.
The filing states, "As of December 31, 2025, we had 3,155 employees worldwide."
Related
ServiceNow Sample Modified · Jul 30, 2026 Amadeus Amadeus needed a cloud-ready service backbone that could expose its sprawling technology estate, unify customer support across multiple tools, and speed data access for a travel platform whose outages and delays… 30M items held in Amadeus' configuration database Configuration Management Database · Customer Service Management · IT Operations Management ServiceNow Sample Modified · Jul 30, 2026 ServiceNow ServiceNow's global technical support engineers were struggling to keep up with more than 115,000 customer instances and roughly 304,000 monthly active users because too much time was spent on low-priority manual tasks… $167 million saved with self-service Customer Service Management · IT Service Management · Now Platform 
Mistral Sample Apr 24, 2026 SAP SAP needed to reduce repetitive support demand, break down knowledge silos, and automate business processes for regulated, multilingual customers without sacrificing European data localization or trust in AI-generated… 100% European, locally hosted AI, maintained in SAP-operated infrastructure. Microsoft Teams · Mistral Large · SAP Build Work Zone 
