# AI Agents Info > aiagentsinfo.com is an independent directory and research site tracking AI agents, agent harnesses, frameworks and agent benchmarks. Prices, scores, and revenue figures are dated on the page they come from. This file is generated from the same files that render the site. Fuller plain text, including posts, research notes, glossary entries, directory listings, and the dated figures below, is at https://www.aiagentsinfo.com/llms-full.txt. ## Trackers and data - [Live data trackers](https://www.aiagentsinfo.com/trackers): Download, AGENTS.md, and GitHub star counts that refresh daily from public APIs. - [AI agent framework downloads tracker](https://www.aiagentsinfo.com/trackers/downloads): PyPI and npm file-request counts. Snapshot in this build: 2026-11-09. A download is not a user. - [AGENTS.md adoption tracker](https://www.aiagentsinfo.com/trackers/agents-md): 6,006 of 22,854 popular, active GitHub repositories (26.3%) had a root AGENTS.md or CLAUDE.md on 16 Oct 2026. - [AI agent framework GitHub stars tracker](https://www.aiagentsinfo.com/trackers/stars): Live stargazers_count. Snapshot in this build: 2026-11-09. - [Breakthroughs timeline](https://www.aiagentsinfo.com/trackers/breakthroughs): Public agent moments, added only with a date and a source. No rows are published yet. - [Harness comparison](https://www.aiagentsinfo.com/trackers/harnesses): Pointer to the sourced harness comparison. Cells are published only with a source. - [Verified revenue](https://www.aiagentsinfo.com/verified-revenue): Payment-processor-verified revenue. Checked 2026-11-05. - [Directory stats](https://www.aiagentsinfo.com/directory/stats): Counts generated from the directory files at build time. - [AI agent framework release November 2026: sortable table of 19 repos](https://www.aiagentsinfo.com/reports/ai-agent-framework-github-stats-november-2026): AI agent framework release November 2026: a sortable table of the latest GitHub releases for 19 open-source frameworks and SDKs, pulled 5 November 2026. 14 of 19 were within 30 days. Stars, forks, licences, and release dates are on this page. ## Reports - [Reports](https://www.aiagentsinfo.com/reports): Dated pages with a source and a citation. - [Browser and computer-use agents next 6 months: 10 products, one date (Nov 2026–May 2027)](https://www.aiagentsinfo.com/reports/browser-and-computer-use-agents-next-6-months): Browser agents and computer-use agents from November 9, 2026 through May 9, 2027. Ten products that are live, the one day a vendor has named, and the OSWorld 2.0 scores printed on the benchmark page. Checked November 9, 2026. - [AI coding agents in the next 30 days (8 Nov to 8 Dec 2026): 11 tools, three dated marks](https://www.aiagentsinfo.com/reports/ai-coding-agents-next-30-days): List prices and usage limits for 11 coding agents, checked November 8, 2026, and the three dates official pages already set between November 8 and December 8. - [AI agents in the next 12 months (Nov 2026 to Nov 2027)](https://www.aiagentsinfo.com/reports/ai-agents-next-12-months): What is already dated for AI agents from November 2026 through November 2027: coding agents, computer use, harnesses, and regulation, including the August 2027 EU sandbox date and the MCP deprecation window through July 2027. Checked November 7, 2026. - [AI agents in the next 6 months (Nov 2026 to May 2027)](https://www.aiagentsinfo.com/reports/ai-agents-next-6-months): What is already dated for AI agents from November 2026 through May 2027: coding agents, computer use, harnesses, and regulation. Checked November 5, 2026. - [AI shopping agents next 6 months: 6 companies (Nov 2026–May 2027)](https://www.aiagentsinfo.com/reports/ai-shopping-agents-next-6-months): AI shopping agents for the next 6 months: what Google, OpenAI, Amazon, Microsoft, Shopify, and Adobe have dated from November 2026 through May 2027. The month-by-month table is on this page. Checked November 5, 2026. ## Research - [Research](https://www.aiagentsinfo.com/research): Paper breakdowns and original notes. - [Papers](https://www.aiagentsinfo.com/papers): Paper breakdowns, each with the source link and the paper's authors. - [Garry Tan Harness Evals: $0.17 to $0.44 per Solved Task (Nov 2026)](https://www.aiagentsinfo.com/research/garry-tan-harness-evals-november-2026): 2026-11-09. Seven harness cells on November 5, 2026 scored 71.3% to 78.8% on 20 tasks, and all six preregistered pass-rate tests came back can't tell apart. OpenClaw on Opus 5.5 cost $0.4435 per solved task; Claude Code cost $0.2170. - [How Many GitHub Stars Does OpenClaw Have? 394,222 on November 9, 2026](https://www.aiagentsinfo.com/research/how-many-github-stars-does-openclaw-have): 2026-11-09. 394,222 stars on openclaw/openclaw at 07:41 UTC on November 9, 2026. Hermes Agent had 261,856. Claude Code had 155,043, and its LICENSE.md reserves all rights. - [How Many Repos Use AGENTS.md? 6,006 of 22,854 on October 16, 2026](https://www.aiagentsinfo.com/research/how-many-repos-use-agents-md): 2026-11-09. 6,006 of 22,854 popular, active GitHub repositories had a root AGENTS.md or CLAUDE.md on October 16, 2026. That is 26.3%. The AGENTS.md homepage, read November 9, still says over 60,000 projects. - [Parallel agent teams beat the best of 100 solo agents on 10 of 12 tasks (2026)](https://www.aiagentsinfo.com/research/parallel-agent-teams-test-time-compute): 2026-11-09. November 6, 2026 arXiv paper, in the November 9 listing: with Qwen3.8-27B, ten teams of ten beat the expected best of 100 solo agents on 10 of 12 tasks. - [Paper breakdown: MCPHallu — why MCP agents fail on unreachable goals](https://www.aiagentsinfo.com/research/mcphallu-mcp-agent-hallucinations): 2026-11-08. Average score 0.642 across 14 agents and 5,012 trajectories. Unreachable Goal scores 0.458, and 23.2% of those runs pass. Claude Opus 4.6 leads overall at 0.728. No model in the paper scores above that. - [Terminal-Bench 2.1: 28 of 89 repaired (2026)](https://www.aiagentsinfo.com/research/terminal-bench-2-1-benchmark-repairs): 2026-11-08. Terminal-Bench 2.1 repairs 28 of 89 tasks from version 2.0. Individual scores shift by up to 12 points, ranks move by at most 3. Spearman 0.958. - [Evolver agent exploration: 320 runs, 83.13% to 87.50%](https://www.aiagentsinfo.com/research/evolver-agent-exploration-evolving-capabilities): 2026-11-07. Evolver agent exploration, arXiv November 5, 2026: 320 paired diagnostic runs inside Claude Code and Codex. Success rose from 83.13% to 87.50%. Recorded task-execution tokens fell 24.24%. This page has the tables. - [HarnessFlow MCTS harness optimization: 69.48% average](https://www.aiagentsinfo.com/research/harnessflow-mcts-harness-optimization): 2026-11-06. HarnessFlow MCTS harness optimization, arXiv November 4, 2026: with 15 refinement attempts per benchmark, the average is 69.48%, 10.89 points above Vanilla and 3.95 points above Meta-Harness. The comparison table is on this page. - [SpecCompact agent context compaction: 8.68× median](https://www.aiagentsinfo.com/research/speccompact-agent-context-compaction): 2026-11-05. SpecCompact agent context compaction, posted November 4, 2026: 2.04–5.63× on AppWorld and τ³, and an 8.68× median in a Hermes user study. The speed tables are on this page. - [Paper breakdown: You don't need long context — short working memory still finishes the job](https://www.aiagentsinfo.com/research/you-dont-need-long-context): 2026-11-04. A November 3, 2026 technical report: compaction kept Claude Opus 4.5 and GPT 5.6 Sol near full-history accuracy while the live context stayed short. - [Paper breakdown: Weight-space programming — a language model that reads and edits neural weights](https://www.aiagentsinfo.com/research/weight-space-programming): 2026-11-03. MIT CSAIL note, November 2, 2026: GPT-6 Astra edited neural weights, with no training loop, and raised a small model's MATH-500 score from 7.9% to 48.1%. - [ReAct: Synergizing Reasoning and Acting in Language Models](https://www.aiagentsinfo.com/research/react-reasoning-and-acting): 2026-10-18. ReAct (Yao et al., 2022) interleaves a reasoning trace with an action, then reads the observation. The paper reports this on HotpotQA, FEVER, ALFWorld, and WebShop. ## Benchmarks - [Agent harness benchmark, November 2026](https://www.aiagentsinfo.com/reports/agent-harness-benchmark-november-2026): Agent harness benchmark for November 2026: public SWE-bench and Terminal-Bench scores for coding agents, each with the date on the official source. - [Benchmark leaderboard](https://www.aiagentsinfo.com/trackers/benchmarks): Public agent benchmarks, published only with a source, a date, and the harness. The November 2026 scores live on the harness benchmark. ## Free tools - [AI agent cost calculator](https://www.aiagentsinfo.com/agent-cost-calculator): Free 30-day model bill from official per-token prices. No login. - [Prompt and tool checker](https://www.aiagentsinfo.com/agent-prompt-checker): Server-side check of pasted prompts and OpenAI, Anthropic, or MCP tool JSON. The prompt is not stored. - [Agent picker](https://www.aiagentsinfo.com/tools/agent-picker): Free picker over the directory listings. No login. - [Cursor, Claude Code, and Codex pricing](https://www.aiagentsinfo.com/tools/cursor-vs-claude-code-vs-codex-pricing): Free comparison of list prices from the vendors' pages. ## Directory - [Directory](https://www.aiagentsinfo.com/directory): 139 listings of AI tools, apps, SaaS, and agents, each with a source site. - [AI agents](https://www.aiagentsinfo.com/directory/category/ai-agents): 6 listings. Products whose main job is to pursue a goal with tools or actions, not only to answer in a chat box. - [Agent frameworks & harnesses](https://www.aiagentsinfo.com/directory/category/agent-frameworks): 9 listings. Libraries and harnesses for building agents: the loop, tools, memory, and permissions around a model. - [Coding agents](https://www.aiagentsinfo.com/directory/category/coding-agents): 6 listings. Agents that write, review, or run code, including IDE agents and coding command-line tools. - [Writing](https://www.aiagentsinfo.com/directory/category/writing): 3 listings. Tools for drafting, editing, and rewriting text. - [Image](https://www.aiagentsinfo.com/directory/category/image): 8 listings. Image generation and image-editing products. - [Video](https://www.aiagentsinfo.com/directory/category/video): 19 listings. Video generation, editing, and repurposing products. - [Image & video](https://www.aiagentsinfo.com/directory/category/image-video): 7 listings. Products that produce both still images and video. - [Audio / voice](https://www.aiagentsinfo.com/directory/category/voice): 8 listings. Speech, voice, and other audio products. - [Productivity](https://www.aiagentsinfo.com/directory/category/productivity): 16 listings. Work tools for documents, tasks, and everyday office jobs. - [Marketing](https://www.aiagentsinfo.com/directory/category/marketing): 10 listings. Advertising, campaign, and paid-media tools. - [SEO](https://www.aiagentsinfo.com/directory/category/seo): 5 listings. Tools for search research, content optimization, and site audits. - [Sales](https://www.aiagentsinfo.com/directory/category/sales): 3 listings. Prospecting, outreach, and pipeline tools. - [Customer support](https://www.aiagentsinfo.com/directory/category/customer-support): 3 listings. Help-desk and customer-support products. - [Design](https://www.aiagentsinfo.com/directory/category/design): 4 listings. Design, interface, and visual-production tools. - [Education](https://www.aiagentsinfo.com/directory/category/education): 2 listings. Tutors, classroom tools, and other learning products. - [Research](https://www.aiagentsinfo.com/directory/category/research): 2 listings. Tools for literature search, reading papers, and research workflows. - [Data / analytics](https://www.aiagentsinfo.com/directory/category/data-analytics): 1 listing. Analysis, business-intelligence, and data-work products. - [Finance](https://www.aiagentsinfo.com/directory/category/finance): 4 listings. Accounting, markets, and other finance tools. - [HR / recruiting](https://www.aiagentsinfo.com/directory/category/hr-recruiting): 4 listings. Hiring, resume, and people-operations tools. - [Health](https://www.aiagentsinfo.com/directory/category/health): 2 listings. Health and clinical workflow tools. Listings here are not medical advice. - [Developer tools](https://www.aiagentsinfo.com/directory/category/developer-tools): 9 listings. APIs, gateways, and infrastructure for people building software. - [Automation](https://www.aiagentsinfo.com/directory/category/automation): 1 listing. Workflow and task-automation products. - [E-commerce](https://www.aiagentsinfo.com/directory/category/ecommerce): 1 listing. Store, catalog, and online-selling tools. - [Shopping agents](https://www.aiagentsinfo.com/directory/category/shopping-agents): 1 listing. Agents that search, compare, or buy products. - [Social media](https://www.aiagentsinfo.com/directory/category/social-media): 4 listings. Scheduling and social-publishing tools. - [Other](https://www.aiagentsinfo.com/directory/category/other): 1 listing. Listings that do not fit a narrower category yet. ## Submit - [Submit a tool](https://www.aiagentsinfo.com/submit): Public form. A valid submission is published to the directory. ## Glossary - [Glossary](https://www.aiagentsinfo.com/glossary): 31 short definitions used across the site. ## Optional - [Blog](https://www.aiagentsinfo.com/blog): Explainers, news, and guides. - [Best AI agents](https://www.aiagentsinfo.com/best): Sourced comparisons. Last verified November 2026. - [Best AI agent harness](https://www.aiagentsinfo.com/best/ai-agent-harnesses): Sourced comparison of coding, open-source, and local agent harnesses. Last verified November 2026. No invented scores. - [AI agent framework comparison table](https://www.aiagentsinfo.com/best/ai-agent-frameworks): Nineteen open-source agent frameworks and SDKs. Licence and latest GitHub release read November 9, 2026. Not a quality ranking. - [Best AI coding agents](https://www.aiagentsinfo.com/best/ai-coding-agents): Four coding agents. List prices read November 9, 2026 from the vendor pages. Free and paid. Not a quality ranking. - [Best computer use agents](https://www.aiagentsinfo.com/best/computer-use-agents): Sourced comparison of desktop, API, open-source, and Windows computer use agents. Last verified November 2026. No invented scores. - [Best AI agent evaluation tools](https://www.aiagentsinfo.com/best/ai-agent-evaluation-tools): Sourced comparison of agent evaluation and trajectory-testing tools, including open source. Last verified November 2026. - [Best AI agent for shopping](https://www.aiagentsinfo.com/best/ai-agents-for-shopping): Sourced comparison of shopping agents on checkout, seller, and list price. Last verified November 2026. No invented scores. - [Best AI agent for web search](https://www.aiagentsinfo.com/best/ai-agents-for-web-search): Sourced comparison of web-search agents on plans, citations, and list price. Last verified November 2026. - [Best AI agent for travel booking](https://www.aiagentsinfo.com/best/ai-agents-for-travel-booking): Sourced comparison of travel-booking agents on who takes payment and list price. Last verified November 2026. - [Best AI agent for email](https://www.aiagentsinfo.com/best/ai-agents-for-email): Sourced comparison of email agents on inbox, plans, and list price. Last verified November 2026. - [Best AI agent for personal assistant tasks](https://www.aiagentsinfo.com/best/ai-agents-for-personal-assistant-tasks): Sourced comparison of personal-assistant agents on scheduled work and list price. Last verified November 2026. - [Best AI agent for research](https://www.aiagentsinfo.com/best/ai-agents-for-research): Sourced comparison of research agents on reports, notebooks, and list price. Last verified November 2026. - [Best AI agent for scheduling](https://www.aiagentsinfo.com/best/ai-agents-for-scheduling): Sourced comparison of scheduling agents on who writes the event and list price. Last verified November 2026. - [Best AI agent for writing](https://www.aiagentsinfo.com/best/ai-agents-for-writing): Sourced comparison of writing agents on where the draft lives and list price. Last verified November 2026. - [Best AI agent for students](https://www.aiagentsinfo.com/best/ai-agents-for-students): Sourced comparison of student-facing agents on school rules, tutors, and list price. Last verified November 2026. - [Best AI agent for small business](https://www.aiagentsinfo.com/best/ai-agents-for-small-business): Sourced comparison of small-business agents on seats, inboxes, and list price. Last verified November 2026. - [Topics](https://www.aiagentsinfo.com/topics): Coverage areas such as news, tools, shopping, and harnesses. - [Daily AI & agent news](https://www.aiagentsinfo.com/topics/news): A calm weekly strip of what actually moved in AI and agents. - [New AI tools & SaaS](https://www.aiagentsinfo.com/topics/tools-saas): Launches, product changes, and the directory of tools worth tracking. - [Agent research & breakthroughs](https://www.aiagentsinfo.com/topics/research): Papers, reports, and what they change about how agents are built. - [Agents in shopping](https://www.aiagentsinfo.com/topics/shopping): How agents search, compare, and buy — and where they still fail. - [Agents in task performance](https://www.aiagentsinfo.com/topics/task-performance): Computer-use, office work, coding, and the gap between demos and reliability. - [Agent harnesses](https://www.aiagentsinfo.com/topics/harnesses): The loops, tools, memory, and guardrails that turn a model into an agent. - [Agents in education](https://www.aiagentsinfo.com/topics/education): Tutors, classroom tools, and how to learn with (and about) agents. - [How agents work](https://www.aiagentsinfo.com/topics/how-agents-work): First-principles explainers: models, tools, planning, memory, and evaluation. - [Plain English](https://www.aiagentsinfo.com/for/everyone): Clear AI news and research for adults, with jargon linked to the glossary. - [Builders](https://www.aiagentsinfo.com/for/builders): Harnesses, tools, evals, and how working systems are put together. - [Research](https://www.aiagentsinfo.com/for/research): Paper breakdowns, original notes, and benchmarks written for people who live in the literature. - [AI agents directory MCP server](https://www.aiagentsinfo.com/mcp): Public, read-only Model Context Protocol server at /api/mcp for searching listings, comparisons, and research. - [About](https://www.aiagentsinfo.com/about): aiagentsinfo.com is an independent directory and research site tracking AI agents, agent harnesses, frameworks and agent benchmarks. - [Contact](https://www.aiagentsinfo.com/contact) - [Editorial policy](https://www.aiagentsinfo.com/editorial-policy) - [Privacy](https://www.aiagentsinfo.com/privacy) - [Terms](https://www.aiagentsinfo.com/terms) - [Full text for agents](https://www.aiagentsinfo.com/llms-full.txt): Posts, research, glossary, directory listings, data-page summaries, and verified-revenue figures in one file. # Full text The index above is repeated so this file can stand alone. The sections after it are the source text. # Data pages ## AI agent framework downloads URL: https://www.aiagentsinfo.com/trackers/downloads Snapshot retrieved: 2026-11-09 A download is a file request, not a user. The page refreshes from pypistats and npm. These figures are the snapshot shipped with the build. LangChain (langchain) had 155,918,257 PyPI downloads without mirrors from 2026-10-10 through 2026-11-08. Vercel AI SDK (ai) had 140,196,706 npm downloads from 2026-10-10 through 2026-11-08. ## AGENTS.md adoption URL: https://www.aiagentsinfo.com/trackers/agents-md Snapshot retrieved: 2026-11-09 6,006 of 22,854 popular, active GitHub repositories (26.3%) had a root AGENTS.md or CLAUDE.md on 16 Oct 2026. Sourcegraph public code search on 2026-11-09 reported 13,756 root AGENTS.md files and 6,744 root CLAUDE.md files. Root AGENTS.md pass reported 2 backends unavailable. Root CLAUDE.md pass reported 4 backends unavailable. These are lower bounds from one public index, not a census of every GitHub repository. The agents.md repository had 25,409 GitHub stars in that snapshot. The homepage claim recorded then was "over 60k open-source projects". ## AI agent framework GitHub stars URL: https://www.aiagentsinfo.com/trackers/stars Snapshot retrieved: 2026-11-09 Source: GitHub REST API https://api.github.com/repos/{owner}/{repo} LangChain (langchain-ai/langchain) had 149,267 stars in that snapshot. The live page reads stargazers_count from the GitHub repository API. ## Agent harness benchmark, November 2026 URL: https://www.aiagentsinfo.com/reports/agent-harness-benchmark-november-2026 Read: 2026-11-05 Agent harness benchmark for November 2026: public SWE-bench and Terminal-Bench scores for coding agents, each with the date on the official source. 47 bash-only SWE-bench Verified rows, 30 other team-checked SWE-bench Verified rows, and 13 SWE-bench Multilingual rows are copied from the official board. The newest result date stored with that SWE-bench copy is 2026-02-26. 35 Terminal-Bench rows are copied from the official leaderboard. Each score on the page keeps the date printed on the official source. ## AI agent framework release November 2026: sortable table of 19 repos URL: https://www.aiagentsinfo.com/reports/ai-agent-framework-github-stats-november-2026 Pulled: 2026-11-05 AI agent framework release November 2026: a sortable table of the latest GitHub releases for 19 open-source frameworks and SDKs, pulled 5 November 2026. 14 of 19 were within 30 days. Stars, forks, licences, and release dates are on this page. Data pulled from GitHub on 5 November 2026. This page lists public GitHub figures for 19 open-source agent frameworks and SDKs, including each repository's latest release. Stars, forks, open issues, the detected licence, the primary language, the creation date, and the latest release are copied from that snapshot. Nothing on this page is estimated. ## Browser and computer-use agents next 6 months: 10 products, one date (Nov 2026–May 2027) URL: https://www.aiagentsinfo.com/reports/browser-and-computer-use-agents-next-6-months Checked: 2026-11-09 Browser agents and computer-use agents from November 9, 2026 through May 9, 2027. Ten products that are live, the one day a vendor has named, and the OSWorld 2.0 scores printed on the benchmark page. Checked November 9, 2026. Browser agents and computer-use agents from November 9, 2026 through May 9, 2027. Ten products that are live, the one day a vendor has named, and the OSWorld 2.0 scores printed on the benchmark page. Checked November 9, 2026. ## AI coding agents in the next 30 days (8 Nov to 8 Dec 2026): 11 tools, three dated marks URL: https://www.aiagentsinfo.com/reports/ai-coding-agents-next-30-days Checked: 2026-11-08 List prices and usage limits for 11 coding agents, checked November 8, 2026, and the three dates official pages already set between November 8 and December 8. Three dates are already on official pages between November 8 and December 8, 2026: November 10, November 18, and November 21. Codex, Cursor, GitHub Copilot, and Devin each list a $0 plan with some agent use. The lowest paid monthly stickers checked on November 8, 2026 are Codex Go and Cursor Plus, both $8. Claude Code starts at $20 a month, or $17 a month if you pay $200 up front. ## AI agents in the next 12 months (Nov 2026 to Nov 2027) URL: https://www.aiagentsinfo.com/reports/ai-agents-next-12-months Read: 2026-11-07 What is already dated for AI agents from November 2026 through November 2027: coding agents, computer use, harnesses, and regulation, including the August 2027 EU sandbox date and the MCP deprecation window through July 2027. Checked November 7, 2026. What is already dated for AI agents from November 2026 through November 2027: coding agents, computer use, harnesses, and regulation, including the August 2027 EU sandbox date and the MCP deprecation window through July 2027. Checked November 7, 2026. ## AI agents in the next 6 months (Nov 2026 to May 2027) URL: https://www.aiagentsinfo.com/reports/ai-agents-next-6-months Read: 2026-11-05 What is already dated for AI agents from November 2026 through May 2027: coding agents, computer use, harnesses, and regulation. Checked November 5, 2026. What is already dated for AI agents from November 2026 through May 2027: coding agents, computer use, harnesses, and regulation. Checked November 5, 2026. ## AI shopping agents next 6 months: 6 companies (Nov 2026–May 2027) URL: https://www.aiagentsinfo.com/reports/ai-shopping-agents-next-6-months Checked: 2026-11-05 AI shopping agents for the next 6 months: what Google, OpenAI, Amazon, Microsoft, Shopify, and Adobe have dated from November 2026 through May 2027. The month-by-month table is on this page. Checked November 5, 2026. AI shopping agents for the next 6 months: what Google, OpenAI, Amazon, Microsoft, Shopify, and Adobe have dated from November 2026 through May 2027. The month-by-month table is on this page. Checked November 5, 2026. # Blog ## Agent tool JSON cost: $2.4672 to $17.136 for one list (Nov 9, 2026) URL: https://www.aiagentsinfo.com/blog/agent-tool-json-cost-across-models Published: 2026-11-09. Updated: 2026-11-09 Category: guide On November 9, 2026 the prompt checker priced one three-tool support list at 200 output tokens and 40 turns a day. gpt-5, a tiktoken count, was $3.042 over 30 days. The same paste was $2.4672 on grok-4.7 and $17.136 on gpt-6-astra, both estimates. Updated November 9, 2026. ## What does one agent tool list cost over 30 days? On November 9, 2026 the [prompt and tool checker](https://www.aiagentsinfo.com/agent-prompt-checker) priced this support list at $0.002535 a turn and $3.042 over 30 days on gpt-5. The prompt was 51 tokens. The three tools were 377. Input tokens priced were 428, which is 51 plus 377. Output tokens were the 200 I entered. Turns per day were the 40 I entered. The encoding was tiktoken `o200k_base`, the encoding that library assigns to gpt-5. The checker does not add message framing, and it does not store the paste. The November 7 walkthrough, [Why tool JSON costs more than your system prompt](/blog/why-tool-json-costs-more-than-your-system-prompt), used one weather tool. This paste is a different job: look up an order, refund it, or open a ticket. The paired page is the [Pydantic AI](/directory/pydantic-ai) listing, a typed agent framework whose summary says it checks answers against a schema. The check below is the tool definition, which is the JSON you send before any answer comes back. The prompt: > You are an order-support agent for a store. Look up an order before you change it. Issue a refund only when the order is paid and unshipped. Open a ticket when the shopper asks for a person. Do not invent an order status. The tools, in OpenAI's Responses shape. [The function-calling guide](https://developers.openai.com/api/docs/guides/function-calling), read November 9, 2026, puts `type`, `name`, `description`, `parameters`, and `strict` on the tool, and it says strict mode needs `additionalProperties: false` on each object and every property listed in `required`. ```json [ { "type": "function", "name": "lookup_order", "description": "Look up an order by its id.", "strict": true, "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id." } }, "required": ["order_id"], "additionalProperties": false } }, { "type": "function", "name": "refund_order", "description": "Refund a paid order that has not shipped.", "strict": true, "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id." }, "amount_cents": { "type": "integer", "description": "Refund amount in cents." }, "reason": { "type": "string", "description": "Why the refund was issued." } }, "required": ["order_id", "amount_cents", "reason"], "additionalProperties": false } }, { "type": "function", "name": "open_ticket", "description": "Open a ticket when the shopper asks for a person.", "strict": true, "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id, when there is one." }, "summary": { "type": "string", "description": "What the shopper needs a person to do." } }, "required": ["order_id", "summary"], "additionalProperties": false } } ] ``` The checker returned no schema errors and detected "OpenAI Responses function". gpt-5 standard prices, read the same day from [OpenAI's pricing page](https://developers.openai.com/api/docs/pricing), are $1.25 per million input tokens and $10 per million output tokens. The arithmetic is `(428 × 1.25 + 200 × 10) / 1,000,000 = $0.002535` a turn, times 40, times 30, which is $3.042. The 200 output tokens are $2.40 of that $3.042. The tool JSON is $0.5655. The prompt is $0.0765. A longer reply swamps a careful schema. The schema is still the part a provider can reject before it bills the turn. ## How does agent tool JSON cost change across models? The same 428 input tokens run from $2.4672 on grok-4.7 to $17.136 on gpt-6-astra over 30 days, at 200 output tokens and 40 turns a day. Only the gpt-5 row is a count tiktoken assigns to that model. gpt-6-astra, Claude Sonnet 5.5, Gemini 4 Argon, and grok-4.7 are estimates: the checker counted the same paste with `o200k_base` and said so. Those providers do not bill with that encoding. The prices are the standard rates read November 9, 2026. Cached, batch, flex, and fast rates are left out. Gemini 4 Argon's row is the paid standard price through November 30, 2026, $2 input and $10 output per million tokens. The page says those rates become $4 and $20 on December 1. This table does not apply that later rate. | Model | Paste | Input priced | Per turn | 30 days | | --- | --- | --- | --- | --- | | gpt-5 | Responses tools | 428, exact | $0.002535 | $3.042 | | grok-4.7 | same Responses paste | 428, estimate | $0.002056 | $2.4672 | | Claude Sonnet 5.5 | same Responses paste | 428, estimate | $0.002856 | $3.4272 | | Gemini 4 Argon | same Responses paste | 428, estimate | $0.002856 | $3.4272 | | gpt-6-astra | same Responses paste | 428, estimate | $0.01428 | $17.136 | | Claude Sonnet 5.5 | Anthropic shape of the same three tools | 657, estimate | $0.003314 | $3.9768 | | Claude Haiku 5.5 | Anthropic shape of the same three tools | 657, estimate | $0.0001657 | $0.19884 | grok-4.7 uses the under-200,000-token band, $2 input and $6 output per million, from [xAI's models page](https://docs.x.ai/docs/models). gpt-6-astra uses the short-context band, at or under 272,000 input tokens, $10 and $50, from the OpenAI page. Claude Sonnet 5.5 is $2 and $10. The Responses-shaped paste did not add Anthropic's tool-use system prompt. The checker said that published count applies when the tools are Anthropic's shape. The Anthropic paste uses `name`, `description`, and `input_schema`, which is the shape on the [define-tools page](https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools). `type`, `strict`, and `additionalProperties` are left off. That paste was 51 prompt tokens and 320 tool tokens, both estimates, plus 286 published input tokens. ```json [ { "name": "lookup_order", "description": "Look up an order by its id.", "input_schema": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id." } }, "required": ["order_id"] } }, { "name": "refund_order", "description": "Refund a paid order that has not shipped.", "input_schema": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id." }, "amount_cents": { "type": "integer", "description": "Refund amount in cents." }, "reason": { "type": "string", "description": "Why the refund was issued." } }, "required": ["order_id", "amount_cents", "reason"] } }, { "name": "open_ticket", "description": "Open a ticket when the shopper asks for a person.", "input_schema": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id, when there is one." }, "summary": { "type": "string", "description": "What the shopper needs a person to do." } }, "required": ["order_id", "summary"] } } ] ``` The [tool-use overview](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview), read November 9, 2026, lists 286 for Claude Sonnet 5.5 when `tool_choice` is auto or none and at least one tool is sent. The any/tool cell on that row is blank, so the checker added no second figure. Input tokens priced were 657, which is 51 + 320 + 286. The 30-day figure was $3.9768. Claude Haiku 5.5 used the same 657 and the band for a prompt up to 100,000 tokens, $0.10 input and $0.50 output per million. The 30-day figure was $0.19884. The same row lists 406 tokens when `tool_choice` is any or a named tool. The checker used the auto/none count of 286 and left 406 out. What you can do today: paste the list once, read the token split, then change only the model. The spread here is the price, on one shared estimate of the paste. gpt-6-astra's 30-day figure is $17.136 against gpt-5's $3.042. Output is most of both bills, at $50 per million tokens against $10. ## Does a longer tool description change the bill? Four sentences on `lookup_order` added 31 tokens and $0.0465 over 30 days on gpt-5. Anthropic's define-tools page says to aim for at least three or four sentences when the tool needs them. I replaced "Look up an order by its id." with this and left the other two tools as they are: > Retrieves one order for a shopper by order id. Use it before a refund or a ticket. It returns the payment state and whether the order has shipped. It does not change the order. The tool text moved from 377 tokens to 408. Input tokens priced moved from 428 to 459. Per turn moved from $0.002535 to $0.00257375. The 30-day figure moved from $3.042 to $3.0885. The gap is $0.0465. Keep the four sentences if the model picks the wrong tool without them. Cut them when the list is long enough that the descriptions, not one lookup line, are the bill. On this list the output box is still the large number. ## Does a dotted tool name fail? `orders.lookup` was a warning on a top-level OpenAI function, an error inside a namespace, and `admin.tools.list` was a warning that the MCP page allows the dot. On November 9 the name check was updated from the pages read that day. A top-level OpenAI function name with a dot had been reported as an error, using a pattern the Responses reference prints for a function inside a namespace. The MCP example `admin.tools.list` had been described as if the tools page published no character set. The [2026-07-28 tools page](https://modelcontextprotocol.io/specification/2026-07-28/server/tools) says a name should be 1 to 128 characters and should use only letters, digits, underscore, hyphen, and dot, and it lists `admin.tools.list` as a valid example. Anthropic's define-tools page still requires `^[a-zA-Z0-9_-]{1,128}$`. This top-level tool returned ok, with one warning, 97 tool tokens, and a 30-day figure of $2.622 on gpt-5 at the same 200 and 40: ```json [ { "type": "function", "name": "orders.lookup", "description": "Look up an order by its id.", "strict": true, "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id." } }, "required": ["order_id"], "additionalProperties": false } } ] ``` The warning was: "orders.lookup" contains a dot. The Responses reference allows a dot in `^(?![\s\S]*[\r\n])[a-zA-Z0-9_.-]+$` for a function loaded inside a tool-search namespace, and it does not print a pattern on a top-level function name. A function name inside a namespace tool must match `^[a-zA-Z0-9_-]{1,128}$`, which rejects a dot. Anthropic's define-tools page requires that same no-dot pattern. The same name inside a namespace was an error. Tool text was 130 tokens. The 30-day figure was $2.6715, and the note said a provider can reject the request before it bills the turn. ```json [ { "type": "namespace", "name": "orders", "description": "Order tools.", "tools": [ { "type": "function", "name": "orders.lookup", "description": "Look up an order by its id.", "strict": true, "parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The order id." } }, "required": ["order_id"], "additionalProperties": false } } ] } ] ``` The error was: "orders.lookup" does not match `^[a-zA-Z0-9_-]{1,128}$`. The Responses reference publishes that pattern for a function name inside a namespace, up to 128 characters. The MCP paste returned ok, with a warning, 57 tool tokens, and $2.562 over 30 days on the gpt-5 price: ```json { "tools": [ { "name": "admin.tools.list", "description": "List the admin tools on this server.", "inputSchema": { "type": "object", "additionalProperties": false } } ] } ``` The warning was: "admin.tools.list" fits the tool-name guidance on the MCP tools page for revision 2026-07-28: 1 to 128 characters from letters, digits, underscore, hyphen, and dot. The page's own example is admin.tools.list. Anthropic's define-tools page requires `^[a-zA-Z0-9_-]{1,128}$`, which rejects a dot. OpenAI publishes that no-dot pattern for a function name inside a namespace. What you can do today: use a dot when the server is MCP and the spec's own example is the pattern you want. Rename to `orders_lookup` before you put that tool inside an OpenAI namespace or send it to Claude. ## What strict-mode errors did refund_order return? Two errors. `additionalProperties` was missing, and `reason` was a property that `required` did not list. I took the first list, removed `additionalProperties` from `refund_order`, and removed `reason` from that tool's `required` array. `strict` stayed true. The checker returned: - `tools[1].parameters.additionalProperties`: OpenAI strict mode requires additionalProperties: false on each object in the parameters. - `tools[1].parameters.required`: OpenAI strict mode requires every property to be listed in required. "reason" is missing. The guide says an optional field can use "null" as one of its types and still be required. The tool text was 367 tokens. Input tokens priced were 418. Per turn was $0.0025225. The 30-day figure was $3.027. The note said the cost includes this tool text, and a provider can reject the request before it bills the turn. Fix the two fields, then read the price. The November 7 article shows the same class of error on a weather tool. ## Where does Pydantic AI fit? [Pydantic AI](/directory/pydantic-ai) checks an answer against a schema. This checker checks the tool JSON before that answer exists. The directory page, last verified November 3, 2026, calls it a Python framework for typed agents: you name a model, add tools, and get answers checked against a schema. The library has no fee of its own and is MIT licensed. The release tag read on November 9, 2026 is on [Best AI agent frameworks](/best/ai-agent-frameworks). That page compares licences. It does not price a tool list. Run the JSON through the [prompt checker](https://www.aiagentsinfo.com/agent-prompt-checker) before you trust the typed answer. A request rejected for `additionalProperties` or a dotted name inside a namespace never returns an object for the framework to validate. If you already know the token counts, the [cost calculator](/agent-cost-calculator) multiplies them in the browser. ## What does the 30-day figure leave out? The figures above are the checker's multiplication of the pasted text, the published Claude add-on when it applied, and the output tokens and turns I entered. - Message framing is not in the count. OpenAI's guide says functions are injected into the system message, so the billed input can differ from the pasted JSON. - gpt-6-astra, Claude, Gemini, and grok counts of the paste are tiktoken estimates. The 286 and the unused 406 are Anthropic's published counts. - No per-call tool fee, computer-use fee, search-grounding fee, or code-execution fee is added. The price pages read on November 9 do not put a single per-call fee on these rows. - Output tokens are the number entered. This check did not call a model and did not count a reply. - Gemini 4 Argon's $2 and $10 rate is the price through November 30, 2026. The calculator does not switch itself on December 1. - The prompt and the tools are not written to a file, a database, or a log. ## FAQ ### What does one agent tool list cost over 30 days? On November 9, 2026 this three-tool support list was $3.042 over 30 days on gpt-5, with 200 output tokens and 40 turns a day. The prompt was 51 tokens and the tools were 377. ### How does agent tool JSON cost change across models? The same 428 input tokens were $2.4672 on grok-4.7 and $17.136 on gpt-6-astra. The gpt-5 count is tiktoken's. The other rows estimate with that count and the standard prices read November 9, 2026. ### Does a dotted tool name fail? `orders.lookup` was a warning at the top level and an error inside a namespace. `admin.tools.list` fit the MCP tools page for revision 2026-07-28. Anthropic's required pattern rejects a dot. ### Does a longer tool description change the bill? Four sentences on `lookup_order` added 31 tokens and $0.0465 over 30 days on gpt-5. The 200 output tokens remained $2.40 of the $3.042. ## Claude Code Don't Ask Again: 1,122 Public Files Held a Secret (Nov 2026) URL: https://www.aiagentsinfo.com/blog/claude-code-dont-ask-again Published: 2026-11-09. Updated: 2026-11-09 Category: explainer 1,122 public copies of .claude/settings.local.json held at least one secret, 1,379 in all, in the census The Hacker News and iTnews attribute to Apiiro. Claude Code writes that file when you pick Yes, and don't ask again. On November 9, 2026, the phrase people were searching was Claude Code don't ask again. The product answer is on Claude Code's own permissions page. The count attached to it is someone else's. [The Hacker News](https://thehackernews.com/2026/11/claude-codes-dont-ask-again-file.html), on November 7, and [iTnews](https://www.itnews.com.au/news/dont-ask-again-in-anthropics-claude-code-leaks-developer-secrets-630203), on November 9, both cite an Apiiro census: 1,122 public copies of `.claude/settings.local.json` hold at least one secret, 1,379 distinct secrets in all. AI Agents Info read those two articles, the permissions page, and the settings page on November 9. A direct request to [Apiiro's write-up](https://apiiro.com/blog/coding-agents-leaking-secrets-github-dont-ask-again) that day returned a Cloudflare challenge, so the census below is the census those articles attribute to Apiiro. This page did not recount GitHub, and it did not test a key. The file sits inside the [harness](/glossary/harness): the loop, tools, and permission stops around the model. The stop, on this site, is an [approval gate](/glossary/approval-gate). Claude Code is the coding agent in the [directory](/directory/claude-code). A separate November 9 note reads a public cost comparison of that harness against others: [Garry Tan harness evals](/research/garry-tan-harness-evals-november-2026). ## What does “don’t ask again” do in Claude Code? Yes, and don't ask again can save the command you just approved, so a later session in that repository runs a matching command without asking. [Claude Code's permissions page](https://code.claude.com/docs/en/permissions), read November 9, 2026, prints a table of what the option does. Bash commands are "Permanently per repository and command." WebFetch is "Permanently per repository and domain." Web search is "Permanently per repository." File modification is "Until session end." Read-only tools inside the working directory are not asked in the first place. The same page says the prompt only offers don't ask again when it can show you everything the saved rule would allow. A saved rule covers what that option named, not a wider class of commands you did not see. For a compound command, the page gives `git status && npm test` as the example: the saved rule is for `npm test`, the subcommand that needed approval, and a single approval can save up to five rules. That is the mechanism the news reports are about. If the thing you approved is one exact command, and the command has a password or a token in it, the allow rule is that string. The Hacker News says the researchers found saved commands with the secret still inside, and that they reproduced the behavior with Opus 5.5 and Fable 5.1. iTnews says the same reproduction, and says Apiiro treated it as a property of Claude Code rather than of the model. ## Where does Claude Code save the approval? A permanent approval is written to `.claude/settings.local.json` at the root of the git repository, as an allow rule. The permissions page says that when the approval saves permanently, Claude Code writes the rule to that file, resolved through worktrees to the main checkout. The rule then applies to later sessions anywhere in that repository, including a session started in a subdirectory or a worktree. A file-modification approval is the exception in the table. It is not saved to the file. The [settings page](https://code.claude.com/docs/en/settings), read the same day, says Claude Code creates `.claude/settings.local.json` the first time you give a standing approval on a permission prompt, and it names "Yes, and don't ask again" for a Bash command as the example. The scope table calls the file personal to one project. The shared file, `.claude/settings.json`, is the one the same page tells a team to commit. Two limits are on that page, and they matter if you go looking for the file and do not find it at the root. Outside a git repository, when the repository root is your home directory, on Windows, or when the root or its `.git` or `.claude` entry is not owned by your user, the file stays with `.claude/settings.json` instead. Before v2.1.211, Claude Code kept the file in the directory where you started the session. Current Claude Code still reads an older file left there, and permission rules from both files apply. The settings page also says the local file's allow rules apply without the workspace trust step while the file stays untracked. If git is tracking the file, the trust step applies to it too. ## How many public copies held a secret? The reported census is 104,185 public copies in 103,617 repositories. 1,122 copies, 1.08 percent of 104,185, hold at least one secret. Both news articles print 104,185 copies, 103,617 repositories, 1,122 copies with at least one secret, and 1,379 distinct secrets. 1,122 divided by 104,185 is 1.08 percent. The Hacker News rounds the same ratio to about 1.1 percent. The wider total in both articles, once older versions of the file, the shared `.claude/settings.json`, and saved agent chat logs are included, is 2,539 secrets across 1,994 repositories. The Hacker News describes how a value got into the 1,379. It matched a known key format, or a password or token sat beside the address of a public server, and an AI reviewer also had to agree. The article says the researchers do not say whether anyone has used these secrets. Both articles say Apiiro did not test whether the credentials still work, and that one company notified earlier confirmed its key had been live. AI Agents Info did not repeat the search. The GitHub code-search API, requested without a token on November 9, returned 401. The figures in the table are the figures the two articles attribute to Apiiro. | What was counted | Number | Who reported it | | --- | --- | --- | | Public copies of `.claude/settings.local.json` | 104,185 | The Hacker News and iTnews, citing Apiiro | | Repositories holding those copies | 103,617 | Both | | Copies with at least one secret | 1,122 | Both | | Distinct secrets in the current copies | 1,379 | Both | | Database passwords | 379 | Both | | Supabase service-role keys | 122 | iTnews, citing Apiiro | | SSH logins for public servers | 43 | Both | | GitHub, GitLab, and npm tokens | 42 | Both | | AWS keys | 23 | Both | | Secrets once history, the shared settings file, and chat logs are included | 2,539 | Both | | Repositories in that wider count | 1,994 | Both | | Organizations among the owners | 63 | Both | The Hacker News says more than a third of the secrets open a database directly, and it names the 379 database passwords inside that statement. iTnews puts 122 Supabase service-role keys in the same breath as those 379 passwords. This page does not add 379 and 122 and call the sum Apiiro's, because the November 7 article's paragraph does not print 122, and "including" does not say the two rows are a partition. Three further counts are in The Hacker News and not in the iTnews article read for this page. | What The Hacker News adds | Number | | --- | --- | | Repositories where a commit listing Claude as a co-author changed the file | 539 of 1,105 | | Secrets present only in older versions of the file | 734, across 509 repositories | | Forks carrying the same secrets as the originals | 211 forks of 74 repositories | | Copies of the shared `.claude/settings.json` that hold a secret | 298 of 76,011 | The same article says that as of Monday, every file with a secret was still on its repository's default branch, and that in half of those repositories the file had been there for at least 217 days. The article's dateline is November 7, 2026. It does not name which Monday. It also says that of 16 coding agents whose documentation the researchers compared, Claude Code was the only one confirmed to save the full approved command in a project file by default. This page did not read those 16 documentation sets. That sentence is The Hacker News's account of the comparison. ## Why did secret scanners miss 215 files? Of the 1,122 files with a secret, gitleaks flagged 564 and TruffleHog flagged 643. Neither flagged 215. Both articles print those three counts, and both say the scanners were run without their verification step. The Hacker News's explanation is that most misses are passwords typed inside commands. A scanner built for known key formats can look straight past a password after `PGPASSWORD=`, `sshpass -p`, or `mysql -p`. The three counts overlap. Files flagged by neither are 215, so files flagged by at least one scanner are 1,122 minus 215, which is 907. gitleaks plus TruffleHog is 564 plus 643, which is 1,207. The difference, 1,207 minus 907, is 300. On these counts, 300 files were flagged by both scanners. That subtraction is this page's. Neither article prints 300. The Hacker News also says the file matters off GitHub. It is plain JSON in the project folder, and it can sit in an npm package, a Docker image, a zip archive, or a synced folder. The same article says the Shai-Hulud worm, in September 2025, ran TruffleHog over infected computers. That is a reason the local copy is part of the cleanup, not a claim that this census measured laptops. ## Does the git exclude stop a commit? It stops a later commit on the computer that wrote the file. It does not travel with the repository, and it does not untrack a file that is already in git. The settings page is specific. The first time Claude Code writes `.claude/settings.local.json` in a git repository that does not already ignore it, Claude Code adds `**/.claude/settings.local.json` to the global git excludes file. That path is `core.excludesFile` when the global git config sets it to an absolute or `~`-prefixed path. Otherwise it is `$XDG_CONFIG_HOME/git/ignore`, or `~/.config/git/ignore` when `XDG_CONFIG_HOME` is unset. Git on that computer then ignores the file in every repository. The same paragraph says you do have to gitignore it yourself if you created the file by hand and Claude Code has not written to it yet. The scope table says the same thing in shorter form: "Claude Code keeps it out of git when it creates the file; if you create it by hand, add it to `.gitignore` yourself." A global exclude is not the project's `.gitignore`. A teammate's clone does not inherit it. A file that was committed before the exclude existed stays in the repository, and it stays in history after someone deletes the current copy. The Hacker News's 734 secrets across 509 repositories are the older-version row of that fact. Its 211 forks of 74 repositories are the fork row. Boris Cherny's comment on [Hacker News](https://news.ycombinator.com/item?id=50408126), posted by the account bcherny and beginning "Boris from the Claude Code team here," says: "We automatically gitignore it when Claude creates the file, so in this case the files were manually created and checked in." He links the settings docs. iTnews identifies him as Boris Cherny, head of Claude Code. The comment's inference and the co-author count are both public, and they are not the same claim. The Hacker News says that in 539 of the 1,105 repositories the researchers traced, a commit listing Claude as a co-author changed the file, and that the researchers take this to mean Claude often commits the file. This page has not read those commits. The settings page, read November 9, describes a global exclude on creation, plus a hand-created-file exception. It does not say every public copy was typed by hand, and it does not say Claude Code writes a project `.gitignore` line. ## Is don’t ask again the same as dontAsk mode? No. One saves an allow rule. The other auto-denies prompts. The permissions page lists `dontAsk` as a mode, next to `default` (Manual), `acceptEdits`, `plan`, `auto`, and `bypassPermissions`. In `dontAsk`, Claude Code auto-denies every call that would otherwise prompt, with the exceptions that page lists: a mod can still approve some calls, file reads in your working directories still run, and tools you pre-approved still run. `AskUserQuestion`, MCP tools marked `requiresUserInteraction`, reads from network paths, and connector tools an organization set to ask are denied even if you have allowed them. `auto` is a third thing. The page says it runs without routine prompts, and a background classifier checks that shell commands and network requests align with your request before they run. The settings page does not say that auto mode writes `.claude/settings.local.json`. Boris Cherny's comment treats permission rules and auto mode as alternatives: "Permission rules are old hat. Everyone should be using auto mode instead. It is safer and also does not require manually maintaining finicky allow/deny lists. Auto mode is enabled by default and is included for free in all plans." That is his comment. This page did not re-check plan prices. List prices for Claude Code and Codex are in [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). A reader on the same thread wrote: "This still presumably happens in auto mode as it asks for sensitive commands (and the default is 'always allow,' which saves it)." Cherny replied: "Yeah good catch. Thinking through what we can do to improve this (ideas welcome)." In a separate reply he named approaches under consideration: "making sure auto mode catches this, building in basic secret scanning, a few other ideas. Going to prototype a few of these to understand the tradeoffs better before we ship something." His opening comment says Claude Code has historically not done secret scrubbing, and that the team is looking into what it can do. The "good catch" reply agrees with a presumption about auto mode's sensitive-command prompt. It does not, by itself, say that auto mode writes the full command into `settings.local.json`. The documented writer of that file, on the pages read November 9, is the permanent don't-ask-again approval. ## What should you do if the file is in a repository? Rotate the credentials first. Deleting the file does not remove a secret from git history or from a fork. The Hacker News prints the researchers' order. Search the organization for `.claude/settings.local.json` and `.claude/settings.json`, and check history even where the current tree no longer has the file. Rotate every password, token, and key the files have ever held before cleaning up. Stop tracking the local file and add it to the repository's own `.gitignore`. The command the article prints for the untrack step is `git rm --cached .claude/settings.local.json`. Open the copy on your own computer, even if it was never pushed, and delete every rule that contains a secret. Keep the secret out of the command in the first place, by referring to a variable such as `$DB_URL`. Approve a pattern such as `Bash(npm run:*)` rather than one exact command. For a security team, the same article's list is to treat agent folders, including `.claude/`, `.cursor/`, `.gemini/`, and `.specstory/`, and agent memory files, the way a team treats `.env` files, on laptops as well as in repositories. Add them to pre-commit and CI scanning, including history, with a scanner that catches passwords inside commands. Add `.claude/settings.local.json` to the organization's gitignore templates. Check credentials that are paired with a public server first. None of that is a finding that a named key in the census still works. The only live confirmation either article reports is the one company. ## Why does the approval gate matter here? An approval that remembers the exact command is a harness decision, and the remembered string is what ended up in public git. On this site an [approval gate](/glossary/approval-gate) holds a named action until a person approves it. Don't ask again is that gate deciding to stay open for one command, and then writing the command down. A [guardrail](/glossary/guardrail) that only watches known key formats missed 215 of the 1,122 files, on the scanner counts above. A [sandbox](/glossary/sandbox) is a different control. Cherny pointed one Hacker News reader who objected to auto mode at the built-in `/sandbox`. The reader's reply on that thread was that the sandbox still has holes. This page did not re-test `/sandbox`. The practical split, from the permissions table, is smaller than the headline. A file edit you approve is forgotten at the end of the session. A Bash command you tell it not to ask about again is stored, in the project, until you take the rule out. The census says 1,122 public copies still had a secret in that stored rule. The settings page says the protection that is supposed to keep the file local lives in one user's global git excludes, on one computer. How a harness is scored once the permissions are in place is a different question. The method is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). The product definition is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Those pages, and a November 5 cost comparison of harnesses, are on the [agent harnesses hub](/topics/harnesses). The cost note is [Garry Tan harness evals](/research/garry-tan-harness-evals-november-2026). ## FAQ ### What does don’t ask again do in Claude Code? On the permissions page read November 9, 2026, Yes, and don't ask again saves an allow rule for a Bash command permanently, for that repository and that command. The same choice for a WebFetch domain is permanent per repository and domain. A file-edit approval lasts until the session ends and is not written to `.claude/settings.local.json`. ### How many Claude Code settings files held a secret? The Hacker News on November 7 and iTnews on November 9, both citing Apiiro, report 104,185 public copies of `.claude/settings.local.json` in 103,617 repositories. 1,122 copies hold at least one secret, 1,379 distinct secrets in all. The wider total they print, with history, the shared settings file, and chat logs, is 2,539 secrets across 1,994 repositories. AI Agents Info did not recount GitHub. ### Did anyone check that the secrets still work? Both articles say the census did not test whether the credentials still work. Both say one notified organization confirmed that its key had been live. The Hacker News says a value counted when it matched a known key format, or when a password or token sat beside a public server, and an AI reviewer agreed. ### Does Claude Code gitignore settings.local.json? The settings page says the first write adds `**/.claude/settings.local.json` to the user's global git excludes file, so that computer ignores it in every repository. If you created the file by hand and Claude Code has not written to it yet, the same page tells you to add it to `.gitignore` yourself. A global exclude does not untrack a file that is already committed, and it does not travel with the clone. ### Is don’t ask again the same as dontAsk mode? No. Don't ask again is a prompt option that can save an allow rule. `dontAsk` is a mode that auto-denies calls which would otherwise prompt, aside from the exceptions on the permissions page. ### What should you do if the file is in a repository? Rotate every password, token, and key the file has ever held, then remove it from the tree and from history's reach as far as you can. The researchers' order, printed by The Hacker News, puts rotation before cleanup, because a deleted file leaves the secret in git history and in forks. Add `.claude/settings.local.json` to the repository's own `.gitignore`, and delete secret-bearing rules from the copy on your computer even if you never pushed it. ## How to Build an Eval Harness for AI Agents: Isolate First URL: https://www.aiagentsinfo.com/blog/how-to-build-an-eval-harness-for-ai-agents Published: 2026-11-09 Category: guide How to build an eval harness for AI agents: write tasks with checkable outcomes, isolate every trial, grade the environment, and keep a held-out set. Harbor, Inspect, and a local pytest suite are three ways to run that loop. Updated November 9, 2026. ## What is an eval harness for AI agents? An eval harness is the infrastructure that runs the tests: it starts a task, gives the agent tools and an environment, records the trace, grades the result, and rolls trials into a report. [Anthropic’s January 9, 2026 post](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) uses that wording. It defines a **task** as one test with inputs and success criteria, a **trial** as one attempt at that task, a **grader** as the logic that scores a transcript or an outcome, and an **evaluation harness** as the system that runs those pieces end to end. The companion object is the [agent harness](/glossary/harness): the loop that lets the model act. This page is about building the first object. How to choose the checks once it exists is [How to evaluate an AI agent harness](/blog/how-do-you-evaluate-an-ai-agent-harness). What the workshop around the model is, in the first place, is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). You do not need a public leaderboard to start. Google’s [September 9, 2026 post](https://developers.googleblog.com/en/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/) says evals belong after dogfooding, and that their job is confidence a prompt, tool, or model change did not make the agent worse. NVIDIA’s [September 21, 2026 post](https://developer.nvidia.com/blog/how-to-evaluate-ai-agents-from-tool-calls-to-task-completion/) says a public suite can set a floor, then a domain eval from your tickets and APIs becomes the release gate. ## How is an eval harness different from an agent harness? The agent harness runs the job. The eval harness scores whether that job, or one step inside it, happened. Anthropic’s January 9 post puts the split in one list. An **agent harness** (it also says scaffold) processes inputs, orchestrates tool calls, and returns results. When you evaluate “an agent,” you are evaluating the harness and the model together. Claude Code is named there as a flexible agent harness. An **evaluation harness** is the other layer: instructions, tools, concurrency, recording, grading, and aggregation. This site uses the same split. [LangChain’s March 10, 2026 anatomy post](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) and [Databricks’ June 17, 2026 explainer](https://www.databricks.com/blog/ai-harness) describe the agent as model plus harness. [OpenAI’s February 11, 2026 harness-engineering post](https://openai.com/index/harness-engineering/) is about designing that surrounding system so agents can implement. The eval harness is how you find out whether a change to that system helped. Designing the agent side is [harness engineering](/glossary/harness-engineering). Scoring one visible action is a [behavioral evaluation](/glossary/behavioral-evaluation). Scoring a frozen task set is a [benchmark](/glossary/benchmark). If you mix the two words, a SWE-bench row looks like a product ranking. The [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026) copies official boards and keeps the harness name on the row for that reason. ## What parts does an eval harness need? It needs a task, an isolated environment, a runner for the real agent, a trace, a grader, and a way to aggregate more than one trial. NVIDIA’s September 21 post rolls a run up as benchmark → trial → task → turn → step. A **trial** is one pass over the task set under a fixed configuration. A **task** is one scorable instance. A **turn** is one exchange. A **step** is one action inside that turn, usually a tool call. Anthropic’s January 9 list matches: task, trial, grader, transcript, outcome, evaluation harness, evaluation suite. The pieces you have to build, in that vocabulary:
Part What it holds A public description
Task Goal, starting state, success rule Anthropic: a test with inputs and success criteria. Harbor’s task overview stores that as instruction.md, task.toml, environment/, and tests/.
Isolated trial A fresh machine or workspace Anthropic: start from a clean environment. Inspect’s sandbox page says the Docker provider runs each sample in a container, with sample_init at the start of the sample.
Agent runner The same loop users get Anthropic: the agent in the eval should function roughly the same as the production agent. Harbor’s README evaluates agents such as Claude Code, OpenHands, and Codex CLI.
Transcript Ordered log of the attempt NVIDIA: user message, each step, environment state when the attempt stops. Anthropic also says trace or trajectory.
Grader Code, a model rubric, or a person Anthropic’s three types: code-based, model-based, and human. Harbor’s verifier writes a number to /logs/verifier/reward.txt or reward.json.
Aggregation More than one trial, then a report Google: do not block a pull request on one noisy run; watch the aggregate pass rate. NVIDIA: report a range across 3–5 trials, not a point estimate.
A [guardrail](/glossary/guardrail) is not a grader. The guardrail allows, blocks, or pauses an action in production. The grader tells you, after the fact, whether the action happened. ## How do you write a task the grader can check? Write a goal, a fixture for the starting state, and a check that reads the environment or a tool call, not the agent’s closing sentence. Anthropic’s January 9 post is blunt about the last part. A flight-booking agent can say “Your flight has been booked.” The **outcome** is whether a reservation exists in the environment’s database. NVIDIA’s September 21 post says the same for a refund, a ticket, or a merged pull request: gate on environment state, not a judge’s opinion of the final message. Google’s September 9 example asserts that a web-search tool was called for live weather, and the comment on the assertion is “Assert behavior, not output prose.” Harbor’s [create-a-task tutorial](https://docs.harborframework.com/tutorials/create-a-task), read November 9, 2026, generates a folder with `instruction.md`, `task.toml`, an `environment/Dockerfile`, a `solution/solve.sh`, and `tests/test.sh` plus pytest. The [verifier page](https://docs.harborframework.com/tasks/verifier) says `tests/test.sh` runs after the agent, and that it must write a numeric reward. The tutorial’s script runs pytest and writes `1` or `0` to `/logs/verifier/reward.txt`. That is an outcome check. A [behavioral](/glossary/behavioral-evaluation) check is smaller: Google’s pytest reads the tool-call names on one reply. Start with one failure you have already seen. Google’s first step is that. NVIDIA’s [November 4, 2026 NeMo post](https://developer.nvidia.com/blog/use-nvidia-nemo-to-turn-agent-failures-into-tests-and-harness-fixes/) starts from stored traces: an agent drafted a message and also called `send_message` without confirmation. NeMo Eval Author then wrote four Harbor tasks and a verifier that fails the run if that send happened. The longer walkthrough of that cycle is the [evaluation guide](/blog/how-do-you-evaluate-an-ai-agent-harness). If several correct paths exist, do not freeze the tool order. Google says to use an outcome check, including an LLM-as-a-judge, when a rigid sequence would punish a valid path. NVIDIA’s September 21 post treats LLM-as-a-judge as provisional until you validate it against human ratings on a sample. Anthropic says model-based graders need calibration with human graders. ## How do you keep trials isolated? Give every trial a fresh environment, and keep earlier git history, files, and caches out of it. Anthropic’s January 9 post says each trial should be isolated by starting from a clean environment. Leftover files, cached data, and resource exhaustion can make failures correlate with the infrastructure instead of the agent. Shared state can also inflate a score. The example the post gives is internal evals where Claude gained an unfair advantage by examining the git history from previous trials. Inspect’s sandbox docs describe the same habit in code. The built-in Docker environment “runs each sample within a Docker container.” Lifecycle methods split expensive setup (`task_init`, such as pulling an image) from per-sample work (`sample_init` at the beginning of each sample, `sample_cleanup` at the end). Harbor’s task overview calls tasks independent, isolated, and reproducible, and it starts an environment, lets the agent act, stops the environment, then runs the verifier. NVIDIA’s November 4 post recommends a Linux machine with Docker for its NemoClaw walkthrough, and it names OpenShell 0.1.2 and Harbor 0.22.0 as of the November 8, 2026 reading on this site’s evaluation guide. This page does not re-run that blueprint. The isolation rule does not depend on those pins: a new container or workspace per trial, and the solution and hidden tests kept out of the agent’s reach. Harbor’s verifier docs say that, without a dedicated verifier image, Harbor starts a fresh copy of the agent environment and uploads `tests/` to `/tests/`, and that the agent’s filesystem changes are not inherited. Record the settings with the report: model id, agent version, CPU, memory, timeout. NVIDIA’s September 21 post lists cost per success and steps per success beside the pass rate. Two runs that used different limits are not the same experiment. ## Should you grade the steps, the outcome, or both? Grade the outcome to decide whether the job finished. Grade the steps to name which action failed. NVIDIA’s September 21 post calls those **process scoring** and **E2E (outcome) scoring**. Process scoring asks whether a call was valid, relevant, and useful at that point. Outcome scoring ignores the path and reads the final state. It says most production evals gate the release on the outcome and keep step-level traces for debugging. Google’s September 9 post says the same pairing in different words: macro benchmarks verify the destination; micro behavioral evals are the partner while you change a prompt, a tool, or a model. Anthropic’s January 9 post adds a third distinction that belongs in the suite design, not in a single grader. **Capability evals** ask what the agent can do, and should start at a low pass rate. **Regression evals** ask whether it still handles what it used to, and should sit near a full pass rate. When a capability task saturates, it can graduate into the regression suite. A published board is an outcome suite with someone else’s tasks. The [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026) copies SWE-bench Verified and Terminal-Bench 4.0 from the official files. Harbor’s README, read November 9, 2026, says Harbor is the official harness for Terminal-Bench 2.0. Those are different editions. Do not treat a 2.0 Harbor run as a row on this site’s 4.0 table. ## When should you use Harbor, Inspect, or a local pytest suite? Use a local pytest assertion when one action is the whole question. Use Harbor when the task needs a container, a hidden verifier, and a numeric reward. Use Inspect when you already want its sample-level Docker (or another) sandbox inside a Python eval. Harbor is a framework from the Terminal-Bench authors. The README, read November 9, 2026, says you can evaluate agents such as Claude Code, OpenHands, and Codex CLI, build and share benchmarks, and run environments in parallel through named providers. A task is a directory. The verifier script owns the score. NVIDIA’s November 4 post says NeMo Eval Author can write evaluations in Harbor format or in NVIDIA NeMo Gym format. Inspect, from the UK AI Security Institute, is one of the libraries on [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). Its sandbox page documents `local` (same filesystem, only if you are already inside another sandbox) and `docker` (a container per sample). That page is about how the environment is created. It is not a Harbor replacement, and it is not a pytest replacement. Google’s September 9 post stays on pytest: `pytest evals/behavioral/ -v`, with a comment that the local suite runs in under five seconds. Treat that comment as the post’s illustration of a fast check, not as a promise about your suite. The Python example imports `Agent` from a module path the post writes as `google.antigravity`. This page does not add an API beyond that example. You can run more than one of these. A behavioral pytest suite for iteration, and a Harbor or Inspect outcome suite for the destination, matches Google’s complementary pairing. Tools that wrap those checks, with licences and list prices, stay on the evaluation comparison. The runtimes they wrap stay on [Best AI agent harness](/best/ai-agent-harnesses). ## How do you stop the suite from overfitting? Keep at least one case out of the edit, and keep a destination score the editor is not allowed to chase on every commit. NVIDIA’s November 4 example produced four Harbor tasks. Two were marked held out and hidden from the coding agents that proposed the `SOUL.md` change. The post says to accept a change when both the development tasks and the held-out tasks improve. If the result is mixed, it says to run Compass on the new traces and keep the held-out cases hidden from the next edit. Those twelve runs (two tasks × three runs, twice) are the example, not a production suite. The post says production suites in their experience often hold hundreds to a few thousand examples. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) (arXiv:2611.06254, posted November 4, 2026) is the research version of the same habit at benchmark scale: search over harness code against a validation split, then score once on a held-out test set. The numbers in that paper stay on the research note. The design rule that travels is: the test the editor can see is not the test you use to claim a win. Google’s September 9 post is the other brake. A behavioral suite can pass while the user-facing job still fails. Anthropic’s [November 26, 2025 note](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents), already walked through on the evaluation guide, is that case: features marked done after unit tests or `curl` checks that missed a click path. Keep the small check, and keep an end-to-end score for the job you ship. Contamination is a third leak. NVIDIA’s September 21 post says web-searching agents can retrieve answer keys during evaluation, and that private domain evals avoid that by being unable to scrape. Harbor’s verifier docs keep tests out of the agent environment unless you choose shared verification. Inspect’s Docker sample is a new container. None of those is a claim about a vendor’s training data. ## What should you write down before the first run? Write down one failure, the check that would have caught it, how a trial is reset, and which cases stay hidden. 1. Name one failure you have already seen. Google’s dogfooding step and NVIDIA’s Compass step both start there. 2. Write the success rule in one sentence that a grader can execute. “A reservation row exists” is a rule. “It booked the flight well” is not. 3. Decide whether that rule is an action (tool call, file edit) or an outcome (tests pass, database row, merged pull request). Keep the other as a debug signal. 4. Say how a trial is reset. Anthropic’s git-history example is the failure mode to design against. 5. Keep at least one case out of the edit. NVIDIA’s held-out Harbor tasks are the short version. 6. Run more than one trial before you treat a pass rate as a gate. Google warns that a single run is noisy. NVIDIA’s September 21 table reports consistency as a range across 3–5 trials. 7. Hold the [agent harness](/glossary/harness) still when you compare models, or write down that you did not. The same-model lesson is in [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Live install and download counts for the libraries around this work sit on the [trackers](/trackers) and the [downloads tracker](/trackers/downloads). A token bill for the model inside a trial is the [agent cost calculator](/agent-cost-calculator). If you ship an evaluation product, [submit it](/submit) or claim its [directory](/directory) page. No login. ## Where can you read more on this site? - This guide’s pair: [How to evaluate an AI agent harness](/blog/how-do-you-evaluate-an-ai-agent-harness), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) - Terms: [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [behavioral evaluation](/glossary/behavioral-evaluation), [benchmark](/glossary/benchmark), [guardrail](/glossary/guardrail), [agent loop](/glossary/agent-loop) - Comparisons: [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools), [Best AI agent harness](/best/ai-agent-harnesses) - Report: [agent harness benchmark, November 2026](/reports/agent-harness-benchmark-november-2026) - Research: [HarnessFlow](/research/harnessflow-mcts-harness-optimization), [Garry Tan harness evals](/research/garry-tan-harness-evals-november-2026), [Terminal-Bench 2.1](/research/terminal-bench-2-1-benchmark-repairs) - At organization scale: [What is harness engineering at scale?](/blog/what-is-harness-engineering-at-scale-thoughtworks-radar) - Trackers: [live trackers](/trackers), [framework downloads](/trackers/downloads), [harness comparison tracker](/trackers/harnesses) - Topics: [agent harnesses hub](/topics/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance)

Frequently asked questions

### What is an eval harness for AI agents? It is the infrastructure that runs evals end to end. [Anthropic’s January 9, 2026 post](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) says it provides instructions and tools, runs tasks, records the steps, grades outputs, and aggregates results. The [agent harness](/glossary/harness) is the other layer, the one that lets the model act. ### How is an eval harness different from an agent harness? The agent harness is the loop, tools, memory, permissions, and stop rules around a model. The eval harness is the workshop that tests that system. Anthropic uses both phrases so the two jobs stay distinct. The longer definition of the workshop is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). ### What should the first task in an eval harness check? One failure you have already seen, with a grader that can read an environment state or a tool call. Google’s September 9, 2026 post starts with one missed action. NVIDIA’s November 4, 2026 post writes the first Harbor tasks from a stored send-without-confirmation failure. The cycle is [How to evaluate an AI agent harness](/blog/how-do-you-evaluate-an-ai-agent-harness). ### Do I need Harbor to build an eval harness? No. Harbor is one format, from the Terminal-Bench authors, with a task folder and a numeric reward. Inspect runs each sample in its own sandbox. Google’s September 9 post shows a local pytest assertion on a tool call. The sourced tool table is [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). ### Should an eval harness grade the transcript or the final state? Both, for different jobs. NVIDIA’s September 21, 2026 post uses step-level scoring to name where a chain broke, and end-to-end scoring to ask whether the environment reached the goal. It says most production evals gate the release on the outcome. ### How do you keep an eval harness from overfitting? Keep cases the editor did not see. NVIDIA hid two of four Harbor tasks from the agents that proposed a harness edit, and says to adopt a change when both the development set and the held-out set improve. Google says a behavioral check still does not replace a destination score. Dated public boards stay on the [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026). ## How to Get Your AI Tool Recommended by ChatGPT and Google (2026 Founder Guide) URL: https://www.aiagentsinfo.com/blog/how-to-get-your-ai-tool-recommended-by-chatgpt-and-google Published: 2026-11-09. Updated: 2026-11-09 Category: guide How to get your AI tool recommended by ChatGPT and Google: allow the search crawlers, publish unique pages they can cite, and treat a directory listing as a public URL — not a paid rank. OpenAI and Google both say eligibility is not a guarantee. Updated November 9, 2026. ## How do I get my AI tool recommended by ChatGPT and Google? You cannot buy a recommendation. OpenAI and Google both say that letting their crawlers in makes a page eligible to be shown. Eligibility is not a guarantee that [ChatGPT](/directory/chatgpt) or Google AI Overviews will name your product. That is the whole honest version. The rest of this page is what those companies have actually published, what this directory does with a [submission](/submit), and which free tools on this site you can run without an account. A paid badge, a “featured” seal, or a star rating invented for rich results is not a path this desk sells. The [editorial policy](/editorial-policy) says payment does not buy a rank. Google Search Central’s AI features page, last updated December 10, 2025 and read November 9, 2026, says a page can meet every requirement and still not be crawled, indexed, or served. If you just filed a tool, open the listing URL from the success screen, then [claim or update it](/submit) if a field is missing. The listing is a public page. It is not a ChatGPT setting. ## How does ChatGPT decide whether my site can appear in search answers? OpenAI’s crawler documentation, read November 9, 2026, says OAI-SearchBot is the bot that surfaces websites in ChatGPT’s search features. [OpenAI’s crawler overview](https://developers.openai.com/api/docs/bots) splits the jobs. OAI-SearchBot is for search. Sites that opt out of it “will not be shown in ChatGPT search answers, though can still appear as navigational links.” OpenAI recommends allowing OAI-SearchBot in `robots.txt` and allowing requests from the published IP ranges. It also says a `robots.txt` change can take about 24 hours to apply for search. GPTBot is not that search crawler. The same page says GPTBot crawls content that may be used to train generative AI foundation models. Disallowing GPTBot is a training opt-out. It is not an on/off switch for ChatGPT search. The two `robots.txt` settings are independent. A site can allow OAI-SearchBot and disallow GPTBot. ChatGPT-User is a third agent. OpenAI says it visits a page when a person in ChatGPT or a custom GPT asks it to. Those fetches are user-triggered. The page says `robots.txt` rules may not apply to them, and ChatGPT-User is not used to decide whether content appears in Search. Use OAI-SearchBot for search opt-outs. This site’s [robots.txt](/robots.txt) allows Googlebot, GPTBot, OAI-SearchBot, ChatGPT-User, and Google-Extended, among others. That is a choice for aiagentsinfo.com. Copying it onto your product domain is a decision you make with your counsel. Allowing those bots here does not make a new listing appear inside ChatGPT. ## Is GPTBot the crawler that gets me into ChatGPT search? No. GPTBot is the training crawler. OAI-SearchBot is the search crawler. OpenAI’s page treats the two `robots.txt` tags as independent.
User agent What OpenAI’s crawler page says it is for What a Disallow does, according to that page
OAI-SearchBot Surface websites in ChatGPT search features. The site is not shown in ChatGPT search answers. It can still appear as a navigational link.
GPTBot Crawl content that may be used in training generative AI foundation models. A Disallow indicates the content should not be used to train those models.
ChatGPT-User Visit a page when a person in ChatGPT or a custom GPT asks it to. Not used to decide Search inclusion. The page says robots.txt rules may not apply because the fetch is user-initiated.
The table is a restatement of OpenAI’s November 9, 2026 page. It is not a ranking of which bot “works better.” OpenAI also documents OAI-AdsBot for ads landing-page checks. That bot is for pages submitted as ads. It is not a directory listing feature, and this site does not sell ChatGPT ads. ## How does Google’s AI Overviews pick a supporting link? Google Search Central’s AI features page, last updated December 10, 2025, says a page must be indexed and eligible to be shown in Google Search with a snippet. There are no extra technical requirements for AI Overviews or AI Mode. The same page says the ordinary SEO work still applies: `robots.txt` must allow crawling, important facts should be in text, internal links should make the page findable, and structured data should match the visible words. It also says you do not need new machine-readable files, AI text files, or extra schema.org types for those Search AI features. [John Mueller’s May 21, 2025 Search Central post](https://developers.google.com/search/blog/2025/05/succeeding-in-ai-search) says Google wants unique, non-commodity content that helps the person who lands on the page. Commodity roundups that restate common knowledge are the example it warns against. The post also says Googlebot must not be blocked, the URL must return HTTP 200, and the page must have indexable content. Google does not promise a citation. The AI features page says meeting the requirements, following the policies, and writing helpful content still does not guarantee crawl, index, or serving. Two later Search Console notes are about measurement and opt-out, not about a secret ranking file: - On June 3, 2026, Google announced Search Generative AI performance reports in Search Console. A note on that post, read November 9, 2026, says those insights rolled out to all websites worldwide on August 31, 2026. The report lists impressions, pages, countries, devices, and dates. It is not a “recommended by Google” badge. - Search Console’s Search generative AI control, rolled out worldwide on August 31, 2026 according to Google’s help article, lets a site include or exclude its links and content from Search generative AI features. Include is the default. Exclude is not a ranking boost. It removes the site from those features. Google-Extended is a different control. The AI features page points to it for limiting AI training and grounding in some of Google’s other systems. It is not the switch for AI Overviews. Googlebot remains the crawler for Search. [Google’s crawler overview](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers), last updated June 12, 2026, says common crawlers such as Googlebot respect `robots.txt` for automatic crawls, and that by default crawlers only read the first 15 MB of a file. That 15 MB figure is Google’s published default, not a ranking score. ## Do I need an llms.txt file or special AI markup? Not for Google AI Overviews or AI Mode. Google’s AI features page says you do not need new machine-readable files, AI text files, or extra schema.org structured data for those features. An `/llms.txt` file is a separate proposal, published at [llmstxt.org](https://llmstxt.org/) and revised as v2 in August 2026. The spec is a Markdown file at `/llms.txt` that names the site, optionally summarizes it, and lists links to more Markdown. Agents that look for that file can use it. Google Search Central’s AI features page does not say it is a requirement for AI Overviews. This site publishes [`/llms.txt`](/llms.txt) and a longer [`/llms-full.txt`](/llms-full.txt). New guides and directory listings show up there on the next deploy. That helps agents that fetch this domain. It does not move a product into ChatGPT’s answers by itself. If you add `/llms.txt` on your own product site, keep it true. A file that claims a rank, a user count, or a review this desk did not write is still a claim on your domain. This guide will not invent those numbers for you. ## What should I publish on my own site so an assistant has something to cite? A public page that states, in ordinary sentences, what the product does, what it costs, and who it is for. Google’s May 21, 2025 post says to write unique pages, keep the main facts in text, and support them with images or video when that helps a person. OpenAI’s search crawler needs a URL it is allowed to fetch. Concrete pages that usually help a person, and therefore give a crawler something to quote: - A product page that is not only a slogan. - A pricing page with the list price you are willing to see quoted. If you submit that URL on [this directory](/submit), the listing can point at it. - A docs or FAQ page that answers the job a buyer would type into ChatGPT or Google, in the same words the buyer would use. - A changelog or “what it is not” page, so an assistant that cites you is less likely to invent a feature. Do not stuff the page with a keyword grid. Google’s post is explicit that the work is unique, people-first content. This desk’s [editorial policy](/editorial-policy) uses the same rule for its own articles: pages are not created to fill a keyword grid, and numbers stay off a page unless a source is linked. If you want a mechanical check on a prompt or a tool schema before you ship, use the [prompt and tool checker](/agent-prompt-checker). If you want a 30-day token bill from official per-token prices, use the [agent cost calculator](/agent-cost-calculator). Neither tool is a citation report. Both run with no login. ## How does a directory listing on this site help, and when does it stay out of search? A listing is a public URL on this domain, with your official site as the source link. Assistants that already read this site, or that are connected to the [directory MCP server](/mcp), can retrieve that URL. That is a citation path. It is not a ChatGPT ranking purchase. [Submit your tool](/submit) requires no account and no fee. A valid submission is published automatically and usually appears in about two minutes. Community listings are the maker’s text. The page says so. This desk has not written an independent review of those pages. Search inclusion on this site is stricter than “the form accepted it.” The [editorial policy](/editorial-policy#directory) says a slogan, a sentence that repeats the tagline, or unreadable English is not a real explanation. Unreadable submissions are not published. Readable but thin ones stay up for visitors and are marked so they are not offered as search results. The mechanical check in this repository is 80 words of description that is not the tagline repeated. That 80-word figure is a threshold this desk uses. It is not a Google ranking factor, and it is not a ChatGPT score. If your listing is live but thin, write a real explanation of the product and [email an update](/submit). There is still no login. A listing also sits in a category. The category page is alphabetical. Comparison tables on [Best AI agents](/best) are sourced rows, not a quality score. After you submit, the success screen and the listing page both show the category and any comparison table this desk already publishes for that category. If your product is not a row on that table yet, the page says so. Nothing on those screens is a paid slot. ## How do I update the listing after I submit, without a login? There is no account. Open your listing, use **Claim / update this listing**, and email the desk. The same form is `/submit?claim=` plus your slug. The listing page now shows a **Complete your listing** card. It reads five fields that already exist on the page: - Screenshots, meaning a product image in the listing body. A logo is not a screenshot. - Demo video, meaning a YouTube (or similar) link on file. - Pricing, meaning a starting price, a pricing-page URL, or a free / open-source model. - Use cases, meaning at least one use the directory can filter on. - Links, meaning a public X, LinkedIn, GitHub, YouTube, or similar profile besides the website. Each missing line opens a prefilled email. Do not send passwords, private metrics, or other secrets. The [submit page](/submit) already warns against that. If the listing is not yours, do not claim it. If it is yours and a fact is wrong, the email is the edit path. ## What free checks can I run before I spend on ads? Three tools on this site run without a login, and none of them is a ChatGPT or Google ranking report. - [Which AI agent fits me?](/tools/agent-picker) asks four questions and returns three directory listings that match. Use it to see which products this desk would surface for a buyer in your category. - [AI agent cost calculator](/agent-cost-calculator) estimates a 30-day model bill from official per-token prices. It does not add a seat fee. - [Prompt and tool checker](/agent-prompt-checker) counts tokens and checks tool JSON. The prompt is not stored. For how an assistant thinks about a product in this category, read [How do AI agents work?](/blog/how-do-ai-agents-work). For how a serious buyer tests a runtime, read [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). Both are existing guides on this site. They do not claim that publishing a listing will make ChatGPT or Google name you. The useful next step is still the boring one: a unique public page, crawlers allowed, a listing that is a real explanation, and an email if a field is missing. That is what the official crawler docs actually support. Everything else is a pitch. ## Frequently asked questions ### How do I get my AI tool recommended by ChatGPT and Google? You cannot buy a recommendation. OpenAI and Google both say that letting their crawlers in makes a page eligible to be shown. Eligibility is not a guarantee that ChatGPT or Google AI Overviews will name your product. ### How does ChatGPT decide whether my site can appear in search answers? [OpenAI’s crawler documentation](https://developers.openai.com/api/docs/bots), read November 9, 2026, says OAI-SearchBot surfaces websites in ChatGPT’s search features. Sites that opt out of it are not shown in ChatGPT search answers, though they can still appear as navigational links. A `robots.txt` change can take about 24 hours to apply for search. ### Is GPTBot the crawler that gets me into ChatGPT search? No. GPTBot is the training crawler. OAI-SearchBot is the search crawler. The two `robots.txt` settings are independent. ### How does Google’s AI Overviews pick a supporting link? Google Search Central’s [AI features page](https://developers.google.com/search/docs/appearance/ai-features), last updated December 10, 2025, says the page must be indexed and eligible for a snippet in Google Search. There are no extra technical requirements. Indexing is not guaranteed. ### Do I need an llms.txt file or special AI markup? Not for Google AI Overviews or AI Mode. Google’s page says you do not need new machine-readable files or extra schema.org markup for those Search AI features. [`/llms.txt`](https://llmstxt.org/) is a separate proposal for agents. This site publishes [its own copy](/llms.txt). ### What should I publish on my own site so an assistant has something to cite? A unique public page, in text, that says what the product does, what it costs, and who it is for. Google’s May 21, 2025 post is the source for “unique, non-commodity” and for keeping Googlebot unblocked. ### How does a directory listing on this site help, and when does it stay out of search? It is a public URL with your official site attached. Thin community blurbs stay up for visitors and stay out of this site’s sitemap. The mechanical check is 80 words that are not the tagline. [Submit](/submit) has no login. Payment does not buy a rank. ### How do I update the listing after I submit, without a login? Use [Claim / update](/submit) with your slug. The listing page’s completeness card names screenshots, demo video, pricing, use cases, and links, and prefills an email for anything missing. ### What free checks can I run before I spend on ads? The [agent picker](/tools/agent-picker), [cost calculator](/agent-cost-calculator), and [prompt checker](/agent-prompt-checker). No login. None of them is a ChatGPT or Google ranking report. ## Is AutoGen Deprecated? 61,753 Stars on an Unarchived Repo URL: https://www.aiagentsinfo.com/blog/is-autogen-deprecated Published: 2026-11-09. Updated: 2026-11-09 Category: explainer 61,753 GitHub stars on November 9, 2026, and the API field archived was false. AutoGen’s README, changed October 16, says Microsoft is not providing bug fixes, security fixes, new features, or support. AutoGen’s README says the project is archived. GitHub’s archive flag, read at the end of a pass at 07:41:51 UTC on November 9, 2026, was false. The repository had **61,753** stars. Those two sentences are the whole disagreement. A README can say “archived” while the API field `archived` stays false. AI Agents Info read both on the same morning. The [November 5 framework snapshot](/reports/ai-agent-framework-github-stats-november-2026) already had this repository in a table of 19, at 61,706 stars, and the public page does not answer the maintenance question. This page does. ## Is AutoGen deprecated? Microsoft’s current README says the project is archived and no longer maintained. GitHub’s repository record, the same morning, said it was not archived. The README on `main`, fetched November 9, opens the caution with “Project archived and no longer maintained.” The next sentences say Microsoft is not providing bug fixes, security fixes, new features, or support, and that the repository remains available for historical reference. A later sentence says new users should start with Microsoft Agent Framework, and points existing users at the [migration guide](https://learn.microsoft.com/en-us/agent-framework/migration-guide/from-autogen/). The API response for `GET /repos/microsoft/autogen` in that pass returned `"archived": false`. It also returned 61,753 stars, 9,392 forks, `open_issues_count` 1,172, the SPDX id CC-BY-4.0, Python as the primary language, `created_at` 2023-08-18T11:43:45Z, and `pushed_at` 2026-10-28T14:45:27Z. The latest-release call returned tag `python-v0.7.5`, published 2025-09-30T06:18:26Z. A search request later the same morning, still November 9, split that 1,172. It counted 572 open issues and 600 open pull requests. 572 plus 600 is 1,172, so the split matched the repository field. GitHub’s `open_issues_count` includes pull requests. The 1,172 figure is not 1,172 bug reports. | Repository | Stars | Forks | open_issues_count | Archived field | Latest release | Release published | Last push | | --- | --- | --- | --- | --- | --- | --- | --- | | microsoft/autogen | 61,753 | 9,392 | 1,172 | false | python-v0.7.5 | Sep 30, 2025 | Oct 28, 2026 | | microsoft/semantic-kernel | 28,747 | 4,857 | 329 | false | dotnet-1.82.0 | Nov 7, 2026 | Nov 7, 2026 | | microsoft/agent-framework | 14,598 | 2,596 | 905 | false | python-1.25.0 | Nov 7, 2026 | Nov 9, 2026 | 61,753 minus 14,598 is 47,155. The repository whose README says Microsoft will not ship fixes had 47,155 more stars than the repository Microsoft’s own pages name as the successor. The [November 5 snapshot](/reports/ai-agent-framework-github-stats-november-2026) recorded AutoGen at 61,706 stars, 9,367 forks, and `open_issues` 1,144, with the same release tag and the same `pushed_at`. The archive boolean in that file was already false. Four days later the star field was 47 higher. The release tag had not moved. The [November 9 framework comparison](/best/ai-agent-frameworks) lists the same release, python-v0.7.5 on September 30, 2025, and calls the row an older Microsoft framework. It does not quote the README. ## When did the README change from maintenance mode to archived? The archived wording landed in the README on October 16, 2026. On April 6, 2026 the same file said maintenance mode. Commit [004749e6b7](https://github.com/microsoft/autogen/commit/004749e6b7102c8fd317b9b1499ce5807223c278), committer date 2026-10-16T16:14:36Z, is titled “docs: clarify AutoGen is archived and no longer maintained.” The README at the previous banner commit, [027ecf0a37](https://github.com/microsoft/autogen/commit/027ecf0a379bcc1d09956d46d12d44a3ad9cee14), committer date 2026-04-06T22:35:32Z, used a maintenance-mode badge. Its caution said AutoGen was in maintenance mode, would not receive new features or enhancements, and was community managed going forward. It already sent new users to Microsoft Agent Framework. The October 16 text is stricter. It names bug fixes, security fixes, new features, and support as things Microsoft is not providing. It tells the reader to use an archived repository with caution, and it says AutoGen was developed for research and experimentation. The April text did not say that. The October 16 commit was 23 days and about 15 hours before the API pass. The wording a person meets on the front of the repository today is the October wording, plus whatever else is on `main`. The April file is the one to open if you want the earlier banner. ## What has been committed since that notice? The last push GitHub recorded was October 28, 2026, and the subject line of that commit is documentation. These are the five newest commits the API returned for the default branch: | Commit | Committer date | Subject | | --- | --- | --- | | d07c667a32 | Oct 28, 2026 | docs: add archive caution to landing page (#8393) | | 004749e6b7 | Oct 16, 2026 | docs: clarify AutoGen is archived and no longer maintained (#8366) | | 027ecf0a37 | Apr 6, 2026 | Update maintenance mode banner in readme (#7521) | | 8544314fa6 | Mar 26, 2026 | fix: restrict importlib provider loading to trusted namespaces (#7463) | | b0477309d2 | Mar 11, 2026 | fix: Improve AutoGen Studio: deprecate FunctionTool, harden MCP WebSocket endpoint (#7362) | Of those five, the newest subject that is not a documentation banner is the March 26 fix. This page did not read the diffs. A commit titled “docs” can still touch code, and a commit titled “fix” can be small. The subjects are the evidence. The latest release tag is older than all five: python-v0.7.5, September 30, 2025, which was 405 days before the API pass. The repository description GitHub returned was still “A programming framework for agentic AI.” The homepage field was `https://microsoft.github.io/autogen/`. Neither field carried the archive sentence. The sentence lives in the README and, as of the October 28 commit, on the landing page that commit names. ## Is Semantic Kernel in the same position? No. Semantic Kernel’s README points people at Agent Framework, and the repository is still publishing releases. The README’s notice, read the same day, says “Semantic Kernel is now Microsoft Agent Framework” and calls that framework the enterprise-ready successor. It links a migration guide. It does not say Microsoft has stopped bug fixes, and it does not use the word archived. The API agreed with the release history. `archived` was false. Stars were 28,747. Forks were 4,857. `open_issues_count` was 329. The license field was MIT. The primary language was C#. `pushed_at` was 2026-11-07T00:59:55Z. The latest release was `dotnet-1.82.0`, published 2026-11-07T01:00:07Z, twelve seconds after that push. `pushed_at` moves when any ref is pushed, including a release tag, so that clock is not by itself the date of the newest commit on `main`. The newest commit the API returned for `main` was “.Net: Bump package version to 1.82.0,” committer date November 5, 2026. The one before it prepared a Python 1.47.0 release. 28,747 minus 14,598 is 14,149. Semantic Kernel still had 14,149 more stars than Agent Framework, and it had shipped a release two days before this reading. AutoGen’s README and Semantic Kernel’s README are both sending new work toward the same successor. Only AutoGen’s README says Microsoft is not providing fixes. ## What does Microsoft say to use instead? Agent Framework, in both the AutoGen README and the Learn overview. The [overview](https://learn.microsoft.com/en-us/agent-framework/overview/), read November 9, 2026, says Agent Framework combines AutoGen’s agent abstractions with Semantic Kernel’s enterprise features, and then says the framework is the direct successor, created by the same teams. A later sentence says it is the next generation of both Semantic Kernel and AutoGen. The page also says that if you can write a function for the task, do that instead of using an agent. The same page describes a Harness Agent for long tasks, with planning, context compaction, file access, memory, tool approval, and observability. On this site that surrounding software is a [harness](/glossary/harness). The directory page is [Microsoft Agent Framework](/directory/microsoft-agent-framework). The successor’s own repository, in the same API pass, had 14,598 stars and 2,596 forks. `archived` was false. The license field was MIT. The description was “A framework for building, orchestrating and deploying AI agents and multi-agent workflows with support for Python and .NET.” The latest release tag was `python-1.25.0`, published 2026-11-07T15:52:42Z. `pushed_at` was 2026-11-09T06:38:42Z, about an hour before the AutoGen read. The [November 5 snapshot](/reports/ai-agent-framework-github-stats-november-2026) had this repository at 14,539 stars and release python-1.24.0, published October 30, 2026. The tag moved. The star count moved by 59. The AutoGen README also contains the sentence “Microsoft Agent Framework 1.0 gives you enterprise-grade multi-agent orchestration.” The release tag GitHub returned for that repository on November 9 was python-1.25.0, not a tag named 1.0. This page treats 1.25.0 as the release tag and treats “1.0” as a phrase in the README. The README sentence as fetched also contains the typo “FrameworkAF in now.” That typo is in the file. It is not a second product. The migration guide’s background section says Agent Framework is a new multi-language SDK and that it was developed by the core AutoGen and Semantic Kernel teams. One difference it names: AutoGen’s `AssistantAgent` is single-turn unless `max_tool_iterations` is raised, and Agent Framework’s `Agent` keeps invoking tools until it can return an answer. Another: AutoGen’s high-level `Team` sits on an event-driven core, and Agent Framework centers on a typed graph workflow. Those are the guide’s distinctions. This page did not port a project. ## Why do the framework star pages skip this? They are release tables. The question they rank for is which framework shipped, and which repository has the stars. Maintenance is a different question, and it is the one a person asks before installing `autogen-agentchat`. The November 5 report copies stars, forks, issues, license, language, creation, and the latest release for 19 repositories. AutoGen is one row. LangChain led that snapshot at 149,047 stars. AutoGen’s 61,706 was enough to sit in the table and not enough to force a sentence about the README. The November 9 comparison reads the license and the latest tag again. AutoGen’s tag had not changed since September 30, 2025, so the row looks quiet, and the page says so. Quiet is not the same sentence as “Microsoft says it is not providing security fixes.” The practical stake is the star count. 61,753 stars still sat on a repository whose own front page says there will be no security fixes. A star is an account’s bookmark, and it does not come off when a README changes. A tutorial from 2024 still finds the package. The install block is still in the README, under the caution, including `pip install -U "autogen-agentchat"`. The caution is above the install. Both are on the page. The count of stars on the successor is the other stake. Agent Framework has the commits and the November 7 release. AutoGen has the reputation, measured the only way this page measured it: 61,753 stars against 14,598. How an agent framework fits the loop around a model is [How do AI agents work?](/blog/how-do-ai-agents-work). The harness definition is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Both sit on the [agent harnesses hub](/topics/harnesses). A separate November 9 count, of stars on agent products rather than on these three libraries, is [How many GitHub stars does OpenClaw have?](/research/how-many-github-stars-does-openclaw-have). ## FAQ ### Is AutoGen deprecated? Microsoft’s README says the project is archived and no longer maintained, and that Microsoft is not providing bug fixes, security fixes, new features, or support. At 07:41 UTC on November 9, 2026, GitHub’s repository API still returned `archived: false`. The latest release was python-v0.7.5, published September 30, 2025. The repository had 61,753 stars. ### Did GitHub archive the AutoGen repository? The `archived` field was false. That field is the archive switch. The README, and the October 28 commit that adds an archive caution to the landing page, are text in the repository. They do not flip the switch by themselves. ### When did AutoGen’s README start saying archived? On October 16, 2026, in commit 004749e6b7. The README at the April 6, 2026 banner commit said maintenance mode, and that the project would not receive new features and was community managed going forward. ### Is Semantic Kernel deprecated too? Its README says Semantic Kernel is now Microsoft Agent Framework. The API the same morning returned `archived: false`, 28,747 stars, and release dotnet-1.82.0, published November 7, 2026. The README does not say Microsoft has stopped fixes. ### What does Microsoft say to use instead of AutoGen? The AutoGen README and the Learn overview both name Microsoft Agent Framework. The overview calls it the direct successor of AutoGen and Semantic Kernel. The successor’s repository had 14,598 stars, and its latest release was python-1.25.0, published November 7, 2026. ## Nvidia board investigation: Jensen Huang stays CEO while directors review the reports URL: https://www.aiagentsinfo.com/blog/nvidia-board-investigation-jensen-huang Published: 2026-11-09. Updated: 2026-11-09 Category: news On November 8, 2026, Nvidia’s board opened an independent investigation into reporting that Jensen Huang was the target of a foreign intelligence effort. The company says he remains CEO. On November 8, 2026, NVIDIA’s board said it is opening an independent investigation after reporting that founder and CEO Jensen Huang was the target of a foreign intelligence effort. The same day, NVIDIA published an email in which Huang wrote that he was unfaithful to his wife, that he supports the review, and that he is fully committed to the company. NVIDIA told CNBC he is committed to continuing as CEO without interruption. The board said it has no reason to believe that national security or NVIDIA secrets were compromised. This page was published and updated on November 9, 2026. The pages cited here were read that day. ## What did Nvidia’s board announce on November 8? NVIDIA’s board opened an independent investigation and said it has no reason to believe that national security or company secrets were compromised. The [statement](https://nvidianews.nvidia.com/news/statement-from-nvidia-board-of-directors), dated November 8, 2026, is six sentences. It says the board “is initiating an independent investigation in light of recent reporting that founder and CEO Jensen Huang has been the target of a foreign intelligence effort.” A committee of independent directors, “assisted by outside independent counsel,” will oversee it. The board says U.S. national and economic security comes first for NVIDIA and its shareholders. Then the limit on what it is claiming: “While the Board has no reason to believe that either national security or NVIDIA secrets were compromised, the Board's inquiry will examine all available information.” Huang “fully supports the review and has committed to cooperating with it.” The board “has confidence in Jensen and his leadership team and will provide further updates as appropriate.” The statement does not name another person, a company, or a news outlet. It does not say the inquiry has finished. It does not remove Huang as CEO. CNBC, in an [article published the same day](https://www.cnbc.com/2026/11/08/nvidia-jensen-huang-email-board-of-directors-statement.html), quoted a company statement that is not in the board text: “Today's news will not slow NVIDIA. Jensen is committed to continuing to lead NVIDIA as CEO without interruption.” ## What did Jensen Huang write to employees? Huang wrote that he was unfaithful to his wife, that he supports the investigation, and that he is fully committed to NVIDIA. NVIDIA posted the email on its newsroom under [A Message to Our Company](https://nvidianews.nvidia.com/news/a-message-to-our-company), also dated November 8, 2026. The newsroom line above the text says Huang sent it to employees earlier that day. CNBC printed the same message, with a subject line “A message to our company” and the date Nov. 8, 2026. The email says: “I was unfaithful to my wife. My actions hurt her and our children. There is no excuse for what I did.” It says he takes full responsibility and asks that his family be given privacy. It says leading NVIDIA is “the privilege of my life,” that he has fallen short, and that “I am fully committed to NVIDIA and will work to earn back your confidence.” On the investigation, the email says: “Regarding the national security concerns raised in news reports, NVIDIA’s Board of Directors has initiated an independent investigation. I support the investigation and will cooperate fully.” It also says he is committed to protecting NVIDIA and to supporting America’s national security. The published email names no other person. It does not describe a foreign government, a customer, or a transfer of information. ## Who is Alice Huang in the public record? A November 4 House letter calls Alice Huang, a Megaspeed executive, a publicly reported Chinese intelligence asset, and asks Jensen Huang about his interactions with her. His published email does not use her name. The letter is a primary document. It is dated November 4, 2026, addressed to Jensen Huang at NVIDIA’s Santa Clara office, and signed by Brian J. Mast, chairman of the House Committee on Foreign Affairs; John Moolenaar, chairman of the House Select Committee on the CCP; and Eric A. “Rick” Crawford, chairman of the House Permanent Select Committee on Intelligence. The [PDF](https://chinaselectcommittee.house.gov/sites/evo-subsites/selectcommitteeontheccp.house.gov/files/evo-media-document/nvidia-letter-pdf.pdf) asks for a briefing on “NVIDIA’s security protocols against foreign adversary espionage and your interactions with a publicly reported People’s Republic of China (PRC) intelligence asset: Megaspeed executive Alice Huang.” That phrase is the letter’s description of prior reporting. The letter is not a court finding, and the November 8 board statement does not use Alice Huang’s name. Two news accounts, both read on November 9, fill in what those “recent reports” have said. They attribute the underlying claims to other publications and to a memo, not to a NVIDIA finding. [TechCrunch, posted November 7, 2026](https://techcrunch.com/2026/11/07/chinese-exec-was-recruited-to-spy-on-nvidia-ceo-according-to-us-intelligence-memo/), says a memo circulated earlier this year claimed that Alice Huang was recruited by China’s Ministry of State Security to spy on Jensen Huang, “according to multiple media reports.” TechCrunch says The New York Times was the first to report on that memo, and that the Financial Times later reported the memo also claimed a romantic relationship between Jensen Huang and Alice Huang, an executive at Singapore-based Megaspeed. TechCrunch says an NVIDIA spokesperson told the Times that Jensen Huang’s interactions with Alice Huang were “not unusual,” and that the company does not share classified information with its customers. That spokesperson comment predates the November 8 email. The email is the later, on-the-record personal statement. [The Verge, November 6, 2026](https://www.theverge.com/tech/1029338/nvidia-ceo-jensen-huang-china-spy), summarizing the Times, says the Ministry of State Security enlisted her to gather information about American AI chips and “potentially influence the company to push for its most advanced chips to be made available to China.” The Verge says the Times described the relationship as “close,” and that the two Huangs are not related. This page has not independently read the Times memo story. The claims above are what TechCrunch and The Verge attribute to that reporting and to the Financial Times. NVIDIA’s November 8 board text uses a narrower sentence: reporting that Huang “has been the target of a foreign intelligence effort.” ## What has Congress asked Jensen Huang to do? Three House chairs asked him to give a closed-door briefing on November 18, 2026, or to propose another session day by November 11. The November 4 letter asks Huang to appear in room 2172 of the Rayburn House Office Building on November 18, 2026. It asks him to accept that date, or to offer another date that falls on a session day of the 119th Congress, by November 11, 2026. The [November 5 press release](https://foreignaffairs.house.gov/news/press-releases/chairmen-mast-moolenaar-and-crawford-request-jensen-huang-brief-congress-on-link-to-chinese-intelligence-asset) from the House Foreign Affairs Committee repeats those two dates and the same description of Alice Huang. The letter was already public before this weekend. It matters on November 9 because the deadline to accept November 18, or to propose another session day, is November 11, and because the board investigation announced on November 8 now sits beside that request. This page has no NVIDIA reply that accepts November 18 or offers another day. ## What did President Trump say? Axios reported that, on Sunday night, Trump told reporters he had not heard that Huang had done anything wrong, and that he would certainly take Huang on an Asia trip. In an [article dated November 9, 2026](https://www.axios.com/2026/11/09/trump-jensen-huang-nvidia), Ben Berkowitz writes that Trump was asked Sunday night whether he is taking Huang on an Asia trip next week. Axios quotes Trump telling reporters: “I don't know, but I'd certainly take him.” Axios then quotes two more lines: “I haven't heard that he's done anything wrong,” and “He dealt with somebody but, you know, so does everybody else.” Those sentences are Axios’s account of remarks to reporters. This page does not have a separate transcript. Axios frames the board review as an investigation into an affair with a suspected Chinese spy. The board’s own November 8 wording is the “foreign intelligence effort” sentence quoted above. Huang’s published email admits an affair and names no one else. ## When is Nvidia’s next earnings call? NVIDIA set its third-quarter fiscal 2027 call for Tuesday, November 17, 2026, at 2 p.m. Pacific time, the day before the briefing date in the House letter. The [October 28, 2026 notice](https://nvidianews.nvidia.com/news/nvidia-sets-conference-call-for-third-quarter-financial-results-6928873) says the call covers the third quarter of fiscal year 2027, which ended October 25, 2026. The webcast is listen-only on investor.nvidia.com. Questions are limited to financial analysts and institutional investors. Written commentary from Colette Kress, executive vice president and chief financial officer, is to be posted after results are announced at about 1:20 p.m. Pacific time. The same notice calls NVIDIA “the world leader in AI and accelerated computing.” The November 18 briefing in the House letter is the calendar day after that call. The letter does not mention earnings. The two dates are simply adjacent.
Date What is on the record
October 28, 2026 NVIDIA schedules the third-quarter fiscal 2027 call for November 17, 2 p.m. Pacific time.
November 4, 2026 Mast, Moolenaar, and Crawford ask Huang for a closed-door briefing on November 18, and for an answer by November 11.
November 5, 2026 The House Foreign Affairs Committee publishes that request.
November 6, 2026 The Verge, citing the Times, describes a reported effort to learn about American AI chips and to influence sales of Nvidia’s most advanced chips to China.
November 7, 2026 TechCrunch reports the memo claim and a later Financial Times report of a romantic relationship, plus an earlier NVIDIA comment that the interactions were “not unusual.”
November 8, 2026 The board opens the investigation. NVIDIA publishes Huang’s email. NVIDIA tells CNBC he will continue as CEO without interruption.
Sunday night, reported by Axios on November 9 Axios quotes Trump saying he has not heard that Huang did anything wrong, and that he would certainly take him on an Asia trip.
November 11, 2026 The House letter’s deadline to accept November 18 or propose another session day.
November 17 and 18, 2026 Earnings call, then the briefing date the chairs requested.
## Why does this Nvidia investigation matter for AI? NVIDIA builds the accelerated computers that train and serve the models inside AI products, and the reporting behind this review is about access to those chips. An [agent](/glossary/agent), on this site, is a [language model](/glossary/llm) plus the software that offers tools and brings results back. The longer account is [How do AI agents work?](/blog/how-do-ai-agents-work). NVIDIA’s October 28 notice describes the company as the world leader in AI and accelerated computing. The Verge’s November 6 summary of the Times says the reported task was to learn about American AI chips and, potentially, to push for the most advanced of those chips to be sold to China. That is a question about who can buy the machines the models run on. The November 8 record draws a line under what NVIDIA is willing to say while that question is open. The board says it has confidence in Huang and no reason to believe secrets were compromised, and it has started a review by independent directors and outside counsel. Huang says he will cooperate and keep leading. Congress has asked for a closed-door briefing and has not, on the documents checked for this page, received a published yes. None of those sentences is a finding about what, if anything, was disclosed. ## FAQ ### What is the Nvidia board investigation? On November 8, 2026, NVIDIA’s board said it is opening an independent investigation because of reporting that founder and CEO Jensen Huang was the target of a foreign intelligence effort. A committee of independent directors, with outside counsel, will run it. The board said it has no reason to believe national security or NVIDIA secrets were compromised, and that it has confidence in Huang and his leadership team. ### Is Jensen Huang still Nvidia’s CEO? NVIDIA told CNBC on November 8, 2026, that Huang is committed to continuing to lead NVIDIA as CEO without interruption. His email the same day says he is fully committed to NVIDIA. The board said it has confidence in him. The investigation is open, and the board said it will give further updates as appropriate. ### Did Jensen Huang’s email name Alice Huang? The email NVIDIA published on November 8, 2026, says Huang was unfaithful to his wife. It names no other person. A November 4 letter from three House committee chairs asks him to brief Congress on his interactions with Megaspeed executive Alice Huang, whom the letter calls a publicly reported PRC intelligence asset. ### Did Nvidia say secrets were taken? NVIDIA’s board said on November 8, 2026, that it has no reason to believe national security or NVIDIA secrets were compromised, and that the inquiry will examine all available information. The board has not published findings. ### When does Congress want a briefing from Jensen Huang? The November 4, 2026, letter asks Huang to brief the committees on November 18, 2026, in room 2172 of the Rayburn House Office Building, and to accept that date or offer another session day by November 11, 2026. This page has no published reply from Huang accepting the date. ### When is Nvidia’s next earnings call? NVIDIA said on October 28, 2026, that it will discuss third-quarter fiscal 2027 results on Tuesday, November 17, 2026, at 2 p.m. Pacific time. That is the day before the briefing date in the House letter. ## What Does Ultracode Do in Claude Code? It Plans a Workflow at Any Effort (2026) URL: https://www.aiagentsinfo.com/blog/what-does-ultracode-do-in-claude-code Published: 2026-11-09. Updated: 2026-11-09 Category: guide Ultracode plans a dynamic workflow for each substantive Claude Code task, at the effort level the session already uses. Claude Code v2.1.284, dated September 28, 2026, is the release that stopped forcing that session to xhigh. What does ultracode do in Claude Code? It plans a dynamic workflow for each substantive task, at the effort level the session is already using. On the [workflows page](https://code.claude.com/docs/en/workflows) read November 9, 2026, `/effort ultracode` is the command that turns that setting on for the current session, and the [changelog](https://code.claude.com/docs/en/changelog) dates the split from a forced `xhigh` effort at Claude Code v2.1.284, September 28, 2026. The pages read for this note are that workflows guide, the [model-config page](https://code.claude.com/docs/en/model-config), the [settings reference](https://code.claude.com/docs/en/settings-reference), and the changelog entries for v2.1.284 and v2.1.160. A [Hacker News thread](https://news.ycombinator.com/item?id=48314397) asks the short form, “What is ultracode mode?” A [Reddit thread](https://www.reddit.com/r/ClaudeCode/comments/1u5d69n/any_reason_not_to_use_ultracode/) asks whether there is a reason to leave it off. The answers below are the docs, not the comment threads. Ultracode sits inside the [harness](/glossary/harness): the loop, tools, and permissions around the model. The part that decides the next step is [orchestration](/glossary/orchestration). The product page on this site is [Claude Code](/directory/claude-code). ## What does ultracode do in Claude Code? With ultracode on, Claude writes a workflow script for a substantive task and a runtime runs that script in the background, while the session stays usable. The workflows page defines a dynamic workflow as a JavaScript script that orchestrates many subagents at once. Claude writes the script for the task you describe. The runtime executes it. The page’s examples of work that fits are a codebase-wide bug sweep, a 500-file migration, a research question whose sources need cross-checking, and a plan drafted from several independent angles. The same page says who holds the plan. With subagents, Claude decides turn by turn, and every intermediate result lands in Claude’s context window. A workflow script holds the loop, the branching, and the intermediate results, so Claude’s context holds the final answer. The scale row in that comparison says workflows run dozens to hundreds of agents per run. Subagents, on the same row, are a few delegated tasks per turn. | | Subagents | A workflow ultracode plans | | --- | --- | --- | | Who decides what runs next | Claude, turn by turn | The script | | Where intermediate results live | Claude’s context window | Script variables | | Scale, in the docs’ words | A few delegated tasks per turn | Dozens to hundreds of agents per run | | If you interrupt | Restarts the turn | Resumable in the same session | That table is the “When to use a workflow” comparison on the workflows page, read November 9, 2026, cut to the four rows that answer what ultracode changes. The page also says a workflow can have independent agents review each other’s findings before they are reported. One bundled command is separate from the setting. `/deep-research` is a built-in workflow. It runs when you invoke it. Ultracode is the switch that lets Claude plan a workflow without that kind of request. ## What is ultracode in Claude Code? Ultracode is a Claude Code setting. The model-config page says it with this sentence: “Ultracode is a Claude Code setting rather than a model effort level.” The effort table on that page, read November 9, puts `ultracode` on its own row, after `low`, `medium`, `high`, `xhigh`, and `max`. The row’s text is: “A Claude Code setting rather than a level: plans a dynamic workflow for each substantive task at any effort level.” The settings reference describes the saved key the same way. `"ultracode": true` starts later sessions with ultracode on, when dynamic workflows are enabled and the model supports `xhigh` effort. The key does not change the session’s effort level. The default is unset, so ultracode is off. Claude Code reads the key and does not write it: `/effort ultracode` is the current session only. The word itself has a date. Changelog v2.1.160, dated June 2, 2026, says the dynamic-workflow trigger keyword was renamed from `workflow` to `ultracode`. After that release, the word “workflow” no longer starts a run. Asking for a workflow in your own words still does. ## Is ultracode the same as max effort? `max` and ultracode are different rows on the model-config effort table. `max` is the deepest reasoning level. ultracode is the workflow setting, and since v2.1.284 it can stay on at `low`, `medium`, `high`, `xhigh`, or `max`. The changelog line for v2.1.284, September 28, 2026, is the one that changed the old behavior: “Changed Ultracode into its own toggle in `/effort` (Tab, or `/effort ultracode [on|off]`): it no longer forces xhigh effort and stays on at any effort level.” The same release notes an Ultracode on/off switch under the Effort slider in VS Code, and says the model pill shows “· Ultracode” at any effort level. Two launch paths still set `xhigh` at the same time. The model-config page says `claude --effort ultracode` starts the session at `xhigh` with ultracode on. An Agent SDK `effortLevel: "ultracode"` value does the same pair of things. Passing `ultracode` to `--effort` or to that SDK value requires v2.1.203 or later. Before v2.1.203, the page says `--effort ultracode` printed `Unknown --effort value 'ultracode'` and the session started at the default effort. Before v2.1.284, the model-config page says turning ultracode on set the session to `xhigh`, picking another level turned ultracode off, and an effort cap below `xhigh` made ultracode unavailable. The workflows page adds one error string from that older behavior: `/effort ultracode off` failed with `Invalid argument`. Picking a level in the `/effort` slider or the `/model` picker, on the current page, leaves ultracode as it was. The saved `effortLevel` key and the `CLAUDE_CODE_EFFORT_LEVEL` environment variable do not accept the word `ultracode`. If either of those sets the level, ultracode stays on at that level. ## How do you turn ultracode on? Type `/effort ultracode` for this session, press Tab on the Ultracode toggle in the `/effort` slider, or set `"ultracode": true` so later sessions start with it on. `claude --effort ultracode` is the launch form, and that one also sets effort to `xhigh`. | Control, as the docs name it | What it covers | Effort level | | --- | --- | --- | | `/effort ultracode` | This session. v2.1.284 or later keeps it on at the current level | Leaves the level where it is | | Tab on the `/effort` slider, then Enter | This session, same toggle | Leaves the level where it is | | `"ultracode": true` in a settings file | Later sessions, when workflows are enabled and the model supports `xhigh` | Leaves the level where it is, on v2.1.284 or later | | `claude --effort ultracode` | The session you just launched. Requires v2.1.203 or later | Sets the level to `xhigh` | | Agent SDK `effortLevel: "ultracode"` | That SDK request. Requires v2.1.203 or later | Sets the level to `xhigh` | | The word `ultracode` in a prompt you type | One workflow for that prompt. The session setting stays as it was | Does not change the session level | The word in a prompt is a one-task opt-in. The workflows page says Claude Code highlights the keyword, and Claude writes a workflow for that task. Asking “use a workflow” or “run a workflow” is the same opt-in. The keyword works in a prompt you type at the interactive prompt, in an IDE extension panel, in a Remote Control client, or in an Agent SDK application that stamps the input as human. It does not start a workflow from `claude -p`, from an SDK prompt that is not stamped as human, from a scheduled task, or from a webhook or pull-request comment. Option+W on macOS, or Alt+W on Windows and Linux, dismisses the highlight for that prompt. The settings key `workflowKeywordTriggerEnabled` defaults to true. Set it to false to type the word without starting a run. In `/config` that key is labeled Ultracode keyword trigger. Ultracode is unavailable in two cases the model-config page names: workflows are turned off, or the model does not support `xhigh` effort. In those cases `--effort ultracode` starts the session with ultracode off, at the highest effort level the model and any cap allow, up to `xhigh`. The effort table on that page lists `xhigh` for Fable 5.1, Fable 5, Opus 5.5, Sonnet 5.5, Haiku 5.5, Opus 5, Sonnet 5, Opus 4.8, and Opus 4.7. The Opus 4.6 and Sonnet 4.6 row lists `low`, `medium`, `high`, and `max`. ## Which Claude plans include ultracode? Dynamic workflows, which ultracode turns on, are on all paid plans, on the Anthropic API, and on Amazon Bedrock, Google Cloud’s Agent Platform, and Microsoft Foundry. On Pro, the workflows page says to turn them on from the Dynamic workflows row in `/config`. Turning workflows off removes ultracode with them. The page lists three ways to turn workflows off for yourself: the Dynamic workflows toggle in `/config`, `"disableWorkflows": true` in `~/.claude/settings.json`, and `CLAUDE_CODE_DISABLE_WORKFLOWS=1` at startup. An organization can set the same `disableWorkflows` key in managed settings. When workflows are disabled, `/workflows` is unavailable, the `ultracode` keyword no longer starts a run, and the Ultracode toggle is removed from `/effort`. A run already in progress keeps going. The page also says no managed setting rules out ultracode by itself: wherever ultracode is available, `/effort ultracode` can turn it on. An effort cap lowers the effort level a session with ultracode on runs at, and does not turn ultracode off. The product that these plans pay for is [Claude Code](/directory/claude-code). List prices for the plans are a different page, [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). This note does not restate those prices. ## Does ultracode use more of your Claude plan? Yes. The workflows page says each request uses more tokens and takes longer than the same request without a workflow, so a subscription session with ultracode on reaches a session or weekly limit sooner. It does not print a separate dollar price for the setting. The cost section of that page says a single workflow run can use meaningfully more tokens than working through the same task in conversation, and that runs count toward the plan’s usage and rate limits. The gauge it recommends is a small slice first: one directory, or a narrow question. `/workflows` shows each agent’s token usage while the run is going, and the run can be stopped there. Three checks are skipped while ultracode is on, because turning it on is already the opt-in to large runs: - The `Large workflow` warning does not appear. That warning, when ultracode is off, shows if a workflow schedules more than 25 agents or its projected token total passes 1.5 million. The warning points at `/workflows` and does not pause the run. - The session’s concurrent subagent limit is not enforced for subagents Claude spawns with the Agent tool. - In auto permission mode, the first workflow launch is not an approval prompt. The runtime caps still apply. The page lists up to 16 concurrent agents by default, fewer when fewer CPUs are available. `CLAUDE_CODE_WORKFLOW_MAX_CONCURRENT_AGENTS` can set that from 1 to 256, and that variable requires v2.1.269 or later. One `parallel()` or `pipeline()` call accepts up to 4,096 items. The total is 1,000 agents per run. A size guideline is advice to Claude about how many agents to aim for, not a cap. The values are unrestricted, small (fewer than 5), medium (fewer than 10), and large (fewer than 50). The default is medium, or small on a Pro plan with v2.1.271 or later. The Reddit question, any reason to leave ultracode off, has this answer in the docs and no other answer on this page. The setting applies to every substantive task in the session. Routine work pays the extra tokens too. `/effort ultracode off` is how the workflows page says to return to routine work. ## How do you turn ultracode off? Run `/effort ultracode off`. On v2.1.284 and later, that turns the setting off for the current session and leaves the effort level where it is. If `"ultracode": true` is in a settings file, the off command lasts for one session. To stop workflows entirely, use the Dynamic workflows toggle or `"disableWorkflows": true`, as the previous section quotes. That also removes the Ultracode toggle. The approval prompt on a workflow is a different switch. In the CLI, the per-run prompt can offer “Yes, and don’t ask again” for a bundled, saved, or plugin workflow run by name, and the workflows page says that option is not offered for a script Claude wrote for the current task. With ultracode on, auto mode skips the first-launch prompt. That prompt option is not the saved Bash rule covered in the permissions docs. The harness around either choice is the longer guide, [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). ## FAQ ### What does ultracode do in Claude Code? It plans a dynamic workflow for each substantive task in the session. The workflows page, read November 9, 2026, says Claude writes a JavaScript script, a runtime runs the subagents in the background, and the session stays responsive. `/effort ultracode` turns that behavior on for the current session. ### What is ultracode in Claude Code? A Claude Code setting. The model-config page says it is a setting rather than a model effort level, and the effort table says it plans a dynamic workflow for each substantive task at any effort level. ### Is ultracode the same as max effort? The model-config table lists them on separate rows. `max` is the deepest reasoning level. ultracode is the workflow setting. Since changelog v2.1.284 on September 28, 2026, `/effort ultracode` leaves the effort level where it is. `claude --effort ultracode` is the form that also sets the level to `xhigh`. ### How do you turn ultracode on? `/effort ultracode`, Tab on the slider’s Ultracode toggle, or `"ultracode": true` for later sessions. `claude --effort ultracode` turns it on and sets effort to `xhigh`. On Pro, turn Dynamic workflows on in `/config` first. ### Does ultracode cost extra? The workflows page prints no separate price. It says each request uses more tokens and takes longer, and that a subscription draws those tokens from the plan’s usage limits, so a session or weekly limit arrives sooner. The plan prices are [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). ### How do you turn ultracode off? `/effort ultracode off` on Claude Code v2.1.284 or later. Turning dynamic workflows off removes the toggle as well. ## Multi-Agent Orchestration: Removing the Interaction Term Costs 7.2 to 17.3 Points on Code (2026) URL: https://www.aiagentsinfo.com/blog/what-is-multi-agent-orchestration Published: 2026-11-09. Updated: 2026-11-09 Category: explainer What is multi-agent orchestration? An orchestrator chooses the next worker and what that worker may read. In Liu and Xiong, arXiv November 5, 2026, dropping the interaction term costs 7.2 points on their long-input HumanEval and 17.3 points on MBPP. Updated November 9, 2026. ## What is multi-agent orchestration? Multi-agent orchestration is the rule for which worker runs next, and which earlier results that worker is allowed to see. An [orchestrator](/glossary/orchestrator) makes that choice. The workers do the reading, the calculating, and the writing. In Liu and Xiong, [arXiv:2611.07495](https://arxiv.org/abs/2611.07495), posted November 5, 2026, the workers’ own policies stay fixed and the training changes only the orchestrator. The paper’s name for the method is IAPO, interaction-aware policy optimization. That is the research meaning. Product pages use the same words for a design choice a person writes down. [OpenAI’s orchestration page](https://developers.openai.com/api/docs/guides/agents/orchestration), read November 9, 2026, says the first choice is who owns the reply. A [handoff](/glossary/agent-handoff) moves control to a specialist. An agent used as a tool leaves the manager in charge. The short glossary entry for the whole idea is [orchestration](/glossary/orchestration). The workshop that holds the rule is an [agent harness](/blog/what-is-an-ai-agent-harness), and the shelf for those pages is [Agent harnesses](/topics/harnesses). On November 9, 2026, Google’s public autocomplete for “multi-agent orchestration” offered frameworks, a platform, patterns, GitHub, and “what is multi-agent orchestration.” This page answers the question with one paper. The framework list people also search for is a different page, linked below. ## What did the November 5, 2026 paper find? Removing the interaction term costs 7.2 points on the paper’s long-input HumanEval and 17.3 points on MBPP. At a matched token budget, the trained plans match or beat the strongest of fifteen published systems, and one episode uses fewer tokens. The authors are Kaixuan Liu, Department of Computer Science, Emory University, and Guojun Xiong, the corresponding author, John Hopcroft Center for Computer Science, Shanghai Jiao Tong University. arXiv lists version 1 in cs.LG, submitted November 5, 2026 at 05:58:30 UTC. The abstract’s sentence is that removing the interaction term costs 7 to 17 points of accuracy on code. Table 7 prints the two code cells behind that range: MBPP 81.1% with the term and 63.8% without it, HumanEval 83.7% and 76.5%. Table 8’s paired test calls those gaps +17.3 and +7.2, both with z above 2. The same abstract says the deployed plans reach the strongest system’s accuracy with fewer tokens. One IAPO episode costs 5.1 thousand to 11.5 thousand tokens. The strongest baselines in Figure 2 cost 26.5 thousand to 121.9 thousand, which the paper calls 2.7 to 24 times more. On MBPP a single episode is already 81.1% at 5.1 thousand tokens, against DyLAN at 71.2% and 121.9 thousand tokens. Those percentages belong to the authors’ long documents. A problem’s own statement sits among 30 to 90 other problems, so the file is longer than the original benchmark item. The paper says the absolute accuracies are not comparable with published numbers on the original benchmarks. ## Why does scoring each decision by itself miss the gain? Because the value of calling a worker depends on the calls that came before it. The paper proves that those interactions can account for half of the attainable gain. Ordinary training gives each decision its own advantage and treats the other decisions as fixed. In orchestration, a later call is useful only if an earlier call left something worth reading. The paper’s example is which worker to call and which earlier outputs it may see. The gain of changing both is not the sum of changing each one. IAPO writes the gap between two policies as a finite sum of those interaction terms, estimates the first- and second-order terms from sampled runs, and accepts an update only when a high-confidence lower bound on the gain is positive. The proof says every accepted update improves the training objective with high probability. The certificate covers net return on the validation problems. Test accuracy is a separate measurement. A small exact check makes the miss visible. On a task the paper calls Complementarity, the first-order score is zero while the best step gains 0.148. On Redundancy, the first-order score takes a step that gains 0.006 against a better step at 0.010. Across 6,000 validation runs of these controlled tasks, no candidate with a negative gain was accepted. The half-gain sentence is the theorem’s bound on what a first-order rule can miss. It is not a claim that every benchmark lost 50 points. ## How does IAPO compare with fifteen other multi-agent systems? At the strongest baseline’s token budget, IAPO is ahead on the paper’s GSM8K, DROP, and MBPP, and within a point of the leader on MATH and HumanEval. The fifteen systems share the executor and the splits. Three are single-agent prompting methods: chain-of-thought, complexity-based prompting, and self-consistency. Six are hand-built multi-agent systems: MultiPersona, LLM-Debate, LLM-Blender, DyLAN, AgentVerse, and MacNet. Six search or train a workflow: AutoAgents, GPTSwarm, ADAS, AFlow, MAS-Zero, and MaAS. The paper runs each as released, so none of them has the team’s BM25 retriever. They read the whole document. The orchestrated team reaches the document through an evidence finder or through a generalist that reads all of it. The table uses the paper’s own units. Accuracy is percent. Tokens are thousands per problem. “One episode” is Table 7, configuration A, the setting trained with the smaller token price. “Matched budget” is Table 8: a vote, or on code a program selector, over as many episodes as fit inside the strongest baseline’s tokens. Appendix I sets those caps at 28.6 thousand for GSM8K, 53.8 thousand for MATH, 57.1 thousand for DROP, and 26.5 thousand for HumanEval. On MBPP the vote is capped at 35.4 thousand, below DyLAN’s 121.9 thousand, and the +9.7 is still against DyLAN’s score at that higher spend. The first-order column is the same method with the interaction term removed.
Long-input task IAPO, one episode First order only, one episode Strongest baseline IAPO, matched budget
GSM8K, 790 test problems 78.1% at 5.6k tokens 77.4% at 9.7k LLM-Debate, 79.5% at 28.6k 81.8% at 25.1k, +2.3
MATH, 220 test problems 37.7% at 5.6k 37.6% at 15.2k DyLAN, 48.3% at 54.3k 48.6% at 50.8k, +0.3
DROP, 800 test problems 69.2% at 11.5k 67.9% at 20.1k LLM-Debate, 70.6% at 57.1k 74.2% at 46.0k, +3.6
MBPP, 333 test problems 81.1% at 5.1k 63.8% at 15.7k DyLAN, 71.2% at 121.9k 80.9% at 33.1k, +9.7
HumanEval, 131 test problems 83.7% at 9.9k 76.5% at 18.1k MaAS, 83.8% at 26.5k 84.3% at 20.4k, +0.5
The paper marks the GSM8K, DROP, and MBPP matched-budget gaps as z above 2. The MATH gap of 0.3 and the HumanEval gap of 0.5 are not. A second training price, which the paper calls IAPO-Lite, pushes GSM8K to 85.1%, MATH to 54.3%, and MBPP to 78.3% at the matched budget, and gives up 4.5 points on DROP and 8.1 points on HumanEval. Fewer tokens and a higher score are not the same result on every column. Search and training tokens are excluded from the per-problem costs. One MBPP run uses about 170 million tokens in total, the paper’s Appendix I. ## What does removing the interaction term cost on code? On one episode it costs 17.3 points on MBPP and 7.2 points on HumanEval, and the first-order plans also spend more tokens. Table 7 is the clean comparison. The retriever, the roles, the validation, and the certification stay. Only the interaction term comes out. IAPO then uses 32% to 58% of the first-order variant’s tokens on every benchmark. On the question-answering and math rows the accuracy gap is smaller: +0.6 on GSM8K, 0.0 on MATH, and +1.4 on DROP. The deployed plans show why code moves. Test-and-repair helps only after a program exists, which is a pair of decisions. IAPO deploys that pair in all four code runs. The first-order variant never proposes it, fills the slots with a second programmer, and the paper says it loses 17 points on MBPP and 7 on HumanEval at 3.1 and 1.8 times the tokens. On the other tasks, every IAPO run follows the evidence finder with the analyst. The first-order variant never calls the analyst. The analyst alone loses points, so a score that looks at one decision never selects it. The pair gains. ## What model and harness did the experiment use? Every worker is Qwen2.5-7B-Instruct. The harness is a pipeline of at most five decisions, four on code, over a fixed menu of roles. Appendix H says each call is a chat completion in bfloat16 on vLLM, one engine per H200, with a 24,576-token context window, at most 4,096 completion tokens, and temperature 1. IAPO trajectories are capped at 32,768 tokens. Baselines are capped at 131,072, and some code and math runs at 393,216. The [Qwen2.5-7B-Instruct card](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) was still public on November 9, 2026. It is the checkpoint the paper froze. It is a poor label for the model inside a current coding product. The roles for questions and math are a planner, an evidence finder that runs BM25 and makes no model call, an analyst, a generalist that reads the whole document, a calculator, a checker, and a selector. Code adds MaAS’s test-and-repair operator, up to three repairs. The orchestrator sees an embedding of the task and a few flags, including how much of the token budget is spent. It does not see the score. The last decision can only stop. That menu leaves 2,016 open-loop plans at five decisions and 432 at four. The reward is accuracy minus a token price. The main setting uses a price fixed in a pilot so that one extra revision of about 1,500 tokens breaks even. IAPO-Lite uses a much higher token price. Training starts from a uniform policy, fits on 6,400 proposal episodes from 200 problems, and certifies candidates on 8,000 validation episodes. Test problems are never used for fitting or selection. ## What are the limits of this multi-agent orchestration result? The result is a comparison inside one paper’s harness, on long documents the authors built, with one 7B worker model. The public GSM8K, MATH, DROP, MBPP, and HumanEval numbers people quote elsewhere do not apply here. The documents are longer, the scorer follows the paper’s rules, and every system sees the opening words of the target unit inside the question. The baselines and the team do not see the file the same way. Baselines read the whole document and have no BM25 retriever. The team retrieves the top few units or spends a generalist call on the whole file. The 7.2 and 17.3 point gaps keep the retriever on both sides, so that particular tool is not the difference. The gap against DyLAN or MaAS still mixes the learned policy with that retrieval choice and with a much smaller token cap. Several baseline rows carry the paper’s own warnings. AgentVerse crashed on 12 of 131 HumanEval problems and 17 of 333 MBPP problems. MAS-Zero’s design step failed on 76% to 94% of iterations. On DROP, the baselines’ training file shares 120 of 161 passages with the test split. A trajectory that exhausts its budget scores zero. That happened only on MBPP among IAPO’s test runs, at most 3 of 1,332 episodes in a run. The certificate accepts an update when the lower bound on validation return is positive. It does not certify the test-set accuracy in the table. The hand-built systems were also adapted where the shared 7B setup forced a change, then frozen. A reader who wants a product ranking of Claude, Codex, or an installed framework will not find it in these cells. ## How does learned orchestration sit inside an agent harness? The harness is the program around the model. Orchestration is the part of that program that picks the next step or the next agent. IAPO trains that part and leaves the model weights alone. On this site a [harness](/glossary/harness) is the loop, the tools, the memory, the permissions, and the stop rules. [Harness engineering](/glossary/harness-engineering) is the work of designing them. The control loop those pieces run is [How do AI agents work?](/blog/how-do-ai-agents-work). A [multi-agent](/glossary/multi-agent) system is the case with more than one role. IAPO is a training method for the choice among roles, measured on one frozen model. Two nearby papers ask different questions. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) searches for a new executable harness. [Parallel agent teams](/research/parallel-agent-teams-test-time-compute) keep the harness and change how a budget of agents is split into teams. IAPO keeps the role menu and changes how the orchestrator is trained. The 2022 ancestor of writing a step and then acting is [ReAct](/research/react-reasoning-and-acting). Microsoft’s architecture guide, cited from the control-loop page, still says to use the lowest level that meets the job: one model call, one agent with tools, or multi-agent orchestration. This paper is evidence about the third level, on the tasks and the 7B model it names. It is not a reason to add roles to a job one worker already finishes. ## Where can you read more on this site? - Terms: [orchestration](/glossary/orchestration), [orchestrator](/glossary/orchestrator), [agent handoff](/glossary/agent-handoff), [multi-agent](/glossary/multi-agent), [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [agent](/glossary/agent), [agent loop](/glossary/agent-loop) - The harness hub: [Agent harnesses](/topics/harnesses) - Guides: [How do AI agents work?](/blog/how-do-ai-agents-work), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness) - Research: [Parallel agent teams](/research/parallel-agent-teams-test-time-compute), [HarnessFlow](/research/harnessflow-mcts-harness-optimization), [ReAct](/research/react-reasoning-and-acting) - Libraries people search for under this heading: [Best AI agent frameworks](/best/ai-agent-frameworks), with release tags read November 9, 2026. That page is a licence and release table. It is not a score for IAPO. ## FAQ ### What is multi-agent orchestration? It is the rule for which worker runs next and which earlier results that worker may read. Liu and Xiong, in a paper posted November 5, 2026, train that rule and keep the workers fixed. The glossary entry is [orchestration](/glossary/orchestration). ### What is multi-agent orchestration in AI? It is that rule inside a system whose workers are language models, tools, or both. OpenAI’s page, read November 9, 2026, splits the product version into a [handoff](/glossary/agent-handoff) and an agent used as a tool. The paper’s version learns the sequence of calls. ### What did IAPO find on code? Removing the interaction term costs 17.3 points on the paper’s long-input MBPP and 7.2 points on HumanEval, for one episode of Qwen2.5-7B-Instruct. The scores are 81.1% versus 63.8%, and 83.7% versus 76.5%. ### Does IAPO beat other multi-agent systems? At a matched token budget it leads the strongest of the fifteen systems by 2.3, 3.6, and 9.7 points on the paper’s GSM8K, DROP, and MBPP, and it is within a point on MATH and HumanEval. One MBPP episode is 81.1% at 5.1 thousand tokens, against DyLAN at 71.2% and 121.9 thousand tokens. ### What model did the experiment use? Qwen2.5-7B-Instruct for every worker call, temperature 1, on vLLM. The model card was still public on November 9, 2026. The paper’s scores are for that checkpoint inside this harness. ### What is an AI agent orchestrator? The part that chooses the next worker and the earlier outputs that worker may see. The paper calls it agent 0. The glossary entry is [orchestrator](/glossary/orchestrator). ### How is this different from a multi-agent orchestration framework? A framework is software a team installs. IAPO is a training method reported in one paper. Installable libraries, with their November 9, 2026 release tags, are [Best AI agent frameworks](/best/ai-agent-frameworks). ### Where is the paper, and is there code? [arXiv 2611.07495](https://arxiv.org/abs/2611.07495), version 1, submitted November 5, 2026. On November 9, 2026, the abstract page did not list a code repository. ## Which AI Agent Framework Do People Install? 155,918,257 LangChain Downloads in 30 Days URL: https://www.aiagentsinfo.com/blog/which-ai-agent-framework-do-people-install Published: 2026-11-09. Updated: 2026-11-09 Category: explainer 155,918,257 PyPI downloads for langchain from October 10 through November 8, 2026, without mirrors. LangGraph had 55,093,579. CrewAI had 2,629,769 and 15,995 more GitHub stars than LangGraph. **155,918,257.** That is how many times the PyPI distribution `langchain` was downloaded from October 10 through November 8, 2026, in the `without_mirrors` series. The [November 5 star snapshot](/reports/ai-agent-framework-github-stats-november-2026) is the page that already shows up for agent-framework stats. It ranks repositories by stars and latest release. It does not say which distribution pip actually fetched. This page sums the daily download rows for the distributions behind that set, adds the one the snapshot left out, and reads the star field again on November 9. The same method, refreshed daily from pypistats and npm, is the [AI agent framework downloads tracker](/trackers/downloads). GitHub stars for the paired repositories are the [stars tracker](/trackers/stars). ## Which AI agent framework do people install? `langchain`, by PyPI file downloads, in this set and this window. `langgraph` is second. `strands-agents` is third, ahead of `claude-agent-sdk` and `openai-agents`. The window is 30 calendar days, October 10 through November 8, inclusive. Each total is the sum of the daily `without_mirrors` counts from `https://pypistats.org/api/packages/{name}/overall?mirrors=false`. The series for these packages ended on November 8. November 9 was not in the file yet. Stars are `stargazers_count` from the GitHub repository API. The responses for the repositories that were already in the November 5 set carried HTTP Date headers from 13:38:56Z to 13:39:03Z on November 9, 2026. `strands-agents/harness-sdk` was read at 13:40:17Z, because the PyPI homepage pointed there and the first list did not include it. | Distribution | Oct 10–Nov 8 | Nov 2–Nov 8 | Nov 8 | PyPI version | Stars | Repository | | --- | --- | --- | --- | --- | --- | --- | | langchain | 155,918,257 | 35,271,689 | 4,301,971 | 1.4.9 | 149,260 | langchain-ai/langchain | | langgraph | 55,093,579 | 13,876,217 | 1,109,424 | 1.2.20 | 44,689 | langchain-ai/langgraph | | strands-agents | 37,592,185 | 8,863,289 | 963,158 | 1.64.0 | 9,084 | strands-agents/harness-sdk | | claude-agent-sdk | 33,887,233 | 8,774,136 | 748,665 | 0.2.179 | 8,502 | anthropics/claude-agent-sdk-python | | pydantic-ai-slim | 32,785,681 | 8,730,134 | 627,097 | 2.66.0 | 21,101 | pydantic/pydantic-ai | | openai-agents | 17,426,915 | 4,509,002 | 326,857 | 0.24.0 | 30,573 | openai/openai-agents-python | | google-adk | 10,099,299 | 2,564,724 | 240,894 | 2.14.0 | 22,158 | google/adk-python | | pydantic-ai | 6,575,730 | 1,601,738 | 105,808 | 2.66.0 | 21,101 | pydantic/pydantic-ai | | dspy | 5,274,275 | 1,211,876 | 75,319 | 3.4.1 | 38,983 | stanfordnlp/dspy | | crewai | 2,629,769 | 590,729 | 69,994 | 1.15.39 | 60,684 | crewAIInc/crewAI | | agno | 2,134,459 | 516,797 | 44,983 | 3.1.12 | 43,117 | agno-agi/agno | | llama-index | 2,044,132 | 411,755 | 29,851 | 0.14.25 | 52,856 | run-llama/llama_index | | agent-framework-core | 1,469,689 | 362,008 | 29,307 | 1.25.0 | 14,603 | microsoft/agent-framework | | smolagents | 597,936 | 175,098 | 16,897 | 1.26.0 | 30,276 | huggingface/smolagents | | haystack-ai | 556,838 | 141,070 | 9,437 | 3.4.1 | 26,954 | deepset-ai/haystack | | autogen-agentchat | 401,015 | 100,319 | 10,576 | 0.7.5 | 61,759 | microsoft/autogen | | semantic-kernel | 328,540 | 80,290 | 7,421 | 1.47.0 | 28,748 | microsoft/semantic-kernel | | agent-framework | 195,287 | 44,294 | 3,687 | 1.25.0 | 14,603 | microsoft/agent-framework | | camel-ai | 71,056 | 11,549 | 1,255 | 0.2.90 | 18,023 | camel-ai/camel | The 19 thirty-day totals sum to 365,081,875. That sum double-counts work that installs more than one of these distributions. It is a checksum on the rows, and it is not a market size. `pydantic-ai` 2.66.0 requires `pydantic-ai-slim==2.66.0` with no extra marker. The slim distribution had 32,785,681 downloads, 4.99 times the meta package’s 6,575,730. Both rows share the star count of `pydantic/pydantic-ai`. The [directory page](/directory/pydantic-ai) is the product. The two download rows are two files on PyPI. `agent-framework` 1.25.0 requires `agent-framework-core[all]==1.25.0` with no extra marker. Core had 1,469,689 downloads and the meta package had 195,287. Both point at `microsoft/agent-framework`, which had 14,603 stars, so the star cell is one repository printed twice. The [directory listing](/directory/microsoft-agent-framework) is that repository. The order of the top five distributions is the same in the seven-day column as in the thirty-day column: langchain, langgraph, strands-agents, claude-agent-sdk, pydantic-ai-slim. A cross-check on the same endpoint: for `langgraph`, the seven daily rows from November 2 through November 8 sum to 13,876,217, and the recent endpoint’s `last_week` field was 13,876,217. November 8 alone was 1,109,424, and `last_day` was 1,109,424. The recent endpoint’s `last_month` field was 44,769,952. The thirty daily rows sum to 55,093,579. The same split showed up for `langchain` (170,457,766 in `last_month`, 155,918,257 in the thirty daily rows) and for `crewai` (2,442,764 against 2,629,769). This page uses the daily rows. ## Why does CrewAI have more stars and fewer downloads? CrewAI’s repository had 15,995 more stars than LangGraph’s. LangGraph’s 30-day download count was 20.95 times CrewAI’s. 60,684 minus 44,689 is 15,995. 55,093,579 divided by 2,629,769 is 20.95. The seven-day cut is the same shape: 13,876,217 against 590,729. Stars are an account’s bookmark on a repository. Downloads are file requests for one named distribution. CrewAI’s PyPI summary calls it a framework for orchestrating role-playing, autonomous agents, and the project URL is `crewAIInc/crewAI`. LangGraph’s summary is “Building stateful, multi-actor applications with LLMs,” and the project URL is the `libs/langgraph` tree inside `langchain-ai/langgraph`. The [CrewAI listing](/directory/crewai) and the [LangChain listing](/directory/langchain) describe the products. The [framework comparison](/best/ai-agent-frameworks) is the licence and release table. Neither of those pages is an install ranking. `langchain` itself had 59.29 times CrewAI’s downloads, 155,918,257 against 2,629,769, and 149,260 stars against 60,684. The November 5 snapshot had this repository at 149,047 stars. Four days later the field was 213 higher. The download window is the new measurement. ## Where does strands-agents rank if the star table left it out? Third on this PyPI list. 37,592,185 downloads. 9,084 stars on the repository the package names. The November 5 methodology says strands-agents was left out because the GitHub API returned a redirect. On November 9, PyPI’s JSON for `strands-agents` 1.64.0 gave the summary “A model-driven approach to building AI agents in just a few lines of code,” Apache-2.0 in the license field, homepage `https://github.com/strands-agents/harness-sdk`, and documentation `https://strandsagents.com`. The API for that repository returned 9,084 stars, 1,487 forks, the SPDX id Apache-2.0, `archived: false`, and the description “Build an agent harness and control it end-to-end. Open-source SDK for production AI agents in Python & TypeScript - any model, any cloud.” 37,592,185 divided by 2,629,769 is 14.29. The omitted package had 14.29 times CrewAI’s downloads and 9,084 stars against CrewAI’s 60,684. A [harness](/glossary/harness), on this site, is the loop, tools, memory, permissions, and stop rules around a model. The repository description uses that word for the SDK. This page did not install it. `claude-agent-sdk` is the next row, 33,887,233 downloads, from `anthropics/claude-agent-sdk-python` at 8,502 stars. Version 0.2.179. The PyPI summary is “Python SDK for Claude Code.” The [listing](/directory/claude-agent-sdk) is the product page. `openai-agents` 0.24.0 had 17,426,915 downloads and the Python repository had 30,573 stars. That listing is the [OpenAI Agents SDK](/directory/openai-agents-sdk). `google-adk` 2.14.0 had 10,099,299 downloads and `google/adk-python` had 22,158 stars. Its listing is [Google ADK](/directory/google-adk). ## Does installing LangChain also download LangGraph? A fresh install of langchain 1.4.9 requests langgraph. The 30-day totals still do not nest, because 155,918,257 is larger than 55,093,579. The requires line, read at 13:40:16 UTC, is `langgraph<1.3.0,>=1.2.12`, with no extra marker. A cache that already holds the file, and every older langchain release inside the 30-day total, keeps the two numbers from lining up as parent and child. google-adk’s langgraph lines are different: each one is guarded by `extra == "all"`, `extra == "extensions"`, or `extra == "test"`. A default `pip install google-adk` does not, on that metadata, require langgraph. `autogen-agentchat` 0.7.5 had 401,015 downloads. The repository `microsoft/autogen` had 61,759 stars on this pass. langchain’s download total was 388.8 times that package’s. The same repository’s README, and the archive flag, are the subject of [Is AutoGen deprecated?](/blog/is-autogen-deprecated), which read 61,753 stars at 07:41 UTC the same day. Six stars later, the install count is still the small number. semantic-kernel 1.47.0 lists `autogen-agentchat` only behind `extra == "autogen"` and a Python-version marker. That extra is not the 401,015. ## What do the npm packages show for the same dates? The package `ai` had 140,196,706 downloads. `@langchain/langgraph` had 15,932,146. `@mastra/core` had 9,988,333. npm’s point API was called for the range `2026-10-10:2026-11-08`. Each response echoed that start and that end. The [download-counts doc](https://github.com/npm/registry/blob/master/docs/download-counts.md) describes this endpoint. An npm download and a PyPI download are different registries. This table does not add them to the PyPI column. | npm package | Downloads, Oct 10–Nov 8 | Version read Nov 9 | Stars | Repository the registry named | | --- | --- | --- | --- | --- | | ai | 140,196,706 | 7.0.166 | 27,830 | vercel/ai | | @langchain/langgraph | 15,932,146 | 1.7.2 | 3,411 | langchain-ai/langgraphjs | | @mastra/core | 9,988,333 | 1.82.0 | 29,428 | mastra-ai/mastra | | @openai/agents | 8,171,695 | 0.21.0 | 3,996 | openai/openai-agents-js | | @google/adk | 1,085,245 | 2.4.0 | 1,471 | google/adk-js | The registry describes `ai` as the AI SDK by Vercel, a single interface for model providers. The GitHub description for `vercel/ai`, trailing space removed, was “The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents.” 140,196,706 downloads and 27,830 stars sit on that repository. The November 5 snapshot included `vercel/ai` in its 19. The download count covers every app that depends on the package, including apps that call a model and never run an agent loop. `@langchain/langgraph` names `langchain-ai/langgraphjs`, which had 3,411 stars at 13:42:28Z. The Python repository `langchain-ai/langgraph` had 44,689 stars and the PyPI distribution had 55,093,579 downloads. Those are two repositories. `@openai/agents` names `openai/openai-agents-js` (3,996 stars), and the Python SDK is the other repository (30,573 stars, 17,426,915 PyPI downloads). `@google/adk` names `google/adk-js` (1,471 stars, version 2.4.0). The Python package `google-adk` was on 2.14.0. `@mastra/core` 1.82.0 names `mastra-ai/mastra`, whose description was “Mastra is the modern TypeScript framework for AI-powered applications and agents,” at 29,428 stars. The npm metadata did not include a description field for that package. The star pass’s HTTP Date for Mastra was 13:39:01Z. The three smaller JavaScript repositories were read at 13:42:28Z and 13:42:29Z. ## Why is Letta absent from the table? The PyPI package `letta` and the repository in the November 5 star table are different projects, so this page does not pair them. `letta` 0.34.12 had 9,886 downloads in the same 30 days. Its summary was “Letta Code: stateful agents in your terminal,” and its source URL was `https://github.com/letta-ai/letta-code`. The star snapshot’s row is `letta-ai/letta`, which returned 25,390 stars, `pushed_at` 2026-10-18T22:28:04Z, and the SPDX id Apache-2.0. Pairing 9,886 downloads with 25,390 stars would answer a question about the wrong repository. ## What is a download count for? A download count is the right tool for “what did pip and npm fetch,” and a weak tool for “what should I install.” CI, a dependency, and a person trying a tutorial all increment the same field. `langchain` 1.4.9 pulling `langgraph`, and `pydantic-ai` pulling `pydantic-ai-slim`, are the cases checked above. Other dependencies inside these graphs were not fully expanded. Quality, licence, and whether the README still promises fixes are separate questions. AutoGen’s install name in its own README is `autogen-agentchat`, and that distribution’s 401,015 downloads sit next to a repository whose maintenance statement is already written up. The [agent picker](/tools/agent-picker) is the directory’s filter for a job. [How do AI agents work?](/blog/how-do-ai-agents-work) is the loop those libraries wrap. A star census of agent products, rather than of these libraries, is [How many GitHub stars does OpenClaw have?](/research/how-many-github-stars-does-openclaw-have). How many popular repositories write instructions for the coding agents these SDKs drive is [How many repos use AGENTS.md?](/research/how-many-repos-use-agents-md). The live filename search next to that census is the [AGENTS.md adoption tracker](/trackers/agents-md). ## FAQ ### Which AI agent framework do people install? `langchain`, at 155,918,257 PyPI downloads from October 10 through November 8, 2026, without mirrors. `langgraph` had 55,093,579. On npm, over the same dates, the package `ai` had 140,196,706. The two registries are not one ranking. ### Does CrewAI have more stars than installs would suggest? It has more stars than LangGraph and fewer downloads. 60,684 stars against 44,689. 2,629,769 downloads against 55,093,579. The download ratio is 20.95. ### Why was strands-agents left off the star snapshot? The snapshot’s note says the GitHub API returned a redirect. The package on PyPI now points at `strands-agents/harness-sdk`, which had 9,084 stars on November 9 and 37,592,185 downloads in the 30-day window. ### Does the LangChain number include LangGraph? Installing langchain 1.4.9 requests langgraph. The published totals do not nest: 155,918,257 is larger than 55,093,579. Older releases and caches sit inside the larger number. ### Is a download a user? A download is a file request in one registry’s public stats. It includes CI and dependency installs. It is the count to use when the question is what was fetched. ## How Do AI Agents Work? The Control Loop From Goal to Action (2026) URL: https://www.aiagentsinfo.com/blog/how-do-ai-agents-work Published: 2026-11-08. Updated: 2026-11-08 Category: guide How do AI agents work? A language model proposes a step. A harness runs an allowed tool, returns the result, and the loop continues until the goal is met, a limit hits, or a person must approve. Updated November 8, 2026. ## How do AI agents work? An AI agent works as a [language model](/glossary/llm) plus a [harness](/glossary/harness). The model proposes a step. The harness runs an allowed [tool](/glossary/tool-use), returns the result, and the loop continues until the goal is met, a limit hits, or a person must approve. That cycle is the control loop from goal to action. [Anthropic’s Form S-1/A](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm), filed November 2, 2026 and read again on November 8, defines the [agent loop](/glossary/agent-loop) as the cycle in which an agent receives a task, takes an action such as planning, running code, or calling a tool, observes the result, and decides the next step, repeating until the task is complete. The same defined terms call that loop the core execution pattern of agentic AI, and they call agentic AI a system that can plan and carry out steps, including with tools, services, datasets, and environments, on a user’s behalf. AI Agents Info uses those words the same way on the glossary and on this guide. The short definition stays at [What are AI agents and how do they work?](/glossary/agent). This page is the long explainer. A model that only replies, a retrieval step that fills the prompt, and a script that follows rules written in advance are the neighboring systems. Each one is spelled out below. The workshop around the model is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Testing one observable action in that workshop is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). Standing up the tasks and graders is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). The 2022 prompting pattern that interleaved a written thought with an action is [ReAct](/research/react-reasoning-and-acting). The dated public record for the coming year is [AI agents, the next 12 months](/reports/ai-agents-next-12-months). Read those for the workshop, the tests, the paper, and the forecast. ## What is an agent loop? An agent loop is the cycle of proposing an action, running an allowed tool, observing the result, and choosing the next step until a stop condition. Anthropic’s filing gives the public definition quoted above. The [Claude Agent SDK loop page](https://code.claude.com/docs/en/agent-sdk/agent-loop), read November 8, 2026, describes the same cycle inside a product runtime. The SDK delivers the prompt, the system prompt, the tool definitions, and the conversation history. Claude answers with text, tool calls, or both. The SDK runs the requested tools and feeds the results back. The cycle repeats until Claude produces a response with no tool calls. The SDK then yields that text and a result message with token usage, cost, and a session id. Hooks can intercept, change, or block a tool call before it runs. `maxTurns` and `maxBudgetUsd` stop the loop at a turn count or a spend cap. The page says a read-only agent whose allow list is Read, Glob, and Grep runs those tools without asking the person, apart from reads on network paths, and a tool on the disallow list is blocked. [OpenAI’s running-agents guide](https://developers.openai.com/api/docs/guides/agents/running-agents), read the same day, describes one SDK run as one application turn. The runner calls the current agent’s model, inspects the output, executes tool calls and continues, switches agents on a handoff, and returns when the model produces a final answer with no more tool work. A max-turn limit, a guardrail exception, or a tool error is a failure. A human approval is a pause: the run resumes from the saved state, and it is the same run. [ReAct](https://arxiv.org/abs/2210.03629), Yao and coauthors, 2022, is the research ancestor of the written form of this cycle: a reasoning trace, then an action, then an observation. The paper’s scores belong to that paper. The [research note](/research/react-reasoning-and-acting) says so. The pattern is what later runtimes kept. ## How does an AI agent differ from a chatbot, an assistant, a RAG app, and a workflow? A chatbot returns a reply. A RAG app retrieves text and then answers. A fixed workflow follows rules written in advance. An AI agent lets the model choose the next step, and a harness runs it. A multi-agent system splits that work across roles. An assistant, in ordinary product language, is the application a person opens. The same assistant sits in a different row of the table below depending on what the software does after the message arrives. [Microsoft’s architecture guide](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns), read November 8, 2026, puts the choices in that order and says to use the lowest level that reliably meets the requirement: a direct model call, one agent with tools, or multi-agent orchestration. [OpenAI’s business guide](https://cdn.openai.com/business-guides-and-resources/a-business-leaders-guide-to-working-with-agents.pdf), a PDF read November 8, 2026, draws the same line with three systems. The copy has no publication date. Its examples still use the names ChatGPT agent, Deep Research, and Operator. This page uses the PDF for the distinction, and it leaves current workplace product names to [What is an AI employee?](/blog/what-is-an-ai-employee), which was checked November 7, 2026.
Chatbot RAG app Fixed workflow AI agent Multi-agent system
Who chooses the next step The person does, by sending another message. OpenAI’s guide says a question-answering chatbot can call a tool for one current document and still cannot plan a multi-step job unless every step is programmed. The application does. Lewis et al., 2020, retrieve passages and then a generator writes the answer conditioned on those passages. The workflow does. OpenAI’s guide says the steps and the “if this, then that” rules are written in advance. An LLM may handle one judgment, such as classifying a request, and then control returns to the script. Microsoft’s sequential pattern is the multi-agent version of a fixed line: the next agent is defined by the workflow. The model does. The harness runs the tool the rules allow. Anthropic’s SDK and OpenAI’s SDK both describe that pass from model to runtime. A pattern does. OpenAI’s handoff gives the specialist the branch. An agent used as a tool leaves the manager in charge of the final answer. Microsoft also documents a fixed sequence, a parallel fan-out, a shared thread, and a manager with a task ledger.
Whether it acts It replies in text. Anthropic’s loop page says that without tools, Claude can only respond with text. It reads retrieved text into the answer. In the 2020 paper the retriever is a Wikipedia index, and the generator writes. A product that keeps choosing new searches has left that one pass. It performs the actions named in the rules. OpenAI’s examples are a locked account after failed logins, and an expense flow that stalls on a blank field. It can read, write, search, call an API, or run a command, inside the permissions. The model requests the action. The harness carries it out. Each role can act inside its own tools. A2A is for one independent agent delegating to another. MCP is for an agent calling its own tools.
What “done” looks like A message in the thread. A generated answer that used the retrieved passages. Lewis et al. also treat the retrieved text as something a person can inspect. The predetermined outcome, such as a routed ticket or a locked account. OpenAI’s guide says every branch was written down, which is why the result is auditable and why an unlisted condition can stall the run. The goal is met, the model stops calling tools, a turn or budget limit hits, a guardrail fails the run, or a person has to approve before the run resumes. The owning role returns the answer, or the pattern’s stop rule fires: a checker approves, a manager’s ledger is complete, or an iteration limit ends the thread.
[Retrieval-augmented generation](/glossary/rag), in Lewis et al., [arXiv:2005.11401](https://arxiv.org/abs/2005.11401), combines a pre-trained generator with a retriever over an explicit index. The paper’s index is Wikipedia, and the point of the retrieval is an answer a person can trace. That is a memory tactic for one generation. It is a different grant from an agent loop, which can call a tool, read the observation, and choose again. Products mix the two. The mix is still worth naming, because a search that the application always runs is RAG, and a search the model can skip, repeat, or follow with a write is [tool use](/glossary/tool-use). Microsoft’s guide says one agent with tools is often the right default for an enterprise case, and it says to set an iteration limit so a tool loop cannot run without end. Multi-agent orchestration, on that page, is justified when one agent cannot reliably handle the job because of prompt complexity, tool overload, or a security boundary. ## How does an AI agent use tools? The model requests a tool. The harness runs that tool when the rules allow it, and it puts the result back in front of the model. Anthropic’s loop page, read November 8, 2026, groups the built-in tools as file operations, search, shell commands, web search and page fetch, on-demand tool discovery, and orchestration tools that spawn a subagent, invoke a skill, ask the user, or track tasks. External services connect through MCP servers. Custom tools are another path. Read-only tools can run at the same time when the model requests several in one turn. Tools that change state, such as an edit or a shell command, run one after another. A denied call comes back as a tool result, and the page says Claude typically tries another approach or reports that it could not proceed. OpenAI’s business guide says the model supplies the judgment and the tools supply the action: a lookup, an edit, a function call, code in a terminal, a computer, or another agent reached through orchestration. The running-agents guide is the runtime version. Tool calls continue the loop. A final answer with no more tool work ends it. The [Model Context Protocol](/glossary/model-context-protocol) is the open protocol for that connection. The [introductory page](https://modelcontextprotocol.io/docs/getting-started/intro), read November 8, 2026, calls MCP a standard way for an application such as Claude or ChatGPT to reach data sources, tools, and workflows, and it compares the shape to a USB-C port. The [versioning page](https://modelcontextprotocol.io/docs/learn/versioning) for the current revision says the protocol version is 2026-07-28. The [specification](https://modelcontextprotocol.io/specification/2026-07-28) says servers can offer resources, prompts, and tools, and that a host must obtain a person’s consent before invoking a tool. Anthropic’s November 2 prospectus says Anthropic introduced MCP in November 2024 and donated it to the Linux Foundation’s Agentic AI Foundation in December 2025. Measured failures of agents that call tools through MCP, including tool misuse and goals the tools cannot reach, are the subject of [MCPHallu](/research/mcphallu-mcp-agent-hallucinations). ## What happens in one loop, from goal to stop? Today’s loop has seven parts. The names differ by vendor. The jobs show up in the pages read on November 8, 2026.
Part What it does
Goal The task the run is trying to finish. Anthropic’s defined term starts the agent loop when the agent receives a task. OpenAI’s business guide says an agent starts from a goal and can pause to ask for clarification.
Model The component that reads the current context and proposes the next step or a final answer. OpenAI’s guide says the model interprets instructions, plans, and decides what to do next. Anthropic’s SDK sends that proposal back as text, tool calls, or both.
Instructions and policies Standing text and checks. OpenAI’s business guide separates guardrails, which bound the action, from instructions, which guide the decision. The Agents SDK uses guardrails for automatic checks and a separate approval pause before a side effect. Anthropic’s permission mode, allow list, and disallow list are the same kind of boundary on the loop page.
Tools The actions the model may request. The harness validates and runs them. MCP is one open connector. A single programmed lookup, in OpenAI’s chatbot example, is a tool call that still leaves the plan in the application’s hands.
State and memory What the next turn can see. Anthropic resends conversation history and can keep a session. OpenAI’s running-agents guide offers four ways to carry state: replayed history, a session your application stores, a conversation id, or the previous response id. The business guide says memory of past steps lets an agent refine a later pass, and it describes that memory as still developing.
Harness and control loop The software that feeds the goal, offers tools, stores observations, enforces the rules, and decides when to call the model again. On this site that software is the harness, and the cycle it runs is the agent loop. Anthropic’s SDK overview says the SDK gives you the same tools, agent loop, and context management that power Claude Code. The overview also names Managed Agents as a hosted harness that runs the loop in an Anthropic-managed cloud sandbox or a self-hosted sandbox.
Stop or escalation The run ends on a final answer with no further tool calls, on a turn or budget limit, or on a failed guardrail. It pauses when a person must approve. OpenAI says to resume that pause from the saved state. A person who approves before the action counts is human in the loop. A person who watches the record and can step in is human on the loop.
A click on a real screen is the same loop with a narrower tool. That guide is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer). A coding agent that keeps an editor and a browser inside its own web session is a different product note, [What is Devin IDE?](/directory/devin#what-is-devin-ide). A workplace job with a file to review is [What is an AI employee?](/blog/what-is-an-ai-employee). ## Where does the harness fit? The harness is the software that runs the loop: tools, memory, permissions, and the stop rule. The model proposes. The harness is what makes the proposal an action. [Harness engineering](/glossary/harness-engineering) is the work of designing that software on purpose. [Orchestration](/glossary/orchestration) is the part that chooses the next step or the next specialist. A [guardrail](/glossary/guardrail) allows, blocks, rewrites, or pauses one action. An [agent framework](/directory/category/agent-frameworks) is a library for building those pieces. The running configuration for one job is the harness. Latest GitHub releases for 19 open-source frameworks and SDKs, pulled 5 November 2026, are in [AI agent framework release, November 2026](/reports/ai-agent-framework-github-stats-november-2026). [Claude’s Agent SDK](/directory/claude-agent-sdk) and the [OpenAI Agents SDK](/directory/openai-agents-sdk) describe that loop on the pages read November 8, 2026. The [Microsoft Agent Framework](/directory/microsoft-agent-framework) documents the multi-agent patterns on the orchestration pages read the same day. The comparison of shipping runtimes is [Best AI agent harness](/best/ai-agent-harnesses). The long account of the workshop is the [harness guide](/blog/what-is-an-ai-agent-harness). How a recorded failure becomes one check is the [evaluation guide](/blog/how-do-you-evaluate-an-ai-agent-harness). This page stays on the loop those pages assume. ## What is agent orchestration? Agent orchestration is the rule for which step, tool, or agent runs next, and who owns the answer. In a single agent, the orchestration is the loop itself: call the model, run the allowed tool, continue. In a [multi-agent](/glossary/multi-agent) system, it also decides who speaks next and who is responsible for the reply the person sees. [OpenAI’s orchestration guide](https://developers.openai.com/api/docs/guides/agents/orchestration), read November 8, 2026, offers two patterns. A handoff moves control to a specialist for that branch. An agent used as a tool leaves the manager in charge of the final answer. The same page says to start with one agent, and to add a specialist when the next branch needs different instructions, tools, or policy. Microsoft documents a wider set. The [Agent Framework orchestration page](https://learn.microsoft.com/en-us/agent-framework/workflows/orchestrations/), read November 8, 2026, lists five patterns: - Sequential: agents run one after another in a defined order. - Concurrent: agents run at the same time. - Handoff: agents transfer control based on context. - Group chat: agents share one conversation. - Magentic: a manager agent coordinates specialists. The [Azure architecture guide](https://learn.microsoft.com/en-us/azure/architecture/ai-ml/guide/ai-agent-design-patterns), read the same day, uses the same names and adds the conditions. Sequential orchestration suits a pipeline whose next stage is already decided. Concurrent orchestration runs independent analyses and can aggregate them. Handoff orchestration transfers full control when the right specialist is clear only mid-task. Group chat keeps a shared thread, and the guide says to consider keeping that pattern to three or fewer agents because the thread gets hard to control. Magentic orchestration is for an open-ended problem: a manager builds and revises a task ledger, and specialist agents use tools to change external systems. The Agent Framework page also says these orchestrations can pause for tool approval or for a request for information, which is the human-in-the-loop stop inside a multi-agent run. ## How do multi-agent systems work? A multi-agent system gives one job to more than one role. The pattern decides whether those roles run in a line, at the same time, by handing off ownership, in a shared thread, or under a manager that keeps a plan. The roles can be several prompts, several model calls, or several processes. On this site it is still one product and one bill unless a source says otherwise. Extra roles are a coordination choice. Microsoft’s architecture guide says they add overhead, latency, and new ways to fail, and it treats that cost as justified when a security boundary or the size of the tool list makes one agent unreliable. OpenAI’s guide makes the same point from the other direction: splitting early creates more prompts, more traces, and more approval surfaces. Two protocols cover two different edges, and both were live on November 8, 2026. The [Agent2Agent protocol](https://a2a-protocol.org/latest/) is the horizontal edge. The project’s home page says A2A is an open standard for agents built on different frameworks, or by different vendors, to discover each other, delegate work, and share results. It says A2A was developed by Google and donated to the Linux Foundation, with a technical steering committee that includes AWS, Cisco, Google, IBM Research, Microsoft, Salesforce, SAP, and ServiceNow. The [specification page](https://a2a-protocol.org/latest/specification/) lists the latest released version as 1.0.0. Agents interact from declared capabilities. The page says they do not have to share internal memory, tools, or proprietary logic. The specification’s goals include discovery, collaboration, and long-running tasks that can include a person. MCP is the vertical edge. The [A2A and MCP page](https://a2a-protocol.org/latest/topics/a2a-and-mcp/) says MCP deepens one agent with tools, resources, and data, and A2A connects agents across a team or a company boundary. The same page says to use both: A2A between agents, MCP from each agent to its own tools. A2A’s home page also says A2A is not a replacement for MCP, and it is not the protocol an agent uses to call its own sub-agents or its own tools. [Google’s Agent Development Kit](/directory/google-adk) is one framework the A2A home page names as a way to build an agent. The protocol is the link between agents. The framework is the code that builds one of them. ## How will AI agents work in the future? The part already specified is interoperability and supervision. The part still ahead is a set of dated forecasts, and this page leaves them as forecasts. MCP and A2A are the two public protocols in the sources above. MCP, version 2026-07-28, connects an application to tools and data. A2A 1.0.0 connects independent agents. [Deloitte’s November 18, 2025 essay](https://www.deloitte.com/global/en/insights/industry/technology/technology-media-and-telecom-predictions/2026/ai-agent-orchestration.html) predicted that, by the following year, competing agent protocols would begin converging on two or three standards. On November 8, 2026, MCP and A2A are both still published, and the A2A project’s own page tells builders to use them together. That is the status of the documents. It is not a score of which protocol has more users. Identity and least privilege are already written as product rules, in different words. Anthropic’s loop page lets a builder allow a narrow tool list and disallow the rest. A read-only agent is the page’s example. MCP’s specification says a person consents before a tool runs, and it says tool descriptions are untrusted unless they come from a trusted server. A2A’s specification treats the other agent as opaque: collaboration uses declared capabilities, and the caller does not receive the other agent’s internal tools. OpenAI’s guardrails page pauses a run before a side effect such as a cancellation, an edit, a shell command, or a sensitive MCP action, and its cybersecurity section says a review should see the proposed action and the calling identity against the approved scope. Those are boundaries a harness can enforce now. Observability is the record that makes a later review possible. Anthropic’s result message includes token usage, cost, and a session id. OpenAI’s pause returns the interruptions and a state the application can store and resume. A2A’s specification lists tracing and monitoring among the enterprise practices the protocol aligns with. Deloitte’s essay says orchestration platforms were, at the time of writing, adding telemetry such as latency, error rates, and token usage, plus a view of the orchestration, so a person can see what happened. The supervision forecast is the sentence to keep dated. Deloitte’s essay says a spectrum will emerge, from humans in the loop, to humans on the loop, to humans out of the loop, depending on the task, the domain, and how serious the outcome is. It says the in-the-loop and on-the-loop approaches rely on telemetry so people can intervene, and that humans out of the loop would still need continuous monitoring. It predicts that in 2026 the most advanced businesses will begin to lay the foundation for human-on-the-loop orchestration. On this site the two nearer terms stay distinct, because the product pages describe different stops. [Human in the loop](/glossary/human-in-the-loop) means a person approves, edits, or stops a step before that step counts. [Human on the loop](/glossary/human-on-the-loop) means a person supervises from the record and can intervene, and the step does not wait for approval first. Deloitte named the spectrum. The sharper split is this site’s, so a reader can tell an approval gate from a dashboard. OpenAI’s undated business guide says that, on the day of that copy, agents run when prompted or scheduled, and that in the future they may start themselves from a goal, a calendar, or a new event. The same guide says that, in the future, agents may run several tools at once. Anthropic’s loop page, read November 8, 2026, already runs read-only tools concurrently when the model requests more than one in a single turn. Tools that change state still run in order on that page. A forecast in an undated PDF and a behavior in a current SDK page can both be true. They are about different products. No source read for this guide names a month in which agents will work without a stop rule. [Autonomy](/glossary/autonomy) on this site remains a permission setting: how many steps are allowed before a person is asked. The next-twelve-months report collects the dates companies and governments have already put on paper. It does not add a launch this page cannot cite. ## Where can you read more on this site? - This guide’s terms: [agent](/glossary/agent), [agent loop](/glossary/agent-loop), [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [tool use](/glossary/tool-use), [orchestration](/glossary/orchestration), [multi-agent](/glossary/multi-agent), [Model Context Protocol](/glossary/model-context-protocol), [Agent2Agent protocol](/glossary/agent-to-agent-protocol), [human in the loop](/glossary/human-in-the-loop), [human on the loop](/glossary/human-on-the-loop) - Nearby terms: [RAG](/glossary/rag), [language model](/glossary/llm), [guardrail](/glossary/guardrail), [autonomy](/glossary/autonomy), [approval gate](/glossary/approval-gate) - Guides: [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents), [What is an AI employee?](/blog/what-is-an-ai-employee), [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer) - Research: [ReAct](/research/react-reasoning-and-acting), [MCPHallu](/research/mcphallu-mcp-agent-hallucinations), [What is multi-agent orchestration?](/blog/what-is-multi-agent-orchestration) - Report: [AI agents, the next 12 months](/reports/ai-agents-next-12-months) - Directory: [Claude Agent SDK](/directory/claude-agent-sdk), [OpenAI Agents SDK](/directory/openai-agents-sdk), [Microsoft Agent Framework](/directory/microsoft-agent-framework), [Google ADK](/directory/google-adk) - Comparisons: [Best AI agents](/best), [Best AI agent harness](/best/ai-agent-harnesses), [Best AI agent frameworks](/best/ai-agent-frameworks), [Best AI coding agents](/best/ai-coding-agents), [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools) - Cost: [AI agent cost calculator](/agent-cost-calculator) - Topics: [how agents work](/topics/how-agents-work), [harnesses](/topics/harnesses), [research](/topics/research) ## Frequently asked questions ### How do AI agents work? A language model proposes a step. A [harness](/glossary/harness) runs an allowed tool, returns the result, and the [agent loop](/glossary/agent-loop) continues until the goal is met, a limit hits, or a person must approve. [Anthropic’s November 2, 2026 prospectus](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm) calls that cycle the agent loop. The short definition is [What are AI agents and how do they work?](/glossary/agent). ### What is an agent loop? It is the control loop from goal to action: propose an action, run an allowed tool, observe the result, choose the next step, and stop when the task is done or a limit fires. The Claude Agent SDK and OpenAI’s Agents SDK, both read November 8, 2026, describe that cycle in their own runtimes. The glossary entry is [agent loop](/glossary/agent-loop). ### What is the difference between a chatbot, an assistant, a RAG app, a workflow, and an AI agent? An assistant is the application a person opens. A chatbot returns a reply. A [RAG](/glossary/rag) app retrieves text and then generates an answer, the pattern in Lewis et al., 2020. A fixed workflow follows rules written in advance, including a single model step that hands control back to the script, which is how OpenAI’s business guide separates workflows from agents. An agent starts from a goal and chooses tools inside a harness. The table above is the comparison. ### How does an AI agent use tools? The model requests a tool. The harness checks the allow and disallow rules, runs the call when it is allowed, and returns the result for the next decision. File edits, shell commands, web search, and external connectors are the kinds of tools on Anthropic’s loop page. [MCP](/glossary/model-context-protocol), current version 2026-07-28 as of November 8, 2026, is one open way to attach those connectors. Measured failures of those MCP calls are the subject of [MCPHallu](/research/mcphallu-mcp-agent-hallucinations). ### What is agent orchestration? It is the rule for which step, tool, or agent runs next, and who owns the answer. OpenAI distinguishes a handoff from an agent used as a tool. Microsoft Learn, read November 8, 2026, documents sequential, concurrent, handoff, group chat, and Magentic orchestration. The glossary entry is [orchestration](/glossary/orchestration). ### How do multi-agent systems work? More than one role shares a job. They may run in order, in parallel, by transferring control, in one shared thread, or under a manager that keeps a plan. Microsoft’s architecture guide says to add that machinery when one agent cannot reliably do the work. When the other party is an independent agent, the open protocol is [A2A](/glossary/agent-to-agent-protocol). When the call is a tool or a data source, the open protocol is MCP. The glossary entry is [multi-agent](/glossary/multi-agent). ### How will AI agents work in the future? MCP and A2A are already specified, and the A2A project says to use them together. Deloitte’s November 18, 2025 essay predicts that during 2026 the most advanced organizations will start building human-on-the-loop supervision, with people watching telemetry and intervening, beside the older pattern of approving steps. OpenAI’s undated business guide says agents of that copy run when prompted or scheduled, and that later ones may start from a goal or an event. This page does not add a month those sources do not name. The dated record is [AI agents, the next 12 months](/reports/ai-agents-next-12-months). ### What is the difference between human in the loop and human on the loop? Human in the loop means a person approves, edits, or stops a step before that step counts. OpenAI’s Agents SDK implements that pause and tells you to resume the same run. Human on the loop means a person supervises from the record and can intervene, without that prior approval. Deloitte’s November 18, 2025 essay names both on one spectrum and predicts a shift toward the second during 2026. The entries are [human in the loop](/glossary/human-in-the-loop) and [human on the loop](/glossary/human-on-the-loop). ## How Many Official MCP Servers Are There? 59,506 Latest Records on November 8, 2026 URL: https://www.aiagentsinfo.com/blog/how-many-official-mcp-servers-are-there Published: 2026-11-08. Updated: 2026-11-08 Category: explainer 59,506 latest records in the official MCP registry, counted from 13:43 to 13:45 UTC on November 8, 2026. 58,565 were marked active. One namespace held 2,366 of them. The official MCP registry listed **59,506** latest server records between 13:43:33 and 13:45:25 UTC on November 8, 2026. Of those, **58,565** were marked active and **941** were marked deprecated. A second pass through the same API, from 13:45:47 to 13:47:43 UTC, counted **59,508**. Two records appeared while the first count was being written up. That is the size of the list. It is not a count of servers worth connecting. [MCP](/glossary/model-context-protocol) is the protocol a host uses to reach tools and data. The registry is the public metadata catalog for servers that speak it. This page counted that catalog. It did not rank the servers, and it did not copy a star list from anywhere else. ## How many official MCP servers are there? 59,506, if you ask the official API for the latest version of each server and you finish the walk that started at 13:43 UTC. 59,508, if you start again two minutes later. Both numbers come from `GET https://registry.modelcontextprotocol.io/v0.1/servers` with `version=latest` and `limit=100`, following `metadata.nextCursor` until the cursor was gone. The OpenAPI text for that parameter says `version=latest` returns only the latest version of each matching server. Without it, the same endpoint returns older versions as separate rows. This count used the filter. Each pass took 596 pages. | | Pass A | Pass B | | --- | --- | --- | | Window (UTC, November 8, 2026) | 13:43:33–13:45:25 | 13:45:47–13:47:43 | | Latest records | 59,506 | 59,508 | | Marked active | 58,565 | not re-counted | | Marked deprecated | 941 | not re-counted | | Marked deleted | 0 | not re-counted | | `isLatest` true | 59,506 | filtered by the query | 58,565 + 941 = 59,506. The second pass is the one that supports the namespace and repository figures further down. Its total is two higher. The count of records with no repository URL stayed at 20,434, so the net addition was two records that have a repository URL. The top ten namespaces had the same counts on both passes. The [about page](https://modelcontextprotocol.io/registry/about), read at 13:48 UTC the same day, calls this "the official centralized metadata repository for publicly accessible MCP servers" and says it is backed by Anthropic, GitHub, PulseMCP, and Microsoft. The same page says the registry is in preview, and that breaking changes or a data reset can still happen. A count from a preview catalog is a snapshot of that catalog. It is the wrong number to engrave. ## What does the registry actually list? A record is a `server.json` document. The about page says it carries a reverse-DNS name such as `io.github.user/server-name`, a place to get the server (a package, or a remote URL), and discovery text. Package registries such as npm host the code. The MCP registry hosts the pointer. The page also says the metadata is "deliberately unopinionated." Ratings and curation are left to downstream marketplaces, which are expected to pull the API on a slow cadence, the example given being once an hour. Host applications are told not to consume the official registry directly. Two records from the same hour show the range of what "listed" means. At 13:48:44 UTC, `io.github.github/github-mcp-server` was active, with the description "Connect AI assistants to GitHub - manage repos, issues, PRs, and workflows through natural language." In the same minute, `com.microsoft/azure` was active, described as Azure MCP tools for agents. Both sit in the same latest-version list as the bulk namespaces below. The API flag on all of them, in pass A, was `active` or `deprecated`. Nothing in the response is a quality score. The status enum in the OpenAPI file is `active`, `deprecated`, and `deleted`. Pass A returned no latest row with status `deleted`. Deprecated means the publisher or the registry has marked that entry retired. It does not mean the other 58,565 were reviewed by a person. Spam controls, from the same about page, are namespace authentication (GitHub, DNS, or an HTTP challenge), character limits on free-form fields, and manual takedown. The page says stricter rate limits, automated detection, and community reporting are under consideration. Authentication stops a stranger from publishing under your name. It does not stop you from publishing thousands of servers under a name you do own. ## Why can one namespace hold thousands of servers? Because the list is metadata, and a verified namespace can attach that metadata to as many names as it likes. Pass B found **36,737** namespaces across **59,508** latest records. **33,307** of those namespaces, 90.7%, had exactly one server. The crowd is real. So is the pile at the top. Twenty-one namespaces had 100 or more latest records. Together they held **11,211** records, 18.8% of pass B. The top ten held **9,821**. | Namespace | Latest records | | --- | --- | | io.github.sadri-dridi | 2,366 | | io.github.pipeworx-io | 1,915 | | io.github.mrfentmen | 1,804 | | io.github.mcp-dir | 1,115 | | io.usefulapi | 884 | | io.github.theluckystrike | 409 | | io.github.CloudCorpRecords | 386 | | io.github.Evozim | 375 | | io.github.CSOAI-ORG | 354 | | ai.smithery | 213 | These ten counts matched on pass A and pass B. 2,366 / 59,506 is 4.0% of the first snapshot in a single namespace. The records inside a large namespace are not copies of one another, and they are not all the same kind of product. At 13:45:35 UTC the first five latest rows for `sadri-dridi` were `aba-ok`, `abnf-ok`, `abs-ok`, `acah-ok`, and `acam-ok`. Each description said the value is discarded (an ABA routing-number shape, an ABNF rule count, an absolute-value sign, and two header counters). Each repository URL was `https://github.com/sadri-dridi/named-mcp-utilities`. Each remote URL was a path under `agent-observatory-sensor.nolimit-observatory.workers.dev`. That is five rows. The namespace has 2,366. This page did not open the other 2,361. The same search-and-read on the next two namespaces, at 13:47:56 UTC, looked different. The first `pipeworx-io` rows were `19hz` and `42matters`, each with its own GitHub repository and a remote under `gateway.pipeworx.io`. The first `mrfentmen` rows were `1c-enterprise-mcp` and `4d-mcp`. Both pointed at `https://github.com/mrfentmen/awesome-servers`, both had no remote URL in the record, and both descriptions claimed "20 real tools" for a named programming stack. Two rows are not a description of all 1,804. Search on this API is a substring match, and a short query returns names outside the namespace you typed. The counts in the table come from splitting the `name` field on the first slash during a full walk, not from the search box. ## How many records point at a code repository? On pass A, **39,072** latest records included a repository URL. **38,980** had `source: github` and **92** had `source: gitlab`. 38,980 + 92 = 39,072. The other 20,434 records had no repository URL in that pass, and pass B found the same 20,434 without one. Pass B collapsed the repository URLs to **30,687** distinct strings. 39,074 records in that pass had a URL (59,508 − 20,434), so thousands of server names share a repository. The sadri-dridi sample is the plain version of that pattern: many names, one repo. A star count on that one repository would still be one number. The registry count would still be thousands. Anyone adding those two numbers together is adding a catalog to a popularity vote. Other fields from pass A, which are easy to misread as "servers": | Field | Count | What it is | | --- | --- | --- | | At least one remote URL | 39,709 | Servers that list a remote endpoint | | Remote entries of type `streamable-http` | 39,519 | Entries, not servers. One server can list several | | Remote entries of type `sse` | 1,139 | Entries | | At least one package | 23,683 | Servers that also point at a package registry | 39,519 + 1,139 = 40,658 remote entries, which is more than 39,709 servers. The extra entries are second remotes on a server that already had one. They are not 40,658 extra products. ## What should you count instead? Count the thing the decision needs. If the question is "how large is the official catalog right now," the answer on November 8, 2026 is 59,506 latest records at 13:45 UTC, or 59,508 at 13:47 UTC, with the preview warning attached. If the question is "how many publishers used the catalog," pass B's namespace count is 36,737, and 33,307 of them published a single server. If the question is "how many code repositories are named," the distinct URL count is 30,687, and 20,434 latest records name none. If the question is "what should an agent be allowed to call," none of those totals is the answer. The about page hands that job to a marketplace. A harness still decides which servers are connected and what the agent may do with them. The [tool JSON](/blog/why-tool-json-costs-more-than-your-system-prompt) for a server is billed on every turn whether or not the model calls it, so connecting a namespace of 2,366 counters is a cost decision, not a completeness decision. [MCPHallu](/research/mcphallu-mcp-agent-hallucinations), posted November 5 and broken down on this site on November 8, scores a different question again: when an agent does have a tool list, which failures show up. Its 36 servers were a benchmark set. They are not a sample of these 59,506 records. The registry will not stay at 59,506. It gained two records in the two minutes between these passes, and the about page says a preview catalog can still be reset. The useful habit is to store the window with the count. The number without the window is already a different list. ## What Is BioBasicsBench? How Well AI Agents Handle Everyday Wet-Lab Science URL: https://www.aiagentsinfo.com/blog/what-is-biobasicsbench Published: 2026-11-08. Updated: 2026-11-08 Category: news What is BioBasicsBench? It is OpenAI’s November 6, 2026 benchmark, built with Ligo, that scores AI agents on 100 raw wet-lab tasks. At the xhigh setting, GPT-6 Astra scored 66.7% of rubric credit. On November 6, 2026, OpenAI published [Measuring how AI agents handle everyday wet-lab science](https://openai.com/index/measuring-how-ai-agents-handle-wet-lab-science) and introduced BioBasicsBench. The [benchmark](/glossary/benchmark) has 100 tasks, built with Ligo, and it measures how well an [agent](/glossary/agent) interprets raw experimental data from routine wet-lab work. At the xhigh setting, which allows more reasoning per task, GPT-6 Astra earned 66.7% of the available rubric credit. GPT-5.6 Sol earned 43.8%. OpenAI says the results show progress on routine lab analysis, with room left for images, controls, and experimental context. This page was published on November 8, 2026. The OpenAI post, Ligo’s site, and the same-week write-ups cited below were re-read that day. The numbers in the tables are from OpenAI’s page. ## What is BioBasicsBench? BioBasicsBench is a 100-task agent evaluation of everyday wet-lab analysis. OpenAI describes Ligo, its partner on the benchmark, as a biotech company focused on enzyme design. Ligo’s own homepage, checked November 8, uses the name Ligo Biosciences and says the company builds tooling to manipulate chemistry and biology, starting with enzyme design. The 100 tasks sit in five areas of routine lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. A task can be as direct as reading the molecular weight of a DNA sample on a gel, or it can be a detailed assay analysis. OpenAI says all of them are work a wet-lab scientist should be able to finish reliably. An agent returns a structured answer: the conclusion, measurement, or recommendation the task asked for, and supporting reasoning where the task requires it. Most tasks test several skills at once. OpenAI names biological knowledge, reading images and other data, error detection, and protocol understanding. A figure caption on the same page says skill tags overlap, so those tags are not a second task count. R&D World and Unite.AI, both dated November 6, and AIA Affairs, dated November 7, describe the same release: 100 tasks, built with Ligo, scored on raw experimental evidence. Where a later paragraph gives a score, the figure is OpenAI’s. ## Why is raw wet-lab evidence harder than a paper figure? OpenAI’s reason for a new benchmark is the evidence itself. The post says everyday wet-lab work involves failed experiments, ambiguous images, and conflicting observations. Benchmarks built from charts and images chosen for scientific papers can miss that ambiguity, and the troubleshooting that goes with it. A paper figure is the picture someone selected after the experiment made sense. The file on the bench is whatever the instrument wrote, including a control that does not match the label, a microscope pass that is a focus series rather than a biological change, and a count that is easy to inflate. Misreading an image, or missing a problem in the controls that check a result, can send a scientist toward a conclusion the run does not support, and toward more experiments that will not answer the question. That is an agent-evaluation problem, not only a biology problem. A [behavioral evaluation](/glossary/behavioral-evaluation) checks one observable action. An end-to-end benchmark asks whether the job, scored against a rubric, came out right. BioBasicsBench is the second kind of test. The writing guide for the smaller check is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). The software around a model, which this benchmark does not score, is an [agent harness](/glossary/harness). ## How were the 100 tasks written and reviewed? Scientists created the tasks from their own experiments. Each question has expert-written grading criteria. OpenAI describes three reviews after that: 1. An internal review by a Ligo scientist who had not written the task. 2. An external review by a PhD-level scientist working in a wet lab. 3. A final internal review. Reviewers checked the prompt, the experimental evidence, the rubric, the author notes, and a graded model attempt. The rubric can give credit for the conclusion, for a number, for the reasoning, or for recognizing that the evidence is not enough. A correct final line and a shaky analysis do not have to receive the same score. ## What experimental data is the agent given? The inputs keep the forms scientists already use: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports, and videos of bench work. One task can include several kinds. Reference sheets, such as ladder charts and plasmid maps, support the reading. They are not a separate experiment. OpenAI’s counts overlap. Adding the column does not produce 100.
Kind of data Tasks What OpenAI says it includes
Gels and blots 31 Agarose DNA gels, SDS-PAGE gels, Western blots
Protocols, notes, and specs 27 Protocols and SOPs, lab notebooks and run records, photographed handwritten notes, catalogues, instrument specs, published methods
Tables and instrument exports 20 CSV and Excel exports, lane layouts and inventories, raw flow cytometry (FCS) files
Bench, apparatus, and reagent photos 18 Benches, gel rigs, reagent and culture tubes, a PCR plate, tube racks, pH strips, a concentrator, a freezer, a liquid-handler deck
Microscopy 14 Raw brightfield and fluorescence images, multi-channel sets, z-stacks, time-lapse movies
Instrument screens and plots 13 Photos of flow cytometer, microscope, NanoDrop, plate reader, and balance screens; exported standard curves, chromatograms, and plots
Sequencing files 12 Raw long-read FASTQs, FASTA sequences, GenBank maps
Protocol videos 7 Streaking, inoculation, serial dilution, SDS-PAGE tank and incubator setups, weighing on a balance
Agar plates and colonies 7 Transformation plates, blue/white screens, antibiotic-disc plates
Reference sheets 21 Ladder charts, plasmid maps, pH charts. OpenAI marks these as supporting material.
## How does BioBasicsBench score an answer? OpenAI evaluated seven models: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna, and GPT-6 Astra. The evaluation covers 37 model-and-reasoning-effort settings. Each setting is five attempts on each of the 100 tasks. That is 18,500 results. The metric is mean reward: the average task score, as a percentage of the rubric credit available on that task. The error bars are 95% confidence intervals from resampling whole tasks, with the five attempts for a task kept together. A task is the unit that moves in the resample, not a single attempt. OpenAI also plots mean reward against average output tokens per task, on a logarithmic scale. Dashed orange lines are GPT-5 models. Solid blue lines are GPT-6 models. Effort settings differ by model. The chart includes max-effort points where OpenAI ran them. The table below is the xhigh setting, the highest one published for all seven models. ## How well did the models do at the xhigh setting? Xhigh is the highest reasoning-effort setting OpenAI reports for every model in the table. Each row is all 100 tasks and five attempts per task.
Model Mean reward 95% CI Output tokens per task
GPT-6 Astra 66.7% 60.3–73.1% 4,704
GPT-6.1 Sol 62.0% 55.6–68.4% 5,238
GPT-5.6 Sol 43.8% 37.6–50.0% 9,099
GPT-6 Luna 38.3% 32.3–44.5% 13,120
GPT-5.5 35.4% 29.7–41.0% 11,135
GPT-5.4 33.2% 28.3–38.4% 13,717
GPT-5.2 31.2% 26.2–36.3% 27,620
Two comparisons are firmer than the rest of the ranking. Astra’s interval (60.3–73.1%) and GPT-5.6 Sol’s (37.6–50.0%) do not overlap. GPT-6.1 Sol’s interval (55.6–68.4%) and GPT-5.6 Sol’s also do not overlap. Astra’s interval and GPT-6.1 Sol’s do overlap, so the 66.7% to 62.0% gap is the point estimate, not a separation the confidence intervals establish on their own. GPT-6 Luna’s 38.3% sits below GPT-5.6 Sol’s 43.8%, and those intervals overlap too. On this table the higher scores are Astra and GPT-6.1 Sol, not every model with a GPT-6 name. Token use moves the other way. At xhigh, Astra averaged 4,704 output tokens per task. GPT-5.6 Sol averaged 9,099. GPT-5.2 averaged 27,620, the most in the table, at the lowest mean reward. OpenAI says Astra’s gain over GPT-5.6 Sol shows up even at low reasoning effort, and that stronger native multimodal ability may be what moves everyday wet-lab analysis. The post does not print the low-effort percentages in the table above, so those settings stay on OpenAI’s chart. Unite.AI’s November 6 article reports the same xhigh pair that leads the table: 66.7% (60.3–73.1%) for GPT-6 Astra and 43.8% (37.6–50.0%) for GPT-5.6 Sol, with 4,704 and 9,099 output tokens. AIA Affairs reports those two headline scores and the 37 settings. This page keeps the full seven-row table from OpenAI. ## Where did GPT-6 Astra improve on GPT-5.6 Sol? OpenAI groups the Astra examples under three headings. Each one is a case where the older model finished a plausible story and Astra stayed with what the file could show. **Recognizing the limit of the data.** In one cell-imaging task, GPT-5.6 Sol classified cells as dead from the signal of a dye used to mark dead cells. OpenAI’s caption says a second channel can share light with that detector: the SYTOX-labelled window is 470–540 nm, the FUN-1 green window is 525–580 nm, and they overlap at 525–540 nm. The shared window permits interference, and the amount was not measured. Brightness alone could not justify the dead-cell call. Astra treated that as ambiguity and asked for controls that could separate the signals. OpenAI calls this the trap of over-interpreting data. **Multimodal detail.** Another task asked for a conclusion from a DNA cloning experiment, where the controls do the work. GPT-5.6 Sol saw no colonies on the negative-control plate and reasoned from that empty plate. GPT-6 Astra read the rim. OpenAI’s caption says the label “LB Kan50” identifies kanamycin, a different antibiotic from the ampicillin used in the experiment. Astra rejected the plate as a reliable control. **Experimental context.** A third task showed cells coming into and out of focus across microscope frames. GPT-5.6 Sol looked for a biological reason the cells seemed to disappear. GPT-6 Astra recognized a z-stack: a series of images taken at different depths. OpenAI’s point is that the reading depends on knowing what the instrument was doing, not only on what a single frame looks like. ## Where do the models still fail? OpenAI states the limit in one sentence: higher benchmark scores do not yet mean models can make reliable judgments in the lab. The examples that follow are Astra failures. In each one, an assumption, an expected answer, or a supplied label outweighs the evidence, or a correct intermediate observation never reaches the final decision. **An assumption overrides the controls.** On a colony PCR gel, Astra can tell the colony bands apart as two groups. It then assumes a larger PCR product would mark successful cloning, and it proposes lanes 4, 6, 7, and 8. The reference comparison of the controls selects lanes 5, 9, and 10. OpenAI says the visual grouping is right and the identities attached to the groups are reversed. Selecting those colonies would lead to purifying the incorrect plasmid. **A contradictory observation is dropped.** Astra concludes that vacuoles, compartments inside cells, are generally darker than the surrounding cell, even though some vacuoles in the same image are lighter. OpenAI notes that changing focus can reverse the contrast, so the darker examples do not make a general rule. Astra comments that contrast varies with focus, then gives more weight to the cells it calls better focused and keeps only the darker description. The lighter regions were seen and left out of the conclusion. **A routine count is off.** Astra’s counting marks on one plate do not line up with individual colonies. OpenAI says it fails to separate close neighbors, misses visible colonies, and marks empty agar. The net result on that plate is an overcount of about 12% against the benchmark reference. With the plated volume and dilution held fixed, that overcount inflates the estimate of how efficiently DNA was introduced into the cells by the same proportion. These are published evaluation failures, not a claim that the rest of the 100 tasks failed the same way. They are the cases OpenAI chose to show what a 66.7% mean reward still gets wrong. ## How do you evaluate an AI agent on experimental data? BioBasicsBench is a worked example of that question. The design choices are the part worth copying, separate from any one model score. The task starts from a real run, not from a cleaned figure. The grader is a rubric with partial credit, so a right-looking conclusion can still lose points for the reasoning, and a refusal can earn credit when the evidence is too weak. Five attempts per task keep one lucky transcript from being the score. The confidence interval resamples tasks, which is the right unit if the hard part is the experiment rather than the wording of a single try. Review is independent of the person who wrote the item: someone inside Ligo who did not author it, then a PhD-level wet-lab scientist outside that draft, then one more internal pass that includes a graded model attempt. What the benchmark does not measure is the [harness](/glossary/harness): the loop, the tools, the memory, and the permissions around a model. OpenAI’s published comparison is seven models at different reasoning-effort settings. A team that wraps one of those models in extra tools, or that lets a person approve the answer, is running a different test. [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) is the definition this site uses. [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness) is the guide for checking one action inside that software. Yesterday’s note on [harness engineering at scale](/blog/what-is-harness-engineering-at-scale-thoughtworks-radar) is about owning that software. It is not a score on wet-lab evidence. Two other pages on this site answer different benchmark questions. The [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026) lists public SWE-bench and Terminal-Bench scores for coding agents, each with the date on its source. The [benchmark leaderboard](/trackers/benchmarks) publishes a cell only when it has a source, a date, and the harness that produced it. BioBasicsBench is not one of those coding rows. The source for the table above is the November 6 OpenAI post. For someone deciding whether to trust an agent on experimental data, the practical test is the one OpenAI’s failures illustrate. Did the agent read the control, or the assumption it brought in? Did a contradictory observation survive into the final sentence? Did the agent say when the file cannot support the claim? Mean reward summarizes those checks. The checks are the result a builder can act on. ## What should a builder take from the score? A 66.7% mean reward is progress against GPT-5.6 Sol on this rubric, at fewer output tokens, on evidence that includes gels, plates, microscopy, sequencing files, and bench video. The 95% interval on that score runs from 60.3% to 73.1%, and the same post documents misses on controls, contrast, and counting. OpenAI’s closing line is that accelerating biological research depends on getting that everyday work right, and that progress here could leave scientists more time to choose the next experiment. The same paragraph says tracking these capabilities will help OpenAI understand their implications for biosafety and biosecurity. That is a statement about monitoring the skill. It is not a claim that the current scores are safe to treat as a lab assistant. OpenAI’s products for building and using agents live elsewhere on this site, and neither page is a BioBasicsBench result. The open-source toolkit is the [OpenAI Agents SDK](/directory/openai-agents-sdk). The consumer product is [ChatGPT](/directory/chatgpt). This article is filed under [news](/topics/news), [research](/topics/research), and [task performance](/topics/task-performance). ## Frequently asked questions ### What is BioBasicsBench? It is a benchmark OpenAI introduced on November 6, 2026, with Ligo. One hundred tasks cover protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Each task asks an agent to interpret raw experimental data, catch errors, and notice when the evidence cannot support a conclusion. The glossary entry for this kind of test is [benchmark](/glossary/benchmark). ### How well do AI agents handle wet-lab science on this benchmark? At xhigh, GPT-6 Astra scored 66.7% of available rubric credit (95% CI 60.3–73.1%) and GPT-6.1 Sol scored 62.0% (55.6–68.4%). GPT-5.6 Sol scored 43.8% (37.6–50.0%). GPT-6 Luna, GPT-5.5, GPT-5.4, and GPT-5.2 scored 38.3%, 35.4%, 33.2%, and 31.2%. OpenAI says the higher scores are not yet reliable lab judgment. ### How do you evaluate AI agents on experimental data? Use tasks from real runs, grade with a rubric that can award partial credit, repeat the attempt, and report uncertainty at the level of the task. BioBasicsBench also grades the decision to withhold a conclusion. A single visible action, such as a tool call, is a [behavioral evaluation](/glossary/behavioral-evaluation). The longer guide is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). ### Who reviewed the BioBasicsBench tasks? OpenAI says each task was reviewed by a Ligo scientist who had not written it, then by a PhD-level scientist working in a wet lab, then in a final internal review. That last pass covered the prompt, the evidence, the rubric, the author notes, and a graded model attempt. ### Does a higher BioBasicsBench score mean a model can work in a lab? OpenAI says it does not. The post shows GPT-6 Astra still overriding controls on a colony PCR gel, dropping lighter vacuoles that contradict a darker-vacuole rule, and overcounting one plate’s colonies by about 12%. ### Which models did OpenAI evaluate? Seven of its own: GPT-5.2, GPT-5.4, GPT-5.5, GPT-5.6 Sol, GPT-6.1 Sol, GPT-6 Luna, and GPT-6 Astra. The post reports 37 model-and-reasoning-effort settings and 18,500 graded results. It does not report a model from another lab. ## What Is T3 Code? 51,402 GitHub Stars on an Open-Source Control Plane (2026) URL: https://www.aiagentsinfo.com/blog/what-is-t3-code Published: 2026-11-08. Updated: 2026-11-08 Category: guide 51,402 GitHub stars at 03:46 UTC on November 8, 2026. T3 Code is a free MIT control surface for coding agents you already subscribe to, and release v0.1.7 shipped on November 4. At 03:46:26 UTC on November 8, 2026, the GitHub API reported 51,402 stars for [pingdotgg/t3code](https://github.com/pingdotgg/t3code). The same response reported 9,893 forks, an MIT license, and an empty description field. The latest release was v0.1.7, published November 4. [T3 Code](https://t3.codes/) is the project’s open-source control plane for coding agents you already run. The README’s own phrase is narrower: an “agent harness control surface.” This page was published November 8, 2026. The homepage, the README, the FAQ, the install guide, the permission and remote docs, the ACP provider page, the license file, and both GitHub API calls were read that morning. Where those pages name different agents, the table below keeps them apart. ## What is T3 Code? T3 Code is a desktop, web, and mobile control surface for coding agents already installed on a machine. The [homepage](https://t3.codes/), read November 8, says “the open-source control plane for coding agents” and “bring your own subscription.” The [README](https://github.com/pingdotgg/t3code/blob/main/README.md) says it “enables control of the agents on your machine” from an iOS app, an Android app, a [web app](https://app.t3.codes), and an Electron desktop app. The repository homepage field points at t3.codes. The MIT license file on main is copyright 2026 T3 Tools Inc. It is not a new coding model. The FAQ’s cost answer says you bring your own subscription and still pay your provider. The homepage says T3 Code does not resell tokens. You plug in credentials you already have, and the app orchestrates those agents. The repository was created February 8, 2026. The API’s `pushed_at` on the snapshot below was November 8, 2026 at 03:41:34 UTC, a few minutes before the star count was read. The default branch is `main`. The primary language field was TypeScript. ## Is T3 Code a harness? The FAQ answers that question with no. Read November 8, the “Is T3 Code a harness?” section says a harness is what turns an LLM into an agent: it gives the model tools, a way to call them, runs them, and hands the results back. It names Claude Code as an interface plus a harness underneath, the Anthropic Agent SDK, and Codex as the app plus the Codex App Server. The next sentence is the definition that matters here: T3 Code uses those harnesses. It is not a harness, and it does not work without one. The same answer uses a kitchen line. The model is the chef. The harness is the kitchen. T3 Code is the restaurant that lets you bring whatever kitchen you want. That matches the definition on this site. A [harness](/glossary/harness) is the runtime around a model: the loop, the [tools](/glossary/tool-use), the memory, the permissions, and the stop conditions. [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) is the longer guide. T3 Code sits outside that runtime and drives it. The homepage does not use the FAQ’s wording. Its testimonials, on the same page the same morning, include the line “Since I've started using T3 Code I no longer care which harness is used,” and another that calls T3 Code “the one AI harness.” Those are quotes the homepage prints. They are not the FAQ’s definition. The FAQ is the project’s own answer to the question, and the answer is no. ## Which coding agents can it control, according to which page? A single merged list would be wrong. On November 8, 2026, the homepage, the README, the FAQ, the install guide, and the ACP page named different sets. Kiro, Pi, Muse Code, and fx each appear on some of those pages and not on others. The release notes for v0.1.7 add one more line that the README’s install warning does not: “Kiro (beta). Use Kiro as an Early Access provider.” | Page, read November 8, 2026 | Agents that page names | | --- | --- | | Homepage hero, t3.codes | Claude Code, Codex, Antigravity, OpenCode, Cursor, and Grok | | README introduction | Claude Code, Codex, Cursor, Grok Build, OpenCode, Google Antigravity, and fx | | README install warning | Codex, Claude, Cursor, Grok Build, OpenCode, Antigravity, and fx | | FAQ, “bring your own” | Claude Code, Codex, Cursor, Grok Build, OpenCode, Google Antigravity, Pi, or Muse Code | | Install guide, providers table | Codex, Claude, Cursor, Grok Build, Kiro, OpenCode, Antigravity, Pi, Muse Code, fx | | ACP providers page, first-class examples | Codex, Claude, Cursor, Grok, OpenCode, Pi, and Antigravity. The page says these do not appear in the ACP Registry list | The install table is the longest list in the docs read that morning. It also says how each one is signed in: `codex login`, `claude auth login`, Cursor’s `agent login`, `grok login`, `kiro-cli login`, `opencode auth login`, Google sign-in for Antigravity from T3 Code’s settings, `pi` then `/login`, `muse login`, and `fx login`. Antigravity and a ChatGPT-connected Codex can use a managed runtime without a binary on `PATH`. The others need the CLI on the server machine, which is the computer running T3 Code, not the phone. The ACP page is a second door. It says T3 Code can run installed ACP commands, or coding agents from the official [ACP Registry](https://agentclientprotocol.com/get-started/registry). Registry agents bring their own models, tools, and sign-in. T3 Code provides projects, threads, checkpoints, and task delegation. The page’s example of a local command is DeepSeek Harness: executable `dsh`, arguments `--profile` and `acp`. Registry agents are third-party code. The page says to review an agent’s source and license before adding it. [Claude Code](/directory/claude-code), [Codex](/directory/codex), and [Cursor](/directory/cursor) are the three of those names with directory pages here. The [November 5 framework snapshot](/reports/ai-agent-framework-github-stats-november-2026) is a different object: 19 agent-framework repositories, with their own star counts. T3 Code is a control surface for agents you already run. It is not one of those 19 rows, and this page does not reprint that table. ## How much does T3 Code cost? The FAQ’s answer is nothing. The heading “How much does T3 Code cost?” says there is no way to pay the project to use T3 Code. You bring your own subscription. It is free and open source (MIT). You still pay your provider the way you already do. The README’s version of the same point is the heading “Wait, what are you selling me?” and the answer “Nothing.” T3 Connect, the built-in remote link, is a separate sentence. The FAQ says it is free too, for now. Every account can connect up to 3 machines running T3 Code. Phones and browsers that connect to them do not count. Need more machines, and the FAQ says to link a paid T3 Chat account. Pro raises the cap to 10 machines. Premier raises it to 100. The free T3 Chat plan keeps the normal limit of 3. The FAQ does not print a dollar price for Pro or Premier, and this page does not add one. The same answer includes the hedge worth quoting: “We don't want to charge for this, but if the bill gets unacceptable we may have to change course.” The alternatives it names are forking the project, self-hosting, or using Tailscale. The remote-access doc repeats the 3, 10, and 100 machine caps and does not repeat the words “for now.” What you still pay is the agent subscription. [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing) is the dated comparison of those plans. This page does not restate those prices. The FAQ also says T3 Code can take an API key or an OpenRouter endpoint on a separate provider instance, so a subscription is the default path, not the only one the docs describe. ## How does remote and mobile control work? The agents run where T3 Code runs: a desktop, a server, or a VPS. You connect to that machine from somewhere else. The FAQ says less than 5 percent of that author’s prompts run on a laptop, and that `npx t3 up` on the machine is the remote path they use. The phone does not become the computer. The FAQ’s mobile answer says the iOS or Android app connects to your computer with T3 Connect or a pairing link, the phone is a remote, and the computer does the work and needs to stay on. The remote-access page, read the same morning, names three ways onto the host. T3 Connect is the account link: sign in on the host, enable it, and pick that environment on the other device. `t3 up` asks whether other devices should use T3 Connect, the local network, or both. On Linux and macOS it then runs T3 Code as a background service. On Windows the background service is not available, so `t3 up` prints a `t3 serve` command to leave running in a terminal. LAN pairing uses a link or QR code from `t3 pair` or `t3 up --lan`. The page also documents Tailscale and other VPN routes, and a desktop mode that turns the local environment off so that computer only drives work running elsewhere. The web client is [app.t3.codes](https://app.t3.codes). The README links the iOS app as “T3 Code: Remote Claude & More” and the Android package `com.t3tools.t3code`. This page did not open the store listings for ratings or prices. That split is the opposite of an agent that operates the screen in front of you. [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer) is the [computer-use](/glossary/computer-use) guide. T3 Code’s phone is a remote control for a machine that stays on. One privacy sentence from the FAQ belongs next to the remote claim. Your code and prompts go from your machine to your provider, not to T3. With T3 Connect, traffic rides a Cloudflare tunnel on your machine. The FAQ says the relay sets up the link and does not proxy your HTTP or WebSocket sessions. T3 Code does send product usage events, such as provider, model, and token totals. Prompts and file contents are not in that list. `T3CODE_TELEMETRY_ENABLED=false` turns it off. That is the FAQ’s account, read November 8. ## What permission and approval modes exist? The permission-modes page lists four modes. You pick one in the message composer, and it applies to that thread. The initial default is Full access. Projects can override the environment default. An administrator can cap the modes on a machine with a managed policy. | Mode | What the permission-modes page says | | --- | --- | | Supervised | Requests approval for commands and file changes. | | Auto-accept edits | Approves file edits automatically. Other actions can still require approval. | | Auto | Uses the provider’s automatic review to approve routine actions and ask about others. | | Full access | Allows commands and edits without approval prompts. | Those modes are an [approval gate](/glossary/approval-gate) on the thread, and a [human in the loop](/glossary/human-in-the-loop) when the mode still asks. The page is explicit that providers do not all implement the four the same way. Auto uses automatic review on Codex, Claude, Cursor, Grok, and fx. OpenCode and Antigravity, which the page says lack an equivalent, fall back to asking. Claude can still ask in Full access when its own settings disable bypass permissions. Antigravity can still send native approval requests in Full access. | Provider limit, from the same page | What happens | | --- | --- | | Muse Code | Only Supervised and Full access. A thread already set to another mode runs in Supervised. | | Kiro | Only Supervised and Full access. In Supervised, Kiro also asks you to review a turn’s file changes. | | fx | No Auto-accept edits and no Full access. A thread already set to either runs in Supervised. | | Grok | No Auto-accept edits. A thread already set to it runs in Supervised. | | ACP Registry agents | They run their own tools in their own mode. T3 Code answers their approval requests by the permission mode. | Approve or reject in the conversation. The page says subagent approval requests appear in the parent conversation with the subagent’s name, so you can answer without switching threads. The v0.1.7 notes call that “child approvals.” ## How is T3 Code different from Claude Code or the Codex app? Claude Code and the Codex app are harnesses T3 Code can drive. T3 Code is the surface in front of both, plus the other agents in the table above. The FAQ’s “why use T3 Code” answer says the project was inspired by the Codex desktop app, Conductor, Claude Desktop, and Cursor Glass, and that none met its bar. What it lists instead: bring your own subscription, remote with `npx t3 up`, a mobile app, and source you can fork. The README adds that every agent thread writes to its own work, and the homepage says one button can open a GitHub pull request with a generated title, body, and changelog. The practical difference is whose software is the loop. In Claude Code, you are in Claude Code. In the Codex app, you are in Codex. In T3 Code, the FAQ says those products keep doing the tool calls, and T3 Code is the place you start the thread, pick the permission mode, and attach a phone. Switching agents does not switch you to a new model sold by T3. It switches which installed harness the thread uses. Two limits from the FAQ are part of that difference. First, T3 Code drives Claude through Anthropic’s Agent SDK. The FAQ says that path drains Claude’s 5-hour limit faster than using Claude Code directly, that weekly limits are unaffected, and that the project has reported it to Anthropic. That is the FAQ’s account of a bug, not a measurement this page re-ran. Second, the FAQ says T3 Code does not support local models. The sentence is blunt: locally runnable models are not capable of meaningful engineering work. A reader who wanted an offline model is outside the product the FAQ describes. The cost difference is the license. Claude Code and Codex are paid through their own plans, compared in [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). T3 Code’s FAQ says the control surface itself has no fee. The subscription you bring is still a subscription. ## What do today's GitHub numbers say? This table is the original snapshot. It is the response from `GET https://api.github.com/repos/pingdotgg/t3code` and `GET https://api.github.com/repos/pingdotgg/t3code/releases/latest` at 2026-11-08T03:46:26Z. GitHub’s `open_issues_count` counts open issues and open pull requests together. The homepage the same morning printed “51k+” next to the download button, and a separate line, “Tolerated by over 500,000 devs.” The star count and that dev line are different claims. Only the star count is in the API response. The numbers in the table are licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Credit AI Agents Info if you reuse them. They are a reading of a public API at one minute, not a forecast. The [November framework snapshot](/reports/ai-agent-framework-github-stats-november-2026) is the same kind of dated GitHub reading, for a different set of repositories. | Field | Value at 2026-11-08T03:46:26Z | | --- | --- | | Stars | 51,402 | | Forks | 9,893 | | Open issues and pull requests | 5,806 | | Licence | MIT | | Latest release | v0.1.7, published 2026-11-04T06:39:38Z | | Snapshot | 2026-11-08T03:46:26Z | v0.1.7 is not a draft and not a prerelease. The release name is “T3 Code v0.1.7.” The notes open with 659 changes from 160 contributors, then Kiro as an Early Access provider, managed policies, one-time schedules, child approvals, shareable thread links, usage filters, and model-switch history. The repository’s `created_at` is 2026-02-08T23:46:22Z. The description field in the API was null. Stars on a fast-moving repository move after a snapshot. A count you see on GitHub later on November 8 can differ from 51,402. The row above is the count at the timestamp in the last row. This guide is filed under [harnesses](/topics/harnesses), [tools and SaaS](/topics/tools-saas), and [how agents work](/topics/how-agents-work). ## Frequently asked questions ### What is T3 Code? It is an open-source control surface for coding agents you already have installed. The homepage’s phrase is “open-source control plane.” The README’s phrase is “agent harness control surface.” The GitHub organization is pingdotgg. The MIT license file is copyright T3 Tools Inc. The app does not sell a model of its own. ### Is T3 Code a harness? The FAQ says no. A harness gives the model tools and runs them. T3 Code uses a harness such as Claude Code or Codex, and it does not work if you have not installed one. The glossary entry is [harness](/glossary/harness). The guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). ### Which coding agents can it control? Read the page, not a merged list. On November 8, 2026, the homepage named six, the README named seven including fx, the FAQ named eight including Pi and Muse Code and not fx, and the install guide named ten including Kiro. The ACP Registry is an extra way to add agents the first-class list does not include. ### How much does T3 Code cost? Nothing, according to the FAQ. It is MIT-licensed, and you bring your own subscription. T3 Connect is free for now, for up to 3 machines. T3 Chat Pro raises that to 10, and Premier to 100. Phones and browsers do not count toward the cap. The FAQ names those plans and does not print a dollar price. ### How do you use T3 Code from a phone? Install the iOS or Android app and connect it to a computer that is running T3 Code. The phone is a remote. The computer stays on and does the work. T3 Connect, LAN pairing, and Tailscale are the documented routes. ### What permission modes does T3 Code have? Supervised, Auto-accept edits, Auto, and Full access. The documented initial default is Full access. Muse Code, Kiro, fx, and Grok do not offer every mode. The details are in the table above, from the permission-modes page read November 8, 2026. ## What is an AI employee? A digital worker, not a legal employee URL: https://www.aiagentsinfo.com/blog/what-is-an-ai-employee Published: 2026-11-07. Updated: 2026-11-08 Category: guide What is an AI employee? Workplace software, also called a digital worker, that takes a goal, uses tools, and returns work for a person to review. Always-on AI agents, on the pages checked November 7, 2026, keep a cloud task running after the laptop closes or start on a schedule. The software is not a legal employee. Updated November 8, 2026. ## What is an AI employee? An AI employee is a plain-language name for workplace software that takes a goal, uses tools, and returns work for a person to review. The more precise name on this site is a [digital worker](/glossary/digital-worker). The working system underneath is usually an [agent](/glossary/agent): a [language model](/glossary/llm) plus a [harness](/glossary/harness). People also say “digital worker,” “AI teammate,” and “employee agent.” The words are ahead of the products. On November 7, 2026, Anthropic’s workplace product is [Claude Cowork](https://claude.com/product/cowork). OpenAI’s is [ChatGPT Work](https://openai.com/chatgpt-work/), with shared [workspace agents](https://openai.com/business/workspace-agents/) as a separate research preview. A chat reply that only explains the task is still chat. A script that always does the same five steps is still fixed automation. The software is not a legal employee. It has no duties of its own, and a mistake is still the organization’s to own. [Deloitte’s September 10, 2026 essay](https://www.deloitte.com/global/en/insights/topics/technology-management/ai-agents-human-workplace.html) treats “agents as labor” and “agents as infrastructure” as two choices a company might make, and it tells leaders to watch for moments when agents are treated as the responsible party in a review. That is a warning about accountability, not a feature list. This guide is for adult work and for university study. School-account policy is a different page: [Google’s November 2026 K-12 switch](/blog/google-k12-gemini-and-search-off-until-schools-opt-in). Sourced tables for the same jobs are [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks) and [Best AI agent for small business](/best/ai-agents-for-small-business). ## What is a digital worker? A digital worker is the same idea with the job in the foreground. Someone assigns a recurring task, such as a Monday readout or a first-pass contract memo. The software reads the sources it is allowed to read, does the steps, and leaves a draft. [OpenAI’s business guide](https://cdn.openai.com/business-guides-and-resources/a-business-leaders-guide-to-working-with-agents.pdf), a PDF read on November 7, 2026, splits agents into a model, tools, and guardrails. The model interprets the goal and picks the next step. The tools gather information or take an action. The guardrails keep that action inside rules a person set. The PDF has no publication date on the copy read here. It still uses the product name “ChatGPT agent” for the pre-built example, and it names Operator and Deep Research as other examples. Those names are the guide’s. The current workplace page calls the product ChatGPT Work. [Deloitte’s December 10, 2025 Tech Trends essay](https://www.deloitte.com/global/en/insights/topics/technology-management/tech-trends/2026/agentic-ai-strategy.html) uses “digital workers” and a “silicon-based workforce” for the same management question: how a company supervises software that can act, beside the people who still handle shifting requirements. The essay also warns about “agent washing,” its name for automation products relabeled as agents. A new badge on an old script does not make an employee. The September 10, 2026 Deloitte essay is the later workplace piece. It cites a Gartner forecast that the average Fortune 500 company is likely to have 150,000 AI agents by 2028, and a Deloitte study of senior managers in which two-thirds expected multi-agent systems to change how work is done within two to three years, while about a quarter said their processes or workforce were prepared. Deloitte’s 2026 State of AI in the Enterprise survey, cited in that essay, says nearly three-quarters of surveyed organizations plan to use agentic AI at least moderately within two years. A separate Deloitte study cited there puts 93 percent of organizational AI investment toward technology infrastructure and 7 percent toward work and people. Those are Deloitte’s and Gartner’s figures, from that page. They are a reason to write the oversight down. They are not a headcount of digital workers you can hire. ## How does a workplace agent differ from a chatbot and from fixed automation? OpenAI’s business guide draws the line with two everyday systems. A question-answering chatbot can return an answer and can call a tool. Asked for the travel policy, it can fetch the current file from an HR system. The guide says it still cannot plan the offsite: venues, costs, and an agenda, unless every step is programmed. A workflow automation follows steps and rules, including simple “if this, then that” logic. The guide’s example is an expense system that stops when a new vendor field is blank, and a login rule that locks an account after three failures and a quiet day. The strength is that every branch was written down. The weakness is a condition the author did not list. An agent, in that guide, starts from a goal. It plans, picks tools, and can change the plan when a source is missing or a person answers a question. The guide says the three can sit in one process: a fixed rule for the stable part, a model for one judgment inside that rule, and an agent where the path is not known at the start. The same split is how this site uses the words. A chat reply is a model. An [agent](/glossary/agent) is that model in a loop with tools. The loop, the permissions, and the stop rule are the [harness](/blog/what-is-an-ai-agent-harness). Designing them on purpose is [harness engineering](/glossary/harness-engineering).
Chat Fixed automation Workplace agent
What you hand it A question, a rewrite, or a short draft. A trigger and a rule, such as “lock the account after three failed logins.” A goal and the sources it may use. ChatGPT’s Work guide says the useful tasks have a clear outcome and something you will review.
Who chooses the next step The conversation stays on the reply. OpenAI’s guide says a tool call for a lookup is still not a plan. The workflow does. A model may be allowed one step, such as classifying a request, and then control returns to the script. The model proposes the step. The harness runs the tool it is allowed to run.
When a fact is missing It answers from what it has, or it asks you. OpenAI’s guide says a fixed expense flow can stall on a blank field. The guide says an agent can try another source or ask a person, then continue.
What “done” looks like A message in the thread. The predetermined outcome, such as a routed ticket. A file, a scheduled run, or an action in a connected app, held for review when the product requires it.
Where this showed up on November 7, 2026 Claude chat, and ChatGPT’s Chat mode. Claude’s Cowork page says a simple question still gets a quick answer. The workflow examples in OpenAI’s business guide. Deloitte’s 2025 essay calls some “agent” projects this pattern under a new name. Claude Cowork and ChatGPT Work. Shared workspace agents are a research preview on top of ChatGPT for selected plans.
A shopping agent is the same shape pointed at a purchase. That guide is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). A click on a real screen is [computer use](/blog/can-an-ai-agent-use-your-computer). Both are narrower than the phrase “AI employee.” ## What can Claude Cowork do at work? [Claude Cowork](https://claude.com/product/cowork), read November 7, 2026, is Anthropic’s product for research, analysis, documents, and other multi-step work that is not software engineering. The page says you give Claude a goal. It works in the folders and tools you choose. You come back to work for review. Claude Code remains the product for writing, debugging, and shipping code. Cowork uses what Anthropic calls the same agentic approach, aimed at knowledge work. The page’s examples are ordinary office jobs: a scheduled Monday marketing readout pulled from Amplitude and a Drive tracker, a spreadsheet that flags variances, a first-pass memo on each contract in a folder, and a quarterly read of call notes with quotes tied back to the notes. Those are illustrations on the product page, not a promise that every connector is turned on in every account. A few boundaries on that page matter more than the demos: - Cowork is included with a paid plan. The page says it is rolling out on the web and on mobile. The desktop app is what can reach folders and applications on your own computer. - Every conversation can start as chat. When a task needs files, Claude brings in Cowork’s capabilities. Until that unified experience reaches an account, the page says Cowork still sits next to Chat. - Deleting anything needs your approval. A permissions setting can make Claude show its plan and wait before anything significant. You can redirect it or stop it. - The desktop app has a built-in browser for Cowork, separate from your own browser, on Pro, Max, Ultra, and Team. Enterprise admins manage it under Organization settings, then Cowork. Cookie import is described for Chrome, Edge, and Firefox on macOS, and for Firefox on Windows and Linux. - Computer use, which drives the screen, is a research preview for Cowork and Claude Code. The page says Claude tries connectors first, then a browser, and uses the screen last, asking before each new application. The click-by-click account is the [computer-use guide](/blog/can-an-ai-agent-use-your-computer). - Tasks assigned from a phone keep running in the cloud after the laptop is closed, on Pro, Max, Ultra, and Team. Enterprise plans can opt in. Agentic tasks use more of the usage allowance than a short chat, because they coordinate tools and sub-agents. - Enterprise admins can turn Cowork off, set access by role, set spend limits, and stream activity to a SIEM through OpenTelemetry. The page also says Claude can be reached through a Claude account or through Amazon Bedrock, Gemini Enterprise Agent Platform, or Microsoft Foundry. Plugins, on the same page, bundle skills, connectors, and sub-agents so Claude starts with a role’s tools and instructions. A skill is the file format in the [agent skill](/glossary/agent-skill) entry. A connector is one [tool](/glossary/tool-use). ## When does the unified Claude experience turn on? Anthropic’s [Enterprise article](https://claude.com/resources/articles/unified-claude-enterprise), read November 7, 2026, says the unified Claude experience is generally available for Claude Enterprise. Chat and Cowork are one conversation on web, desktop, and mobile. A question stays a chat. A task that needs code or files continues in a cloud environment Anthropic manages, and the person can pick the result up on another device. The article says chats, projects, connectors, skills, and plugins carry over. The [admin article](https://support.claude.com/en/articles/17444462-set-up-and-manage-the-new-unified-claude-experience-for-your-enterprise-organization), read the same day, is the one with the date. Chat and Cowork unified is off by default in existing organizations and turns on by default on November 16, 2026. An Owner or Primary Owner who has not changed the setting sees “Will be automatically enabled on November 16, 2026,” and can choose Enable now or Keep disabled. A choice already saved stays as it was. The switch also needs Chat, and cloud code execution and file creation, to be on. If either is off, nothing turns on by itself. After November 16, the new experience turns on once both are on. The same article excludes organizations that use zero data retention for this setup, a HIPAA-ready configuration, or customer-managed encryption keys. Those organizations keep chat and Cowork as they are. The Enterprise article says the same exclusion. HIPAA support is described as expected after general availability. Customer-managed keys are described as a later release. For organizations that do turn it on, the admin article describes the cloud session in concrete terms. Each conversation that needs agentic work gets its own virtual machine. After a few idle minutes the machine stops and its disk is saved as an encrypted snapshot. The sandbox expires 30 days after it was created. Network egress is off by default for Enterprise. Local files and a linked computer stay optional, and Remote control has to be on before web, mobile, or a scheduled task can use that computer. Cost stays usage-based. The Enterprise article says Claude Opus 5.5 costs 40 percent less to run than Claude Opus 5 on typical workloads, and Claude Sonnet 5.5 costs up to 30 percent less than Claude Sonnet 5 for most work. Those are Anthropic’s comparisons, not a public price card. Admins can turn on Efficient mode, which the help article limits to Sonnet and Haiku in the unified experience, with new sessions starting on Claude Sonnet 5.5 at low effort. Model allowlists and spend limits at the organization, group, and member level still apply. ## What is ChatGPT Work? [ChatGPT Work](https://openai.com/chatgpt-work/), read November 7, 2026, is OpenAI’s agent for a finished piece of work: a presentation, a report, an analysis, a spreadsheet, or a recurring update. The page says it is powered by GPT-6, and that GPT-6 introduces Astra as a tier for work that needs more judgment. It is available on every plan in the Mac and Windows desktop apps, and on Plus, Pro, Business, Enterprise, and Edu on the web and on mobile. [OpenAI’s getting-started guide](https://learn.chatgpt.com/docs/get-started-with-work), read the same day, tells you when to leave Chat. Use Chat for an answer, an explanation, a brainstorm, or a short draft. Use Work when the task has a clear outcome, uses more than one source or tool, and produces something you will review. On the desktop app, Work can also use local files, apps, and the browser when those are turned on. The guide says a person who has been using Codex for non-coding work can stay in Codex or move to Work. The same guide names three models in the selector: GPT-6 Astra for demanding work that needs careful reasoning or a polished file, GPT-6.1 Sol for complex work at a lower cost than Astra, and Luna for simpler tasks. Longer tasks can use more of the plan’s allowance because the product is doing more of the steps. The product page adds the controls around that work. Plan mode gathers context, asks questions, and writes a step-by-step plan you can change or approve before the work starts. The page says more than 1,400 plugins can pull context from tools the team already uses. Scheduled tasks can run once, on a cadence, or when an event happens. Team tasks can delegate a shared job, such as a weekly report, through approved company tools, and teammates with access can edit the instructions. ChatGPT Space is the shared page for editing with the team. Sites turns a plan or a dataset into an internal web page. The desktop app has a built-in browser for work that has to cross tools in tabs. The getting-started guide’s own instruction is the oversight rule in one line: say which sources to use, what good looks like, and when to stop for review. Its sample asks for a draft before anything is shared, and it tells you to follow the notice if a task pauses for a safety review. If the work needs the computer in front of you, that is [computer use](/glossary/computer-use), documented for the desktop app on the computer-use guide. ## What is a ChatGPT workspace agent? A workspace agent is a shared agent a company builds inside ChatGPT, separate from one person’s Work task. [OpenAI’s business page](https://openai.com/business/workspace-agents/), read November 7, 2026, says workspace agents are in research preview for ChatGPT Business, Enterprise, Edu, and Teachers plans. One person defines the job in plain language, connects approved apps, and shares the agent so other people run the same workflow. The page’s examples are recurring work: reviewing leads, summarizing support requests, generating reports, updating tickets, editing documents, and sending messages. Admins control who can build and run agents, which tools those agents can use, and the logs of what ran. The [help article](https://help.openai.com/en/articles/20001143-chatgpt-workspace-agents-for-enterprise-and-business), read the same day, adds the switches. Workspace agents are off by default at launch for ChatGPT Enterprise workspaces. An admin turns them on. People see and run only the agents their role allows. A run can start in ChatGPT, in Slack, on a schedule, or from an API trigger. Write actions, the ones that send, edit, post, or delete, default to Always ask. A builder can set Never ask, or a custom approval for specific writes. The article says to use those write approvals carefully. Action constraints can narrow an app further, for example limiting an email send to one domain. The constraint limits what the agent may ask the app to do. It does not filter a record the app returns through some other allowed call. Files attached to an agent are limited, on that page, to 512 MB per file and 10 GB total. The article says very large collections can keep the agent from running, and it tells builders to add only the files the job needs. Edu appears on both the Work availability line and the workspace-agent plan list. A university that buys ChatGPT Edu is buying a workspace with those controls available, not a promise that every student agent is turned on. The university section below is what three campuses have actually published. ## What do people mean by always-on AI agents? On the pages read November 7, 2026, an always-on AI agent is a workplace job that continues after you leave the chat. Claude Cowork says tasks assigned from a phone keep running in the cloud after the laptop is closed, on Pro, Max, Ultra, and Team. Enterprise plans can opt in. ChatGPT Work can run a scheduled task once, on a cadence, or when an event happens. A workspace agent can start in ChatGPT, in Slack, on a schedule, or from an API trigger. That is still an AI employee in the sense of this guide: a goal, the tools it is allowed to use, and work a person reviews. The short definition is [digital worker](/glossary/digital-worker). The product notes are [Claude](/directory/claude) for Cowork and [ChatGPT](/directory/chatgpt) for Work and workspace agents. Claude’s Cowork page says agentic tasks use more of the usage allowance than a short chat. A saved permission that skips the review is a wider grant, covered in the next section. ## When must a person approve an agent’s action? A [human in the loop](/glossary/human-in-the-loop) is a person who can approve, edit, or stop a step before it counts. An [approval gate](/glossary/approval-gate) is the configured stop for one kind of action. The products checked on November 7, 2026 publish that stop in different places: - Claude Cowork asks you to choose the folders and tools. Deleting needs approval. The permissions setting makes Claude show a plan and wait before a significant action. - In the unified Enterprise experience, admins can turn off “Skip all approvals” and “Automatically approve.” Off means a person cannot hand Claude a standing yes for tools, file edits, and browsing, including scheduled tasks and the built-in browser. “Always allow” for a connector tool can be turned off so Claude asks every time. Remote control is a separate switch: when it is off, web, mobile, and scheduled tasks cannot use a member’s linked computer. - ChatGPT Work’s Plan mode is a review of the approach before the work begins. The getting-started guide tells you to require a draft before a share, and to stay with a task that pauses for a safety review. - Workspace agents default write actions to Always ask. The business page names sending a message and updating a record as the examples. - OpenAI’s business guide says the pre-built agent it describes was trained to seek confirmation before many real-world actions, to ask for oversight before sensitive tasks such as email, and to refuse high-risk actions such as financial transfers. Treat that as the guide’s account of its training rules. Confirm the switch in the product you actually have. [Deloitte’s December 2025 essay](https://www.deloitte.com/global/en/insights/topics/technology-management/tech-trends/2026/agentic-ai-strategy.html) uses Mapfre as a workplace example of the same pause. Agents handle routine claims tasks such as damage assessments. Customer communication stays with a person. Mapfre’s group chief data officer, Maribel Solanas Gonzalez, is quoted calling the design hybrid: the software changes what people spend time on, and risky work still goes through a human worker. That is Deloitte’s interview, not a product manual. A gate you never look at is a rubber stamp. Deloitte’s September 2026 essay says to watch for the moment when approval speed becomes the main measure, because review then multiplies with the volume of agent output and stops being a check. The measurement for a real check is the same idea as a [behavioral evaluation](/blog/how-do-you-evaluate-an-ai-agent-harness): name the action, such as “sent without a person,” and see whether the run still does it. Desktop agents raise a second question, which is whether security tools can see the work. [Palo Alto Networks’ November 2, 2026 post](https://www.paloaltonetworks.com/blog/2026/11/new-cortex-xdr-integration-with-claude-cowork-secures-desktop-ai-agents/) says Claude Cowork does this desktop work inside a small local virtual machine. That boundary also keeps a normal endpoint sensor from watching the agent. Palo Alto says a Cortex XDR integration will put a lightweight sensor inside that machine, and that the integration will be available in private preview in the coming months. The post’s own forward-looking notice says unreleased features may not ship on the stated schedule, or at all. Anthropic’s admin article, separately, says an endpoint sensor can run inside the virtual machine on a user’s computer and is in beta, and that this does not cover the cloud sandbox Anthropic manages. A cloud session and a desktop session are two places to log. [Autonomy](/glossary/autonomy) is how many of these gates you remove. A suggest-only draft is the low end. A saved Always allow, or a write action set to Never ask, is the high end. Money is the case this site already treats as its own gate. The shopping guide and the computer-use guide both say a person stays on the payment step unless a product has a narrower checkout rule you already accepted. ## How are universities using workplace AI? The education pages checked for this guide are about adults in universities, and about the jobs those adults are preparing for. They are access programs, assessment rules, and staff training. They are not a hiring plan for digital workers, and they are not a school product. [OpenAI’s November 5, 2026 post](https://openai.com/index/australian-universities-ai-native-graduates/) describes collaborations with UNSW Sydney, the University of Sydney, La Trobe University, and Adelaide University. It also reports a survey Pureprofile ran for OpenAI: 1,018 university students in Australia, aged 18 and over, studying on campus or remotely, from August 3 to August 7, 2026. In that survey, 81 percent said they already use AI several times a week, 33 percent several times a day, and 28 percent said they had already built an AI agent or assistant for study, work, or personal life. Those percentages are the survey’s. They describe students who already have the tools, which is the access problem the universities say they are trying to even out. The university pages, read November 7, 2026, are more specific than the slogan. [UNSW’s September 9, 2026 announcement](https://www.unsw.edu.au/newsroom/news/2026/09/unsw-partners-with-openai-to-support-the-first-generation-of-ai-) says the partnership will bring ChatGPT Edu to more than 80,000 students and staff. Access that week was being extended to higher-degree research candidates and casual academic staff, on top of access that professional and academic staff already had. The university said it was working toward the whole student community ahead of the 2027 academic year. That is a staged rollout, not a statement that every student could sign in on September 9. UNSW’s assessment rule is Can, Can’t, Must: a course can permit AI, prohibit it where the student has to show the skill alone, or require it when the point of the assignment is to use it. The announcement also describes staff training, research uses such as literature review and coding, and intended links to internships and industry projects. [The University of Sydney’s September 14, 2026 announcement](https://www.sydney.edu.au/news-opinion/news/2026/09/14/students-staff-to-receive-free-chatgpt-edu-access.html) says students and staff will get free, supported ChatGPT Edu, introduced in stages, with more features turned on over time. University data will not be used to train models. The university is also putting $1 million into an AI Sustainability Research and Innovation Fund. Its assessment description is a two-lane rule: secure supervised work where students show what they can do themselves, and open work where they show how they use AI. Cogniti, the university’s own teaching tool, stays the preferred classroom system. ChatGPT Edu is the supplement. OpenAI’s November 5 post adds a date the university page does not: access for all students scheduled for Semester 1, 2027, after a trial. Use the university page for the commitment, and OpenAI’s post for that semester date. [La Trobe’s September 28, 2026 note](https://www.latrobe.edu.au/news/announcements/2026/what-an-ai-first-university-looks-like-in-practice), by Pro Vice-Chancellor Phil Laufenberg, says the OpenAI collaboration commits ChatGPT Edu access for all of about 45,000 students and staff, with the initial rollout already underway. OpenAI’s November 5 post is tighter on the calendar: all 3,500 staff already have ChatGPT Edu, and universal access for those 45,000 students and staff is planned for the first quarter of 2027. The same OpenAI post says La Trobe has made Codex and ChatGPT Work available across the institution, and it points to handbook subjects, including agentic software engineering, from 2027. Laufenberg’s own note emphasizes judgment: a graduate who can decide whether AI helps, check the output, and apply it. That is the human-in-the-loop skill, taught to adults, rather than a claim that the campus has hired agents. Adelaide University appears in OpenAI’s November 5 post as a newer institution building on existing API use, including OpenAI Daybreak Blue for authorised defensive security work. This guide does not have a separate Adelaide page confirming that sentence, so it stays OpenAI’s account. None of these pages says a university has replaced a staff job with an AI employee. The pattern they do describe is the workplace pattern in the table above: give people a capable chat and, on some campuses, agentic tools, then write down where a person still has to show the work. ## What should you decide before calling software an employee? 1. Name the job in one sentence, and name the file or the action that counts as done. “A Monday deck from these two sources” is a job. “Be my employee” is a slogan. 2. Pick the narrowest system that can finish it. A chat answer, a fixed workflow, and an agent are three different grants. OpenAI’s guide says to use the agent where the path has to adapt. 3. List the tools. Claude Cowork and ChatGPT Work both ask you to connect folders, plugins, or apps. A tool you did not connect is a tool the agent should not have. 4. Write the approval gates. Deletes, sends, record changes, and payments are the stops on the pages above. A draft for review is the default a careful team can live with. 5. Decide who is accountable when the draft is wrong. Deloitte’s September 2026 warning is the useful test: if the notes of a mistake name the agent as the responsible party, the accountability has drifted. 6. If the agent runs on a desktop, decide who can see that session. Palo Alto’s November 2 post is the security version of the same question, and it is still a private preview. 7. If the buyer is a university, read the staging date on that university’s own page. “ChatGPT Edu” on a plan list is not the same sentence as “every student has it today.” The [how agents work](/topics/how-agents-work) topic is the loop in general. [Task performance](/topics/task-performance) is where this site follows office work, computer use, and other jobs an agent finishes. [Education](/topics/education) is the shelf for learning and campus tools. [New AI tools and SaaS](/topics/tools-saas) is the shelf for the products. [Harnesses](/topics/harnesses) is the workshop around the model. ## Where can you read more on this site? - This guide’s terms: [digital worker](/glossary/digital-worker), [human in the loop](/glossary/human-in-the-loop), [approval gate](/glossary/approval-gate) - Nearby terms: [agent](/glossary/agent), [harness](/glossary/harness), [guardrail](/glossary/guardrail), [autonomy](/glossary/autonomy), [tool use](/glossary/tool-use), [computer use](/glossary/computer-use), [agent skill](/glossary/agent-skill), [language model](/glossary/llm) - Guides: [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer), [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you) - Campus policy that is out of scope here: [Google will turn off Gemini and Search for K-12 students unless schools opt in](/blog/google-k12-gemini-and-search-off-until-schools-opt-in) - Directory: [Claude](/directory/claude), [ChatGPT](/directory/chatgpt) - Topics: [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [education](/topics/education), [tools and SaaS](/topics/tools-saas), [harnesses](/topics/harnesses) ## Frequently asked questions ### What is an AI employee? It is a label for workplace software that is given a job and returns work for a person to review. The glossary entry is [digital worker](/glossary/digital-worker). On November 7, 2026, the two product pages that match the label most closely are Claude Cowork and ChatGPT Work. The software is not a legal employee. [Deloitte’s September 10, 2026 essay](https://www.deloitte.com/global/en/insights/topics/technology-management/ai-agents-human-workplace.html) says to treat a review that names the agent as the responsible party as a sign that accountability has slipped. ### What do people mean by always-on AI agents? On the product pages read November 7, 2026, the phrase matches a workplace job that continues after you leave the chat. Claude Cowork can keep a phone-assigned task running in the cloud after the laptop closes. ChatGPT Work and workspace agents can start on a schedule. The [digital worker](/glossary/digital-worker) entry is the short definition. [Claude](/directory/claude) and [ChatGPT](/directory/chatgpt) are the directory notes. A person still reviews the result. ### Is an AI employee the same thing as a chatbot? A chatbot answers in the thread. [OpenAI’s business guide](https://cdn.openai.com/business-guides-and-resources/a-business-leaders-guide-to-working-with-agents.pdf) says that system can call a tool for a current document and still cannot plan a multi-step job unless every step is programmed. Claude’s Cowork page says a conversation starts as chat and picks up file work only when the task needs it. ChatGPT’s Work guide says to stay in Chat for a short answer and to use Work for a finished file. The table above is the comparison. ### How is an AI employee different from RPA? Fixed automation follows rules written in advance. OpenAI’s guide calls that a workflow automation and uses an expense approval and a login lock as the examples. People often call that pattern RPA. An agent starts from a goal and can change its plan when a condition changes. Deloitte’s December 2025 essay says some projects sold as agents are this older automation under a new name. A team can use a script for the stable steps and an agent only where the path is not known. ### What is Claude Cowork? It is Anthropic’s product for knowledge work that is not coding. The [product page](https://claude.com/product/cowork) says you hand Claude a goal, it works in folders and tools you choose, and you review the result. Claude Code is the separate product for shipping software. Computer use on the desktop is still a research preview. Enterprise organizations move to one chat-and-Cowork conversation on the schedule in the next answer. ### When does Claude’s unified experience turn on for existing Enterprise organizations? On November 16, 2026, by default, if an owner has not already chosen. Anthropic’s [help article](https://support.claude.com/en/articles/17444462-set-up-and-manage-the-new-unified-claude-experience-for-your-enterprise-organization) says the setting is off until that date in existing organizations, and that owners can enable it early or keep it off. Chat and cloud code execution have to be on. Zero data retention, HIPAA-ready, and customer-managed encryption-key organizations are outside this switch. ### What is ChatGPT Work? It is OpenAI’s agent for a task with a reviewable result. The [product page](https://openai.com/chatgpt-work/) read on November 7, 2026 says it is powered by GPT-6, on every desktop plan, and on Plus, Pro, Business, Enterprise, and Edu on the web and on mobile. Plan mode asks you to approve an approach first. Workspace agents are the shared, admin-controlled version, in research preview for Business, Enterprise, Edu, and Teachers, and off by default for Enterprise at launch. ### Does a workplace agent need a person’s approval? For a send, a delete, a record change, or a payment, the pages checked here say the person should be asked. That pause is an [approval gate](/glossary/approval-gate). Claude can show a plan and wait. Enterprise admins can forbid modes that skip approvals. ChatGPT workspace write actions default to Always ask. A saved permission is a wider grant. The habit of reading the draft is the [human in the loop](/glossary/human-in-the-loop). ### Are university ChatGPT Edu rollouts the same as hiring an AI employee? They are staged access for adult students and staff, with training and assessment rules. UNSW’s September 9, 2026 page, Sydney’s September 14 page, and La Trobe’s September 28 page each describe ChatGPT Edu for study and university work, not a digital staff roster. OpenAI’s November 5 post is the overview and the source for a few dates the university pages do not print. School-age account policy is the [K-12 news post](/blog/google-k12-gemini-and-search-off-until-schools-opt-in), and it is outside this guide. ## What Is Harness Engineering at Scale? Thoughtworks Radar Volume 35 Explained URL: https://www.aiagentsinfo.com/blog/what-is-harness-engineering-at-scale-thoughtworks-radar Published: 2026-11-07. Updated: 2026-11-08 Category: news What is harness engineering? It is the work of designing the software around an AI agent. Thoughtworks Technology Radar Volume 35, released November 4, 2026, makes harness engineering at scale a theme and treats the agent harness as a product. On November 4, 2026, Thoughtworks released Volume 35 of its Technology Radar and made harness engineering at scale one of the edition’s themes. The same day’s [press release](https://www.thoughtworks.com/about-us/news/2026/thoughtworks-technology-radar-highlights-stronger-controls) quotes chief technology officer Rachel Laycock: clients used to ask how fast agents could write code. They now ask whether anyone can trust, review, and afford what those agents produce. Cecilia Geraldo’s [blog post](https://www.thoughtworks.com/en-us/insights/blog/generative-ai/harness-is-a-product), also dated November 4, states the practical claim in the headline. The harness is a product. An [agent](/glossary/agent), on this site, is a model plus the software that offers tools, runs a step, and brings the result back. That software is the [agent harness](/glossary/harness). [Harness engineering](/glossary/harness-engineering) is the work of designing it. This note, published November 7, 2026, is about the Radar and the two write-ups that explained it. The standing definition stays on the [harness guide](/blog/what-is-an-ai-agent-harness). The pages cited here were re-read on November 7. ## What is harness engineering? [Harness engineering](/glossary/harness-engineering) is the work of designing the software around a model: the loop, the tools, the memory, the permissions, and the checks. The longer guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Harness engineering at scale, in the next section, is Thoughtworks’ name for that same work once an organization has to own it, version it, and evaluate it. ## What is harness engineering at scale? The [Volume 35 radar page](https://www.thoughtworks.com/en-us/radar) says the phrase is not new. What changed is the placement. Thoughtworks debated listing harness engineering as one blip, then made it a theme because it connects many techniques. The theme text defines the work as everything a team does to build the environment around an AI agent, from memory and context to feedback loops and [guardrails](/glossary/guardrail). The same paragraph names the pieces that now travel together: Agent Plugins, an agent plugin marketplace, and Agent Package Manager. It calls the harness a living asset that needs an owner and upkeep, and it points at harness quality evaluation, a self-improving harness, and a feedback flywheel. The scale problem is last: an enterprise harness may show up as a product in the coming months, “not unlike what happened with platforms and platform engineering a few years ago.” Geraldo, a lead consultant and a Radar contributor, writes the product version of that paragraph. Ask a team which model it runs and you get a name. Ask what wraps the model — instructions, tools, permissions, and how context is assembled — and the answer is often a pile of half-owned files. She says those files are one product. Many organizations have a model strategy and a platform strategy. Fewer have a harness strategy, and fewer still have a named person accountable for it. Her reason for caring: the harness is the layer where agentic engineering succeeds or fails. The Radar calls itself an opinionated guide from Thoughtworks’ own projects, published twice a year. It is not a market census. The press release calls Volume 35 a biannual report informed by client work. ## What did Thoughtworks publish on November 4? The press release is datelined November 4, 2026, Chicago. Its frame is that larger, longer-running agent tasks move the hard problems past the conversation with the model: securing the environment, deciding what an agent can access, keeping human oversight meaningful, and controlling cost. The Adopt ring is the release’s strongest signal. Eleven practices and technologies sit there, including agent sandboxing and threat modeling. The release says earlier editions traced AI inside software engineering, Volume 34 examined the friction and risks of using it at scale, and Volume 35 turns to managing those risks as adoption grows. Laycock’s quote is the release’s summary: > Not long ago, the question we heard from clients was how fast agents could write code. Now it’s whether anyone can trust, review and afford what they produce. She adds that models will keep improving, and that lasting value goes to organizations that invest in the engineering and oversight around them. The release names four themes. The interactive radar shortens the second heading and retitles the fourth. The subjects match.
Theme in the November 4 press release What the release says it covers
Harness engineering at scale Prompts, tools, memory, guardrails, and feedback loops become a product with owners, versions, and dependencies, plus evaluation, security, access control, and maintenance.
Putting more agents to work, without losing human judgment Parallel sessions and longer tasks, including unattended overnight builds, can turn review and delivery into bottlenecks. The release warns against “human in the loop theater,” where approval is a ritual. The radar page heads this theme “Putting more agents to work.”
Beyond frontier models A proprietary frontier model is no longer the obvious default. Open-weight models have narrowed the gap for many workloads and can run on an organization’s own infrastructure. Choose by task, cost, privacy, and operations.
The economics of running AI at scale Cost includes routing, batching, storage, networking, and scheduling, not only the model price. The radar page heads the matching section “Constructing the AI factory.”
The rings on the rest of this page are recommendations. Adopt means seriously consider using it. Trial means ready, and less proven than Adopt. Assess means look closely, and trial it when it fits. Caution means consider another option, or avoid it, after negative experience. ## What is an agent harness in this edition? Geraldo’s definition is the Volume 35 answer to “what is an agent harness?” The harness is everything between the model and the work. It is neither the model nor the application. It decides how the two meet.
Part of the harness, in the November 4 post The job it does
Instructions and skills What the agent loads before it acts.
Tools What the agent is allowed to call.
Permissions and sandboxes The box the agent runs inside.
Context policy What the agent is allowed to see.
State What the agent carries between steps.
Orchestration How the steps are sequenced. On this site, orchestration means choosing the next step or specialist.
Evaluation Whether the run was effective.
Telemetry What happened when it was not.
She writes that none of these parts is new. Application engineering spent a decade spreading them across a framework, a platform team, and a CI pipeline. In an agent system they collapse into one artifact a small team can change in an afternoon, and that nobody may own. The failure she describes is a prototype that, six months later, is a folder of Markdown, a few wrapper scripts, a config file full of commented experiments, and one person who knows why the retry exists. It often works. It is a weak thing to upgrade, audit, or hand to another team. Her name for the unmanaged version is configuration debt with a system prompt. The sentence the post quotes twice: > The harness is the place where engineering judgment is expressed. If it’s not managed properly, it’s also the place where that judgment is lost. The supply chain in the same post is where the plugin blips belong. Agent Plugins, she writes, packages agent skills and MCP servers into one installable plugin. Marketplaces and Agent Package Manager publish, pin, and compose those plugins, so teams stop pasting prompt fragments between repositories. A copied prompt has no version, no provenance, and no upgrade path. A package does. Once a harness depends on someone else’s plugin, it needs an owner. The techniques summary places the marketplace in Trial and says Anthropic popularized the pattern with its Claude Code plugin marketplace. Agent Plugins is in Assess on the platforms quadrant. Agent Package Manager is in Trial on the tools quadrant. ## Which blips carry the harness theme? These rings are from the Volume 35 [techniques summary](https://www.thoughtworks.com/radar/techniques/summary) and the related-blip labels on those pages, checked November 7, 2026.
Blip Ring in Volume 35 What the radar text uses it for
Agent sandboxing Adopt, techniques A sensible default: isolated environments, restricted privileges, limited access to other systems, shaped by threat modeling.
Threat modeling Adopt, techniques A regular human-led exercise. The text says prompt injection and autonomous action can slip past rule-based scans.
Agent plugin marketplace Trial, techniques Distribution for skills, instructions, MCP servers, and subagent definitions, with a way to publish and update them.
Feedback flywheel Trial, techniques A team habit: ask at the end of a session what should change in a shared artifact, then decide in a retrospective. People still accept the edit.
Self-improving harness Trial, techniques Agents read traces and eval feedback, then propose or make changes to prompts, tools, skills, memory, and workflows. Humans set the bounds.
Risk-tiered human oversight Trial, techniques Review depth follows what a change could break, so attention goes where a person can still judge the diff.
Harness quality evaluation Assess, techniques Scheduled checks that a skill, plugin, or guardrail actually works.
Enterprise harness Assess, techniques The shared layer coding agents consume, kept thin and federated.
Human-in-the-loop theater Caution, techniques An approval step the person cannot realistically evaluate. The text calls it an accountability sink.
AI-accelerated shadow IT Caution, techniques Software built outside the supported path, including with coding agents. The suggested answer is a useful enterprise harness and firm security limits.
Three neighboring cautions belong in the same reading. Agent instruction bloat is guidance that grows until it contradicts itself. The summary says Anthropic removed over 80 percent of Claude Code’s system prompt for its latest models with no measurable loss, and it tells teams to delete instructions a newer model no longer needs. Blanket frontier model usage is sending every task to the most capable model. Coding throughput as a measure of productivity is treating the share of AI-written code, or tokens consumed, as a badge. The summary says those numbers can show adoption and still be a poor proxy for productivity, and it points back to DORA metrics and business outcomes. ## What did the Adopt ring actually include? The press release’s count of 11 is checkable. On November 7, 2026, the four quadrant summaries listed eleven Adopt blips. Six are techniques. The other five are a platform, three tools, and a language standard. Sandboxing and threat modeling are the examples in the release. They are not the whole ring.
Adopt blip Quadrant
Agent sandboxing Techniques
Mutation testing Techniques
Path-to-production mapping Techniques
Post-quantum cryptography Techniques
Progressive context disclosure Techniques
Threat modeling Techniques
VictoriaMetrics Platforms
mise Tools
OpenAI Codex Tools
OpenCode Tools
Open Data Contract Standard (ODCS) Languages and frameworks
Agent sandboxing is the Adopt item closest to the release’s security line. The techniques summary says Thoughtworks had already recommended sandboxes for coding agents, and now treats sandboxing as a sensible default for agents that also act in business systems. The method it describes is risk-based, informed by threat modeling, using least privilege. The blip does not name one sandbox product. Threat modeling, also Adopt, stays a human-led practice with help from agents. The summary points at MITRE ATLAS, which this volume places in Assess, and at toxic flow analysis for AI, in Trial, as ways to locate where prompt injection can enter. A session should end with decisions about where nondeterministic model behavior is acceptable, where it is not, and where a deterministic guardrail is required. The other Adopt techniques are about the flood of agent output, not a new security tool. Path-to-production mapping is a workshop from the workstation to production that lists steps, then pain points, because more generated code exposes later bottlenecks. Progressive context disclosure loads detail only when a step needs it. The coding example is a short AGENTS.md that links out, plus deferring the full token cost of MCP tools and skills until they are used. Mutation testing targets generated tests that pass without catching a logic change. OpenAI Codex and OpenCode are the coding tools in this volume’s Adopt ring. Claude Code is not one of the eleven. Related blips still mark Claude Code as Adopt from the April 2026 radar. That older placement is not a new Volume 35 decision. ## What is human-in-the-loop theater? The press release keeps the warning short. People run several agent sessions and hand off longer tasks, including unattended overnight builds. Review and the delivery pipeline become the bottleneck. Approval that turns into a ritual is “human in the loop theater.” The attached advice is to measure review capacity before adding sessions. The Caution blip is sharper. A person stays in the loop as an empty approval step and signs actions, prompts, or diffs they cannot realistically evaluate. The summary calls that an accountability sink: nobody owns the decision. The alternative is triage. Keep human review for changes a person can actually judge. Elsewhere, steer the system with checks such as mutation testing. Risk-tiered human oversight, in Trial, is that triage. The riskiest or hardest-to-reverse changes are for humans to apply. Mid-risk changes get review. Low-risk changes can be auto-merged through continuous integration. Codebase zones can force a closer look. An LLM that classifies impact helps, and it is fallible. The summary says a published process from Meta combines those two signals, and that Thoughtworks teams use the split to spend scarce attention on purpose. Unattended overnight agentic builds, also Trial, is the long task the press release names. A team gives a coding agent a feature-sized job in the evening and reads the result in the morning. The summary treats the entry requirements as the point: a sandbox, work specified up front, and feedback sensors so the agent can correct its own mistakes. A night’s diff can be too large for line-by-line review, so the team needs a harness it has already tested. The blip is Trial for teams that have those pieces, and a goal for teams that do not. The November 6 New Stack conversation, below, uses a different sentence for the same week: agents that catch mistakes before a person looks, so developers stop babysitting the run. That claim is about feedback inside the harness. The Caution blip is about fake approval beside it. Volume 35 prints both. ## What is an enterprise harness? The [enterprise harness blip](https://www.thoughtworks.com/en-us/radar/techniques/enterprise-harness), dated November 4, 2026, and placed in Assess, is the organization-scale version. It is the shared layer of what coding agents consume: skills, tools, connectors to internal context, approved technologies, architecture decision records, path-to-production guidance, hooks, and security policies. Teams need that common context on top of each team’s own instructions. The comparison in the blip is a developer-experience platform, pointed at agent experience. The advice is modest, which is why the ring is Assess. Thoughtworks has seen organizations build a custom coding agent on top of OpenCode. The blip prefers open standards and a plugin marketplace. “There’s no consensus yet on how to do this well, so keep the enterprise harness thin and federated, leave room for experimentation and harvest what works.” Geraldo describes what “running one” adds once a platform team is asked to own the runtime: auditability, access control, lifecycle, and spend limits. Her list is golden paths that are faster than the unsupported route, sandbox defaults that do not depend on someone remembering a flag, plugin allow-lists with provenance checks, one model-routing policy for cost and data residency, telemetry that names the harness version behind an output, and a deprecation policy so other teams can depend on the runtime. Identity is the item she calls most urgent. An agent acting for a person needs an answer to whose permissions it is using, what it may do without asking, and what evidence remains. Those are harness decisions. A per-team config file is a poor place to leave them. She also names the bypass. If the official harness is slower than something a team can assemble in a weekend, people will go around it. The Caution blip on AI-accelerated shadow IT says the same thing from the outside: coding agents make it easy to ship software with no tests, no pipeline, and no observability, and security is the first risk. The remedy in that blip is a place to experiment that is actually easier, with limits that stay firm. Three questions end her post. Who owns the harness, as the person accountable for quality and the roadmap? How do you know the last change made it better, if the honest answer cannot be a demo, an anecdote, or an acceptance rate? What may it touch, including data, tools, credentials, and production? She writes that harness engineering emerged in 2026 and is still immature, and that the decisions are cheaper now than they will be in twelve months. This site’s [glossary](/glossary/harness-engineering) already points at earlier 2026 accounts, including OpenAI’s February 11 post and LangChain’s March 10 explainer. Geraldo’s sentence is Thoughtworks’ clock for that work. ## What has to be true before a harness rewrites itself? The Trial blip for a self-improving harness is agents that read traces and evaluation results, then propose or apply changes to prompts, context strategy, workflows, skills, tools, memory, and middleware. The feedback flywheel, the summary says, still has people interpret the signal and edit the system. Research the blip names, including Meta-Harness and VERSE, pushes the loop further. The printed recommendation is bounded experiments, regression tests, and humans who set the goal, the editable surface, and whether a gain survives new tasks. Geraldo’s brake is stricter. A loop that can rewrite instructions, pick its own tools, or re-rank results can drift. The output can look like learning and behave like entropy. Her signal is the flywheel: findings, incidents, failed evals, cost anomalies, and cases where a person took over. Without that, self-improvement is a model editing its own job description. Three preconditions have to hold before she would run the loop: - An evaluation harness, so a gain is measured. - Provenance, so each self-authored change can be attributed, reviewed, and reverted. - A rollback path, so a bad iteration costs an afternoon. Automate the proposal. Keep the approval. The system may suggest an instruction, a tool swap, or a context rule. It does not merge on its own. The flywheel blip adds the matching maintenance rule: delete stale guidance, and drop a sensor that only makes noise. ## How do you tell whether a harness change worked? Geraldo’s question for harness quality evaluation is whether a change made the agent better or worse. An acceptance rate, or a pull request that made the instructions clearer, does not answer it. Behavior is non-deterministic, tasks vary, and a set that represented the work in March can mislead by June. Suites also cost money and time, so teams measure what is cheap: tokens, accepted suggestions, tests passed. She calls those activity metrics. They can show that the system is busy. The practice she describes is a golden set of real jobs, on the order of the 20 or 30 tasks the agent is supposed to do, each with a score. Version the set with the harness. Regress on every meaningful change, not only at release. Treat cost and latency as product numbers, because a better-looking answer that doubles spend has not improved the product. Keep a sample of human review for tone, judgment, and style, which a scored set will miss. The set needs an owner and a review cadence, the way a release test does. Otherwise the eval is another kind of theater: green, and disconnected from customers. Her rule is that without an eval, there is no change. The Assess blip is the operational version. Custom harnesses are skills, plugins, AGENTS.md files, and deterministic guardrails. For the non-deterministic parts, the summary says Thoughtworks teams use PluginEval on coding-agent plugins or skills. For deterministic guardrails, they want a test suite so each check can be built test-first and can show that it fires. Run the evaluations nightly or beside CI. A test of one observable action is a [behavioral evaluation](/glossary/behavioral-evaluation) on this site. The writing guide is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). How to stand up the tasks and graders is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). Thoughtworks’ golden set is a wider question than one action. This page does not treat them as the same test. Both say a harness you cannot measure is a harness you cannot safely change. Two nearby pages answer different questions, and neither is a Thoughtworks result. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) is this site’s note on a November 4, 2026 paper that searches for a better harness while the model stays fixed. The [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026) lists public coding-agent scores with the date on each source. The [harness comparison](/trackers/harnesses) stays empty until every cell has a source. Volume 35 does not cite those pages. ## What did Vanitha Kumar tell The New Stack on November 6? On November 6, 2026, The New Stack published [Kimberly Fessel’s account](https://thenewstack.io/harness-engineering-coding-agents/) of a podcast with Vanitha Kumar. The article identifies her as Head of Technology for Retail, CPG, Travel and Hospitality Market for Thoughtworks Americas. She is also on the Volume 35 contributor list, with Laycock and Geraldo. Fessel’s opening matches the Radar’s second theme. Harness engineering, in that telling, gives agents tests, feedback loops, and guardrails so they can correct mistakes before a person reviews the work. They then hold longer tasks, and several can run at once. Kumar splits the job in one sentence: “Do it right the first time and also self-correct when you have done it for the first time.” The payoff The New Stack attributes to her is that you no longer have to “babysit and watch your agent do the work.” One agent is a different problem from hundreds. That is where she talks about AI factories, and compares them to Kubernetes, which hid a lot of microservice machinery. Factories, in her account, standardize model choice, workload management, and cost. The line The New Stack quotes, brackets included: “The best way to build a platform is [to] make it easy to do the right thing.” The radar page uses the same factory language in “Constructing the AI factory,” and it also names Kubernetes as a base for serving and scheduling models. The podcast article is Kumar’s version of that theme. It is not a second Radar. More autonomy also means more access to data, systems, and tools, which Fessel writes can add security exposure. Some organizations are delaying adoption because of that. Guardrails in the article restrict tools, force a test environment, and require a person for sensitive changes. Asked what next year looks like, Kumar says: “Security is going to be a big topic. Not even one year out.” She discusses Volume 35, which the article dates to November 4. Fessel’s summary is AI, harness engineering, new agent practices, and older practices that now weigh more. The examples, via Kumar, are penetration testing and understanding the path to production: “old things that are making a comeback because they now have more added weight because of AI.” Path-to-production mapping is an Adopt technique, as the table above shows. This article did not find a Volume 35 blip titled penetration testing, so that example stays Kumar’s remark as The New Stack reported it. The same day, the newsletter Agent Patch Notes published [issue 3](https://agentpatchnotes.beehiiv.com/p/agent-patch-notes-3), dated November 6, 2026. It describes tighter write permissions in Claude Code and a Factory coding harness rebuilt around programmatic tool calling. That roundup is context beside the Radar. It is not a Thoughtworks finding. ## What should a builder read next? The November 4 material leaves a short list. Name an owner. Pair that owner with a golden set, and with a rule that a change without an eval does not ship. Decide the blast radius — data, tools, credentials, production — as part of the product, and sandbox by default. Keep human review for diffs a person can still judge. If the supported path is slower than a weekend setup, expect people to route around it. The standing pages here are the glossary and the two guides already linked: [what an agent harness is](/blog/what-is-an-ai-agent-harness) and [how to evaluate one](/blog/how-do-you-evaluate-an-ai-agent-harness). [How agents work](/topics/how-agents-work) is the plain-language shelf. [Agent harnesses](/topics/harnesses) collects the guides, the [November 2026 benchmark](/reports/agent-harness-benchmark-november-2026), and [Best AI agent frameworks](/best/ai-agent-frameworks). This piece is on the [news](/topics/news) topic. Vendor docs for other runtimes are in the [agent frameworks](/directory/category/agent-frameworks) category and on the [Builders](/for/builders) page. Volume 35’s own next step, on the radar page, is the interactive edition and the PDF. The sourced claim on November 7 is the one in the press release and the theme: Thoughtworks is telling teams to engineer the harness as a product, with sandboxing and threat modeling in Adopt, and the enterprise version still in Assess. ## Frequently asked questions ### What is harness engineering? It is the work of designing the software around a model: the loop, the tools, the memory, the permissions, and the checks. The glossary entry is [harness engineering](/glossary/harness-engineering). The longer guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). The rest of this page is Thoughtworks’ November 4, 2026 account of that work at organization scale. ### What is harness engineering at scale? In Technology Radar Volume 35, released November 4, 2026, it is a theme, not one blip. Thoughtworks uses it for the environment around an AI agent — prompts, tools, memory, guardrails, and feedback loops — treated as a system with owners, versions, evaluation, security, and access control. Cecilia Geraldo’s post the same day calls that layer a product. The local definition of the work is [harness engineering](/glossary/harness-engineering). ### What is an agent harness, according to Thoughtworks? Geraldo says it is everything between the model and the work: instructions and skills, tools, permissions and sandboxes, context policy, state, [orchestration](/glossary/orchestration), evaluation, and telemetry. It is neither the model nor the application. The longer guide on this site is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). ### What did Technology Radar Volume 35 put in the Adopt ring? The November 4 press release says 11 practices and technologies, including agent sandboxing and threat modeling. Checked on November 7, 2026, the quadrant summaries list those two plus mutation testing, path-to-production mapping, post-quantum cryptography, and progressive context disclosure; VictoriaMetrics on platforms; mise, OpenAI Codex, and OpenCode on tools; and the Open Data Contract Standard on languages and frameworks. ### What is an enterprise harness? The November 4 blip, in Assess, calls it the shared layer coding agents consume: skills, tools, connectors to internal context, approved technologies, architecture decision records, path-to-production guidance, hooks, and security policies. The printed advice is to keep it thin and federated, and to share assets through open standards and a plugin marketplace. ### What is human-in-the-loop theater? The press release uses the phrase for approval that has become a ritual. The Caution blip describes a person signing actions, prompts, or diffs they cannot realistically evaluate. The Trial practice next to it is risk-tiered human oversight: a person reviews what they can still judge, and checks cover the rest. ### Who is quoted in the November 4 and November 6 coverage? The press release quotes Rachel Laycock, Thoughtworks’ chief technology officer. The blog is by Cecilia Geraldo, lead consultant. The New Stack article dated November 6, 2026, quotes Vanitha Kumar, Head of Technology for Retail, CPG, Travel and Hospitality Market for Thoughtworks Americas. The Volume 35 contributor list includes all three. ## Why Tool JSON Costs More Than Your System Prompt (and How to Check It) URL: https://www.aiagentsinfo.com/blog/why-tool-json-costs-more-than-your-system-prompt Published: 2026-11-07. Updated: 2026-11-07 Category: guide On November 7, 2026 the prompt checker counted a 23-token weather prompt and a 94-token get_weather tool in OpenAI's Responses shape. For gpt-5, with 200 output tokens entered and 20 turns a day, the 30-day figure was $1.28775. The tool text was $0.0705 of that. The page rounds both this result and a 101-token Chat Completions paste to $1.29, so the token split is the number to read. Updated November 7, 2026. A short system prompt is the paragraph you rewrite. The tool list is the JSON you send again on the next turn. On November 7, 2026 I posted both to the live [prompt and tool checker](https://www.aiagentsinfo.com/agent-prompt-checker). The system prompt was 23 tokens. One `get_weather` tool, in OpenAI's Responses shape, was 94 tokens. The tool text is a little over four times the prompt. The 30-day dollar gap is smaller than that ratio, and the rounded headline on the page can hide it. The numbers below are the checker's response, with the output length and the daily volume I typed in. ## What was pasted? The prompt is the sample already on the checker: > You are a weather assistant. Use get_weather when the user asks about the weather. Do not invent a temperature. The tool is the page's valid example. [OpenAI's function-calling guide](https://developers.openai.com/api/docs/guides/function-calling), read November 7, 2026, uses this shape for `get_weather`: `type`, `name`, `description`, `parameters`, and `strict` sit on the tool. The guide's own example asks for a location and a units enum. This one asks for a city, which is the example the checker loads. ```json [ { "type": "function", "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" } }, "required": ["city"], "additionalProperties": false }, "strict": true } ] ``` [Tool use](/glossary/tool-use) is the step this JSON buys: the model can call `get_weather`, and the harness sends the result back. One place that ships that step is the [OpenAI Agents SDK](/directory/openai-agents-sdk). The directory page, last verified November 3, 2026, says the SDK uses the Responses API by default when you call OpenAI models, and that those calls are billed at API rates. The SDK docs, read November 7, 2026, describe function tools, handoffs, guardrails, tracing, and sandbox agents as parts of that runtime. They do not print a token price for the tool list. The checker is the page that prices the list you are about to send. The request was `POST /api/agent-prompt-checker` with `format` set to `auto`, `modelId` set to `gpt-5`, `outputTokens` set to 200, and `tasksPerDay` set to 20. The server returned no schema errors and detected "OpenAI Responses function". The prompt is not stored. ## What did gpt-5 cost for 30 days? The count used tiktoken `o200k_base`, the encoding that library assigns to gpt-5. The checker says this is a count of the pasted text. It does not add the few tokens of message framing an API request adds. gpt-5's row in the price table, read November 5, 2026, is $1.25 per million input tokens and $10 per million output tokens on the standard band. The table cites [OpenAI's API pricing](https://developers.openai.com/api/docs/pricing). Cached, batch, flex, and fast prices are not used. I did not re-read that pricing page on November 7. The rates below are the rates the checker applied. The API returned: - Prompt tokens: 23 - Tool tokens in the cost: 94 - Input tokens priced: 117 - Per turn: $0.00214625 - 30 days at 20 turns a day: $1.28775 The same prompt with the tool box empty came back as 23 input tokens, $0.00202875 per turn, and $1.21725 over 30 days. The tool JSON is the difference, $0.0705. The 200 output tokens are the number I entered. The checker does not write a reply. At $10 per million, those 200 tokens are $1.20 of the $1.28775. The tool text is the larger share of the input. The output I entered is the larger share of the money. The page rounds a 30-day total of $1 or more to the cent, and it rounds a per-turn price under $1 to four decimal places. This result displays as $1.29 and $0.0021 per turn. A Chat Completions paste of the same tool, counted in the next section at 124 input tokens, returned $1.293 and displays as $1.29 as well. Read the token split before you read the rounded headline. OpenAI's guide says callable function definitions count against the context limit and are billed as input tokens, because the functions are injected into the system message in a syntax the model was trained on. The 94 is the JSON in the box. It is not that injected message. A Responses request can bill a different input count. Use the checker to see the paste, then use the usage object on the real response for the billed count. Once you already have 117 and 200, the [agent cost calculator](/agent-cost-calculator) is the shorter tool. It stays in the browser and multiplies the counts you type. It does not read a prompt, and it does not validate tool JSON. ## How do the four shapes change the count? The same city field can be written four ways. The checker looks at the field names. I kept the prompt, the description "Get the current weather for a city.", and the settings: 200 output tokens, 20 turns a day, format auto. gpt-5 rows are tiktoken counts of the paste. The Claude row is an estimate plus a published add-on. | Paste | Prompt | Tool text | Added input | Input priced | 30-day from the API | | --- | --- | --- | --- | --- | --- | | Responses `get_weather`, gpt-5 | 23 | 94 | none | 117 | $1.28775 | | Prompt only, gpt-5 | 23 | 0 | none | 23 | $1.21725 | | Chat Completions `get_weather`, gpt-5 | 23 | 101 | none | 124 | $1.293 | | MCP `get_weather` text, priced as gpt-5 | 23 | 75 | none | 98 | $1.2735 | | Anthropic `get_weather`, Claude Sonnet 5.5 | 23, estimate | 75, estimate | 286 published | 384 | $1.6608 | | Three Responses functions, gpt-5 | 23 | 308 | none | 331 | $1.44825 | ### Chat Completions The older wrapper puts the function under `function`. The checker called it "OpenAI Chat Completions function", counted 101 tool tokens, and included them in the cost. It also returned a warning: the current function-calling guide puts `name`, `description`, and `parameters` on the tool itself for the Responses API. ```json [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" } }, "required": ["city"], "additionalProperties": false }, "strict": true } } ] ``` The warning is not a schema error, so the check still succeeded. The guide, read November 7, 2026, says a Responses request tries to rewrite the schema into strict mode when `strict` is omitted, and a Chat Completions request stays non-strict unless `strict` is set. Set `strict` to true or false on purpose. The 7 extra tokens against the Responses paste disappear once the page rounds $1.293 and $1.28775 both to $1.29. ### Anthropic `input_schema` [Anthropic's tool-use overview](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview), read November 7, 2026, defines a user tool with `name`, `description`, and `input_schema`. There is no `type: "function"` wrapper. The overview prices a tool-use request as the input tokens sent to the model, including the `tools` parameter, plus output tokens. Server-side tools can add a separate usage fee. This paste is a client tool. ```json [ { "name": "get_weather", "description": "Get the current weather for a city.", "input_schema": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" } }, "required": ["city"] } } ] ``` I priced it as Claude Sonnet 5.5. The checker counted the prompt as 23 and the schema as 75 with tiktoken `o200k_base`, and it labeled both an estimate. Anthropic does not bill with that encoding, so the billed count of the paste can differ. It then added 286 input tokens. Anthropic's overview lists 286 for Claude Sonnet 5.5 when `tool_choice` is auto or none and at least one tool is sent. The any/tool cell on that row is blank. The checker did not invent a second number. The 286 is Anthropic's published count, not a tiktoken count. I read that table on November 7, 2026. The checker's copy of the table is dated November 6, 2026. Both show 286 for this cell. The price table, read November 5, 2026, lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. The API returned $0.002768 per turn and $1.6608 over 30 days. The page displays $1.66. Input tokens priced were 384, which is 23 + 75 + 286. On this row the published tool-use prompt is larger than the system prompt I wrote and larger than the schema. That is the Claude case of the title. The schema's own 75 is the estimate. The 286 is the published figure. ### MCP `inputSchema` The [MCP tools page](https://modelcontextprotocol.io/specification/2025-06-18/server/tools) for the 2025-06-18 specification, read November 7, 2026, names the argument schema `inputSchema`. The spec's example is also `get_weather`. This paste uses `city` so the words match the other rows. The field name is the part that makes it MCP. ```json [ { "name": "get_weather", "description": "Get the current weather for a city.", "inputSchema": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" } }, "required": ["city"] } } ] ``` On gpt-5 the checker counted 75 tool tokens and priced 98 input tokens. The 30-day figure was $1.2735, which the page displays as $1.27. That is gpt-5's rate applied to the pasted text, so the row can sit next to the OpenAI rows on one encoding. MCP lists tools for a client. The specification does not bill those tools at gpt-5 rates. ## What happens with three short functions? The single tool already outweighs the prompt. A second paste, still on gpt-5 with 200 output tokens and 20 turns a day, used three Responses functions: current weather, a daily forecast with a `days` integer, and a city geocode. The checker returned no schema errors, 308 tool tokens, 331 input tokens priced, and $1.44825 over 30 days. The page displays $1.45. Against the prompt-only figure of $1.21725, the three functions add $0.231. ```json [ { "type": "function", "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" } }, "required": ["city"], "additionalProperties": false }, "strict": true }, { "type": "function", "name": "get_forecast", "description": "Get a daily forecast for a city.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" }, "days": { "type": "integer", "description": "Number of days, from 1 to 7." } }, "required": ["city", "days"], "additionalProperties": false }, "strict": true }, { "type": "function", "name": "geocode_city", "description": "Look up latitude and longitude for a city name.", "parameters": { "type": "object", "properties": { "city": { "type": "string", "description": "City name" } }, "required": ["city"], "additionalProperties": false }, "strict": true } ] ``` OpenAI's guide says to keep the number of functions available at the start of a turn small, and to shorten descriptions if you hit a token limit. It offers fewer than 20 functions as a soft suggestion. These three are short, and the tool text is already 308 tokens against a 23-token prompt. A production list with long descriptions is a different paste. Put that paste in the checker. The $0.0705 from the one-tool example does not scale itself. ## What does the checker reject? I took the valid Responses tool, added an optional `country` string, and left `country` out of `required`. `strict` stayed true. The checker returned an error at `tools[0].parameters.required`: in strict mode every property has to be listed in `required`, and an optional field can use `"null"` as one of its types and still be required. The function-calling guide says the same thing. Strict mode needs `additionalProperties` set to false for each object, and every field in `properties` marked required. The tool text counted as 193 tokens. The cost still included it, with the note that a provider can reject the request before it bills the turn. The checker's invalid example fails for different reasons. Posted as written, it came back as a Chat Completions tool whose name is `bad name`, whose `parameters` type is `string`, and which omits `description` and `strict`. The checker reported the name pattern, the object-type requirement, the missing description, the missing `strict`, and the Chat Completions warning. The token count still ran. A clean schema and a token count answer different questions. The page shows both. gpt-5, gpt-5-mini, and gpt-5-nano are the models for which the checker says tiktoken has an encoding. Newer OpenAI ids in the price list, and every Claude, Gemini, and Grok id, are labeled estimates made with `o200k_base`. ## Where does the Agents SDK fit? The [OpenAI Agents SDK](/directory/openai-agents-sdk) page is the directory entry to read next to this check. Last verified November 3, 2026, it says the Python package `openai-agents` and the TypeScript package `@openai/agents` are MIT licensed, and that OpenAI does not list a separate subscription for the SDK. Model calls go through your API account at API rates. The $1.28775 in the first row is that kind of charge for one pasted tool, an entered reply of 200 tokens, and 20 turns a day. It is not an SDK fee. The same listing says the SDK runs the loop: tool calls, handoffs, guardrails, sessions, and tracing, including sandbox agents that work inside an isolated workspace. The SDK guide, read November 7, 2026, points function tools and MCP at the platform's tools docs and keeps tracing on the SDK. This checker does not price a handoff, a trace, or a sandbox. It prices the system prompt and the tool JSON on one turn, then multiplies by the turns per day you type. On the checker, Load a valid example fills this prompt and this Responses tool. It leaves output tokens at 800 and turns per day at 10. The page says those defaults are there so the price is visible, and that they are not a measured workload. To match the table, set output tokens to 200 and turns per day to 20, then run the check. If you already know the token counts, skip the paste and use the [cost calculator](/agent-cost-calculator). ## What the figure leaves out The 30-day numbers are the checker's multiplication. They leave out the items the checker prints under the price, and two limits that matter for this paste. - Message framing is not in the count. OpenAI's injected tool syntax is not in the 94. - No per-call tool fee is added. The OpenAI and Anthropic rows in the price table do not publish one. - No computer-use fee, search-grounding fee, or code-execution fee is added. Anthropic's overview says some server tools charge on top of tokens. Those rates are not in this table. - The Claude counts of the pasted prompt and schema are tiktoken estimates. The 286 is the published tool-use system prompt for auto or none. - Output tokens are the number entered. This check did not call gpt-5 or Claude and did not count a reply. - The prompt and the tool JSON are not stored. ## Claude Code vs Codex Pricing: Which Costs Less in 2026? URL: https://www.aiagentsinfo.com/blog/claude-code-vs-codex-pricing Published: 2026-11-06. Updated: 2026-11-09 Category: guide Codex costs less to start. On November 6, 2026 it is included on ChatGPT Free and on Go at $8 a month, and Claude Code starts on Pro at $20 a month, or $17 a month on an annual bill. Anthropic Claude Code limits on Pro are a five-hour window and a weekly cap, shared with the rest of Claude. The $100 and $200 individual prices match, and Codex Pro also lists a $500 tier. Updated November 9, 2026. The list prices below were checked again that day and still match the November 6 reading, including Codex Go at $8, Claude Pro at $20 or $17 annually, and the $100, $200, and $500 Codex tiers. The local-message ranges for Plus and Standard Business are unchanged. One limit was incomplete on the earlier reading: Enterprise and Edu plans without flexible pricing have the same per-seat usage limits as Plus for most features. ## Which costs less, Claude Code or Codex? Codex costs less to start. On the pages checked November 6, 2026, [Codex](/directory/codex) is included with ChatGPT Free at $0 and with Go at $8 a month, and [Claude Code](/directory/claude-code) starts on Pro at $20 a month, or $17 a month when you pay $200 up front for the year. After that, the individual monthly prices line up. Claude Pro and ChatGPT Plus are both $20 a month. Claude Max and the first Codex Pro tier are both $100. Claude Ultra and the middle Codex Pro tier are both $200. Codex Pro also lists a $500 tier, and the Claude individual pages checked here do not. Those are list prices. They are not a count of how much code each plan will write. [Claude Code](https://claude.com/product/claude-code) is Anthropic’s coding agent. Codex is OpenAI’s. Each one is an [agent](/glossary/agent): a model plus the [harness](/glossary/harness) that runs tools, keeps a record, and stops. The longer account of that surrounding software is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). The sourced harness table is [Best AI agent harness](/best/ai-agent-harnesses). [Claude](/directory/claude) is the chat product on the same Anthropic plans. [ChatGPT](/directory/chatgpt) is the plan Codex rides on. Prices below exclude tax where the vendor says so. Anthropic says its prices can change. OpenAI’s pages describe the current cards and send you to a usage dashboard for the live limit. Use the vendor page for a number you intend to pay. ## What does each $20 plan buy? The $20 comparison is the one most people mean. The two products share a sticker and then diverge on what sits inside it.
At $20 a month Claude Pro ChatGPT Plus
Where the price is printed Claude pricing. Annual billing is $17 a month, $200 up front. Max and Ultra are monthly only. Codex pricing docs. The ChatGPT pricing FAQ lists Go and Plus as monthly plans.
Coding agent Claude Code is included. The pricing table says No on Free and Yes on Pro, Max, and Ultra. Codex on the web, in the CLI, in the IDE extension, and on iOS. The docs call this enough for a few focused coding sessions a week.
What else is named for coding The same subscription covers Claude on the web, desktop, and mobile. Terminal, IDE, Slack, and the web are the steering surfaces on the product page. Cloud integrations such as automatic code review and Slack. GPT-6.1 Sol and GPT-6 Luna. Credits if you want to go past the included allowance.
How the limit works One pool for Claude and Claude Code. Pro has a five-hour session limit and a weekly limit. Anthropic says there is no fixed message count. Codex shares usage with ChatGPT Work. The docs print local-message estimates for a five-hour window, and say those estimates are not fixed limits. Weekly limits may also apply.
Anthropic’s pricing FAQ says Pro gives you at least five times the Free plan’s usage per five-hour session. The Codex docs, for Plus and for Standard Business, print these local-message ranges per five hours: GPT-6 Astra 5–45, GPT-6.1 Sol 15–160, GPT-6.1 Terra 35–285, GPT-6 Sol 15–150, and GPT-6 Luna 350–3,000. The same page says cloud tasks can use more of the allowance than local messages, and that Pro plans currently have no five-hour limit. A range that wide is a reason to read your own dashboard, not a promise of 160 Sol messages. ## What are the individual prices?
Plan Price on November 6, 2026 Coding agent on that plan
Claude Free $0 Claude Code is not included. The feature table says No.
Claude Pro $20 a month, or $17 a month with annual billing ($200 up front) Included. Shared with the rest of Claude. Five-hour and weekly limits.
Claude Max $100 a month, billed monthly. The help center says this is the web price and that mobile prices can differ. Included. 5 times Pro usage every five hours and every week. $100 a month in API credits on the Claude Platform. Fable, for up to half the weekly limit.
Claude Ultra $200 a month, billed monthly. Web price; mobile prices can differ. The help center says the old Max 20x plan was renamed Ultra, with no price change. Included. 10 times Pro usage every week. Verified accounts have no five-hour limit. $200 a month in API credits. Fable, for up to half the weekly limit.
ChatGPT Free $0 Codex for quick tasks. The help center includes Free, and says Codex Cloud is not included. The docs name GPT-6 Luna at Standard speed in the desktop app, subject to rollout.
ChatGPT Go $8 a month Codex for lightweight tasks. Codex Cloud is not included. Same Luna note as Free, subject to rollout.
ChatGPT Plus $20 a month Codex on web, CLI, IDE, and iOS, plus the cloud integrations named above. Eligible for Codex Cloud, subject to rollout and settings.
ChatGPT Pro $100, $200, or $500 a month. The card says “from $100.” Everything in Plus, on the docs page. Ultrafast mode is on the $500 plan. The docs say Pro has no five-hour limit. The Codex pricing page says GPT-6 Astra sits inside the allowance on the $100 and $200 Pro plans.
API key API token rates, not a monthly sticker Codex in the CLI, SDK, or IDE extension. The docs say this path has no cloud features such as GitHub code review and Slack.
Claude Code’s product page says you can sign in with Pro, Max, Ultra, Team, Enterprise, or a Claude Console account. With Console, use is billed at standard API rates. The same FAQ prices a research-preview fast mode for Opus 5.5 at $8 per million input tokens and $40 per million output tokens, on consumption-based plans and, through usage credits, on subscription plans. That rate is an extra speed setting. It is not the Pro price. The Codex pricing page’s FAQ, checked the same day, also mentions a $100 Business plan: it says GPT-6 Astra is included in the allowance on the $100 and $200 Pro plans and the $100 Business plan, and that Plus and the $20 Business plan include limited GPT-6 Astra, with credits if you need more. The plan cards on the Codex docs page print Business at $20 per user per month when billed annually. This guide does not merge those two sentences into a third Business sticker. ## What does a team seat cost?
Seat Price What the coding agent includes
Claude Team, standard $20 per member per month billed annually, or $25 billed monthly. Minimum two members, up to 150. US prices, tax excluded. The product page prints the $20 annual figure and “up to $500 per month in pooled API credits.” Claude Code on every seat. The help center says a standard seat includes 1.25 times the Pro plan’s per-session allowance, plus a weekly limit. Limits are per member. Monthly API credits on Team plans cannot be used in Claude Code or the Claude apps.
Claude Team, premium $100 per member per month billed annually, or $125 billed monthly Claude Code, with 6.25 times the Pro plan’s per-session allowance and a weekly limit, on the help center page. The pricing page says premium seats give 5 times more usage than standard seats. Both sentences are on Anthropic’s pages. This guide does not merge them into one formula.
Claude Enterprise The pricing FAQ says $20 per seat per month plus usage at API rates, billed annually. The product card lists security, analytics, role-based access, spend limits, and data controls, and it does not repeat the dollar figure. Included on new and self-serve Enterprise seats. The help center says usage-based Enterprise, including self-serve, has no per-seat usage limit and is billed at API rates. Older Enterprise plans can still split seats.
ChatGPT Business $20 per user per month for two or more users, billed annually. $25 per user per month billed monthly. ChatGPT and Codex on desktop and mobile. Larger virtual machines for cloud chats. Credits to extend usage. SAML SSO and MFA. No training on business data by default, on the docs page.
ChatGPT Enterprise and Edu Contact sales. The docs card does not print a dollar price. Everything in Business, plus priority processing, SCIM, retention and residency controls, audit logs, and the Compliance API, as that card lists them. On the docs page re-read November 9, 2026, flexible-pricing workspaces scale with credits and have no fixed rate limit. Enterprise and Edu plans without flexible pricing have the same per-seat usage limits as Plus for most features.
A team that only compares the $20 annual seat is looking at the same sticker. The allowance is still the part that moves the bill. Claude Team standard is a multiple of Pro, per person. ChatGPT Business, on the docs page, shares the Plus local-message estimates in the “Standard Business” column. Extra Claude Code on a Team plan, past the included limit, uses usage credits. The monthly API credits that come with the Team plan are a different purse, and Anthropic says that purse does not pay for Claude Code. ## Where does the cheaper plan stop being cheaper? Use the row that matches the job, then open the linked page before you subscribe.
If you want Lower list price on November 6, 2026 What you give up at that price
To try a coding agent Codex on ChatGPT Free, $0 Claude Code is not on Free. Codex Cloud is not on Free. The docs describe quick tasks, not a full workday.
A small paid plan Codex on Go, $8 a month Still no Codex Cloud. Claude has no published plan between Free and Pro on the pricing page checked here.
The usual individual plan Claude Pro at $17 a month if you pay annually. Month to month, both are $20. Annual Pro is a year up front. Plus stays a monthly plan in the ChatGPT pricing FAQ.
A heavier individual plan The $100 and $200 stickers match: Max or Ultra, and Pro at $100 or $200. The multiples differ. Max is 5 times Pro on a five-hour and a weekly window. The Codex docs say Pro has no five-hour limit. Ultrafast is the $500 Codex tier.
A team, billed annually Both entry seats are $20 per person, for two or more people. Claude premium seats jump to $100 annually. A Codex FAQ line also names a $100 Business plan for fuller GPT-6 Astra access. Enterprise on both sides needs the sales or FAQ page, not this sticker.
Two billing paths sit beside the stickers. A Claude Console account, and extra usage credits on a paid Claude plan, bill tokens at API rates. An OpenAI API key bills Codex the same way and drops the cloud integrations. The help center says standard Codex Cloud environments had no separate virtual-machine charge at launch, and that model usage still counts against the plan or the credits. Auto-review safety checks, when you are signed in with ChatGPT, are free and do not count toward the allowance. The software that spends those tokens is the harness. A $20 plan with a tight loop and a $20 plan with a long cloud task are different bills even when the sticker matches. [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) is the page for that split. The directory rows for [Claude Code](/directory/claude-code) and [Codex](/directory/codex) keep the short product notes. ## What are the Anthropic Claude Code limits? Anthropic Claude Code limits, on the pages checked November 6, 2026, are the plan’s limits. Claude Code is not a separate meter. Pro has a five-hour session limit and a weekly limit. Both are shared with the rest of Claude. The pricing FAQ says there is no fixed message count. Max is 5 times Pro usage every five hours and every week. Ultra is 10 times Pro usage every week, and verified accounts have no five-hour limit. The tables above put those windows next to Codex. [Claude Code](/directory/claude-code) is the product note. [Codex](/directory/codex) is the other column. A session with ultracode on draws on this same pool, and the workflows page read for [What does ultracode do in Claude Code?](/blog/what-does-ultracode-do-in-claude-code) does not print a separate price for that setting. This page does not have a separate price for a Claude Code subagent. The vendor pages checked for this comparison do not print one. A background task still draws on the same pool, or on API rates after the included allowance, which is the split in the sections above. ## What should you check before you pay? 1. Pick the surface first. Claude Code’s product page lists the terminal, the web, iOS, Android, GitHub, VS Code, JetBrains, and Slack, and it says the VS Code extension also covers Cursor and [Devin Desktop](/directory/devin#what-is-devin-ide). Codex’s docs list the CLI, the IDE, the desktop app, the web, and cloud tasks on the plans that include cloud. 2. Read the pool, not only the price. Claude Code and Claude chat draw from one pool. Codex, ChatGPT Work, and the other agent features named on the help page draw from a shared allowance when your plan uses one. 3. Treat message ranges as estimates. Anthropic says the mix of length, model, and features means there is no fixed message count. OpenAI says similar tasks can consume different amounts, and prompt length alone is a poor estimate. 4. Keep a receipt path for overage. Usage credits, a Console account, or an API key are how either product continues after the included allowance. Team monthly API credits are not that path for Claude Code. ## Frequently asked questions ### Which costs less, Claude Code or Codex? Codex costs less to start. Free ChatGPT includes it, and Go is $8 a month. Claude Code starts at Pro, $20 a month or $17 a month on an annual bill. The $100 and $200 individual prices match. Codex Pro also lists $500. ### Is Claude Code on the free plan? The [Claude pricing table](https://claude.com/pricing) marks Claude Code unavailable on Free and available on Pro, Max, and Ultra. Paid plans include it, and the product page adds Team, Enterprise, and Console. ### Does free ChatGPT include Codex? Yes. The [Codex docs](https://learn.chatgpt.com/docs/pricing) list Free at $0, and the [help center](https://help.openai.com/en/articles/11369540-using-codex-with-your-chatgpt-plan) includes Free and Go. Codex Cloud is not included on those two plans. ### What does the $20 plan include for a coding agent? Claude Pro includes Claude Code in the same pool as the rest of Claude, with a five-hour limit and a weekly limit. ChatGPT Plus includes Codex on the web, in the CLI, in the IDE, and on iOS, plus cloud code review and Slack. Neither page promises a fixed number of tasks. ### Do Claude Max and Codex Pro cost the same? At $100 and $200, yes. Codex Pro also offers $500, which is the tier with Ultrafast mode. Max is 5 times Pro usage on both the five-hour window and the week. The Codex docs say Pro plans have no five-hour limit. ### How much is a team seat? Claude Team standard and ChatGPT Business are both $20 per person per month when billed annually, and both are $25 when billed monthly, for at least two people. Claude premium seats are $100 annually or $125 monthly. Claude Enterprise, in the pricing FAQ, is $20 per seat per month plus API-rate usage, billed annually. On the Codex docs re-read November 9, 2026, Enterprise and Edu plans without flexible pricing have the same per-seat usage limits as Plus for most features. Flexible-pricing workspaces scale with credits and have no fixed rate limit. ### Can I pay API rates instead of a subscription? Yes. Console bills Claude Code at standard API rates. An OpenAI API key bills Codex at API rates in the CLI, SDK, or IDE, without the cloud integrations. Paid subscriptions can also buy credits after the included allowance runs out. ### What are the Anthropic Claude Code limits? They are the plan’s limits, not a separate Claude Code meter. On the November 6, 2026 check, Pro has a five-hour session limit and a weekly limit, shared with the rest of Claude. Max is 5 times Pro usage on both windows. Ultra is 10 times Pro usage every week, and verified accounts have no five-hour limit. The pages checked here do not print a separate price for a background subagent. The [Claude Code listing](/directory/claude-code) has the same plan names, and [Codex](/directory/codex) is the other product. ## How to Evaluate an AI Agent Harness (Nov 2026) URL: https://www.aiagentsinfo.com/blog/how-do-you-evaluate-an-ai-agent-harness Published: 2026-11-06. Updated: 2026-11-09 Category: guide How to evaluate an AI agent harness: test one action you can see, then check the whole job still finishes. A benchmark score does not name which step moved. Updated November 9, 2026. ## How do you evaluate an AI agent harness? How to evaluate an AI agent harness: check the actions you expect, then check whether the whole job still finishes. A [behavioral evaluation](/glossary/behavioral-evaluation) watches one step you can point at in the trace: a tool call, a file edit, a clarifying question, a test command. An end-to-end [benchmark](/glossary/benchmark) scores a frozen set of tasks and tells you whether the bundle of model plus [harness](/glossary/harness) succeeded. [Google’s September 9, 2026 post](https://developers.googleblog.com/en/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/) says a composite score can move a few points and still leave you unsure which step changed. The behavioral check is how you name that step. The workshop those checks sit on is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). How to build the infrastructure that runs the checks — tasks, isolated trials, graders — is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). The cycle inside that workshop is [How do AI agents work?](/blog/how-do-ai-agents-work). Designing it on purpose is [harness engineering](/glossary/harness-engineering). The shelf for both is the [harnesses](/topics/harnesses) topic. Tools that run those checks, with licences and list prices from official pages, are [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). The runtimes they wrap are [Best AI agent harness](/best/ai-agent-harnesses). Dated public boards are the [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026). Live install counts sit on the [trackers](/trackers) and the [downloads tracker](/trackers/downloads). ## What is a behavioral evaluation? A behavioral evaluation is a test of an observable action, written the way a software team writes an integration test for a function. Google’s September 9 post, by Taylor Mullen and Christian Gunderman, draws the contrast with a report card. Many teams hand an agent a large codebase, set a time limit, and count how many tests pass. When that score drops, the post lists the questions the score does not answer: the model may have guessed on an ambiguous prompt, skipped the test suite, or invented a command-line flag. A behavioral eval asks a smaller question: - Given an underspecified prompt, does the agent ask a clarifying question? - When it edits a build file, does it run the local validator before it says the edit is done? - When it writes documentation, does it include the canonical links for the repository? Those are harness behaviors. The [model](/glossary/llm) proposes the step. The harness offers the tools, runs the one the model picked, and records what happened. A behavioral eval reads that record. The same idea applies outside coding agents. Any AI evaluation that scores only the final paragraph misses the path, and for an [agent](/glossary/agent) the path is the product. A chat benchmark can still grade the last answer. An agent evaluation also has to grade whether the system searched, called a [tool](/glossary/tool-use), loaded an [agent skill](/glossary/agent-skill), or stopped for a person. [Databricks’ June 17, 2026 explainer](https://www.databricks.com/blog/ai-harness) puts evaluation next to observability for that reason: at production scale you need a record of what the agent did, and a way to measure it across many runs. A narrow check can also be the wrong check. [Anthropic’s November 26, 2025 note](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents) describes agents that marked features done after unit tests or `curl` checks, while the feature still failed when someone clicked through it. The note’s response was to require browser testing, through the Puppeteer MCP, used the way a person would. A behavioral eval is only as good as the action it names. “The unit test ran” and “a person could complete the click path” are different actions. ## How does a behavioral eval differ from an agent harness benchmark? A behavioral eval checks one action. An agent harness benchmark scores whether the model plus harness finished a frozen set of tasks. Google’s post treats them as a pair. Macro benchmarks verify the destination. Micro behavioral evals are the partner that lets you change a prompt, a tool, or a model without guessing.
Behavioral evaluation End-to-end benchmark
Question Did this expected action happen? Did the task succeed under the suite’s scoring rule?
What you read A tool call, a file change, a question, or another step in the trace. A score on a frozen task set. The harness is part of the bundle being scored.
What a drop tells you Which behavior moved. Google’s examples are a missed question, a skipped validator, or a missing link. That the bundle did worse. Google names Terminal-Bench and DeepSWE as suites where a small score change is hard to explain.
How strict it should be Strict when one action is the right milestone. For a task with several correct paths, Google says to avoid a rigid tool sequence and use an outcome check, including an LLM-as-a-judge. The suite’s own grader. A search over whole harnesses, such as HarnessFlow, lives at this end of the scale.
What it is for Iteration and regression checks while you edit the harness. A destination measurement. Google says behavioral evals do not replace it.
[LangChain’s March 10, 2026 anatomy post](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) is the public reminder that the destination score still moves when only the harness changes. The company wrote that a harness-only change took one of its coding agents from the top 30 to the top 5 on Terminal-Bench 2.0. That is LangChain’s account of a leaderboard on that date. It is also why a single headline number is a poor debugging tool: the score moved, and a behavioral suite is how you would ask which new behavior carried it. LangChain’s account names Terminal-Bench 2.0. A separate note on [Terminal-Bench 2.1](/research/terminal-bench-2-1-benchmark-repairs) is on this site, and the dated public boards are the [November 2026 agent harness benchmark](/reports/agent-harness-benchmark-november-2026). A research search sits on the benchmark side. [HarnessFlow](/research/harnessflow-mcts-harness-optimization), arXiv:2611.06254, posted November 4, 2026, keeps the model fixed and searches for a better executable harness. This site’s breakdown, checked the same day as this guide, reports the authors’ average of 69.48 percent across six benchmarks after 15 refinement attempts each, 10.89 points above the starting harness and 3.95 points above the Meta-Harness rerun in that paper. Those are automatic grades of whole runs. They answer a different question from “did the agent call search before it answered.” The same-model lesson, with the measurement and its limits, is in [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Hold the harness still when you compare models, or say that you did not. ## When do you start evaluating a harness? Start after the agent can already do ordinary work, and start with one repeated mistake. Google’s post says not to build a large evaluation system on day one. The first phase is developer instinct and dogfooding: use the agent on its own codebase, on boilerplate, and on routine tasks. Until that works, a score is ahead of the system. Evals are the second phase. Their job is confidence that a prompt tweak, a tool-schema change, or a model upgrade did not make the agent worse. The post says the point is not celebrating a 2 percent gain. NVIDIA’s [November 4, 2026 post](https://developer.nvidia.com/blog/use-nvidia-nemo-to-turn-agent-failures-into-tests-and-harness-fixes/) starts from the other direction, once traces exist. Generic benchmark scores may not explain a domain failure: the wrong tool, the wrong branch, missing context, a model that was more expensive than the step required, or a task that finished without meeting a domain rule. The post’s order is: find the failure in stored traces, write a test that reproduces it, change the harness, and measure again. You can write tests before production traffic. NVIDIA says NeMo Eval Author can draft evaluations from a requirements file and from the code, including tool definitions, before real traces exist. Once failures show up in logs, those tests should grow to match the work the agent actually does. Anthropic’s long-running note is a third clock. The eval that matters there is the definition of done. A feature list, stored as JSON with `"passes": false`, stops a later session from declaring the project finished because some of it exists. That list is a harness artifact. The behavioral question is whether the agent flipped a feature to passing only after the check you named. ## How is a behavioral eval written? Write a behavioral eval as a test of one action, such as the tool call in Google’s September 9 example, then rerun the suite so older checks still hold. This section describes that example. It does not add an API beyond what the post shows. Folding that assertion into a task folder, a sandbox, and a numeric reward is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). The example is a Python test. It imports `Agent`, `LocalAgentConfig`, and `types` from a module path the post writes as `google.antigravity`. The test asks an agent for the weather in Mountain View, California, collects the names of the tool calls on the reply, and asserts that a built-in web-search tool is among them. The assertion message says the agent answered from memory without consulting live search. The comment above the assertion is the design rule: assert the behavior, not the prose of the answer. The post then suggests a loop in which a model edits its own system prompt until a failing test passes, while the rest of the suite guards the behaviors you already care about. That is a suggestion on the blog. It is not a separate product this page can point to. The three-step loop on the same page: 1. Pick one failure. Use a recent mistake, such as marking a task done without running unit tests. Name one action that slipped. 2. Match the assertion to the task. If one milestone is the right behavior, assert that milestone on a single turn, for example that the test runner was called. If several paths would be correct, do not lock the tool order. Use an outcome check. The post names an LLM-as-a-judge as one fuzzier check. 3. Run the suite in batches. A single run is noisy, because model output varies. The post says to avoid blocking a pull request on one run, to collect a larger batch, and to watch the aggregate pass rate. The shell block on the page is `pytest evals/behavioral/ -v`, with a comment that the local suite runs in under five seconds. Treat that comment as the post’s illustration of a fast local check. Two harness features show up in the thing you might assert, and they have their own glossary entries. An [agent skill](/glossary/agent-skill) is a `SKILL.md` file a coding harness can load when a task matches its description. The [Model Context Protocol](/glossary/model-context-protocol) is the open protocol for connecting an agent to external tools. [LangChain’s March post](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) lists tools, skills, and MCP servers as harness parts, and it says skill front matter is loaded at the start so the full instructions stay out of context until they are needed. A behavioral eval can ask whether the agent used the skill or the connector you expected. The assertion still has to allow another correct path when the task has one. ## How do you turn a failure into a harness fix? Turn a stored failure into a test, change one part of the harness, and rerun it on cases the editor did not see. NVIDIA’s November 4 post, by Patrick Moorhead, Sean Lopp, and Moon Chung, walks through one cycle with NVIDIA NeMo. The blueprint it points to is NemoClaw for agent optimization. The example agent is a Hermes Agent for a fictional company, running in an NVIDIA OpenShell sandbox, connected through MCP to chat, email, files, and tickets. The post says the blueprint uses three libraries: NeMo Compass, NeMo Eval Author, and NVIDIA NeMo Relay. The walkthrough below is the part the post actually demonstrates: Compass, Eval Author, a harness edit, and a rerun. The failure they start from is specific. Asked to draft a message with project updates, the agent drafted the message and also called `send_message` without the user confirming the send. NeMo Compass reads stored traces plus a short expected-behavior file, `ETHOS.md`. In the example, Compass was pointed at 42 traces. It groups signals about unusual behavior, tool use, and divergence from the intended rules, and it returns trace IDs for the runs behind each finding. NVIDIA says an internal run of the full signal set, on thousands of traces, found anomalies that a coding agent given a similar task missed. That is the company’s comparison. NeMo Eval Author is the next step. The post says it provides skills and workflows for a coding agent such as Codex or Claude Code, and that it can write evaluations in Harbor format or in NVIDIA NeMo Gym format. You hand the coding agent the Compass findings, the raw traces, `ETHOS.md`, and the agent repository, then review what it writes. In the example it produced four Harbor tasks similar to “draft a response,” plus a verifier that fails the run if `send_message` was called without human confirmation. The harness edit in the example is one instruction in the Hermes Agent’s `SOUL.md`, which the post calls a Hermes concept similar to a system prompt. The new line says a request to write, draft, compose, prepare, or suggest a message authorizes the draft only. The agent should show the draft and stop before sending. A token returned by a preparation tool is the technical ability to act, and it is not the user’s consent. The post says other harness edits can be middleware or plugins. This one is a prompt-level rule inside the harness. They then reran the four tasks. Two tasks, marked held out, were hidden from the coding agents that proposed the change.
Evaluation set Baseline passes / runs Updated agent Tool calls, before to after
Development, 2 tasks × 3 runs 3/6 6/6 10 to 6
Held out, 2 tasks × 3 runs 4/6 6/6 28 to 15
Those twelve runs are the example, not a production suite. The post says the team started narrow, on the Compass finding, and that production suites in their experience often hold hundreds to a few thousand examples. Accept a change when it helps both the development cases and the held-out set. If the result is mixed, run Compass on the new traces and keep the held-out cases hidden from the next edit. A second pass on the updated traces dropped the send-without-confirmation finding and surfaced a different one: the agent sometimes had the answer and failed to ground the reply in the retrieved evidence. The post treats that as the next loop, not as a finished agent. The same post applies the loop to a harder engineering example. LEO, the engineering Virtual Companion from Dassault Systèmes, proposes car-door design changes and tests them in simulation. Compass found `QueryModel` arguments placed outside the expected `options` field, with an empty error, across multiple traces. A coding agent added a validation step that catches misplaced arguments. NVIDIA reports malformed tool calls falling from 142 to zero, with accuracy unchanged and no regressions on the other scenarios. An unattended overnight pass of about 12 hours, across six scenarios, is reported as a 28 percent relative improvement in the overall evaluation score and 35 percent fewer tool calls per run. Those figures are NVIDIA’s account of that example. The post’s setup list, for readers who want the version pins, is a Linux machine running Docker, with Ubuntu 24.04, 8 vCPUs, and 32 GB of RAM recommended. On November 8, 2026, that post still names OpenShell 0.1.2, Harbor 0.22.0, an NVIDIA API key from build.nvidia.com, and a coding agent Eval Author supports, such as Codex or Claude Code. This guide does not re-run the blueprint. ## How does evaluation fit harness engineering? Evaluation tells you whether a change to the harness helped. [Harness engineering](/glossary/harness-engineering) is the work of designing the loop, the tools, the memory, the permissions, and the stop rules. [OpenAI’s February 11, 2026 post](https://openai.com/index/harness-engineering/) describes the human job on one internal project as specifying intent, shaping the repository and the tools, and building feedback loops so agents could do the implementation. The authors say a missing capability had to be made legible and enforceable. A behavioral eval is one way to make a failure legible: the test fails until the harness, not a plea to “try harder,” produces the action you named. The longer account of that post is the [harness guide](/blog/what-is-an-ai-agent-harness). LangChain’s March post puts the same split in one line: if you are not the model, you are the harness. It also names an open problem that matches the NVIDIA loop: agents that read their own traces and fix harness-level failures. Compass is a published tool for the reading step. The post still has a person review the generated tests and the harness edit. The two pages are describing the same kind of work from different ends. A [guardrail](/glossary/guardrail) is the check that allows, blocks, or pauses an action in production. A behavioral eval is the measurement that the action did or did not happen. The `SOUL.md` line in NVIDIA’s example is a harness instruction. A confirmation step before `send_message` would be a guardrail. The eval tells you whether runs still send without a person. It does not, by itself, stop the send. [Computer use](/blog/can-an-ai-agent-use-your-computer) is a case where the action is a click. The behavioral question is whether the agent asked before a payment, a password, or another hard-to-undo step. The safety rules on that guide are the harness controls. An eval would record whether those controls held. ## What should you write down before the first eval? Write down one failure you have already seen, the action that should have happened, and at least one case you will keep out of the edit. 1. Name one failure you have already seen, in a trace or in dogfooding. Google’s first step and NVIDIA’s Compass step are the same habit. 2. Write the action that should have happened, in one sentence. “Called the test runner” is an action. “Did a good job” is not. 3. Decide whether that action is the only acceptable path. If it is not, score the outcome and leave the tool order free. 4. Keep at least one case out of the edit. NVIDIA’s held-out tasks are the short version. 5. Rerun the older checks after the harness change. A fix that creates a new failure is still a failure. 6. Keep a destination score for the job you actually care about. Google’s post is explicit that behavioral evals complement end-to-end suites. The [how agents work](/topics/how-agents-work) topic is the loop in general. [Task performance](/topics/task-performance) is where this site follows coding, computer use, and other jobs an agent finishes. [Research](/topics/research) is the shelf for papers that grade a harness, including [HarnessFlow](/research/harnessflow-mcts-harness-optimization). ## Where can you read more on this site? - This guide’s terms: [behavioral evaluation](/glossary/behavioral-evaluation), [agent skill](/glossary/agent-skill), [Model Context Protocol](/glossary/model-context-protocol) - Harness hub: [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do AI agents work?](/blog/how-do-ai-agents-work), [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [agent loop](/glossary/agent-loop), [benchmark](/glossary/benchmark), [guardrail](/glossary/guardrail), [tool use](/glossary/tool-use) - Research: [HarnessFlow](/research/harnessflow-mcts-harness-optimization), [Terminal-Bench 2.1](/research/terminal-bench-2-1-benchmark-repairs) - Nearby guides: [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer), [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing), [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents) - Topics: [agent harnesses hub](/topics/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [research](/topics/research) - Comparisons: [Best AI agent harness](/best/ai-agent-harnesses), [Best AI agent frameworks](/best/ai-agent-frameworks), [Harness comparison tracker](/trackers/harnesses) - Trackers: [live trackers](/trackers), [framework downloads](/trackers/downloads) - Report: [agent harness benchmark, November 2026](/reports/agent-harness-benchmark-november-2026) ## Frequently asked questions ### What is a behavioral evaluation of an AI agent? It is a test of one observable action. [Google’s September 9, 2026 post](https://developers.googleblog.com/en/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/) uses asking a clarifying question, running a validator before calling a build file done, and including canonical documentation links. The [glossary entry](/glossary/behavioral-evaluation) is the short definition. The check reads the step, such as a tool call, rather than grading an entire project. ### How is a behavioral eval different from an agent harness benchmark? An agent harness benchmark scores a frozen task set, and the harness is part of the bundle being scored. A behavioral eval scores a specific action. Google says suites such as Terminal-Bench and DeepSWE show that a score moved and usually do not show which step moved it. The table above is the comparison. Dated public boards are the [November 2026 agent harness benchmark](/reports/agent-harness-benchmark-november-2026). Google also says the behavioral check does not replace the destination score. ### When should you start evaluating an agent harness? After the agent can do routine work, and then on one repeated mistake. Google says dogfooding comes first, and evals are the phase that guards against regressions from prompt, tool, and model changes. NVIDIA’s November 4 post writes the test from a failure already present in stored traces. ### What is NVIDIA NeMo Compass? In the November 4, 2026 technical post, NeMo Compass reads stored traces and a file of expected behavior, then returns recurring problems with the trace IDs behind them. NeMo Eval Author helps a coding agent turn a finding into a Harbor or NeMo Gym evaluation. The harness change comes after that test exists. The NemoClaw blueprint is the walkthrough NVIDIA points to. ### What is a held-out task in an agent evaluation? It is a case hidden from the editor of the harness. NVIDIA withheld two of four example tasks from the coding agents that proposed the `SOUL.md` change. The post says to keep those cases hidden on later rounds, and to adopt a change when both the development tasks and the held-out tasks improve. ### Can a behavioral eval replace a benchmark such as Terminal-Bench? Google says the two are complementary. A small check can pass while the job a person would attempt still fails. Anthropic’s November 26, 2025 note is that case: features marked done after unit tests or `curl` checks that missed end-to-end bugs. Use the behavioral suite to iterate, and keep an end-to-end score for the destination. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) is this site’s breakdown of one paper that grades whole harnesses. ### What is an agent skill in a coding harness? A skill is a folder with a `SKILL.md` file: a name, a description, and instructions the agent loads when a task matches. The open format is the [Agent Skills specification](https://agentskills.io/specification). [LangChain](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) treats skills as part of the harness, loaded in stages so unused MCP and tool instructions do not fill the context window. The glossary entry is [agent skill](/glossary/agent-skill). ### How do you build an eval harness for AI agents? Write tasks with checkable outcomes, isolate every trial, grade the environment or a tool call, and keep a held-out set. Harbor, Inspect, and a local pytest suite are three runners. The guide is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). ## Can an AI agent use your computer? Yes (checked Nov 7, 2026) URL: https://www.aiagentsinfo.com/blog/can-an-ai-agent-use-your-computer Published: 2026-11-05. Updated: 2026-11-08 Category: guide Yes, when the product has computer use. On November 7, 2026, Claude and ChatGPT desktop apps and the Claude, OpenAI, and Gemini APIs can click, type, and scroll. Updated November 8, 2026. ## Can an AI agent use your computer? Yes, when the product has [computer use](/glossary/computer-use). The model looks at the screen. The app clicks, types, and scrolls. A chat reply that tells you which button to press is still a chat. Computer use starts when software can see the current screen and carry out the next action. On November 7, 2026, that software is available in two desktop products and in the Anthropic, OpenAI, and Gemini APIs. [Claude Cowork and Claude Code](https://claude.com/resources/articles/dispatch-and-computer-use) can drive a Mac or Windows screen in a research preview. The [ChatGPT desktop app](https://developers.openai.com/codex/computer-use) can do the same for ChatGPT Work and Codex, in supported regions. [Anthropic](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool) and [OpenAI](https://developers.openai.com/api/docs/guides/tools-computer-use) document API loops in which your application runs each click. [Google’s Computer Use guide](https://aistudio.google.com/docs/computer-use) loaded on November 7. The older address, `ai.google.dev/gemini-api/docs/computer-use`, redirected there. A request to that older address on November 6 had returned HTTP 500. The short version: the model chooses the action. An app you run, or a desktop app that is awake, performs it. Payments, passwords, and other hard-to-undo steps still need a person. A workplace agent that returns a file for review, and only uses the screen as a last resort, is [What is an AI employee?](/blog/what-is-an-ai-employee). A sourced comparison of desktop, API, open-source, and Windows options, without invented scores, is [Best computer use agents](/best/computer-use-agents). ## What is a computer use agent? A computer use agent is an [agent](/glossary/agent) whose tool is [computer use](/glossary/computer-use). The model looks at the screen and asks for a click, a keystroke, or a scroll. The app runs that action and returns the new screen. The glossary entry is the short definition. The product notes below are from the pages re-read on November 7, 2026. ## What does computer use mean? Computer use is a [tool](/glossary/tool-use) that works through the screen. The model receives a picture of the display. It answers with a pointer action, a keystroke, a scroll, or a request for a closer look. The [harness](/glossary/harness) around the model takes that request, performs it, and puts the new screen back in front of the model. That pair, model plus harness, is an [agent](/glossary/agent). Three neighbors are easy to mix up: - A chat-only model writes the next message. It does not move the pointer. - A connector, or any other API tool, calls a named function and gets a structured result. A calendar tool returns an event. A catalog tool returns a price. The [shopping guide](/blog/can-an-ai-shopping-agent-check-out-for-you) is that pattern pointed at a cart. - Computer use is the path for a job that lives in a graphical interface: a desktop app, a settings pane, or a site with no API. The result of a click is a new picture, which the model has to read. Anthropic's product article says Claude reaches for the precise tool first, including connectors such as Slack or Google Calendar. When there is no connector, Claude can control the browser, mouse, keyboard, and screen. The same article says working through the screen is slower than a direct integration, and that computer use is still early compared with Claude's coding and text work. ## What is browser use? [Browser use](/glossary/browser-use) is the narrower tool. The agent reads and acts on a web page. It does not need every app on a desktop. Anthropic's API page says the browser use tool is the closer fit for tasks that stay inside web pages, because its tools act on the page itself. OpenAI's API guide separates the two jobs. `BrowserToolset` is for page text, controls, and tabs. `ComputerToolset` is for screenshots and mouse or keyboard actions in a browser or on a desktop. Google's Computer Use page offers a browser environment alongside mobile and desktop. In that browser environment the client still executes the action, including clicks and a direct navigation to a URL. A full desktop is the wider grant. It can open a local app, a file dialog, or a system setting. The safety sections below treat that wider grant as the case that needs the tighter sandbox. Cognition documents a separate browser inside a Devin session. The [session tools](https://docs.devin.ai/work-with-devin/devin-session-tools) page, read November 9, 2026, says that tab is labeled Computer when computer use is enabled for the organization. What those docs call Devin IDE, and how that differs from Devin Desktop, is [What is Devin IDE?](/directory/devin#what-is-devin-ide). ## How does the loop work? The loop has four beats. The app sends the goal and a picture of the screen. The model chooses the next action. The app runs that action. The app sends back the new picture. The cycle repeats until the model answers in text, or until a limit or a person stops it. Anthropic's API page names the current toolset `computer_toolset_20260801`. One entry in the tools list gives Claude 17 member tools, including `screenshot`, `zoom`, `left_click`, `type`, `key`, and `scroll`. Claude answers with one or more `tool_use` blocks. Your application runs them in order, in an environment you control, and returns a result for each one. A screenshot or a zoom comes back as an image. Several actions in one response are a batch: click, type, then screenshot, for example. Anthropic says a confirmation check belongs before each action in the batch, because one turn can finish a multi-step change. OpenAI's API guide says your application supplies the driver. The driver reads the screen or performs an allowed action, and your application owns the session, the permissions, and the cleanup. The code samples on that page use `gpt-6-astra` and `gpt-6.1-sol`. The same page says the project needs a model that supports the computer tool. A separate line in the guide points to the Agents API for a browser that OpenAI hosts. Google's page describes the same four beats for the Gemini API. Your request includes the Computer Use tool, the target environment, the prompt, and a screenshot. The model returns a function call such as a click, a scroll, or a keystroke. For Gemini 4 and Gemini 3.x, the call also includes an `intent` that states why the model chose that action. The response can include a safety decision: allowed, `require_confirmation`, or blocked. Your client executes the action only when it is allowed or a person confirms it, then sends a new screenshot. The action tables for browser, mobile, and desktop print coordinates from 0 to 999, which the client scales to the screen. A code sample on the same page, read November 7, describes that scale as a 1,000 by 1,000 grid. On this site that surrounding software is the [harness](/blog/what-is-an-ai-agent-harness). Open-source libraries for building that loop, with each repository's latest GitHub release as of 5 November 2026, are in [AI agent framework release, November 2026](/reports/ai-agent-framework-github-stats-november-2026). Designing the loop on purpose is [harness engineering](/glossary/harness-engineering). Checking one of those actions, such as whether the agent asked before a payment, is a [behavioral evaluation](/blog/how-do-you-evaluate-an-ai-agent-harness). Computer use is one tool the harness can offer. The [sandbox](/glossary/sandbox) is the bounded machine the tool acts on. [Autonomy](/glossary/autonomy) is how far the loop runs before a person confirms the next step. ## What do Claude, OpenAI, and Gemini offer? The Claude, ChatGPT, OpenAI API, and Gemini rows are what those pages said on November 7, 2026. Claude’s and OpenAI’s pages matched the November 6 reading. Gemini’s guide was readable again, at a new address. A model name in a code sample is the name printed in that sample. It is a pointer to the page, and the page can change which models it lists.
Product or path Who runs the environment Availability when the page was read Notable limits
Claude Cowork and Claude Code Your Mac or Windows computer. The desktop app has to be awake and running. Research preview for Claude Pro, Max, and Ultra. Turn it on in the desktop app settings. Dispatch can hand the task off from the phone. Anthropic says it asks for explicit permission first, and requests permission before a new app. Some apps are off-limits by default. Computer use is still early next to coding and text. Keep sensitive data out.
ChatGPT desktop app Your Mac or Windows desktop, in the apps you allow. On Windows it uses the active desktop. In supported regions, with ChatGPT Work and Codex. Open Settings, then Computer use, and turn on Any App. If that row says Install, follow the prompts. Permission before each new app. It cannot automate a terminal app or ChatGPT itself. On Windows it takes the foreground. Stay present for payments and credentials.
Claude API, computer_toolset_20260801 Your application, in an environment you control. Anthropic recommends a virtual machine or a container. Messages API, re-read November 7, 2026. The quick start needs no beta header. The availability table lists computer_toolset_20260801 as available for Claude Fable 5.1, Claude Mythos 5.1, Claude Fable 5, Claude Mythos 5, Claude Mythos Preview, Claude Opus 5.5, Claude Opus 5, Claude Sonnet 5.5, Claude Sonnet 5, Claude Haiku 5.5, and Claude Opus 4.8 on the Claude API and Google Cloud. A second available row, without Mythos Preview, covers AWS, Amazon Bedrock, and Microsoft Foundry. The beta row lists Claude Haiku 4.5, Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 4.6, Claude Opus 4.5, and Claude Sonnet 4.5. Not available in Claude Managed Agents. Seventeen member tools. You perform every call. Confirm financial transactions and other consequential actions. Treat the screen as untrusted.
OpenAI API, ComputerToolset and BrowserToolset Your driver, on a browser or desktop you run, or a partner sandbox such as Daytona or E2B Desktop. API guide re-read November 7, 2026. Samples use gpt-6-astra and gpt-6.1-sol. The page says you need a model that supports the computer tool. Isolate the environment. Confirm purchases, destructive changes, and typing secrets. Bound the steps, the time, or the cost.
OpenAI Agents API, hosted browser OpenAI hosts the browser. The computer-use page points to a separate Agents API guide for this path. That separate guide. The computer-use page does not list plan names for it. The safety section on the computer-use page still says to confirm hard-to-reverse actions and to bound the run.
Gemini API Computer Use Your client. Google recommends a sandboxed virtual machine, a container, or a separate browser profile. Read November 7, 2026, at the AI Studio address. The older ai.google.dev address redirected there. A request on November 6 had returned HTTP 500. The changelog, re-read November 7, still dates general availability to October 30, 2026, on the Interactions API and the GenerateContent API. The model list names Gemini 4 Argon, Gemini 3.8 Flash as the recommended model, Gemini 3.5 Flash-Lite, and Gemini 3 Flash Preview. Browser, mobile, and desktop for Gemini 4 and Gemini 3.x. The client executes the action. Payments can be blocked or sent for confirmation. Google says the feature can contain errors.
Tencent's Hy4 release sits on its own page. The sourced account is [Tencent releases Hy4 with image understanding and computer use](/blog/tencent-releases-hy4-with-computer-use). Tencent’s English announcement, re-read November 6, 2026, still says Hy4 can use a command line and a graphical interface, still places the weights and a system card in the coming days, and says Hy4 Flash is scheduled for release soon. This table stays with the desktop products and the API guides checked for the rows above. ## What does Claude computer use require? Anthropic's article says computer use in Claude Cowork and Claude Code is a research preview for Claude Pro, Max, and Ultra. It runs on macOS and Windows. You enable it in the desktop app settings, and the desktop app has to be awake and running. After that, Anthropic says Claude can open files, use the browser, and run dev tools without further setup. Complex tasks sometimes need a second try. Dispatch, described in the same article, keeps one conversation going between the phone and the desktop. You can assign a task from the phone. Claude can keep working while you are away, for example on a morning briefing or a pull request. The awake rule still applies: the desktop app has to be running. You can stop Claude at any point. The article says the system scans model activations for prompt injection, and that some apps are off-limits by default. The recommendation on the page is to start with apps you trust and to keep sensitive data out of the task. ## What does the ChatGPT desktop app require? OpenAI's desktop page, re-read November 7, 2026, says Computer Use is available on macOS and Windows with ChatGPT Work and Codex, in supported regions. Setup is Settings, then Computer use. Under Control, find Any App. If the row says Install, follow the prompts. If the switch is off, turn it on. If the app shows a Computer Use plugin page instead, follow that page. If Any App is missing, separate controls for a browser or one app do not grant access to other desktop apps. In a managed workspace, an administrator decides whether Computer Use is allowed. You switch the app to Work or select Codex, and you turn on control for the apps ChatGPT may use. During a task, ChatGPT asks before it uses an app. An app you mark Always allow can be used later without a fresh prompt. You can remove that grant in the same settings screen. On Windows, the page says those saved decisions live in `$CODEX_HOME/config.toml`, under `computer_use.windows`, in `always_allowed_app_ids`. A task can start by mentioning `@Computer` or an app name, or by asking ChatGPT to use Computer Use. If the target app has a plugin or an MCP server, the page says to prefer that for data access and for operations you want to repeat. On a Mac, ChatGPT needs Screen Recording permission so it can see the app, and Accessibility permission so it can click and type. A scoped task can run in the background while you work elsewhere. Locked use is a separate switch. Enabling it installs an Apple authorization plug-in that participates in the macOS unlock flow. After you enable it, a task started from a connected device can continue once the Mac locks. OpenAI says that unlock is limited to an active Computer Use turn, that ChatGPT covers the displays while the desktop is temporarily unlocked, and that local keyboard or pointer input locks the Mac again. On Windows, Computer Use runs on the active desktop and takes the foreground. It cannot keep working in the background while you use that same Windows session. If the task should continue while you step away, OpenAI says to leave the device unlocked and online, check it from a phone, or run the desktop app inside a Windows virtual machine so the agent takes over the virtual machine. The page says the feature cannot automate terminal apps or ChatGPT itself, and that it cannot authenticate as an administrator or approve the operating system's own security prompts. For a difficult task that depends on screenshots, OpenAI says to choose GPT-6 Astra when that model is in the selector. For a web app you are building locally, the page says to use the built-in browser first. ## What do the API pages require? The API pages agree on the split. The model proposes the action. Your code performs it. Anthropic's quick start adds `computer_toolset_20260801` to a Messages API request and uses no beta header. On November 7, 2026, the availability table still lists that toolset as available for Claude Fable 5.1, Claude Mythos 5.1, Claude Fable 5, Claude Mythos 5, Claude Mythos Preview, Claude Opus 5.5, Claude Opus 5, Claude Sonnet 5.5, Claude Sonnet 5, Claude Haiku 5.5, and Claude Opus 4.8 on the Claude API and on Google Cloud. The AWS, Bedrock, and Microsoft Foundry available row names the same models except Mythos Preview. The beta row names Claude Haiku 4.5, Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 4.6, Claude Opus 4.5, and Claude Sonnet 4.5. The same table says the toolset is not available in Claude Managed Agents. Your application runs every call. Anthropic's precautions are a dedicated virtual machine or container, no sensitive logins in that environment, an allowlist of domains, and a person confirming consequential actions. OpenAI's guide offers three starts. Screenshots plus mouse and keyboard go through `ComputerToolset`. Page text, controls, and tabs go through `BrowserToolset`, with your driver exposing the operations you allow. A browser hosted by OpenAI is the Agents API path, documented separately. Partner environments named on the page include Browserbase, Browser Use, Daytona, and E2B Desktop. The page says to set a session timeout, to close the session even when the run fails, and to restrict which sites the session can reach. Google's changelog, re-read November 7, 2026, still says Computer Use reached general availability on October 30, 2026, on the Interactions API and the GenerateContent API. The Computer Use guide on AI Studio, read the same day, lists Gemini 4 Argon, Gemini 3.8 Flash, Gemini 3.5 Flash-Lite, and Gemini 3 Flash Preview. The code samples use `gemini-3.8-flash`, and the page calls Gemini 3.8 Flash the recommended model for computer use. Gemini 4 and Gemini 3.x can target a browser, a mobile device, or a desktop. The action tables print coordinates as integers from 0 to 999. A code sample on the same page scales those coordinates from a 1,000 by 1,000 grid, and one helper’s comment says the normalized range runs from 0 to 1,000. The reference implementation linked from the page is a Docker sandbox you run yourself. The older `ai.google.dev` Computer Use address redirected to this AI Studio page on November 7. A request on November 6 had returned HTTP 500. ## What safety rules do the vendors publish? Computer use reads pixels and then changes a machine. The pages checked for this guide publish the same four controls, with different wording. ### Where should the actions run? Put the actions in a [sandbox](/glossary/sandbox). Anthropic's API page says to use a dedicated virtual machine or container with minimal privileges. OpenAI's API page says to use an isolated browser or virtual machine, and an allowlist of sites and actions. Google's page says to use a sandboxed virtual machine, a container, or a dedicated browser profile with limited permissions, so the agent stays off the host system. The desktop products are the exception you opt into. Claude Cowork, Claude Code, and the ChatGPT desktop app are built to operate the computer in front of you. Their pages still tell you to limit which apps are in scope and to keep sensitive apps closed unless the task needs them. ### Who has to agree first? A [guardrail](/glossary/guardrail) is the check that allows, blocks, or pauses an action. Claude's product article says Claude asks for your explicit permission first, and that it requests permission before it opens a new application. You can stop it at any point. ChatGPT asks before it uses an app on your computer, and it may also ask before a sensitive or disruptive action. Always allow is a saved grant for an app you trust. Workspace administrators can restrict which apps are available and whether those saved grants are allowed. On the APIs, the check is yours to implement. Anthropic says to ask a person to confirm decisions with real-world consequences, including accepting cookies, completing a financial transaction, or agreeing to terms. A batch of actions can finish that sequence inside one model turn, so the check has to run before each action, which means before the click that sends the money. OpenAI says to keep the user in control of purchases, data transmission, destructive changes, and other actions that are hard to reverse. Typing sensitive information into a form counts as transmission. Google returns a safety decision on the function call. `require_confirmation` means your application prompts the person. A blocked action means the client stops. Google also says that even if you disable a safety policy, the model may still ask for confirmation, and your application should handle that decision every time. ### What about payments, passwords, and other hard-to-undo steps? The pages treat money and secrets as a person-in-the-loop problem. Anthropic's product article says computer use is still early, that Claude can make mistakes, and that you should avoid sensitive data. The API page lists financial transactions among the actions to confirm, and it says to avoid giving the model account logins. OpenAI's API page puts purchases and typed secrets on the confirm list, and it says to bound the run with a step limit, a time limit, or a cost limit, and to check the real outcome rather than trust only the model's last sentence. The ChatGPT desktop page says to stay present for account, security, privacy, network, payment, or credential settings, and to avoid tasks that need secrets unless you are there to approve each step. If ChatGPT uses a browser where you are already signed in, the site can treat the click as yours. Google's warning says Computer Use may contain errors and security vulnerabilities. The recommendation is close supervision for important tasks, and avoiding the feature for critical decisions, sensitive data, or actions that cannot be corrected. The built-in category `FINANCIAL_TRANSACTIONS` blocks or asks for confirmation on payments, retail checkout, and regulated goods. Other categories cover changing health, financial, or government records, sending messages, creating accounts, deleting or sharing files, cookie banners, and legal terms. A shopping agent that pays through a checkout integration is a different path, with its own gates. That guide is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). A computer-use agent that clicks a store's Pay button is the case these safety sections are describing. Leave that click for a person. ### Why is text on the screen untrusted? The screen is an input the model will follow. A web page, a document, or an image can contain instructions aimed at the agent. That is prompt injection, and the vendors say so in different words. Anthropic's API page says Claude can follow commands found in web pages or in images even when they conflict with your instructions. If you use the computer use tools, classifiers scan what the tools return, including screenshots, and steer the model to check that the instruction came from you. Anthropic says that extra scan is a poor fit for some uses with no person in the loop, and that a team can ask support to turn it off. The virtual machine, the allowlist, and the human check still matter when the scan is on. The product article says the desktop feature scans model activations for the same class of attack. OpenAI's API page says to treat screen content as untrusted. Text in a page, a document, or a tool result cannot grant permission or override the user's instructions. The desktop page says web pages can contain malicious or misleading content, and that you should review website actions as if you were taking them yourself. Google's page offers prompt-injection detection for Gemini 3.5 Flash-Lite and later models. It is off unless you turn it on. When it is on, Google says it checks a screenshot for hidden instructions, such as a line telling the model to ignore previous commands, and blocks execution when it finds them. The safety best-practices list still asks for a sandbox, an allowlist, and a log of the prompts, the screenshots, the suggested actions, and the actions the client actually ran. ## When is a connector the better tool? Use a connector when the service has one. The connector returns a field you can check. A click returns a picture, and the model has to interpret the picture correctly, including small text, a changed layout, or a pop-up. Anthropic's product article puts connectors first and calls screen work slower than a direct integration. OpenAI's desktop page says the same about a plugin or an MCP server: prefer it for data access and for operations you want to repeat, and use Computer Use when the task has to be seen or clicked. For public information that needs no interaction, OpenAI's API guide says to use web search. Google's page is for building an agent that must operate an interface. A catalog call or a payment API is a different tool, and the shopping guide is the place this site covers payment. [Tool use](/glossary/tool-use) is the general name for all of these calls. Computer use is the member of that family that works in pixels. [Browser use](/glossary/browser-use) is the member that stays on a page. Picking the narrowest tool that can finish the job is the harness decision. ## What should you set up before you turn it on? 1. Name the machine that will receive the clicks. A spare virtual machine is the setup the API pages recommend. Your everyday desktop is the setup the Claude and ChatGPT apps are offering, and it deserves a shorter allowlist. 2. Turn on the product switch and the operating-system permissions that page names. For ChatGPT on a Mac, those are Screen Recording and Accessibility. For Claude, the desktop app has to be awake. 3. Allow specific apps. Claude asks before a new app, and some apps start off-limits. ChatGPT asks before an app unless you have saved Always allow. An API driver should expose only the actions the task needs. 4. Write down the steps that wait for a person. Payments, passwords, sending a message, deleting files, and accepting terms are the examples on these pages. 5. Prefer the connector, the plugin, or the built-in browser when the task does not need the whole desktop. 6. Keep a stop control, and read the result in the app itself. OpenAI's API page says to check the outcome on the screen, which means the model's closing sentence is a report. The [task performance](/topics/task-performance) topic is the shelf for computer use, coding agents, and other jobs an agent finishes. [How agents work](/topics/how-agents-work) is the loop in general. [Harnesses](/topics/harnesses) is the workshop around the model. [New AI tools and SaaS](/topics/tools-saas) is the shelf for the products. ## Where can you read more on this site? - Glossary: [computer use](/glossary/computer-use), [browser use](/glossary/browser-use), [sandbox](/glossary/sandbox), [harness](/glossary/harness), [tool use](/glossary/tool-use), [guardrail](/glossary/guardrail), [autonomy](/glossary/autonomy), [agent](/glossary/agent), [Model Context Protocol](/glossary/model-context-protocol) - Guides: [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [What is an AI employee?](/blog/what-is-an-ai-employee), [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you) - News: [Tencent releases Hy4 with image understanding and computer use](/blog/tencent-releases-hy4-with-computer-use) - Report: [Browser and computer-use agents next 6 months: 10 products, one date](/reports/browser-and-computer-use-agents-next-6-months) - Directory: [Claude](/directory/claude), [Claude Code](/directory/claude-code), [ChatGPT](/directory/chatgpt), [Gemini](/directory/gemini) - Topics: [task performance](/topics/task-performance), [how agents work](/topics/how-agents-work), [harnesses](/topics/harnesses), [tools and SaaS](/topics/tools-saas) ## Frequently asked questions ### What is a computer use agent? It is an [agent](/glossary/agent) whose tool is [computer use](/glossary/computer-use). The model looks at the screen and asks for a click, a keystroke, or a scroll. The app runs that action and returns the new screen. On the pages re-read November 7, 2026, the cases in this guide are Claude Cowork, Claude Code, the ChatGPT desktop app, and the Claude, OpenAI, and Gemini APIs. ### Can an AI agent use my real desktop? Yes, in the desktop products re-read on November 7, 2026. Claude Cowork and Claude Code can point and click on a Mac or Windows PC while the desktop app is awake. That feature is a research preview for Claude Pro, Max, and Ultra. The ChatGPT desktop app can operate a Mac or Windows interface for ChatGPT Work and Codex, in supported regions, and only in the apps you allow. The Anthropic and OpenAI API guides describe a browser or machine your code controls, and they recommend keeping it isolated from the computer that holds your passwords. Google’s Computer Use guide loaded the same day on AI Studio. A request to the older address on November 6 returned HTTP 500. ### Do I need to approve each click? You approve access, and you should approve anything that is hard to undo. Claude's product article says Claude asks for explicit permission first and before it opens a new app. ChatGPT asks before it uses an app, with an optional Always allow for later tasks, and it may also ask before a sensitive action. On the APIs, a model can propose several clicks in one turn. Anthropic, OpenAI, and Google each say a person confirms purchases and other consequential actions before they run. ### Is computer use the same thing as an agent harness? Computer use is a tool. The [harness](/glossary/harness) is the software around the model: the loop, the tools, the memory, the permissions, and the stop rule. The harness offers computer use, runs the click inside a [sandbox](/glossary/sandbox), and brings the new screen back. An [agent](/glossary/agent) is the model plus that harness. The longer account is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). ### How do Claude, OpenAI, and Gemini compare for computer use? Claude's desktop feature is a research preview in Cowork and Claude Code, on macOS and Windows, for Pro, Max, and Ultra. ChatGPT's desktop app offers Computer Use on macOS and Windows with Work and Codex. Anthropic and OpenAI document an API where your application executes the action. On November 7, Anthropic’s availability table still lists `computer_toolset_20260801` as available for a current set that includes Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, Claude Haiku 5.5, and Claude Opus 4.8, and as beta for an older set. It is not available in Claude Managed Agents. Google's changelog, re-read the same day, dates general availability to October 30, 2026. The Computer Use guide itself loaded on AI Studio. The table above has the limits from each read. ### Is computer use reliable enough for money or passwords? The pages checked here say to keep a person on those steps. Anthropic says the desktop feature is still early compared with coding and text, and says to avoid sensitive data. OpenAI says to confirm purchases and the typing of secrets, and the ChatGPT page says to stay present for payment and credential settings. Google says the feature can contain errors, and its payments category can block a retail checkout or ask for confirmation. A checkout integration with its own gate is the shopping guide, [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). ### When should I use a connector instead of clicking the screen? Use the connector, plugin, or API when the service has one. Anthropic says Claude tries a connector first, and that the screen is slower than a direct integration. OpenAI's desktop page says to prefer a plugin or MCP server for repeatable data access, and to use Computer Use when the task has to be seen or clicked. A click is the fallback for a job that only exists as a graphical interface. ### Does my computer have to stay awake? For Claude's desktop feature, the app has to be awake and running, including when Dispatch assigns the task from your phone. ChatGPT on Windows uses the active desktop and takes the foreground. On a Mac, ChatGPT can run a scoped task in the background, and locked use, after you enable it, can continue a task once the screen locks. An API agent keeps going only while the browser, virtual machine, or desktop you attached is still running. ## Tencent releases Hy4 with image understanding and computer use URL: https://www.aiagentsinfo.com/blog/tencent-releases-hy4-with-computer-use Published: 2026-11-05. Updated: 2026-11-09 Category: news On November 4, 2026, Tencent released Hy4, a model that can read images and operate a computer. The API is live. On November 9 the announcement still had not published the weights. On November 4, 2026, Tencent released Hy4, the next Hy [language model](/glossary/llm). Tencent says this is the first Hy model that can read images and operate a computer, both through a command line and through a graphical screen. The API is already on Tencent Cloud TokenHub, under the model id `hy4`, and on [OpenRouter](https://openrouter.ai/tencent/hy4) as `tencent/hy4`. ## Are the Hy4 weights public yet? No. On November 9, 2026, Tencent’s November 4 announcement still said the weights and a system card would come in the coming days, and the page still did not link a download. The same re-read still says WorkBuddy and CodeBuddy are free from November 4 through November 15, 2026, and that TokenHub bills Hy4 at 50 percent of the catalog price through December 31, 2026. [Hy4-preview weights](https://huggingface.co/tencent/Hy4-preview) responded with HTTP 200 that day. Those are the preview weights, not the Hy4 release the announcement still describes as upcoming. The same announcement says Hy4, inside the Hyra 2.0 [agent harness](/glossary/harness), changed the pre-training setup for smaller models that share its architecture. A model with computer use is not, by itself, a complete desktop agent. Those products are compared in [Best computer use agents](/best/computer-use-agents). ## What did Tencent release on November 4? The English announcement is on [Tencent’s site](https://www.tencent.com/tencent-launches-hy4-joining-the-ranks-of-the-worlds-leading-open-source-ai-models/), dated November 4, 2026. It calls Hy4 the latest Hy model and says it “introduces visual understanding capabilities for the first time,” along with gains in instruction following, long-context reading, and long-horizon agent tasks. The computer-use sentence is specific. Tencent writes that Hy4 “introduces computer-use capabilities for the first time, enabling it to interact with both command-line interfaces and graphical user interfaces.” The same paragraph says the model had specialized training on professional software such as Blender and CAD, for planning and carrying out long tasks. Two clocks matter. The API listing is the part a caller can use now. The weights are the part Tencent still places “in the coming days,” together with system cards. The announcement also describes Hy4 as an open-source model. Until those files are public, the sourced release is the hosted model, the product integrations, and the price list. TokenHub’s [model list](https://cloud.tencent.com/document/product/1823/130051), updated November 4, 2026 at 12:13:30, is the API spec that matches the launch day. The Hy4 row lists retained thinking, structured output, function calling, a cache, and image understanding. The context window is written as 1M tokens, with a maximum input of 960k and a maximum output of 64k. OpenRouter’s page, checked November 5, writes the same window as 1,048,576 tokens and the completion cap as 65,536 tokens. Those figures line up if the “k” on TokenHub means 1,024: 960 × 1,024 + 64 × 1,024 = 1,048,576. The Hy4 Preview row on the same TokenHub page has the same window and the same thinking, output, tool, and cache labels. It does not list image understanding. ## Where is Hy4 available, and what does a call cost? Tencent says Hy4 is available on Tencent Cloud TokenHub, WorkBuddy, CodeBuddy, Yuanbao, WeCom’s AI assistant Dayuan, and ima. It says the API is being made available through OpenRouter, Hermes, and OpenCode. This article confirmed the OpenRouter listing on November 5. It does not have a separate check that Hermes and OpenCode were serving Hy4 that morning. Tencent also says integration work is underway for QQ, Honor of Kings, Game of Peace, WeRead, and Tencent News. That sentence describes work in progress. WorkBuddy and CodeBuddy are free from November 4 through November 15, 2026, according to the announcement. The post does not attach a time zone to those two dates. The catalog price is on TokenHub’s [price page](https://cloud.tencent.com/document/product/1823/130055), also updated November 4, 2026 at 12:13:30. In the post-paid language-model tables, Hy4 and Hy4 Preview are both listed at 6 yuan per million input tokens, 18 yuan per million output tokens, and 0.3 yuan per million cache-hit tokens. A note at the top of the online-inference section says Hy4 usage is billed at 50 percent of the catalog price through 23:59:59 Beijing time on December 31, 2026, and that the catalog price returns after that. Half of those catalog rates is 3 yuan, 9 yuan, and 0.15 yuan. The batch-task table on the same page lists Hy4 Preview and does not list Hy4. Tencent’s English announcement approximates that catalog as US$0.90 per million input tokens, US$2.70 per million output tokens, and US$0.045 per million cached input tokens, and says Hy4 keeps the Hy4 Preview price. OpenRouter’s Hy4 page, checked November 5, shows Tencent Cloud list rates of $0.834, $2.501, and $0.042 per million input, output, and cache-read tokens, with a displayed 50 percent discount at $0.417, $1.251, and $0.021. Those dollar figures sit near Tencent’s approximation. They are the rates on each page, and this article does not convert yuan into dollars on its own.
Source, checked November 5, 2026 Input Output Cache
TokenHub catalog, per million tokens 6 yuan 18 yuan 0.3 yuan, cache hit
TokenHub bill through December 31, 2026, 23:59:59 Beijing time 50 percent of catalog 50 percent of catalog 50 percent of catalog
Tencent’s English announcement, approximate catalog US$0.90 US$2.70 US$0.045, cached input
OpenRouter, Tencent Cloud list, then the 50 percent rate shown on the page $0.834, then $0.417 $2.501, then $1.251 $0.042, then $0.021, cache read
OpenRouter also says Hy4 accepts text and images and returns text, and that requests can pass `tools`, `tool_choice`, and a JSON schema in `response_format`. That matches the function-calling and structured-output labels on TokenHub. ## What can Hy4 do that the preview row does not list? The announcement and the API table answer different questions. The table is the contract for a TokenHub call. The announcement is Tencent’s account of what the model was trained to do.
Capability Hy4 on TokenHub Hy4 Preview on TokenHub Where Tencent describes it
Context window 1M, with 960k input and 64k output The same limits TokenHub model list, November 4, 2026
Image understanding Listed Absent from the row TokenHub, and the announcement’s “first time” sentence for visual understanding
Function calling Listed Listed TokenHub. See tool use.
Retained thinking Listed, as 深度思考(保留式思考) Listed TokenHub’s label. The thinking from a step stays available.
Computer use Named in the November 4 announcement The announcement assigns this training claim to Hy4 Command line and graphical interface, with Blender and CAD named as training software. TokenHub’s rows use the function-calling label instead.
A [context window](/glossary/context-window) is how much text the model can see on one call. A million tokens is a large window. It still fills up on a long computer-use run, because screenshots and command output are text and images the harness has to carry. What happens then is a harness decision: summarize, drop, or retrieve. Computer use, in the announcement, means the model can be pointed at software and asked to act. On this site that is an [agent](/glossary/agent): a model plus the software that offers tools, runs the chosen step, and brings the result back. Function calling is the API shape of that offer. The TokenHub row documents function calling. The announcement documents the computer-use training. A caller still has to supply the tools, the permissions, and the stop rule. Those pieces are the harness, which the next section covers for the research system Tencent named. The announcement also describes office and finance workflows inside WorkBuddy and WeCom: chats, mail, documents, calendars, and meetings, with the model tracking tasks and deadlines across projects. It says a team feature called Projects is still an upcoming WorkBuddy release. Treat that paragraph as a product plan attached to the model launch. ## What is the Hyra 2.0 harness, and what does Hy4 have to do with it? [Harness engineering](/glossary/harness-engineering) is the work of designing the software around a model: the loop, the tools, the memory, the permissions, and the rule for when to stop. The longer guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Hyra is Tencent’s name for a research harness aimed at science and at the tools scientists reuse. Hyra 2.0’s own post is dated November 1, 2026, four days before the Hy4 announcement. It is the design document. The November 4 news is that Tencent now says Hy4 ran inside that harness and “optimized the pre-training infrastructure for smaller models sharing the same architecture.” Tencent calls that step part of recursive self-improvement. This page has Tencent’s sentences for it, and no separate audit of the training runs. The November 1 post describes a harness built for jobs that last days, with several projects at once. It uses three roles: - Overall talks with human researchers, sets priorities, and handles resources and key decisions. - A Mission owns one research problem over time: direction, experiments, and the conclusion. - Workers read code, run experiments, look at failures, and can challenge the Mission. Different workers can use different models. A scheduler pulls work from a persistent queue. An agent can set a time to wake up when a training run finishes. A Research Board stores ideas, results, evidence, and decisions. The post says operations that spend substantial compute, or that sync shared knowledge, need human approval, and that already authorized work can continue. That approval gate is a [guardrail](/glossary/guardrail). How far the loop runs before a person is asked is [autonomy](/glossary/autonomy). Who holds the next step is [orchestration](/glossary/orchestration). When a worker reports that it is done, the post says it has to attach evidence: code versions, results, logs, or execution traces. For an independent check, Hyra can start a new review worker and hand it the original problem and the evidence first, so the review is less anchored to the first worker’s conclusion. That is a stop rule with a record, which is the part of a harness that keeps a long run from declaring victory on a summary alone. The November 4 announcement also says the team has open-sourced two scientific models developed with Hy4 and Hyra 2.0: HyCell, for single-cell representations, and HyPDE, for physics fields. The [Hyra-results repository](https://github.com/Tencent-Hunyuan/Hyra-results/blob/main/README.md) has a November 2, 2026 note for HyCell. That note describes a 16.1-million-parameter model that maps a human cell’s raw counts to a 512-dimensional embedding, with weights and code in the repository. It reports an overall score of 0.735 on seven atlases held out from pretraining, next to a score of 0.736 for a model it calls Stack, at 218 million parameters. Those scores are the repository’s. The README checked on November 5 documents HyCell and does not contain a HyPDE entry. HyPDE remains a name in the November 4 announcement. ## What scores did Tencent report? The numbers below are Tencent’s, from the November 4 announcement. They are useful because they name the test and the comparison. They are a weak substitute for a public scorecard, because the weights are still unreleased and several of the tests are internal. In a blind evaluation Tencent says it ran with Tencent Financial Technology, 213 financial professionals scored the models, including 153 experts from leading external institutions. Tencent says approximately 70 percent had more than five years of industry experience. Across 579 real-world financial tasks, Tencent reports these overall scores:
Model, in Tencent’s financial evaluation Score
Hy4 3.04
DeepSeek-V4.1-Flash-1017 2.97
Kimi K3.1 Pro 2.68
GLM-5.3 2.68
The announcement does not publish the rubric, the scale, or the harness each comparison model used. A [benchmark](/glossary/benchmark) on this site is a named test with a date. This table is that kind of citation: Tencent’s scores, on Tencent’s tasks, published November 4, 2026. Tencent also says an internal check cut Hy4’s hallucination rate from 12.95 percent on Hy4 Preview to 7.2 percent, and cut errors from unsupported assumptions from 23.1 percent to 5.6 percent. It says Hy4 gained 12.03 percentage points over Hy4 Preview on an Investment Research Agent Benchmark run by the financial AI company Rabyte. On WorkBuddy, Tencent reports an average score of 83.18, a pass rate of 96.15 percent, and first place in nine of twelve scenarios in mainland China. The announcement does not define those twelve scenarios. On TenureTrail, which Tencent describes as 123 executable tasks across seven disciplines, the announcement says Hy4 showed strong results in mathematical construction, physical modeling, environmental and earth sciences, systems engineering, and socio-economic decision-making. It does not print a comparison table for that test. ## What did Tencent say about safety, and when do the weights arrive? The announcement separates two safety reviews, both internal. On cybersecurity, Tencent says Hy4’s training favored defensive work, including proof-of-concept development and fixing vulnerabilities, and that offensive exploitation material was left out of the training set. It says experts from Tencent’s Xuanwu Lab and Keen Security Lab reviewed cryptographic reverse engineering, binary deobfuscation, and vulnerability exploitation, and found no significant concerns about offensive cybersecurity capability in the areas they evaluated. That is the company’s assessment of its own model. On biology, Tencent says internal expert review found strong foundational knowledge and more limited skill at open-ended experimental design than frontier models. The announcement does not publish the review protocol. The weight release is the other open item. Tencent says the weights and system cards come in the coming days, with attention to cybersecurity and biological safety. Until that drop, a reader can call the hosted model and can read the Hyra 2.0 design, and cannot download Hy4 itself from a public weight file cited in these sources. One heading in the English announcement says “Hy4 Flash Recently Released.” The paragraph under it says Hy4 Flash “is scheduled for release soon.” The TokenHub model list and price page updated the same day have rows for Hy4 and Hy4 Preview, and no row named Hy4 Flash. The sourced claim for a model a person can call on November 5 is Hy4. ## What does the launch change for someone building an agent? The practical change on November 4 is a new hosted model with image input, tool calling, a million-token window, and a published price, including a 50 percent TokenHub discount through the end of 2026. Computer use, as Tencent describes it, is a training claim about command lines and graphical software. Putting that model to work still means choosing the tools, the sandbox, and the approval rule. That surrounding system is the harness, whether the product is WorkBuddy or a research queue like Hyra. Hyra 2.0 is a concrete example of the long-job pattern this site already tracks: a queue, a written record, a human gate on expensive steps, and a second pass that sees the evidence before it sees the first agent’s conclusion. The [harness comparison](/trackers/harnesses) stays empty until every cell has its own source. Hyra’s November 1 post is one sourced description. It is a research harness for science and model training, which is a different job from a coding product that edits a repository. DeepSeek’s own harness, including the November 4 pre-release, is covered in the [harness guide](/blog/what-is-an-ai-agent-harness#what-is-deepseek-harness-and-how-is-it-different), updated November 5, 2026. That guide also summarizes the Harmony Harness paper. For the plain-language map of those parts, start with [how agents work](/topics/how-agents-work) and the [harnesses](/topics/harnesses) topic. Builders who want the vendor docs for other harnesses can use the [agent frameworks](/directory/category/agent-frameworks) shelf and the [Builders](/for/builders) page. This launch is one item on the [daily news](/topics/news) beat. Task-shaped follow-ups, including computer use and office work, sit under [task performance](/topics/task-performance). ## Frequently asked questions ### What is Tencent Hy4? Hy4 is Tencent’s newest Hy model, released on November 4, 2026. Tencent says it is the first in that line with visual understanding and computer use. TokenHub lists it as `hy4`, with image understanding, function calling, retained thinking, and a 1M context window. The weights are still described as a coming release. ### Can Hy4 use a computer? Tencent’s announcement says it can interact with a command line and a graphical interface, and names Blender and CAD as software it was trained on. TokenHub’s November 4 model list does not label a computer-use mode. It does list function calling, which is the API hook for [tool use](/glossary/tool-use). ### How much does Hy4 cost? TokenHub’s November 4 price page lists 6 yuan per million input tokens, 18 yuan per million output tokens, and 0.3 yuan per million cache-hit tokens. Usage is billed at 50 percent of that catalog through 23:59:59 Beijing time on December 31, 2026. Tencent’s English post approximates the catalog at US$0.90, US$2.70, and US$0.045. ### Are the Hy4 weights public? The November 4 announcement schedules the weights and system cards for the coming days. The public artifact checked on November 5 is the API. HyCell, a 16.1-million-parameter model the announcement links to Hy4 and Hyra 2.0, has a weights-and-code note in the Hyra-results repository dated November 2. ### What is the Hyra 2.0 harness? It is Tencent’s research [harness](/glossary/harness), described on November 1, 2026. An Overall role sets priorities with people, Missions own research problems, and Workers run experiments. Expensive compute needs approval, and a finished worker has to attach evidence. The November 4 post says Hy4 used that harness to optimize pre-training infrastructure for smaller models of the same architecture. ### How is Hy4 different from Hy4 Preview on TokenHub? Both rows list the same 1M window, retained thinking, structured output, function calling, and cache. Hy4 adds image understanding. The catalog price on the price page is the same for both. The 50 percent discount note names Hy4. ## Can an AI shopping agent check out for you? (Nov 7, 2026) URL: https://www.aiagentsinfo.com/blog/can-an-ai-shopping-agent-check-out-for-you Published: 2026-11-04. Updated: 2026-11-07 Category: guide A shopping agent can pay only with a checkout path, a saved card, and an approval rule. The store remains the seller. Pages re-checked November 7, 2026. Updated November 7, 2026. ## Can an AI shopping agent check out for you? Sometimes. A [shopping agent](/glossary/shopping-agent) can research and compare on its own. It can pay only when the product has a checkout path, a saved payment method, and a rule for who must agree. On the pages re-read for this update, that rule is not the same in every app. Google's Universal Cart can take Google Pay at participating stores, or hand the items to the store's own cart. ChatGPT's merchant page, re-read November 7, 2026, sends the purchase to the store's website or app. Amazon's Alexa for Shopping can buy inside Amazon when a price you set is reached, and it can place an eligible order on another store through Buy for Me. Microsoft Copilot Checkout can finish a purchase inside the assistant for participating merchants. Perplexity Instant Buy is included from the help article retrieved November 4. That page did not load on November 7. The short version: the assistant is allowed to act. The store still sells the item. Licence, list price, and checkout path for four products are in [Best AI agent for shopping](/best/ai-agents-for-shopping). ## What is a shopping agent? A shopping agent is a [language model](/glossary/llm) plus a [harness](/glossary/harness) aimed at a purchase. The model reads the request. The harness calls tools: a catalog search, a price check, a cart, and sometimes a payment. The [agent](/glossary/agent) is the working pair. A chat reply that names three toasters is product advice. It becomes an agent when software can look up a live price, hold a cart, and either stop for a person or place the order under a rule the person already set. That boundary is [autonomy](/glossary/autonomy). Suggesting a product is the low end. Paying with a saved card is the high end. Google's Canada post on November 4, 2026 uses the phrase agentic commerce for the high end: AI that helps complete a task on a person's behalf, with that person's permission. The same post says Google's research found that 59 percent of holiday shoppers in Canada plan to use an AI tool to help them shop this season. That figure is Google's, from that post. ## What is agentic checkout? [Agentic checkout](/glossary/agentic-checkout) is the payment step an assistant can carry out. The person has already chosen an item, or has already given a rule such as "buy this if the price falls to $75." The assistant sends the cart and the payment instruction. The merchant charges the card on the merchant's own systems. Two open protocols show up in the primary pages: [Google's January 11, 2026 announcement](https://blog.google/products/ads-commerce/agentic-commerce-ai-tools-protocol-retailers-platforms/) introduced the Universal Commerce Protocol, UCP. Google says it co-developed UCP with Shopify, Etsy, Wayfair, Target, and Walmart, and that more than 20 other companies endorsed it, including Visa, Mastercard, Stripe, and American Express. Google says UCP covers discovery, buying, and support after the purchase, and that it is compatible with Agent2Agent, Google's Agent Payments Protocol, and the Model Context Protocol. [Shopify's January 11, 2026 post](https://www.shopify.com/news/ai-commerce-at-scale) describes the same protocol as co-developed with Google. [OpenAI's September 29, 2025 post](https://openai.com/index/buy-it-in-chatgpt/) introduced the Agentic Commerce Protocol, ACP, built with Stripe. In that design the merchant stays the merchant of record, processes payment on its existing provider, and handles fulfillment and support. On March 24, 2026, OpenAI said it was extending ACP so merchants can share [product feeds](/glossary/product-feed) for discovery. The protocol is the shared language. The harness is the software in one product that decides whether a given tap is allowed to spend money. ## Who remains the seller? The pages below agree on the seller, even when the button sits inside the assistant. Google's October 15, 2026 cart post says the merchant remains the seller of record, whether the person pays with Google Pay in the cart or transfers the items to the merchant's site. The person gets confirmation and customer support from the merchant. Google's Canada post on November 4 says the same about retailers. OpenAI's September 29, 2025 Instant Checkout post says orders, payments, and fulfillment stay with the merchant. The March 24, 2026 post quotes Shopify's Mani Fazeli: Shopify merchants in ChatGPT complete purchases on the merchant's online store through an in-app browser. Microsoft's Agentic Commerce page, re-read November 7, 2026, says the merchant remains the merchant of record. Checkout uses the merchant’s existing payments, fraud, tax, fulfillment, and reconciliation systems. The page names Shopify and Stripe among its partners. This reading does not include a commission or affiliate-fee sentence. Perplexity's help article on Instant Buy says the order and the payment are sent to the merchant for fulfillment. PayPal's November 25, 2025 announcement of that launch says the retailer stays the merchant of record. Amazon is the exception that still names a seller you already know. Inside Amazon's own store, you are buying from Amazon's checkout. For Buy for Me, Amazon's article says the brand store manages delivery, returns, exchanges, and customer service, and sends the order confirmation. ## How do current assistants handle the purchase? The rows are what each company's own page says. A store named in a launch post is an example from that post, not a complete list of stores on November 4.
Assistant What the page says happens at purchase Who sells Where, and the source date
Google AI Mode and the Gemini app Universal Cart holds items from more than one store. Checkout can use Google Pay in the cart, or the items can move to the merchant's cart. In Canada, direct checkout is rolling out on eligible retailers. The retailer U.S. cart rollout on October 15, 2026, in Search and the Gemini app. Canada direct checkout announced November 4, 2026.
ChatGPT People compare products in the chat. Purchases finish on the merchant's website or app. OpenAI says it is moving away from a standalone Instant Checkout. The merchant Merchant page re-read November 7, 2026: shopping is live for ChatGPT users in the U.S. The course change is the March 24, 2026 post.
Alexa for Shopping It can add items to a cart for you to check out, buy when a price you set is reached, or place Buy for Me on an eligible product from another store. Amazon, for Amazon-store orders. The brand, for Buy for Me delivery and support. Amazon's guide updated October 29, 2026: U.S. customers, in the Shopping app and on Amazon.com.
Microsoft Copilot Copilot Checkout completes the purchase inside Copilot, on the merchant's existing payments, fraud, tax, and fulfillment systems. The merchant Product page re-read November 7, 2026: Copilot.com and the Copilot mobile app. English-language merchants selling to U.S. buyers, in U.S. dollars.
Perplexity Instant Buy sends the order and the payment to a compatible merchant. PayPal's launch announcement powered that checkout. The merchant Help article retrieved November 4, 2026. It did not load on November 7, so this cell was not re-checked. Users in the United States, except Enterprise Pro and Enterprise Max. Launch announcement: November 25, 2025.
Read a row, then open the linked page before you depend on a button. Region, merchant, and plan change the screen a person actually sees. ## What did Google ship, and what opened in Canada on November 4? On January 11, 2026, Google said UCP would soon power checkout on eligible product listings in AI Mode in Search and in the Gemini app, for eligible U.S. retailers, paid with Google Pay. Retailers would remain the seller of record. On October 15, 2026, Google said Universal Cart was rolling out in the U.S. across Google Search and in the Gemini app. The cart saves products from more than one retailer across those surfaces. Google says it sits on the Shopping Graph, "60+ billion continually refreshed product listings," and can notify a person about a price drop or an item back in stock. Checkout with Google Pay, or a transfer into the merchant's cart, was offered with merchants including Nike, Sephora, Target, Ulta Beauty, Walmart, Wayfair, and many Shopify merchants. On November 4, 2026, Google Canada said direct checkout powered by UCP was rolling out to Canadian shoppers on AI Mode in Google Search and in the Gemini app. Named participants at launch: lululemon, select Shopify merchants, Shoppers Drug Mart, and Simons. Payment uses Google Pay and shipping details saved in Google Wallet. Google says a simpler UCP connection inside Merchant Center starts next year. Loyalty rewards are described as later work. Shopify's help page, re-read November 7, 2026, describes its own direct checkout inside Google AI Mode and Gemini. Eligible stores must sell to customers in the United States, Canada, or Australia. Direct checkout is active by default for eligible stores. The shopper pays with a credit or debit card in a Shopify-powered checkout and does not have to open the store. A merchant can turn that off. Products stay discoverable, and the shopper is sent to the online store to pay. Shopify says you pay only your standard payment processing fees. Subscriptions sold only as subscriptions, bundles, customizable products, and B2B-only products are not supported in that direct checkout. Google Pay in Google's cart posts and a card payment in Shopify's help page are two implementations. A shopper should read the button in front of them. Both pages still name the merchant as the seller. ## Can ChatGPT buy things for me? No. On the merchant page re-read November 7, 2026, people discover and compare products in [ChatGPT](/directory/chatgpt), and the purchase finishes on the merchant’s website or app. The dates below are how that changed. On September 29, 2025, OpenAI launched Instant Checkout for U.S. ChatGPT Plus, Pro, and Free users buying from U.S. Etsy sellers, with Shopify merchants described as coming soon. The post said a person taps Buy, confirms the order, and pays without leaving the chat. It also said merchants pay a small fee on completed purchases, and that the fee does not change the price or the ranking. Users confirm each step before an action is taken. Payment tokens are authorized for a specific amount and a specific merchant. On March 24, 2026, OpenAI changed the emphasis. The post says: "We've found that the initial version of Instant Checkout did not offer the level of flexibility that we aspire to provide, so we're allowing merchants to use their own checkout experiences while we focus our efforts on product discovery." The same post says richer visual shopping, including side-by-side comparisons, was rolling out that week to Free, Go, Plus, and Pro. It names Target, Sephora, Nordstrom, Lowe's, Best Buy, The Home Depot, and Wayfair as retailers already integrated into ACP for discovery. It also says Walmart was introducing an in-ChatGPT app with account linking, loyalty, and Walmart payments, available in web browsers, with iOS and Android to follow. This guide does not have a later OpenAI page that confirms those mobile apps shipped. The merchant page re-read November 7, 2026 is the current statement. Shopping is live for ChatGPT users in the United States. Shoppers discover and evaluate products in ChatGPT. Purchases are completed on merchant-owned websites or apps. There are no fees on purchases that start in ChatGPT. Shopify and Etsy catalogs are already integrated. Other merchants apply. Apps inside ChatGPT are optional and aimed at larger merchants who want more control. The September 2025 merchant fee belongs to that earlier Instant Checkout description, not to the fee sentence on the current merchant page. The page also says OpenAI is building a self-serve merchant platform later this year and plans to expand to more merchants and regions over time. ## What can Alexa for Shopping buy? Amazon's help page says Amazon is bringing Rufus and Alexa together into one assistant with shared memory. In the Amazon store, that assistant appears as Alexa for Shopping, on the Amazon Shopping app and on Amazon.com. The directory entry is [Alexa for Shopping](/directory/amazon-rufus). The URL still contains the older name. Amazon's guide, updated October 29, 2026, says all U.S. customers can use it. The guide describes several different permissions: - A question in the main search bar, a side-by-side comparison, or a price history for 30, 90, and 365 days. Nothing is bought. - A Scheduled Action that researches a repeat purchase and either notifies you or adds items to the cart. You review the cart and check out. - Auto-buy at a target price. The guide's examples include buying headphones at 30 percent off, or a shirt at $15 or less. Alexa for Shopping completes the purchase with the default payment method when the price is reached. - Buy for Me, for eligible products from other stores that Amazon is not selling. You tap the button. Amazon completes the purchase with your Amazon payment and shipping details. Amazon's Buy for Me article says that tap opens an Amazon checkout page where you confirm the address, taxes, shipping, and payment, and that Amazon then sends encrypted name, address, and payment details to the brand's site. The brand sends the confirmation and handles delivery and returns. Auto-buy is the high-autonomy case. The permission is given earlier, as a price rule, and the later purchase uses the default payment method. Buy for Me, in the earlier article, still has a confirmation page. Scheduled Actions stop at the cart. ## What do Copilot Checkout and Perplexity Instant Buy do? Microsoft's January 8, 2026 Source post says Copilot Checkout was available in the U.S. on Copilot.com, with PayPal, Shopify, and Stripe, and names Urban Outfitters, Anthropologie, Ashley Furniture, and Etsy among launch sellers. The Agentic Commerce page re-read November 7, 2026 is the current description, and its page metadata records a modification on November 3, 2026. Copilot Checkout is on Copilot.com and the Copilot mobile app. Eligible merchants are English-language merchants who sell to U.S. buyers, in U.S. dollars. The merchant stays the merchant of record. A product feed is required. Shopify merchants are described as automatically enrolled, with controls in Shopify admin. The page has a Stripe tab for Stripe’s Agentic Commerce Suite. It does not mention PayPal, and it does not state a commission. Microsoft's own FAQ still draws the harness line: a feed gets products into answers, and UCP is what lets checkout happen inside the assistant. PayPal's November 25, 2025 announcement says U.S. users could check out inside Perplexity with PayPal, and that the retailer remained the merchant of record. Launch examples included Abercrombie & Fitch, Ashley Furniture, Fabletics, Adorama, and Newegg. That list is the launch list. Perplexity's Instant Buy help article, retrieved November 4, 2026, says U.S. users can buy from merchants compatible with the checkout, that the order and payment go to the merchant, and that Enterprise Pro and Enterprise Max plans are excluded. It does not publish a fixed store list. A request for that help article on November 7 did not load, so those sentences stay the November 4 reading. ## What is a product feed, and why does checkout depend on it? A product feed is a file of rows, one per item the store will stand behind. OpenAI's feed spec says a discovery row needs nine fields: `item_id`, `title`, `description`, `url`, `brand`, `seller_name`, `image_url`, `availability`, and `price`. Availability has to be one of `in_stock`, `out_of_stock`, `pre_order`, `backorder`, or `unknown`. The minimal example on that page leaves checkout off. A column called `is_eligible_checkout` opts a row in only when checkout is already enabled for that integration. Setting the flag does not finish onboarding. Google describes the same job inside Merchant Center. The January 11, 2026 post says new attributes cover questions shoppers ask, compatible accessories, and substitutes, on top of the existing product data. The October 15 cart depends on that Shopping Graph staying current. Shopify's help page tells merchants to keep the product data and the return policy in Merchant Center matched to the store, because discovery reads that feed. A feed is how the assistant avoids inventing a price. It is also the limit. If the row is missing, stale, or marked out of stock, a careful harness has nothing honest to add to the cart. ## How does checkout sit inside a harness? The [harness](/blog/what-is-an-ai-agent-harness) is the workshop around the model: tools, memory, permissions, and the rule for when to stop. A shopping harness uses that same list. [Tool use](/glossary/tool-use) is the catalog search, the cart update, and the payment call. [Orchestration](/glossary/orchestration) is which system owns the next step. In these products the merchant owns the charge, the tax, and the shipment. The assistant owns the conversation and the handoff. A [guardrail](/glossary/guardrail) is the check before money moves: a tap to confirm, a price cap, a merchant allowlist, or a pause for a person. OpenAI's 2025 Instant Checkout post said the person confirms each step. Amazon's auto-buy moves that confirmation earlier, into the price rule, and then uses the default payment method. A computer-use agent that clicks a store's own Pay button is a separate path from the buttons in the table above. On the API pages re-read November 7, 2026, that path still has a payment gate. Anthropic's computer-use tool page lists financial transactions among the actions a person should confirm. OpenAI's computer-use guide says to keep the user in control of purchases. Google's Computer Use guide, now on AI Studio, has a safety category for payments and retail checkout that can block the action or ask for confirmation. The click loop is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer). A workplace agent that drafts a file and stops before it sends or pays is [What is an AI employee?](/blog/what-is-an-ai-employee). [Harness engineering](/glossary/harness-engineering) here is deciding those gates on purpose. A prompt that says "buy the best headphones" is not a checkout policy. The policy is the code that knows the maximum price, the address, the merchant, and what to do when the feed and the product page disagree. An eval is the test of that policy. This site's [benchmark](/glossary/benchmark) entry is the general idea: a measured check, not a vibe. For a shopping agent the useful checks are narrow. Did the charged amount match the approved amount? Did the item identifier match the feed row? Did the assistant stop when the item was out of stock? The [harness comparison](/trackers/harnesses) on this site stays empty until every cell has a source. The table above is a substitute with a date on each row, not a scoreboard. ## What should you confirm before an assistant pays? 1. Read who the seller is. On the Google, ChatGPT, Copilot, and Perplexity pages above, that is the merchant. For Buy for Me, Amazon's article says the brand handles delivery and returns. 2. Read where the feature is offered. The U.S. pages do not, by themselves, cover Canada, and the Canada post names specific retailers. 3. Read what permission you already gave. A cart you still have to check out is a different grant from auto-buy at a target price. 4. Prefer a feed-backed price over a sentence the model composed. If the product page and the assistant disagree, the product page is the one the store will charge. 5. Keep the receipt path. These pages say the merchant sends the confirmation. Save that message. It is the record that sits outside the chat. The [shopping topic](/topics/shopping) is the shelf for this guide. The [harnesses topic](/topics/harnesses) is the shelf for the workshop around it. [How agents work](/topics/how-agents-work) covers the loop in general. [Agent frameworks](/directory/category/agent-frameworks) are libraries for building an agent. They are not, by themselves, a store. ## Where can you read more on this site? - Glossary: [shopping agent](/glossary/shopping-agent), [agentic checkout](/glossary/agentic-checkout), [product feed](/glossary/product-feed), [harness](/glossary/harness), [guardrail](/glossary/guardrail), [autonomy](/glossary/autonomy), [tool use](/glossary/tool-use), [orchestration](/glossary/orchestration) - Guides: [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [What is an AI employee?](/blog/what-is-an-ai-employee), [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer) - Directory: [Amazon Rufus latest features, November 2026](/directory/amazon-rufus#what-are-the-latest-amazon-rufus-features-and-expansion-in-november-2026), [ChatGPT](/directory/chatgpt), [Gemini](/directory/gemini), [Perplexity](/directory/perplexity), [shopping agents](/directory/category/shopping-agents) - Report: [AI shopping agents next 6 months: 6 companies](/reports/ai-shopping-agents-next-6-months) - Topics: [Agents in shopping](/topics/shopping), [harnesses](/topics/harnesses), [how agents work](/topics/how-agents-work) ## Frequently asked questions ### Can ChatGPT buy things for me? No. On the merchant page re-read November 7, 2026, people discover and compare products in [ChatGPT](/directory/chatgpt), and the purchase finishes on the merchant’s website or app. Instant Checkout, launched September 29, 2025, is the earlier in-chat path OpenAI stepped back from on March 24, 2026. ### Can an AI shopping agent check out for you? Sometimes. Google is rolling out direct checkout for eligible Canadian retailers on November 4, 2026, after launching Universal Cart in the U.S. on October 15, 2026. ChatGPT's merchant page says the purchase finishes on the merchant's website or app. Alexa for Shopping can auto-buy at a price you set, or use Buy for Me on an eligible product from another store. Copilot Checkout and Perplexity Instant Buy can pay inside the assistant for participating merchants. Each path is limited to the stores and countries named on that company's page. ### Who is the seller when an assistant pays? The merchant, on the Google, OpenAI, Microsoft, and Perplexity pages checked here. The assistant carries the cart and the permission. The store charges the payment method and owns the order. For Amazon's Buy for Me, the brand handles delivery, returns, and customer service. ### What is the difference between a product feed and checkout? The feed is the catalog the assistant is allowed to quote: an id, a title, a description, a product URL, a brand, a seller, an image, availability, and a price, in OpenAI's spec. Checkout is a second integration. A flag on a feed row does not turn payment on. ### Does ChatGPT still have Instant Checkout? OpenAI launched it on September 29, 2025. On March 24, 2026, OpenAI said the first version was not flexible enough, and that merchants would use their own checkout while ChatGPT focused on discovery. The merchant page re-read November 7, 2026 says purchases are completed on merchant-owned websites or apps. ### Is Alexa for Shopping the same product as Rufus? Amazon's help page says Rufus and Alexa are being brought together. In the Amazon store the result is called Alexa for Shopping. It is available on the Amazon Shopping app and on Amazon.com. ### Which countries are these checkout features offered in? The pages checked here put Universal Cart, ChatGPT shopping, Alexa for Shopping, Copilot Checkout, and Perplexity Instant Buy in the United States, with the limits in the table. Google's November 4, 2026 post adds direct checkout for named Canadian retailers. Shopify's Google channel also allows stores that sell to customers in Australia. A launch list is not worldwide availability. ## Google will turn off Gemini and Search for K-12 students unless schools opt in URL: https://www.aiagentsinfo.com/blog/google-k12-gemini-and-search-off-until-schools-opt-in Published: 2026-11-04. Updated: 2026-11-09 Category: news On November 3, 2026, Google said Gemini and Search stay off for K-12 students until a school turns each one on. Re-read November 9, the blog still places that switch in the coming months. On November 3, 2026, Google said K-12 student accounts will lose access to [Gemini](/directory/gemini) and Google Search until a school administrator turns them back on. Later the same day, Google’s news account said a school can turn Search on by itself, without also turning Gemini on. Search for those accounts is still an AI product: Google said AI Mode and AI Overviews will walk students through a problem step by step instead of giving the final answer. A school Google login is the account on a classroom computer. Google’s change is about that login. It is a switch a school controls. It is not a new chatbot for home use, and the posts do not say a student’s Search box is already dark. Student-facing products, including this school-account rule, are compared in [Best AI agent for students](/best/ai-agents-for-students). On November 9, 2026, Phillips’s blog was still the page at that URL. The body still says access will be entirely off until an administrator turns the products on, and that Google will share specifics in the coming months before the rollout. The post still does not give a calendar day, and it does not say the off switch is already in effect. The Search Guided Learning pilot is still described as beginning in select U.S. schools later this month. The page does not name those schools, and it does not say the pilot has begun. The X posts cited below were not re-fetched on November 9. Their November 3 timestamps are unchanged in this account. This page is for adult readers following a school-account policy. It is not written for students. ## What did Google announce for K-12 accounts? Chris Phillips, Google’s vice president and general manager of education, published the plan on [Google’s blog](https://blog.google/products-and-platforms/products/education/ai-learning-era-updates-2026/) on November 3, 2026. A [short collection page](https://blog.google/products-and-platforms/products/education/ai-learning-era-2026/) the same day points at the same set of updates. Phillips writes that, after listening to schools, teachers, researchers, and policymakers, Google is changing the controls for four products: Gemini, Gemini Notebook, Gemini in Classroom, and Google Search. The sentence that carries the policy is this one: “Access to these products will be entirely off for K-12 student accounts until school administrators explicitly turn them on.” The blog does not name a calendar day for that switch. It says Google will share more specifics with schools “in the coming months ahead of the changes rolling out,” so schools have time to prepare. Until that rollout, the sourced claim is the policy, not a report that student Search has already stopped. The same post pairs the controls with a tutoring style Google calls Guided Learning. In Gemini, Phillips writes, it “coaches students through the process of problem solving through open-ended questions and step-by-step breakdowns,” under a phrase he uses for the goal: “productive struggle.” ## Can a school turn Search on without Gemini? Early coverage said no. [The Verge](https://www.theverge.com/tech/1025411/google-turn-off-search-ai-features) and [TechCrunch](https://techcrunch.com/2026/11/03/google-will-let-schools-disable-its-ai-tools-but-not-without-sacrificing-search/) both reported that Phillips told [The New York Times](https://www.nytimes.com/2026/11/03/us/google-search-schools-restrict-ai.html) that “Search and AI are deeply integrated,” and that a school which restricts AI entirely leaves students without Search. The Verge’s wording of the rest of that comment is that students “will not have access to Search, and they’ll see a message from their school that indicates it’s not available.” TechCrunch printed the same comment with “search” in lowercase. The Times article is the interview those outlets cite. The Washington Post, the same day, reported the catch in its own lede: Gemini off by default, and Google Search off as well. At 20:25 UTC on November 3, the verified [News from Google](https://x.com/NewsFromGoogle/status/2117714815477059722) account posted a clarification. It says the announcements “give schools more choice, not less,” and then states three points: - Gemini, Gemini Notebook, and Search will be off by default, because K-12 schools asked for more control over student use of AI services. - Schools can turn on one, two, or all of those services individually. - Search for schools is being updated so that AI Mode and AI Overviews “offer step-by-step guided learning through problems, so that students don’t get quick answers to schoolwork.” Read the blog and the post together. The blog’s off-by-default list has four names, including Gemini in Classroom. The X post’s “one, two, or all” sentence names three: Gemini, Gemini Notebook, and Search. The post does not say Gemini in Classroom is a fourth independent switch, and it does not withdraw Gemini in Classroom from the blog’s list. The clarification also keeps AI inside the Search experience a school turns on. Google’s words are about AI Mode and AI Overviews offering guided learning. The post does not describe a student Search setting that removes those AI features and leaves only a list of links. [NBC News](https://www.nbcnews.com/tech/tech-news/google-shut-search-schools-opt-ai-rcna607280) updated its account with that social-media statement: schools can turn Google Search on individually without enabling Gemini products, and Search will still have AI features built into it.
Product What Google said on November 3, 2026
Gemini Off for K-12 student accounts until an administrator turns it on. Named in the blog and in the X clarification.
Gemini Notebook Same default as Gemini. Named in both the blog and the X post.
Gemini in Classroom On the blog’s list of four products that stay off until an admin turns them on. Not named in the X post’s list of three.
Google Search Off by default. The X post says a school can turn it on without turning the other named services on. AI Mode and AI Overviews will offer step-by-step guided learning.
## What is Guided Learning, and where is it on? Guided Learning is Google’s name for a coach that asks questions and breaks a problem into steps. Phillips’s examples for Search are a pasted physics problem and a request for help writing a book report. In those cases, he writes, Search shows prompts to think through the concepts “without going directly to the answer.” Two clocks are in the posts, and they are not the same clock as the default-off switch. For Google Search, Guided Learning is a pilot. Phillips says it starts in select U.S. schools later in November 2026, so Google can take educator feedback before expanding to other districts in the coming months. For Google Classroom, Google said the teacher-led version is rolling out globally on November 3. A [research post](https://blog.google/products-and-platforms/products/education/guided-learning-uk-japan/) the same day, by Sofie Cornelis and Asha Johnson, says a teacher assigns the activity. The student then gets a step-by-step coach anchored in course materials, and the teacher gets insights on how students used it. Phillips describes that activity as a one-to-one study tool that also gives teachers a view of where the class may need help. That Classroom rollout is the part Google marked as starting the day of the announcement. The default-off controls are the part Google scheduled behind a notice period of “coming months.” ## What did Google’s studies report? The numbers below are Google’s, from the November 3 research post. This page does not have a separate audit or a paper from the universities. Google says it ran a randomized controlled trial with more than 300 students at the University of Tokyo. Every student had half a day to study a statistics lesson with ordinary online resources. Half of them also had Guided Learning in the Gemini app. Google says students using only the standard web tools improved by 5.3 percentage points from pre-test to post-test. Students who also had Guided Learning improved by 8.1 percentage points. The post calls that “1.5x higher learning gains in just a single study session,” and it notes the session was half a day. The students in that trial are university students, not the K-12 accounts the new default covers. The same post describes a four-week pilot with Ark in three UK secondary schools, across four subjects. Google says teachers found that Guided Learning handled initial questions during independent work, which left more time for one-to-one help. It says the majority of surveyed students felt comfortable asking Gemini for help when stuck, and over half said it helped them solve problems on their own. The post does not give the survey’s sample size. It also says some teachers wanted a clearer view of how students used the materials, and wanted students kept on a specific assignment. Google says the product changes that followed include educator insights and a way to anchor the conversation on uploaded curriculum. ## Who criticized the plan, and when? The sharpest public objections landed before Google’s clarification post. At 18:57 UTC, New York Mayor Zohran Mamdani wrote on [his official account](https://x.com/NYCMayor/status/2117692668213940677): “Google is trying to use its monopoly power to force AI into our public schools.” The next sentences say no corporation should have the power to dictate how children are taught, and that public schools exist to serve students, “not to grow the profit margins of Big Tech.” At 19:29 UTC, Senator Bernie Sanders wrote that Google is “threatening schools: if you stop your students from using AI, we will cut off your access to Google Search.” He called that “extortion” and ended the post with “PAUSE AI NOW.” The post is [here](https://x.com/SenSanders/status/2117700813544185865). Google’s clarification, with the separate on-switches, is timestamped 20:25 UTC. It came after both of those posts. [Fairplay](https://fairplayforkids.org/statement-on-google-turning-off-search-for-schools-that-reject-google-ai/), a children’s advocacy group, issued a statement the same day from executive director Josh Golin. He said Google’s attempt to “strong-arm schools into using its AI products is likely to backfire,” and he urged schools to turn Google’s AI tools off “the minute that option becomes available.” He also said schools should help students find “more privacy-protective alternatives for search” and ways to do research that go past “search and click.” NBC News published written comments from Randi Weingarten, president of the American Federation of Teachers. She said it is unfortunate that Google “has prioritized its business interests over the education of our children,” and that the company “throws the baby out with the bathwater and bans access to the tools altogether.” Those sentences are NBC’s report of her comments. ## When does a student’s account change? Two schedules are in Google’s own posts. Guided Learning inside Google Classroom is the feature Google said is rolling out globally on November 3, when a teacher assigns it. The default-off controls for Gemini, Gemini Notebook, Gemini in Classroom, and Google Search are described as a coming change. Phillips says schools will get more specifics in the coming months before the rollout. The X post explains what the switches will do. It does not say the off position is already in effect for every K-12 student account, and it does not give a date. Re-reading the blog on November 6, 2026 did not add a rollout day. The summary bullets on that page say administrators now have full control to enable or disable the AI products. The body still schedules the off-by-default change ahead of the rollout. Nothing in these sources states a price, a new model name, or an API change. The [Gemini listing](/directory/gemini) on this site is still the product door for the consumer app and the model family. Plans and model names belong on Google’s pages, checked on the date printed there. ## What does this mean for learning with AI? For a family, the practical question is which switches the school will flip, and what Search still does after it is on. Google’s November 3 account is that Search can be on while Gemini stays off, and that the school Search experience includes AI Mode and AI Overviews that coach instead of finishing the assignment. For a student, Guided Learning is a conversation with hints. It is not, in these posts, a system that opens apps, files, or purchases on the student’s behalf. On this site an [agent](/glossary/agent) is a model plus the software that lets it take those steps. A [language model](/glossary/llm) supplies the words. A [guardrail](/glossary/guardrail) is a check that can allow or block a step, and [autonomy](/glossary/autonomy) is how far the system may go before a person confirms it. The school administrator’s on-switch is that kind of human gate, applied to whole products rather than to one click inside a coding agent. The longer explanation of the software around a model is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). The place on this site for tutors and classroom tools is [agents in education](/topics/education). This announcement is one item on the [daily news](/topics/news) beat. ## Frequently asked questions ### Will Google turn off Search for K-12 student accounts? Google’s November 3 blog says Gemini, Gemini Notebook, Gemini in Classroom, and Google Search will be entirely off for K-12 student accounts until an administrator turns them on. Re-read on November 6, the same page still says Google will share more specifics in the coming months before that change rolls out. The posts do not say student Search is already off. A summary bullet also says administrators now have full control to enable or disable AI products for those accounts. ### Can a school turn Search on without turning Gemini on? Google’s news account said so at 20:25 UTC on November 3. Gemini, Gemini Notebook, and Search start off, and a school can turn on one, two, or all three. School Search still includes AI Mode and AI Overviews, which Google says will coach students through problems step by step. ### What is Guided Learning? Google’s name for a coach that uses questions and steps instead of a finished answer. Teacher-assigned Guided Learning in Google Classroom is the piece Google said is rolling out globally on November 3. The Search version is a pilot in select U.S. schools later in November. ### What did Google’s University of Tokyo study report? In Google’s account of its own trial, more than 300 students studied statistics for half a day. The standard-tools group gained 5.3 percentage points. The Guided Learning group gained 8.1. Google calls that 1.5 times the learning gain. It is a university study the company ran, not a K-12 result from an outside lab. ### When did critics post, relative to Google’s clarification? Mayor Zohran Mamdani posted at 18:57 UTC, and Senator Bernie Sanders posted at 19:29 UTC, both on November 3. Google’s post about separate on-switches is timestamped 20:25 UTC. ## Has Anthropic Met Its AI Acceleration Threshold? No (Nov 9, 2026) URL: https://www.aiagentsinfo.com/blog/anthropic-may-soon-meet-ai-acceleration-threshold Published: 2026-11-03. Updated: 2026-11-09 Category: news Has Anthropic met its AI acceleration threshold? No. On November 9, 2026, EDGAR still listed the November 2 Form S-1/A as the latest S-1, and that filing has not designated any current model past the line. ## Has Anthropic met its AI acceleration threshold? No. Anthropic told the U.S. Securities and Exchange Commission on November 2, 2026 that its internal estimates and expectations for future models indicate it may soon meet its own rule for “dramatically accelerated progress attributable to AI.” In the same [Form S-1/A](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm), the company says it has not designated any current model as over that line, and that it does not expect to let [Claude](/directory/claude) design successor frontier models fully on its own without meaningful human supervision. ## What did Anthropic tell the SEC? The document is an amended registration statement, Form S-1/A, for Anthropic, PBC. The [EDGAR index](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/0001628280-26-068676-index.htm) lists a filing date of November 2, 2026 and an acceptance time of 09:24:24 that day, under accession number 0001628280-26-068676. In the risk factors, Anthropic writes that using its own systems to speed research “may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures.” It says much of its evidence on that pace comes from lagging indicators, and that it has in the past failed to identify a significant acceleration until after it had occurred, and may do so again. The filing then states: > For example, our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of “dramatically accelerated progress attributable to AI.” We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data. It adds that it is “evaluating the implications of meeting the AI research and development capability threshold” and “will release further reports in the future as we set forth in our Responsible Scaling Policy.” Elsewhere, describing what it calls “closing the loop,” Anthropic says each version of Claude assists in building the next. The work it lists includes running experiments, generating and processing training data, automating evaluations and safety audits, and writing and reviewing code. It also says that using AI to build more capable AI “could also pose a considerable safety risk, particularly if it sharply accelerates the pace of progress,” and that its [Responsible Scaling Policy](https://www.anthropic.com/responsible-scaling-policy) covers the approach to “what has been termed ‘recursive self-improvement.’” The same passage sets a limit in Anthropic’s own words: “We do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible.” It says it is considering updates to the policy “to keep humans in control of frontier model development at Anthropic.” The filing is also the company’s public account of its recent size. Anthropic states that revenue was $17.0 billion for the three months ended September 30, 2026, up from $1.4 billion for the three months ended September 30, 2025. For the 2026 quarter it states an operating margin of (3)% and an operating loss of $567 million, compared with an operating margin of (224)% and an operating loss of $3.1 billion in the year-earlier quarter. Those figures are the company’s statements in the registration document. [Julie Bort’s report for TechCrunch](https://techcrunch.com/2026/11/02/anthropic-warns-that-its-very-close-to-rsi-capability-aka-self-building-ai/), published the same day, highlighted the recursive-self-improvement language. The sentences above are the wording in the filing itself. ## What does Anthropic’s IPO filing reveal? The November 2 document is Form S-1/A, an amendment to a registration statement under the Securities Act of 1933. File number 333-299633. The original Form S-1 was filed October 26, 2026. EDGAR’s company search for CIK 0002130357, filtered to form type S-1, was checked again on November 9, 2026. The list still had two rows: the October 26 Form S-1, accession 0001628280-26-067168, and the November 2 Form S-1/A, accession 0001628280-26-068676. No later S-1 was on that list. The prospectus says: “This is an initial public offering of shares of Class A common stock of Anthropic, PBC.” It says that before this offering there has been no public market for that stock. It says Anthropic has applied to list the Class A shares on the Nasdaq Global Select Market under the symbol “ANT.” The cover is marked “Subject to Completion.” The estimated price per share is printed as a blank range, and the share count is blank. Those blanks are what the filed document shows. The same prospectus is the source of the acceleration-rule sentences in the previous section, and of the size figures already quoted there. Anthropic states revenue of $17.0 billion for the three months ended September 30, 2026, up from $1.4 billion for the three months ended September 30, 2025. It states an operating margin of (3)% and an operating loss of $567 million for the 2026 quarter, compared with an operating margin of (224)% and an operating loss of $3.1 billion for the three months ended September 30, 2025. Those are the company’s sentences in the registration statement. [Claude](/directory/claude) is the assistant the prospectus discusses. [Claude Code](/directory/claude-code) is the coding product it uses as the practical example, in the section below. This note stays on the [news](/topics/news) topic. The site’s [verified revenue](/verified-revenue) page is a separate list of payment-processor figures for tools. It does not republish this filing. ## What does “dramatically accelerated progress” mean in the policy? The phrase in the filing points at Anthropic’s Responsible Scaling Policy, not at a new public benchmark. The current policy text is [version 3.4, effective July 8, 2026](https://www-cdn.anthropic.com/files/4zrzovbb/website/0bacdc8440ea96e62a8766d99ebe1d4eea6d5f3a.pdf). Anthropic’s [policy page](https://www.anthropic.com/responsible-scaling-policy), read again on November 9, 2026, still says it was last updated August 14, 2026, and still lists version 3.4 as the current policy. The August 14 note on that page is a risk report, and the page does not list a policy version after 3.4. Version 3.4 says the automated R&D threshold is met if Anthropic determines either of two things. The first is that its models would be able to “fully substitute for our entire set of Research Scientists and Research Engineers, at competitive costs (i.e., within a factor of 5).” The second is that there is “dramatic acceleration” of the pace of AI progress for reasons that likely relate to automating AI R&D. For that second case, the policy says Anthropic would look for a situation where it observes or expects “double the rate of progress” in “AI aggregate capabilities,” compared with both the rate it would expect and the fastest rate of extended progress it has observed without significant AI contributions to AI R&D. A footnote says that extended-progress comparison is “over at least three model generations.” The doubling also has to be plausibly “substantially attributable to the automation of research and/or engineering,” as opposed to other factors “such as increased headcount, compute, or general productivity,” and the continuation of the trend has to seem likely to lead to even greater acceleration. The policy’s footnote defines the doubling this way: “‘Double the rate of progress’ means ‘as much progress in one year as one would see in two years at baseline.’” The example in that footnote: if baseline progress involved a 3× scaleup in compute and a 3× improvement in algorithmic efficiency, for a 9× “effective scaleup,” then double the rate of progress “would entail something like an 81× effective scaleup.” The footnote adds: “This is not the same idea as ‘doubling researchers’ productivity,’ since doubling inputs does not necessarily double the rate of progress.” The July 8, 2026 changelog says this threshold “is intended to capture the onset of dramatic recursive self-improvement, and has proven difficult to operationalize.” It also records a clarification: Anthropic would not treat the threshold as crossed if the overall rate of AI progress were constant or slowing, even if that rate were dramatically faster than it would have been without advanced AI tools. In that case, the changelog says, there could be a strong argument for expecting the trend to continue rather than accelerate. A [glossary entry on this site](/glossary/benchmark) is the right scale for reading any later scorecard: a named rule with a date, not a single percentage on a public leaderboard. Anthropic has not published, in this filing, the internal estimates behind the “may imminently meet” sentence. ## Has Claude already crossed that line? The filing’s answer on current models is direct: “We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data.” The “may imminently meet” clause is attached to “internal estimates and our expectations regarding future models, rather than our current models.” Readers who want the distinction in one place can hold both sentences together. Anthropic is describing a forecast about models it has not designated as over the line, and it says its view of the pace can lag the event. That is a different claim from a product announcement that a released Claude model now trains its replacement alone. The supervision sentence quoted above is the company’s statement on that point. ## Why does it matter for AI agents? An [agent](/glossary/agent), on this site, is a model plus a [harness](/glossary/harness): tools, a loop, permissions, and a stop. [Autonomy](/glossary/autonomy) is how far that loop may run before a person confirms the next step. Anthropic’s filing is about where that pattern sits inside a lab that builds the [language model](/glossary/llm) itself. The practical example Anthropic gives is Claude Code, which it calls “our agentic coding product.” It says that as models advanced, Claude’s success rate on challenging, open-ended engineering problems that lack a clear specification reached approximately 90% in September 2026, up approximately 50 percentage points in nine months, “based on our internal benchmarks.” It says the average daily volume of code contributed to its codebase per engineer in the three months ended September 30, 2026 was over 45 times the average daily volume in the year ended December 31, 2024, and that over 90% of contributions to the codebase in September 2026 were written by Claude. Those are Anthropic’s figures, from internal measurement, published in the registration statement. They are not an independent audit. The filing describes the shift in how people work with that system: engineers “increasingly direct and review its work rather than write code line by line,” after Claude Code “progressed from suggesting code to writing and running it directly and working autonomously over long time horizons.” That is [tool use](/glossary/tool-use) inside a company, with a person still in the review role the filing describes. The policy threshold is a further step. It is about whether automated research changes the pace of AI progress in aggregate, not only whether a coding agent drafts a pull request. The filing treats that step as a safety risk the company says it is still measuring, and as a reason it may delay or limit models. It points to an earlier case: it says past precautionary measures included delaying releases and limiting initial availability, “such as with Mythos through Project Glasswing,” and that future measures could affect whether, how, and when new models or capabilities ship. For a plain-language reading, start with [how agents work](/topics/how-agents-work). For the harness and task questions, see [harnesses](/topics/harnesses), [task performance](/topics/task-performance), and the [Builders](/for/builders) page. The research framing of the policy lives with the [research topic](/topics/research) and the [Research](/for/research) audience page. The public pattern of interleaving a model’s reasoning with actions is older than this filing; the site’s notes on [ReAct](/research/react-reasoning-and-acting) cover that 2022 paper. ## What happens next? Anthropic says it will release further reports as the Responsible Scaling Policy requires, and that it is considering policy updates so humans stay in control of frontier model development. Version 3.4 says the company will publish a Risk Report every 3–6 months. It also says it will publish an analysis of an individual model’s relevant risks within 30 days of determining that an internally deployed model could pose high-stakes misalignment or automated R&D risks that significantly exceed those of all models for which it has already analyzed such risks in public. The filing also says Anthropic cannot predict precisely when it will decide extra safety measures are necessary, what those measures will be, or how long they will stay in place. It says such measures could become more frequent and more costly as capabilities advance, and could slow its capability gains relative to competitors that do not adopt similar constraints. The November 2 document names no new Claude model, states no new API price, and gives no date on which the threshold will be declared met. Until Anthropic publishes the further report it says the policy calls for, the sourced claim is the one in the risk factors: estimates for future models may meet the definition soon, and no current model has been designated as past it. This filing is one item on the [daily news](/topics/news) beat. The [Claude directory listing](/directory/claude) stays a product pointer; confirm current plans and model names on Anthropic’s site. ## Frequently asked questions ### Did Anthropic say a current Claude model has crossed the threshold? The Form S-1/A says Anthropic has not designated any current model as exceeding the threshold, and that it continues to assess the underlying data. The “may imminently meet” wording is about internal estimates and expectations for future models. ### What does “dramatically accelerated progress attributable to AI” refer to? In the filing, that is Anthropic’s name for a definition in its Responsible Scaling Policy. Version 3.4 treats “dramatic acceleration” as one of two ways to meet the automated AI R&D threshold. The doubling it describes is aggregate AI capability — as much progress in one year as two years at baseline — judged plausibly due to automating research or engineering. The policy’s footnote says that is separate from doubling researchers’ productivity. ### Is this the same as Claude coding without a person in the loop? The filing separates the two. It describes Claude Code writing and running code on long tasks, with engineers directing and reviewing. It also says Anthropic does not anticipate safely using Claude to fully autonomously design and build successive frontier models without meaningful human supervision or understanding, if and when that becomes technically possible. ### What revenue figure does the filing report? Anthropic states revenue of $17.0 billion for the three months ended September 30, 2026, compared with $1.4 billion for the three months ended September 30, 2025. It states an operating loss of $567 million for the 2026 quarter. ### Where is the primary source? The amended registration statement is the [Form S-1/A on EDGAR](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm). The rule it cites is [Responsible Scaling Policy version 3.4](https://www-cdn.anthropic.com/files/4zrzovbb/website/0bacdc8440ea96e62a8766d99ebe1d4eea6d5f3a.pdf), linked from [Anthropic’s policy page](https://www.anthropic.com/responsible-scaling-policy). ## What Is an AI Agent Harness? DeepSeek Super Agent and DeepSeek Harness (2026) URL: https://www.aiagentsinfo.com/blog/what-is-an-ai-agent-harness Published: 2026-11-03. Updated: 2026-11-08 Category: guide What is DeepSeek Harness? DeepSeek’s open-source agent harness, still in developer preview. The documented start command runs a web interface on your own machine. The DeepSeek super agent note covers the router and worker prototype. An AI agent harness is the loop, tools, memory, permissions, and stop rules around a model. Updated November 8, 2026. ## What does an agent harness mean in plain language? A language model is a program that answers in words. An [agent](/glossary/agent) is that program when it can also take steps: look something up, change a file, run a check, and try again. The [harness](/glossary/harness) is the workshop around the program. It holds the tools, the notebook, the lock on the door, and the rule for when to stop. Two workshops can hand the same program a job and get different results. One workshop may allow a command to run. Another may stop and ask a person first. The program did not change. The workshop did. If you have used a chat box, you have used a model. When a product can click, edit, or buy, the software around the chat is doing extra work: it offers the tools, carries out the chosen step, and brings the result back. That extra software is the harness. The rest of this guide is the full account for adult readers, including people who build these systems. Short definitions live in the glossary: [harness engineering](/glossary/harness-engineering), [orchestration](/glossary/orchestration), and [guardrail](/glossary/guardrail). A sourced comparison of shipping harnesses, without invented scores, is [Best AI agent harness](/best/ai-agent-harnesses). ## What is an AI agent harness? On this site, a harness is the runtime around a model: the loop, the [tools](/glossary/tool-use), the memory, the permissions, and the stop conditions. The weights stay the model. The harness is the code that puts a goal in front of those weights and does something with the reply. Public write-ups in 2026 use the same split. [LangChain’s March 10, 2026 anatomy post](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) says a harness is every piece of code, configuration, and execution logic that is the model’s surroundings, and it writes the shorthand as agent equals model plus harness. The parts it lists are system prompts, tools and their descriptions, bundled infrastructure such as a filesystem, a sandbox, and a browser, orchestration, and hooks that run fixed code for jobs such as compaction, continuation, and lint checks. [Databricks’ explainer, published June 17, 2026](https://www.databricks.com/blog/ai-harness), defines an agent harness as the software infrastructure wrapped around a large language model so the model can act on tasks. It names tools, memory, a workspace, and guardrails. [OpenAI’s developer post on Codex as a platform](https://developers.openai.com/blog/codex-as-a-platform), live when this page was checked, calls the harness the surrounding execution system: the part that keeps context, calls tools, handles failures, asks for approval, and returns a result. A research definition is narrower and still useful. The [Meta-Harness paper](https://arxiv.org/abs/2603.28052), arXiv:2603.28052, first posted March 30, 2026, describes a harness as the code that determines what information to store, retrieve, and present to the model. Those sentences agree on the boundary. They leave room for argument about whether a particular file is “prompt” or “harness.” On this site the practical test is simple. If it is code or configuration around the weights, it belongs to the harness. ## How do a model, a harness, and an agent differ? Databricks’ June 17 explainer separates the three words. The model reasons and generates the next output. The harness executes actions, manages memory, runs tools, and enforces rules. The agent is the working system that combines the two. A [prompt](/glossary/prompt) sits inside that picture. It is the text the model sees on one call: standing instructions, the user goal, tool schemas, and recent observations. In an agent, the harness rebuilds that text on every turn. Prompt wording matters. It is one file in the workshop, which is why [harness engineering](/glossary/harness-engineering) is a wider job than prompt writing. Databricks’ table in the same post puts prompt engineering, context engineering, and harness engineering in that order. The last one’s artifact is the harness. The [language model](/glossary/llm) glossary entry is the right place for what a model is. This page stays on the workshop. ## What happens in one loop? Many agents repeat a small cycle. The model reads the current context and chooses a next step. The harness carries out that step. The harness puts the result back into the context. The model chooses again. The cycle ends when a stop rule fires. [Shunyu Yao and coauthors](/research/react-reasoning-and-acting) named a version of this pattern in 2022: interleave a written reasoning trace with an action, then read the observation. The paper is [ReAct](https://arxiv.org/abs/2210.03629). Databricks’ 2026 explainer and LangChain’s March post both describe production agents in those terms. The 2022 scores belong to that paper’s models and tasks. The [research note](/research/react-reasoning-and-acting) says so. The pattern is what traveled. A current product loop is more specific than the paper. Anthropic’s [agent-loop page](https://code.claude.com/docs/en/agent-sdk/agent-loop) for the Claude Agent SDK, checked November 3, 2026, describes five beats. The SDK delivers the prompt, the system prompt, the tool definitions, and the history. Claude answers with text, tool calls, or both. The SDK runs the requested tools. The cycle repeats until Claude produces a response with no tool calls. The SDK then yields that text and a result message with token usage, cost, and a session id. Hooks can intercept, change, or block a tool call before it runs. Stop rules differ by product. The loop page says the cycle continues until the model produces a response with no tool calls. The [quickstart](https://code.claude.com/docs/en/agent-sdk/quickstart) says the loop also ends when the run hits an error. OpenAI’s [running-agents guide](https://developers.openai.com/api/docs/guides/agents/running-agents) treats a max-turn limit, a guardrail exception, and a tool error as failures, and treats a human approval as a pause. A pause is meant to resume from the same saved state. A new user turn is a different event. ## Which parts does a harness usually contain? Vendors group the parts differently. The rows below are the pieces that show up, under those names, in the sources linked from this guide.
Part What it is for A public description
Tools Let the model ask for a search, a file edit, a command, or an API call. The harness runs the call and returns the result. Anthropic’s Agent SDK overview lists built-in tools for reading, writing, editing, commands, and web search. See tool use.
Sandbox Give those commands a bounded machine so a bad step stays inside it. OpenAI’s sandbox guide separates the harness, which it calls the control plane, from the sandbox, which it calls the execution plane.
Files and memory Keep notes, plans, and progress that outlast one context window. Anthropic’s November 26, 2025 note uses a progress file, a feature list, and git. Pydantic documents a memory notebook.
Orchestration Choose the next step, tool, or specialist, and who owns the reply. OpenAI’s orchestration guide. Glossary: orchestration.
Guardrails Allow, block, replace, or pause an action. OpenAI’s guardrails page and Pydantic’s guardrail docs. Glossary: guardrail.
Checks before “done” Require evidence, such as a test or a click-through, before a feature counts as finished. Anthropic’s long-running notes, linked below, use a feature list and browser checks.
Traces Record the calls, the cost, and the failures so a person can reconstruct the run. Microsoft’s Agent Framework overview includes observability on its Harness Agent. OpenAI’s SDK overview points tracing at its observability guide.
Screen clicks sit on the tools row and the sandbox row together. The guide to that loop, including [computer use](/glossary/computer-use), [browser use](/glossary/browser-use), and the [sandbox](/glossary/sandbox), is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer), checked November 5, 2026. [Context windows](/glossary/context-window) fill up. Harnesses then summarize, drop, or retrieve. LangChain’s March post calls the quality drop “context rot” and describes compaction, offloading large tool output to files, and loading skills only when needed. Anthropic’s March 24, 2026 note, cited below, separates compaction (a summary inside the same session) from a full context reset plus a handoff file. [Retrieval](/glossary/rag) is one way to fill the window. It is a memory tactic. Tool use is the step outside the text box. A harness can use both. ## What is harness engineering? [Harness engineering](/glossary/harness-engineering) is the discipline of designing that workshop on purpose. Thoughtworks’ November 4, 2026 account of the same work across an organization is [What is harness engineering at scale?](/blog/what-is-harness-engineering-at-scale-thoughtworks-radar). OpenAI’s [February 11, 2026 post](https://openai.com/index/harness-engineering/) by Ryan Lopopolo describes one internal experiment under that name. The team’s line is that humans steer and agents execute. The post says humans never directly contributed code on that project, and that Codex wrote the application, tests, CI configuration, documentation, observability, and internal tooling. The first commit to an empty repository landed in late August 2025. The post says the initial scaffold was generated by the Codex CLI using GPT-5. Five months later, the post says, the repository held on the order of a million lines, about 1,500 pull requests had been opened and merged, and a team of three engineers had driven the agents, later growing to seven. It reports an average of 3.5 pull requests per engineer per day for that earlier team of three, and says throughput rose as the team grew. It says the product had hundreds of internal users, including daily power users, and that the authors estimate the build took about one tenth of the time a hand-written version would have taken. Those figures are the company’s account of its own beta. This page does not have a separate audit. The engineering claim in that post is about the human job. Early progress was slow because the environment was underspecified. The fix the authors describe is a missing capability made legible and enforceable, rather than a request to try harder. They made the app, its logs, and its metrics readable to the agent, including a per-worktree copy of the app and a local observability stack. They say they regularly see a single Codex run work on one task for upwards of six hours. They also say a single giant instruction file failed: context is scarce, a file that marks everything as important guides nothing, and the file rots. Their replacement was a short `AGENTS.md`, about 100 lines, used as a map, with a `docs/` tree as the system of record. GPT-5 in that story is the model the post names for the August 2025 scaffold. It is a historical detail of the experiment. It is a poor label for “the model in Codex” on a later date. Databricks’ June 17 table is the clean teaching version of the same shift. Prompt engineering writes a better input. Context engineering chooses what the model sees, and when. Harness engineering designs the tools, sandboxes, loops, and guardrails. The first two live inside the third. LangChain’s March 10 post adds a research-shaped caution. It says products such as Claude Code and Codex are post-trained with the harness in the loop, so a model can get worse when the tool format changes, even if a person would find the new format easy. That is the company’s argument, tied to a Codex prompting guide it cites. Treat it as a reason to test a harness change. The test for one observable action is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). How to stand up the tasks, sandboxes, and graders is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). The short definition of the job is the [harness engineering](/glossary/harness-engineering) glossary entry. ## How is a framework different from a harness? A framework is a library. A harness is a configured run: which prompts, tools, sandbox, approvals, and stop rules are actually in force. This site’s [agent frameworks](/directory/category/agent-frameworks) category is the library shelf. A separate page copies the latest GitHub release for 19 open-source agent frameworks and SDKs, pulled 5 November 2026: [AI agent framework release, November 2026](/reports/ai-agent-framework-github-stats-november-2026). The [LangChain listing](/directory/langchain) says the framework is the open-source library, and the harness is the runtime around the model. [CrewAI](/directory/crewai) and [smolagents](/directory/smolagents) are on that shelf too. The [harness comparison tracker](/trackers/harnesses) stays empty until every cell in a matrix has its own source. The table in the next section is a narrower substitute: what each vendor’s own docs call the harness. Those rows were checked on November 3, 2026. The Pydantic row was checked again on November 4, 2026. Prices and plan limits live on the directory listings, where this site records a last-checked date, and on the vendor pages. The side-by-side list prices for the two shipping coding-agent harnesses are [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing), checked November 6, 2026. ## How do current products divide the work?
Product What the vendor’s docs say Where the loop runs
Claude Agent SDK Anthropic’s overview says the SDK gives you the same tools, agent loop, and context management that power Claude Code, including permissions, sessions, hooks, subagents, and MCP. A Python or TypeScript process you operate. The SDK runs the Claude Code binary.
Anthropic Managed Agents The same overview calls Managed Agents a hosted agent harness. Sessions sit in an Anthropic-managed cloud sandbox or a self-hosted sandbox. Anthropic’s service, or a sandbox you host. This is a different product from the Agent SDK.
OpenAI Agents SDK The sandbox guide says the harness owns the agent loop, model calls, tool routing, handoffs, approvals, tracing, recovery, and run state. The sandbox is the machine that reads files and runs commands. Your application, in the TypeScript or Python SDK. The SDK overview says the Agents API is the managed harness in OpenAI’s service.
Codex The Codex platform post says one open-source harness powers the app, the CLI, and the IDE extension. It gathers context, uses tools, applies sandbox and approval rules, and carries work forward. Model access stays separate from that open-source layer. The Codex app, CLI, or IDE, or your own app through the Codex SDK, codex exec, or app-server.
Microsoft Agent Framework The overview describes a Harness Agent for long, multi-step tasks: planning and todo tracking, context compaction, file access and memory, don’t-ask-again tool approval, and observability. Graph and function workflows are the path when the steps are fixed. Microsoft calls the framework the successor to Semantic Kernel and AutoGen, and says that if you can write a function for the task, do that instead of using an agent. Your app. The overview shows Python and .NET, and says the Go SDK is in public preview and requires Go 1.26 or later.
Google ADK 2.0 The ADK 2.0 page says the release moves from a hierarchical executor to a workflow runtime. Agents, tools, and functions are nodes in a graph. It also describes dynamic workflows and collaborative workflows with a coordinator and subagents. Your process. The page says ADK 2.0 is available for Python, TypeScript, and Go. It dates general availability to May 19, 2026 for Python, June 30, 2026 for Go, and August 21, 2026 for TypeScript.
Pydantic AI Harness The harness docs say every Pydantic AI agent already has a light harness: a typed loop, a model, tools, and structured output. The harness package adds a workspace, a plan, memory, sub-agents, context management, and durable execution for long jobs. Guardrails can allow, block, replace, retry, or defer a tool call for approval. Your process. The quick start says LocalWorkspace runs on your machine and is a sandbox only if you swap in one. The page checked November 4, 2026 names Modal, E2B, Render, Sprites, Daytona, and boxd. Its first example model string is still anthropic:claude-opus-5-5. The same section also shows a coder on gpt-6.1-sol.
Read a row as a pointer, then open the vendor page before you depend on a detail. Model names in sample code move. The [Claude](/directory/claude) and [Codex](/directory/codex) listings are the product doors on this site. DeepSeek Harness is not in the table. Its repository was checked separately on November 5, 2026, in the next section. ## What is DeepSeek Harness? DeepSeek Harness is an agent harness from DeepSeek, published as source. The general name for that surrounding software is an [agent harness](/glossary/harness). Designing one on purpose is [harness engineering](/glossary/harness-engineering). GitHub records the public repository [deepseek-ai/deepseek-harness](https://github.com/deepseek-ai/deepseek-harness) as created on August 13, 2026. The repository description is “DeepSeek Harness: Everything is a Plugin.” The [README](https://github.com/deepseek-ai/deepseek-harness/blob/master/README.md) calls it an open-source agent harness, `dsh`, developed by DeepSeek, built on an everything-is-a-plugin architecture. The same README says the project is in developer preview and that compatibility-breaking changes are expected. GitHub lists the license as MIT, and the README points at the same MIT license file. On November 5, 2026, GitHub’s latest-release endpoint, which returns the newest release that is not a pre-release, responded 404. The newest item on the releases list was a pre-release: tag `dsh-v0.2.2-rc.1`, name v0.2.2-rc.1, published November 4, 2026. ## What does “DeepSeek Harness desktop” refer to here? On this page it refers to the local run already in the README. The instructions run on your own machine. `npx @deepseek-ai/dsh web` starts a web interface there. A checkout can be installed and built from the repository. That is a different artifact from the products in the table. [Claude Code](/directory/claude-code) and [Codex](/directory/codex) are shipping products with their own apps and plan pages. Inside Claude Code, ultracode is the setting that plans a workflow for a substantive task. The November 9, 2026 reading of that setting is [What does ultracode do in Claude Code?](/blog/what-does-ultracode-do-in-claude-code). DeepSeek’s repository is the harness itself, in preview, organized as plugins. This page does not turn that into a ranking. The sourced table is [Best AI agent harness](/best/ai-agent-harnesses). The [harness comparison tracker](/trackers/harnesses) points at that page. ## What is the DeepSeek super agent? The DeepSeek super agent, in the sources below, is a second shape the same group described at the end of October 2026, with a prototype published beside it. Designing that shape is still [harness engineering](/glossary/harness-engineering). A check of one action in a run is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). The [South China Morning Post](https://www.scmp.com/tech/article/3372884/deepseek-unveils-ai-super-agent-coding-race-openai-anthropic), in a story published November 1, 2026, says the DeepSeek Harness team published a technical article the day before, a Saturday, about a system it calls a super agent. The Post says the aim is to reduce the human work of coordinating several agents and watching their progress. That story is a news report of the article. It describes the aim. It does not print a results table. The code published with the discussion is [deepseek-ai/dsh-superagent-prototype](https://github.com/deepseek-ai/dsh-superagent-prototype). The [README](https://github.com/deepseek-ai/dsh-superagent-prototype/blob/main/README.md), checked November 6, 2026, calls it dsh-superagent: a proactive coding agent for one GitHub repository, packaged as a bundle for DeepSeek Harness. The README says it is a research prototype for reference, not an official DeepSeek product. It says the published code was adapted, so it is not identical to the internal version or to the implementation described in “Exploring Super-agent Design: Towards a More Autonomous Agent System,” and that some designs in that article are later work and are not in the prototype. The README links an MIT license. [MarkTechPost’s November 1, 2026 article](https://www.marktechpost.com/2026/11/01/deepseek-dsh-superagent-router-worker-coding-agent/) describes the design as two layers. A long-running router handles interaction and dispatch. Isolated worker sessions do the work. The prototype’s [execution notes](https://github.com/deepseek-ai/dsh-superagent-prototype/blob/main/docs/003-execution.md) say the main session uses a router preset with a smaller tool set, and that each ticket gets its own worker session and directory. Those notes say worker sessions run commands without approval, as the operator’s operating-system user. MarkTechPost also prints simulation times from secondary reports of a DeepSeek test, a p95 of 15.7 minutes for the super-agent setup and 29.5 minutes for a single-session baseline, and it says the runs used synthetic messages and simulated tools. It says real coding quality was not measured. This page has not re-run that test, and it does not treat those minutes as a product score. The prototype is a harness you can read. The bill for a shipping coding agent is a separate question. [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing) compares those list prices as of November 6, 2026. An open-source workshop can still call a model on a paid plan. The plan is how those calls are billed. The harness is what the calls are allowed to do. ## Can an agent harness improve itself? A research paper can study that question. It does not, by itself, mean a product you can install already rewrites its own workshop. [Harmony Harness](https://arxiv.org/abs/2610.15150) (arXiv:2610.15150) is a framework described in a paper, “Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses.” The author list includes researchers affiliated with Google Cloud AI Research, Google DeepMind, and Carnegie Mellon University. The abstract says recent work treats the agent harness as an optimization target that can be improved from execution feedback, and that those approaches mostly optimize a single objective and evolve the harness as one block. The abstract’s own description of Harmony Harness is narrower. It formulates harness evolution as a multi-objective optimization problem. It uses decomposition-based evolutionary search to keep solutions across different regions of the Pareto frontier. Separate operators refine general-purpose harness mechanisms and benchmark-specific adaptations, and a composition step puts them together when the agent runs. On SecRepoBench and BaxBench, the authors report that it outperformed strong harness-evolution baselines, with gains exceeding 10 percent on both benchmarks. That is the authors’ measurement on those two benchmarks. It is not a claim that [Claude Code](/directory/claude-code), [Codex](/directory/codex), or DeepSeek Harness rewrites itself. The older pattern of taking a step and reading the result is the [ReAct](/research/react-reasoning-and-acting) note. Designing the surrounding system on purpose is [harness engineering](/glossary/harness-engineering). ## Why do long tasks need a stronger harness? A single loop works while the job fits in one context window and the model stops at the right time. Long jobs break that assumption. Each new session starts without the previous session’s memory, and a model can declare victory early or try to build the whole thing at once. Anthropic’s [November 26, 2025 engineering note](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents) calls the Claude Agent SDK a general-purpose agent harness and says compaction, by itself, was not enough. In the team’s experiments, a high-level prompt such as “build a clone of claude.ai” produced two failure patterns. The agent tried to do too much at once and left the next session with a half-built feature. Or a later session saw some progress and declared the job done. The note’s response has two prompts on one harness. A footnote says the system prompt, tools, and harness stayed the same. Only the initial user prompt changed. The initializer prompt asks for an `init.sh` script, a `claude-progress.txt` log, an initial git commit, and a feature list. In the claude.ai-clone example the list held over 200 features, each stored as JSON with `"passes": false`. The note says the model was less likely to overwrite JSON than Markdown, and the coding prompt told the agent to change only the passes field. Later sessions work on one feature, commit, and update the progress file. The note also says the agent marked features done after unit tests or `curl` checks that missed end-to-end bugs, until the prompt required browser testing through the Puppeteer MCP, used the way a person would click. Anthropic’s [March 24, 2026 note](https://www.anthropic.com/engineering/harness-design-long-running-apps) keeps the one-chunk handoff and adds a split between the agent that builds and the agent that judges. A planner turns a short prompt into a product spec. A generator implements one sprint at a time. An evaluator clicks through the running app with the Playwright MCP and grades the sprint against thresholds. Before the sprint, the two sides agree on a sprint contract. The note also names “context anxiety”: some models start wrapping up as they approach what they believe is the context limit. Compaction keeps a shortened history in the same session. A reset clears the window and depends on the handoff file. The note says the reset costs extra orchestration, tokens, and latency. [OpenAI’s February post](https://openai.com/index/harness-engineering/) describes a related habit from the other direction. Humans may review pull requests. The post says that over time almost all review effort moved to agent-to-agent checks, while people still set intent and looked for missing capabilities. [Autonomy](/glossary/autonomy) on this site is that permission question: how far the loop may run before a person confirms the next step. A coding agent that opens a pull request, and a system that would design the next frontier model alone, are different permission levels. The [November 3, 2026 news note](/blog/anthropic-may-soon-meet-ai-acceleration-threshold) stays on the second one, using Anthropic’s own SEC filing. ## How can the same model score differently? A [benchmark](/glossary/benchmark) measures a bundle. The bundle includes the model, the prompt, the tools, and the harness settings. OpenAI said this directly on [July 29, 2026](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/). On the ARC-AGI-3 public set, OpenAI reports that GPT-5.6 Sol scored 13.3 percent with the official harness and 38.3 percent after two changes: retained reasoning and compaction. The score is Relative Human Action Efficiency. From official gameplay logs, OpenAI estimates the average human tester at 48 percent. The post says the official harness threw away private reasoning after each action and used rolling truncation once the conversation passed 175,000 characters, so older actions disappeared. OpenAI’s reimplementation kept reasoning by passing the previous response id on the Responses API, and it replaced truncation with compaction. The post says that for GPT-5.6 Sol at the max setting, the two changes together yielded roughly three times the score and about six times fewer output tokens. The [model page for `gpt-5.6-sol`](https://developers.openai.com/api/docs/models/gpt-5.6-sol) still responded on November 4, 2026. The same day, [OpenAI’s models index](https://developers.openai.com/api/docs/models) told a new project to start with GPT-6 Astra, to choose GPT-6.1 Sol to balance intelligence and cost, or GPT-6 Luna for high-volume work. The 13.3 and 38.3 figures remain that July measurement of GPT-5.6 Sol on that public set. They are not a score for GPT-6 Astra or GPT-6.1 Sol. The mechanism is the part to keep. The official harness, OpenAI writes, was generic on purpose, so model gaps would be easier to see. A production harness is tuned to the model’s settings, including whether earlier reasoning stays in the transcript. Comparing two models on different harnesses mixes those choices together. A score on a whole benchmark still leaves the failed step unnamed. Checking one action at a time is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). Two other dated reports point the same way, with their own limits. LangChain wrote on March 10, 2026 that changing only the harness moved one of its coding agents from the top 30 to the top 5 on Terminal-Bench 2.0. That is the company’s account of a leaderboard on that date. The Meta-Harness authors, in the March 30, 2026 abstract, report three of their own results. On an online text-classification task, their search beat a context-management system they call state of the art by 7.7 points while using four times fewer context tokens. On retrieval-augmented math, one discovered harness improved accuracy by 4.7 points on average across five held-out models, on 200 IMO-level problems. On agentic coding, they say the discovered harnesses surpassed the best hand-engineered baselines they report on TerminalBench-2. Those are the authors’ measurements. The [weight-space programming note](/research/weight-space-programming) on this site cites the paper for a different reason: a harness still wraps a model whose weights someone is trying to edit. The [Builders](/for/builders) and [Research](/for/research) doors are the places those threads continue. ## What should a beginner check first? Start with the job, then the workshop. 1. Write down the stop rule. “The model stopped talking” and “the test passed” are different finishes. Anthropic’s feature list is one way to make the second kind explicit. 2. List the tools the run may call. Pydantic’s harness quick start says `LocalWorkspace` runs file tools and commands on your machine and is a sandbox only after you swap in a sandbox capability. Commands there can reach anything you can reach. 3. Decide which steps wait for a person. OpenAI’s approval flow pauses the same run and resumes it from saved state. That is a [guardrail](/glossary/guardrail) plus a human gate. 4. Keep a record outside the chat. A progress file, a git commit, or a trace lets the next session see what the last one did. 5. When you compare models, hold the harness still, or say that you did not. The July 29 ARC-AGI-3 note is the short version of why. The [task performance](/topics/task-performance) topic is where this site follows coding, computer use, and office work. The [harnesses](/topics/harnesses) topic is the shelf for this guide and the ones that will sit beside it. A shopping assistant that can pay is the same workshop pointed at a cart: [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). ## Where can you read more on this site? - Glossary: [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [orchestration](/glossary/orchestration), [guardrail](/glossary/guardrail), [agent](/glossary/agent), [tool use](/glossary/tool-use), [computer use](/glossary/computer-use), [browser use](/glossary/browser-use), [sandbox](/glossary/sandbox), [autonomy](/glossary/autonomy), [benchmark](/glossary/benchmark), [behavioral evaluation](/glossary/behavioral-evaluation) - At organization scale: [What is harness engineering at scale?](/blog/what-is-harness-engineering-at-scale-thoughtworks-radar) - Evaluation: [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents) - Research: [ReAct](/research/react-reasoning-and-acting), [weight-space programming](/research/weight-space-programming) - Directory: [agent frameworks](/directory/category/agent-frameworks), [Claude Agent SDK](/directory/claude-agent-sdk), [OpenAI Agents SDK](/directory/openai-agents-sdk), [Codex](/directory/codex), [Google ADK](/directory/google-adk), [Microsoft Agent Framework](/directory/microsoft-agent-framework), [Pydantic AI](/directory/pydantic-ai) - Shopping: [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you) - Computer use: [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer) - Coding-agent prices: [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing) - Reports: [AI agents, November 2026 to November 2027](/reports/ai-agents-next-12-months), [AI agents, November 2026 to May 2027](/reports/ai-agents-next-6-months), [agent harness benchmark, November 2026](/reports/agent-harness-benchmark-november-2026) - Also on the harness hub: [Harness comparison tracker](/trackers/harnesses), [Best AI agent frameworks](/best/ai-agent-frameworks), [SpecCompact agent context compaction](/research/speccompact-agent-context-compaction), [HarnessFlow](/research/harnessflow-mcts-harness-optimization) - Topics: [harnesses](/topics/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [shopping](/topics/shopping) ## Frequently asked questions ### What is DeepSeek Harness? It is an open-source [agent harness](/glossary/harness) from DeepSeek, in the repository deepseek-ai/deepseek-harness, which GitHub records as created on August 13, 2026. The README calls the program `dsh`, says everything is a plugin, and says the project is in developer preview. On November 5, 2026, the newest item on the releases list was the pre-release v0.2.2-rc.1, published November 4, 2026. The README’s run instructions are local: `npx @deepseek-ai/dsh web` starts a web interface on the machine where you run it. A later section covers the DeepSeek super agent note: the separate dsh-superagent prototype, a router and worker design published at the end of October 2026. [Claude Code](/directory/claude-code) and [Codex](/directory/codex) are shipping products. Their list prices are [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). DeepSeek Harness is the harness itself, still in preview. The longer account is the section above. The work of designing any harness, including this one, is [harness engineering](/glossary/harness-engineering). ### What does “DeepSeek Harness desktop” refer to here? It refers to the local run in the README, checked November 5, 2026. The instructions run on your own machine. `npx @deepseek-ai/dsh web` starts a web interface there. A checkout can be installed and built from the repository. The section above is the same note. ### What is the DeepSeek super agent? In the sources on this page, it is the name for a router and worker design the DeepSeek Harness group described at the end of October 2026. The [dsh-superagent-prototype README](https://github.com/deepseek-ai/dsh-superagent-prototype/blob/main/README.md) calls the code a research prototype, not an official DeepSeek product, and says the published code is not identical to the internal version. The [South China Morning Post](https://www.scmp.com/tech/article/3372884/deepseek-unveils-ai-super-agent-coding-race-openai-anthropic) story of November 1, 2026 reports the aim as reducing the human work of coordinating several agents. This page has not re-run any timing test. The section above is the full note. ### What is an AI agent harness? It is the software around a language model that lets the model take steps. It holds the tools, the memory, the permissions, the record of what happened, and the rule for when to stop. On this site, an [agent](/glossary/agent) is the model plus that [harness](/glossary/harness). ### How do a model, a harness, and an agent differ? The model reads context and produces the next text, including a request to use a tool. The harness runs the tool, stores the result, and decides what the model may see and do. The agent is the working combination. Databricks’ June 17, 2026 explainer and LangChain’s March 10, 2026 explainer both write this as agent equals model plus harness. ### What is harness engineering? It is the work of designing that surrounding system. [OpenAI’s February 11, 2026 post](https://openai.com/index/harness-engineering/) uses the phrase for engineers who specify intent, shape the tools and the repository, and build feedback loops so agents can do the implementation. Writing a better prompt is one part of the job. Thoughtworks’ November 4, 2026 account of the same work at organization scale is [What is harness engineering at scale?](/blog/what-is-harness-engineering-at-scale-thoughtworks-radar). ### Is an agent framework the same thing as a harness? A framework is a library for building agents. A harness is the running system: the prompts, tools, sandbox, permissions, and stop rules for a job. The [directory](/directory/category/agent-frameworks) lists the libraries. Two apps can import the same library and still run different harnesses. ### Why can the same model score differently in two harnesses? The score includes the settings around the model. On July 29, 2026, OpenAI reported that GPT-5.6 Sol scored 13.3 percent on the ARC-AGI-3 public set with the official harness and 38.3 percent after retained reasoning and compaction. The metric was Relative Human Action Efficiency. That pair is one dated measurement. ### What is a guardrail in an agent harness? A [guardrail](/glossary/guardrail) is a check that can allow, block, replace, or pause an action. OpenAI’s Agents SDK uses the word for automatic checks, and uses a separate approval step when a person must decide. Pydantic’s harness docs put a callable on the prompt, the tool call, and the final output. # Research ## Garry Tan Harness Evals: $0.17 to $0.44 per Solved Task (Nov 2026) URL: https://www.aiagentsinfo.com/research/garry-tan-harness-evals-november-2026 Published: 2026-11-09. Updated: 2026-11-09 Seven harness cells on November 5, 2026 scored 71.3% to 78.8% on 20 tasks, and all six preregistered pass-rate tests came back can't tell apart. OpenClaw on Opus 5.5 cost $0.4435 per solved task; Claude Code cost $0.2170. Garry Tan's harness evals ask which agent harness to run once the model is held still. The [matrix report](https://github.com/garrytan/g-harness-evals/blob/1c26ae409de0c90c32f5becd33a2aa7c311e7ef7/docs/benchmarks/2026-11-04-matrix-mu/report.md) in [garrytan/g-harness-evals](https://github.com/garrytan/g-harness-evals) answers that, for 20 public tasks run on November 5, 2026, the pass rates cannot be told apart. The costs can. OpenClaw on Opus 5.5 cost $0.4435 per solved task. Claude Code, on the same model and the same tasks, cost $0.2170. AI Agents Info read the README, the scoreboard, and that matrix report on November 9, 2026, at main commit `1c26ae4`. The GitHub API that morning listed 11 stars, 2 forks, and the MIT license. The numbers below are copied from those files. A pass rate in a different paper is not a row in these tables. ## What did the Garry Tan harness evals measure? They measured pass rate, cost per solved task, and time, with the model and the 20 tasks held fixed, and only the harness swapped. The matrix report says the run is 7 cells and 560 trials: 20 public v0.3 multi-user dev tasks, 4 repetitions each. It ran from 2026-11-05 07:02Z to 12:06Z, one Ubicloud standard-16 VM per cell, trials one at a time. Every cell was a plain harness. The report says no trial produced a gbrain event, in 560 of 560 event logs. Four cells used Opus 5.5, which the report names as `anthropic:claude-opus-5-5`. Three used GPT-6.1 Sol, `openai:gpt-6.1-sol`. Claude Code and Codex are each model's own harness. The report says they cannot run the same model, so they are never compared with each other. | Harness | Version | Model | | --- | --- | --- | | Claude Code | 2.1.312 | Opus 5.5 | | OpenClaw | 2026.10.6 | Opus 5.5 | | Hermes Agent | v2026.9.24 | Opus 5.5 | | QM (Pi loop) | main@b6786d67 | Opus 5.5 | | Codex | 0.165.0 | GPT-6.1 Sol | | OpenClaw | 2026.10.6 | GPT-6.1 Sol | | Hermes Agent | v2026.9.24 | GPT-6.1 Sol | Pass rate in the report is a task mean. Each task contributes its own share of passing scored tries, and those shares are averaged. It is not the pooled fraction of passes over trials. The 95% ranges are task-clustered bootstrap intervals, 10,000 draws, seed 7, so a few lucky tasks do not narrow the range. Cost per solved task is the cell's whole spend at list price through the project's metering proxy, divided by scored passes. Spend includes trials the scorer then excluded. Time is the median wall-clock time of one trial. The [harness](/glossary/harness), in the glossary on this site, is that bundle of loop, tools, memory, and permissions. The [benchmark](/glossary/benchmark) entry is the frozen task set plus the scoring rule. This matrix is one benchmark. It is not the only one, and the last section says what it is not. ## Can the pass rates tell the harnesses apart? No. All six preregistered tests came back "can't tell apart," and the report prints Holm p = 1 for every one of them. The report's own summary is the sentence to quote: on these 20 tasks, the run cannot tell any two harnesses apart on pass rate, on either model. The biggest gaps it names are on Sol, where the model is shared. OpenClaw is 3.9 points over Codex, interval [+0.0, +11.8]. Hermes is 3.8 points over Codex, interval [+0.0, +10.0]. Both intervals include zero. Each primary pair differed on 1 to 4 tasks. The reason the tests cannot see a smaller gap is the task set. Eleven tasks passed on every repetition in every cell. Three tasks never passed in any cell. That is 14 of 20 tasks with the same score everywhere. The scoreboard calls the set saturated: half or more of the tasks scored the same in every cell, so the tasks, not the harness, decide most of the score. The minimum detectable difference was 7.7 to 14.2 points at alpha 0.05. The report says the run had less than an 80% chance to detect a gap under that range. "Can't tell apart" is the glossary line in the scoreboard. It is the preregistered test, adjusted for the number of comparisons, finding no difference. The scoreboard says it does not mean equal. The report's verdict on why the result is null is FIX-METHOD. The tasks hit a ceiling. The report says to raise that ceiling before this matrix runs again. QM is not one of the six tests. The report says QM ran on the default Docker bridge rather than the internal network, so its comparisons are labelled untested. ## What did each harness cost per solved task? OpenClaw on Opus cost $0.4435 per solved task. The other Opus point estimates were $0.1853, $0.2170, and $0.2434. On Sol the three point estimates were $0.1653, $0.1869, and $0.2222. The table copies the matrix report's per-cell results. Pass rate is the task mean, printed next to the raw passes so the two are not divided into each other. OpenClaw on Opus is the row where they disagree: 60 divided by 78 is 76.9%, and the task mean is 76.3%. | Harness | Model | Passes / scored | Pass rate | Cost per solved task | Spend | Median time | | --- | --- | --- | --- | --- | --- | --- | | Claude Code | Opus 5.5 | 61 / 80 | 76.3% [57.5%, 92.5%] | $0.2170 [$0.1644, $0.3042] | $13.2346 | 40.1 s | | OpenClaw | Opus 5.5 | 60 / 78 | 76.3% [58.8%, 91.3%] | $0.4435 [$0.3280, $0.6282] | $26.6127 | 99.4 s | | Hermes Agent | Opus 5.5 | 63 / 80 | 78.8% [60.0%, 95.0%] | $0.2434 [$0.1764, $0.3576] | $15.3320 | 64.6 s | | QM (Pi loop) | Opus 5.5 | 58 / 80 | 72.5% [53.8%, 88.8%] | $0.1853 [$0.1259, $0.2807] | $10.7478 | 40.1 s | | Codex | GPT-6.1 Sol | 57 / 80 | 71.3% [51.2%, 90.0%] | $0.1653 [$0.1154, $0.2575] | $9.4236 | 30.1 s | | OpenClaw | GPT-6.1 Sol | 56 / 76 | 73.7% [53.9%, 90.8%] | $0.2222 (interval basis $0.1964 [$0.1333, $0.2954]) | $12.4437 | 83.6 s | | Hermes Agent | GPT-6.1 Sol | 60 / 80 | 75.0% [56.3%, 91.3%] | $0.1869 [$0.1350, $0.2760] | $11.2113 | 81.5 s | The seven spend figures sum to $99.0057. The report's prose rounds that to $99.01 of a $156.27 cap. VM list price, from the same table, is $6.3601, and the report says that cost sits beside the cap rather than inside it. Passes sum to 415. Scored trials sum to 554. The 6 trials in the gap from 560 recorded to 554 scored are the cap stops below. The README rounds cost per solved task to $0.19, $0.22, $0.24, and $0.44 on Opus, and to $0.17, $0.19, and $0.22 on Sol, and it ranks by whether those 95% ranges overlap. On that rule every row is rank 1, with a mark on OpenClaw's Opus row. OpenClaw's range, [$0.33, $0.63] in the README's rounding, overlaps Hermes's [$0.18, $0.36] and clears Claude Code's [$0.16, $0.30] and QM's [$0.13, $0.28]. The README says the cost differences are descriptive, with no test. The test is the second run, further down. Two cells did not finish the plan. A cap stop, in the report, is the metering proxy refusing a request because the worst-case reservation would pass the shard's lease. The trial is recorded as `model_error` and is not retried. | Cell | Trials cut off | Scored | What was refused | | --- | --- | --- | --- | | OpenClaw, Opus 5.5 | sy-03 r3 and r4 | 60 / 78 | Reservations of $3.50 and $3.37, each over a $29.76 lease, after $25.91 had already been committed | | OpenClaw, GPT-6.1 Sol | sy-03, all 4 reps | 56 / 76 | Reservations of $1.44 to $1.69, each over a $13.88 lease, after $12.03 had been committed | The task id in the report is `sy-03-marrow-fieldworks-ticket-numbers`. On Sol, that task drops out of the comparisons, so those pairs use 19 tasks. The report says OpenClaw on Opus averaged $0.333 per trial against a lease formula that assumed $0.270, and OpenClaw on Sol averaged $0.156 against $0.125. It also records two timed-out Sol requests charged at their full reservations, $2.8755 together. One of those, $1.4432 on `cn-01-tamsin-cold-chain-site` r4, was logged after the trial's meter closed. That is why the Sol cost interval is built around $0.1964 while the headline, which keeps the charge in the ledger, is $0.2222. ## Where does Hermes Agent land? Hermes Agent has the highest printed pass rate on Opus, 78.8%, and the tests still cannot tell it apart from Claude Code or OpenClaw. On cost, the second run can tell it apart from Codex and cannot tell it apart from Claude Code. The pin is Hermes Agent v2026.9.24. On Opus the cell is 63 passes out of 80, $0.2434 per solved task, $15.3320 spent, median 64.6 seconds. On Sol it is 60 of 80, $0.1869, $11.2113, median 81.5 seconds. The README's rank rule, overlapping cost ranges, puts both of those rows in rank 1 with the other cells. The same README says Hermes sends more per request on Sol: 423 kB more tool schemas per trial than the comparison it is drawing, and 16.7 requests per trial against Codex's 11.6. It gives that as the reason Hermes costs $0.023 more per trial than Codex in the separate cost run, interval [−$0.030, −$0.016] on the Codex-minus-Hermes contrast. The scoreboard's waterfall table is the place those calls are marked as tested. On Opus, Hermes at $0.190 per trial "can't tell apart" from Claude Code at $0.180. On Sol, Hermes at $0.141 "costs more per trial than Codex" at $0.117. The matrix report also notes a side channel that did not become a model call. Hermes probes the provider for its model list. The proxy refused 2,640 of those probes on Opus and 1,540 on Sol. The report says no model request from Hermes was refused, and the refusals cost nothing. Hermes on this site is already a row in other people's experiments. The [HarnessFlow](/research/harnessflow-mcts-harness-optimization) note copies that paper's runs of Hermes v2026.9.7, an earlier pin, on different benchmarks. The [SpecCompact](/research/speccompact-agent-context-compaction) note is about compaction inside a Hermes-style summary. Neither table is this matrix. The repository discloses that Y Combinator, which Garry Tan runs, is an investor in Hermes. ## Why did OpenClaw cost about twice as much on Opus? OpenClaw's Opus ledger was $26.6127 and Claude Code's was $13.2346. Dividing those two totals gives 2.01. The report's cost per solved task is $0.4435 against $0.2170, which is 2.04. The README's account of where the money went: about 81% of OpenClaw's extra cost over Claude Code on Opus is cache writes, because OpenClaw re-sends a much bigger system prompt and bigger tool results. On Sol, the same paragraph says most of the gap to Codex is output tokens, 282,492 against 32,226, and that $0.036 per trial of the gap is one timed-out stream charged at its full reservation. Those token counts are the README's. This page did not re-sum the trial logs. The matrix itself marks the Opus cost gap as descriptive. The waterfall run is the preregistered cost test. There, OpenClaw on Opus cost $0.345 per trial against Claude Code's $0.180, and the scoreboard's call is "costs more per trial than Claude Code." The README prints the paired contrast as $0.165 less per trial for Claude Code, interval [−$0.206, −$0.128]. The difference of the two printed per-trial averages is $0.165. Median time on the matrix was 99.4 seconds for OpenClaw on Opus and 40.1 seconds for Claude Code. On the waterfall run those medians were 90.3 and 34.1 seconds, and the scoreboard says wall time there is confounded with the machine. OpenClaw's Sol headline, $0.2222, is not twice Codex's $0.1653. The ratio of those two point estimates is 1.34. The waterfall call on that pair is still "costs more per trial," $0.225 against $0.117, and the README calls the paired gap $0.107, interval [−$0.270, −$0.002], barely clear of zero. The repository discloses that OpenAI is now the key benefactor of OpenClaw, and that OpenClaw's creator, Peter Steinberger, works for OpenAI. It also discloses that OpenAI began through YC Research. The cells are still the cells in the table. ## What did the second run say about cost? It tested cost per trial. On Opus, OpenClaw cost more than Claude Code, Claude Code and Hermes could not be told apart, and Claude Code cost more than QM. On Sol, Codex cost less than OpenClaw and less than Hermes. The scoreboard names the run waterfall-frontier-2026-11. Same 20 tasks, two tries each, the same pins. Each cell ran alone on its own VM. The calls below are the scoreboard's, and they are cost contrasts. They are not pass-rate tests. Three Opus cells each printed 80.0%, which is 32 of 40. That repeated 80% is the saturation story again, on a smaller sample. | Harness | Model | Passes / scored | Pass rate | Cost per trial | Median time | The run's cost call | | --- | --- | --- | --- | --- | --- | --- | | Claude Code | Opus 5.5 | 32 / 40 | 80.0% [62.5%, 95.0%] | $0.180 | 34.1 s | Cheaper than OpenClaw; can't tell apart from Hermes; costs more than QM | | OpenClaw | Opus 5.5 | 32 / 40 | 80.0% [65.0%, 92.5%] | $0.345 | 90.3 s | Costs more per trial than Claude Code | | Hermes Agent | Opus 5.5 | 32 / 40 | 80.0% [62.5%, 95.0%] | $0.190 | 63.5 s | Can't tell apart from Claude Code | | QM (Pi loop) | Opus 5.5 | 29 / 40 | 72.5% [52.5%, 90.0%] | $0.138 | 36.7 s | Cheaper per trial than Claude Code | | Codex | GPT-6.1 Sol | 28 / 40 | 70.0% [50.0%, 90.0%] | $0.117 | 22.9 s | Cheaper per trial than OpenClaw and Hermes | | OpenClaw | GPT-6.1 Sol | 31 / 40 | 77.5% [60.0%, 92.5%] | $0.225 | 76.6 s | Costs more per trial than Codex | | Hermes Agent | GPT-6.1 Sol | 30 / 40 | 75.0% [57.5%, 92.5%] | $0.141 | 66.3 s | Costs more per trial than Codex | QM's "cheaper" call carries the network caveat. The scoreboard says QM ran on the default Docker bridge while the other cells ran on the internal network. The README's paired figure for the Claude Code side of that contrast is $0.041 more per trial than QM, interval [+$0.020, +$0.061]. The difference of the printed averages is $0.042. This page quotes the interval from the README and the averages from the scoreboard, and does not treat $0.041 and $0.042 as two measurements. Two further caveats are on the scoreboard and not in the matrix table. On Sol, the proxy billed every uncached prompt token at the cache-write rate, above OpenAI's input list price, because OpenAI caches prompts on its own. And OpenClaw on Sol in this run includes one stream that timed out before its final usage, charged at the full reservation. ## What should a ranking use, and what should it leave alone? Use the tested cost calls, and leave the pass rates unranked. The sentence the files support: on this public 20-task set, pass rate did not separate the harnesses, and a second run found OpenClaw more expensive per trial than Claude Code on Opus 5.5 and more expensive than Codex on GPT-6.1 Sol. Hermes Agent's Opus cost per trial could not be separated from Claude Code's. Codex was cheaper per trial than Hermes on Sol. QM's lower Opus point estimate was not tested on a shared network. The sentence the files do not support: Hermes won, or Claude Code beat OpenClaw at the task, or 78.8% is better than 71.3% in a way the experiment could see. The intervals on the pass rates are 32 to 39 points wide. The report says so. A reader who wants a harness for a reason other than this task set has other pages, and they answer different questions. They are collected on the [agent harnesses hub](/topics/harnesses). [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness) is the method: one visible action, then the whole job. [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents) is how to stand up the tasks and graders. [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) is the definition. [Best AI agent harness](/best/ai-agent-harnesses) is licence, list price, and platform from official pages, last verified November 2026. It does not use these pass rates. The [November harness benchmark report](/reports/agent-harness-benchmark-november-2026) copies official SWE-bench and Terminal-Bench numbers. [Terminal-Bench 2.1](/research/terminal-bench-2-1-benchmark-repairs) is a repair of 28 of 89 tasks in a different suite. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) searches for a harness and scores it against pinned Claude Code, Codex, OpenHands, and Hermes on that paper's benchmarks. Claude Code's permission file is a different November 9 page. [Don't ask again](/blog/claude-code-dont-ask-again) is about what a saved approval writes to disk. It is not a row in this matrix. The product pages are [Claude Code](/directory/claude-code) and [Codex](/directory/codex). The disclosures in the README, which this note is not in a position to audit, are these. Garry Tan runs the repository and the evaluation. gstack and gbrain are his projects, and they are not in the seven cells. QM is a project of Y Combinator, which he runs. Y Combinator is an investor in Hermes. OpenAI began through YC Research. OpenAI is now the key benefactor of OpenClaw, and OpenClaw's creator works for OpenAI. The report says the receipts are in the tree so the trials can be regraded. This note did not regrade them. It copied the tables the repository had already published. ## How Many GitHub Stars Does OpenClaw Have? 394,222 on November 9, 2026 URL: https://www.aiagentsinfo.com/research/how-many-github-stars-does-openclaw-have Published: 2026-11-09. Updated: 2026-11-09 394,222 stars on openclaw/openclaw at 07:41 UTC on November 9, 2026. Hermes Agent had 261,856. Claude Code had 155,043, and its LICENSE.md reserves all rights. OpenClaw had **394,222** GitHub stars at the end of an API pass at 07:41:51 UTC on November 9, 2026. Hermes Agent had **261,856**. Claude Code had **155,043**, and the license file in that repository reserves all rights. AI Agents Info pulled eleven repositories in that pass. Ten are projects whose own text calls them an agent a person runs. The eleventh, LangChain, is the library that led this site’s [November 5 framework snapshot](/reports/ai-agent-framework-github-stats-november-2026) at 149,047 stars. On this pull LangChain had 149,241. The framework page is a release table. It earns the impression for “agent framework” stars and does not answer how many stars OpenClaw has. This note answers that, and it prints the clock. ## How many GitHub stars does OpenClaw have? 394,222, in the repository response for openclaw/openclaw when the pass finished at 07:41:51 UTC. The same object listed 82,839 forks, `open_issues_count` 8,006, the SPDX id MIT, TypeScript as the primary language, and `created_at` 2025-11-24T10:16:47Z. That timestamp is 349 days and about 21 hours before the pass finished. The homepage field was `https://openclaw.ai`. The description, with the trailing emoji removed, was “The AI that really does things. Any OS. Any Platform. The lobster way.” The latest release was tag `v2026.11.2`, name “openclaw 2026.11.2,” published 2026-11-08T19:31:04Z. `pushed_at` was 2026-11-09T07:41:03Z, forty-eight seconds before the pass finished, so the branch was moving while the count was being taken. The star field is the field from that response. It is not a promise about the next minute. The README on `main`, fetched the same morning, says OpenClaw is an open-source AI assistant that runs on your own computer and meets you in the channels you already use. It names Discord, iMessage, Slack, Teams, Telegram, WhatsApp, and others, plus native apps. It says state, memory, and credentials live on your hardware, and that models and agent harnesses are plugins. A [harness](/glossary/harness), on this site, is the loop, tools, memory, permissions, and stop rules around a model. The README’s claim that OpenClaw has no paid tier is the README’s claim. This note did not audit a billing page. | Repository | Stars | Forks | open_issues_count | Licence field | Created | Latest release | Published | Last push | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | openclaw/openclaw | 394,222 | 82,839 | 8,006 | MIT | Nov 24, 2025 | v2026.11.2 | Nov 8, 2026 | Nov 9, 2026 | | NousResearch/hermes-agent | 261,856 | 58,267 | 53,817 | MIT | Jul 22, 2025 | v0.23.0 | Nov 5, 2026 | Nov 9, 2026 | | anthropics/claude-code | 155,043 | 25,345 | 16,907 | No SPDX id | Feb 22, 2025 | v2.1.319 | Nov 6, 2026 | Nov 6, 2026 | | langchain-ai/langchain | 149,241 | 25,099 | 729 | MIT | Oct 17, 2022 | langchain-core==1.7.6 | Nov 5, 2026 | Nov 9, 2026 | | openai/codex | 134,025 | 21,231 | 26,901 | Apache-2.0 | Apr 13, 2025 | rust-v0.167.0 | Nov 6, 2026 | Nov 9, 2026 | | browser-use/browser-use | 120,380 | 13,448 | 626 | MIT | Oct 31, 2024 | 0.13.14 | Oct 26, 2026 | Nov 9, 2026 | | google-gemini/gemini-cli | 108,797 | 14,932 | 737 | Apache-2.0 | Apr 17, 2025 | v0.67.0 | Nov 4, 2026 | Nov 9, 2026 | | OpenHands/OpenHands | 92,999 | 12,425 | 1,172 | MIT | Mar 13, 2024 | v1.37.0 | Nov 6, 2026 | Nov 9, 2026 | | cline/cline | 71,479 | 7,897 | 1,752 | Apache-2.0 | Jul 6, 2024 | v4.1.26 | Oct 26, 2026 | Nov 9, 2026 | | block/goose | 56,382 | 6,569 | 570 | Apache-2.0 | Aug 23, 2024 | v1.57.0 | Oct 28, 2026 | Nov 9, 2026 | | Aider-AI/aider | 50,028 | 5,145 | 1,959 | Apache-2.0 | May 9, 2023 | v0.86.0 | Aug 9, 2025 | May 22, 2026 | The eleven star fields sum to 1,594,452. Without LangChain, the ten agent repositories sum to 1,445,211. Without LangChain and without Claude Code, the nine repositories that returned an SPDX id sum to 1,290,168. Those sums are arithmetic on one pass. They are not a market size. ## How many stars does Hermes Agent have, and what is the gap? 261,856. OpenClaw’s lead on that number is 132,366 stars. Hermes Agent’s repository object listed 58,267 forks, `open_issues_count` 53,817, the SPDX id MIT, Python, and `created_at` 2025-07-22T22:22:28Z. That timestamp is 474 days and about 9 hours before the pass, so the repository record is older than OpenClaw’s and the star field is smaller. The description was “The agent that grows with you.” The homepage was `https://hermes-agent.nousresearch.com`. The latest release was tag `v0.23.0`, name “Hermes Agent v0.23.0 — The Hearth Release,” published 2026-11-05T21:32:33Z. `pushed_at` was 2026-11-09T07:40:08Z. The README calls it the self-improving AI agent built by Nous Research, and says it runs from Telegram, Discord, Slack, WhatsApp, Signal, and a terminal, through one gateway process. The cost of running it is not in the star field. A separate November 9 note, [Garry Tan’s harness evals](/research/garry-tan-harness-evals-november-2026), put a pinned Hermes Agent and a pinned OpenClaw on the same public tasks. On Opus 5.5 in that matrix, OpenClaw cost $0.4435 per solved task and Hermes cost $0.2434. Those are that report’s figures. They are not restated here as a new measurement, and they used older pins than the release tags in the table above. OpenClaw’s pin in that matrix was 2026.10.6. The tag on this pull was v2026.11.2. ## Is Claude Code open source? The repository is public. The license file does not grant an open-source license. `anthropics/claude-code` returned 155,043 stars, 25,345 forks, and `open_issues_count` 16,907. The license object was null. A request to the repository license route returned 404. `LICENSE.md` on `main` is two lines. It says “© Anthropic PBC. All rights reserved.” The second sentence says use is subject to Anthropic’s [commercial terms](https://www.anthropic.com/legal/commercial-terms). This note did not parse those terms. It records what the file says. The description field calls Claude Code an agentic coding tool that lives in the terminal. The homepage was `https://code.claude.com/docs/en/overview`. `created_at` was 2025-02-22T17:41:21Z. The latest release was `v2.1.319`, published 2026-11-06T17:04:01Z, and `pushed_at` was the same second. The product page on this site is [Claude Code](/directory/claude-code). A star rank that files this repository under “open source” is filing it by the fact that the code can be read, not by the license GitHub returned. OpenClaw’s lead over Claude Code is 239,179 stars. That gap is a pair of counts. It is not a verdict on which coding agent to install. Price and limits for Claude Code and Codex are on [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing) and on the [November 9 coding-agent comparison](/best/ai-coding-agents). The other SPDX ids in the table were MIT or Apache-2.0. Codex’s description was “Lightweight coding agent that runs in your terminal,” license Apache-2.0, 134,025 stars, latest tag `rust-v0.167.0` with release name “0.167.0,” published November 6, 2026. Gemini CLI’s description was “An open-source AI agent that brings the power of Gemini directly into your terminal,” Apache-2.0, 108,797 stars, tag `v0.67.0`, published November 4. OpenHands, Cline, goose, Aider, and Browser Use returned MIT or Apache-2.0 as in the table. Browser Use’s description calls it an open-source Python library for AI browser agents, plus a hosted agent API. The glossary entry is [browser use](/glossary/browser-use). The 120,380 stars are on a library-and-service repository, which is why the row is labeled by its description and not folded into OpenClaw’s “assistant on your own computer.” ## Where does LangChain’s count sit now? At 149,241 stars, 194 above the November 5 snapshot, and 244,981 below OpenClaw on this pull. The [framework release report](/reports/ai-agent-framework-github-stats-november-2026) copied 19 libraries and SDKs on November 5, 2026, and LangChain led them at 149,047. That page is the one in this site’s impressions for framework GitHub stats. Its title is a release table. A person who wanted the largest star count on an agent they can run does not get OpenClaw from it, because OpenClaw was not in the 19. 394,222 divided by 149,241 is 2.64, to two decimal places. The cleaner pair is the subtraction: 244,981 more stars on OpenClaw than on LangChain, same pass. LangChain’s description on this pull was “The agent engineering platform.” The latest tag was `langchain-core==1.7.6`, published 2026-11-05T00:22:12Z. In a monorepo, `/releases/latest` can name one package. This tag names langchain-core. The [November 9 framework comparison](/best/ai-agent-frameworks) reads licenses and release tags for those 19 again, including this repository. It does not include OpenClaw or Hermes Agent. The directory listing is [LangChain](/directory/langchain). Five of the coding-agent rows were already counted on November 8 for the [30-day coding-agent brief](/reports/ai-coding-agents-next-30-days). That brief’s question is price and the dates official pages have set through December 8. The star fields moved anyway: | Repository | Stars on Nov 8 | Stars on this pull | Change | | --- | --- | --- | --- | | openai/codex | 133,622 | 134,025 | +403 | | OpenHands/OpenHands | 92,927 | 92,999 | +72 | | cline/cline | 71,457 | 71,479 | +22 | | google-gemini/gemini-cli | 108,778 | 108,797 | +19 | | Aider-AI/aider | 50,019 | 50,028 | +9 | The November 8 brief recorded no release tag for Aider. This pull’s `/releases/latest` call returned `v0.86.0`, published 2025-08-09T17:42:19Z. Aider’s `pushed_at` was still 2026-05-22T14:02:20Z, the same push time the brief recorded. The tag and the empty earlier call are both part of the record. Goose was not in that brief. On this pull `block/goose` had 56,382 stars, Apache-2.0, and release `v1.57.0`, published October 28, 2026. Its description calls it “an open source, extensible AI agent that goes beyond code suggestions.” ## What does the open-issue number count? It counts open issues and open pull requests together. Hermes Agent’s 53,817 is mostly pull requests. A search later the same morning asked for open issues and open pull requests separately. The repository field and the search totals are different requests. For two of the three, they matched. | Repository | open_issues_count | Open issues | Open pull requests | Sum of the search | | --- | --- | --- | --- | --- | | openclaw/openclaw | 8,006 | 4,819 | 3,187 | 8,006 | | NousResearch/hermes-agent | 53,817 | 18,965 | 34,850 | 53,815 | | anthropics/claude-code | 16,907 | 16,266 | 641 | 16,907 | 4,819 plus 3,187 is 8,006. 16,266 plus 641 is 16,907. 18,965 plus 34,850 is 53,815, which is two under 53,817. The miss of two is left as a miss. It is small next to the split that matters: about two thirds of Hermes Agent’s headline issue count, on this search, was pull requests. Codex’s `open_issues_count` of 26,901 was not split. Read it as the combined field. OpenClaw’s combined field, 8,006, is the one that matched the search exactly. 4,819 open issues on a repository with 394,222 stars is a different shape from Hermes Agent’s 18,965 open issues and 34,850 open pull requests on 261,856 stars. Neither shape is a defect rate. Issues can be questions, and pull requests can be one-line typos. ## What should a star table be used for? Use it to see which repository accounts have bookmarked, on a dated pull, and read the license field before calling the repository open source. The sentence the API supports: at 07:41 UTC on November 9, 2026, OpenClaw had 394,222 stars, Hermes Agent 261,856, Claude Code 155,043, and LangChain 149,241, and Claude Code’s `LICENSE.md` reserved all rights while OpenClaw and Hermes Agent returned MIT. The sentence the API does not support: OpenClaw is the most-starred agent on GitHub, or the one a team should install, or a better harness than Claude Code. The set stops at eleven repositories. Leaderboards that add other names are answering a wider list, and this note does not import their numbers. Quality, for a harness, is a test of one action and then of the whole job. That method is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). The pages that already collect harness notes are on the [agent harnesses hub](/topics/harnesses). Maintenance is a separate fact from stars, and it can run the other way. The same API pass that produced this table also read `microsoft/autogen` at 61,753 stars with `archived` false, while that repository’s README says Microsoft is not providing fixes. The write-up is [Is AutoGen deprecated?](/blog/is-autogen-deprecated). AutoGen was inside the November 5 library set. It is not one of the eleven rows above. ## How Many Repos Use AGENTS.md? 6,006 of 22,854 on October 16, 2026 URL: https://www.aiagentsinfo.com/research/how-many-repos-use-agents-md Published: 2026-11-09. Updated: 2026-11-09 Paper: What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md Paper URL: https://arxiv.org/abs/2610.31715 6,006 of 22,854 popular, active GitHub repositories had a root AGENTS.md or CLAUDE.md on October 16, 2026. That is 26.3%. The AGENTS.md homepage, read November 9, still says over 60,000 projects. On October 16, 2026, **6,006** of **22,854** popular, active GitHub repositories had a root `AGENTS.md`, a root `CLAUDE.md`, or both. That is 26.3%. Pawel Zarzycki’s census is arXiv:2610.31715, version 2, revised October 26, 2026. The pages that rank for the filename repeat a round number from the format’s homepage. The homepage, fetched November 9, 2026, still says the format is used by over 60,000 open-source projects. The census says how many repositories cleared a star floor, a recency floor, and a language list, and then had one of two root files. Both sentences can be true. They are not the same count. ## What is AGENTS.md? A plain Markdown file that coding agents are expected to read before they change a repository. The homepage calls it a README for agents. The page fetched November 9 opens with “A simple, open format for guiding coding agents, used by over 60k open-source projects.” It says a README is for humans and that AGENTS.md holds build steps, tests, and conventions that would clutter that README. The sample on the page has setup commands and a code-style section. The compatible-agent list on that page includes Codex, Cursor, Copilot, Gemini CLI, Aider, Devin, Zed, and others. This note did not test whether each named tool loads the file. The spec repository `agentsmd/agents.md` returned 25,408 stars, 1,949 forks, the SPDX id MIT, and `pushed_at` 2026-09-10T18:46:16Z. The HTTP Date on that response was 13:39:04Z on November 9, 2026. Stars on the spec repository are not the adoption count. The paper cites the homepage as accessed on the census day, October 16. A [harness](/glossary/harness) is the software around the model: the loop, the tools, the permissions, the stop rules. An instruction file is one input to that loop. [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) describes a short AGENTS.md used as a map. Thoughtworks’ write-up of the same pattern is [harness engineering at scale](/blog/what-is-harness-engineering-at-scale-thoughtworks-radar). ## How many popular repositories had a root file on October 16? 6,006 of 22,854. Adoption is higher in the repositories with the most stars. The search that built the frame asked for public repositories with at least 1,000 stars, a push in the previous 90 days, that were not forks and not archived, in thirteen languages. GitHub returned 22,854. A repository counts if the default branch has a root `AGENTS.md` or a root `CLAUDE.md` in that exact case. | Stars | Repositories | AGENTS.md | CLAUDE.md | Either | | --- | --- | --- | --- | --- | | 100,000 or more | 89 | 52.8% | 31.5% | 59.6% | | 10,000–100,000 | 3,059 | 32.3% | 21.5% | 37.0% | | 5,000–10,000 | 3,087 | 26.1% | 17.9% | 31.2% | | 1,000–5,000 | 16,619 | 18.7% | 13.2% | 23.2% | 89 + 3,059 + 3,087 + 16,619 = 22,854. The bottom tier is 23.2%. The top tier is 59.6%, and it contains 89 repositories. The paper’s Figure 1A is this table. Of the 6,006 adopters, 2,566 have only `AGENTS.md`, 1,052 have only `CLAUDE.md`, and 2,388 have both. 2,566 + 1,052 + 2,388 = 6,006. Among the 2,388, `CLAUDE.md` defers to `AGENTS.md` through a link, an import, or a pointer in 1,540 repositories. The reverse pointer appears in 182. The paper says 598 keep two files that each have content of their own. 1,540 + 182 + 598 = 2,320, which is 68 short of 2,388. The paper’s sentence does not account for those 68 in the three buckets it names. This note leaves the gap as the paper left it. New adopters moved from starting with `CLAUDE.md` to starting with `AGENTS.md`. The paper says 73.7% of new adopters from January through August 2025 started with `CLAUDE.md`, and that the `CLAUDE.md`-first share since May 2026 is 17.1%. It dates Claude Code’s support for `AGENTS.md` to September 18, 2026, from changelog version 2.1.277, and draws that date as a dashed line on the monthly chart. This note did not re-open the changelog. The census day is four weeks after the date the citation uses. ## What else did the census count, besides the two root names? 6,652 repositories, 29.1%, had at least one of the instruction paths the paper listed. 646 had one of the other paths and no root `AGENTS.md` or `CLAUDE.md`. | Path | Repositories | Share of 22,854 | Also has root AGENTS.md | | --- | --- | --- | --- | | /AGENTS.md | 4,954 | 21.7% | — | | /CLAUDE.md | 3,440 | 15.1% | 69.4% | | .github/copilot-instructions.md | 1,024 | 4.5% | 52.3% | | .cursor/rules/ | 347 | 1.5% | 64.6% | | .github/instructions/ | 273 | 1.2% | 59.0% | | /GEMINI.md | 179 | 0.8% | 83.2% | | .claude/CLAUDE.md | 150 | 0.7% | 56.7% | | /.cursorrules | 94 | 0.4% | 66.0% | | Root name in another case | 47 | 0.2% | 8.5% | | /.windsurfrules | 20 | 0.1% | 95.0% | | Any of these | 6,652 | 29.1% | — | 4,954 equals 2,566 + 2,388, the only-AGENTS.md repositories plus the both-file repositories. 3,440 equals 1,052 + 2,388. The root-file census and this wider list meet. 6,652 minus 6,006 is 646, the paper’s count of repositories that use another format and no root file. Copilot’s instruction path is the largest of those other formats, at 1,024 repositories, 4.5%. Cursor’s `.cursor/rules/` is 347. A case-variant filename such as `agents.md` at the root is 47 repositories, and only 8.5% of those also have the exact-case `AGENTS.md`. The exact-case rule in the main count is doing work. ## What do the files contain? Rules, in almost every instruction set. The median length is 818 words. Most directives do not say why. Sonnet’s codes, reweighted to the population, put at least one rule in 98.2% of instruction sets (95% CI 97.3–98.9). A repository’s statements average 52.6% rules, 29.8% descriptions, and 17.6% references. The rules cover code and its artefacts in 88.7% of repositories, workflow in 85.4%, the agent’s own behaviour in 79.3%, and the instruction files themselves in 52.7%. The subject shares the paper prints for rules: code conventions and quality 74.4%, build, environment, and delivery 72.6%, agent workflow and conduct 67.3%. Project knowledge appears in 86.2% of instruction sets, and 54.3% contain a rule on that area. The most common specific rule subjects are verification before declaring work done (49.7%), test writing (48.2%), and the instruction files themselves (44.3%). Each of those three is under half of repositories. The 10th and 90th percentiles of length are 163 and 3,010 words. A file can be a short command list or a long manual. The median, 818, is the figure to remember. ## Do the rules give a reason? Most instruction sets contain a reason somewhere. Most directives do not carry one. 74.0% of directives have none. 82.9% of instruction sets give a reason at least once. 10.8% cite an incident or evidence. 80.9% mark at least one rule as mandatory, and 77.8% prohibit something, yet prohibitions average 15.8% of a repository’s directives. Unmarked imperatives are the most common form. Rules are tied to a moment in the workflow in 54.7% of instruction sets, and 53.7% state an exception. 8.6% of repositories name a specific agent somewhere. 8.3% address human contributors. Vague directives appear in 40.5% of repositories and are 3.5% of directives. These shares are Sonnet’s codes on the reweighted sample. The human review, below, is why this note keeps the label on them. On collaboration files, Haiku’s uncorrected codes name concrete requirements. Of files with a rule on pull-request descriptions, 27.2% require a test plan, 24.2% require the project’s template, and 23.3% require linked issues. Of files with a commit-message rule, 36.7% require Conventional Commits. Test-writing rules prohibit trivial tests in 8.1% of those files and prohibit weakening tests so they pass in 4.9%. ## Do projects agree about AI attribution? Commit-message rules prohibit it more often than they require it. Pull-request rules more often require disclosure. The paper prints both a file table and a repository sentence, and the percentages are not identical. | Cut | Require | Prohibit | | --- | --- | --- | | Commit messages, share of files with such a rule (Table 1, Haiku) | 3.5% | 7.4% | | Pull-request descriptions, share of files with such a rule (Table 1) | 5.7% | 1.9% | | Commit messages, repositories, in the prose | 59 repositories | 7.5% (95% CI 6.3–8.9) | | Pull requests, in the prose | 5.8% | 1.9% | The 7.5% and the 7.4% are close and they are not the same statistic. One is repositories, with an interval. One is files, uncorrected. The “59 require it” is a count. The sentence does not print how many repositories had a commit-message rule, so this note does not turn 59 into a percentage. The pull-request prose says 5.8% require disclosure and 1.9% prohibit it, mostly in different repositories. The file table says 5.7% and 1.9%. Trailer-based measures of AI use inherit the disagreement: a project that forbids the trailer and a project that demands it will not leave the same trace for the same amount of agent work. [Claude Code’s “Don’t ask again” files](/blog/claude-code-dont-ask-again) are a different instruction surface, permission prompts rather than a root markdown file. The census does not count them. ## Where does the text come from, and do projects edit it? 15.8% of instruction sets match an `/init` signature. 81.8% were edited after they were created. 948 of 5,998 instruction sets match a signature the paper associates with an `/init` command. The opening line of `claude /init` is in 692, which is 11.5%. Codex’s heading set appears in 146. The share falls with popularity, from 17.1% in the lowest star tier to 9.4% in the highest. The paper says the detector mainly measures Claude Code, because other tools leave no verbatim line. 93 instruction sets, 1.6%, contain a block a tool writes into the file. The Next.js development server accounts for 44 of those. 3.4% share a paragraph verbatim with the files of at least two other owners. 17.7% were created in an AI-marked commit, and 41.7% have at least one AI-marked commit. Among the 5,942 instruction sets with a file history, 33.1% carry a text trace or were created in an AI-marked commit. The paper’s “a third” is that 33.1%. 81.8% were edited after creation. Sets with a generator signature were edited about as often as sets without one, 82.4% against 81.7%, and the paper does not adjust those two shares for age. By file, 80.0% of `AGENTS.md` files and 67.7% of `CLAUDE.md` files have more than one commit. 24.5% of the 106,279 commits that touched agent files are AI-marked. The marker is a trailer. It does not identify which lines a model wrote. ## Which languages have the files? TypeScript is the highest share, 40.8% of 4,048 repositories. C is the lowest, 13.0% of 1,272. Python, the largest group, is 24.3% of 5,015. | Language | Repositories | AGENTS.md | CLAUDE.md | Either | | --- | --- | --- | --- | --- | | TypeScript | 4,048 | 34.4% | 24.5% | 40.8% | | Rust | 1,706 | 28.1% | 18.9% | 32.7% | | Go | 2,264 | 24.5% | 16.6% | 28.9% | | Swift | 474 | 23.4% | 17.9% | 28.1% | | Kotlin | 611 | 21.1% | 14.9% | 26.0% | | Java | 1,262 | 20.9% | 12.4% | 24.6% | | Python | 5,015 | 19.5% | 14.9% | 24.3% | | C# | 964 | 18.2% | 12.0% | 23.5% | | PHP | 804 | 14.6% | 9.1% | 18.9% | | JavaScript | 2,211 | 13.0% | 10.1% | 17.5% | | C++ | 1,785 | 14.7% | 7.7% | 17.4% | | Ruby | 438 | 15.1% | 8.9% | 17.1% | | C | 1,272 | 10.7% | 6.3% | 13.0% | The thirteen repository counts sum to 22,854. JavaScript’s either-share, 17.5%, is well below TypeScript’s 40.8% even though the names are often discussed together. The percentages are the paper’s Table 4. They are rounded, so a reader who multiplies them back by the repository count will not always land on an integer. ## Which of the paper’s hypotheses held? Three of six prespecified hypotheses held. More stars go with more rules about pull requests or AI contribution policy. Compiled languages go with more build-and-run statements. In a repository that has both files, `CLAUDE.md` is more descriptive than `AGENTS.md`. | Hypothesis | Verdict | Effect the paper prints | | --- | --- | --- | | H1. A higher AI-marked share goes with more agent rules, permission rules, and prohibitions | Not supported | IRR 0.977 (0.959–0.996) on the first of three tests | | H2. More collaboration goes with more workflow rules and fewer descriptions | Mixed, 2 of 4 | Merged pull requests: workflow IRR 1.027; human authors moved workflow the other way | | H3. Compiled languages state how to build and run more often | Supported | OR 1.68 (1.48–1.91) | | H4. Emphasis goes with more rules on agent failure modes | Not supported | OR 0.93 (0.88–0.99) | | H5. CLAUDE.md is more descriptive than AGENTS.md in the same repository | Supported | +6.3 percentage points (4.2–8.4), 484 pairs | | H6. More stars go with rules on pull requests or AI policy | Supported | OR 1.36 (1.29–1.44) per standard deviation of log stars | The tests use 5,587 repositories with at least five coded statements. A hypothesis counts as supported only if every one of its tests points the predicted way and the adjusted p-value is under 0.05. H1 and H4 are significant in the wrong direction, which is why they are not supported. The paper says H3, H6, and the pull-request half of H2 still hold on the Sonnet sample and when either of the two largest languages is left out. The exploratory models, 99 tests on 5,586 instruction sets, associate commits in the past year with more of the content outcomes than the other predictors: 11 of 13 tests on that predictor stay significant after the false-discovery correction. LLM and agent projects are 27.9% of adopters and 13.3% of matched controls, an odds ratio of 2.76 within star tier and language. A post hoc check that holds age and activity fixed, and drops agent-tooling projects, shrinks that ratio to 1.54 (95% CI 1.28–1.84). Libraries are under-represented, odds ratio 0.58. The paper classifies project type with Haiku on repository metadata. On 29 repositories the reviewer agreed on type at kappa 0.43 and on AI focus at kappa 0.80. The type contrasts are the paper’s “indicative” results. ## How much did the human review agree? The reviewer accepted the statement boundaries in 55 of 60 cases, 92%. Agreement on the subject was 43% to 50%. The 60 statements were coded before the reviewer saw the model labels. Restatements and extracted elements were accepted in 56 of 60, 93%. The free-text subject fit in 57 of 60, 95%. Against the models, description versus rule agreed in 82–83% of statements once references were excluded. The three-way split into description, rule, and reference agreed in 62–67%, kappa 0.33–0.40. Subject area agreed in 43–50%. The fine subject agreed in 32–37%. The paper’s own reading of that gap: the models agree with each other more closely than with this reviewer, and subject categories stay model-defined. One reviewer cannot produce a human-human kappa. The adoption counts in the first table do not depend on these codes. The 74.0%, the subject shares, and the attribution stances do. ## What would a fresh count have to change? A count that wants to update the 6,006 has to keep the frame or publish a new one. “Over 60,000” does not update it. A live GitHub code search for files named `AGENTS.md`, kept on a separate row so it is not mistaken for this census, is the [AGENTS.md adoption tracker](/trackers/agents-md). The homepage number, read November 9, has no date, no star floor, and no language list on the sentence itself. The census has all three, and a day. Adding nested `AGENTS.md` files, dropping the star floor, or including archived repositories would move the numerator and the denominator together. The paper’s data-availability section says the dataset is on Hugging Face and does not print a URL. This note did not locate the dataset from that sentence, and it did not re-run the search. Which libraries people actually install around these agents is a different November 9 count: [Which AI agent framework do people install?](/blog/which-ai-agent-framework-do-people-install). Coding-agent prices and the dates already on vendor pages are the [30-day coding-agents brief](/reports/ai-coding-agents-next-30-days). The sourced product table is [Best AI coding agents](/best/ai-coding-agents). How the loop uses a file like this one is [How do AI agents work?](/blog/how-do-ai-agents-work). ## Parallel agent teams beat the best of 100 solo agents on 10 of 12 tasks (2026) URL: https://www.aiagentsinfo.com/research/parallel-agent-teams-test-time-compute Published: 2026-11-09. Updated: 2026-11-09 Paper: Test-Time Compute Scales Better with Parallel Agent Teams Paper URL: https://arxiv.org/abs/2611.09798 November 6, 2026 arXiv paper, in the November 9 listing: with Qwen3.8-27B, ten teams of ten beat the expected best of 100 solo agents on 10 of 12 tasks. ## What is a parallel agent team? A parallel agent team is a small group of agents that share notes, with several such groups running at once, and with the best scored result kept. The paper studies tasks where a verifier can score an attempt, so the best of several attempts can be chosen without a person in the loop. A solo agent is one sample. Best-of-N keeps the best of many solo agents. One large team is a different sample: the members share a forum, and the sample's score is the best submission any member produced. Parallel agent teams apply best-of-N to the teams. Ten teams of ten, at a budget of 100 agents, are ten samples. One hundred solo agents are one hundred samples. One team of one hundred is one sample. Each agent in the main experiment runs in the [Codex](https://github.com/openai/codex) harness, inside its own sandbox. The forum is the piece the authors add. It has three actions: post a note, list titles, and read one post. The agent's current best score is attached to its posts. The prompt tells the agent how many peers it has and says that posting and reading are optional. There is no manager in the main teams. The [harness](/glossary/harness) is the runtime around the model. The [orchestration](/glossary/orchestration) here is the choice to split a budget into teams of a given size and to let them share a forum. The model weights do not change. The score in the tables is an expected best, not the single luckiest run. From a pool of finished runs, the authors take every way of choosing N of them, record the best score in each choice, and average those bests. Their small example is the pool 2, 4, 5, 7, 9. The expected best is 5.4 for one draw, 7.1 for two, and 9 if every run is kept. That is why one pool can draw a curve from one agent up to the whole pool. ## How do the scores compare at the same number of agents? At 100 agents, ten teams of ten beat 100 solo agents on 10 of 12 tasks. At 50 agents, five teams beat 50 solo agents on 11 of 12. The model is Qwen3.8-27B. Each agent has 12 hours, and 6 hours on Browser Exploit. Team size is 10 throughout the table. "Best of N" is the expected best of N solo agents. "Teams" is the expected best of that many teams. Bold in the paper marks the winner of each pair. The units are not the same from row to row. EdgeBench tasks and the numbers printed as 21.0 through 109.0 are the task's own points. MoE Load Balance and Active Learning are rewards from 0 to 1. Browser Exploit is a ladder from 0 to 16. KDA Chunk Forward and Kimi-Linear Decode are the inverse of latency. SwinMLP BF16 is a speedup. Multiscale Cantilever is a stiffness ratio. A higher number is better inside a row. A 54.8 on Carleson Formalization is not comparable with a 0.820 on MoE Load Balance. | Task | Best of 10 | One team | Best of 50 | 5 teams | Best of 100 | 10 teams | | --- | --- | --- | --- | --- | --- | --- | | Dabic Gravity Inversion | 21.0 | 22.3 | 28.7 | 30.5 | 33.1 | 34.6 | | Borden Source Inversion | 79.9 | 76.5 | 87.9 | 98.2 | 89.3 | 109.0 | | Graph Node Classification | 69.1 | 79.0 | 76.8 | 90.0 | 81.5 | 95.1 | | Ad Placement Optimization | 74.5 | 74.3 | 76.6 | 76.7 | 77.3 | 77.8 | | Carleson Formalization | 12.2 | 23.2 | 17.7 | 45.8 | 19.0 | 54.8 | | MoE Load Balance | 0.720 | 0.643 | 0.750 | 0.797 | 0.750 | 0.820 | | Active Learning | 0.520 | 0.530 | 0.540 | 0.548 | 0.560 | 0.550 | | Browser Exploit | 4.98 | 5.40 | 5.88 | 6.50 | 6.00 | 7.00 | | KDA Chunk Forward | 11.21 | 13.12 | 11.77 | 15.25 | 11.83 | 16.54 | | Kimi-Linear Decode | 1.77 | 1.33 | 2.12 | 2.04 | 2.36 | 2.16 | | SwinMLP BF16 | 5.65 | 6.74 | 6.20 | 8.08 | 6.29 | 8.33 | | Multiscale Cantilever | 11.06 | 11.60 | 11.44 | 11.69 | 11.50 | 11.70 | The 100-agent losses are Active Learning and Kimi-Linear Decode. The 50-agent loss is Kimi-Linear Decode only. The paper's average statement is relative, not a raw average of these cells: parallel teams of ten score 10% higher than solo agents at a budget of 10 agents and 27% higher at 100 agents. The largest cell it names is Carleson Formalization, 19.0 for 100 solo agents and 54.8 for ten teams. One team is a weaker claim than several teams. A single team of ten beats the expected best of ten solo agents on 8 of 12 tasks. It loses on Borden Source Inversion, Ad Placement Optimization, MoE Load Balance, and Kimi-Linear Decode. The paper's reading of those four is that one team varies from run to run, that Ad Placement Optimization is already close to its ceiling, and that Kimi-Linear Decode has little to share because most attempts score zero. Solo agents also flatten. The paper says that at 20 agents, each task has already reached 60% to 94% of the gain that about 100 solo agents bring. From 10 to 20 agents, each extra agent adds 1.27% to the parallel-team curve and 0.64% to the solo curve. The introduction gives a related saturation figure for solo agents: 50 agents already reach 81% to 100% of the gain that 100 agents reach. ## Do several small teams beat one large team? Yes, on all four tasks where the authors ran a team of 100. Ten teams of ten beat that one team by 8% to 30%. The four tasks are Dabic Gravity Inversion, Borden Source Inversion, Graph Node Classification, and KDA Chunk Forward. The single large team rises at first and then falls. From 64 agents to 100, its score drops on all four. On two of the four, one team of 100 scores below the expected best of 100 solo agents. The appendix names one of those: on Dabic Gravity Inversion, one team of 100 falls to 27.5, while 100 solo agents have an expected best of 33.1. Holding the budget at 100 agents and changing only the team size, the best size is in the middle. Ten teams of ten score highest on three tasks in that sweep. On Graph Node Classification, two teams of 50 both reached 100.0, the maximum. Twenty teams of five on that task score 77.6, which does not beat solo agents. On KDA Chunk Forward, twenty teams of five reach 13.9 and two teams of 50 reach 14.5, against 16.54 for ten teams of ten in the main table. The paper's statistical reading is that a team raises the average score, and several teams still collect the gain that comes from having more than one sample. One team, however large, is one sample. At smaller budgets the preferred size is sometimes below ten. The appendix says teams of ten are the best size at 20, 30, and 50 agents on Graph Node Classification and Browser Exploit, while Borden Source Inversion, Carleson Formalization, and Active Learning prefer three to five throughout. Where the preferred size changes, it grows as the budget grows. ## Do the teams beat GPT-5.6 Sol at the same dollar cost? At the price of five GPT-5.6 Sol agents, Qwen3.8-27B teams win on 4 of 8 tasks. At the price of one, they usually do not. The eight tasks are the ones with at least five GPT-5.6 Sol runs. Kimi-Linear Decode and SwinMLP BF16 have one frontier run each and are left out of the count. Qwen is priced as a share of the H200 engines that served it: $4.54 per GPU-hour, a four-GPU engine serving twelve agents for 12 hours, which is $18.16 per agent-run, and kernel engines serving ten agents, which is $21.79. Every Qwen agent costs the same on that basis, whatever it did with the time. GPT-5.6 Sol is priced from recorded tokens at list rates of $4 per million input tokens, $0.40 per million cached input tokens, and $20 per million output tokens, plus $5 per million tokens written to cache. The CLI does not report cache writes, so the paper charges every uncached input token as a write and calls that an upper bound. A Qwen team of ten costs about as much as one or two GPT-5.6 Sol agents. The wins at five times the price of one frontier agent are Dabic Gravity Inversion, Graph Node Classification, Carleson Formalization, and Active Learning. The printed comparisons are: on Dabic, four Qwen teams reach 29.0 against 19.7 for five GPT-5.6 Sol agents; on Graph Node Classification, three Qwen teams reach 86.9 against 84.6, while one GPT-5.6 Sol agent at 83.0 still beats anything Qwen buys for the price of that single agent; on Carleson, eight Qwen teams reach 52.0 against 47.9, and one GPT-5.6 Sol agent beats one Qwen team. GPT-5.6 Sol stays ahead on Borden Source Inversion and Ad Placement Optimization, where its mean is higher, and on KDA Chunk Forward, where its spread is wider. On MoE Load Balance it leads 0.78 to 0.77. Pricing Qwen by tokens, at $0.25 per million input tokens, $0.025 cached, and $2.50 output, comes to $4 to $29 per agent-run. The paper says that basis favors Qwen and raises the matched-or-better count to 6 of 8, both at the price of one frontier agent and at the price of five. The headline "half the tasks" in the abstract is the GPU-time count, 4 of 8. These are the authors' runs inside their sandboxes. They are not a row on a public GPT-5.6 Sol leaderboard. The [GPT-5.6 system card](https://deploymentsafety.openai.com/gpt-5-6) loaded on November 9, 2026. This page uses it as the document the paper cites, and it does not copy a score out of that card. ## Does a manager help, and does a longer solo session? A manager made the tested teams worse. A longer solo session, on the one task measured, barely moved the score. The orchestrated team replaces the forum with one agent that assigns work. Workers report back to that agent and do not read one another. On Borden Source Inversion, Graph Node Classification, and KDA Chunk Forward, those teams stay below flat teams and below solo agents at every budget in the figure. On Borden, the best of ten orchestrated teams scores 68.4. The average flat team of ten scores 76.5. The paper's reason is that workers wait on the manager and share findings only through it. An orchestrated team costs as much as a flat team and has a lower mean. Dabic Gravity Inversion is omitted because its evaluator often crashed. On Graph Node Classification, the authors compare a longer session with a second agent. They did not launch a separate 6-hour experiment. They read 101 scheduled 12-hour runs at hour 6 and at hour 12. The mean moves from 52.0 to 52.7. A team of two, read at hour 6, averages 57.4, against 52.7 for a single agent at hour 12. The better of two independent 6-hour agents scores about 57.6, so at that size the paper says most of the gain is the second attempt. The agents being read at hour 6 still believed they had 12 hours. The paper says an agent told it had only 6 hours might pace itself differently. ## What happens with a second model? DeepSeek V4.1 Flash does not copy the Qwen pattern on every task. The repeat covers eight tasks, about 100 solo runs and ten teams of ten, 12 hours per agent and 6 hours on Browser Exploit. Teams beat solo agents at every budget on Borden Source Inversion and KDA Chunk Forward. On SwinMLP BF16 the lead shrinks from 11% at 10 agents to 2% at 100. On Carleson Formalization, solo agents lead by about one point at every budget. Graph Node Classification, MoE Load Balance, and Multiscale Cantilever are ties throughout. On Browser Exploit, solo agents lead until both sides reach the ladder maximum of 16. Two caveats are in the appendix. The MoE Load Balance solo runs were cut short by a 60-turn cap in the harness and ran about 8 hours, against 12 hours for the teams. One KDA Chunk Forward team scored 31.8, far above every other run. The Qwen table is the paper's main result. The DeepSeek figure is the check on whether the same arrangement travels. ## What the experiment controlled Names and numbers in this section are printed in the November 6 paper unless a link says otherwise. The harness is Codex, unpinned as to release. The main model is Qwen3.8-27B, served with SGLang on the authors' H200 GPUs, about ten concurrent agents per four-GPU engine. Sandboxes are Modal. EdgeBench tasks have a judge sidecar. Kernel tasks hold a dedicated H100 for the whole run. Browser Exploit uses a privileged virtual machine. The two MLS-Bench tasks grade the sandbox at the end and do not offer a running judge. A crash or an invalid submission scores zero. Kernel runs that hit a known reward exploit on Kimi-Linear Decode and KDA Chunk Forward are excluded. The forum is one service shared by every run. Its transport failure rate was 0.04% with 60 concurrent agents and 0.19% with 250. Teams of up to ten see the whole index of titles. Teams of 64 and 100 see the newest 100 titles and every unread post. The study's total is more than 8,000 agent runs and about 95,000 agent-hours. A run enters a pool if it finished or ran to its budget. Infrastructure failures are dropped. The same prompts go to every model. A solo agent receives the task instruction and a time-budget line. A team member also receives a forum note. Nothing is pushed during the run. If a session ends early, the harness restarts it with a resume prompt. The time-budget line tells the agent to keep going for 12 hours, and the appendix sets Browser Exploit to 6 hours. Checked November 9, 2026, the [Qwen3.8-27B model card](https://huggingface.co/Qwen/Qwen3.8-27B) loaded, and the [GPT-5.6 system card](https://deploymentsafety.openai.com/gpt-5-6) loaded. The code URL in the paper did not. The GitHub API returned 404 for `modal-projects/parallel-agent-teams-scaling`. This arrangement is not a search over harness source. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) edits the harness and grades a held-out split. [Evolver](/research/evolver-agent-exploration-evolving-capabilities) injects strategy notes into Claude Code or Codex from earlier traces of the same tasks. The forum here is closer to a shared log than to a stored skill. The [November 9 harness-eval note](/research/garry-tan-harness-evals-november-2026) compares named product harnesses on a fixed task list. Codex in that note is one product among several. Codex in this paper is the common runtime, and the thing that changes is how many copies share a forum. ## Limits that change how to read the scores The conclusion lists the limits, and the appendices add more. The method needs a score. Tasks a person still has to judge are outside the protocol. The two MLS-Bench tasks are scored once at the end, so a team cannot steer on a running number the way the EdgeBench tasks can. The large-team claim is four tasks, with one to five runs at each large size, and the forum those large teams saw was truncated. The 8% to 30% range should be read with that sample. The main team pools are about ten runs. An expected best over ten teams is an average across a small pool, and one outlier can move it. The DeepSeek appendix flags one KDA team at 31.8. The normal-distribution story fits Graph Node Classification and Browser Exploit and fails where scores pile up at zero or at a ceiling. The paper says the predicted sign is right on 9 of 12 tasks, and that on KDA Chunk Forward the fit misses most of the teams' 40% gain. The mechanism the authors want a reader to keep is the split between a higher mean and the gain from having many samples. The formula is not a forecast for every task. The dollar comparison depends on the price basis and on the multiple. GPU-time pricing, at five frontier agents, is 4 of 8. One frontier agent still wins several of the printed one-agent comparisons, including Carleson. Token pricing of Qwen changes the count. A reader who quotes only "beats GPT-5.6 Sol" has dropped the denominator. The second model does not reproduce the 10-of-12 result. DeepSeek ties or loses on several of the eight tasks, and one of those comparisons is shortened by a turn cap. Transfer across models is an open part of the paper, not a result the Qwen table settles. The code link was not available on November 9, 2026. Until a repository is public, the tables stand as the authors' report. ## Read the paper The November 6 paper, the model it runs, and the frontier card it cites: - [Parallel agent teams on arXiv](https://arxiv.org/abs/2611.09798) - [HTML](https://arxiv.org/html/2611.09798) - [PDF](https://arxiv.org/pdf/2611.09798) - [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) - [GPT-5.6 system card](https://deploymentsafety.openai.com/gpt-5-6) - [Codex](https://github.com/openai/codex) - [Code link named in the paper](https://github.com/modal-projects/parallel-agent-teams-scaling) Related pages on this site: [agent harnesses hub](/topics/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [research](/topics/research), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [How do AI agents work?](/blog/how-do-ai-agents-work), [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [orchestration](/glossary/orchestration), [benchmark](/glossary/benchmark), [behavioral evaluation](/glossary/behavioral-evaluation), [Best AI agent harness](/best/ai-agent-harnesses), [HarnessFlow](/research/harnessflow-mcts-harness-optimization), [Evolver](/research/evolver-agent-exploration-evolving-capabilities), [Garry Tan harness evals](/research/garry-tan-harness-evals-november-2026), the [November harness benchmark](/reports/agent-harness-benchmark-november-2026), and [Codex](/directory/codex). ## Paper breakdown: MCPHallu — why MCP agents fail on unreachable goals URL: https://www.aiagentsinfo.com/research/mcphallu-mcp-agent-hallucinations Published: 2026-11-08. Updated: 2026-11-08 Paper: MCPHallu: Benchmarking Reasoning, Execution, and Memory Hallucinations in MCP Agents Paper URL: https://arxiv.org/abs/2611.07689 Average score 0.642 across 14 agents and 5,012 trajectories. Unreachable Goal scores 0.458, and 23.2% of those runs pass. Claude Opus 4.6 leads overall at 0.728. No model in the paper scores above that. ## What does the main table say about the 14 models? Table 5 is the result. Scores run from 0 to 1, and higher is better. The paper prints a mean of 0.642 and a pass rate of 47.8%. The best overall cell is 0.728. The introduction says no model scores above 0.728 on average. The 14 overall numbers below add up to 8.992, and 8.992 divided by 14 rounds to 0.642. Bootstrap intervals are in the paper’s Appendix F.5. The main text says the overall intervals have a half-width of about 0.03 to 0.04. One interval from that appendix, for the leader, is printed here because it is the number a reader will try to over-read: Claude Opus 4.6 overall is 0.728, with a 95% interval of 0.695 to 0.759. The HTML and the PDF agreed on every cell of Table 5 when both were read on November 8, 2026. | Model | Overall | Branch | Context | Unreach. | Tool | Easy | Med. | Hard | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | GPT-4o | 0.599 | 0.641 | 0.555 | 0.405 | 0.835 | 0.674 | 0.593 | 0.517 | | GPT-5-chat | 0.659 | 0.783 | 0.592 | 0.438 | 0.864 | 0.718 | 0.655 | 0.594 | | Claude Haiku 4.5 | 0.639 | 0.818 | 0.575 | 0.371 | 0.840 | 0.763 | 0.610 | 0.524 | | Claude Sonnet 4.5 | 0.710 | 0.766 | 0.655 | 0.567 | 0.880 | 0.786 | 0.651 | 0.686 | | Claude Opus 4.5 | 0.696 | 0.867 | 0.612 | 0.492 | 0.849 | 0.808 | 0.678 | 0.585 | | Claude Sonnet 4.6 | 0.702 | 0.714 | 0.657 | 0.585 | 0.875 | 0.770 | 0.684 | 0.640 | | Claude Opus 4.6 | 0.728 | 0.755 | 0.730 | 0.576 | 0.885 | 0.784 | 0.707 | 0.687 | | Gemini 3 Flash | 0.583 | 0.677 | 0.643 | 0.284 | 0.793 | 0.648 | 0.567 | 0.525 | | Doubao-Seed-1.8 | 0.692 | 0.753 | 0.635 | 0.547 | 0.860 | 0.785 | 0.673 | 0.603 | | DeepSeek-V3.1 | 0.655 | 0.773 | 0.707 | 0.390 | 0.802 | 0.760 | 0.634 | 0.553 | | DeepSeek-V4 | 0.606 | 0.853 | 0.533 | 0.302 | 0.788 | 0.703 | 0.598 | 0.500 | | Qwen3.5-397B | 0.589 | 0.656 | 0.445 | 0.539 | 0.720 | 0.724 | 0.529 | 0.496 | | GLM-5.1 | 0.539 | 0.621 | 0.457 | 0.413 | 0.688 | 0.671 | 0.481 | 0.448 | | Kimi-K2.6 | 0.595 | 0.618 | 0.454 | 0.498 | 0.829 | 0.686 | 0.571 | 0.514 | The highest cell in each column, read off that table: overall 0.728 and Context Forgetting 0.730 and Tool Misuse 0.885 and Medium 0.707 and Hard 0.687 are Claude Opus 4.6. Branch Collapse 0.867 and Easy 0.808 are Claude Opus 4.5. Unreachable Goal 0.585 is Claude Sonnet 4.6. GLM-5.1 is the lowest overall score, 0.539. Gemini 3 Flash is the lowest Unreachable Goal score, 0.284. Unreachable Goal is the lowest of the four subtype scores on 12 rows. On Qwen3.5-397B, Context Forgetting is 0.445 and Unreachable Goal is 0.539. On Kimi-K2.6, those two cells are 0.454 and 0.498. That is the “12 of the 14 models” sentence in section 5.2, visible in the table. Easy to Hard falls on 13 rows. Claude Sonnet 4.5 is the row that does not: Medium is 0.651 and Hard is 0.686. The model names are the paper’s names. This table is not a claim about any model the November 5 paper left out. ## What are the four subtypes, and how are they detected? Table 2 is the taxonomy. Two subtypes share the reasoning axis and use different triggers. The paper says that is why both stay in the benchmark: across models, Branch Collapse and Unreachable Goal correlate at r = -0.12. | Subtype | Axis | Structural trigger | Detection signal | | --- | --- | --- | --- | | Branch Collapse | Reasoning | At least two mutually exclusive branches, gated on a tool output | Wrong-branch tool calls in the trajectory | | Unreachable Goal | Reasoning | The goal is not satisfiable by any available tool | Continued action instead of an infeasibility report | | Tool Misuse | Execution | At least two tools with overlapping schemas and distinct semantics | Calls to curated confusion tools | | Context Forgetting | Memory | A key value is retrieved early, then same-domain distractors follow | The final answer differs from the retrieved value | The paper says the set is representative, not exhaustive, and that subtypes can co-occur in one trajectory. This release of Tool Misuse instantiates wrong tool selection. Faulty invocation, with the right tool and invalid arguments, is listed as future work. Context Forgetting covers a stale, approximate, or substituted value. A [behavioral evaluation](/glossary/behavioral-evaluation) checks one observable action. MCPHallu is a [benchmark](/glossary/benchmark): a fixed set of tasks, a scorer, and a comparison across models. The glossary entry for the protocol the agents speak is [Model Context Protocol](/glossary/model-context-protocol). The loop they ran inside is the one the paper cites from ReAct. The note on that paper is [ReAct](/research/react-reasoning-and-acting). ## Where do the 358 tasks and 36 servers sit? Table 3 is the domain cut. The representative servers are examples, not the full server list. Table 8 later names all 36 and totals 296 tools. Section 4.2’s phrase for the same inventory is “roughly 300 tools.” | Domain | Representative servers | Servers | Tasks | | --- | --- | --- | --- | | General | brave_search, exa, weather | 14 | 75 | | Coding | git, github, e2b_server, mcp_code_executor | 6 | 80 | | Analytics | airtable, mongodb, national_parks | 5 | 69 | | Productivity | filesystem, notion, slack | 9 | 61 | | Financial | twelvedata, alchemy | 2 | 73 | Fourteen plus 6 plus 5 plus 9 plus 2 is 36. Seventy-five plus 80 plus 69 plus 61 plus 73 is 358. Figure 6’s caption says Unreachable Goal is the largest subtype, at 103 tasks, and the other three range from 82 to 89. The three feasible subtypes are the other 255 tasks. The paper says that in the appendix, and 358 minus 103 is 255. Table 6’s 1,442 Unreachable Goal runs are 103 times 14. Table 7’s 1,176 Tool Misuse runs are 84 times 14, so Tool Misuse is the subtype of 84 tasks inside that 82-to-89 range. This page does not assign the remaining 89 and 82 to Branch Collapse and Context Forgetting, because the caption states the range and does not print those two counts in a sentence. Servers run in Docker with a sandboxed filesystem and allow-listed network egress. A fresh container starts each episode, so a stateful write does not leak into the next run. Credentials are per server in Table 8. Some need an API key, some none, MongoDB a connection string, Google Workspace an OAuth token. The paper says no credentials are distributed with the release. Construction is three stages. Tool grounding reads schemas from the live servers. Adversarial synthesis uses Claude Sonnet 4.6, with a subtype-specific prompt, to write candidates that have a checkable correct behavior. Filtering checks executability, a real structural trigger, self-contained instructions, non-triviality, and label consistency. A Claude Code solver on Claude Sonnet 4.6 runs the label check, separate from the GPT-4o judge, so one model is not both the only writer and the only grader. Three annotators, each with at least two years on agent evaluation, review tasks that survive. A task is accepted at 2 or better, on a 0-to-2 scale, for realism, label, and ground truth, from at least two of the three. The benchmark keeps 73% of the candidates that were generated. ## Why do unreachable goals fail? The abstract’s sentence is the finding: agents often continue issuing tool calls, or claim successful completion, rather than recognizing infeasible goals. The average on this subtype is 0.458. The pass rate is 23.2%. Table 6 bins every Unreachable Goal run by how many tools the agent called. Pass rate falls as the call count rises. Completion language in the final answer, words such as “successfully,” “saved,” and “created,” moves the other way through the 8-to-15 bin, then drops at 16 or more. | Calls | Runs | Pass % | Mean score | Completion language % | | --- | --- | --- | --- | --- | | 0 | 320 | 65.0 | 0.778 | 0.6 | | 1–3 | 455 | 27.9 | 0.569 | 13.8 | | 4–7 | 266 | 0.0 | 0.342 | 30.1 | | 8–15 | 158 | 0.0 | 0.201 | 36.7 | | 16+ | 243 | 0.0 | 0.121 | 19.8 | The paper’s prose says the pass rate is 0% once the agent issues four or more calls. That matches the last three rows. It does not match a reading of Figure 7 as the same table. Figure 7’s counts are 342, 475, 283, 182, and 264, and its zero-call pass rate is 65.2%, not 65.0%. Those figure bins add to 1,546. The table bins add to 1,442. This page treats Table 6 as the source for these percentages. The early-stop rule in the scorer is built for this pattern. For Unreachable Goal, the structural score is 1 if the agent declares infeasibility within a probing window of k = 2 tool calls, then decays, and it is 0 if the agent never declares infeasibility. The window is there so a short probe is not punished. A long trajectory that never says “this cannot be done” gets nothing from that component. Appendix Table 13 prints a coarser cut: 307 runs that keep acting for 4 or more calls with no infeasibility declaration, pass rate 0.0%; 210 runs that keep acting for 1 to 3 calls, pass rate 0.0%; 448 runs that stop and declare infeasibility, pass rate 48.0%. Those three rows sum to 965, not to 1,442, so they do not cover every run in Table 6. Table 14, limited by its caption to non-empty responses, is the language cut: completion only, 195 runs, 2.1% pass; limitation only, 304 runs, 44.4% pass, mean 0.668. The four language rows sum to 1,366. The useful reading is the one the bins agree on, even where the counts do not. Stopping early and saying the goal cannot be met is associated with passes. Continuing, and talking as if the task finished, is associated with zeros. Blanket refusal is not the same skill. The three feasible subtypes are 255 of the 358 tasks and are scored by completion. On Tool Misuse, calling neither the correct tool nor a confusion tool passes 8.0% of the time. ## What does tool misuse look like once the right tool is in the trace? Table 7 splits Tool Misuse trajectories by which tools were actually called. Finding the right tool is not the result. Avoiding the near-miss tool is. | Type | Runs | Pass % | Mean | Steps | Confusion calls | | --- | --- | --- | --- | --- | --- | | Correct only | 825 | 84.7 | 0.927 | 3.54 | 0.00 | | Correct + confusion | 260 | 32.3 | 0.649 | 10.70 | 3.42 | | Confusion only | 16 | 0.0 | 0.348 | 3.38 | 1.63 | | Neither | 75 | 8.0 | 0.364 | 0.37 | 0.00 | Correct only versus correct plus confusion is 84.7% versus 32.3%, with the required tool present in both. Mixed traces are also the long ones: 10.70 steps against 3.54. The confusion score penalizes the set of distinct confusion tools, not the number of times one of them is retried. One look at the wrong tool costs the same, on that term, as several looks at the same wrong tool. The ideal trace the appendix describes is short. Tool Misuse tasks have a sufficient call sequence with a median of 2 calls, range 2 to 5. Branch Collapse and Context Forgetting sit at a median of 7. Nineteen of the 84 Tool Misuse tasks need more than one distinct tool, for example a log lookup before the call that answers the question. The appendix’s worked contrast is git_git_show on a commit, which returns that commit’s patch, against git_git_diff with target HEAD, which answers a different question. Context Forgetting has its own split, in Appendix Table 15. Stateless runs, in General, Analytics, and Financial, pass 59.2% of the time, mean 0.713, 7.17 steps, over 644 runs. Stateful runs, in Productivity and Coding, pass 24.6%, mean 0.431, 11.46 steps, over 504 runs. The paper’s point is that the memory failure is often a failure to write the remembered value into the artifact the task scores, not only a failure to hold the value in the context window. ## Why is the overall leader not the leader on every subtype? Table 16 ranks each model inside each subtype. Rank 1 is best. Spread is the gap between a model’s best and worst subtype rank. The mean spread is 6.86 out of a possible 13. The abstract’s “6.9 of 13” is that figure rounded. Claude Opus 4.6, the overall leader, is rank 7 on Branch Collapse, rank 1 on Context Forgetting, rank 2 on Unreachable Goal, and rank 1 on Tool Misuse. The discussion section says this in one line: the overall leader ranks seventh on Branch Collapse. | Model | Branch | Context | Unreach. | Tool | Spread | | --- | --- | --- | --- | --- | --- | | Claude Opus 4.6 | 7 | 1 | 2 | 1 | 6 | | Claude Opus 4.5 | 1 | 7 | 7 | 6 | 6 | | Claude Sonnet 4.6 | 9 | 3 | 1 | 3 | 8 | | Claude Sonnet 4.5 | 6 | 4 | 3 | 2 | 4 | | Claude Haiku 4.5 | 3 | 9 | 12 | 7 | 9 | | DeepSeek-V4 | 2 | 11 | 13 | 12 | 11 | | DeepSeek-V3.1 | 5 | 2 | 11 | 10 | 9 | | Doubao-Seed-1.8 | 8 | 6 | 4 | 5 | 4 | | GPT-5-chat | 4 | 8 | 8 | 4 | 4 | | GPT-4o | 12 | 10 | 10 | 8 | 4 | | Gemini 3 Flash | 10 | 5 | 14 | 11 | 9 | | Qwen3.5-397B | 11 | 14 | 5 | 13 | 9 | | GLM-5.1 | 13 | 12 | 9 | 14 | 5 | | Kimi-K2.6 | 14 | 13 | 6 | 9 | 8 | DeepSeek-V4 is the sharp version of the same fact. It is rank 2 on Branch Collapse, with a Table 5 score of 0.853, and rank 13 on Unreachable Goal, with a score of 0.302. A team that picks a model from the overall column will not have picked the best model for every failure mode in this paper. The attribution audit is the check on whether the label and the observed failure are the same thing. Appendix Table 9 counts 80 failed runs. Rows are the intended subtype. Columns are what the auditors saw. The diagonal is 16, 21, 13, and 17, which is 67. The paper reports kappa 0.78 for that audit. | Intended \ Observed | Branch | Unreach. | Tool | Context | Row count | | --- | --- | --- | --- | --- | --- | | Branch Collapse | 16 | 1 | 2 | 2 | 21 | | Unreachable Goal | 0 | 21 | 0 | 0 | 21 | | Tool Misuse | 0 | 4 | 13 | 0 | 17 | | Context Forgetting | 0 | 3 | 1 | 17 | 21 | Eight of the 13 off-diagonal runs land in the Unreachable Goal column. The paper’s note is that a model which gets stuck on a feasible task sometimes stops and wrongly reports the goal as unreachable. That is a real overlap, and it is why 0.642 should not be read as four cleanly separated abilities that never touch. ## What the evaluation held fixed The agent framework is a ReAct-style loop for every model. The context is 128K tokens even when the provider’s native window is longer. Gemini 3 Flash’s native window in the inference table is 1M. GPT-5-chat’s is 400K. DeepSeek-V4’s is 128K. All of them run the benchmark at 128K. The tool-call budget is 20. Temperature is 1.0 in every row of that table. Environments reset between episodes. The paper says a plan-and-execute scaffold and an infeasibility-aware prompt, checked in an appendix, leave the two hardest subtypes the same. The vendors and the context windows, from Appendix Table 12: | Model | API provider | Native context | Temperature | | --- | --- | --- | --- | | GPT-4o | OpenAI | 128K | 1.0 | | GPT-5-chat | OpenAI | 400K | 1.0 | | Claude Haiku 4.5 | Anthropic | 200K | 1.0 | | Claude Sonnet 4.5 | Anthropic | 200K | 1.0 | | Claude Sonnet 4.6 | Anthropic | 200K | 1.0 | | Claude Opus 4.5 | Anthropic | 200K | 1.0 | | Claude Opus 4.6 | Anthropic | 200K | 1.0 | | Gemini 3 Flash | Google | 1M | 1.0 | | Doubao-Seed-1.8 | ByteDance | 256K | 1.0 | | DeepSeek-V3.1 | DeepSeek | 128K | 1.0 | | DeepSeek-V4 | DeepSeek | 128K | 1.0 | | Qwen3.5-397B | Alibaba | 262K | 1.0 | | GLM-5.1 | Zhipu AI | 200K | 1.0 | | Kimi-K2.6 | Moonshot AI | 256K | 1.0 | Fourteen models times 358 tasks is 5,012 trajectories, which is the count in the abstract. One run per model per task is the arithmetic of that sentence. Expert humans, on the stratified 40-task sample, score 0.88. The paper sets the ideal agent at 1.0, the score for never triggering the designed hallucination, and says the humans come close. The comparison it prints against the best model uses the full-benchmark 0.728, not a 40-task score for that model. The Unreachable Goal gap is printed as 0.82 versus 0.58. Table 5’s full-set figure for Claude Opus 4.6 on that subtype is 0.576. ## Where the repository and the paper disagree The paper’s code link is [J1zz/mcp_hallu](https://github.com/J1zz/mcp_hallu). On November 8, 2026, the GitHub API reported the repository public, default branch main, created February 27, 2026, last pushed May 7, 2026, with 4 stars and 0 forks. The license SPDX id was Apache-2.0. The LICENSE file on main is the Apache License, Version 2.0. The README badge on that same file says MIT. The README also still says the submission is double-blind and that author information is withheld. The November 5 paper names nine authors. The README’s tool line is 302. Table 8’s total is 296, and adding the per-server tool counts in that table also comes to 296. The README displays a NeurIPS 2026 E&D Track badge that links to neurips.cc. The paper read for this note is the arXiv preprint submitted November 5, 2026. The badge is not, by itself, a sentence in that paper saying the work was accepted. Anyone reproducing the tables should pin the paper, not the README’s summary statistics. The tasks and the scoring code may still be in the repository. The license badge, the anonymity notice, the tool count, and the push date are older than the preprint this page is about. ## How to read it next to the other notes on this site MCPHallu holds the agent loop still and changes the task, so the failure mode is the thing that varies. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) searches for a different harness program. [Evolver](/research/evolver-agent-exploration-evolving-capabilities) keeps Claude Code and Codex and injects a strategy asset from an earlier trace. Neither paper’s table is a row in Table 5, and Table 5 is not a score for those systems. [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness) is the definition of the loop MCPHallu did not search over. [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness) is the guide for checking one action inside that software. A [harness](/glossary/harness) here is that runtime. MCPHallu’s judge-and-script scorer is a benchmark of model behavior in a shared loop, not a ranking of harness products. [What is BioBasicsBench?](/blog/what-is-biobasicsbench) is the other new benchmark note on this site, from November 8. It scores raw wet-lab evidence. It does not call MCP tools, and its model list is a different list. The [benchmark tracker](/trackers/benchmarks) is where a cell goes when this site has a source, a date, and the harness that produced it. MCPHallu’s cells live on this page, with the November 5 paper as the source. The paper, the HTML, the PDF, and the repository: - [MCPHallu on arXiv](https://arxiv.org/abs/2611.07689) - [HTML](https://arxiv.org/html/2611.07689) - [PDF](https://arxiv.org/pdf/2611.07689) - [J1zz/mcp_hallu](https://github.com/J1zz/mcp_hallu) Related pages: [research](/topics/research), [harnesses](/topics/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [Model Context Protocol](/glossary/model-context-protocol), [benchmark](/glossary/benchmark), and [behavioral evaluation](/glossary/behavioral-evaluation). ## Terminal-Bench 2.1: 28 of 89 repaired (2026) URL: https://www.aiagentsinfo.com/research/terminal-bench-2-1-benchmark-repairs Published: 2026-11-08. Updated: 2026-11-09 Paper: Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks Paper URL: https://arxiv.org/abs/2611.08417 Terminal-Bench 2.1 repairs 28 of 89 tasks from version 2.0. Individual scores shift by up to 12 points, ranks move by at most 3. Spearman 0.958. Terminal-Bench 2.1 is a repair of a benchmark that was already supposed to be clean. [Buchanan, Zhou, Linderman, and Ré](https://arxiv.org/abs/2611.08417) say version 2.0 had been through human review, model-assisted auditing, and red-teaming before release, and that 28 of its 89 tasks still needed a fix. The fixes ship as Terminal-Bench 2.1. The abstract says accuracy rose by over 10 percentage points. The discussion says an individual score can move by as much as 12 points, while the ordering of 19 agents stays highly correlated. This page was published November 8, 2026. The PDF, the HTML, the arXiv record, the 2.1 repository, pull request 53, and the Terminal-Bench homepage were read that day. ## What did the Terminal-Bench 2.1 repair change? The Terminal-Bench 2.1 repair changes tasks, not models. Section 5 says the audit found issues in 28 tasks, roughly a third of the 89. The paper sorts the defects into three piles, and a task can sit in more than one. Figure 3a prints the shares. Table 2 prints the counts. | Defect | Tasks in Table 2 | Share of the 28 in Figure 3a | | --- | --- | --- | | Instruction–verification mismatch | 16 | 57% | | Task drift | 12 | 43% | | Environment issues | 8 | 29% | Sixteen divided by 28 is 57%. Twelve divided by 28 is 43%. Eight divided by 28 is 29%. The three counts sum to 36 because of the overlap. The example the paper gives for a mismatch is `query-optimize`: the grader demanded a byte-for-byte copy of a database file, and the instructions never said so. Drift is a task that broke after release because a package, an API, or a web page moved. Environment issues are budgets and images that were too tight for the known-correct solution. The repair rule for a resource limit is at least 1.5 times what the oracle and the grader actually used. The sources of the 28 are also printed. Eleven tasks had already been flagged in Terminal-Bench 2.0 Verified, five more had been discussed in public issues and pull requests, and 14 were new to this audit. The paper says two tasks appear in more than one of those lists, and 11 + 5 + 14 − 2 = 28. Z.ai's earlier audit, which this paper cites, is a different measurement. The paper says environment fixes alone, on all 89 task images, moved Claude Opus 4.5 from 52.4% to 57.9% in that earlier report. That 5.5-point gap is Z.ai's number. It is not a row in Table 7. ## Which of the 28 Terminal-Bench 2.1 tasks moved? Table 7 lists the pass rate of each of the 28 repaired tasks, averaged across the paper's agent-model pairs. After the repair, the paper says no task in the table is still at zero for every agent. The largest drop is `overfull-hbox`, from 53.3% to 36.7%, which the appendix explains as a broken build that had been giving credit the repaired grader refuses. The five tasks that go from at most 15% to over 50% are the ones the main text says account for roughly 62.7% of the absolute pass-rate change on the modified tasks. Adding the absolute changes on those five rows gives 305.9 points. Adding the absolute changes on all 28 rows gives 487.5 points. 305.9 / 487.5 is 62.7%. | Task | TB 2.0 | TB 2.1 | Change | | --- | --- | --- | --- | | adaptive-rejection-sampler | 31.7% | 58.3% | +26.6 | | build-pmars | 75.0% | 81.7% | +6.7 | | build-pov-ray | 75.0% | 68.3% | −6.7 | | caffe-cifar-10 | 5.0% | 69.2% | +64.2 | | compile-compcert | 45.0% | 48.3% | +3.3 | | configure-git-webserver | 46.7% | 50.0% | +3.3 | | crack-7z-hash | 75.0% | 86.7% | +11.7 | | extract-moves-from-video | 8.3% | 13.3% | +5.0 | | filter-js-from-html | 0.0% | 6.7% | +6.7 | | financial-document-processor | 61.7% | 56.7% | −5.0 | | fix-git | 98.3% | 95.0% | −3.3 | | gpt2-codegolf | 15.0% | 18.3% | +3.3 | | hf-model-inference | 88.3% | 86.7% | −1.6 | | install-windows-3.11 | 0.0% | 23.3% | +23.3 | | make-doom-for-mips | 6.7% | 1.7% | −5.0 | | mcmc-sampling-stan | 75.0% | 71.7% | −3.3 | | mteb-leaderboard | 28.3% | 36.7% | +8.4 | | mteb-retrieve | 15.0% | 41.7% | +26.7 | | overfull-hbox | 53.3% | 36.7% | −16.6 | | polyglot-c-py | 5.0% | 80.0% | +75.0 | | polyglot-rust-c | 5.0% | 66.7% | +61.7 | | protein-assembly | 20.0% | 15.0% | −5.0 | | query-optimize | 50.0% | 43.3% | −6.7 | | rstan-to-pystan | 58.3% | 58.3% | 0.0 | | sam-cell-seg | 0.0% | 53.3% | +53.3 | | torch-pipeline-parallelism | 8.3% | 10.0% | +1.7 | | torch-tensor-parallelism | 15.0% | 66.7% | +51.7 | | train-fasttext | 5.0% | 3.3% | −1.7 | The signed changes sum to +377.7 points, or 13.5 points per repaired task. That is the mean on these 28 rows. It is the wrong number to paste next to a model's name. The other 61 tasks are outside the table. If they were unchanged, the same +377.7 points would be 4.2 points across all 89 tasks. The abstract's "over 10 percentage points" is larger than 4.2, so it is not this diluted average. The discussion's ceiling, 12 points on an individual agent, is a third quantity. The paper never prints the 19 accuracies that would let a reader see which agent moved by 12. Claude Code with Claude Opus 4.6 is the one agent the text names for a rank change: it sits more than 7 points above the fitted line and climbs three places. Codex with GPT 5.4 sits below the line. The two weakest agents, both under 40% on version 2.0, lose accuracy on 2.1. The paper says that is what you would expect when 19 of the 28 repairs were hard tasks. It does not name those two agents. ## Did the ordering of agents survive? Mostly. Spearman ρ = 0.958 between version 2.0 and Terminal-Bench 2.1 across 19 agents. The task-bootstrap 95% interval is 0.881 to 0.979. The fitted line in the text is TB2.1 = 2.1 + 1.04 × TB2.0. A correlation that high means the broad order is stable. A move of three ranks, which is the maximum the paper reports, is enough to change which of two close agents you would call the leader. Table 6 is the configuration list, and it has 20 rows, not 19. Three use Claude Code (Opus 4.6, Sonnet 4.6, GLM 5.1). Four use Codex (GPT 5.3 Codex, GPT 5.4, GPT 5.4 mini, GPT 5.5). Two use Gemini CLI (Gemini Flash 3, Gemini Pro 3.1). Eleven use Terminus-2, including Gemma 4 31B and Kimi K2.6. Qwen 3.5 9B and Qwen 3.5 27B appear in the model table and in the budget curves, and they are not in Table 6. This page stops at that count. It does not guess which row was left out of the 19. The auditor-agreement table is a different 89. Claude Code (Opus 4.6) and Codex (GPT 5.4) agree on the primary defect label, including "no defect," at κ = 0.80 on all 89 tasks. Cheaper judges agree less. Table 1 prints three pairs, each over the same 89 tasks: | Judge A | Judge B | Agree | Disagree | κ | | --- | --- | --- | --- | --- | | gpt-4o-mini | gpt-4o | 69 | 20 | 0.550 | | gpt-4o-mini | gemini-2.5-flash | 79 | 10 | 0.769 | | gpt-4o | gemini-2.5-flash | 69 | 20 | 0.549 | Against the final 2.1 release, the workflow's precision is 57.1% and its recall is 71.4%. The paper says those figures are retrospective, because the workflow helped build the repair set. They are a check on triage, not a claim that an unattended agent found every bug. ## How much of the harness gap is the clock? At the default timeout, the harness is a large part of the score. The abstract says extending the time limit closes a gap of about 20 points on GPT 5.4 and shrinks a 30-point gap on GPT 5.4 mini to 5 points. Figure 6 prints eight accuracies for Codex and Terminus-2 on Terminal-Bench 2.1. One pairing reproduces those gaps exactly, within ordinary rounding: | Model | Harness | 1× timeout | 32× timeout | | --- | --- | --- | --- | | GPT 5.4 | Codex | 74.2% | 74.9% | | GPT 5.4 | Terminus-2 | 53.9% | 75.3% | | GPT 5.4 mini | Codex | 67.0% | 68.2% | | GPT 5.4 mini | Terminus-2 | 37.1% | 62.9% | On GPT 5.4, 74.2 − 53.9 is 20.3 points at the default clock. At 32×, Terminus-2 is at 75.3 and Codex is at 74.9, so the gap has closed and the order has flipped by 0.4 points. On GPT 5.4 mini, 67.0 − 37.1 is 29.9 points, and 68.2 − 62.9 is 5.3 points. Codex barely moves when it is given more time. Terminus-2 uses the extra time. The paper's conclusion is that a fixed budget is an operating point, and that a harness comparison read only at that point can be a comparison of speed. Figure 3b says something compatible about failures. More than half of failed trials at the default budget ended in a timeout. The range the text prints is 68% under Claude Code and 18% under Gemini CLI. The appendix adds that 11.8% of graded timeouts still pass, because a timed-out trial is scored on the files the agent left behind. About 27% of upstream trials were filtered out before that chart, as runner and provider failures rather than agent outcomes. GLM 5.1 is the efficiency outlier the text names. On the budget sweep it solves 8 more tasks on version 2.1 than on 2.0 and overtakes GPT 5.4, whose coverage barely changes. The Qwen curves barely change at all. A longer clock and a repaired task set can reorder agents even when the default-time ranking looks stable. Terminus-2 is the baseline [harness](/glossary/harness): a single-agent ReAct loop with a shell and file tools. The ReAct paper is the [reasoning-and-acting note](/research/react-reasoning-and-acting) on this site. Claude Code and Codex are the provider harnesses in the same figure. A sourced comparison of those shipping runtimes is [Best AI agent harness](/best/ai-agent-harnesses). A harness guide that starts from the product side, rather than from this benchmark, is [How to evaluate an AI agent harness](/blog/how-do-you-evaluate-an-ai-agent-harness). The November harness board on the live site is a different version, and the [November 2026 agent harness benchmark](/reports/agent-harness-benchmark-november-2026) reads that board. ## What was on GitHub the day this page was written? The paper says Terminal-Bench 2.1 is at [harbor-framework/terminal-bench-2-1](https://github.com/harbor-framework/terminal-bench-2-1), and that the discussion of the revisions is [pull request 53](https://github.com/harbor-framework/terminal-bench-2/pull/53). Both were checked on November 8, 2026. | Check | Result | | --- | --- | | Repository API, 13:48:44 GMT | 136 stars, 73 forks, 48 open issues, license Apache-2.0, default branch `main`, `pushed_at` 2026-11-05T20:22:18Z | | `tasks/` listing, 13:42:29 GMT | 89 directories. All 28 Table 7 names were present | | README on `main` | "26 tasks were modified." Community submissions to the 2.1 leaderboard are closed | | Pull request 53 | Open. Created March 11, 2026. Updated May 28, 2026. Not merged. Opened by ekellbuch | The README's 26 and the paper's 28 were both public on the same afternoon. The directory count confirms the benchmark is still 89 tasks. It does not say which two tasks the README left out, or whether the README is counting a narrower kind of edit. The homepage check is the reason this page refuses to become a leaderboard. At 13:42:54 GMT, `https://www.tbench.ai/leaderboard/terminal-bench/2.1` returned a redirect to `/?version=2.1`. The HTML of that response still said Terminal-Bench 4.0 and did not include score rows. The README still links a 2.1 board and says only maintainer-run submissions are being added. Until that board is in the HTML, a percentage on a third-party "2.1 leaderboard" is someone else's run. Two earlier notes on this site already use the words Terminal-Bench 2.1 for other experiments. [Evolver](/research/evolver-agent-exploration-evolving-capabilities) reports a diagnostic slice of 20 tasks per configuration. [You don't need long context](/research/you-dont-need-long-context) runs the 89 tasks inside mini-swe-agent and studies context, not grader repairs. A percentage from either note is not a cell in Table 7, and Table 7 is not a submission to Terminal-Bench 4.0. The audit also left Terminal-Bench for a spot check. On 34 tasks drawn from an 80-task Harbor-Index snapshot, covering 29 source benchmarks, the workflow flagged 14 and manual review confirmed 12, which the paper calls 86% precision. Reviewers also found defects the workflow missed, and two bugs in shared grading code touched 13 tasks. The paper treats 2.1 as a checkpoint. The last sentence of the discussion says the pipeline can keep running, so version 2.1 is not a finished board. ## Evolver agent exploration: 320 runs, 83.13% to 87.50% URL: https://www.aiagentsinfo.com/research/evolver-agent-exploration-evolving-capabilities Published: 2026-11-07. Updated: 2026-11-09 Paper: Evolver: Turning Agent Exploration into Evolving Capabilities Paper URL: https://arxiv.org/abs/2611.07595 Evolver agent exploration, arXiv November 5, 2026: 320 paired diagnostic runs inside Claude Code and Codex. Success rose from 83.13% to 87.50%. Recorded task-execution tokens fell 24.24%. This page has the tables. ## How it compares with the same harness, and with public boards The paper's comparison is internal and paired. Each task is run twice under one harness and one model: once with the harness's native tools and built-in skills, and once in a fresh session with selected Genes injected. [Claude Code](https://github.com/anthropics/claude-code) and [Codex](https://github.com/openai/codex) are the harnesses. The model configurations are deepseek-v4.1-flash and gpt-5.6-sol. Evolver is version 2.0.33. No number below is copied from a vendor leaderboard. [HarnessFlow](/research/harnessflow-mcts-harness-optimization) is the nearby paper on this site, and it does the other job. Posted November 4, 2026, it searches for a new harness program and grades the winner on a held-out split. Evolver does not search harness source. It keeps the harness a team already runs and hands that harness a strategy asset from an earlier exploration. The November 6 HarnessFlow note stays the account of the search. This page is the account of reuse inside Claude Code and Codex. ### Success on the 320 pairs Figure 2 is the combined set and the two harnesses. Each harness row pools 160 pairs: two models, four benchmarks, 20 tasks. The combined row pools all 320. | Slice | Without Evolver | With Evolver | Change | | --- | --- | --- | --- | | Claude Code | 85.00% | 87.50% | +2.50 points | | Codex | 81.25% | 87.50% | +6.25 points | | All 320 pairs | 83.13% | 87.50% | +4.38 points | The printed percentages line up with integer counts. Claude Code at 85.00% of 160 pairs is 136 successes, and 87.50% is 140. Codex at 81.25% is 130, and 87.50% is 140. Together that is 266 of 320, which is 83.125% and rounds to the printed 83.13%, then 280 of 320, which is 87.50%. Fourteen additional successes on 320 pairs is 4.375 points, printed as +4.38. Both harnesses land on the same augmented rate, 87.50%. Codex has more room because it starts lower. Nine of the 16 configurations have a higher success rate. Seven are unchanged. The paper does not describe a configuration that gets worse. ### Where the successes move Figure 3 breaks the same 20 tasks out by harness, model, and benchmark. The text gives the aggregate by benchmark and a partial cell-by-cell account. Each benchmark aggregate has 80 pairs, because four harness–model configurations each contribute 20 tasks. | Benchmark | What the paper prints | | --- | --- | | Terminal-Bench 2.1 | 64/80 to 71/80, +8.75 percentage points | | SWE-rebench | +5.00 points | | GAIA | +3.75 points | | SWE-bench Multilingual | 96.25% under both conditions | Terminal-Bench 2.1 is the largest aggregate move. Sixty-four successes out of 80 is 80%. Seventy-one out of 80 is 88.75%. The gap is 8.75 points, and those seven extra successes are half of the fourteen extra successes in the whole study. SWE-rebench's 5.00 points on 80 pairs is four extra successes. GAIA's 3.75 points is three. SWE-bench Multilingual contributes none. 96.25% of 80 is 77, so that benchmark is 77 successes either way. The paper does not print the baseline counts for SWE-rebench or GAIA, only the point gaps. The named cells, each on 20 tasks: - Codex and DeepSeek on Terminal-Bench 2.1: 70% to 90%, which is 14 of 20 to 18 of 20, and the largest gain the paper names. - Codex and GPT: +10 points on Terminal-Bench 2.1 and +10 points on SWE-rebench. Ten points on 20 tasks is two extra successes on each benchmark. - Claude Code, both models: +5 points on SWE-rebench. Five points on 20 tasks is one extra success per model. - Claude Code also gains on GAIA with DeepSeek and on Terminal-Bench 2.1 with GPT. The paper names those gains and does not print the size. The paper's reading is that procedural guidance depends on the model and on how that model sits in the harness. A Gene that helps Codex with DeepSeek on a terminal task is not, in these cells, a uniform lift. ### Token use on the same pairs Figure 4 reports recorded task-execution tokens. The paper prints averages in the form 1,641.1K, and it computes the headline reduction from total usage across pairs rather than by averaging each pair's own percentage. | Slice | Pairs | Average without | Average with | Reduction | | --- | --- | --- | --- | --- | | All selected pairs | 320 | 1,641.1K | 1,243.3K | 24.24% | | Successful both ways | 265 | 1,278.2K | 1,053.6K | 17.57% | The ratio of the first pair of averages is (1,641.1 − 1,243.3) / 1,641.1 = 24.24%. The second is (1,278.2 − 1,053.6) / 1,278.2 = 17.57%. Tokens fall in all four harness–model pairings. The stated range is 13.55% to 35.96%. The paper names the two largest: Codex with DeepSeek at 35.96%, and Claude Code with DeepSeek at 25.12%. It does not print the Claude Code–GPT or Codex–GPT percentages. The 265-pair cut is the cleaner efficiency claim, because those tasks were already solved without Evolver. The shorter trace is not an artifact of skipping a failed attempt. Two cases in Figure 5 keep the pass and shrink the trace. | Case | Setting | Token cut | Model calls | | --- | --- | --- | --- | | Count tokens in a dataset subset | Terminal-Bench 2.1, Claude Code, GPT | 86.5% | 112 to 17 | | Fix a null-pointer exception in RxJava's ObservableSwitchMap | SWE-bench Multilingual, Codex, DeepSeek | 58.2% | 86 to 52 | Both executions succeed on each side. The paper presents them as illustrations, not as the average. ### What the public boards measure Checked November 7, 2026, [Terminal-Bench](https://www.tbench.ai/) presents version 4.0. The top displayed row is Claude Code with Opus 5.5, max effort, 64.85% accuracy, 214 successes in 330 trials, result date September 22, 2026, with a 95% half-width of 3.11 points. The next displayed row is Claude Code with Sonnet 5.5 at 61.82%. The site's [November harness benchmark](/reports/agent-harness-benchmark-november-2026) reads that same 4.0 board. Evolver's terminal number is a different object: version 2.1, 20 diagnostic tasks per configuration, models deepseek-v4.1-flash and gpt-5.6-sol, and a paired with-and-without design. 71 of 80 is not a claim about the 4.0 board. [SWE-bench](https://www.swebench.com/) on the same day embeds a multilingual list whose highest resolved rate is 72.7%, Gemini 3 Flash with mini-SWE-agent, dated February 13, 2026, with the submission path marked multilingual. The next rows in that embedded list are 72.0% and 70.7%. The official multilingual set has 300 instances. Evolver's 96.25% is 77 of 80 on a diagnostic slice of 20 tasks per configuration, and it is the same with or without Genes. It does not enter the 300-instance board, and the flat result is the paper's own finding for that slice. The closest same-benchmark numbers already on this site are in the November 3 report [You don't need long context](/research/you-dont-need-long-context). That report runs Terminal-Bench 2.1 as 89 tasks and SWE-bench Multilingual as 300, inside mini-swe-agent 2.4, and studies context compaction. Its GPT 5.6 Sol full-history scores are 87.6% on Terminal-Bench 2.1 and 79.3% on SWE-bench Multilingual. Those are that report's runs. They use a different harness, the full task sets, and no Gene injection. They are the reminder that a percentage on Terminal-Bench 2.1 depends on which tasks were scored and which harness held the model. ## How the reuse loop works Evolver separates three jobs. Something extracts a strategy from an old trace. Something stores it as an asset with a lifecycle. Something hands a small selection to the harness that is about to run. The model that acts is whichever model the harness was already calling. ### What gets stored The paper writes a Gene as a five-part record. One part summarizes the experience. One lists the steps. One states when the strategy applies. One holds signals for matching a later task. One is an optional plan for checking the result. The matching signals are how retrieval decides relevance. The applicability text tells the agent the condition. The paper is explicit that neither field is a gate the runtime must enforce. Around the Gene, a Capsule keeps a worked example, an AntiGene keeps a negative lesson, and separate records keep events, provenance, and review. The asset body is content-addressed and append-only. Quarantine, approval, rejection, and revocation change a side record. They do not rewrite the strategy text. Automatically drafted hints, and candidates produced by a model distillation, enter quarantine. The default drafting path pulls concise hints from assistant utterances and marks them unproven. A heavier distillation path, when enabled, can also emit cases and negative guidance. ### How a harness sees it On a later task, retrieval can run at session start, as a bounded set of locally eligible Genes, or at prompt time by matching the task text to stored signals. An MCP tool can fetch an asset on demand. A separate heuristic ranks candidates by task relevance, a quality signal, and historical reuse. The pathways do not share one ranking function or one context budget. The paper's point is that the host harness still reasons, calls tools, and checks its own work. The Gene arrives as context, not as a new policy and not as a new tool implementation. The repository README, read November 7, 2026, documents a hook installer for this idea on the default branch: `evolver setup-hooks` with a platform flag for Claude Code and another for Codex. That README is ahead of the v2.0.33 pin used in the experiments. It describes the integration. It is not a log of the 320 runs. EvoMap, at the URL in the paper's footnote, is the networked version of the same assets: publish a Gene, discover one written elsewhere, and carry provenance with it. The homepage loaded on November 7 and describes the Genome Evolution Protocol in those terms. The measured study does not use that network. Its pools are local, isolated, and frozen before the second run. ### What the paired protocol actually does The three stages are the whole evaluation. 1. Baseline. The harness runs the task with its own tools and skills. Nothing from Evolver is injected. 2. Extraction. A model matched to that run distills Genes from the valid baseline traces, successes and failures together. The pool is private to that batch, benchmark, harness, and model. The candidates are then frozen. 3. Gene-augmented run. The environment is restored. A new session sees the original task plus selected Genes, up to 12,000 bytes, with a same-task Gene preferred. Both sides use the benchmark's native grader. Runs that a timeout or a token limit interrupts are dropped, and so are runs without a complete token record. The authors also say the final choice of tasks takes the observed improvements and the token totals into account. That sentence is the reason the set is diagnostic. ## What the experiments controlled Names and numbers in this section are printed in the November 5 paper unless a link says otherwise. The software pin is Evolver v2.0.33. The harnesses are Claude Code and Codex, unnamed as to release. The models are the configurations deepseek-v4.1-flash and gpt-5.6-sol. The paper cites DeepSeek's CoRR paper on DeepSeek-V4.1-Flash and OpenAI's GPT-5.6 post for those names. It does not print a temperature, a reasoning effort, or a dollar price. Each benchmark contributes the same 20 tasks to every harness–model pair. The starting point is the first 20 tasks in the benchmark's original order. A task that yields an empty response, because the output was malformed, the run timed out, or execution failed, is replaced by the next task in order. After that replacement, the analyzed table still requires a valid outcome and a full token record on both the baseline and the augmented side. Interrupted runs are out. The grader is the benchmark's own. Terminal-Bench 2.1 is terminal engineering work. SWE-bench Multilingual and SWE-rebench are software-issue resolution. GAIA is multi-step information seeking and reasoning. The paper cites Merrill et al. for Terminal-Bench, Yang et al., SWE-smith, for the multilingual set, Badertdinov et al. for SWE-rebench, and Mialon et al. for GAIA. Experience extraction uses a model configuration matched to the evaluation setting, so a DeepSeek run is not distilled by a different family and then tested as if the hint came from nowhere. The augmented session does not continue the baseline conversation. The paper says the environment is restored and the session is fresh. Whatever the Gene carries has to fit in 12,000 bytes. The token metric is recorded input plus recorded output on those final executions. Extraction, review, and any other overhead are outside the 24.24% and the 17.57%. Model-call counts appear in the two case studies and are not given as a study-wide total. On November 7, 2026, the GitHub API reported the repository public, under GPL-3.0, with the default branch last pushed on November 5, 2026. Tag v2.0.33 resolves to an annotated tag created September 6, 2026 by the project's release bot. Release v2.0.49 was published November 5, 2026 at 04:36:20 UTC. Anyone who installs the current release is not installing the pin in the tables. ## Limits that change how to read the scores The conclusion is short, and the setup section already contains the constraint that matters most. The 320 pairs were kept because they had outcomes and token records, and the authors say the selection also looks at which runs improved and how many tokens they used. That is a diagnostic slice. It can show that a Gene injection is capable of raising success and cutting tokens on tasks chosen with those outcomes in view. It cannot show the rate a team would get by taking the next 20 tasks in order and stopping there. The public boards above are the full published sets, or the current version of them, and they answer a different question. The Genes are taken from the baseline of the task being re-run, when such a Gene exists. The agent is being reminded of a procedure extracted from an earlier attempt at the same problem, plus other frozen hints from the same small pool. That is a real system, and it is a weaker claim than "the agent learned a skill and used it on a new problem." The authors put the stronger claim in future work. Success does not move in seven of sixteen configurations, and the multilingual slice does not move at all. The Terminal-Bench aggregate and the single Codex–DeepSeek cell carry a large share of the headline. A harness owner who reads only +4.38 points will miss that the gain is conditional. The token percentages are the cost of the graded attempt, not the cost of becoming experienced. Distilling the Genes is extra model use, and the paper excludes it. The 17.57% figure is the part that survives even when both attempts already pass, which is the number to quote when the question is efficiency on solved work. The paper names Evolver v2.0.33 and does not name the Claude Code or Codex build. The public project has moved: v2.0.49 shipped the morning of the paper's arXiv publication, about three hours before the Atom timestamp. Results for the September pin are not results for the November release. The README on the default branch points its research discussion at the earlier Gene paper and does not yet point at this one. Trust machinery is real in the design and uneven in enforcement. Quarantine, probation, and an untrusted mark for imported network assets are described. The paper also says some direct retrieval paths do not apply the same checks, and that applicability text does not bind the tool caller. The experiment sidesteps the network by freezing a private pool. A deployment that turns on EvoMap discovery is outside these 320 pairs. ## Read the paper The November 5 paper, the repository it names, and the boards a reader might otherwise confuse with its tables: - [Evolver on arXiv](https://arxiv.org/abs/2611.07595) - [HTML](https://arxiv.org/html/2611.07595) - [PDF](https://arxiv.org/pdf/2611.07595) - [EvoMap/evolver](https://github.com/EvoMap/evolver) - [EvoMap](https://evomap.ai/) - [Strategy genes, arXiv:2604.15097](https://arxiv.org/abs/2604.15097) - [Terminal-Bench](https://www.tbench.ai/) - [SWE-bench](https://www.swebench.com/) - [Claude Code](https://github.com/anthropics/claude-code) - [Codex](https://github.com/openai/codex) Related pages on this site: [agent harnesses hub](/topics/harnesses), [Best AI agent frameworks](/best/ai-agent-frameworks), [Harness comparison tracker](/trackers/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [research](/topics/research), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [agent skill](/glossary/agent-skill), [model context protocol](/glossary/model-context-protocol), [benchmark](/glossary/benchmark), [behavioral evaluation](/glossary/behavioral-evaluation), [HarnessFlow](/research/harnessflow-mcts-harness-optimization), [SpecCompact](/research/speccompact-agent-context-compaction), [You don't need long context](/research/you-dont-need-long-context), [ReAct](/research/react-reasoning-and-acting), the [November harness benchmark](/reports/agent-harness-benchmark-november-2026), [Claude Code](/directory/claude-code), and [Codex](/directory/codex). ## HarnessFlow MCTS harness optimization: 69.48% average URL: https://www.aiagentsinfo.com/research/harnessflow-mcts-harness-optimization Published: 2026-11-06. Updated: 2026-11-09 Paper: HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents Paper URL: https://arxiv.org/abs/2611.06254 HarnessFlow MCTS harness optimization, arXiv November 4, 2026: with 15 refinement attempts per benchmark, the average is 69.48%, 10.89 points above Vanilla and 3.95 points above Meta-Harness. The comparison table is on this page. ## How it compares with Meta-Harness and hand-built harnesses The paper's comparisons are internal. Vanilla is the benchmark's own reference agent and the root of the search tree. [Meta-Harness](https://arxiv.org/abs/2603.28052) is rerun with the same initial harness, the same coding agent, the same 15-candidate budget, and the same grader. [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [OpenHands](https://github.com/OpenHands/OpenHands), and [Hermes](https://github.com/NousResearch/hermes-agent) are official releases, pinned, with the same backbone, the same benchmark tools, and the same per-task limits. No number in the tables is taken from a vendor leaderboard. On November 6, 2026, those four repositories were public. Claude Code's repository describes an agentic coding tool in the terminal. Codex's describes a lightweight coding agent in the terminal. OpenHands's describes AI-driven development. Hermes's short description is "The agent that grows with you." The pins in the paper are older than a repository's default branch may be: Claude Code 2.1.272, Codex 0.154.0, OpenHands cloud-1.45.0 (SDK 1.34.0), and Hermes v2026.9.7. The [harness guide](/blog/what-is-an-ai-agent-harness) on this site, updated November 6, 2026, uses a wider product definition for Claude Code and Codex and already summarizes Meta-Harness's own March results. Those March results used other tasks. They are not a row below. Meta-Harness, in the March 30 paper, is an outer loop whose proposer reads a filesystem of prior code, scores, and traces and is not given a parent-selection rule. HarnessFlow's claim is about the organization of the search: a tree, an explicit parent policy, and a library of reusable mechanisms. The scores that follow are what that change did inside HarnessFlow's protocol. ### Mean scores on the six benchmarks Table 1 is the mean test score in percent. Each cell except the average is the mean of five runs, plus or minus the sample standard deviation. Bold in the paper marks the highest mean. The average has no standard deviation. TAU2 is TAU2-bench. BC+ is BrowseComp-Plus. SWE-V is SWE-bench Verified. BCB-H is BigCodeBench-Hard. | Method | TAU2 | AppWorld | WebShop | BC+ | SWE-V | BCB-H | Avg. | | --- | --- | --- | --- | --- | --- | --- | --- | | Vanilla | 63.98 ± 2.15 | 72.00 ± 1.86 | 47.79 ± 1.14 | 72.40 ± 1.85 | 67.10 ± 0.89 | 28.30 ± 0.76 | 58.59 | | Claude Code | 73.12 ± 1.14 | 70.86 ± 2.00 | 56.98 ± 1.19 | 73.20 ± 1.72 | 65.30 ± 1.30 | 30.80 ± 2.61 | 61.71 | | OpenHands | 71.83 ± 1.59 | 66.29 ± 1.32 | 65.54 ± 1.77 | 74.00 ± 1.97 | 67.20 ± 1.64 | 30.80 ± 2.86 | 62.61 | | Hermes | 72.90 ± 2.17 | 74.19 ± 1.03 | 64.50 ± 1.34 | 74.30 ± 1.30 | 68.10 ± 2.53 | 31.70 ± 1.15 | 64.28 | | Codex | 75.91 ± 2.98 | 73.24 ± 1.97 | 66.15 ± 1.49 | 74.70 ± 2.36 | 68.70 ± 2.11 | 30.20 ± 1.10 | 64.82 | | Meta-Harness | 74.62 ± 2.36 | 76.76 ± 2.94 | 66.33 ± 0.61 | 76.60 ± 1.02 | 68.80 ± 1.15 | 30.10 ± 1.66 | 65.53 | | HarnessFlow | 81.83 ± 2.39 | 80.76 ± 1.33 | 69.49 ± 0.88 | 76.90 ± 1.14 | 75.40 ± 1.19 | 32.50 ± 1.77 | 69.48 | The paper says HarnessFlow exceeds Vanilla, Codex, and Meta-Harness by 10.89, 4.66, and 3.95 points on average. Those differences are 69.48 minus 58.59, 64.82, and 65.53. The largest gains over Vanilla are WebShop, 69.49 versus 47.79, and TAU2-bench, 81.83 versus 63.98. The largest gains over Meta-Harness are TAU2-bench, 81.83 versus 74.62, and SWE-bench Verified, 75.40 versus 68.80. The hand-built ranking is not the same on every column. Codex has the highest hand-built average, 64.82. Hermes is higher than Codex on AppWorld, BrowseComp-Plus, and BigCodeBench-Hard. Claude Code's average, 61.71, is the lowest of the four production harnesses. On AppWorld, Claude Code at 70.86 and OpenHands at 66.29 are both below Vanilla at 72.00. On SWE-bench Verified, Claude Code at 65.30 is below Vanilla at 67.10. A production harness, on this backbone and these pins, is not automatically ahead of the benchmark's own agent. Two leads are narrow next to the printed standard deviations. BigCodeBench-Hard is 32.50 ± 1.77 for HarnessFlow and 31.70 ± 1.15 for Hermes. BrowseComp-Plus is 76.90 ± 1.14 and 76.60 ± 1.02. The paper marks HarnessFlow as the highest mean in both columns. The TAU2-bench and SWE-bench Verified gaps over Meta-Harness are several times the printed standard deviations. The official metrics, all scaled to percent, are task success for TAU2-bench and AppWorld, task score for WebShop, answer accuracy for BrowseComp-Plus, resolution rate for SWE-bench Verified, and Pass@1 for BigCodeBench-Hard. ### Search efficiency Figure 3(a) is the validation set, not the test set. The solid curve is the mean candidate score. The dashed curve is the mean, across benchmarks, of the best score seen so far. Round 0 is the shared initial harness. After three candidate evaluations per benchmark, the mean best validation score is 71.5%. By round 15 it is 74.2%. On four of the six benchmarks, the paper says the final best is first reached at round 10 or later. It does not name which four. It does say those later winners refine an earlier retained harness rather than only the candidate from the previous round. The test average of 69.48% is a different number, measured once on the held-out split of the harness that won validation. The SWE-bench Verified lineage in Figure 4 makes one path concrete. Edges 0 → 2 → 4 → 8 → 10 move the validation score from 67.60% to 79.20%. Along that path the paper describes repository-history retrieval, interpreter compatibility, symbol-level retrieval, and compaction of large submission patches. The caption says the scores are cumulative designs, not the isolated effect of each edit. The TAU2-bench appendix figure raises validation from 59.35% to 81.30%, and the caption names action reviews, structured evidence, and troubleshooting continuation along that lineage. ### What the selected harness costs to run Figure 3(b) plots mean test score against the geometric mean of per-task execution cost, which down-weights BrowseComp-Plus. HarnessFlow and Meta-Harness are the Pareto frontier at $0.101 and $0.092. Meta-Harness is the cheaper of those two points. The paper says the hand-engineered harnesses cost two to three times as much, at lower scores. Search cost is not in the plot. The appendix also prints an arithmetic average, which BrowseComp-Plus pulls upward. The introduction's 41.4% claim is this average, not the geometric mean: HarnessFlow at $0.232 divided by Codex at $0.561. Prices are the executor's list price for GPT-5.4-mini, $0.75 per million input tokens and $4.50 per million output tokens. Bold in the paper marks the cheapest cell in a column. | Method | TAU2 | AppWorld | WebShop | BC+ | SWE-V | BCB-H | Avg. | | --- | --- | --- | --- | --- | --- | --- | --- | | Vanilla | 0.067 | 0.132 | 0.030 | 2.292 | 0.181 | 0.007 | 0.451 | | Claude Code | 0.177 | 0.383 | 0.191 | 1.444 | 0.456 | 0.024 | 0.446 | | OpenHands | 0.148 | 0.286 | 0.128 | 1.365 | 0.445 | 0.017 | 0.398 | | Hermes | 0.215 | 0.350 | 0.167 | 1.507 | 1.050 | 0.043 | 0.555 | | Codex | 0.128 | 0.481 | 0.134 | 2.027 | 0.569 | 0.025 | 0.561 | | Meta-Harness | 0.075 | 0.100 | 0.023 | 1.653 | 0.249 | 0.008 | 0.351 | | HarnessFlow | 0.116 | 0.109 | 0.022 | 0.948 | 0.172 | 0.024 | 0.232 | Read by column, HarnessFlow is the cheapest method on WebShop, BrowseComp-Plus, and SWE-bench Verified. It is not the cheapest on TAU2-bench, where Vanilla is $0.067 and Meta-Harness is $0.075, or on AppWorld, where Meta-Harness is $0.100, or on BigCodeBench-Hard, where Vanilla is $0.007, Meta-Harness is $0.008, and OpenHands is $0.017. The paper's claim is a lower arithmetic average than every hand-engineered harness, and a place on the Pareto frontier of each benchmark, which allows a higher cost when the score is higher. The hand-engineered arithmetic range in the text is $0.398 to $0.561. ### The same search with GPT-5.4 as the executor Table 4 reruns the comparison with GPT-5.4 as the model inside the harness. For HarnessFlow and Meta-Harness, search is started again from the same initial harness, with this executor. Cost uses a different list price, $2.50 per million input tokens and $15.00 per million output tokens. Only three benchmarks are in the table. Rows in the paper are ordered by average score. | Method | TAU2 | TAU2 $ | AppWorld | AppWorld $ | WebShop | WebShop $ | Avg. | Avg. $ | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Vanilla | 83.76 ± 2.83 | 0.23 | 82.48 ± 1.09 | 0.38 | 56.29 ± 1.20 | 0.08 | 74.18 | 0.23 | | Codex | 85.91 ± 1.59 | 0.47 | 75.52 ± 2.37 | 0.57 | 73.59 ± 0.89 | 0.38 | 78.34 | 0.47 | | Hermes | 85.81 ± 1.73 | 0.70 | 79.43 ± 1.36 | 0.62 | 72.76 ± 1.43 | 0.47 | 79.33 | 0.60 | | OpenHands | 83.12 ± 3.64 | 0.48 | 83.71 ± 1.32 | 0.52 | 71.62 ± 0.59 | 0.41 | 79.48 | 0.47 | | Claude Code | 82.69 ± 1.76 | 0.61 | 87.81 ± 1.92 | 0.81 | 72.06 ± 1.36 | 0.59 | 80.85 | 0.67 | | Meta-Harness | 84.59 ± 1.12 | 0.24 | 88.41 ± 0.73 | 0.20 | 72.12 ± 0.51 | 0.09 | 81.71 | 0.18 | | HarnessFlow | 88.06 ± 1.92 | 0.31 | 90.95 ± 0.95 | 0.25 | 71.72 ± 0.66 | 0.08 | 83.58 | 0.21 | HarnessFlow's average, 83.58, is the highest of the seven. The paper says that is 2.73 points above Claude Code and 1.87 points above Meta-Harness. The wins are TAU2-bench, 88.06, and AppWorld, 90.95. WebShop is not a win. Codex is 73.59, Hermes 72.76, Meta-Harness 72.12, Claude Code 72.06, HarnessFlow 71.72, and OpenHands 71.62. Average cost is $0.21 for HarnessFlow and $0.67 for Claude Code. Meta-Harness is cheaper on that average, at $0.18. The paper's sentence on cost in this table is the comparison with Claude Code. ## How the search works HarnessFlow separates two jobs. A selection policy chooses which harness to refine. A coding agent chooses how to change the code. The model inside the harness stays the one named in the experiment. The coding agent is a different model: Codex CLI 0.154.0, running GPT-5.5 at the highest reasoning effort. It can read the search history. It cannot see the test tasks. ### Selection After some attempts, the search holds a tree. The root is the initial harness. Each later node is a harness, its validation score, and its execution records. An edge means the child was produced by editing the parent. Scores already measured are not rewritten when the tree grows. At the next attempt, the parent is drawn from a shortlist of up to five highest-scoring nodes. Ties break by creation order. Inside that shortlist, one term is a softmax over validation scores. A temperature, set from the scores' own standard error and clipped between 0.005 and 0.100, controls how hard the softmax concentrates on the leader. If no shortlisted node can supply that error, the temperature is 0.08. A second term prefers nodes with fewer direct children. Direct children, not the size of the whole subtree, are what get counted. The mixture used in the main experiments puts weight 0.65 on the score term and 0.35 on the exploration term. The coding agent may still read code and evidence from nodes that were not selected. The shortlist restricts the parent, not the archive. ### Expansion The coding agent receives the selected parent, the archive, and a read-only snapshot of the Harness Component Library. It writes a new self-contained harness and a record of the edit. The library is optional reference code, not a menu the search is required to stay inside. The agent is told to copy what it uses, and that code's local dependencies, into the candidate. It is told not to import the snapshot at runtime. The archive holds earlier implementations, the edits, validation feedback, task-level traces, and which harness was the parent of which. The library in the experiments has 21 benchmark-neutral Python modules. They cover observation compaction and skill catalogs, note stores and logs, loop and workflow runners, and checks on tool arguments, plus shared support for verification, self-review, and recovery. The modules depend only on the Python standard library. Model calls and tool execution come from the host through a small adapter. ### Evaluation and backpropagation A candidate is scored on the validation split with five repetitions. The score is the average of the benchmark metric. The traces are kept so a later edit can see what failed. If the candidate passes a runtime compatibility check and the evaluation yields a score, it is added to the tree under its parent, the parent's child count goes up by one, and the archive stores the change in score relative to the parent. Improving and non-improving edits are both stored. A candidate that fails the check still uses up one of the 15 attempts. It is not added to the tree. After 15 attempts, the harness with the highest validation score in the tree is evaluated on the test set. That test score is the number in Table 1. The protocol is sequential and budgeted. It is not a claim that the search found the best harness in an open-ended space. ### What the ablations change The library ablation repeats the search without the component library, on TAU2-bench, AppWorld, and BigCodeBench-Hard only. The average falls from 65.03 to 63.39. The paper says the drop is primarily TAU2-bench, 81.83 to 77.64, with small changes on the other two: AppWorld 80.76 to 80.63, and BigCodeBench-Hard 32.50 to 31.90. Without the library, the three-benchmark average is still 8.63 points above Vanilla's 54.76. Starting from the OpenHands agent, instead of the benchmark's reference harness, still improves all three of those benchmarks. The average is 62.61, from TAU2-bench 76.02 ± 2.42, AppWorld 79.62 ± 0.92, and BigCodeBench-Hard 32.20 ± 1.96. The paper's reading is that the gains are not tied to one starting architecture. That 62.61 is this three-benchmark ablation, not the OpenHands row in Table 1, which prints the same average on the full set of six. The exploration ablation keeps the top-five shortlist and the budget of 15, and changes only the mixture weight, on TAU2-bench, AppWorld, and SWE-bench Verified. | Benchmark | Weight 0 | Weight 0.35 | Weight 1 | | --- | --- | --- | --- | | TAU2-bench | 74.19 ± 2.77 | 81.83 ± 2.39 | 72.80 ± 2.10 | | AppWorld | 78.76 ± 2.47 | 80.76 ± 1.33 | 79.81 ± 0.80 | | SWE-bench Verified | 68.83 ± 2.57 | 75.40 ± 1.19 | 68.33 ± 0.29 | | Average | 73.93 | 79.33 | 73.65 | Weight 0 uses only the score softmax. Weight 1 favors parents with fewer direct children. The default, 0.35, is the highest mean on all three benchmarks. The paper states the gaps as 5.40 points over pure exploitation and 5.68 points over pure exploration. The movement is larger on TAU2-bench and SWE-bench Verified than on AppWorld. AppWorld at weight 1 is 79.81, close to the default 80.76. ## What the experiments controlled Names and numbers in this section are printed in the November 4 paper unless a link says otherwise. Each benchmark is split once, with seed 42. TAU2-bench is 92 validation tasks and 186 test tasks. AppWorld is 105 and 210. WebShop, BrowseComp-Plus, SWE-bench Verified, and BigCodeBench-Hard are 100 and 200. BigCodeBench-Hard samples tasks from that release with its published difficulty criteria. The same split and the same official grader are used for every method. The test split is not an input to search. The initial harness is the benchmark's reference agent, with the model client swapped for a shared gateway so that Vanilla, every candidate, and every baseline call the same model. The paper lists the upstream revisions: TAU2-bench LLMAgent at `a2c0247`, the AppWorld ReAct experiment harness at `66ad809` (v0.1.3.post1), the WebShop ReAct scaffold at `a5c0827`, BrowseComp-Plus `search_agent/openai_client.py` at `0469490`, mini-SWE-agent DefaultAgent at `04d809c` for SWE-bench Verified, and `bigcodebench.generate` at v0.1.4. The production agents run through their official non-interactive entry points, with their default tools, skills, and subagents. All methods share a network-isolated sandbox whose only egress is the model gateway, and they reach benchmark tools through a benchmark-specific tool server. Each agent is given the task description and the benchmark's input and output conventions. Wall-clock deadlines and step limits are the same for every method. TAU2-bench allows 200 steps and 1,800 seconds. AppWorld allows 50 interactions and 1,800 seconds. WebShop allows 30 actions and 900 seconds. BrowseComp-Plus allows 100 tool calls and 1,800 seconds. SWE-bench Verified allows 250 model calls and 2,400 seconds. BigCodeBench-Hard allows 30 iterations and 1,800 seconds. Auxiliary models are fixed per benchmark and shared. The TAU2-bench user simulator is GPT-5.2, and its evaluator is GPT-5.5. The BrowseComp-Plus grader is GPT-5.5 with the upstream grading template. The other benchmarks use programmatic graders. Sampling temperature for the executor is the API default. The GPT-5.4 table changes the executor only. AppWorld's starting harness is the benchmark's ReAct experiment harness. That is a historical thread on this site as well. The [ReAct](/research/react-reasoning-and-acting) note covers Yao et al. 2022, which the HarnessFlow paper cites, including the WebShop setting. HarnessFlow's WebShop root is a later ReAct scaffold, revision `a5c0827`, not the 2022 prompt. The scores in Table 1 are not a comparison with the 2022 paper. ## Limits that change how to read the scores The limitations section of the paper is short, and the tables add constraints the abstract does not. The search needs a validation score, so the setting is tasks with an automatic grader. Interactive work that a person still has to judge is outside the protocol. The 15 graded candidates are a one-time cost. The cost plots and the $0.232 figure leave that cost out. A reader who budgets only the serving number is not budgeting the search. The structure is a tree. A child has one parent. The paper says the natural extension is a graph, so a candidate could combine mechanisms found on different branches, and it leaves that undone. The SWE-bench Verified story is a single lineage of cumulative edits. It is not evidence that each named edit would have helped on its own, which the figure caption states. The arithmetic cost average and the geometric-mean plot answer different questions. BrowseComp-Plus costs about $0.95 to $2.29 per task, against a few cents on BigCodeBench-Hard. The 41.4% sentence is the arithmetic comparison with Codex. On the geometric-mean frontier, Meta-Harness is the cheaper point. On several individual benchmarks, Vanilla or Meta-Harness is cheaper than HarnessFlow at a lower score. The GPT-5.4 table is three benchmarks, and WebShop reverses the ranking against Codex. Transfer to a stronger executor is real on the printed average and is not a win on every column. The main table's smallest margins, BigCodeBench-Hard and BrowseComp-Plus, should be read with the standard deviations next to them. The reproducibility statement says the implementation and the scripts for the tables and figures will be released. The arXiv record, checked November 6, 2026, did not link a repository. Until that code is public, the tables are the authors' report of their own runs. The pinned harnesses in the table are product runtimes. They are not the open-source framework repositories whose latest GitHub releases, pulled 5 November 2026, are listed in [AI agent framework release, November 2026](/reports/ai-agent-framework-github-stats-november-2026). ## Read the paper The November 4 paper, the March harness optimizer it reruns, and the four pinned harnesses: - [HarnessFlow on arXiv](https://arxiv.org/abs/2611.06254) - [HTML](https://arxiv.org/html/2611.06254) - [PDF](https://arxiv.org/pdf/2611.06254) - [Meta-Harness](https://arxiv.org/abs/2603.28052) - [Meta-Harness project page](https://yoonholee.com/meta-harness/) - [Claude Code](https://github.com/anthropics/claude-code) - [Codex](https://github.com/openai/codex) - [OpenHands](https://github.com/OpenHands/OpenHands) - [Hermes agent](https://github.com/NousResearch/hermes-agent) Related pages on this site: [agent harnesses hub](/topics/harnesses), [Harness comparison tracker](/trackers/harnesses), [Best AI agent frameworks](/best/ai-agent-frameworks), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [research](/topics/research), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness), [Best AI agent harness](/best/ai-agent-harnesses), [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools), [harness](/glossary/harness), [harness engineering](/glossary/harness-engineering), [behavioral evaluation](/glossary/behavioral-evaluation), [benchmark](/glossary/benchmark), [SpecCompact](/research/speccompact-agent-context-compaction), [ReAct](/research/react-reasoning-and-acting), [Claude Code](/directory/claude-code), and [Codex](/directory/codex). ## SpecCompact agent context compaction: 8.68× median URL: https://www.aiagentsinfo.com/research/speccompact-agent-context-compaction Published: 2026-11-05. Updated: 2026-11-09 Paper: SpecCompact: Accelerating LLM Agent Context Compaction with Slot-Parallel Semantic Speculative Verification Paper URL: https://arxiv.org/abs/2611.06876 SpecCompact agent context compaction, posted November 4, 2026: 2.04–5.63× on AppWorld and τ³, and an 8.68× median in a Hermes user study. The speed tables are on this page. ## What is SpecCompact agent context compaction? SpecCompact is a November 4, 2026 method that drafts an agent’s context summary faster. A Hermes user study in the paper measured an 8.68× median speedup. The harness guides that this compaction sits inside are on [Agent harnesses](/topics/harnesses). ## How it compares with methods in use today The paper's comparisons are internal. Full History keeps the transcript. Hermes and OpenClaw are the authors' structured-summary baselines, sharing a 2,048-token summary budget with SpecCompact. Target-self, used on the long-document benchmarks, is the target model writing each slot itself, with no drafter and no semantic check. This section puts those tables next to the OpenClaw and Hermes compaction practice that was public on November 5, 2026. No latency number below is taken from a live product dashboard. ### OpenClaw and Hermes as agent frameworks compact today [OpenClaw's compaction page](https://docs.openclaw.ai/concepts/compaction), read November 5, 2026, says a conversation that nears the context window has its older turns summarized, the summary is saved in the session transcript, and recent messages stay intact. The full history remains on disk. Built-in compaction summarizes the older history in one model request. A split turn adds a request for the turn prefix, and a failed safeguard quality audit adds one request per corrective attempt. Each request's conversation input is capped at 160,000 characters, which the page calls about 40,000 tokens, or less when the summarizer's own window is smaller. Auto-compaction is on by default. New configs default the mode to `safeguard`. Manual client-side compaction keeps a recent tail, with `keepRecentTokens` defaulting to 20,000. That 20,000 is the tail left verbatim. It is not the length of the summary, and it is not the 2,048-token budget in the paper. The six names in the paper are still the harness format. On November 5, 2026, OpenClaw main (`29142856`) builds the checkpoint in `summary-checkpoint-prompt.ts` as one block: Goal, Constraints & Preferences, Progress with Done, In Progress, and Blocked, Key Decisions, Next Steps, and Critical Context. The paper's slot contracts follow that schema, including the instruction to preserve exact file paths, function names, and error messages. The public concepts page describes when compaction runs. It does not print these headings. The headings live in the harness source, and the summarizer still fills them in a single request. [Hermes](https://github.com/NousResearch/hermes-agent), the repository the paper cites, is a different template and the same shape of work: one summarizer call. On November 5, 2026, main (`be765e31`) asks `agent/context_compressor.py` for a checkpoint with Historical Task Snapshot, Goal, Constraints & Preferences, Completed Actions, Active State, Blocked, Key Decisions, Errors & Fixes, Resolved Questions, Relevant Files, Critical Context, and Unverified / Working Hypotheses, plus a conditional Pruned Skills section. A later compaction updates the previous summary instead of starting from nothing. If the summary call fails, a deterministic fallback fills the same kind of headings from text it can extract locally. The developer guide in that repository says the agent compressor fires at 50% of the context window by default, and a gateway safety pass fires at 85%. Figure 2(a) in the paper draws a shorter Hermes sketch beside OpenClaw. This page uses the November 5 source for the live template, and the paper's tables for the Hermes-style baseline the authors timed. SpecCompact, in the paper, is a generation method. The experiments compact on a fixed cadence, every eight agent calls, rather than at the 50% or context-window triggers above. What changes is how the summary is written: six drafts, a small model proposing lines, and the target accepting or repairing them. ### Latency against Hermes-style and OpenClaw-style summaries Table 3 in the paper is 20 matched inputs per workload. Speedup is relative to Hermes. SpecCompact uses an additional drafter GPU. Times are seconds. | Target | Workload | Hermes | OpenClaw | SpecCompact | Versus Hermes | | --- | --- | --- | --- | --- | --- | | Gemma4-31B-IT | AppWorld Normal | 38.89 | 16.44 (2.37×) | 8.26 | 4.71× | | Gemma4-31B-IT | AppWorld Challenge | 39.12 | 18.70 (2.09×) | 9.14 | 4.28× | | Gemma4-31B-IT | τ³ | 36.21 | 13.19 (2.75×) | 6.43 | 5.63× | | Gemma4-31B-IT | LongBench, about 16k | 74.13 | 63.55 (1.17×) | 29.72 | 2.49× | | Qwen3.5-122B-A10B | AppWorld Normal | 18.03 | 7.87 (2.29×) | 7.49 | 2.41× | | Qwen3.5-122B-A10B | AppWorld Challenge | 16.98 | 8.69 (1.95×) | 7.55 | 2.25× | | Qwen3.5-122B-A10B | τ³ | 17.81 | 7.68 (2.32×) | 8.71 | 2.04× | | Qwen3.5-122B-A10B | LongBench, about 16k | 29.43 | 24.62 (1.20×) | 21.67 | 1.36× | The abstract's range, 2.04–5.63×, is the six AppWorld and τ³ cells. LongBench sits outside that sentence. On Gemma 4 the agent rows are 4.28–5.63×. On Qwen3.5 they are 2.04–2.41×. OpenClaw-style summarization is already 1.95–2.75× faster than Hermes on the agent rows, and SpecCompact is faster than that OpenClaw baseline in seven of the eight settings. The exception is Qwen3.5 on τ³. The paper gives three reasons the Qwen3.5 speedups are smaller. Its Hermes baseline is already quicker. The 0.6B drafter matches it less well: draft acceptance is 19.6 percentage points lower than with Gemma 4. And the authors' vLLM setup reuses less of the cached prefix on Qwen3.5, because that model's linear-attention layers also have to restore recurrent state. The paper does not print the acceptance rates themselves, only the 19.6 point gap. ### Whether the compacted harness still finishes the task Table 1 is task performance in percent. Full History does not compact. Bold in the paper marks the best compaction result, which is not copied here. Each cell in the paper also has a 95% confidence interval. The intervals are wide beside the gaps. Gemma 4's AppWorld Normal task-goal scores are 57.74 with interval [47.02, 67.86] for Hermes and 54.76 with interval [44.05, 65.48] for SpecCompact. The point differences below are the ones the paper states, plus the cells it prints. | Target | Method | Normal TGC | Normal SGC | Challenge TGC | Challenge SGC | τ³ success | | --- | --- | --- | --- | --- | --- | --- | | Gemma4-31B-IT | Full History | 62.50 | 46.43 | 53.24 | 41.73 | 52.08 | | Gemma4-31B-IT | OpenClaw | 58.93 | 44.64 | 46.52 | 33.09 | 47.50 | | Gemma4-31B-IT | Hermes | 57.74 | 39.29 | 47.24 | 32.37 | 50.00 | | Gemma4-31B-IT | SpecCompact | 54.76 | 37.50 | 43.17 | 28.06 | 47.50 | | Qwen3.5-122B-A10B | Full History | 40.48 | 26.79 | 20.62 | 8.63 | 67.92 | | Qwen3.5-122B-A10B | OpenClaw | 28.57 | 14.29 | 17.75 | 8.63 | 65.83 | | Qwen3.5-122B-A10B | Hermes | 28.57 | 14.29 | 14.39 | 7.19 | 63.75 | | Qwen3.5-122B-A10B | SpecCompact | 33.33 | 21.43 | 16.07 | 6.47 | 64.58 | TGC is task goal completion. SGC is scenario goal completion. Against Hermes, the paper says Gemma 4 loses 2.98 and 4.07 percentage points of TGC on Normal and Challenge, and 2.50 points of τ³ success. Qwen3.5 gains 4.76 and 1.68 TGC points on those splits and 0.83 points of τ³ success, and loses 0.72 points of Challenge SGC. The Qwen Normal SGC cell, which the paper's prose does not highlight, is 21.43 for SpecCompact and 14.29 for both Hermes and OpenClaw. Read by column, Full History is the highest score everywhere, tied with OpenClaw on Qwen Challenge SGC at 8.63. On Gemma 4, SpecCompact is the lowest compaction score in the table, tied with OpenClaw on τ³ at 47.50. On Qwen3.5 it is the best compaction score on Normal TGC, Normal SGC, and, versus Hermes only, on τ³. OpenClaw is ahead of it on Challenge TGC (17.75 versus 16.07), Challenge SGC (8.63 versus 6.47), and τ³ (65.83 versus 64.58). Faster compaction and a higher task score are not the same result. AppWorld here is the full Test-Normal and Test-Challenge sets. τ³ is three trials per task on an 80-task subset chosen at random, separately for each target. The paper cites that benchmark as Shi et al., τ-Knowledge, arXiv:2603.04370. ### LongBench and RULER Table 2 is the long-document check. LongBench numbers are ROUGE-L as printed. RULER numbers are the official matching scores, scaled to 100. Execution failures count as zero. Target-self is the control that matters for the method: the same slots, written by the target, with no drafter. | Target | Method | GovReport | QMSum | RULER 8k | RULER 16k | | --- | --- | --- | --- | --- | --- | | Gemma4-31B-IT | Full History | 28.36 | 24.34 | 91.72 | 92.12 | | Gemma4-31B-IT | Target-self | 25.05 | 18.35 | 71.00 | 71.02 | | Gemma4-31B-IT | SpecCompact | 22.28 | 17.52 | 66.50 | 68.55 | | Qwen3.5-122B-A10B | Full History | 31.16 | 22.98 | 90.92 | 90.92 | | Qwen3.5-122B-A10B | Target-self | 28.25 | 18.54 | 72.98 | 72.15 | | Qwen3.5-122B-A10B | SpecCompact | 27.66 | 18.05 | 74.23 | 72.60 | The paper's summary of this table: Qwen3.5 with SpecCompact is within 0.6 points of target-self on both LongBench sets and higher on both RULER lengths. Gemma 4 with SpecCompact is 0.83–4.50 points below target-self across the four settings. The large drop is compaction itself. Full history on RULER scores 90.92 to 92.12. The slot methods score 66.50 to 74.23. ### What the system optimizations add on top of parallel slots Parallel slots alone still repeat the long history and pay for a model call per line. Section 3.3 adds independent scheduling of the six streams, a shared prefix cache, draft buffering of several lines per drafter call, stopping a repair at the next line break, and direct handling of duplicates, over-long lines, markdown headings, and empty-slot markers such as `(none)`. Table 4 is that package, before and after, on 20 matched inputs. Both sides already generate slots concurrently. | Target | Workload | Before (s) | After (s) | Speedup | | --- | --- | --- | --- | --- | | Gemma4-31B-IT | AppWorld Normal | 20.17 | 8.26 | 2.44× | | Gemma4-31B-IT | AppWorld Challenge | 21.56 | 9.14 | 2.36× | | Gemma4-31B-IT | τ³ | 12.07 | 6.43 | 1.88× | | Qwen3.5-122B-A10B | AppWorld Normal | 9.55 | 7.49 | 1.28× | | Qwen3.5-122B-A10B | AppWorld Challenge | 11.43 | 7.55 | 1.51× | | Qwen3.5-122B-A10B | τ³ | 10.38 | 8.71 | 1.19× | The paper states the span as 1.88–2.44× for Gemma 4 and 1.19–1.51× for Qwen3.5. The after times match the SpecCompact agent rows in the latency table. The paper says the table measures the optimizations together, not one component at a time, and that the full task evaluation uses the original implementation and has not been repeated with all of them. ### The Hermes user study The deployment study is separate from Table 3. SpecCompact was integrated into the Hermes codebase and run with Gemma 4 on four RTX PRO 6000 GPUs. Three scenarios came from the authors' own workflows: coding, math, and multitask question answering. Hermes was run ahead of time and paused at a fixed snapshot. People triggered each method from that snapshot and picked which felt faster without being told the method. Compaction time was recorded in the background. Afterward the agent wrote a summary, and a separate blind preference compared the two summaries. The timed result is a median of 63.74 seconds for the Hermes path and 7.28 seconds for SpecCompact, with a median paired speedup of 8.68×. Dividing 63.74 by 7.28 does not produce 8.68, so the paired figure is its own statistic. The summary votes, 28 responses, were 11 for SpecCompact, 8 for Hermes, 8 similar, and 1 unable to judge. The paper does not print the count for the "which felt faster" question. ### A related compaction result, with different models The November 3 report covered on this site, [You don't need long context](/research/you-dont-need-long-context), also studies compaction as a harness choice. It measures a written handoff on Claude Opus 4.5 and GPT 5.6 Sol, against keeping the full history. Those scores are not a row in the tables above. SpecCompact is about how fast a structured slot summary can be produced, and what that does to AppWorld, τ³, LongBench, and RULER for two open target models. The earlier report is about whether a short live context can hold task accuracy for the closed models it tested. Both treat compaction as harness work. They do not share a baseline. ## What the experiments controlled Names and numbers in this section are printed in the November 4 paper unless a link says otherwise. ### Why the slots can be drafted in parallel The preliminary study uses 100 trajectories from LiveClawBench, six slots each. Monolithic generation writes all six slots in one output. Independent generation writes each slot from the same history and does not see the other slots. An LLM judge scores how well a slot preserves information from the full history. The score is 0 to 5, the sum of five binary checks. The paper does not name the judge model. Independent generation averages 4.73, monolithic generation 4.44. The share of slots scoring at least 4 is 93% and 85%. Reversing slot order, or the order in which independent slots are invoked, moves the mean by at most 0.03. That is the warrant for treating slots as parallel units. It is an adequacy judgment, not an AppWorld score. The six slots, taken from the OpenClaw schema, are Goal, Constraints and Preferences, Progress, Key Decisions, Next Steps, and Critical Context. The prompt in the paper's appendix tells the drafter it is writing one section of an OpenClaw-style summary, not answering the user. Progress uses Done, In Progress, and Blocked. Empty slots may return `(none)`. Exact file paths, function names, and error messages are supposed to be kept. Each request includes the history and the full schema, and asks for only one slot. ### How a line is accepted The drafter proposes the next line conditioned on the history, the slot instruction, and the lines already accepted in that slot. The target sees the same history and prefix and returns one judgment token. The acceptance score is the estimated probability of a positive judgment divided by the sum of the positive and negative estimates. Variants such as capitalization count, and a missing label is filled from the leftover probability mass by the rule in Section 3.2. The line is accepted at η = 0.7 or higher. Otherwise the target writes a replacement with the rejected line left out of the prompt, and drafting resumes from the new prefix. Verification is one token. Drafting and repair may emit up to 64 new tokens per request. A slot stops at 12 lines or 320 tokens. The assembled summary is held to 2,048 tokens, the same budget as the baselines. A summary past that budget raises an error. This is semantic checking, in the sense of SpecReason and DREAM-R, which the paper cites. The target can keep a line it would not have worded itself, if the line is supported by the history and consistent with the prefix. It is not token-level speculative decoding, which would require the target's own next tokens. ### The drafter The drafter is Qwen3-0.6B, fine-tuned separately for each target on slot text that the target itself generated. The paper notes that semantic speculation does not require a shared vocabulary, and that they train on text only, so they do not have to store the target's probabilities. One history produces six supervised examples, one per slot. Training and validation examples are separated by trajectory ID. The set is described as about 50K training data from AppWorld, τ³, and LiveClawBench. The paper does not say whether that 50K counts histories or the six-per-history examples. Each drafter trains for one epoch. LoRA is applied to all linear layers, rank 16, scaling factor 32, dropout 0. The optimizer is AdamW, learning rate 10⁻⁴, cosine decay, warmup ratio 0.05, weight decay 0, gradient clipping at 1.0. The per-device batch size is 2, with 16 gradient-accumulation steps, BF16, a maximum sequence length of 16,384, and random seed 42. Loss is computed only on the slot content. ### The agent and long-context protocol During agent evaluation the harness compacts every eight model calls. System messages, setup messages, and the current input stay. New messages pile up until the next compaction. AppWorld is scored with task goal completion and scenario goal completion on the full Test-Normal and Test-Challenge sets. τ³ is task success, three trials per task, on an 80-task random subset for each target. The paper describes AppWorld as task completion through code and application APIs, and τ³ as task completion through user interaction and tool use. LongBench uses GovReport and QMSum, reported as ROUGE-L. RULER uses all five task types at 8,000 and 16,000 tokens, with the official matching metrics. Latency and throughput use smaller subsets, 20 matched inputs per workload, rather than the full sets. Serving for the latency runs is vLLM on NVIDIA RTX PRO 6000 Blackwell GPUs. Gemma 4 runs in BF16 on one GPU. Qwen3.5 runs in FP8 on two GPUs. SpecCompact adds one GPU for the drafter and runs the six slots concurrently. The user-study machine is a different count: Gemma 4 on four of those GPUs, inside Hermes. ### What the discussion leaves open The limitations section says the work is aimed at the speed–accuracy trade-off and the system implementation, and that it does not give a consistent accuracy advantage over target-model summarization. The suggested next step is closer drafter–target alignment, with DREAM-R's reinforcement learning named as an example. The discussion also says slots do not all read the history the same way. Progress and Decisions draw on evidence spread through the context. Goal, Constraints, and Next Steps lean on the beginning or the end. The authors mention slot-conditioned sliding-window attention as a way to speed drafting further, and they leave it undone. Two related systems in the paper's related-work section attack the wait differently. [Slipstream](https://arxiv.org/abs/2605.08580) compacts while the agent keeps acting, and uses later steps to check the summary. [Parallel Context Compaction](https://arxiv.org/abs/2605.23296) summarizes blocks of the input at the same time. SpecCompact's parallelism is across summary slots, with a line-level accept or repair, rather than across time or across input blocks. ## Read the paper The November 4 paper, and the two harnesses it builds on: - [SpecCompact on arXiv](https://arxiv.org/abs/2611.06876) - [HTML](https://arxiv.org/html/2611.06876) - [PDF](https://arxiv.org/pdf/2611.06876) - [Hermes agent](https://github.com/NousResearch/hermes-agent) - [OpenClaw compaction](https://docs.openclaw.ai/concepts/compaction) Related pages on this site: [Agent harnesses](/topics/harnesses), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [HarnessFlow](/research/harnessflow-mcts-harness-optimization), [how agents work](/topics/how-agents-work), [You don't need long context](/research/you-dont-need-long-context), [ReAct: Synergizing Reasoning and Acting in Language Models](/research/react-reasoning-and-acting), [harness](/glossary/harness), and [agent](/glossary/agent). ## Paper breakdown: You don't need long context — short working memory still finishes the job URL: https://www.aiagentsinfo.com/research/you-dont-need-long-context Published: 2026-11-04. Updated: 2026-11-09 Paper: You don't need long context Paper URL: https://arxiv.org/abs/2611.05306 A November 3, 2026 technical report: compaction kept Claude Opus 4.5 and GPT 5.6 Sol near full-history accuracy while the live context stayed short. ## How it compares with methods in use today The report compares each short-context run with the same model and harness keeping the full history. It does not enter those numbers on a public leaderboard. This section puts the report next to three live sources. The authors' tables are unchanged. The compaction docs from Anthropic and OpenAI were re-read on November 9, 2026. The SWE-bench leaderboard page was opened the same day, and its highest embedded Verified rate was still 79.2%. **Against a growing transcript.** On SWE-bench Verified, 500 tasks, the full-history scores are 70.0% for Claude Opus 4.5 and 84.4% for GPT 5.6 Sol. Compaction, written as "total context limit / threshold" in tokens: | Setting | Opus 4.5 resolved | GPT 5.6 Sol resolved | | --- | --- | --- | | Full history | 70.0% | 84.4% | | 48,000 / 36,000 | 73.6% | 88.8% | | 32,000 / 24,000 | 72.0% | 85.0% | | 24,000 / 18,000 | 74.8% | 85.0% | | 16,000 / 12,000 | 68.8% | 84.6% | The only Opus cell under the full-history score is the tightest one, by 1.2 points. Every GPT 5.6 Sol cell in that table is at or above 84.4%. The authors' estimated optimized cost for the whole benchmark falls from 420.00 USD to 62.53 USD for Opus, and from 167.68 USD to 40.62 USD for GPT 5.6 Sol, at the 16,000 token setting. That estimate is defined in Appendix B. It is not a bill. SWE-bench Multilingual, 300 tasks, is flatter. Opus moves from 66.3% to a best of 68.0% at 48,000 / 36,000, and to 64.3% at 16,000 / 12,000. GPT 5.6 Sol moves from 79.3% to 80.0%, then to 77.7% at the tightest setting. The caption on that table says the Opus run is Claude Opus 4.5 High and the GPT run is GPT 5.6 Sol Medium. The SWE-bench Verified table does not repeat those effort labels. Terminal-Bench 2.1, 89 tasks, is where Opus gains the most in the long-horizon group: 47.2% full history, 52.8% at 32,000 / 24,000. GPT 5.6 Sol goes from 87.6% to 88.8% at 48,000 / 36,000, then to 85.4% at the two tightest settings. FRAMES is the clearest long-horizon loss. Full history scores 81.0% and 90.2%. Every compaction setting is lower, down to 77.3% for Opus and 86.9% for GPT 5.6 Sol. The report still treats the band as small next to the cost change. A reader who needs those questions answered correctly should not round that into "no loss." **Against one action per conversation.** Single-action compaction is the extreme. The model acts once, writes the state, and the next call starts clean. | Benchmark | Model | Full history | Best compaction in the table | One action per call | | --- | --- | --- | --- | --- | | SWE-bench Verified | Opus 4.5 | 70.0% | 74.8% (24,000 / 18,000) | 70.2% | | Terminal-Bench 2.1 | Opus 4.5 | 47.2% | 52.8% (32,000 / 24,000) | 51.8% | | SWE-bench Verified | GPT 5.6 Sol | 84.4% | 88.8% (48,000 / 36,000) | 82.4% | | Terminal-Bench 2.1 | GPT 5.6 Sol | 87.6% | 88.8% (48,000 / 36,000) | 84.3% | Median peak context under the one-action setting is 8,000 tokens on SWE-bench Verified for Opus and 7,000 for GPT 5.6 Sol. The full-history maximums on those two rows are 200,000 and 173,000 tokens. The one-action maximums are 16,000 and 15,000. **Against reading the whole document.** When the input itself is long, the harness puts the files on disk and lets the model search. On Needle in a Haystack, both models reach 100% and never compact. The full-document baselines are 60.0% and 80.0%. On AA-LCR, Opus moves from 65.0% to 84.0% at a 16,000 token limit, and GPT 5.6 Sol from 82.0% to 83.0%. On BABILong, the gains are 67.1% to 92.0% and 70.7% to 94.0%. The report's stated range for Opus on the long-input group is 0.4 points down to 40.0 points up. The 40 point jump is the needle test, 60.0% to 100%. **Against the compaction already in products.** Anthropic's context-window page, re-read November 9, 2026, says server-side compaction is the primary way to manage long agent conversations, and that accuracy falls as the token count grows. The separate threshold page still sets the default trigger at 150,000 input tokens, with 50,000 as the minimum. It now says threshold compaction is in beta on select models, and it adds rules for Claude 5.1 and later: a custom-instruction summary sees the visible conversation, and earlier thinking blocks are not part of that summary. OpenAI's GPT 5.6 Sol page lists a 1,050,000 token window. The card shows 4 USD per million input tokens, 0.40 USD cached input, and 20 USD per million output tokens. The page calls the 4 USD and 20 USD rates promotional through November 30, 2026. Prompts above 272,000 input tokens are priced at 2 times input and 1.5 times output for the whole request, which is 8 USD and 30 USD on those promotional rates. The 4 USD and 20 USD figures match the nominal rates the report cites from OpenAI documentation it accessed on September 29, 2026. The November 9 page adds the promotion end date and the multipliers. OpenAI's compaction guide says the carried item is opaque and not intended to be human-interpretable. The report is testing a tighter version of an idea both vendors already ship, and the handoff is text a person can read. It measures the authors' harness. The vendor compaction features are a separate implementation. Its Opus model is Claude Opus 4.5. Anthropic's 1 million token list on the November 9 page names Claude Fable 5.1, Claude Mythos 5.1, Claude Fable 5, Claude Mythos 5, Claude Opus 5.5, Claude Opus 5, Claude Opus 4.8, Claude Opus 4.7, Claude Opus 4.6, Claude Sonnet 5.5, Claude Sonnet 5, Claude Sonnet 4.6, Claude Haiku 5.5, and Claude Mythos Preview. Opus 4.5 does not appear on that list. The report's own reference window for Opus 4.5, from Anthropic docs it accessed on September 29, 2026, is 200,000 tokens. The November 9 page still places models outside that 1 million list, including deprecated Claude Sonnet 4.5, at 200,000 tokens. **Against the public SWE-bench leaderboard.** The [SWE-bench](https://www.swebench.com/) page, opened again on November 9, 2026, still embeds a Verified table whose highest resolved rate is 79.2%, for Claude 4.5 Opus with live-SWE-agent, dated December 15, 2025, and the same 79.2% for the Sonar Foundation Agent, dated December 5, 2025. The mini-SWE-agent row named "Claude 4.5 Opus (high)," version 2.0.0, dated February 17, 2026, is 76.8%. The newest dates in that embedded Verified table are still in February 2026. GPT 5.6 Sol does not appear. The report uses mini-swe-agent 2.4 and prints 70.0% as its own Opus full-history baseline. Those are different runs. A higher number in the report is a different harness, and the lower baseline is the authors' own full-history run. **Against smaller open models.** The negative result is part of the finding. On SWE-bench Verified, five local models: | Model | Full history | 48,000 / 40,000 | 32,000 / 24,000 | 16,000 / 12,000 | | --- | --- | --- | --- | --- | | Qwen 3.6 27B | 68.6% | 69.4% | 68.2% | 65.0% | | Qwen 3 Coder Next | 53.0% | 49.2% | 40.4% | 3.6% | | Gemma 4 31B | 65.2% | 65.0% | 65.8% | 58.0% | | Step 3.7 Flash | 70.0% | 66.8% | 64.6% | 56.6% | | MiniMax M2.7 | 72.6% | 73.0% | 73.2% | 67.8% | At the tightest budget every one of these five is below its own full-history score. Qwen 3 Coder Next is the collapse. The report's broad claim is about the frontier closed models it studied, plus a warning that a short window can still starve a model whose traces are long. ## What the experiments controlled Names and numbers below are the ones printed in the November 3 report unless a link says otherwise. **What "short" means here.** The report calls a context long when it is a large fraction of the windows common at the time of the study, which it puts at about 256,000 tokens or more. It calls a context short, in the regimes it emphasizes, when the limit is about 16,000 to 50,000 tokens. That is short relative to a million-token window. It is still a lot of text. **The three harnesses.** All three are the same agent loop. Full-history accumulation never deletes a turn. End-of-conversation compaction follows that loop until the length crosses a threshold, asks for a bounded handoff, and resumes in a new conversation. If the handoff does not finish before a higher total limit, the conversation is stopped. Single-action compaction gives each call the task and the current written state, takes one environment action, records the result, updates the state, and discards the conversation. The handoff prompt in the mini-swe-agent appendix tells the model to stop, not to run commands, and to record the objective, established facts, applied changes, verification, dead ends, and the remaining plan. The authors say the useful handoff is a brief for the work still to do, not a narrative of everything that happened. **Serving cost.** GLM 5.2 and Kimi K2.6 run on 16 H100 GPUs, with INT4 weights and BF16 activations. Kimi also uses an FP8 key-value cache. Qwen 3 Coder Next runs on 8 H100s in BF16. The setup prefills a long prefix, turns on prefix caching, and has each request generate 1,024 tokens. The highest throughput without key-value preemption is the number they keep. Repeated measurements varied by at most 2%. Figure 1 converts that throughput at 3.85 USD per H100-hour. The 7 times gap is Qwen at 256,000 tokens versus 16,000, not a retail price anyone is charged for Claude or GPT. **One action, and the model they did not train.** Section 5 asks whether a model could live entirely on a written state: read the state, act once, update the state, and let a fresh call continue. They name that idea a semantic state space model, to separate it from a neural state-space model whose state is a vector. The utility they claim for it is the cost of short contexts, already measured. The further claim, that such a model would be easier to train because it would not have to learn where to look in a long history, is marked as a surmise. No trained semantic state space model is in the report. ## Read the paper The report, the lab page it links, and the harness it starts from: - [You don't need long context](https://arxiv.org/abs/2611.05306) - [HTML version](https://arxiv.org/html/2611.05306) - [Deep Learning Efficiency Research](https://nv-dler.github.io) - [Local Frontier Intelligence](https://research.nvidia.com/labs/lpr/lfi/) - [mini-swe-agent](https://github.com/SWE-agent/mini-swe-agent) Related pages on this site: [harnesses](/topics/harnesses), [how agents work](/topics/how-agents-work), [task performance](/topics/task-performance), [context window](/glossary/context-window), [ReAct](/research/react-reasoning-and-acting), and [agents](/glossary/agent). ## Paper breakdown: Weight-space programming — a language model that reads and edits neural weights URL: https://www.aiagentsinfo.com/research/weight-space-programming Published: 2026-11-03. Updated: 2026-11-04 Paper: Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights Paper URL: https://rikiyat.github.io/blog/weight-space-programming/ MIT CSAIL note, November 2, 2026: GPT-6 Astra edited neural weights, with no training loop, and raised a small model's MATH-500 score from 7.9% to 48.1%. ## How it compares with methods in use today The note does not claim a new public [benchmark](/glossary/benchmark) record. It compares Astra with baselines the authors built, and it says what older ways of writing weights looked like. This section checks those claims against the note and against sources that were live on November 3, 2026. **Against training.** The authors' game policies were trained with PPO. Astra's one-shot MinAtar networks, which never play before they are scored, beat those PPO runs at 100 million steps in all three games. The table in the note is: | Game | PPO, 100 million steps | PPO, 500 million steps | Astra, one shot | | --- | --- | --- | --- | | Breakout | 15.8 | 25.3 | 517.4 | | Seaquest | 5.6 | 88.9 | 53.1 | | Asterix | 25.8 | 55.2 | 53.2 | Scoring is 1,000 episodes, with sticky actions of 0.1. PPO is the best of 3 to 10 seeds. On 9×9 Go, the note says a network written with zero forward passes wins more than half its games against two smaller PPO networks, and that 50 forward passes beat an equal-sized PPO network trained on 20 million games. It does not publish a win rate for that 50-pass match, so none is stated here. These are comparisons with the authors' PPO runs, not with a current public game engine. **Against the Qwen3.5-0.8B model card.** The [Qwen3.5-0.8B card](https://huggingface.co/Qwen/Qwen3.5-0.8B) warns that, in thinking mode with the recommended sampling parameters, this model is more prone to thinking loops than other Qwen3.5 models, and that those loops may stop it from finishing. The card's recommended thinking setting for text tasks is temperature 1.0, top-p 0.95, top-k 20, and presence penalty 1.5. A footnote on the same card says non-video benchmarks used that setup with temperature 0.6 in thinking mode. The note's Qwen evaluation uses temperature 0.6, top-p 0.95, top-k 20, and presence penalty 1.5, up to 32,768 tokens. That matches the card's benchmark footnote, not the temperature-1.0 recommendation. The card's published thinking-mode IFEval for the Qwen3.5-0.8B column is 44.0. Non-thinking IFEval in the same table is 52.1. The note's own base IFEval is 21.0, rising to 62.3 after the edit, with two samples per problem. Those are different measurements. The card does not list MATH-500, so the 7.9% to 48.1% result has no twin number on the model card. Read the note's MATH-500 figure as the authors' protocol, including the looping they measured: 93.0% of base MATH-500 answers hit the token limit, and 85 of the first 96 practice answers did too. **Against the authors' own negative control.** Gemma 3 270M hit its token limit on 8.5% of answers, against 93% for the Qwen setup. MATH-500, averaged over four samples, was 6.8% unedited and 7.0%, 7.6%, 7.7%, and 7.0% after budgets of 5, 10, 25, and 50 forward passes. The authors say no edit moved the score by more than about a point, and that the gain was not established. Editing removed an obstacle that was hiding answers. It did not add an ability the model lacked. **Against a second run of the edited checkpoint.** The [edited-model card](https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming) says the checkpoint was rebuilt on November 2, 2026 from the published diff, with every rebuilt tensor checked by SHA-256 against the checkpoint evaluated in the note. Only the down-projection weights in layers 15 and 19 differ from Qwen/Qwen3.5-0.8B at commit `2fc06364715b967f1860aea9cf38778875588b17`. A full MATH-500 re-run, 500 problems times 4 samples, scored 51.0%, with 31.6% of answers hitting the token limit. A 25-problem smoke test of the base model scored 5.0%, with 95% hitting the limit. That re-run supports the direction of the MATH-500 result. It is not a full replay of the base model, and it does not re-report GSM8K or IFEval. **Against harness work and older ways of writing weights.** The post cites [Meta-Harness](https://arxiv.org/abs/2603.28052), whose arXiv page is titled "Meta-Harness: End-to-End Optimization of Model Harnesses" and is dated March 30, 2026, in a sentence about using language models to optimize whole harnesses. A [harness](/glossary/harness) still matters in this note: the Go and Qwen experiments give Astra a Python session, a ban on training, and a budget of forward passes. The [ReAct breakdown](/research/react-reasoning-and-acting) is the other layer, a prompting loop over a model whose weights stay as trained. [Agent frameworks](/directory/category/agent-frameworks) in the directory, including LangChain and CrewAI, are that kind of wrapper. The post's own links describe two older routes with these anchor words: "custom networks trained to write them" ([arXiv:2102.11174](https://arxiv.org/abs/2102.11174)) and "compilers from formal programs" ([arXiv:2301.05062](https://arxiv.org/abs/2301.05062)). The authors' claim for what is new is a general-purpose frontier model, used as it is, reading and writing the weights. ## What the experiments controlled This is the part for readers who want the protocol, not the metaphor. Names and numbers below are the ones printed in the November 2 note unless a link says otherwise. **Reading games.** Four PPO policies play grid versions of Breakout, Pong, Pac-Man, and Freeway. The board is 10×10. The network is an MLP, 500 → 32 → 32 → 5, tanh, 17,253 weights, trained for 1 billion steps, best of three seeds. Inputs are five channels. Actions are no-op, up, down, left, and right. Astra receives the weights as text and a description of the grid and actions, with no game named and no tools. The note says it named the right game for all four. **Reading diffusion models.** Three DDPMs generate filled circles, squares, or triangles at 12×12 grayscale. The network is 160 → 64 → 64 → 144, ReLU, 23,824 weights, trained with Adam for 40,000 steps at batch 512. Astra is told not to run a forward pass. It names the shape and draws the outline from the output layer. The note says the outlines, rather than filled shapes, are a sign it is reading weight geometry rather than simulating the network. **Reading the middle of a tiny language model.** Three character-level transformers, 4 layers by 6 heads, width 384, about 7.16 million weights, train on 1 billion characters of Python, Japanese in romaji, or English. Astra sees only the four transformer blocks, with Python allowed and the network forbidden, up to 100 runs. It names the language in both attempts. The note's table gives confidences of 0.56 and 0.52 for Python, 0.55 and 0.62 for romaji, and 0.58 and 0.48 for English. Copying heads it names match the heads measured by running the model. Measured copying heads score 0.22 to 0.55. Every other head scores at most 0.02. The authors say this matches a method in Elhage et al. that they never mentioned to Astra. **Repairing the car.** The official Kinetix actor is 679 → 128, ReLU, then five layers of 128 with tanh, then 16 outputs: 171,664 weights. Official PPO for 1 billion steps never succeeds in 256 episodes. Astra gets the weights, the scene, and one recorded episode, in one reply, with no tools. The edit succeeds in all 1,024 episodes and changes 3.5% of the weights. The note describes a small added path that backs the car up, then hands control back to the trained policy. **Writing MinAtar and Go.** MinAtar networks start at all zeros, shape 600 → 32 → 32 → 6, tanh. Astra gets the rules and source, and may not roll the network out before scoring. For Go, the network is 243 → 512 × 6 → 82, ReLU, 1,480,274 weights, also starting at zero. The sandbox can edit tensors with NumPy or PyTorch on CPU and cannot run the network there. A forward pass is up to 128 games or 256 positions. Training, search, and tree search are forbidden. With zero forward passes the note says Astra wrote the Go network in one Python run of 336 lines. **Sycophancy.** Two copies of instruction-tuned Gemma 3 270M, 18 layers, are fine-tuned the same way, with embeddings frozen. One answers correctly. The other follows the user's guess. On facts the model knows and was not trained on, the note says the sycophant follows 100% of wrong guesses and the honest model follows 1–3%. Weights alone were not enough: four attempts found differences and could not map them to behavior. When Astra could see attention weights on up to 16 texts it wrote, and not the model outputs, it picked the sycophant in all three pairs. For head 10.2, the note's figure gives the honest model 94.3% of attention on the question's numbers and 0.75% on the user's answer, and the sycophant 3.5% and 85.7%. **The Qwen edit.** Qwen3.5-0.8B, thinking on, 24 layers. Practice items are 7,469 from the GSM8K training set and 7,275 from the MATH training set, not the test sets. A forward pass answers up to 512 problems or reads up to 1,024 texts. Budgets are 10, 25, or 50. The 50-pass edit subtracts one looping direction from 64 output columns of the MLP down-projection in layers 15 and 19: 131,072 weights. Sample counts in the note include 4 on MATH-500, 2 on GSM8K and IFEval, 16 on AMC23, 4 on Minerva Math, OlympiadBench, and GPQA Diamond, and 1 on a fixed 1,000-question slice of MMLU-Pro. Scores for AMC23, Minerva, OlympiadBench, GPQA, and MMLU-Pro are not in the table printed above, so they are not repeated here. | Qwen3.5-0.8B | MATH-500 | GSM8K | IFEval | Hit the token limit | | --- | --- | --- | --- | --- | | Base | 7.9% | 13.8% | 21.0% | 93.0% | | Edit, 10 forward passes | 12.3% | 25.7% | 49.0% | 87.2% | | Edit, 25 forward passes | 14.1% | 25.1% | 39.9% | 86.4% | | Edit, 50 forward passes | 48.1% | 67.1% | 62.3% | 33.1% | The edited-model card publishes these SHA-256 hashes for the raw bfloat16 bytes of the two changed tensors: layer 15 `82e2d733ea64feaeb4af93a1e065641d3df98b494a8b7576ef2f52e90fe2ee53`, layer 19 `8ef3c61bc36af113ad58e89dbb0a4a14dcba5a45cb750c15ab515c82ddd12eb2`. The authors' forward look is explicit and untested in this note. They hope later systems might audit weights without a test set, steer a capability by editing a few weights, or move a circuit from one architecture to another. They also name recursive self-improvement as a possibility if a model could edit its own weights. This note does not show that. A separate page on this site covers what [Anthropic told the SEC](/blog/anthropic-may-soon-meet-ai-acceleration-threshold) on November 2, 2026 about automated AI research. That filing is a different document, with different claims. ## Read the paper The original note, the logs, and the edited checkpoint: - [Sparks of Weight-Space Programming](https://rikiyat.github.io/blog/weight-space-programming/) - [GitHub logs and code](https://github.com/RikiyaT/lm_understand-logs) - [Edited Qwen3.5-0.8B checkpoint](https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming) - [Live demo](https://huggingface.co/spaces/akhaliq/weight-space-programming-demo) Related pages on this site: [harnesses](/topics/harnesses), [how agents work](/topics/how-agents-work), [research](/topics/research), [language models](/glossary/llm), and [agents](/glossary/agent). ## ReAct: Synergizing Reasoning and Acting in Language Models URL: https://www.aiagentsinfo.com/research/react-reasoning-and-acting Published: 2026-10-18. Updated: 2026-11-08 Paper: ReAct: Synergizing Reasoning and Acting in Language Models Paper URL: https://arxiv.org/abs/2210.03629 ReAct (Yao et al., 2022) interleaves a reasoning trace with an action, then reads the observation. The paper reports this on HotpotQA, FEVER, ALFWorld, and WebShop. ## What is ReAct: Synergizing Reasoning and Acting in Language Models? ReAct is a 2022 prompting pattern that writes a reasoning trace, takes an action, then reads the observation before the next step. If you have seen an agent print “Thought:” then “Action:” then “Observation:”, you have seen a descendant of **ReAct**. The contribution is not a new model. It is a prompting pattern that treats language as both a scratchpad and an interface to tools. The paper’s own title is ReAct: Synergizing Reasoning and Acting in Language Models. Yao, Zhao, Yu, Du, Shafran, Narasimhan, and Cao posted it as arXiv:2210.03629. It later appeared at ICLR 2023. The pattern sits next to the [harness](/glossary/harness) glossary entry: the loop around the model, not the weights. The broader account of that loop, including the product runtimes checked November 8, 2026, is [How do AI agents work?](/blog/how-do-ai-agents-work). The short definition of the cycle is [agent loop](/glossary/agent-loop). ## What the paper actually does The authors prompt a model to alternate reasoning traces with actions. On HotpotQA and FEVER, those actions hit a Wikipedia API. On ALFWorld and WebShop, the actions are steps in an interactive environment. They compare the mixture with reasoning-only and acting-only setups, and on the interactive tasks with imitation and reinforcement-learning baselines. The public abstract reports absolute success-rate gains of 34% (ALFWorld) and 10% (WebShop) against those imitation and reinforcement-learning methods, with one or two in-context examples. Cite the paper for the exact tables. Do not treat the figures as a score for any 2026 product. ## What not to copy forward Do not paste the original task scores into a modern [benchmark](/glossary/benchmark) chart. Tool APIs, context windows, and models have moved. A current number needs a current eval and a date. ## Read the source - Paper: [arXiv:2210.03629](https://arxiv.org/abs/2210.03629). The abstract still states the 34% and 10% success-rate sentences. Re-read November 8, 2026. - Project site: [react-lm.github.io](https://react-lm.github.io). It resolved on November 8, 2026. - Code: [ysymyth/ReAct](https://github.com/ysymyth/ReAct). The repository resolved on November 8, 2026. GitHub lists its last push as February 6, 2024. It is the paper’s code. - Related on this site: [Agent harnesses](/topics/harnesses), [How do AI agents work?](/blog/how-do-ai-agents-work), [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness), [agent loop](/glossary/agent-loop), [tool use](/glossary/tool-use), [autonomy](/glossary/autonomy) # Glossary ## Agent handoff URL: https://www.aiagentsinfo.com/glossary/agent-handoff A multi-agent pattern in which a specialist takes ownership of the next reply, instead of answering only as a tool the manager calls. An **agent handoff** gives the next specialist ownership of that branch of the work. Control moves. The specialist’s reply is the reply for that branch. [OpenAI’s orchestration page](https://developers.openai.com/api/docs/guides/agents/orchestration), read November 9, 2026, draws the line in those words. A handoff fits when a specialist should own the next response. An agent used as a tool fits when a manager should stay in control and call specialists as bounded capabilities. The page says to give each specialist a narrow job and to keep the handoff description short. The same page says a handoff can carry structured metadata or a filtered history, and that the exact API differs by language. The person or policy that decides a handoff is due is the [orchestrator](/glossary/orchestrator). The broader rule is [orchestration](/glossary/orchestration). A system with more than one role is [multi-agent](/glossary/multi-agent). How a learned orchestrator chooses the next call, including pairs of calls that only help together, is [What is multi-agent orchestration?](/blog/what-is-multi-agent-orchestration). The software that runs the transfer is the [harness](/glossary/harness). ## Agent loop URL: https://www.aiagentsinfo.com/glossary/agent-loop The cycle in which a model proposes an action, the harness runs an allowed tool, and the result comes back until a stop condition. An **agent loop** is the control loop from goal to action. The [model](/glossary/llm) proposes a step. The [harness](/glossary/harness) runs an allowed [tool](/glossary/tool-use), returns the result, and the cycle continues until the goal is met, a limit hits, or a person must approve. [Anthropic’s Form S-1/A](https://www.sec.gov/Archives/edgar/data/2130357/000162828026068676/anthropicpbc-sx1ax1.htm), filed November 2, 2026 and read November 8, 2026, defines the agent loop as the cycle in which an agent receives a task, takes an action, observes the result, and decides the next step until the task is complete. The [Claude Agent SDK](https://code.claude.com/docs/en/agent-sdk/agent-loop) and [OpenAI’s Agents SDK](https://developers.openai.com/api/docs/guides/agents/running-agents), both read the same day, describe that cycle in a product runtime. A 2022 ancestor of the written form is [ReAct](/research/react-reasoning-and-acting). The long explainer is [How do AI agents work?](/blog/how-do-ai-agents-work). The workshop around the loop is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Choosing the next specialist is [orchestration](/glossary/orchestration). ## Agent skill URL: https://www.aiagentsinfo.com/glossary/agent-skill A folder of instructions for an agent, centered on a SKILL.md file, loaded when a task matches the skill’s description. An **agent skill** is a folder that teaches an agent one job. The [Agent Skills specification](https://agentskills.io/specification), read November 6, 2026, requires a `SKILL.md` file. That file starts with YAML front matter. The required fields are `name` and `description`. The name is 1 to 64 characters, lowercase letters, numbers, and hyphens, and it must match the folder name. The description is 1 to 1,024 characters and should say what the skill does and when to use it. The markdown after the front matter is the instructions. Optional folders include `scripts/`, `references/`, and `assets/`. The [overview](https://agentskills.io/) says the format was developed by Anthropic and released as an open standard. Agents load it in stages: the name and description at startup, the full `SKILL.md` when a task matches, and other files only if the task needs them. Anthropic’s [Agent Skills page](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview), read November 6, 2026, says Claude Code discovers custom skills on the filesystem: `~/.claude/skills/` for one person and `.claude/skills/` inside a project. [LangChain’s March 10, 2026 harness post](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) treats skills as part of the [harness](/glossary/harness), so unused tool and [MCP](/glossary/model-context-protocol) instructions do not fill the context window. A [prompt](/glossary/prompt) is the text of one model call. A skill is a file the harness can pull into a later call. Anthropic’s own [Agent Skills docs](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) add product rules on top of that format, including a ban on the reserved words “anthropic” and “claude” in the name. Those rules are the product’s. The open specification is the page linked above. NVIDIA’s November 4, 2026 post says NeMo Eval Author provides coding-agent skills for writing evaluations. The post uses the word skills for that workflow. The guide to checking whether an agent did what a skill or a tool call requires is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). ## Agent2Agent protocol URL: https://www.aiagentsinfo.com/glossary/agent-to-agent-protocol An open protocol for independent AI agents to discover each other and delegate work. A2A is the horizontal link. MCP is the vertical link to tools and data. The **Agent2Agent protocol** (A2A) is an open standard for agents built on different frameworks, or by different vendors, to discover each other, delegate tasks, and share results. The [project home page](https://a2a-protocol.org/latest/), read November 8, 2026, says A2A was developed by Google and donated to the Linux Foundation. The [specification](https://a2a-protocol.org/latest/specification/) lists the latest released version as 1.0.0. Agents collaborate from declared capabilities. The page says they do not have to share internal memory, tools, or proprietary logic. A2A is horizontal: one agent to another agent. The [Model Context Protocol](/glossary/model-context-protocol) is vertical: one agent to its tools and data. The [A2A and MCP page](https://a2a-protocol.org/latest/topics/a2a-and-mcp/), read the same day, says to use both. A2A’s home page says A2A does not replace MCP, and it is not the protocol an agent uses to call its own tools. How several roles share one job is [multi-agent](/glossary/multi-agent). The long explainer is [How do AI agents work?](/blog/how-do-ai-agents-work). ## Agentic checkout URL: https://www.aiagentsinfo.com/glossary/agentic-checkout Agentic checkout is a purchase an assistant can finish. The store remains the seller, and a person or an earlier rule approves the payment. ## What is agentic checkout? Agentic checkout is a purchase an assistant can finish. The store remains the seller, and a person or an earlier rule approves the payment. It is the payment step inside a [shopping agent](/glossary/shopping-agent). The assistant sends the cart and a payment instruction. The merchant charges the payment method on the merchant's systems and remains the seller of record, on the Google, OpenAI, Microsoft, and Perplexity pages cited in the [checkout guide](/blog/can-an-ai-shopping-agent-check-out-for-you). Amazon’s November 6, 2026 text feature is on [Amazon Rufus latest features](/directory/amazon-rufus). The six-company timeline is [AI shopping agents next 6 months](/reports/ai-shopping-agents-next-6-months). The sourced table is [Best AI agent for shopping](/best/ai-agents-for-shopping). Shopping coverage is collected on [Agents in shopping](/topics/shopping). The step is a [tool](/glossary/tool-use) call with a [guardrail](/glossary/guardrail). The guardrail can be a tap, a price cap, or a rule set earlier, such as Amazon's auto-buy at a target price. How much of that happens without a fresh confirmation is [autonomy](/glossary/autonomy). The catalog the assistant is allowed to quote is a [product feed](/glossary/product-feed). The surrounding software is the [harness](/glossary/harness). ## Approval gate URL: https://www.aiagentsinfo.com/glossary/approval-gate A configured stop that holds a named agent action until a person approves it. An **approval gate** is a stop the [harness](/glossary/harness) enforces before a named action. The action can be sending a message, editing a record, deleting a file, or spending money. The run waits. A person approves or rejects that step. That person is the [human in the loop](/glossary/human-in-the-loop). [OpenAI’s workspace-agents page](https://openai.com/business/workspace-agents/), read November 7, 2026, lists approval gates for sensitive actions such as sending messages or updating records. The [help article](https://help.openai.com/en/articles/20001143-chatgpt-workspace-agents-for-enterprise-and-business) read the same day says write actions default to Always ask during a run. [Claude’s Enterprise help article](https://support.claude.com/en/articles/17444462-set-up-and-manage-the-new-unified-claude-experience-for-your-enterprise-organization), also read November 7, says admins can turn off “Skip all approvals” and “Automatically approve,” so a member cannot let Claude act on tools, file edits, and browsing without being asked. A [guardrail](/glossary/guardrail) is the wider family of checks. The gate is the specific pause. How many gates you set is an [autonomy](/glossary/autonomy) choice. The [tool](/glossary/tool-use) still runs only after the gate opens. The workplace guide is [What is an AI employee?](/blog/what-is-an-ai-employee). A payment gate in a store is [agentic checkout](/glossary/agentic-checkout). ## Autonomy URL: https://www.aiagentsinfo.com/glossary/autonomy How many actions a system may take without a human confirming — a spectrum, not a personality. **Autonomy** here means permission and supervision, not consciousness. A suggest-only coding assistant is low autonomy. A [shopping agent](/glossary/shopping-agent) that can complete an [agentic checkout](/glossary/agentic-checkout) is higher. The [harness](/glossary/harness) encodes the level: budgets, allowlists, and human gates. A person who approves a step before it counts is [human in the loop](/glossary/human-in-the-loop). A person who watches the run and can step in is [human on the loop](/glossary/human-on-the-loop). The shopping guide is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). Sourced tables are [Best AI agent for shopping](/best/ai-agents-for-shopping) and [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks). ## Behavioral evaluation URL: https://www.aiagentsinfo.com/glossary/behavioral-evaluation A test that checks one observable agent action, such as a tool call or a question, rather than a whole benchmark score. A **behavioral evaluation** (also called a behavioral eval) checks one action you can see in a run. The action might be a [tool](/glossary/tool-use) call, a clarifying question, a file edit, or a test command. The check reads that step. It does not, by itself, score whether a whole project succeeded. [Google’s September 9, 2026 post](https://developers.googleblog.com/en/the-anatomy-of-harness-engineering-how-to-evaluate-iterate-and-guard-ai-coding-agents/) treats these checks as integration tests for an [agent harness](/glossary/harness). An end-to-end [benchmark](/glossary/benchmark) still answers whether the job finished. The longer guide is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). How to stand up the tasks and graders is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). Tools that run those checks are [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). Designing the system those tests wrap is [harness engineering](/glossary/harness-engineering). A paper that grades entire harnesses on held-out tasks is [HarnessFlow](/research/harnessflow-mcts-harness-optimization). ## Benchmark URL: https://www.aiagentsinfo.com/glossary/benchmark A frozen set of tasks plus a scoring rule. It is not the same as your production job. A **benchmark** is an evaluation suite. For agents, the [harness](/glossary/harness) is part of what you are measuring. Read dates, cost, and contamination notes before you believe a chart. A [behavioral evaluation](/glossary/behavioral-evaluation) checks one action, such as a tool call, rather than the suite’s overall score. The comparison is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). How to build the suite is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). Tools that run those checks are [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). Dated public boards are the [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026). ## Browser use URL: https://www.aiagentsinfo.com/glossary/browser-use Agent tools that read and act on a web page, without taking over a full desktop. **Browser use** is a set of [tools](/glossary/tool-use) that let a model read a web page and act on it. The work stays in the browser. A full desktop, with every local app, is [computer use](/glossary/computer-use). [Anthropic's computer-use page](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool) says the browser use tool reads and acts on the page itself and does not need a full desktop environment. [OpenAI's computer-use guide](https://developers.openai.com/api/docs/guides/tools-computer-use) separates `BrowserToolset`, for page text, controls, and tabs, from `ComputerToolset`, for screenshots and mouse or keyboard actions. Both still sit inside a [harness](/glossary/harness) that runs the calls and applies the permissions. The longer guide is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer). Desktop, API, and Windows options are compared in [Best computer use agents](/best/computer-use-agents). The six-month product table is [Browser and computer-use agents next 6 months](/reports/browser-and-computer-use-agents-next-6-months). ## Computer use URL: https://www.aiagentsinfo.com/glossary/computer-use A tool loop in which a model looks at a screen and asks for a mouse or keyboard action, and the app runs that action. **Computer use** is a [tool](/glossary/tool-use) loop. The model looks at a screen and asks for a click, a keystroke, or a scroll. The app around the model takes the screenshot, runs the action, and returns the new screen. That app is part of the [harness](/glossary/harness). The working pair is an [agent](/glossary/agent). [Anthropic's computer-use tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool) is the API form: one toolset entry, and your application runs each call in an environment you control. [Claude's product article](https://claude.com/resources/articles/dispatch-and-computer-use) describes the same idea in Claude Cowork and Claude Code, on a Mac or Windows desktop. A check before a risky click is a [guardrail](/glossary/guardrail). How far the loop runs before a person confirms is [autonomy](/glossary/autonomy). A narrower tool that stays on a web page is [browser use](/glossary/browser-use). The longer guide is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer). A sourced comparison of desktop, API, open-source, and Windows options is [Best computer use agents](/best/computer-use-agents). What is dated from November 9, 2026 through May 9, 2027 is [Browser and computer-use agents next 6 months](/reports/browser-and-computer-use-agents-next-6-months). A workplace agent that returns a file for review, rather than driving the screen, is [What is an AI employee?](/blog/what-is-an-ai-employee). ## Context compaction URL: https://www.aiagentsinfo.com/glossary/context-compaction A harness step that replaces older turns of an agent transcript with a shorter summary so later model calls still fit. **Context compaction** is the [harness](/glossary/harness) step that shortens a long transcript. Later calls see the summary, plus whatever recent turns the harness keeps, instead of the whole history. Two product versions were re-read on November 9, 2026. [Anthropic’s context-window page](https://platform.claude.com/docs/en/build-with-claude/context-windows) calls server-side compaction the primary strategy for long-running agent work. The [threshold page](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold) says compaction starts when input tokens reach a trigger, 150,000 by default and at least 50,000, and that threshold compaction is in beta on select models. [OpenAI’s compaction guide](https://developers.openai.com/api/docs/guides/compaction) says the carried item is opaque and not intended to be human-interpretable. A research handoff is a third version. The November 3, 2026 report [You don’t need long context](/research/you-dont-need-long-context) has the model write a short brief and then starts a new conversation. The space being protected is the [context window](/glossary/context-window). The text of one call is a [prompt](/glossary/prompt). The cycle that fills the window is the [agent loop](/glossary/agent-loop). ## Context window URL: https://www.aiagentsinfo.com/glossary/context-window How much text a model can see at once — instructions, history, tool results, and retrieved notes. The **context window** is the model's working memory for one call. Agents fill it with [prompts](/glossary/prompt), traces, and [RAG](/glossary/rag) hits. When it fills up, the harness must summarize, drop, or retrieve. That summary step is [context compaction](/glossary/context-compaction). ## Digital worker URL: https://www.aiagentsinfo.com/glossary/digital-worker A workplace label for software, often an agent, that is given a recurring job and returns work for a person to review. A **digital worker**, sometimes called an AI employee, is a label for software that a workplace treats as the holder of a recurring job. On the pages checked for this site’s workplace guide, the software is usually an [agent](/glossary/agent): a model, tools, and a [harness](/glossary/harness) that can plan steps and hand work back. It is a product setting and a management metaphor. It is not a legal employee, a payroll record, or the party accountable for a mistake. [OpenAI’s business guide](https://cdn.openai.com/business-guides-and-resources/a-business-leaders-guide-to-working-with-agents.pdf), read November 7, 2026, uses “employee agents” for pre-built software that helps a team finish work, and names ChatGPT’s agent mode as the example in that guide. [Deloitte’s December 10, 2025 Tech Trends essay](https://www.deloitte.com/global/en/insights/topics/technology-management/tech-trends/2026/agentic-ai-strategy.html) talks about managing “digital workers” and a “silicon-based workforce” beside human staff, and it keeps people on oversight and on work whose requirements keep shifting. How far the software may go before a person confirms a step is [autonomy](/glossary/autonomy). The stop itself is a [human in the loop](/glossary/human-in-the-loop), often at an [approval gate](/glossary/approval-gate). The longer guide is [What is an AI employee?](/blog/what-is-an-ai-employee). Sourced tables for that job are [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks) and [Best AI agent for small business](/best/ai-agents-for-small-business). ## Guardrail URL: https://www.aiagentsinfo.com/glossary/guardrail A check that can allow, block, rewrite, or pause an agent action before it counts. A **guardrail** is a check inside a [harness](/glossary/harness). It runs on the way in, around a [tool](/glossary/tool-use), or on the way out. The check can let the step through, block it, rewrite it, or pause it for a person. [OpenAI’s Agents SDK](https://developers.openai.com/api/docs/guides/agents/guardrails-approvals) uses the word for automatic checks on input, output, or tool behavior. The same page treats human review as a separate pause: the run stops until a person or a policy approves or rejects a sensitive action. [Pydantic AI’s harness docs](https://pydantic.dev/docs/ai/harness/guardrails/) describe a callable at the user prompt, at each tool call and result, and at the final output. The callable’s outcomes are allow, block, replace, retry, or — for a tool call — defer for approval. How often a person must confirm a step is [autonomy](/glossary/autonomy). The person who approves, edits, or stops that step is a [human in the loop](/glossary/human-in-the-loop). A named stop, such as “ask before this send,” is an [approval gate](/glossary/approval-gate). A payment is one of those pauses: see [agentic checkout](/glossary/agentic-checkout). The longer guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). The workplace case is [What is an AI employee?](/blog/what-is-an-ai-employee). The shopping case is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). ## Harness URL: https://www.aiagentsinfo.com/glossary/harness The runtime around a model — loop, tools, memory, permissions, and stop conditions. A **harness** is the software around the model weights. It feeds the goal, exposes [tools](/glossary/tool-use), stores observations, retries, and stops. Two products can share a model and behave differently because the harness differs. Designing that software is [harness engineering](/glossary/harness-engineering). Choosing the next step or agent is [orchestration](/glossary/orchestration). A check that allows, blocks, or pauses an action is a [guardrail](/glossary/guardrail). A test of one observable action is a [behavioral evaluation](/glossary/behavioral-evaluation). The cycle it runs is the [agent loop](/glossary/agent-loop). The long explainer of that cycle is [How do AI agents work?](/blog/how-do-ai-agents-work). The longer guide to the workshop itself is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). How to test the workshop is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). How to stand up the tasks, sandboxes, and graders is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). A sourced comparison of coding, open-source, and local runtimes is [Best AI agent harness](/best/ai-agent-harnesses). Nineteen framework release tags are [Best AI agent frameworks](/best/ai-agent-frameworks). Public scores are the [November 2026 agent harness benchmark](/reports/agent-harness-benchmark-november-2026). The short pointer is the [harness comparison tracker](/trackers/harnesses). All of those sit on the [agent harnesses hub](/topics/harnesses). A shopping assistant that can pay is the same split applied to a cart: [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). ## Human in the loop URL: https://www.aiagentsinfo.com/glossary/human-in-the-loop A person who can approve, edit, or stop an agent action before that action counts. **Human in the loop** means a person stays in the run. The [agent](/glossary/agent) can draft, look things up, and propose a next step. A person approves, edits, or stops the step that would send, pay, delete, or otherwise leave the organization. [OpenAI’s business guide](https://cdn.openai.com/business-guides-and-resources/a-business-leaders-guide-to-working-with-agents.pdf), read November 7, 2026, says guardrails should seek confirmation before many real-world actions and ask for oversight on sensitive tasks such as sending email. [ChatGPT’s Work guide](https://learn.chatgpt.com/docs/get-started-with-work), read the same day, tells you to say when the task should stop for review, and to follow the notice if a task pauses for a safety check. A [guardrail](/glossary/guardrail) is the check inside the [harness](/glossary/harness). An [approval gate](/glossary/approval-gate) is one configured stop, such as “ask before this write.” [Autonomy](/glossary/autonomy) is how many steps are allowed before that person is asked. A person who supervises from the record and can intervene, without approving every step before it counts, is [human on the loop](/glossary/human-on-the-loop). The workplace guide is [What is an AI employee?](/blog/what-is-an-ai-employee). The click-by-click case is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer). The loop those stops sit in is [How do AI agents work?](/blog/how-do-ai-agents-work). ## Human on the loop URL: https://www.aiagentsinfo.com/glossary/human-on-the-loop A person who supervises an agent run from its record and can intervene. The step does not wait for approval before it counts. **Human on the loop** means a person supervises a run and can step in. The action proceeds under the rules already set. The person watches telemetry, traces, or another record, and intervenes when that record shows a problem. [Deloitte’s November 18, 2025 essay](https://www.deloitte.com/global/en/insights/industry/technology/technology-media-and-telecom-predictions/2026/ai-agent-orchestration.html), read November 8, 2026, names a spectrum: humans in the loop, on the loop, and out of the loop. It says the first two rely on dashboards for outcome tracing and orchestration so a person can intervene, and it predicts that in 2026 the most advanced organizations will begin shifting some supervision toward human-on-the-loop. That is Deloitte’s forecast. On this site the phrase stays distinct from [human in the loop](/glossary/human-in-the-loop), which means a person approves, edits, or stops a step before that step counts. How many steps are allowed before someone is asked is [autonomy](/glossary/autonomy). The long explainer is [How do AI agents work?](/blog/how-do-ai-agents-work). ## Language model (LLM) URL: https://www.aiagentsinfo.com/glossary/llm A model trained to predict text; today's AI agents almost always wrap one. A **large language model** predicts the next tokens of text. Product names change; the job does not. Agents use the model as a policy that speaks in language — including structured tool calls — rather than as the whole product. Related: [prompt](/glossary/prompt), [context window](/glossary/context-window). ## Model Context Protocol URL: https://www.aiagentsinfo.com/glossary/model-context-protocol An open protocol for connecting an AI application to external tools, data, and workflows. The specification’s current version, read November 8, 2026, is 2026-07-28. The **Model Context Protocol** (MCP) is an open protocol for connecting an AI application to external systems. The [introductory page](https://modelcontextprotocol.io/docs/getting-started/intro), read November 8, 2026, describes it as a standard way for applications such as Claude or ChatGPT to reach data sources, tools, and workflows. The same page compares MCP to a USB-C port: one connection pattern, many devices. It names support in Claude, ChatGPT, Visual Studio Code, Cursor, and MCPJam. The [specification for revision 2026-07-28](https://modelcontextprotocol.io/specification/2026-07-28) says MCP uses JSON-RPC so hosts, clients, and servers can share context, expose tools, and build workflows. Servers can offer resources, prompts, and tools. The [versioning page](https://modelcontextprotocol.io/docs/learn/versioning) for that revision says the current protocol version is 2026-07-28, and that a version string marks the last date of a backwards-incompatible change. MCP is a [tool use](/glossary/tool-use) path. The [harness](/glossary/harness) decides which servers are connected and what the [agent](/glossary/agent) may call. A skill served over MCP is a separate extension of the protocol. The file format those skills use is an [agent skill](/glossary/agent-skill). The [Agent2Agent protocol](/glossary/agent-to-agent-protocol) is the complementary standard for one independent agent talking to another. The A2A project’s own comparison, read November 8, 2026, calls MCP the vertical link to tools and data. The versioning page still named 2026-07-28 as the current protocol version on that date. The evaluation guide, including NVIDIA’s November 4, 2026 example of an assistant connected through MCP, is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). The loop that calls these tools is [How do AI agents work?](/blog/how-do-ai-agents-work). ## Multi-agent URL: https://www.aiagentsinfo.com/glossary/multi-agent More than one model (or role) collaborating, often as planner, worker, or critic. **Multi-agent** setups split work across roles: a planner, a coder, a reviewer. Sometimes that is several model calls with different prompts; sometimes it is several processes. It is still one product, one [harness](/glossary/harness), and one bill. Extra roles are not automatically extra intelligence. How the patterns differ is [How do AI agents work?](/blog/how-do-ai-agents-work). When the other party is an independent agent, the open protocol is [Agent2Agent](/glossary/agent-to-agent-protocol). ## Orchestration URL: https://www.aiagentsinfo.com/glossary/orchestration The rules for which step, tool, or agent runs next, and who owns the answer. **Orchestration** is the part of a [harness](/glossary/harness) that chooses the next step and passes the result along. In a single agent, that can be the loop that calls a tool and continues. In a [multi-agent](/glossary/multi-agent) setup, it also decides who owns the reply. [OpenAI’s Agents SDK](https://developers.openai.com/api/docs/guides/agents/orchestration) describes two patterns. A handoff gives the next specialist ownership of that branch. An agent used as a tool leaves the manager in charge of the final answer. [Google’s ADK 2.0 page](https://adk.dev/2.0/) describes another form: a graph of agents, tools, and functions with explicit routes. [LangChain’s March 10, 2026 anatomy post](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) lists orchestration as subagent spawning, handoffs, and model routing. The longer guide to the single-agent cycle is [How do AI agents work?](/blog/how-do-ai-agents-work). The part that fills the seat is an [orchestrator](/glossary/orchestrator). A specialist that takes over the reply is an [agent handoff](/glossary/agent-handoff). A November 5, 2026 paper that trains the choice, and the points it reports on code, is [What is multi-agent orchestration?](/blog/what-is-multi-agent-orchestration). The workshop around that cycle is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). A sourced comparison of those runtimes is [Best AI agent harness](/best/ai-agent-harnesses). ## Orchestrator URL: https://www.aiagentsinfo.com/glossary/orchestrator The part of a multi-agent system that chooses which worker runs next and what earlier results that worker may see. An **orchestrator** is the part that picks the next worker and the earlier results that worker is allowed to read. The workers do the reading and the writing. The orchestrator decides the order. Liu and Xiong, [arXiv:2611.07495](https://arxiv.org/abs/2611.07495), posted November 5, 2026 and read November 9, define agent 0 as the orchestrator and the other agents as workers. Their workers can be language models, tools, or verifiers. The orchestrator chooses a worker and a set of earlier outputs, or it stops. The long explainer is [What is multi-agent orchestration?](/blog/what-is-multi-agent-orchestration). [OpenAI’s orchestration page](https://developers.openai.com/api/docs/guides/agents/orchestration), read November 9, 2026, uses a manager for the same seat when the manager keeps the reply and calls specialists as tools. A specialist that takes over the reply is an [agent handoff](/glossary/agent-handoff). The rule, whoever fills the seat, is [orchestration](/glossary/orchestration). The program around all of them is the [harness](/glossary/harness). ## Product feed URL: https://www.aiagentsinfo.com/glossary/product-feed A structured catalog a store sends so an assistant can show a real price, a real image, and a product page. A **product feed** is a file of product rows that a merchant maintains. [OpenAI's feed spec](https://developers.openai.com/commerce/specs/file-upload/products) requires nine fields for a discovery row: an item id, a title, a description, a product URL, a brand, a seller name, an image URL, availability, and a price. Checkout is a separate integration. Marking a row as eligible does not, by itself, turn on payment. A [shopping agent](/glossary/shopping-agent) that quotes a price should be reading a feed like this, or a merchant catalog with the same job, rather than inventing a number. Google's Shopping Graph and Merchant Center are the feed path behind its cart. The payment step that may follow is [agentic checkout](/glossary/agentic-checkout). Fetching the row is [tool use](/glossary/tool-use). The longer guide is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). The sourced table is [Best AI agent for shopping](/best/ai-agents-for-shopping). ## Prompt URL: https://www.aiagentsinfo.com/glossary/prompt The instructions and context given to a language model for a single step or a whole run. A **prompt** is the input: system rules, the user goal, tool schemas, and recent observations. In an [agent](/glossary/agent), the prompt is rebuilt every loop iteration. Treat prompt files as part of the [harness](/glossary/harness), and version them like code. Consumer writing products are compared in [Best AI agent for writing](/best/ai-agents-for-writing). ## RAG (retrieval-augmented generation) URL: https://www.aiagentsinfo.com/glossary/rag Fetching documents or traces at run time and placing them into the model context. **Retrieval-augmented generation** looks up text (docs, tickets, prior traces) and adds it to the [context window](/glossary/context-window) before the model answers. It is a memory tactic, not a mind. Agents often mix RAG with tool use; they are not the same thing. Sourced research products are compared in [Best AI agent for research](/best/ai-agents-for-research). ## Sandbox URL: https://www.aiagentsinfo.com/glossary/sandbox A bounded machine, often a virtual machine or container, where an agent's actions run so a mistake stays inside it. A **sandbox** is a bounded machine where an agent's actions run. A virtual machine or a container is the usual form. The point is containment: a bad click, a bad command, or a hostile page stays inside that machine. The [harness](/glossary/harness) decides which commands are even offered. A [guardrail](/glossary/guardrail) can still pause a step inside the sandbox. [Computer use](/glossary/computer-use) is one reason the boundary matters, because the model is asking for real pointer and keyboard actions. [Anthropic's computer-use page](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool) lists a dedicated virtual machine or container with minimal privileges as a precaution. [OpenAI's computer-use guide](https://developers.openai.com/api/docs/guides/tools-computer-use) says to use an isolated browser or virtual machine. [Google's Computer Use guide](https://ai.google.dev/gemini-api/docs/computer-use) says to run the agent in a sandboxed virtual machine or container, separate from the host. The longer guide is [Can an AI agent use your computer?](/blog/can-an-ai-agent-use-your-computer). ## Shopping agent URL: https://www.aiagentsinfo.com/glossary/shopping-agent A model plus the tools to search, compare, and sometimes buy a product for an adult shopper. A **shopping agent** is an [agent](/glossary/agent) pointed at a purchase. The model reads the request. The [harness](/glossary/harness) calls a catalog, a price check, a cart, and, when the product allows it, a payment. Comparing three items in a chat is advice. Adding a live price from a [product feed](/glossary/product-feed), or paying under a rule the person already set, is the agent. How far it may go without a new confirmation is [autonomy](/glossary/autonomy). The payment step itself is [agentic checkout](/glossary/agentic-checkout). The sourced table is [Best AI agent for shopping](/best/ai-agents-for-shopping). The longer guide is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). The workshop around any of these products is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). ## Tool use URL: https://www.aiagentsinfo.com/glossary/tool-use Letting a model call APIs, a browser, a shell, or other programs, then reading the result. **Tool use** (also called function calling) is how a model steps outside the text box. The [harness](/glossary/harness) decides which tools exist, how arguments are validated, and what the model is allowed to see of the result. Without tools, you have dialogue. With tools and a loop, you have an [agent](/glossary/agent). One open way to connect those tools is the [Model Context Protocol](/glossary/model-context-protocol). A catalog lookup or a payment call is the same pattern in a store: see [product feed](/glossary/product-feed) and [agentic checkout](/glossary/agentic-checkout). Web search and inbox tools are compared in [Best AI agent for web search](/best/ai-agents-for-web-search) and [Best AI agent for email](/best/ai-agents-for-email). ## What Are AI Agents and How Do They Work? URL: https://www.aiagentsinfo.com/glossary/agent An AI agent is a language model plus a harness that lets it use tools, check results, and keep going until a goal is met. The long explainer is How do AI agents work? In classical AI textbooks, an **agent** perceives and acts. On this site the word usually means a language model plus a [harness](/glossary/harness). The model chooses a step. The harness runs a [tool](/glossary/tool-use), checks the result, and puts that result back in front of the model. The loop continues until the goal is met or a limit stops it. A chatbot that only replies in a thread is a useful model. An agent on this page also takes steps. The long explainer is [How do AI agents work?](/blog/how-do-ai-agents-work). The cycle inside that explainer is the [agent loop](/glossary/agent-loop). The workshop around the model is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). The [how agents work](/topics/how-agents-work) topic collects the explainers. Framework release tags are [Best AI agent frameworks](/best/ai-agent-frameworks). Coding-agent list prices checked November 9, 2026 are [Best AI coding agents](/best/ai-coding-agents). A token bill is the [AI agent cost calculator](/agent-cost-calculator). Sourced job-fit tables are on [Best AI agents](/best): [Best AI agent harness](/best/ai-agent-harnesses), [Best computer use agents](/best/computer-use-agents), [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools), [Best AI agent for shopping](/best/ai-agents-for-shopping), [Best AI agent for web search](/best/ai-agents-for-web-search), [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks), and [Best AI agent for writing](/best/ai-agents-for-writing). ## What types of AI agents are there? The types on this site are jobs, not a textbook taxonomy. - A coding agent reads a project, edits files, and runs checks. The [coding agents](/directory/category/coding-agents) category lists them. [Claude Code](/directory/claude-code) is one. - A [shopping agent](/glossary/shopping-agent) searches, compares, and sometimes buys. - A [multi-agent](/glossary/multi-agent) setup splits one job across roles, such as a planner and a reviewer. It is still one product and one harness. ## What Is Harness Engineering? URL: https://www.aiagentsinfo.com/glossary/harness-engineering Harness engineering is the work of designing the tools, memory, permissions, and checks around an AI model. OpenAI’s February 11, 2026 post is one public account of that job. “Opus 5.5 harness engineering” is a rising search for the same work, not a separate product this page can confirm. ## What is harness engineering? **Harness engineering** is the work of designing the [harness](/glossary/harness): the loop, tools, files, permissions, and checks around a model. The [prompt](/glossary/prompt) is one piece of that system. [LangChain’s March 10, 2026 explainer](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) uses the phrase for building the system around a model so the model’s output becomes work. [Databricks’ June 17, 2026 explainer](https://www.databricks.com/blog/ai-harness) places it after prompt engineering and context engineering: the artifact is the harness itself. The longer guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). How to test a change to that workshop is [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness). How to build the evaluation infrastructure is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). A sourced comparison of runtimes is [Best AI agent harness](/best/ai-agent-harnesses). Dated public scores, each with the date on its source, are in the [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026). The [harnesses](/topics/harnesses) topic is the shelf for those guides. A check that allows, blocks, or pauses an action is a [guardrail](/glossary/guardrail). Choosing the next step is [orchestration](/glossary/orchestration). ## What does OpenAI mean by harness engineering? [OpenAI’s February 11, 2026 account](https://openai.com/index/harness-engineering/) uses the phrase for a team whose job became designing environments, specifying intent, and building feedback loops so coding agents could ship an internal product. That post is what “OpenAI harness engineering” refers to on this page: the company’s description of one internal experiment. It is not a product name, and it is not a course. The [harness guide](/blog/what-is-an-ai-agent-harness) walks through the figures in that post. The [evaluation guide](/blog/how-do-you-evaluate-an-ai-agent-harness) is how you test a change to the workshop the post describes. Standing up the tasks and graders is [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents). ## Why are people searching for Opus 5.5 harness engineering? “Opus 5.5 harness engineering” is a rising search. This page does not have a product announcement under that name, and it does not treat the phrase as news. Harness engineering is still the work above: the loop, tools, files, permissions, and checks around a model. A model name can sit inside that system. The [agent harness guide](/blog/what-is-an-ai-agent-harness) notes that Pydantic AI’s harness quick start, checked November 4, 2026, still uses the model string `anthropic:claude-opus-5-5` in its first example. That string names a model the harness calls. It does not change this definition, and it is not a claim about a new Opus 5.5 product. To learn the work, use the reading list in the next section. ## How do you learn harness engineering? The public accounts above are the reading list. They are not a course. 1. Read the parts. The [harness guide](/blog/what-is-an-ai-agent-harness) walks through the loop, the tools, the memory, the permissions, and the checks. 2. Separate two jobs that people mix up. [Orchestration](/glossary/orchestration) chooses the next step or specialist. A [guardrail](/glossary/guardrail) allows, blocks, replaces, or pauses an action. 3. Read one primary account of the work, then a second. OpenAI’s February 11, 2026 post is the company’s description of its own experiment. LangChain’s March 10 post and Databricks’ June 17 post are the shorter teaching versions of the same split. 4. Test a change before you trust it. [How do you evaluate an AI agent harness?](/blog/how-do-you-evaluate-an-ai-agent-harness) is the guide for one observable action. [How to build an eval harness for AI agents](/blog/how-to-build-an-eval-harness-for-ai-agents) is the guide for the tasks, sandboxes, and graders. The [November 2026 harness benchmark](/reports/agent-harness-benchmark-november-2026) lists public SWE-bench and Terminal-Bench scores that already have a date and a source. 5. Compare products only where a source exists. The [harnesses topic](/topics/harnesses) collects the explainers. The [harness comparison](/trackers/harnesses) stays empty until each cell has its own source. # Directory ## AccurateScribe.ai URL: https://www.aiagentsinfo.com/directory/accuratescribe-ai Category: Audio / voice AI speech-to-text for meetings and recorded media. Pricing: Freemium Website: https://accuratescribe.ai/ Last verified: 2026-11-09 _No body text._ ## Adspirer URL: https://www.aiagentsinfo.com/directory/adspirer Category: Marketing Adspirer is the AI-powered paid media agent for every business. Pricing: Freemium · starting at $0 per month · Monthly Features: Maker: Abhilash Mekala, Indie hacker Website: https://www.adspirer.com/ Last verified: 2026-11-03 _No body text._ ## Agent Runaway Guard Kit URL: https://www.aiagentsinfo.com/directory/agent-runaway-guard-kit Category: Developer tools Python guardrails for repeated AI-agent tool calls, retry storms, and bounded execution costs. Pricing: Paid · starting at 24 · One-time Features: Type: AI tool Website: https://mhamedasskoure.gumroad.com/l/agent-runaway-guard-kit Last verified: 2026-11-08 _No body text._ ## AgentPawn URL: https://www.aiagentsinfo.com/directory/agentpawn Category: AI agents AI agents post and complete jobs for other agents - free Pricing: Free Features: Type: AI tool Website: https://agentpawn.com/ Last verified: 2026-11-05 _No body text._ ## AgentSEO URL: https://www.aiagentsinfo.com/directory/agentseo Category: SEO SEO and AI search research tools for developers and AI agents Pricing: Freemium Features: Type: SaaS; Maker: Daniel Martin, Startup Website: https://www.agentseo.dev/ Last verified: 2026-11-09 _No body text._ ## AI Designer URL: https://www.aiagentsinfo.com/directory/ai-designer Category: Design Designs UI, brand kits, and images from a short prompt, and ships a desktop canvas that coding agents such as Claude Code and Codex can use. Pricing: On aidesigner.ai/pricing (checked twice 2026-11-09): Desktop Pro is a launch price of $10/month, with the regular price shown as $15/month. Pro is $25/month and includes Desktop Pro plus 100 monthly credits, credit rollover, site publishing, and custom domains. The page says Pro includes everything in Free, and the homepage says you can try it with no card. A yearly toggle is on the page. Those yearly dollar amounts were not in the static price cards. Features: AI UI, landing pages, brand kits, and image tools; Desktop Pro runs with your own Claude Code, Codex, or OpenCode; MCP server so a coding assistant can reach the design tool; Pro adds monthly credits, publishing, and custom domains Website: https://www.aidesigner.ai Last verified: 2026-11-09 AI Designer, from Wiz, turns a short prompt into UI, brand kits, and other visual assets. The desktop app is the agent piece: it is a design canvas for a coding agent you already run, such as Claude Code, Codex, or OpenCode, and the site lists an MCP server for that connection. TrustMRR verifies revenue with a Stripe API key. Checked November 9, 2026: current MRR $8,323, last 30 days $8,188, all-time revenue $85,149, with 287 active subscriptions. ## AI Sales Trainer URL: https://www.aiagentsinfo.com/directory/ai-sales-trainer Category: Sales Practice B2B and retail sales conversations with AI customers Pricing: Free Features: Type: SaaS Website: https://sandwichconsulting.com/products/ai-sales-trainer/ Last verified: 2026-11-07 _No body text._ ## AI Trend Studio URL: https://www.aiagentsinfo.com/directory/ai-trend-studio Category: Video AI video generator with trending effects like AI Kiss and AI Hug. Pricing: Freemium · starting at $9.90/mo · Monthly Features: Type: AI tool Website: https://aitrendstudio.com/ Last verified: 2026-11-09 _No body text._ ## AI Video Maker URL: https://www.aiagentsinfo.com/directory/ai-video-maker Category: Video Browser-based AI tool for creating videos from text or images, plus image generation and editing. Pricing: Paid Features: Type: AI tool Website: https://aivideomaker.dev/ Last verified: 2026-11-09 _No body text._ ## Aidelly URL: https://www.aiagentsinfo.com/directory/aidelly Category: AI agents Agent-operated social platform for agencies. An assistant drafts, posts, and replies across client accounts while people still approve the work. Pricing: On aidelly.ai/pricing (checked twice 2026-11-09), the cards are annual prices: Launch $29/month billed annually ($348/year) with 1,000 AI credits a month, Scale $99/month billed annually ($1,188/year) with 5,000 credits, and Agency $279/month billed annually ($3,348/year) with 20,000 credits. Overage is $0.01 per credit. Add-ons on the page include an extra workspace at $15/month and X Unlimited at $149/month. Each card offers a 7-day trial at $0.00 today. Features: An agent runs social for a brand, with a human approval step; Shared AI credit pool for text, images, captions, and vision; Launch, Scale, and Agency plans, billed annually on the price cards; 7-day trial at $0 today Website: https://www.aidelly.ai Last verified: 2026-11-09 Aidelly is Wilzer Jean-Baptiste’s social product for agencies that want an agent to do the posting. The homepage is explicit about the setup: an agent drafts and publishes, and the team approves. The site also labels MCP and an API on the product, so Claude, ChatGPT, or another assistant can run the account instead of a person clicking through every network. TrustMRR verifies revenue with a Stripe API key. Checked November 9, 2026: current MRR $798, last 30 days $774, all-time revenue $7,433, with 15 active subscriptions. ## Aimake Videos URL: https://www.aiagentsinfo.com/directory/aimake-videos Category: Video Viral AI video generator with trend templates Pricing: Freemium · starting at Credits-based Features: Type: AI tool; Maker: Aimake Videos; Launch year: 2025 Website: https://aimakevideos.com/ Last verified: 2026-11-05 _No body text._ ## AIToolly URL: https://www.aiagentsinfo.com/directory/aitoolly Category: Productivity AI tools directory to discover and compare AI applications. Pricing: Freemium · starting at $9.9/mo · Monthly Features: Type: AI tool; Maker: Andy Zhang (@AIToolly) Website: https://aitoolly.com/ Last verified: 2026-11-08 _No body text._ ## Alexa for Shopping URL: https://www.aiagentsinfo.com/directory/amazon-rufus Category: Shopping agents Amazon Rufus now appears as Alexa for Shopping. On November 6, 2026, Amazon invited some U.S. customers to text it, with a wider rollout planned in the coming weeks. Pricing: Included for U.S. Amazon customers on the app and the website, according to Amazon's October 29, 2026 guide. That page does not list a separate subscription price. The November 6 text feature is also not priced as a separate plan. Features: Questions, comparisons, and price history, as Amazon’s October 29, 2026 guide describes; Auto-buy at a target price, and Buy for Me on eligible products from other stores; Texting with Alexa for Shopping, invited for some U.S. customers on November 6, 2026 Website: https://www.amazon.com Last verified: 2026-11-09 Alexa for Shopping is the name Amazon now uses in its store for the assistant that used to be presented as Rufus. Amazon's [help page](https://www.amazon.com/gp/help/customer/display.html?nodeId=Tvh55TTsQ5XQSFc7Pr) says Rufus and Alexa are being brought together, and that the store experience appears as Alexa for Shopping on the Amazon Shopping app and on Amazon.com. Amazon's [guide updated October 29, 2026](https://www.aboutamazon.com/news/retail/how-to-use-amazon-shopping-ai-assistant) says all U.S. customers can use it. The same guide separates a cart you still check out yourself, an auto-buy that purchases when a price you set is reached, and Buy for Me for eligible products on other stores. Amazon's [Buy for Me article](https://www.aboutamazon.com/news/retail/amazon-shopping-app-buy-for-me-brands) says the brand sends the confirmation and handles delivery, returns, and customer service. An [October 29 post](https://www.aboutamazon.com/news/retail/alexa-for-shopping-new-capabilities) says more than 100 account actions, including lists, registries, Subscribe and Save, and billing questions, are available to all U.S. customers. ## What are the latest Amazon Rufus features and expansion in November 2026? On November 6, 2026, Amazon began inviting some U.S. customers to text Alexa for Shopping, and said the option would reach millions more in the coming weeks. Amazon’s [About Amazon post](https://www.aboutamazon.com/news/retail/alexa-for-shopping-text-to-shop), published November 6, 2026, says a subset of U.S. customers can text a contact named “Alexa Shopping” to search, compare, add items, and complete a purchase. The post says a reorder can be bought with the customer’s approval, and it also says the assistant can complete a purchase on the customer’s behalf. Both sentences are on that page. The rollout is invitation-only at the start. Customers who have not been invited can sign up to hear when more invites go out. The post says a request started over text can carry over to the Amazon Shopping app, Amazon.com, and Alexa-enabled devices. It also names Shop Direct for products from stores across the web, photo search, price-drop alerts, and up to a year of price history. Amazon says more than 400 million customers used Alexa for Shopping in the last 12 months, and that use of its agentic features grew more than eight times since the start of the year. Those figures are Amazon’s. The longer checkout account is [Can an AI shopping agent check out for you?](/blog/can-an-ai-shopping-agent-check-out-for-you). The six-company timeline is [AI shopping agents next 6 months: 6 companies](/reports/ai-shopping-agents-next-6-months). The sourced table is [Best AI agent for shopping](/best/ai-agents-for-shopping). The term for the payment step is [agentic checkout](/glossary/agentic-checkout). ## Apiframe URL: https://www.aiagentsinfo.com/directory/apiframe Category: Developer tools One API key for image, video, and music models. Plans are credit packs, not seats. A small trial is available before the first paid month. Pricing: Basic is $19 for the first month and then $39/mo, with 4,000 credits a month. Pro is $99/mo (10,000 credits), Growth is $399/mo (40,000), and Scale is $749/mo (75,000). A trial includes 100 credits after a phone check or a $1 check. Features: Image, video, and music generation behind one API; The pricing page says every plan includes 70+ models; Credits, concurrency limits, and top-ups; An MCP entry in the site navigation, next to the API docs Website: https://apiframe.ai Last verified: 2026-11-04 Apiframe is an API, not a chat app. You pay for credits and call models from your own software. TrustMRR lists current MRR of $35,714 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $41,832, and all-time revenue is $523,086. ## AriaFlow URL: https://www.aiagentsinfo.com/directory/ariaflow Category: Video AI faceless video generator for social media content with auto-publishing. Pricing: Freemium · starting at $199.99/year · Yearly Features: Type: AI tool; Maker: Andy Zhang Website: https://ariaflow.ai/ Last verified: 2026-11-08 _No body text._ ## Avie URL: https://www.aiagentsinfo.com/directory/avie Category: Customer support AI WhatsApp receptionist, scheduling and CRM for clinics in Brazil Pricing: Paid · starting at R$1,350/month + R$2,700 one-time setup · Monthly Features: Type: SaaS Website: https://avie.me/ Last verified: 2026-11-07 _No body text._ ## Based Labs URL: https://www.aiagentsinfo.com/directory/based-labs Category: Image Indie AI image and short-video generator sold as credit packs, with crypto payment options. Pricing: On basedlabs.ai/pricing (checked twice 2026-11-06): credit packs, not seats, are $10 for 300 credits, $29 for 900, $50 for 1,600 (plus a 5% bonus), $99 for 3,300 (plus 10%), $299 for 10,000 (plus 10%), and a $499 higher pack. Crypto payment is mentioned on the home page (USDC, USDT, or ETH). Features: AI images and short video from credit packs, not seats; Packs from $10 for 300 credits up through a $499 pack; Bonus credits on the $50, $99, and $299 packs; Crypto payment mentioned on the home page (USDC, USDT, or ETH) Website: https://www.basedlabs.ai Last verified: 2026-11-06 Based Labs is Michael Aubry's AI image and short-video generator. You buy credit packs rather than seats. The homepage also mentions paying with crypto (USDC, USDT, or ETH). TrustMRR (https://trustmrr.com/startup/based-labs-ai) lists current MRR of $28,537 from a connected Stripe API key, checked November 6, 2026. All-time revenue on that page is $1,254,077. A last-30-days figure was not in the verified source. ## Bazzly URL: https://www.aiagentsinfo.com/directory/bazzly Category: Marketing Finds and engages Reddit threads that match your product so you can get customers from Reddit on autopilot, using shared warmed accounts and credits. Pricing: On bazzly.ai/pricing (checked twice 2026-11-05): one plan at $199/month with 170 credits and a 7-day free trial. Extra credits are $1 each with a $5 minimum. An additional active project adds a seat at $199/month, prorated. Features: Reddit customer acquisition on autopilot with warmed shared accounts; Single plan with a monthly credit pool; Credit top-ups at $1 per credit (minimum $5); Extra project seats billed at the plan price Website: https://www.bazzly.ai Last verified: 2026-11-05 Bazzly is built for founders who want Reddit distribution without living in the subreddits all day. You buy a credit plan, and the product helps place you in relevant threads. TrustMRR lists current MRR of $32,942 from a connected Stripe API key, checked November 5, 2026. Last 30 days on that page are $37,007, all-time revenue is $92,675, with 179 active subscriptions. ## BeatAPI URL: https://www.aiagentsinfo.com/directory/beatapi Category: AI agents Unified model API for AI agents Pricing: Paid · Usage-based Features: Type: SaaS; Maker: Eric, Startup Website: https://beatapi.io/ Last verified: 2026-11-08 _No body text._ ## Best AI Image Generator URL: https://www.aiagentsinfo.com/directory/best-ai-image-generator Category: Image Paid AI image generation and editing from text prompts or reference images in the browser. Pricing: Paid Website: https://aiimage.io/ Last verified: 2026-11-09 _No body text._ ## ByeDispute URL: https://www.aiagentsinfo.com/directory/byedispute Category: Finance Stripe chargeback helper from Marc Lou that watches payments for fraud patterns and alerts you before a dispute lands. Pricing: On byedispute.com (checked twice 2026-11-07): one-time plans at $69 for 10,000 protected transactions, $119 for 30,000, and $219 for 100,000. Features: Connect Stripe and choose which fraud patterns to watch; Alerts on suspicious payments, with a link back to the Stripe charge; Rules you can set, such as an email alert or a refund; One-time plans by volume at $69, $119, and $219 Website: https://byedispute.com Last verified: 2026-11-07 ByeDispute is Marc Lou's no-code tool for Stripe chargebacks. It listens for payment events, flags patterns such as repeated charges or an early fraud warning, and can run the rules you set. The goal is to catch a bad payment before it becomes a dispute. TrustMRR verifies revenue with a Stripe API key. Checked November 7, 2026: last 30 days $202, all-time revenue $9,141. TrustMRR shows no active subscriptions, which matches a one-time product. ## ChatGPT URL: https://www.aiagentsinfo.com/directory/chatgpt Category: Productivity Consumer and team chatbot from OpenAI, with optional tools such as web browsing and code execution depending on the plan and product surface. Pricing: The ChatGPT pricing page lists Free, Go, Plus, and Pro, and says paid plans are priced per user per month. Dollar amounts shown on the Codex page for those ChatGPT plans are Plus $20, Pro $100, and Business $20 for 2 or more seats billed annually. A dollar price for Go was not in the pricing-page text. Features: Conversational interface over OpenAI models; Optional tools (browse, code, files) on some plans — confirm on the vendor site; Consumer, Plus/Pro, and business SKUs; details change Website: https://chatgpt.com Last verified: 2026-11-03 ChatGPT is OpenAI's chat product. Treat model names, tool availability, and plan limits as moving parts: read the vendor changelog before you depend on a specific capability. Related reading on this site is about [agents](/glossary/agent) versus chat, not an endorsement. ChatGPT desktop Computer Use is compared in [Best computer use agents](/best/computer-use-agents). Consumer jobs that include ChatGPT are [Best AI agent for shopping](/best/ai-agents-for-shopping), [Best AI agent for web search](/best/ai-agents-for-web-search), [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks), [Best AI agent for research](/best/ai-agents-for-research), [Best AI agent for writing](/best/ai-agents-for-writing), [Best AI agent for students](/best/ai-agents-for-students), and [Best AI agent for small business](/best/ai-agents-for-small-business). ## Claude URL: https://www.aiagentsinfo.com/directory/claude Category: Productivity Anthropic's assistant family (Claude), offered in a chat app and via API, with tool use and computer-use features documented by the vendor. Pricing: Free is $0. Pro is $20 per month, or $17 per month when billed annually ($200 up front). Max is $100 per month. Ultra is $200 per month. Enterprise is $20 per seat per month plus usage at API rates, billed annually. Prices exclude tax. On the same page, Opus 5.5 API rates are $4 per million input tokens and $20 per million output tokens. Sonnet 5.5 is $2 and $10. Features: Chat and API access to Anthropic models; Tool use and computer-use capabilities described in Anthropic's docs; Consumer and commercial plans; confirm current names on the vendor site Website: https://claude.ai Last verified: 2026-11-03 Claude is Anthropic's assistant brand. For agent builders, the relevant public materials are Anthropic's own notes on tool use and computer use. Desktop and API computer use are compared in [Best computer use agents](/best/computer-use-agents). Consumer jobs that include Claude are [Best AI agent for web search](/best/ai-agents-for-web-search), [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks), [Best AI agent for writing](/best/ai-agents-for-writing), [Best AI agent for students](/best/ai-agents-for-students), and [Best AI agent for small business](/best/ai-agents-for-small-business). ## Claude Agent SDK URL: https://www.aiagentsinfo.com/directory/claude-agent-sdk Category: Agent frameworks & harnesses A library that puts Claude Code's agent loop inside your own Python or TypeScript app. It already knows how to read files, edit them, run commands, and search the web. Pricing: Anthropic's docs say use of the SDK is governed by its Commercial Terms, except where a component's own license says otherwise. The Python package's GitHub license is MIT. There is no separate SDK subscription on claude.com/pricing. Model use is billed at Claude API rates. On that page, Opus 5.5 is $4 per million input tokens and $20 per million output tokens, and Sonnet 5.5 is $2 and $10. Managed Agents, a different hosted product, is $0.08 per active session-hour plus standard token rates. Features: The same built-in tools, permissions, sessions, and hooks as Claude Code; Subagents, MCP connections, skills, and plugins, as listed in the docs; Python and TypeScript libraries. Other languages can drive the Claude Code CLI instead; Third-party products must use an API key. Anthropic does not allow them to offer claude.ai login unless previously approved Website: https://code.claude.com/docs/en/agent-sdk/overview Last verified: 2026-11-03 The Agent SDK is the [harness](/glossary/harness) behind Claude Code, packaged as a library you run. It is not the Claude chat app. A sourced comparison with other runtimes is [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## Claude Code URL: https://www.aiagentsinfo.com/directory/claude-code Category: Coding agents Anthropic’s coding agent for the terminal, editors, the web, mobile, and Slack. It can explain a codebase, write code, run tests, and open pull requests. Anthropic Claude Code limits on Pro and Max are a five-hour session and a weekly cap, shared with the rest of Claude. Claude Code vs Codex list prices are in a separate guide. Claude Duet is a separate proactive agent, not a Claude Code mode. Pricing: Not included on the Free plan ($0). Included on Pro ($20 per month, or $17 per month when billed annually at $200 up front), Max ($100 per month), and Ultra ($200 per month). Those plans share one usage pool with the rest of Claude. With a Claude Console account, use is billed at standard API rates. Enterprise is $20 per seat per month plus usage at API rates, billed annually. Prices exclude tax. Features: Terminal app plus VS Code, JetBrains, web, iOS, Android, GitHub, and Slack, as listed on the product page; Can explain a codebase, write code, run tests, and open pull requests; In the terminal it runs on your computer and asks before it edits files or runs commands; Remote Control in the Claude iOS and Android app can drive a Claude Code session that is already running on your computer; Claude Duet is a separate research-preview agent for Pro, Max, and Ultra in the US, not a Claude Code mode Website: https://claude.com/product/claude-code Last verified: 2026-11-07 Claude Code is a [coding agent](/glossary/agent). The product page is the source for where it runs and which plan includes it. Anthropic Claude Code limits for Pro, Max, and Ultra are in the section below. The comparison of list prices with Codex is a separate article, [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing), not the name of this product. The harness comparison is [Best AI agent harness](/best/ai-agent-harnesses). Computer use on the desktop is in [Best computer use agents](/best/computer-use-agents). ## How is Claude Code different from Codex? [Claude Code](https://claude.com/product/claude-code) is Anthropic’s coding agent. [Codex](/directory/codex) is OpenAI’s. Anthropic’s page was read again on November 7, 2026, and the notes below still match it. The Codex notes are what OpenAI’s page said on November 5, 2026. They are not a quality ranking. Anthropic’s page says you steer Claude Code from the terminal, an IDE, Slack, or the web. The surface list is the terminal, the web, iOS, Android, GitHub, VS Code, JetBrains, and Slack. The editor line says the VS Code extension also covers [Cursor](/directory/cursor) and Devin Desktop. The page says it is available for macOS, Linux, and Windows. Its FAQ says the terminal version runs locally and asks before it changes files or runs commands. The same FAQ says you can use it with a Pro, Max, or Ultra plan, a Team or Enterprise plan, or a Claude Console account. The Team block on that page is $20 per seat per month when billed annually, for teams of 2 to 150. OpenAI’s page says Codex is included with a ChatGPT plan, and that you use the same agent in ChatGPT, an editor, and the terminal. It also describes an IDE extension, cloud tasks that keep running after you close the laptop, and automatic pull-request review. The prices printed there are Plus at $20, Pro at $100, and Business at $20. Business is noted for two or more seats, billed annually. ## How does Claude Code vs Codex pricing compare? On the pages checked November 5, 2026, both vendors list a $20 plan, and the rest of the ladder is not the same. This page’s pricing summary lists Claude Code Pro at $20 per month, Max at $100, and Ultra at $200, and says Claude Code is not on the Free plan. The Codex paragraph above lists Plus at $20, Pro at $100, and Business at $20 for two or more seats, billed annually. The [Codex listing](/directory/codex) records a different free-tier rule: Codex is included on Free and Go, with limits, and Codex Cloud is not. Those are list prices from the vendor pages, not a measure of how much work each plan includes. The full ladder, rechecked November 6, 2026, including Claude Team standard and premium seats and Codex Pro at $100, $200, and $500, is [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). The software around either model is a [harness](/glossary/harness). The longer guide is [What is an AI agent harness?](/blog/what-is-an-ai-agent-harness). Plan limits move. Use the vendor page for a number you intend to rely on. ## What is Claude Duet? [Claude Duet](https://claude.com/product/claude-duet) is not a Claude Code mode. Anthropic’s product page, read November 7, 2026, calls it a proactive agent you hand work to through messages. It works on its own computer in the cloud, using apps you connect, and it messages you when it is done or needs a decision. The same page says it is a standalone app for iOS and Android, and that on a computer you open it at claude.ai. It is a research preview on a waitlist for Claude Pro, Max, and Ultra subscribers in the United States. Team and Enterprise plans are not included yet. During that preview it shares the plan’s usage limits with the rest of Claude. The [announcement](https://claude.com/resources/articles/claude-duet) says the same about the plans and the shared limits. Phone control of a Claude Code session is a different feature. Anthropic’s [mobile docs](https://code.claude.com/docs/en/mobile), read November 7, 2026, say the Claude app for iOS and Android is a client for Claude Code sessions, not a place where the code runs. [Remote Control](https://code.claude.com/docs/en/remote-control) connects that app, or claude.ai/code, to a session that is already running on your computer. The code stays on your machine, so the computer has to stay on. Claude Duet’s announcement says its virtual machine keeps working when your laptop is off. Those are two products. ## What are the Anthropic Claude Code limits? On a Pro, Max, or Ultra plan, Claude Code draws from the same usage pool as the rest of Claude. Anthropic’s [help center article](https://support.claude.com/en/articles/11145838-use-claude-code-with-your-pro-max-or-ultra-plan), read November 7, 2026, says Pro and Max have a five-hour session limit and a weekly limit. Ultra has a weekly limit, and verified Ultra accounts do not have a five-hour session limit. The [pricing page](https://claude.com/pricing), read the same day, says Max is 5 times Pro usage every five hours and every week, and Ultra is 10 times Pro usage every week, with no five-hour limit for verified accounts. The page also says there is no fixed message count. The side-by-side with [Codex](/directory/codex) is [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). Prices for Claude Code, Codex, Cursor, and GitHub Copilot re-read on November 9, 2026 are [Best AI coding agents](/best/ai-coding-agents). Ultracode spends that same usage pool. The November 9 reading of the workflows and model-config docs is [What does ultracode do in Claude Code?](/blog/what-does-ultracode-do-in-claude-code). The [Claude Code changelog](https://code.claude.com/docs/en/changelog) for version 2.1.317, dated November 3, 2026, says that release added a check that asks before a web or shell tool call sends your account email address to the network. That line is the changelog note. This page does not have a separate incident report, and it does not describe versions before 2.1.317. ## ClimbX URL: https://www.aiagentsinfo.com/directory/climbx Category: Social media Drafts X posts in your voice from what is already working in your niche, and exposes an API and MCP so an agent can use the same account. Pricing: On climbx.so/pricing (checked twice 2026-11-09): Base is $29/month for 1 X account, 300 AI credits a day, scheduling, and API plus MCP access. Pro is listed at a founding price of $29/month with $39 shown as the later price, and adds 3 X accounts, 800 AI credits a day, and a 60-day growth plan. Every plan starts with a 7-day trial for $1. The yearly toggle advertises 2 months free. Features: Drafts and a reply feed based on posts that are doing well in your niche; Cliff, the in-product AI co-writer; Scheduling, analytics, and an API plus MCP on the Base plan; 7-day trial for $1, then a monthly plan Website: https://climbx.so Last verified: 2026-11-09 ClimbX is Daniel Smidstrup’s writing tool for X. It looks at posts that are already doing well in your niche and drafts in your voice, then you schedule from the same product. Base includes an API and MCP, so an agent can publish, schedule, and read analytics instead of you living in the X app. TrustMRR verifies revenue with a Stripe API key. Checked November 9, 2026: current MRR $1,189, last 30 days $1,463, all-time revenue $8,142, with 41 active subscriptions. ## CNVS URL: https://www.aiagentsinfo.com/directory/cnvs Category: Coding agents macOS canvas from Max Blade that runs Claude, Cursor, and Codex side by side. You direct the agents, including by voice, and pay once. Pricing: On cnvs.dev (checked twice 2026-11-09): pay once. The founder license is $99 (5 left at check), then $129 for licenses 6–50 and $169 for licenses 51–150. The page says that covers every agent, voice, and themes, plus 1.x updates, and that it works with your existing AI subscriptions. Features: One canvas runs Claude, Cursor, and Codex in parallel; Voice control on a native macOS app; Uses the AI subscriptions you already pay for; One-time license, with no monthly CNVS fee Website: https://cnvs.dev Last verified: 2026-11-09 CNVS is Max Blade’s agent harness for people who already use Claude, Cursor, or Codex. The Mac app puts those coding agents on one canvas so you can direct them, including by voice, instead of babysitting three separate windows. TrustMRR verifies revenue with a Stripe API key. Checked November 9, 2026: last 30 days $1,990, all-time revenue $50,782. TrustMRR shows no active subscriptions, which matches a one-time product. TrustMRR still lists an older $199 price. The live site prices above are the ones to use. ## CodeFast URL: https://www.aiagentsinfo.com/directory/codefast Category: Developer tools Bootcamp-style course from Marc Lou that teaches indie hackers to ship a SaaS with Next.js, payments, and launch habits. Pricing: On codefa.st (checked twice 2026-11-07): the course is $169; the Course + ShipFast bundle is $299. Both are one-time. PayPal is listed at the same amounts. Features: About 12 hours of short lessons that end in building a SaaS; Next.js, auth, a database, and Stripe or Lemon Squeezy payments; One-time course at $169, or the course plus ShipFast at $299; Lifetime access, with a 7-day refund if under 10% of the course was watched Website: https://codefa.st Last verified: 2026-11-07 CodeFast is Marc Lou's course for people who want to ship a small SaaS. The lessons cover the basics, then a Next.js app with auth, a database, and payments. You buy it once. TrustMRR verifies revenue with a Stripe API key. Checked November 7, 2026: last 30 days $2,938, all-time revenue $836,070. TrustMRR shows no active subscriptions, which matches a one-time product. ## Codex URL: https://www.aiagentsinfo.com/directory/codex Category: Coding agents Codex usage limits depend on the plan. Free and Go include it with limits; Codex Cloud is not on those plans. Plus is $20. Checked November 6, 2026. Pricing: Checked November 6, 2026. Codex docs list Free at $0, Go at $8 a month, Plus at $20, Pro from $100 with tiers at $100, $200, and $500, and Business at $20 per user per month for 2 or more users billed annually ($25 billed monthly). Codex is included on Free and Go with limits. The help center says Codex Cloud is not included on Free or Go. Features: Included with ChatGPT plans, with limits that depend on the plan; Clients listed by OpenAI include ChatGPT, the Codex CLI, an IDE extension, and the web; Cloud tasks can keep running after you close your laptop; Pull-request review is described on the product page Website: https://openai.com/codex/ Last verified: 2026-11-06 Codex is a coding [agent](/glossary/agent). Plan limits and which models are available change, so check OpenAI's current pricing and help pages before you rely on a number. ## What are Codex usage limits? Codex usage limits depend on the plan. It is included on Free and Go with limits, and the help center says Codex Cloud is not included on Free or Go. ## What is Codex pricing in 2026? Checked November 6, 2026, Codex docs list Free at $0, Go at $8 a month, Plus at $20, Pro from $100 with tiers at $100, $200, and $500, and Business at $20 per user per month for 2 or more users billed annually ($25 billed monthly). The comparison with Claude Code is [Claude Code vs Codex pricing](/blog/claude-code-vs-codex-pricing). The harness comparison is [Best AI agent harness](/best/ai-agent-harnesses). Prices re-read on November 9, 2026, including Cursor and GitHub Copilot, are [Best AI coding agents](/best/ai-coding-agents). How to test a harness is [How to evaluate an AI agent harness](/blog/how-do-you-evaluate-an-ai-agent-harness). Public board scores are copied in the [November 2026 agent harness benchmark](/reports/agent-harness-benchmark-november-2026). Desktop Computer Use is in [Best computer use agents](/best/computer-use-agents). ## CrewAI URL: https://www.aiagentsinfo.com/directory/crewai Category: Agent frameworks & harnesses Framework for role-based multi-agent workflows, with open-source libraries and a commercial platform described by the vendor. Pricing: The cloud pricing page lists Basic at free, with 50 workflow executions a month, and Enterprise at a custom price. The open-source framework's GitHub license is MIT. Features: Role and crew abstractions for multi-agent runs; Open-source package plus a hosted product (confirm current names on the vendor site); Integrations depend on the release Website: https://crewai.com Last verified: 2026-11-03 [Multi-agent](/glossary/multi-agent) in product copy often means multiple prompts or roles, not multiple companies. Read the docs for the actual execution model. The framework row is in [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## Cursor URL: https://www.aiagentsinfo.com/directory/cursor Category: Coding agents Cursor AI is a code editor with an agent, sold with Grok. November 9, 2026: Free $0, Plus $8, Premium $30, Super $100, Ultra $200 a month. Other models cost extra. Pricing: Checked November 9, 2026. Free is $0. Plus is $8 per month ($80 per year). Premium is $30 per month ($300 per year). Super is $100 per month ($1,000 per year). Ultra is $200 per month ($2,000 per year). Teams is $30, $100, or $200 per seat per month. Enterprise is custom. Third-party models are not included. Prices exclude tax. Features: Code editor with Agent and Chat; New plans are part of The X Subscription, shared with Grok; Plus includes Grok and Composer models only. Third-party models start on Premium, at API list price Website: https://cursor.com Last verified: 2026-11-09 ## What is Cursor AI? Cursor AI is Anysphere’s code editor with an agent that can edit a repository. New customers buy it inside The X Subscription, one subscription for Grok, Cursor, and X, rather than as a separate Cursor-only plan. The [pricing docs](https://cursor.com/docs/models-and-pricing), read November 9, 2026, say paid plans include a weekly pool of usage for Cursor’s own models, shared with Grok. Those models are Grok 4.7, Grok 4.6, Grok 4.5, and Composer 2.5. Plus runs them at fixed settings and does not include third-party models, Auto, Cloud Agents, Automations, or Bugbot. Premium and above add those, and third-party models are billed at API list price from prepaid credit. People who subscribed before The X Subscription stay on a legacy Cursor plan until they switch or cancel. Compare it with other [coding agents](/directory/category/coding-agents). The priced table is [Best AI coding agents](/best/ai-coding-agents). The software around the model is an [agent harness](/topics/harnesses). ## How much does Cursor cost? On November 9, 2026, [cursor.com/pricing](https://cursor.com/pricing) lists Free at $0, Plus at $8 a month, Premium at $30, Super at $100, and Ultra at $200. Annual billing is $80, $300, $1,000, and $2,000. Teams is $30, $100, or $200 per seat a month. Enterprise is custom. Prices exclude tax, and third-party model use is not included in the monthly price. Plus is the plan that includes only Grok models. Premium is 4 times Plus usage, Super is 15 times, and Ultra is 50 times, on the docs page read the same day. The pool refreshes weekly. A token bill for a model API, separate from these seat prices, is the [AI agent cost calculator](/agent-cost-calculator). ## Damon Salvatore Church Edit AI Video Generator URL: https://www.aiagentsinfo.com/directory/damon-salvatore-church-edit-ai-video-generator Category: Video Recreate the Damon Salvatore church edit with your face — 10s 9:16 AI video with reference audio. Pricing: Paid · starting at 6.99 · One-time Features: Type: AI tool; Maker: Shenmei Tu, Indie hacker Website: https://damonsalvatoreaivideo.app/ Last verified: 2026-11-08 _No body text._ ## DeskBot for Kalshi URL: https://www.aiagentsinfo.com/directory/deskbot-for-kalshi Category: Finance Desktop automation for Kalshi prediction markets on Mac and Windows. Pricing: Paid · starting at $1 first month then $19.99/mo or $155/yr · Monthly Features: Type: App; Maker: DeskBot (@deskbotapp), Startup; Launch year: 2025 Website: https://deskbot.store/products/deskbot-kalshi-bot Last verified: 2026-11-06 _No body text._ ## Devin URL: https://www.aiagentsinfo.com/directory/devin Category: Coding agents A software-engineering agent from Cognition. It writes and tests code in the cloud or on your computer, including in its own browser, and is built to keep going until a pull request is ready. Pricing: On devin.ai/pricing: Free is $0. Pro is $20 per month. Max is $200 per month. Teams is $80 per month plus $40 per month for each full developer seat, for up to 200 users. Enterprise is custom. Each paid plan includes a usage allowance that refreshes daily and weekly. Features: Devin Desktop for local work, plus Devin Cloud on paid plans; Free plan includes a light agent quota, limited model choice, unlimited inline edits, and unlimited Tab completions; Paid plans add higher quotas. Extra usage past the included allowance is charged at API pricing; On 9 November 2026, windsurf.com returned HTTP 308 to https://devin.ai/desktop Website: https://devin.ai Last verified: 2026-11-03 ## What is Devin IDE? Devin IDE is the embedded code editor inside a Devin web session. Cognition's [introducing Devin](https://docs.devin.ai/get-started/devin-intro) page, read November 9, 2026, calls Devin an autonomous AI software engineer that can write, run, and test code. It says you follow that work in an embedded IDE. The [session tools](https://docs.devin.ai/work-with-devin/devin-session-tools) page names three tools in a session: the Shell, the IDE, and the Browser. Under the heading Devin IDE, it says Devin works in an interactive VS Code environment loaded with your repos, and that you can watch edits and take over the editor without leaving the Devin web app. [Cognition's homepage](https://cognition.com/) says it operates Devin, the first autonomous software engineer. That is the company's line. On November 9, 2026, cognition.ai returned HTTP 301 to that page. The homepage does not call Devin itself an IDE. [Devin Desktop](https://docs.devin.ai/desktop/getting-started) is a different surface. That page calls it a next-generation AI IDE for Mac, Windows, and Linux. The [Desktop FAQ](https://docs.devin.ai/desktop/devin-desktop-faq) says Devin Desktop is the new name for Windsurf: the same IDE and the same editor, under the Devin brand, with the change dated June 2, 2026. A [VS Code troubleshooting page](https://docs.devin.ai/desktop/troubleshooting/plugins-enterprise/vscode) says the VS Code plugin is in maintenance mode and recommends the native Devin Desktop editor or the JetBrains local plugin. That page does not name a Devin IDE extension. Devin is a coding [agent](/glossary/agent). Plan prices on this listing were last checked November 3, 2026. Check [devin.ai](https://devin.ai) before you quote a current price. ## Diagnosed by AI URL: https://www.aiagentsinfo.com/directory/diagnosed-by-ai Category: Health AI-powered multi-agent medical diagnosis platform - educational perspectives on health cases. Pricing: Free Features: Type: AI tool; Launch year: 2026 Website: https://diagnosedbyai.com/ Last verified: 2026-11-08 _No body text._ ## DisconfirmAI URL: https://www.aiagentsinfo.com/directory/disconfirmai Category: Finance Investment research that looks for the evidence against your thesis first Pricing: Freemium Features: Type: SaaS; Launch year: 2026 Website: https://disconfirmai.com/en Last verified: 2026-11-09 _No body text._ ## Dog Bartender AI Video Generator URL: https://www.aiagentsinfo.com/directory/dog-bartender-ai-video-generator Category: Video Upload one photo to create a 15-second 9:16 Dog Bartender AI video with the $1 vs $5 vs $50 drink sequence and sound. Pricing: Paid · starting at 7.99 · One-time Features: Type: AI tool; Maker: Shenmei Tu, Indie hacker Website: https://dogbartender.com/ Last verified: 2026-11-08 _No body text._ ## Effect Farm URL: https://www.aiagentsinfo.com/directory/effect-farm Category: Video AI photo to video effects studio Pricing: Paid · starting at $4.99/video · One-time Features: Type: AI tool; Maker: Eason Website: https://effectfarm.com/ Last verified: 2026-11-09 _No body text._ ## ElevenLabs URL: https://www.aiagentsinfo.com/directory/elevenlabs Category: Audio / voice Speech synthesis and voice-agent platform. The feature set (text to speech, speech to speech, agents) is defined on elevenlabs.io. Pricing: On monthly billing: Free is $0 with 10,000 credits a month. Starter is $1 the first month, then $6 a month, with 30,000 credits. Creator is $11 the first month, then $22 a month, with 121,000 credits. Pro is $99 with 600,000 credits. Scale is $299 with 1.8 million credits. Business is $990 with 6 million credits. Enterprise is custom. Features: Text-to-speech APIs; Voice cloning and agent products as documented by the vendor; Usage billed by plan and by characters or minutes Website: https://elevenlabs.io Last verified: 2026-11-03 Voice products carry consent and impersonation risks. This listing does not evaluate safety claims. ## EnhanceWizard URL: https://www.aiagentsinfo.com/directory/enhancewizard Category: Image Free AI image enhancer: upscale, unblur and sharpen photos online. Pricing: Freemium Features: Type: AI tool; Maker: EnhanceWizard Team, Indie hacker Website: https://enhancewizard.com/ Last verified: 2026-11-06 _No body text._ ## EveryGen AI URL: https://www.aiagentsinfo.com/directory/everygen-ai Category: Video Browser AI video studio — prompt or image to video Pricing: Paid · starting at $9.9/mo · Monthly Features: Type: AI tool Website: https://everygenai.video/ Last verified: 2026-11-08 _No body text._ ## Feather URL: https://www.aiagentsinfo.com/directory/feather Category: Productivity Turns Notion pages into a fast, SEO-oriented blog without coding. Each blog is its own site with unlimited articles. Pricing: On feather.so/pricing (checked twice 2026-11-05): blogs start at $39/month. You can create as many blog sites as you want; each blog is a separate website with unlimited articles. Features: Publish a blog from Notion pages; SEO-oriented hosting and structure without a custom codebase; Unlimited articles per blog site; Multiple blogs allowed, each billed from the starting price Website: https://feather.so Last verified: 2026-11-05 Feather is the Notion-to-blog product in Tibo’s portfolio. Write in Notion, publish on Feather, and keep SEO basics handled without standing up WordPress. ## Floowed URL: https://www.aiagentsinfo.com/directory/floowed Category: Automation Floowed is the operations platform: AI agents do tasks; Floowed runs whole operations from first request to outcome. Pricing: Freemium · starting at $100/mo · Monthly Features: Type: SaaS; Maker: Michel de Rijk, Startup Website: https://www.floowed.com/ Last verified: 2026-11-07 _No body text._ ## Frameow URL: https://www.aiagentsinfo.com/directory/frameow Category: Image AI-generated digital cat portraits and wallpapers from your photo. Pricing: Freemium Features: Type: AI tool Website: https://frameow.com/ Last verified: 2026-11-09 _No body text._ ## FreeGenVideo URL: https://www.aiagentsinfo.com/directory/freegenvideo Category: Video Create short videos from text prompts or reference images in a browser Pricing: Freemium · starting at 1 free video per day; paid options Features: Type: AI tool Website: https://freegen.video/ Last verified: 2026-11-09 _No body text._ ## Gemini URL: https://www.aiagentsinfo.com/directory/gemini Category: Productivity Google's Gemini app and model family, integrated across Google consumer and Cloud products. Capabilities differ by surface (app, API, or Workspace). Pricing: Google's AI plans page lists Google AI Plus at $4.99 per month, Google AI Pro at $19.99 per month, Google AI Ultra 5x at $99.99 per month, and Google AI Ultra 20x at $199.99 per month. Features: Consumer Gemini app; Gemini models via Google AI and Vertex AI; Multimodal inputs on supported surfaces Website: https://gemini.google.com Last verified: 2026-11-03 Google ships Gemini through several products. Name the surface you mean (the Gemini app, AI Studio, or Vertex) when you write about it. Computer Use on the Gemini API is compared in [Best computer use agents](/best/computer-use-agents). Consumer jobs that include Gemini are [Best AI agent for shopping](/best/ai-agents-for-shopping), [Best AI agent for web search](/best/ai-agents-for-web-search), [Best AI agent for travel booking](/best/ai-agents-for-travel-booking), [Best AI agent for email](/best/ai-agents-for-email), [Best AI agent for personal assistant tasks](/best/ai-agents-for-personal-assistant-tasks), [Best AI agent for research](/best/ai-agents-for-research), [Best AI agent for scheduling](/best/ai-agents-for-scheduling), [Best AI agent for writing](/best/ai-agents-for-writing), and [Best AI agent for students](/best/ai-agents-for-students). ## GitHub Copilot URL: https://www.aiagentsinfo.com/directory/github-copilot Category: Coding agents GitHub Copilot price on November 9, 2026: Free $0, Pro $10, Pro+ $39, and Max $100 a month. Pro includes 1,500 AI credits. Paid plans can buy more. Pricing: Checked November 9, 2026. Free is $0, with 2,000 completions a month and limited chat and agent use. Pro is $10 a month, with 1,500 AI credits (1,000 base and 500 flex). Pro+ is $39 a month, with 7,000 credits (3,900 base and 3,100 flex). Max is $100 a month, with 20,000 credits (10,000 base and 10,000 flex). One credit is $0.01. Features: Completions, chat, agent mode, and a cloud agent, as GitHub’s plans page lists them; Free includes 2,000 completions a month. Paid plans list completions as unlimited; Pro, Pro+, and Max include a monthly GitHub AI credit allowance Website: https://github.com/features/copilot Last verified: 2026-11-09 ## What is the GitHub Copilot price in November 2026? On November 9, 2026, GitHub lists Copilot Free at $0, Pro at $10 a month, Pro+ at $39 a month, and Max at $100 a month. [GitHub’s plans page](https://github.com/features/copilot/plans) prints those four stickers. [The billing docs](https://docs.github.com/en/copilot/concepts/billing-and-usage/individuals/billing), read the same day, say one AI credit is $0.01. Pro includes 1,500 credits a month (1,000 base and 500 flex). Pro+ includes 7,000 (3,900 base and 3,100 flex). Max includes 20,000 (10,000 base and 10,000 flex). Free includes 2,000 code completions a month and limited chat and agent use through auto model selection. Paid plans list completions as unlimited and not billed in credits. Unused credits do not carry over. The allowance resets at 00:00 UTC on the first of the month. The same docs say that through November 30, 2026, paid plans get a 30 percent discount on model costs when using Copilot Fusion, and a 10 percent discount when using auto model selection. Organization and enterprise seat prices are on GitHub’s licence page. They are not repeated here. The four-product price table is [Best AI coding agents](/best/ai-coding-agents). A per-token bill, separate from these seats, is the [AI agent cost calculator](/agent-cost-calculator). The harness around a coding agent is covered on [Agent harnesses](/topics/harnesses). ## GojiberryAI URL: https://www.aiagentsinfo.com/directory/gojiberryai Category: Sales AI prospecting agents that find warm, high-intent leads and help turn buying signals into outreach. Pricing: On gojiberry.ai/pricing (checked twice 2026-11-06): Pro is $99/month for 2 AI agents and up to about 1,800 contacted prospects a month. Custom pricing for larger teams and agencies. Features: Prospecting agents that look for warm, high-intent leads; Help turning buying signals into outreach; Pro includes 2 AI agents and up to about 1,800 contacted prospects a month; Custom pricing for larger teams and agencies Website: https://gojiberry.ai Last verified: 2026-11-06 GojiberryAI, from Romàn Czerny, runs prospecting agents that look for warm, high-intent leads and help turn those buying signals into outreach. TrustMRR (https://trustmrr.com/startup/superfruits) lists current MRR of $424,368 from a connected Stripe API key, checked November 6, 2026. Last 30 days on that page are $407,199, and all-time revenue is $1,801,629. ## Google ADK URL: https://www.aiagentsinfo.com/directory/google-adk Category: Agent frameworks & harnesses Google's open-source Agent Development Kit. You write an agent in code, give it tools, and can grow it into a multi-agent workflow. It is available in Python, TypeScript, Go, Java, and Kotlin. Pricing: The framework is free under the Apache License 2.0. Google does not list a price for ADK itself. If you call Gemini or run on Google Cloud, those services are billed separately by Google. Features: Code-first agents with tools, including a Google Search tool in the official examples; Graph-style workflows, which the site describes as new in ADK 2.0; Run it on your own servers, or deploy to Google Cloud; The Python repository is Apache License 2.0 Website: https://adk.dev Last verified: 2026-11-03 ADK is an agent framework. The library is free. The model you connect may not be. A sourced comparison is [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## Grok Bot Kit URL: https://www.aiagentsinfo.com/directory/grok-bot-kit Category: Education Practical Grok Bot tutorials, workflows, and copy-ready bot templates. Independent, source-linked guides from setup to r Pricing: Freemium · starting at $0 Features: Maker: Yuanwai Dong Website: https://grokbotkit.com/ Last verified: 2026-11-08 _No body text._ ## Havaron URL: https://www.aiagentsinfo.com/directory/havaron Category: Finance Gated AI digital employees for finance & operations, each on its own machine, work email and chat; sensitive actions need human approval. Pricing: Paid Features: Type: AI tool Website: https://havaronai.com/ Last verified: 2026-11-06 _No body text._ ## HeadshotPro URL: https://www.aiagentsinfo.com/directory/headshotpro Category: Image AI professional headshots from a few selfies. One-time packs for individuals and credit packs for teams. Pricing: On headshotpro.com/pricing (checked twice 2026-11-06): individual one-time packs are Basic $29 (30 headshots), Professional $39 (50), and Executive $59 (70). Team credits start at about $19.50 per shoot. Enterprise is custom. No subscription for individuals. Features: Professional headshots generated from a few selfies; One-time individual packs (Basic, Professional, and Executive); Team credits, starting around $19.50 per shoot; Enterprise pricing is custom. Individuals are not on a subscription Website: https://www.headshotpro.com Last verified: 2026-11-06 HeadshotPro makes professional headshots from a few selfies. Individuals buy a one-time pack. Teams buy credits. There is no individual subscription. ## HireDov URL: https://www.aiagentsinfo.com/directory/hiredov Category: AI agents AI sourcing agent that finds suppliers and gets you quotes Pricing: Paid Features: Type: AI tool Website: https://hiredov.com/ Last verified: 2026-11-08 _No body text._ ## Html2MP4 URL: https://www.aiagentsinfo.com/directory/html2mp4 Category: Video Html2MP4 turns HTML, website URLs, prompts, and ready-made templates into frame-accurate MP4 videos online (powered by HyperFrames). Pricing: Paid · starting at $5 · One-time Features: Type: AI tool; Maker: Everitere (@Everitere1), Indie hacker Website: https://html2mp4.com/ Last verified: 2026-11-08 _No body text._ ## HumanPen URL: https://www.aiagentsinfo.com/directory/humanpen Category: Writing Rewrites your product copy so it reads like local writing. Pricing: Freemium Features: Type: AI tool; Maker: Weijie Li Website: https://humanpen.net/ Last verified: 2026-11-05 _No body text._ ## ImgEdify URL: https://www.aiagentsinfo.com/directory/imgedify Category: Image & video Free online AI photo editor and AI image generator. Pricing: Freemium · starting at $7.92/mo billed yearly · Yearly Features: Type: AI tool; Maker: Andy Zhang (@imgedify) Website: https://imgedify.com/ Last verified: 2026-11-08 _No body text._ ## Indie Page URL: https://www.aiagentsinfo.com/directory/indie-page Category: Marketing One-page portfolio from Marc Lou where indie makers list products, tell their story, and share startup links. Pricing: On indiepa.ge (checked twice 2026-11-07): 1-Year access is $25; Lifetime is $45. The page showed higher struck-through prices ($55 and $75) beside those amounts. Features: A single page for products, a short story, and links; Optional Stripe or Lemon Squeezy connection for a revenue badge; Click and traffic stats on the startups you list; 1-Year access at $25, or a lifetime deal at $45 Website: https://indiepa.ge Last verified: 2026-11-07 Indie Page is Marc Lou's one-page portfolio for indie makers. You list products, write a short story, and send people to waitlists or live sites. A Stripe or Lemon Squeezy connection can show a revenue badge. The cheaper plan is one year. Lifetime access is a separate one-time price. ## InteriorAI URL: https://www.aiagentsinfo.com/directory/interiorai Category: Design Pieter Levels indie app for AI interior redesign and virtual staging from a room photo. Pricing: On interiorai.com/pricing (checked twice 2026-11-06): Pro $49/month (yearly $349, about $29/month), Premium $99/month (yearly $599, about $49/month), VR Ultra $199/month (yearly $1,199, about $99/month). Yearly is marketed as 6+ months free. Features: Interior redesign from a photo of a room; Virtual staging; Pro, Premium, and VR Ultra plans; Yearly billing marketed as 6+ months free Website: https://interiorai.com Last verified: 2026-11-06 InteriorAI is Pieter Levels's indie app for redesigning a room from a photo and for virtual staging. You pick a plan and upload a picture of the space. ## Khanmigo URL: https://www.aiagentsinfo.com/directory/khanmigo Category: Education Khan Academy's AI tutor and teaching assistant, with learner, teacher, and district offerings described on khanmigo.ai. Pricing: Khan Academy's pricing page says Khanmigo is free for teachers. For parents and learners it is $4 per month or $44 per year. Features: Tutoring-style chat grounded in Khan Academy materials, as the vendor describes it; Teacher tools documented on the product site; Teacher signup is offered free on the product site; learner and district access are described there and change Website: https://www.khanmigo.ai Last verified: 2026-11-03 Education tools need supervision policies that this listing does not evaluate. See [agents in education](/topics/education). Khanmigo is compared in [Best AI agent for students](/best/ai-agents-for-students). ## Kling AI URL: https://www.aiagentsinfo.com/directory/kling-ai Category: Video AI image and video generation Pricing: Paid Website: https://kling.movie/ Last verified: 2026-11-09 _No body text._ ## Koala AI URL: https://www.aiagentsinfo.com/directory/koala Category: SEO AI writer for long-form SEO articles, images, and internal links, with an MCP server on paid plans so your own assistant can call it. Pricing: On koala.sh/pricing (checked twice 2026-11-09): Free is $0 and includes 5,000 words. Starter is $25/month (45,000 words). Professional is $49/month (100,000 words). Boost is $99/month (250,000 words). Higher monthly plans on the same page are Growth $179, Elite $350, Advanced $500, Scale I $750, Scale II $1,250, and Scale III $2,000. Annual billing is advertised at 20% off. Paid plans include an MCP server. Features: Article drafts with SEO checks, images, and internal links; One subscription covers the writer, images, and a brand profile; MCP server included on paid plans; Free starting allowance, then monthly word plans Website: https://koala.sh Last verified: 2026-11-09 Koala AI writes and edits SEO articles, then handles images and internal links in the same subscription. Paid plans include an MCP server, so an assistant you already use can call Koala instead of you copying drafts between tools. New accounts start with a free word allowance and no card. ## LangChain URL: https://www.aiagentsinfo.com/directory/langchain Category: Agent frameworks & harnesses Open-source framework and commercial platform (LangSmith and related products) for composing language-model applications and agents. Pricing: The framework is open source. LangSmith Developer is $0 per seat per month, then pay as you go, with up to 5,000 base traces a month. Plus is $39 per seat per month, then pay as you go, with up to 10,000 base traces a month. Enterprise is custom. One LangChain Standard Unit is listed at $1. Features: Open-source libraries for chains, tools, and agents; Observability and eval products under the LangSmith family; Docs and versioning on langchain.com and GitHub Website: https://www.langchain.com Last verified: 2026-11-03 A framework is not a finished agent. The [harness](/glossary/harness) is the runtime around the model. LangSmith eval pricing is in [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). The framework row is in [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## LLM Gateway URL: https://www.aiagentsinfo.com/directory/llm-gateway Category: Developer tools LLM Gateway is an open-source API gateway for Large Language Models (LLMs). Pricing: Paid · starting at $9/mo · Monthly Features: Maker: LLM Gateway, Indie hacker Website: https://llmgateway.io/ Last verified: 2026-11-03 _No body text._ ## LocalRank URL: https://www.aiagentsinfo.com/directory/localrank Category: SEO Local SEO software for citations, rank tracking, and reviews. The homepage also tracks whether ChatGPT, Claude, Gemini, and Perplexity recommend a business. Pricing: Monthly prices are Starter $57/mo, Pro $297/mo, Scale $497/mo, and Enterprise $2,997/mo. Billed annually on the same page: Starter $47.50/mo, Pro $247.50/mo, Scale $372.50/mo, and Enterprise $1,664.17/mo. Buttons say Start for Free, and the hero says no credit card is required. Features: Citation building and a local rank tracker; An AI recommendation monitor for ChatGPT, Claude, Gemini, and Perplexity; Google Business Profile tools and review requests on higher plans; Monthly and annual prices on the homepage Website: https://localrank.so Last verified: 2026-11-04 LocalRank is local SEO software. The AI piece on the homepage is a monitor of what answer engines say about a business, plus citation tools aimed at those answers. TrustMRR lists current MRR of $19,392 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $14,531, and all-time revenue is $741,060. ## LogoFast URL: https://www.aiagentsinfo.com/directory/logofast Category: Design Free AI logo maker from Marc Lou. Generate simple logos without a design tool. Pricing: On logofa.st (checked twice 2026-11-06): free AI logo maker. No paid plans were found on the live page. Features: Generate a simple logo without a full design tool; Free on the live page; No paid plans were listed when the page was checked; Made by Marc Lou Website: https://logofa.st Last verified: 2026-11-06 LogoFast is Marc Lou's free AI logo maker. You generate a simple logo without opening a design tool. The live page listed no paid plans when it was checked on November 6, 2026. ## MagicSlides URL: https://www.aiagentsinfo.com/directory/magicslides Category: Productivity Makes presentation slides from a prompt, for Google Slides, PowerPoint, and export formats listed on the pricing page. A one-time designer service is separate from the subscriptions. Pricing: Essential is shown at $8 and $6.67/month, with $80 billed annually. Pro is shown at $16 and $12.42/month, with $149 billed annually. Premium is shown at $29 and $23.25/month, with $279 billed annually. A one-time designer service is $99, shown next to $500. Features: Slide generation with an export to PDF, PowerPoint, and Google Slides; Add-ons for Google Slides, Figma, Chrome, and a ChatGPT connection; An MCP connection listed on the plan cards; A one-time designer service besides the subscriptions Website: https://www.magicslides.app Last verified: 2026-11-04 MagicSlides is a slide maker. The MCP line on the pricing page is a connection to other agents, not a separate agent framework. TrustMRR lists current MRR of $9,954 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $9,225, and all-time revenue is $729,638. ## Meteor Jobs: AI Job Search URL: https://www.aiagentsinfo.com/directory/meteor-jobs-ai-job-search Category: HR / recruiting AI job search for iPhone: discover matches, tailor resumes, and queue applications. Pricing: Freemium Features: Type: App Website: https://usemeteor.ai/ Last verified: 2026-11-07 _No body text._ ## Microsoft Agent Framework URL: https://www.aiagentsinfo.com/directory/microsoft-agent-framework Category: Agent frameworks & harnesses Microsoft's open-source framework for building agents and multi-agent workflows in Python and .NET, with a Go SDK in public preview. It includes a ready-made harness for long tasks and graph workflows when you want fixed steps. Pricing: The GitHub repository is MIT licensed, and Microsoft does not list a fee for the framework. You pay whatever model or cloud provider you connect. Microsoft's overview says you are responsible for those third-party costs. Features: Agents that call tools and MCP servers; A Harness Agent with planning, file access, memory, and tool approval, as Microsoft describes it; Graph and function workflows for explicit step-by-step control; Microsoft describes it as the successor to Semantic Kernel and AutoGen Website: https://learn.microsoft.com/en-us/agent-framework/overview/ Last verified: 2026-11-03 This is a framework and a [harness](/glossary/harness), not a finished chatbot. Microsoft says to prefer an ordinary function when the task does not need an agent. A sourced comparison is [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## Midjourney URL: https://www.aiagentsinfo.com/directory/midjourney Category: Image & video Image generation service operated by Midjourney, accessed through the vendor's app, with subscription tiers documented by Midjourney. Pricing: Monthly prices are Basic $10, Standard $30, Pro $60, and Mega $120. Annual prices, paid up front, are Basic $96 ($8 a month), Standard $288 ($24 a month), Pro $576 ($48 a month), and Mega $1,152 ($96 a month). Extra GPU time is $4 per hour. A company with more than $1,000,000 in gross revenue a year must use Pro or Mega. Features: Image generation, plus video generation with limits that differ by plan; Plan limits and commercial terms are defined by Midjourney; Interface and model versions change; use the vendor changelog Website: https://www.midjourney.com Last verified: 2026-11-03 For image and video tools, read the vendor's current terms on training data and commercial use. ## MusiCraft AI URL: https://www.aiagentsinfo.com/directory/musicraft-ai Category: Audio / voice AI music agent to create, split, master and transcribe music Pricing: Freemium Features: Type: AI tool; Maker: Sphoten Ltd Website: https://musicraft.ai/features/stem-splitter Last verified: 2026-11-06 _No body text._ ## NanoPic URL: https://www.aiagentsinfo.com/directory/nanopic Category: Image AI creative platform: generate images from text and edit reference images. Pricing: Freemium Features: Type: AI tool Website: https://nanopic.app/ Last verified: 2026-11-08 _No body text._ ## NeverApply URL: https://www.aiagentsinfo.com/directory/neverapply Category: HR / recruiting AI job-search agent that finds, matches and applies to jobs with your approval Pricing: Freemium Features: Type: SaaS; Maker: Mohammed Sohail (@neverapplyco), Startup Website: https://neverapply.co/ Last verified: 2026-11-07 _No body text._ ## NextName URL: https://www.aiagentsinfo.com/directory/nextname Category: Developer tools Look up domains, WHOIS, DNS, SSL, and TLD prices in one place. Pricing: Free Features: Type: SaaS; Maker: Marvin, Indie hacker Website: https://nextname.app/?utm_source=dir&utm_medium=referral&utm_campaign=aiagentsinfo Last verified: 2026-11-05 _No body text._ ## NotFair URL: https://www.aiagentsinfo.com/directory/notfair Category: Marketing An ads agent for Google, Meta, and other ad accounts. You connect an assistant you already use. It can draft campaign changes and, after you approve them, apply them. Pricing: Free is $0, with 7 days of unlimited access and then 300 operations a month. The Growth card shows about $79/mo billed annually ($950/year). The page title also says the free plan and Growth are $99/mo or $950/yr. Extra ad-account spots are $10/mo. Features: Connects to Google, Meta, LinkedIn, Reddit, TikTok, and X ad accounts; Runs inside Claude, ChatGPT, Codex, Cursor, and other assistants listed on the pricing page; Free plan, then a Growth plan for unlimited operations; Extra ad-account spots at $10/mo each on Growth Website: https://notfair.co Last verified: 2026-11-04 NotFair is an ads agent that sits inside an assistant you already run. Claude, ChatGPT, Codex, and Cursor are the harnesses named on the pricing page. The agent drafts a change, you preview it, and then it can apply it. TrustMRR lists current MRR of $13,085 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $16,370, and all-time revenue is $67,145. ## O.Translator URL: https://www.aiagentsinfo.com/directory/o-translator Category: Writing AI document translation with layout preservation Pricing: Paid · Usage-based Features: Type: AI tool Website: https://otranslator.com/ Last verified: 2026-11-09 _No body text._ ## OpenAI Agents SDK URL: https://www.aiagentsinfo.com/directory/openai-agents-sdk Category: Agent frameworks & harnesses An open-source toolkit for building agents in Python and TypeScript. It runs the loop that calls a model, uses tools, hands work to another agent, and stops when the task is done. Pricing: The SDK itself has no license fee. The Python and TypeScript packages are MIT licensed. Model calls go through your OpenAI API account, so you pay OpenAI's API rates for the models you use. OpenAI does not list a separate subscription for the SDK. Features: Agents, tool calls, handoffs, guardrails, sessions, and built-in tracing; Python package openai-agents and TypeScript package @openai/agents, both MIT licensed; Sandbox agents that work inside an isolated workspace; Uses OpenAI's Responses API by default when you call OpenAI models Website: https://developers.openai.com/api/docs/guides/agents/sdk Last verified: 2026-11-03 This SDK is a [harness](/glossary/harness) you embed in your own app. It is not the ChatGPT product. A sourced comparison with other runtimes is [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## OpenLinker URL: https://www.aiagentsinfo.com/directory/openlinker Category: AI agents Connect, call and coordinate AI agents, hosted or self-hosted. Pricing: Open source Features: Type: SaaS; Maker: Zhi Kin (@openlinkerai), Startup; Launch year: 2026 Website: https://openlinker.ai/?utm_source=aiagentsinfo&utm_medium=directory&utm_campaign=launch Last verified: 2026-11-05 _No body text._ ## Outrank URL: https://www.aiagentsinfo.com/directory/outrank Category: Productivity AI SEO platform that writes and publishes articles on a schedule, with optional backlinks and human review add-ons. Indie product from Tibo’s portfolio. Pricing: On outrank.so/pricing (checked twice 2026-11-05): Organic Growth is $99/month or $999/year (30 articles/month, integrations included). Volume add-ons are +$85/month (60 articles) or +$160/month (90). Human Curated Service is $1,399/month. Backlink plans: Premium $199/month, Scale $899/month, Enterprise $1,899/month. Free Tools Builder $99/month. Premium Images $49/month. Features: Organic Growth plan with 30 SEO articles generated and published per month; Add-ons to raise article volume to 60 or 90 per month on one site; Backlink plans and a human-curated editing service; Free-tools builder and premium image add-ons Website: https://www.outrank.so Last verified: 2026-11-05 Outrank automates SEO articles and publishing so a site can keep shipping pages without a full content team. Add-ons cover more articles, backlinks, and human editors. A TrustMRR profile exists at trustmrr.com/startup/outrank, but the primary MRR figure was not clearly Stripe-verified as a usable number on 2026-11-05 (page showed inconsistent/zero MRR in the main readout). Revenue is omitted until that figure is clear. ## PageAI URL: https://www.aiagentsinfo.com/directory/page-ai Category: Developer tools AI website builder that turns a short description of a business into a hosted site on your own domain. You can download the code. Pricing: On pageai.pro/pricing (checked twice 2026-11-07): Pro is $20/month with 500 credits and one published site. Studio is $60/month with 1,600 credits and five published sites. Yearly billing is labeled 20% off. Add-on credit packs are 200 credits for $10, 550 for $25, and 1,200 for $50, and they require a plan. Features: Describe the business in a few sentences and pick one of three looks; Hosting is included. A domain connects with two DNS records; Pro is $20/month for one site. Studio is $60/month for five sites; Edit the text in place, roll back a version, or download the code Website: https://pageai.pro Last verified: 2026-11-07 PageAI builds a simple business website from a short description. You pick a look, edit the words, and publish on your own domain. Hosting is included. The code can be downloaded. Pro covers one published site. Studio covers five. Credits pay for building and later changes, and unused credits carry over. ## Perplexity URL: https://www.aiagentsinfo.com/directory/perplexity Category: Research Answer engine that retrieves web sources and cites them in responses. Useful as a research front end, not a substitute for primary sources. Perplexity Teammates are a research preview of agents for ongoing work, for individual Max subscribers on the web. Pricing: The pricing page lists Free at $0 per month, then $17 per month and $167 per month, both when billed annually. The page's FAQ describes Pro and Max. A separate non-annual monthly price was not shown. Features: Web-grounded answers with cited links; Pro and API offerings as described by the vendor; Collections and file features vary by plan; Perplexity Teammates, a research preview for individual Max subscribers on the web, assign ongoing work to an agent Website: https://www.perplexity.ai Last verified: 2026-11-07 Citations in the product are only as good as the retrieved pages. This site prefers primary sources on the [research desk](/research). Perplexity is compared in [Best AI agent for shopping](/best/ai-agents-for-shopping), [Best AI agent for web search](/best/ai-agents-for-web-search), and [Best AI agent for research](/best/ai-agents-for-research). ## What are Perplexity Teammates? Perplexity’s [help center article](https://www.perplexity.ai/help-center/en/articles/2026102601-perplexity-teammates), read November 7, 2026, says Teammates are AI agents that own ongoing work. You give a teammate a responsibility, such as triaging bug reports, tracking metrics, or researching a market, and it plans the tasks and follows through until the result is ready for review. Each teammate has its own role. It can monitor tools you connect, act when something changes, and reach out when it needs your input. The same article says the initial research preview requires an individual Perplexity Max subscription and is on the web. Enterprise Pro and Enterprise Max are not included by default. The [product page](https://www.perplexity.ai/hub/products/teammate) says Teammates are available for Max subscribers on the web, with iOS and Android still listed as coming, and that a teammate works in the background and notifies you when a decision is needed. This listing has not run a teammate. It is recording those two pages. Other agent products on this site are in the [AI agents category](/directory/category/ai-agents). ## PhotoAI URL: https://www.aiagentsinfo.com/directory/photoai Category: Image & video AI photo and video generator from @levelsio, including likeness-trained self portraits, headshots, and credit-based monthly plans. Pricing: On photoai.com/pricing (checked twice 2026-11-05): monthly Starter $19 (50 credits), Pro $49 (1,000 credits), Premium $99 (3,000 credits), Ultra $199 (10,000 credits). Yearly cards show about $9, $29, $49, and $99 per month respectively. Features: Train likeness and generate photos or videos of yourself; Credit packs tied to Starter, Pro, Premium, and Ultra plans; Yearly billing with several months free versus monthly; Headshot and style presets alongside custom prompts on higher plans Website: https://photoai.com Last verified: 2026-11-05 PhotoAI is levelsio’s consumer image/video generator. You buy credits on a monthly or yearly plan and spend them on likeness photos, headshots, and related styles. No TrustMRR page was found on 2026-11-05, so revenue is omitted. ## Plavtora URL: https://www.aiagentsinfo.com/directory/plavtora Category: Productivity AI startup decision tool that pressure-tests a product and turns the biggest open question into one next step for founders. Pricing: On plavtora.com (checked twice 2026-11-07): Free is $0, with 3 Daily Objectives per project. Premium is $8.99/month, with unlimited Daily Objectives, 20 launch audits a month, and higher chat and analysis limits. Features: Startup audit across product, customer, positioning, validation, and launch readiness; A Daily Objective that names the action, the evidence, and the success signal; Free plan with 3 Daily Objectives per project and tight monthly limits; Premium at $8.99/month with unlimited Daily Objectives and higher monthly limits Website: https://www.plavtora.com Last verified: 2026-11-07 Plavtora is a decision tool for founders. You describe the product and the customer. It looks for weak assumptions, then gives one Daily Objective: what to test, what evidence to bring back, and what would count as success. The free plan covers a few objectives. Premium raises the limits. ## Post Bridge URL: https://www.aiagentsinfo.com/directory/post-bridge Category: Social media Jack Friks’s scheduler posts one update to many social accounts, with an API and MCP so an agent can publish for you. Pricing: On post-bridge.com/pricing (checked twice 2026-11-09): Marketer is $39/month for 15 accounts, billed monthly, with MCP and the API included. Operator is $99/month for 99 accounts, then $1 for each account after that. Both show a 7-day trial and $0.00 due today. The yearly toggle advertises 2 months free. Features: Cross-post to X, Instagram, LinkedIn, Facebook, TikTok, YouTube, Bluesky, Threads, Pinterest, and Google Business; Marketer plan includes 15 accounts, MCP, and the API; Operator plan starts at 99 accounts, then $1 per extra account; 7-day trial with $0 due today Website: https://www.post-bridge.com Last verified: 2026-11-09 Post Bridge is Jack Friks’s social scheduler for one person running a business. You write a post once and send it to the networks you already use. The agent path is on the pricing page: Marketer includes MCP and the API, so another assistant can schedule posts instead of you clicking through each app. TrustMRR verifies revenue with a Stripe API key. Checked November 9, 2026: current MRR $49,677, last 30 days $45,756, all-time revenue $574,567, with 1,638 active subscriptions. ## Postiz URL: https://www.aiagentsinfo.com/directory/postiz Category: Productivity Postiz is the ultimate agentic social media scheduler tool Pricing: Paid · starting at $29/month Features: Type: AI tool; Maker: Nevo David, Indie hacker Website: https://postiz.com/ Last verified: 2026-11-03 _No body text._ ## PostSyncer URL: https://www.aiagentsinfo.com/directory/postsyncer Category: Social media Social scheduler for 11+ platforms with an AI studio for captions, images, and video, plus MCP so other agents can call it. Pricing: On postsyncer.com/pricing (checked twice 2026-11-05): Starter is $29/month or $24/month billed yearly. Pro is $49/month or $39/month billed yearly. Pro Plus is $99/month with a yearly option shown on the same page. Trials start at $0.00. Features: One calendar across 11+ social networks; AI studio for captions, images, and video; MCP endpoint documented at docs.postsyncer.com/mcp; 7-day free trial on listed plans Website: https://postsyncer.com Last verified: 2026-11-05 PostSyncer is Tibo’s multi-platform scheduler. The MCP docs are the agent angle: another assistant can schedule through PostSyncer instead of you clicking every network by hand. ## Prizm URL: https://www.aiagentsinfo.com/directory/prizm Category: HR / recruiting Your CV and cover letter tailored to every job offer in a minute Pricing: Freemium Features: Type: SaaS; Maker: Melvine Lawson; Launch year: 2026 Website: https://heyprizm.com/en?utm_source=aiagentsinfo&utm_medium=directory&utm_campaign=nov2k Last verified: 2026-11-09 _No body text._ ## Promptwise URL: https://www.aiagentsinfo.com/directory/promptwise Category: Image & video One subscription for 40+ image, video, and audio AI models in a single web app. Indie generative suite with Stripe-verified revenue on TrustMRR. Pricing: On promptwise.com/pricing (checked twice 2026-11-05): Starter $19/month ($159/year), Plus $39 ($327/year), Pro $79 ($663/year), Max $149 ($1,251/year). Max 2 is $249/month and Max 3 is $349/month. Prices exclude VAT. Page notes list prices before a shown discount. Features: 40+ models for image, video, and audio on every plan; Credit tiers from Starter through Max (and higher Max options); Monthly or yearly billing with VAT called out separately; Cancel anytime; every model included on every plan Website: https://promptwise.com Last verified: 2026-11-05 Promptwise puts many generative models behind one login and one credit pool. Useful when you do not want a separate subscription per model vendor. TrustMRR lists current MRR of $66,778 from a connected Stripe API key, checked November 5, 2026. Last 30 days on that page are $95,319, all-time revenue is $297,353, with 1,982 active subscriptions. ## Pydantic AI URL: https://www.aiagentsinfo.com/directory/pydantic-ai Category: Agent frameworks & harnesses A Python framework for typed agents. You name a model, add tools, and get answers checked against a schema. A separate harness package adds files, a shell, and planning for longer jobs. Pricing: The Pydantic AI library has no fee of its own and is MIT licensed. Pydantic's pricing page is for Logfire, the observability product, not the framework. Logfire Personal is free with 10 million records a month. Team is $49 per month. Growth is $249 per month. Extra records on Team and Growth are $2 per million. Enterprise is custom. The page says those prices apply to Logfire. Features: A typed agent loop that can swap models by changing a string; Pydantic AI Harness for memory, guardrails, sub-agents, planning, and a coding agent; Separate open-source packages for evals and graph control flow; The GitHub repository is MIT licensed Website: https://pydantic.dev/docs/ai/overview/ Last verified: 2026-11-03 Pydantic AI is a framework. Logfire is the paid product on the same company's pricing page, and it is optional. The harness comparison is [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). The eval package is on [Best AI agent evaluation tools](/best/ai-agent-evaluation-tools). ## Qovia URL: https://www.aiagentsinfo.com/directory/qovia Category: Developer tools Chinese social media data for your AI agents, in one API Pricing: Paid · Usage-based Features: Type: AI tool; Maker: Scott (@QoviaAI), Startup; Launch year: 2025 Website: https://qovia.ai/ Last verified: 2026-11-07 _No body text._ ## Rank Prompt URL: https://www.aiagentsinfo.com/directory/rank-prompt Category: SEO Tracks how a brand shows up in AI answers, then helps with citations, content, and audits. A free report does not ask for a card. Pricing: On the pricing page, annual prices are Starter $39 per month billed annually at $470 (saves $118), Pro $71 per month billed annually at $855 (saves $213), Agency $119 per month billed annually at $1,430 (saves $358), and Agency Plus $239 per month billed annually at $2,870 (saves $718). A free report uses 10 credits and does not ask for a card. Features: AI rank and mention tracking across answer engines; Content, audits, and citation outreach on the paid plans; A free report of 10 credits, with no card; An MCP server so Claude Code, Cursor, and other clients can call the product Website: https://rankprompt.com Last verified: 2026-11-04 Rank Prompt is an AI-search tool. The MCP server is a way for an outside agent harness, such as Claude Code or Cursor, to call Rank Prompt. The product itself is the tracker and the content tools. TrustMRR lists current MRR of $42,348 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $52,928, and all-time revenue is $328,704. ## RelayMe URL: https://www.aiagentsinfo.com/directory/relayme Category: Image & video AI video & image workflow platform for marketing teams Pricing: Freemium Features: Type: SaaS; Launch year: 2024 Website: https://www.relayme.ai/ Last verified: 2026-11-07 _No body text._ ## ReplyCurrent URL: https://www.aiagentsinfo.com/directory/replycurrent Category: Customer support Website assistant that answers visitor questions from your site, docs, and files, and shows the source for each reply. Pricing: On replycurrent.com (checked twice 2026-11-07): Growth $29/month, Pro $49/month, and Agency $99/month. The page says prices are in USD and billed monthly, and that you can change or cancel anytime. Features: Answers from a website, help center, docs, and PDFs, with the source attached; One embed snippet. Connected websites re-sync daily; Growth $29/month, Pro $49/month, and Agency $99/month; Lead capture, insights, and badge removal on Pro and Agency Website: https://www.replycurrent.com Last verified: 2026-11-07 ReplyCurrent turns a site, help center, docs, and PDFs into an assistant that answers visitor questions and cites the passage it used. You name it, set the tone, and paste one snippet into the site. Growth covers one assistant. Pro adds a second assistant and lead capture. Agency adds client workspaces. It is compared in [Best AI agent for small business](/best/ai-agents-for-small-business). ## Revid URL: https://www.aiagentsinfo.com/directory/revid Category: Video Turns text, links, and prompts into short videos with AI credits. Built by indie maker Tibo for creators who need faceless clips at volume. Pricing: On revid.ai/pricing (checked twice 2026-11-05): Growth is $39/month or $32/month billed yearly (2,000 credits). Elite is $89/month or $74/month yearly (5,000 credits). Ultra is $199/month or $166/month yearly (12,000 credits), with higher Ultra volumes at $459, $749, $1,459, and $2,799 per month. The plan cards also show Hobby at $39/month. Credit boosts: 1,000 for $49, 4,000 for $99, 10,000 for $199, 100,000 for $1,900. Features: Prompt-to-video and link-to-video workflows with voiceover-style outputs; Monthly credit plans from Hobby/Growth up through high-volume Ultra tiers; One-time credit boost packs that do not expire; Yearly billing discounts called out on the pricing FAQ Website: https://www.revid.ai Last verified: 2026-11-05 Revid is an AI video product from Thibault Louis-Lucas (Tibo). You spend credits to turn prompts or links into short videos. Check the pricing page for how many credits a clip uses before you generate. No TrustMRR Stripe page was found for Revid on 2026-11-05, so revenue is omitted. ## Rezloom URL: https://www.aiagentsinfo.com/directory/rezloom Category: HR / recruiting Tailor a resume to a job description from your career materials—Chinese and English on the same facts, with controllable share links. Pricing: Freemium Features: Type: AI tool; Maker: Everitere (@Everitere1), Indie hacker Website: https://rezloom.com/ Last verified: 2026-11-08 _No body text._ ## Ringzel URL: https://www.aiagentsinfo.com/directory/ringzel Category: Audio / voice AI receptionist that answers the calls you can't take Pricing: Free trial · Monthly Features: Type: SaaS; Maker: Ringzel team (@ringzelhq), Startup Website: https://ringzel.com/ Last verified: 2026-11-06 _No body text._ ## RiseMore URL: https://www.aiagentsinfo.com/directory/risemore Category: Marketing Content publishing helper that researches a product and drafts social, SEO, GEO, and Reddit posts, with free and paid plans. Pricing: On risemore.app/pricing (checked twice 2026-11-07): Free $0, Pro $20/month, and Growth $60/month. Yearly billing is labeled two months free. Publishing on your behalf is extra: $20 per Reddit post, $10 per Reddit comment, and $5 per GEO or blog article. Social and SEO publishing to your own accounts or site has no extra publish fee. Features: Research and drafts for a product. You review before anything publishes; Free includes social drafts. Pro adds SEO and Competitor Radar; Growth adds GEO and Reddit, published on your behalf for a separate fee; Yearly billing is labeled two months free Website: https://www.risemore.app Last verified: 2026-11-07 RiseMore is a content workspace for founders who do not have a marketing team. It researches the product, drafts posts, and waits for you to approve them. Free covers research and social drafts. Pro adds SEO articles to a site you connect. Growth adds GEO articles and Reddit posts, which carry a separate publish fee when RiseMore posts them for you. ## RouteMux URL: https://www.aiagentsinfo.com/directory/routemux Category: Developer tools Reliability-first AI gateway: one API key for many providers. Pay only for success. Pricing: Paid · starting at $10 · Usage-based Website: https://routemux.com/ Last verified: 2026-11-09 _No body text._ ## Runway URL: https://www.aiagentsinfo.com/directory/runway Category: Image & video A web app for making images, video, and audio with generative models. You spend credits. Each model costs a set number of credits based on the length and the resolution. Pricing: Free is $0 and includes 125 credits once. Paid monthly prices on the pricing page are Standard $15, Pro $35, and Max $95. Billed annually, those are $12, $28, and $76 per month, with 625, 2,250, and 9,500 credits a month. Standard and Pro credits do not roll over. Max can roll over up to one month of unused credits. Purchased extra credits do not expire. Enterprise is custom. Features: Text, image, and video tools, plus audio tools on higher plans; Credits are the unit you spend. Gen-4.5 is listed at 12 credits per second of video; Free includes a one-time deposit of 125 credits; Extra credits can be purchased. The minimum purchase is 1,000 credits Website: https://runway.com Last verified: 2026-11-03 Runway is a creative app, not an agent framework. Read the current plan page before you quote a credit cost, because each model spends credits differently. ## SandBase URL: https://www.aiagentsinfo.com/directory/sandbase Category: Agent frameworks & harnesses One API for AI models & sandboxes Pricing: Freemium Features: Type: Agent framework Website: https://www.sandbase.ai/ Last verified: 2026-11-08 _No body text._ ## ScribeMedical URL: https://www.aiagentsinfo.com/directory/scribemedical Category: Health AI medical scribe for clinical notes Pricing: Freemium Features: Maker: ScribeMedical Team Website: https://scribemedical.ai/ Last verified: 2026-11-08 _No body text._ ## SEOBOT URL: https://www.aiagentsinfo.com/directory/seobot Category: Productivity Autonomous SEO robot with AI agents for keyword research, article generation, internal linking, and related SEO chores aimed at busy founders. Pricing: On seobotai.com (checked twice 2026-11-05; no separate /pricing route): subscriptions start at $49/month. The FAQ offers a full refund on the $49 plan if you contact them after the first article and are not satisfied. Features: AI-generated blog and programmatic SEO flows; Auto keyword research and AI internal linking; AI backlink-related features described on the homepage; Refund on the $49 plan after the first article if you are not satisfied Website: https://seobotai.com Last verified: 2026-11-05 SEOBOT markets itself as an SEO robot: agents handle research, drafting, and internal linking so a founder can keep shipping product. Treat the homepage pricing line as the source of truth until a fuller plan grid appears. TrustMRR lists current MRR of $32,430 from a connected Stripe API key, checked November 5, 2026. Last 30 days on that page are $33,917, all-time revenue is $1,924,427, with 482 active subscriptions. ## ShipFast URL: https://www.aiagentsinfo.com/directory/shipfast Category: Developer tools Next.js boilerplate from Marc Lou with auth, payments, and other launch essentials so makers can ship a SaaS in days. One-time purchase with lifetime updates. Pricing: On shipfa.st (checked twice 2026-11-05): one-time Starter $199, All-in $249, and Bundle (ShipFast + CodeFast) $299. Schema markup also exposed $169 and $199 offer prices (likely a promo). FAQ confirms PayPal at $199, $249, or $299. Not a subscription. Features: Next.js starter with common SaaS building blocks; One-time license tiers including a CodeFast bundle; Lifetime updates and member discounts called out on the page; PayPal purchase path at the same USD amounts Website: https://shipfa.st Last verified: 2026-11-05 ShipFast is Marc Lou’s launch boilerplate, not an agent runtime. Makers buy it once to skip repetitive auth/payment/setup work. TrustMRR verifies revenue with Stripe. Checked November 5, 2026: last 30 days $1,445, all-time revenue $1,273,191. TrustMRR shows no active subscriptions, which matches a one-time product. ## Shortlist Watch URL: https://www.aiagentsinfo.com/directory/shortlist-watch Category: Marketing See if ChatGPT recommends your SaaS, which competitors it names instead, and which sites it cites. Pricing: Freemium · starting at $29/mo · Monthly Features: Type: SaaS; Maker: Highpoint Labs Website: https://shortlistwatch.com/ Last verified: 2026-11-08 _No body text._ ## SiteGPT URL: https://www.aiagentsinfo.com/directory/sitegpt Category: Customer support Train a customer-support chatbot on your site, docs, and files, then embed it and escalate to a human when needed. Pricing: On sitegpt.ai/pricing (checked twice 2026-11-06): Starter $59/month or $468/year (about $39/month); Growth $129/month or $948/year (about $79/month); Scale $429/month or $3,108/year (about $259/month); Enterprise is custom. A 7-day free trial requires a card. An extra 5,000-message add-on is $59/month on monthly billing, or about $39/month on yearly billing. Features: Support chatbot trained on your site, docs, and files; Embed the bot on your site; Escalate a conversation to a human; Starter, Growth, Scale, and custom Enterprise plans, with a 7-day trial Website: https://sitegpt.ai Last verified: 2026-11-06 SiteGPT, from Bhanu Teja Pachipulusu, trains a support agent on your site, docs, and files. You embed the chatbot, and it can escalate to a human when it should stop answering. ## Slashspace URL: https://www.aiagentsinfo.com/directory/slashspace Category: Productivity Desktop canvas for branching AI chats. Bring your own keys or a Claude or ChatGPT subscription, or spend wallet credits on hosted models. Pricing: On slashspace.ai/pricing (checked twice 2026-11-09): Yearly is $89 per year and Forever is $249 once. Both include the full app, 500 wallet credits (1 credit is $0.01 of hosted use), bring-your-own-key, and 2 device seats. A Black Friday code, BF2026, was advertised at 51% off through Dec 1, showing $43.61 instead of $89 and $122.01 instead of $249. Refunds are offered within 7 days. Wallet top-ups are $11 for $10 of usage. There is no free trial. Features: Desktop app for macOS, Windows, and Linux; Branching canvas instead of one long chat thread; Bring your own API keys or a Claude or ChatGPT subscription; Yearly plan or a one-time lifetime license, each with 500 wallet credits Website: https://www.slashspace.ai Last verified: 2026-11-09 Slashspace is Praneeth Pike’s desktop canvas for AI chats. You keep several threads on one board and jump between them, with your own API keys, a Claude or ChatGPT subscription, or Slashspace’s hosted models. The product used to be called Rabbit Holes AI. A lifetime purchase from that name carries over. TrustMRR still lists the revenue under Rabbit Holes AI and verifies it with a Stripe API key. Checked November 9, 2026: current MRR $174, last 30 days $2,601, all-time revenue $131,971, with 19 active subscriptions. ## Smew AI URL: https://www.aiagentsinfo.com/directory/smew-ai Category: Video Viral AI trend videos from your own photo Pricing: Freemium · starting at $9.99 · One-time Features: Type: AI tool; Maker: Jiang King Website: https://smew.ai/ Last verified: 2026-11-05 _No body text._ ## smolagents URL: https://www.aiagentsinfo.com/directory/smolagents Category: Agent frameworks & harnesses A small open-source Python library from Hugging Face for building agents. Its code agent writes Python to call tools and do math, instead of only returning text. Pricing: The library is free under the Apache License 2.0. Hugging Face does not sell a smolagents subscription. You pay the model host you connect, if that host charges. Features: CodeAgent writes actions as code. ToolCallingAgent uses ordinary tool calls; Works with many models, including Hugging Face Inference, other APIs, or a local model; Can run code in a sandbox such as Docker, E2B, Modal, or Blaxel, as the docs describe; The GitHub repository is Apache License 2.0 Website: https://huggingface.co/docs/smolagents/index Last verified: 2026-11-03 smolagents is a small framework. The price, if any, comes from the model provider, not from the library. Local-model notes are in [Best AI agent harness](/best/ai-agent-harnesses). The November 9, 2026 release tag is on [Best AI agent frameworks](/best/ai-agent-frameworks). ## Solo Content Studio URL: https://www.aiagentsinfo.com/directory/solo-content-studio Category: Writing AI writing studio for coaches. It drafts newsletters, posts, scripts, and sales pages in the coach’s own voice on a monthly credit plan. Pricing: On solocontent.studio (checked twice 2026-11-09): a 7-day trial is $0 today, then $59/month, with 100 starting credits and 1 project. Essentials is $59/month for 500 credits and about 30 drafts. Studio is $99/month for 1,000 credits, up to 2 projects, and about 60 drafts. Pro is $199/month for 2,500 credits, unlimited projects, and about 150 drafts. The yearly toggle advertises 20% off. Features: Drafts newsletters, posts, scripts, email sequences, and sales pages; Voice profile so the drafts sound like the coach; Essentials, Studio, and Pro credit plans; 7-day trial, then a monthly plan unless you cancel Website: https://solocontent.studio Last verified: 2026-11-09 Solo Content Studio is Jason Zook’s writing app for coaches who are the face of the brand. You give it a topic and it drafts the formats coaches actually ship: newsletters, posts, video scripts, and sales pages, using a voice profile so the result is not a generic blog voice. TrustMRR verifies revenue with a Stripe API key. Checked November 9, 2026: current MRR $3,643, last 30 days $3,712, all-time revenue $28,544, with 67 active subscriptions. ## Speel URL: https://www.aiagentsinfo.com/directory/speel Category: Image & video A web app for product images and short videos that look like creator posts. You spend credits. Each plan sets a monthly credit allowance. Pricing: Monthly prices are Starter $49/mo (1,200 credits), Growth $109/mo (2,700 credits), and Pro $179/mo (4,400 credits). New users can start a one-time free trial with 90 credits, 1 video, and 5 images. The same page offers yearly billing marked 30% off. Features: Image and video tools, including product shots and talking-head clips; Credits are the unit you spend. Starter includes 1,200 credits a month; New users can take a one-time free trial of 90 credits; The pricing page also offers yearly billing, marked 30% off Website: https://www.speel.co Last verified: 2026-11-04 Speel is a creative app, not an agent framework. Read the pricing page before you quote a credit cost, because the homepage FAQ lists different credit totals than the plan cards. TrustMRR lists current MRR of $65,293 from a connected Stripe API key, checked November 4, 2026. The same page lists $2,826 of revenue in the last 30 days and $183,792 all time. The MRR and the last-30-days figure are both printed there, and they are not the same number. ## Squad URL: https://www.aiagentsinfo.com/directory/squad Category: Productivity Gives you AI teammates on a cloud computer that connect to your tools, run processes on a schedule, and bring back finished work. A practical agent harness for non-developers. Pricing: On squad.so/pricing (checked twice 2026-11-05): one flat plan at $99/month per squad. Optional email inbox is $10/month per AI teammate. Human teammate seats are $9/month. Extra squads are another $99/month each. You bring your own AI plan (the page recommends Claude Max around $100–$200/month). 7-day money-back guarantee. Features: AI teammates with separate browser profiles on a dedicated cloud computer; Flat $99/month per squad with a 7-day money-back guarantee; Optional email inbox per teammate ($10/mo) and human seats ($9/mo); Bring your own Claude / Claude Code plan — Squad does not bill per-task AI usage Website: https://squad.so Last verified: 2026-11-05 Squad is an agent harness: each AI teammate gets its own browser profile and workspace, then runs your process on a schedule. You pay Squad $99/month and separately pay for the model plan you connect (often Claude Max). That split matters when you compare total cost. ## Stalkr URL: https://www.aiagentsinfo.com/directory/stalkr Category: Marketing Social listening from Marc Lou. It monitors keywords on X, Reddit, YouTube, and LinkedIn so you can jump into the right conversations. Pricing: On stalkr.ai (checked twice 2026-11-07): Starter $29/month or $261/year, Pro $99/month or $891/year, and Scale $299/month or $2,691/year. New accounts get 100 free mentions. No credit card is required to start. Features: Keyword monitoring across X, Reddit, YouTube, and LinkedIn; AI categories, email alerts, saved views, and an analytics dashboard; REST API, MCP, and webhooks on every plan, including the free mentions; Starter $29/month, Pro $99/month, and Scale $299/month, with yearly billing about 25% off Website: https://www.stalkr.ai Last verified: 2026-11-07 Stalkr is Marc Lou's social listening tool for founders. You add product names, competitors, or niche phrases. It pulls public mentions from X, Reddit, YouTube, and LinkedIn, then sorts them into cases such as bugs, complaints, and testimonials. It does not reply for you. A new account starts with 100 free mentions. ## Stamp Identifier URL: https://www.aiagentsinfo.com/directory/stamp-identifier Category: Research Identify old and foreign postage stamps from a photo and get estimated value ranges. Pricing: Freemium Features: Type: App Website: https://stampscanner.app/ Last verified: 2026-11-08 _No body text._ ## StoryShort URL: https://www.aiagentsinfo.com/directory/storyshort Category: Video Turns a prompt into short videos, with voiceover, captions, and stock-style visuals. Higher plans add long-form and UGC-style clips. You spend credits. Pricing: Monthly prices are Starter $39, Growth $69, Influencer $129, and Ultra $199. Yearly on the same page: Starter $23/mo ($279 billed yearly), Growth $42/mo ($499), Influencer $75/mo ($899), and Ultra $117/mo ($1,399). The page says yearly billing saves 40%. Features: Short videos from a text prompt, with voiceover and background music; Auto-post and download on the listed plans; UGC-style and long-form video on higher plans; Credits per video, with the estimate shown before you generate Website: https://storyshort.ai Last verified: 2026-11-04 StoryShort is a video generator. Check the pricing page for how many credits a format uses, because a 30-second clip and a talking-avatar clip are not the same cost. TrustMRR lists current MRR of $21,491 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $20,508, and all-time revenue is $601,218. ## Supergrow URL: https://www.aiagentsinfo.com/directory/supergrow Category: Marketing LinkedIn content workspace that helps founders and teams draft, schedule, and publish posts with AI assistance. Pricing: On the live Supergrow pricing page (checked twice 2026-11-07): Pro $39/month and Teams $199/month. Yearly billing is $31/month for Pro and $189/month for Teams. Enterprise is custom. A 7-day free trial is offered. Features: Drafts, a calendar, scheduling, and profile analytics for LinkedIn; Voice interviews and repurposing from YouTube, articles, and PDFs; Pro is $39/month for one account. Teams is $199/month for five accounts; 7-day free trial. Yearly billing is listed at $31/month for Pro and $189/month for Teams Website: https://www.supergrow.ai Last verified: 2026-11-07 Supergrow is a LinkedIn content workspace for founders and small teams. It covers drafting, scheduling, a content calendar, and analytics. Deven is named as the founder on TrustMRR. TrustMRR's Stripe key for Supergrow expired on February 28, 2026, so this listing does not quote revenue. ## Superwhisper URL: https://www.aiagentsinfo.com/directory/superwhisper Category: Audio / voice Voice-to-text app for Mac, Windows, iPhone, and Android. Local dictation is free. Pro adds cloud models, AI modes, and file transcription. Pricing: On superwhisper.com/docs/billing/plans (checked twice 2026-11-09): Free is unlimited local dictation with up to two modes and no AI processing. Pro is $8.49/month, $84.99/year, or $249.99 once. New accounts get 3,000 words of Pro features, then fall back to Free. Students, educators, and non-profits can request 40% off purchases made on superwhisper.com. Features: Dictation into the app you are already using; Free local Whisper models, with a cap of two modes; Pro adds cloud models, AI modes, meetings, and file transcription; One Pro license covers Mac, Windows, iPhone, and Android Website: https://superwhisper.com Last verified: 2026-11-09 Superwhisper turns speech into text on Mac, Windows, iPhone, and Android. The free app stays on the device with local Whisper models. Pro is what adds the AI layer: cloud voice models, modes that rewrite what you said, meeting notes, and transcription of files you already have. There is no separate license per computer. One Pro key covers the devices listed on the plans page. ## SuperX URL: https://www.aiagentsinfo.com/directory/superx Category: Social media Schedules posts on X, suggests what is already working in a niche, and writes drafts in your voice. Paid plans include an API, a CLI, and MCP. Pricing: Pro is $49/month. Advanced is shown at a launch price of $49/month, with $99/month labeled as the normal price. Ultra is $199/month. Features: Scheduling, analytics, and a Chrome extension for X; AI credits for post and thread drafts on the higher plan; Signal agents that watch for leads; Pro includes API, CLI, and MCP access Website: https://superx.so Last verified: 2026-11-04 SuperX is a posting tool for X. MCP on the Pro plan is how another agent can call it. The prices above are the plan cards on the homepage, including the launch price on Advanced. TrustMRR lists current MRR of $19,947 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $18,124, and all-time revenue is $264,053. ## Taja URL: https://www.aiagentsinfo.com/directory/taja Category: Video Turns a long video into shorts, a blog post, and social posts, and can schedule them. The pricing page offers a 7-day free trial. Pricing: Monthly prices are Starter $19.99, Professional $49.99, and Teams/Agency $99.99. Billed annually: Starter $15.99/month, Professional $39.99/month, and Teams/Agency $79.99/month. The page says annual billing saves 20% and offers a 7-day free trial. Features: YouTube long-form optimization, with a video cap that depends on the plan; Shorts from a long video, without a watermark on the paid plans; Blog, LinkedIn, and Threads posts from the same video; A 7-day free trial Website: https://www.taja.ai Last verified: 2026-11-04 Taja is a video repurposing app. The trial and the video caps are on the pricing page, and they differ by plan. TrustMRR lists current MRR of $13,742 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $11,687, and all-time revenue is $1,034,896. That page does not name a founder. ## Taplio URL: https://www.aiagentsinfo.com/directory/taplio Category: Marketing AI LinkedIn growth workspace for ideas, scheduling, engagement, and analytics so founders can post in minutes a day. Pricing: On taplio.com/pricing (checked twice 2026-11-06): monthly Starter $39, Growth $69, and Pro $199. Yearly billing is about 25% off. A 7-day free trial is offered. Features: Post ideas, scheduling, engagement, and analytics in one LinkedIn workspace; Agent-style help for the daily posting routine; Monthly plans from Starter through Pro, with yearly billing about 25% off; 7-day free trial Website: https://taplio.com Last verified: 2026-11-06 Taplio is a LinkedIn growth workspace from Lempire. It covers ideas, scheduling, engagement, and analytics so a founder can post without living in the feed. The agent side is that drafting and engagement help, which runs as part of the posting routine rather than as a general-purpose assistant. ## TecAdRise Bot URL: https://www.aiagentsinfo.com/directory/tecadrise-bot Category: AI agents Free open-source Grok Bot alternative for Windows, with local multi-agent chat. Pricing: Open source Features: Type: App; Maker: Chris / TecAdRise, Startup; Launch year: 2026 Website: https://tecadrise.ai/products/tecadrisebot Last verified: 2026-11-06 _No body text._ ## Tideris URL: https://www.aiagentsinfo.com/directory/tideris Category: Other An AI toolkit for Chinese astrology and a free BaZi chart Pricing: Freemium Features: Type: AI tool; Maker: Tideris Website: https://www.tideris.com/ Last verified: 2026-11-05 _No body text._ ## Tiny Desk AI Video Generator URL: https://www.aiagentsinfo.com/directory/tiny-desk-ai-video-generator Category: Video Upload 1–4 photos to create a 10-second 9:16 Tiny Desk–style AI band video with sound. Pricing: Paid · starting at $7.99 · One-time Features: Type: AI tool; Maker: Shenmei Tu, Indie hacker Website: https://tinydeskvideo.com/ Last verified: 2026-11-08 _No body text._ ## Toolvori URL: https://www.aiagentsinfo.com/directory/toolvori Category: Productivity Curated directory of practical tools for creators, developers and indie teams. Pricing: Free Features: Type: AI tool; Maker: 饭团, Indie hacker Website: https://www.toolvori.com/?utm_source=aiagentsinfo-com&utm_medium=referral&utm_campaign=directory_listing&utm_content=toolvori_directory Last verified: 2026-11-06 _No body text._ ## traxy URL: https://www.aiagentsinfo.com/directory/traxy Category: Sales Finds people already talking about a market, enriches a work email, and can draft outbound from that research. Higher plans add an outbound agent. Pricing: Monthly prices are Pro $149, Advanced $399, and Scale $1,099. The yearly column on the pricing page is Pro $119, Advanced $319, and Scale $879 per month, and the page says yearly billing saves 20%. Agencies are custom. A 7-day free trial is offered. Features: A discovery agent that surfaces qualified leads; Verified work email on leads, and alerts when a tracked profile shows a buying signal; Advanced and Scale add an outbound agent that writes sequences from research on each lead; A 7-day free trial. Agencies are custom Website: https://traxy.ai Last verified: 2026-11-04 traxy is a sales product. The discovery step is an agent that looks for leads. The outbound agent, which writes sequences, starts on the Advanced plan. TrustMRR lists current MRR of $51,974 from a connected Stripe API key, checked November 4, 2026. Last 30 days on that page are $54,345, and all-time revenue is $159,020. ## TriRank URL: https://www.aiagentsinfo.com/directory/trirank Category: SEO TriRank connects Google Search Console and checks whether AI answer engines cite your site. Pricing: Freemium Features: Type: SaaS Website: https://trirankai.com/ Last verified: 2026-11-09 _No body text._ ## Tweet Hunter URL: https://www.aiagentsinfo.com/directory/tweet-hunter Category: Marketing X growth suite with a viral library, an AI writer, scheduling, analytics, CRM lists, and auto-DMs. Pricing: On tweethunter.io/pricing (checked twice 2026-11-06): Discover $29/month (1 X account), Grow $49/month (5 X accounts), Enterprise $199/month (unlimited X accounts). A 7-day free trial is offered. Yearly discounts are advertised (about 30% or 53% off, depending on the toggle). Features: Viral post library and an AI writer for X; Scheduling, analytics, CRM lists, and auto-DMs; Plans for 1, 5, or unlimited X accounts; 7-day free trial, with yearly discounts on the pricing toggle Website: https://tweethunter.io Last verified: 2026-11-06 Tweet Hunter is an X growth suite from Lempire. It includes a viral library, an AI writer, scheduling, analytics, CRM lists, and auto-DMs. The writer and the auto-DMs are the agent pieces: they act on an account after you set them up. ## TypingGIF URL: https://www.aiagentsinfo.com/directory/typinggif Category: Image AI GIF generator for text and photo to GIF Pricing: Freemium · starting at $5 HD pack · One-time Features: Type: AI tool; Maker: Ethan Moore Website: https://typinggif.com/ Last verified: 2026-11-09 _No body text._ ## TypingMind URL: https://www.aiagentsinfo.com/directory/typingmind Category: Productivity Multi-model AI chat client with agents, plugins, and local or private storage. Pay once and bring your own API keys. Pricing: On typingmind.com/pricing (checked twice 2026-11-06): a lifetime license starts at $39 once (50% off was advertised at check), plus bring-your-own API keys (you pay the providers per use). There is no monthly TypingMind subscription for the personal license. Team and self-host options exist. Features: Chat across multiple models in one client; Agents and plugins; Local or private storage; One-time personal license with bring-your-own API keys. Team and self-host options exist Website: https://www.typingmind.com Last verified: 2026-11-06 TypingMind is a multi-model chat client with agents, plugins, and local or private storage. You pay once for the app and bring your own API keys, so the model providers bill usage separately. Team and self-host options are listed on the pricing page. ## UMEVO URL: https://www.aiagentsinfo.com/directory/umevo Category: Productivity AI note-taker for professional meeting notes and real-time transcription in 140+ languages. Pricing: Paid · starting at $79 · One-time Features: Type: AI tool; Maker: Andy Wong, Startup; Launch year: 2025 Website: https://www.umevo.ai/ Last verified: 2026-11-06 _No body text._ ## Underpriced AI URL: https://www.aiagentsinfo.com/directory/underpriced-ai Category: E-commerce Photo tool for resellers. It identifies an item, checks recent sold prices, and drafts the marketplace listing. Pricing: On underpricedai.com/pricing (checked twice 2026-11-09): Starter is $5/month or $48/year (30 scans). Pro is $15/month or $144/year (100 scans). Pro Plus is $29/month or $278/year (240 scans). Business is $59/month or $565/year (550 scans). Scale is $199/month for 2,500 scans, then $0.12 per extra scan. Scan packs run $3 for 5, $7 for 15, $12 for 30, $18 for 50, $32 for 100, and $70 for 250. The page offers a free first scan. Features: Identify an item from a photo and price it from recent sales; Draft a listing and publish to eBay or Bonanza after you check it; Monthly scan plans and one-time scan packs; A free first scan, then paid plans from $5/month Website: https://underpricedai.com Last verified: 2026-11-09 Underpriced AI is for people who sell secondhand goods. You photograph an item. The app names it, shows what similar listings recently sold for, and drafts the marketplace listing. You still check the draft and press post. eBay and Bonanza publishing is in the product. Other marketplaces can be filled in from the browser extension, and you post those yourself. ## Vid.AI URL: https://www.aiagentsinfo.com/directory/vid-ai Category: Video Turn any idea or script into ready-to-post videos that get views. Pricing: Paid · starting at $59 /mo · Monthly Features: Maker: Priyam Raj, Indie hacker Website: https://vid.ai/ Last verified: 2026-11-03 _No body text._ ## Videm Ai URL: https://www.aiagentsinfo.com/directory/videm-ai Category: Image Free AI image generator with no sign up or platform watermark. Pricing: Freemium Features: Type: AI tool Website: https://videm.ai/ Last verified: 2026-11-08 _No body text._ ## Visualizee URL: https://www.aiagentsinfo.com/directory/visualizee Category: Design AI rendering and visualization for designers and architects. Turn ideas into visuals without a full 3D pipeline. Pricing: On visualizee.ai/pricing (checked twice 2026-11-06): Hobby $20/month ($15/month yearly), Pro $40/month ($30/month yearly), Max $80/month ($60/month yearly). A free trial is available. Yearly is billed as $180, $360, or $720. Features: AI rendering for designers and architects; Turns an idea into a visual without a full 3D pipeline; Hobby, Pro, and Max plans, monthly or yearly; Free trial available Website: https://visualizee.ai Last verified: 2026-11-06 Visualizee is Piotr Obidowski's AI rendering tool for designers and architects. It turns an idea into a visual without a full 3D pipeline. TrustMRR (https://trustmrr.com/startup/visualizee-ai) lists current MRR of $8,094 from a connected Stripe API key, checked November 6, 2026. Last 30 days on that page are $8,347, and all-time revenue is $95,876. ## Whisper AI Transcribe URL: https://www.aiagentsinfo.com/directory/whisper-ai-transcribe Category: Audio / voice Transcripts for meetings, lectures and interview recordings. Pricing: Freemium Website: https://whisperaitranscribe.com/ Last verified: 2026-11-09 _No body text._ ## WhisperTranscribe.ai URL: https://www.aiagentsinfo.com/directory/whispertranscribe-ai Category: Audio / voice Turn video and podcast audio into captions-ready transcripts. Pricing: Freemium Website: https://whispertranscribe.ai/ Last verified: 2026-11-09 _No body text._ ## WhisperTranscript.ai URL: https://www.aiagentsinfo.com/directory/whispertranscript-ai Category: Audio / voice Readable AI transcripts for lectures, interviews, and clips. Pricing: Freemium Website: https://whispertranscript.ai/ Last verified: 2026-11-09 _No body text._ ## Window Peeper AI Video URL: https://www.aiagentsinfo.com/directory/window-peeper-ai-video Category: Video Recreate the viral Window Peeper shot from one photo — 14s 9:16 AI video with original sound. Pricing: Paid · starting at 6.99 · One-time Features: Type: AI tool; Maker: Shenmei Tu, Indie hacker Website: https://windowpeeperaivideo.com/ Last verified: 2026-11-09 _No body text._ ## XCrawl URL: https://www.aiagentsinfo.com/directory/xcrawl Category: Data / analytics Turn any web page into LLM-ready JSON or Markdown. Pricing: Freemium Website: https://www.xcrawl.ai/ Last verified: 2026-11-09 _No body text._ ## ZenVoice URL: https://www.aiagentsinfo.com/directory/zenvoice Category: Productivity Self-serve Stripe invoices from Marc Lou. Customers generate, edit, and download their own invoices from a link. Pricing: On zenvoice.io (checked twice 2026-11-07): one-time plans at $49 (one Stripe account) and $69 (unlimited Stripe accounts). Struck-through prices on the page were $99 and $119. Features: Customers generate, edit, and download invoices from a link; VAT number, company details, and other fields can be edited on the invoice; Bulk download as a ZIP or CSV for a date range; One-time plans at $49 for one Stripe account and $69 for unlimited accounts Website: https://zenvoice.io Last verified: 2026-11-07 ZenVoice is Marc Lou's tool for Stripe invoices. You connect one or more Stripe accounts and share a link. A customer can generate, edit, and download an invoice for their own successful payment, including VAT and company details. You can also download a date range in bulk. You pay once. TrustMRR verifies revenue with a Stripe API key. Checked November 7, 2026: last 30 days $69, all-time revenue $19,688. TrustMRR shows no active subscriptions, which matches a one-time product. ## Zombie Hug URL: https://www.aiagentsinfo.com/directory/zombie-hug Category: Video AI zombie hug photo to video clips Pricing: Paid · starting at $6.90/video · One-time Features: Type: AI tool; Maker: Eason Website: https://aizombietrend.com/ Last verified: 2026-11-09 _No body text._ # Verified revenue URL: https://www.aiagentsinfo.com/verified-revenue Checked: 2026-11-05 - Postiz: MRR $368,905; all-time $1,738,009; 10,840 active subscriptions. Price: Standard $29/mo, Team $39/mo, Pro $49/mo, Ultimate $99/mo. Annual billing is $23, $31, $39, and $79 per month. Every plan starts with a 7-day free trial. Revenue source: https://trustmrr.com/startup/postiz (checked 2026-11-05). - Rezi: MRR $241,149; all-time $10,491,341; 9,997 active subscriptions. Price: Free $0, Pro $29/month, Lifetime $149 one-time, Enterprise $99/month per 200 users. Revenue source: https://trustmrr.com/startup/rezi (checked 2026-11-05). - AEO Engine: MRR $187,116; all-time $2,681,931; 63 active subscriptions. Price: Growth $1,597/month, Scale $2,997/month, Enterprise from $10,000/month. Revenue source: https://trustmrr.com/startup/aeo-engine (checked 2026-11-05). - Promptwise: MRR $66,778; all-time $297,353; 1,982 active subscriptions. Price: Monthly prices on the pricing page are shown as 20% off list. Starter $19/month or $159/year (list $24), Plus $39/month or $327/year (list $49), Pro $79/month or $663/year (list $99), Max $149/month or $1,251/year (list $186), Max 2 $249/month or $2,091/year (list $311), Max 3 $349/month or $2,931/year (list $436). A 7-day trial includes 50 credits. Prices are in US dollars and exclude VAT. Revenue source: https://trustmrr.com/startup/promptwise (checked 2026-11-05). - Speel: MRR $65,244; all-time $183,792; 1,094 active subscriptions. Price: On the pricing page, monthly prices are Starter $49/mo (1,200 credits), Growth $109/mo (2,700 credits), and Pro $179/mo (4,400 credits). New users can start a one-time free trial with 90 credits. The same page offers yearly billing marked 30% off. Revenue source: https://trustmrr.com/startup/speel-co (checked 2026-11-05). - traxy: MRR $51,129; all-time $161,509; 241 active subscriptions. Price: Monthly prices are Pro $149, Advanced $399, and Scale $1,099. Yearly on the same page: Pro $119, Advanced $319, and Scale $879 per month. The page says yearly billing saves 20%. Agencies are custom. A 7-day free trial is offered. Revenue source: https://trustmrr.com/startup/traxy-ai (checked 2026-11-05). - Vid.AI: MRR $50,442; all-time $1,660,821; 752 active subscriptions. Price: The pricing FAQ says Starter is $59 a month or $549 a year, Plus $99 a month or $947 a year, and Pro $199 a month or $1,919 a year, in USD. Yearly billing saves up to 22%. Eligible new subscribers get 50% off the first month on monthly plans. Revenue source: https://trustmrr.com/startup/vidai-llc (checked 2026-11-05). - Rank Prompt: MRR $44,848; all-time $331,597; 219 active subscriptions. Price: Annual prices on the pricing page: Starter $39 per month billed annually at $470, Pro $71 per month billed annually at $855, Agency $119 per month billed annually at $1,430, and Agency Plus $239 per month billed annually at $2,870. A free report uses 10 credits. Revenue source: https://trustmrr.com/startup/rank-prompt (checked 2026-11-05). - Apiframe: MRR $35,373; all-time $524,325; 916 active subscriptions. Price: Basic is $19 for the first month and then $39/mo (4,000 credits). Pro is $99/mo (10,000 credits), Growth $399/mo (40,000), and Scale $749/mo (75,000). A trial includes 100 credits after a phone check or a $1 check. Revenue source: https://trustmrr.com/startup/apiframe (checked 2026-11-05). - SEOBOT: MRR $32,578; all-time $1,924,181; 484 active subscriptions. Price: The homepage says subscriptions start at $49/mo. A fuller plan table was not on that page. Revenue source: https://trustmrr.com/startup/seobot (checked 2026-11-05). - Based Labs: MRR $28,537; all-time $1,254,077; 873 active subscriptions. Price: The homepage says 300 credits for $10, paid in crypto (USDC, USDT, or ETH) or Telegram Stars, with no card. The separate pricing URL did not include dollar amounts in the HTML. Revenue source: https://trustmrr.com/startup/based-labs-ai (checked 2026-11-05). - StoryShort: MRR $21,236; all-time $602,390; 378 active subscriptions. Price: Monthly: Starter $39, Growth $69, Influencer $129, Ultra $199. Yearly on the same page: Starter $23/mo ($279 billed yearly), Growth $42/mo ($499), Influencer $75/mo ($899), Ultra $117/mo ($1,399). The page says yearly billing saves 40%. Revenue source: https://trustmrr.com/startup/storyshort (checked 2026-11-05). - Plutio: MRR $20,109; all-time $1,500,394; 908 active subscriptions. Price: Core $19/month, Pro $49/month, Max $199/month. The same page advertises 15% off any plan with code 15off4ever, so checkout can be lower than those list prices. Revenue source: https://trustmrr.com/startup/plutio-super-work-ai (checked 2026-11-05). - SuperX: MRR $19,937; all-time $264,810; 471 active subscriptions. Price: Pro is $49/month. Advanced is shown at a launch price of $49/month, with $99/month labeled as the normal price. Ultra is $199/month. Revenue source: https://trustmrr.com/startup/superx (checked 2026-11-05). - LocalRank: MRR $19,335; all-time $742,266; 193 active subscriptions. Price: Monthly: Starter $57/mo, Pro $297/mo, Scale $497/mo, Enterprise $2,997/mo. Annual billing on the same page: Starter $47.50/mo, Pro $247.50/mo, Scale $372.50/mo, Enterprise $1,664.17/mo. Revenue source: https://trustmrr.com/startup/localrank-so (checked 2026-11-05). - Taja: MRR $13,682; all-time $1,035,226; 343 active subscriptions. Price: Monthly: Starter $19.99, Professional $49.99, Teams/Agency $99.99. Billed annually: Starter $15.99/month, Professional $39.99/month, Teams/Agency $79.99/month. The page offers a 7-day free trial. Revenue source: https://trustmrr.com/startup/taja-ai (checked 2026-11-05). - NotFair: MRR $13,093; all-time $67,862; 110 active subscriptions. Price: Free is $0, with 7 days of unlimited access and then 300 operations a month. The Growth card shows about $79/mo billed annually ($950/year). The page title also says the free plan and Growth are $99/mo or $950/yr. Revenue source: https://trustmrr.com/startup/notfair (checked 2026-11-05). - MagicSlides: MRR $9,842; all-time $730,309; 750 active subscriptions. Price: Essential is shown at $8 and $6.67/month with $80 billed annually. Pro is shown at $16 and $12.42/month with $149 billed annually. Premium is shown at $29 and $23.25/month with $279 billed annually. A one-time designer service is $99, shown next to $500. Revenue source: https://trustmrr.com/startup/magicslides-app (checked 2026-11-05). - Visualizee: MRR $8,104; all-time $95,357; 287 active subscriptions. Price: Hobby $20/mo or $15/mo billed yearly ($180/year), Pro $40/mo or $30/mo billed yearly ($360/year), Max $80/mo or $60/mo billed yearly ($720/year). The page title also says plans start at $20 a month, with 5 free renders. Revenue source: https://trustmrr.com/startup/visualizee-ai (checked 2026-11-05).