Theme
AI Resources
ClawMark
ClawMark is a living-world benchmark positioned around multi-day, multimodal coworker agents, with 100 tasks spread across professional domains and tool-backed work environments.
The official repository presents ClawMark as a benchmark for evaluating coworker-style agents across timelines, multimodal evidence, and multiple external systems. Use this as a first read, not a recommendation. Open the original project before trusting details like terms, limits, privacy, cost, setup, or safety.
What it is
A benchmark for coworker agents
ClawMark is framed as an evaluation benchmark rather than an agent product, with the repository centered on long-running work tasks that unfold across several days and several tool environments.
Why it stands out
Multi-day and multimodal pressure
It brings together timelines, real tool backends, multimodal evidence, and deterministic scoring instead of a single-turn prompt or a lighter toy environment.
Availability
Repository with tasks and scoring framework
Public materials are available through a GitHub repository containing the benchmark framework, task layouts, scoring setup, and environment instructions for readers who want to inspect how the evaluation is structured.
Why it matters
Why readers may notice it
Many agent benchmarks still feel too narrow or too short. A benchmark built around multi-day work, changing state, and multimodal evidence gives readers a more demanding lens for judging agent capability.
What readers may want to know
Where it fits
This project fits in the benchmark and dataset layer rather than the model or app layer. It is more relevant to readers evaluating coworker-style agents and agent robustness than to readers looking for a standalone assistant product.
Reporting note
What appears notable
The official materials are useful for checking the benchmark's combination of 100 tasks, 13 professional domains, multimodal raw evidence, cross-environment tool coordination, and deterministic rule-based scoring.
Before using
What readers may want to review
Which task domains and external environments are included in the current benchmark release.
What local setup, credentials, and Docker requirements are needed to run the full evaluation stack.
How the reported scores, runs, and token counts were defined before comparing models too directly.
Reader fit
Who may find it relevant
Readers following agent benchmarks and coworker-agent evaluation work.
Builders who want a harder benchmark for multi-step, multimodal, tool-using agents.
Less relevant for readers focused only on consumer chat products or single-model demos.
Editorial note
Why it is included here
Use ClawMark as a source check on a benchmark style built around longer timelines, multimodal evidence, and changing work environments.
Source links
Original materials
Reader note
Before relying on this entry
LifeHubber lists entries to help readers inspect AI projects, not to endorse them or prove they are safe, suitable, accurate, maintained, or right for a specific use. We do not verify every entry in depth. Before relying on anything listed, review the original materials, terms, privacy practices, limits, and risks that matter for your situation.
Get occasional updates when new AI resources are added
Occasional notes when new AI resources are added. The form below is handled by the mailing-list service, so its own terms apply when you subscribe.
More in Datasets
Keep browsing this category
A few more places to continue in datasets.
General365
meituan-longcat/General365
A manually curated benchmark for general reasoning in LLMs, designed around high difficulty, broad task diversity, K-12-scope knowledge, and hybrid scoring.
LARYBench
meituan-longcat/LARYBench
A benchmark for evaluating latent action representations, with pipelines for action semantics, robotic control regression, and broader vision-to-action alignment.
Monitorability Evals
openai/monitorability-evals
An OpenAI evaluation-data release for studying monitorability, with public eval splits, prompt templates, dataset mappings, and metric code from the Monitoring Monitorability paper.
Related in LifeHubber
Keep the thread going
Follow the next layer with AI Resources for AI projects worth inspecting at the source, AI Guides for decision habits for messy AI choices, AI Access for free and low-cost ways to compare AI model access, AI Ballot for a clearer view of what readers are leaning toward, and AI Radar for AI stories that deserve a second look.