Environment

The agents' workbench

The agents write no free-form kernel code. They drive FreeCAD through a REST command interface with safe operations at a high level of abstraction – every operation is at once a command, a history entry and an undo step. What this environment does is described here in words; the code itself is not published.

In brief: The AI receives a technical drawing and builds a 3D model from it in a real CAD program (FreeCAD). It does not click with a mouse; it sends validated commands to an interface. In the end it has to deliver three things: the model, the traceable build path and a measurement log. Unfamiliar terms are explained in the glossary.

API groups

The command interface bundles the capabilities into a few, well-delimited groups. Each is reachable over HTTP and returns structured, self-healing errors – wrong inputs come back with a correction suggestion instead of failing silently.

Building

Primitives & parametric CAD

Boxes, cylinders, rotationally symmetric base bodies, sketch/pad and further parametric building blocks – each with a clear coordinate contract (base point or centered position) and an accompanying bounding box (the enclosing box), so drawing coordinates are directly usable.

Combining

Boolean operations

Union, difference and intersection on a real B-rep kernel. Volume measurements are secured against the silent pitfalls of boolean evaluation (cross-check plus an independent point sample), so “penetrated” or “not penetrated” is dependable.

Addressing

Semantic selectors

Faces and edges are not addressed via fragile indices like Face6, but semantically (proximity to a point, role, relation). This way a reference survives the topology drift that every new chamfer or hole triggers.

Depicting & measuring

TechDraw projection & measurement

Standard-compliant views are projected from the 3D model and measured machine-readably. This is exactly what carries drawing fidelity and dimensional accuracy: dimensions are re-measured on the projected image, not asserted from the model.

Securing

Validation

B-rep and mesh checking, solid counting, detection of empty or degenerate results, hole-pattern analysis. Validation is the core of every workflow – it separates “topologically valid” from “professionally correct”.

Canonical truth

Command interface & provenance history

Every change runs through the same operation history – agent commands, manual changes in the interface and results of compute jobs on equal footing. The model is thereby readable at any time as an editable, traceable operation list.

The task package

Every case provides the agent with three things — identical across all runs:

  • the technical drawing as a PDF;
  • a machine text-recognition preparation of that drawing: the recognized text (dimension figures, title block) and the cropped view regions as images — generated in advance with an image-recognition model, without manual post-editing;
  • the task statement with the delivery obligation (below).

The preparation makes the drawing machine-accessible, but is deliberately as imperfect as text recognition simply is: it overlooks dimensions or reads them wrong. Part of the task is to close these gaps using the image crops and the vector geometry of the PDF itself — the exemplary measurement log shows how an agent documents exactly that. The package contains no reference data (isolation protocol).

Delivery obligation

A run counts only if the agent delivers a complete, verifiable package. Three components are mandatory:

3D model
The finished model in the losslessly exchangeable STEP format – the measurable geometry.
Build script
The reproducible build path over the command interface: the model as an executable operation sequence (build.py).
Measurement log
The self-documentation (notes.md) with the evidence of the projection check: which dimensions were measured, which assumptions were made.

The measurement log is more than prose: it contains the target-vs-actual comparison of the agent's own TechDraw measurement. This makes it possible later to trace every movement in the figures back to a decision by the agent – the basis of the optimization loop.

Delimitation from the delivery rate: The delivery obligation covers all three components. The delivery rate reported in the results measures the machine-checkable criterion among them — a scorable model, that is, exactly one body (solid) in the STEP. Build script and measurement log are checked in substance (for instance the evidence of the projection check), but do not enter this rate.

Agents & models

As agents, two command-line agents run in their own isolated work environments – each with its own FreeCAD server, so that a crash costs only its own track. The matrix comprises seven models from both providers across several size classes; the two top systems were each measured three times (eleven runs in total):

AgentModelRole
codexgpt-5.6-soltop system (3 runs)
codexgpt-5.6-lunasmall 5.6 model
codexgpt-5.6-terrasmall 5.6 model
codexgpt-5.4-miniprevious generation (generational comparison)
Claude Codeclaude-opus-5top system (3 runs)
Claude Codeclaude-sonnet-5mid-size model
Claude Codeclaude-haiku-4-5small model

All runs proceed in parallel on separate tracks, each with its own server. The full evaluation of all eleven runs – sorted and with delivery rate – is shown in the results.

The agents' rulebook

Above the plain command interface lies a judgment and process layer: the agents' rulebook. It tells the agent not only what works, but what is right – and is handed to the agent in an evidence-free edition, so that it contains no solution-revealing dimensions.

  • Drawing → CAD: anchor to text recognition. Do not estimate dimensions from the raw line image, but model anchored to the supplied preparation, close its gaps via image crops and PDF vector geometry, and check the result both numerically and by multi-view inspection against the source.
  • Projection check. After the build, generate a technical drawing of your own and measure target against actual – spans, wall thicknesses from circle differences, angles from line vectors, breakthrough check. Only once this check passes does a case count as delivered.
  • Plausibility checks. A weight check against the title block, symmetry and clearance checks, feature counts per view. Core principle: anchoring instead of distillation – every rule needs an executable counter-check or the qualified engineer's sign-off.

The rulebook is described here, not printed. What matters for the measurement is: in isolation it contains the rules, but not the solutions.

How the rulebook is improved