What this lab is, why no third-party data appears here, and under which licenses the tools we use are published.
This lab publishes its own measurements only. There is no evaluation procedure for third parties, no submissions, and no competitive element. We show how far AI agents get in our environment — and name the limits of that statement ourselves.
Because divergent interpretations are the real insight. When an agent folds a drawing wrong or builds solid instead of hollow, that is precisely the evidence of what the lab measures. The viewer compares the source video against the reconstruction themselves.
The agents' rules of thumb were sharpened on seven cases. Their result is therefore optimistically biased (training = test) and is reported separately from the remaining independent cases. The clean generalization test is the independent cases. That the anchoring cases nonetheless score worse on average is due to their selection: the very cases where early runs failed — the harder parts — were chosen for anchoring. Details in the methodology.
No. This lab is purely a research and measurement effort: we document how far AI agents get in a CAD environment — there is no product, no service, and no contract processing of drawings. Professional exchange is welcome (contact below).
No. What is measured is a bounded sub-task — reconstructing a 3D model from a fully dimensioned drawing. Even the best system is far from meeting all requirements at once in every case, and tolerances, material choice, simulation, and manufacturing are not part of the measurement at all. Anyone who wants to judge, check, and sign off on the results still needs CAD and design knowledge — the measurement shows how good a tool is becoming, not that a profession is disappearing.
Agent runs are stochastic; the shape fidelity of a case scatters between runs — for individual cases by several tenths — without anything having changed in the model. That is why we measure the top systems multiple times and report the ranges. A single case figure without context cannot be interpreted.
The underlying drawings and models are works of third parties. We decided not to publish them — neither as an excerpt or title block nor as a depiction of the reference model. Cases are referenced solely through their source video and SHA256 checksums; the sources for the original data are given in the respective video description of the author.
Only two things: the embedded original source video and the 3D depictions of our own AI reconstructions. These depictions are our own works derived from the non-protectable dimensional facts of the drawings and are marked as AI reconstructions — not as original models. The only case we show in full is our own, self-designed demo case.
The depictions of our reconstructions can be removed at any time on short notice. On notice of a possible infringement we respond promptly — see contact below and in the imprint.
No. Metric and environment are fully described in substance — every dimension, threshold, and ceiling is explained — but the scoring code, interface code, and rulebook remain internal. What is traceable is therefore the method, not the tool: an independent recomputation of the same figures is not possible from the outside.
The author and person responsible in substance is Dipl.-Ing. Sören Gebbert, managing director of the Institut für holistische Technologieforschung GmbH, which operates this website. Concept, measurement environment, references, and evaluation come from a single hand; the lab publishes its own measurements only.
The CAD kernel of the environment is FreeCAD, free software under the GNU Lesser General Public License (LGPL). FreeCAD is an independent project with no connection to this test lab; we use it as a tool and thank the developer community. Trademark and copyright in FreeCAD rest with the respective rights holders.
By email at info@holistech.ai. The operator is the Institut für holistische Technologieforschung GmbH; mandatory legal information is in the imprint, notes on data protection in the privacy policy.