SnT, University of Luxembourg1 University of Trento2 Fondazione Bruno Kessler3

Accepted at NeurIPS'26 Evaluation Track

NeurIPS 2026

TL;DR

CADTestBench is the first test-based benchmark for Text-to-CAD, based on CADTests, executable software tests operating directly on the CAD Model (B-Rep) that verify whether a generated CAD model satisfies the geometric and topological requirements of the input prompt.

CADTestBench teaser

CADTests can account for ambiguous design prompts (left) and identify subtle structural errors (right).

Abstract

Text-to-CAD has emerged as an important task with the potential to substantially accelerate design workflows. Despite its significance, there has been surprisingly little work on Text-to-CAD evaluation, and assessing CAD model generation performance remains a considerable challenge. In this work, we introduce a new evaluation perspective for Text-to-CAD based on automated testing. We propose CADTestBench, the first test-based benchmark for Text-to-CAD, based on CADTests, executable software tests that verify whether a generated CAD model satisfies the geometric and topological requirements of the input prompt. Using CADTestBench, we conduct comprehensive benchmarking of recent Text-to-CAD methods and further demonstrate that CADTests can also guide CAD model generation, yielding simple baselines that surpass performance of current methods. The CADTestBench dataset and code are publicly available.

CADTests in Action

Prompt

Generated model

Advantages of CADTests

Reference-free

Assess prompt compliance directly, rather than via alignment with a ground-truth solution.

Explicit verification

Every dimension and structural specification is treated as a strict constraint.

Interpretable

Failing tests return explicit log messages describing the cause of failure.

Efficient

Fast and deterministic, avoiding the inference costs of learnable semantic evaluators.

Consistent

Handle invalid generations without aggregating scores on method-specific subsets.

Iterative Refinement Framework

Framework step 1 of 10 Framework step 2 of 10 Framework step 3 of 10 Framework step 4 of 10 Framework step 5 of 10 Framework step 6 of 10 Framework step 7 of 10 Framework step 8 of 10 Framework step 9 of 10 Framework step 10 of 10

Results

Evaluation of Text-to-CAD methods on CADTestBench. Results are reported for both the abstract and detailed partitions. Text-to-CAD baselines are evaluated using different LLM planners. Performance is measured in terms of pass rate (PR), requirement score (RS), and invalidity rate (IR).

Method Planner Detailed Abstract
IR ↓RS ↑PR ↑ IR ↓RS ↑PR ↑
Finetuned Method
Text2CAD–0.0050.4050.0250.0150.6030.085
Train-Free Methods
CadCodeVerifyGPT-4.10.0250.7940.4100.0400.8800.630
10-shotsGPT-5.20.1750.7180.4650.2100.7420.580
SkilledGPT-5.20.1900.6880.4300.1800.7660.600
ReActDeepSeek-V30.0350.7520.3700.0050.8210.390
ReActQwen3-Coder-480B0.0250.7930.4250.0100.8810.605
ReActGPT-5.20.0000.8350.4800.0050.9160.695
ReAct + ImageGPT-5.20.0250.8220.4850.0050.9160.695
10-shotsClaude-4.6-Sonnet0.1550.7440.5100.1150.8300.640
SkilledClaude-4.6-Sonnet0.1850.7270.5050.1300.8100.640
ReActClaude-4.6-Sonnet0.0000.8740.5800.0000.9290.715
Test-based Baselines
CADTestsGPT-5.20.0200.8240.5000.0150.9250.720
CADTests + LogGPT-5.20.0150.8520.5150.0200.9220.700
CADTestsClaude-4.6-Sonnet0.0050.8820.5900.0000.9530.765
CADTests + LogClaude-4.6-Sonnet0.0000.8970.6250.0000.9620.810

Interactive Demo

Browse CADTestBench samples with their generated models, code and CADTests. It opens on the CADTests Claude-4.6-Sonnet run; pick another preloaded experiment and press Load to switch.

BibTeX

@inproceedings{mallis2026cadtests,
  title={Text-to-CAD Evaluation with CADTests},
  author={Dimitrios Mallis and Marco Wang and Ahmet Serdar Karadeniz and Elisa Ricci and Anis Kacem and Djamila Aouada},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2026}
}

Acknowledgements

This work is supported by the National Research Fund (FNR), Luxembourg, under the BRIDGES2021/IS/16849599/FREE-3D project and the JUMP2025/19750191/Morfis project.

University of Luxembourg, SnT and Luxembourg National Research Fund logos