Deep Learning Assisted Optimization for 3D Reconstruction from Single 2D Line Drawings

Jia Zheng1     Yifan Zhu2     Kehan Wang3     Qiang Zou4     Zihan Zhou1
1Manycore Tech Inc.     2Nanjing University of Aeronautics and Astronautics     3University of California, Berkeley     4State Key Lab of CAD&CG, Zhejiang University

Given (a) an input line drawing, we train deep models to predict (b) geometric constraints (e.g., parallel constraints) and (c) initial depth value of the vertices, which are then used in numerical optimization for geometric constraint solving to obtain an accurate and compact 3D model (d).


In this paper, we revisit the long-standing problem of automatic reconstruction of 3D objects from single line drawings. Previous optimization-based methods can generate compact and accurate 3D models, but their success rates depend heavily on the ability to (i) identifying a sufficient set of true geometric constraints, and (ii) choosing a good initial value for the numerical optimization. In view of these challenges, we propose to train deep neural networks to detect pairwise relationships among geometric entities (i.e., edges) in the 3D object, and to predict initial depth value of the vertices. Our experiments on a large dataset of CAD models show that, by leveraging deep learning in a geometric constraint solving pipeline, the success rate of optimization-based 3D reconstruction can be drastically improved.

Interactive Results

Input Image
Wireframe Results
(Novel View)
Mesh Results
Ground Truth


  author  = {Zheng, Jia and Zhu, Yifan and Wang, Kehan and Zou, Qiang and Zhou, Zihan},
  title   = {Deep Learning Assisted Optimization for 3D Reconstruction from Single 2D Line Drawings},
  journal = {CoRR},
  volume  = {abs/2209.02692},
  year    = {2022},


This work was done during Yifan Zhu and Kehan Wang's internships at Manycore Tech Inc. This work was supported in part by the Key R&D Program of Zhejiang Province (No. 2022C01025), the National Natural Science Foundation of China (No. 62102355), the Natural Science Foundation of Zhejiang Province (No. LQ22F020012).