CONIX Publication

We introduce a new silhouette-based representation for modeling clothed human bodies using deep generative models. Our method can reconstruct a complete and textured 3D model of a person wearing clothing from a single input picture. Inspired by the visual hull algorithm, our implicit representation uses 2D silhouettes and 3D joints of a body pose to describe the immense shape complexity of clothing variations. Given a segmented 2D silhouette of a person and its inferred 3D joints from the input picture, we synthesize consistent silhouettes from novel view points around the subject. The synthesized silhouettes that are the most consistent with the input segmentation are fed into a deep visual hull algorithm for robust 3D shape extraction. We then infer the complete texture of the subject’s back-view using the frontal image and segmentation mask as input to a conditional generative adversarial network. Our experiments demonstrate that the proposed silhouette model is a particularly efficient representation and that the appearance of the back-view can be predicted reliably using a deep learning approach. Within the context of single-view 3D modeling, classic methods based on parametric models would fail, while our approach succeeds to produce results are comparable to those obtained from multiple views.