LUCAS: Layered Universal Codec Avatars

Di Liu1,2, Teng Deng1, Giljoo Nam1, Yu Rong1, Stanislav Pidhorskyi1, Junxuan Li1, Jason Saragih1, Dimitris N. Metaxas2, Chen Cao1
1Codec Avatars Lab, Meta, 2Rutgers University
CVPR 2025
Paper arXiv SuppMat

LUCAS: A novel approach for high-fidelity Layered Universal Codec Avatars. We disentangle face and hair into a layered structure, supporting both real-time mesh-based avatar~(45 FPS on mobile) and high-fidelity Gaussian avatar generation. Our universal layered prior model also enables accurate expression and pose transfer, even for unseen subjects, while maintaining visual quality.

Abstract

Photorealistic 3D head avatar reconstruction faces critical challenges in modeling dynamic face-hair interactions and achieving cross-identity generalization, particularly during expressions and head movements. We present LUCAS, a novel Universal Prior Model (UPM) for codec avatar modeling that disentangles face and hair through a layered representation. Unlike previous UPMs that treat hair as an integral part of the head, our approach separates the modeling of the hairless head and hair into distinct branches. LUCAS is the first to introduce a mesh-based UPM, facilitating real-time rendering on devices. Our layered representation also improves the anchor geometry for precise and visually appealing Gaussian renderings. Experimental results indicate that LUCAS outperforms existing single-mesh and Gaussian-based avatar models in both quantitative and qualitative assessments, including evaluations on held-out subjects in zero-shot driving scenarios. LUCAS demonstrates superior dynamic performance in managing head pose changes, expression transfer, and hairstyle variations, thereby advancing the state-of-the-art in 3D head avatar reconstruction.

Method

Overview of LUCAS. (a) Our identity-conditioned hypernetwork generates identity-specific features and untied biases from neutral geometry and appearance data. (b) The expression encoder learns a unified expression code space that enables consistent expression transfer across identities. (c) Given expression code, view direction, and poses, our compositional avatar decoder produces separate geometry and appearance maps for face and hair. These are combined with mean geometry and geometry displacement for multi-mesh rendering, followed by separate pixel decoders for the final avatar generation.

Dehaired head and hair geometries

Our method precisely disentangles dehaired head from hair for different users.

Avatars driving with diverse inputs

Expressions from a source identity (top left) are accurately transferred to multiple personalized avatars, preserving fine details in both wrinkles and hair.

Avatars driving on unseen identities

Our model successfully transfers novel expressions to untrained identities while maintaining precise facial features, particularly around the eyes and mouth regions.

Evaluation of layered representation on head pose animation

Our LUCAS mesh tracks hair strand deformation and aligns with head and neck movements, outperforming uPiCA in dynamic scenarios.

Ablation study on expression code

Expression code improves face-hair synchronization during expressions.

Application on hairstyle switching

We combine face condition from subject B, hair condition from subject C, and expressions from subject A. Our model maintains high-fidelity facial details while accurately preserving the characteristics of both the chosen face and hairstyle.

BibTeX

@inproceedings{liu2025lucas,
      title     = {LUCAS: Layered Universal Codec Avatars},
      author = {Liu, Di and Deng, Teng and Nam, Giljoo and Rong, Yu and Pidhorskyi, Stanislav and Li, Junxuan and Saragih, Jason and Metaxas, Dimitris N. and Cao, Chen},
      booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      month     = {June},
      year      = {2025},
    }