Computer Graphics and Visualization Lab

Computer Graphics and Visualization Lab

University of Hong Kong

Computer Graphics and Visualization

CGVU Lab, led by Prof. Taku Komura, belongs to the Department of Computer Science, the University of Hong Kong. Our research focus is on physically-based animation and the application of machine learning techniques for animation synthesis.

group.png

Meet the Team

Principal Investigator

Avatar

Taku Komura

Professor

Physical Simulation, Character Animation, 3D Modelling

Research Staff

Avatar

Floyd M. Chitalu

Senior researcher, since Nov. 2022.

Physical Simulation

Avatar

Yinghao Huang

Postdoc, since Aug. 2023.

Human Pose Estimation, Human Motion Generation

Avatar

Chen Peng

Postdoc, since Sep. 2023.

Physically-Based Animation, Fluid Simulation

Graduate Students

Avatar

Linxu Fan

PHD, since Nov. 2019.

Physical Simulation

Avatar

Zhiyang Dou

PhD, since Aug. 2020.
Co-supv. by Prof. Wenping Wang.

Character Animation, Geometric Computing

Avatar

Dafei Qin

PhD, since Sep. 2020.

Facial Animation, Neural Rendering

Avatar

Mingyi Shi

PhD, since Nov. 2020.

3D Human Moton, Generative AI

Avatar

Jintao Lu

PhD, since Sept. 2021.

Human Scene Interaction, Motion Control

Avatar

Huancheng Lin

M.Phil., since Sep. 2022.

Physical Simulation

Avatar

Kemeng Huang

PhD, since Sep. 2022.

Physical Simulation, High Performance Computing

Avatar

Guying Lin

MPhil, since Sept. 2022.
Co-supv. by Prof. Wenping Wang.

Neural Implicit Surface Representation

Avatar

Wenjia Wang

PhD, since Jan. 2023.

3D Reconstruction, Human Pose Estimation, Human Motion Generation

Avatar

Zhouyingcheng Liao

PhD, since Jan. 2023.

Neural Cloth Simulation, Character Animation

Avatar

Yuke Lou

M.Phil, since Sept. 2023.

Motion Generation

Avatar

Xiaohan Ye

PhD, since Sept. 2023.

Physics Simulation, Motion Control

Research Assistant

Avatar

Leo Ho

Research Assistant, since Aug. 2023.

Digital Humans, Motion Synthesis

Avatar

Xinyu Lu

Research Assistant, since Sep. 2023.

Physically-Based Animation, Simulation

Recent Publications

Quickly discover relevant content by filtering publications.
CBIL: Collective Behavior Imitation Learning for Fish from Real Videos

CBIL: Collective Behavior Imitation Learning for Fish from Real Videos

Reproducing realistic collective behaviors presents a captivating yet formidable challenge. Traditional rule-based methods rely on hand-crafted principles, limiting motion diversity and realism in generated collective behaviors. Recent imitation learning methods learn from data but often require ground truth motion trajectories and struggle with authenticity, especially in high-density groups with erratic movements. In this paper, we present a scalable approach, Collective Behavior Imitation Learning (CBIL), for learning fish schooling behavior directly from videos, without relying on captured motion trajectories. Our method first leverages Video Representation Learning, where a Masked Video AutoEncoder (MVAE) extracts implicit states from video inputs in a self-supervised manner. The MVAE effectively maps 2D observations to implicit states that are compact and expressive for following the imitation learning stage. Then, we propose a novel adversarial imitation learning method to effectively capture complex movements of the schools of fish, allowing for efficient imitation of the distribution for motion patterns measured in the latent space. It also incorporates bio-inspired rewards alongside priors to regularize and stabilize training. Once trained, CBIL can be used for various animation tasks with the learned collective motion priors. We further show its effectiveness across different species. Finally, we demonstrate the application of our system in detecting abnormal fish behavior from in-the-wild videos.

Analytic rotation-invariant modelling of anisotropic finite elements

Analytic rotation-invariant modelling of anisotropic finite elements

Anisotropic hyperelastic distortion energies are used to solve many problems in fields like computer graphics and engineering with applications in shape analysis, deformation, design, mesh parameterization, biomechanics and more. However, formulating a robust anisotropic energy that is low-order and yet sufficiently non-linear remains a challenging problem for achieving the convergence promised by Newton-type methods in numerical optimization. In this paper, we propose a novel analytic formulation of an anisotropic energy that is smooth everywhere, low-order, rotationally-invariant and at-least twice differentiable. At its core, our approach utilizes implicit rotation factorizations with invariants of the Cauchy-Green tensor that arises from the deformation gradient. The versatility and generality of our analysis is demonstrated through a variety of examples, where we also show that the constitutive law suggested by the anisotropic version of the well-known \textit{As-Rigid-As-Possible} energy is the foundational parametric description of both passive and active elastic materials. The generality of our approach means that we can systematically derive the force and force-Jacobian expressions for use in implicit and quasistatic numerical optimization schemes, and we can also use our analysis to rewrite, simplify and speedup several existing anisotropic \textit{and} isotropic distortion energies with guaranteed inversion-safety.

EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation

EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation

We introduce Efficient Motion Diffusion Model (EMDM) for fast and high-quality human motion generation. Current state-of-the-art generative diffusion models have produced impressive results but struggle to achieve fast generation without sacrificing quality. On the one hand, previous works, like motion latent diffusion, conduct diffusion within a latent space for efficiency, but learning such a latent space can be a non-trivial effort. On the other hand, accelerating generation by naively increasing the sampling step size, e.g., DDIM, often leads to quality degradation as it fails to approximate the complex denoising distribution. To address these issues, we propose EMDM, which captures the complex distribution during multiple sampling steps in the diffusion model, allowing for much fewer sampling steps and significant acceleration in generation. This is achieved by a conditional denoising diffusion GAN to capture multimodal data distributions among arbitrary (and potentially larger) step sizes conditioned on control signals, enabling fewer-step motion sampling with high fidelity and diversity. To minimize undesired motion artifacts, geometric losses are imposed during network learning. As a result, EMDM achieves real-time motion generation and significantly improves the efficiency of motion diffusion models compared to existing methods while achieving high-quality motion generation. Our code is available at \url{https://github.com/Frank-ZY-Dou/EMDM}.

Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models

Surf-D: Generating High-Quality Surfaces of Arbitrary Topologies Using Diffusion Models

We present Surf-D, a novel method for generating high-quality 3D shapes as Surfaces with arbitrary topologies using Diffusion models. Previous methods explored shape generation with different representations and they suffer from limited topologies and poor geometry details. To generate high-quality surfaces of arbitrary topologies, we use the Unsigned Distance Field (UDF) as our surface representation to accommodate arbitrary topologies. Furthermore, we propose a new pipeline that employs a point-based AutoEncoder to learn a compact and continuous latent space for accurately encoding UDF and support high-resolution mesh extraction. We further show that our new pipeline significantly outperforms the prior approaches to learning the distance fields, such as the grid-based AutoEncoder, which is not scalable and incapable of learning accurate UDF. In addition, we adopt a curriculum learning strategy to efficiently embed various surfaces. With the pretrained shape latent space, we employ a latent diffusion model to acquire the distribution of various shapes. Extensive experiments are presented on using Surf-D for unconditional generation, category conditional generation, image conditional generation, and text-to-shape tasks. The experiments demonstrate the superior performance of Surf-D in shape generation across multiple modalities as conditions.

Coverage Axis++: Efficient Skeletal Points Selection for 3D Shape Skeletonization

Coverage Axis++: Efficient Skeletal Points Selection for 3D Shape Skeletonization

We introduce Coverage Axis++, a novel and efficient approach to 3D shape skeletonization. The current state-of-the-art approaches for this task often rely on the watertightness of the input or suffer from substantial computational costs, thereby limiting their practicality. To address this challenge, Coverage Axis++ proposes a heuristic algorithm to select skeletal points, offering a high-accuracy approximation of the Medial Axis Transform (MAT) while significantly mitigating computational intensity for various shape representations. We introduce a simple yet effective strategy that considers shape coverage, uniformity, and centrality to derive skeletal points. The selection procedure enforces consistency with the shape structure while favoring the dominant medial balls, which thus introduces a compact underlying shape representation in terms of MAT. As a result, Coverage Axis++ allows for skeletonization for various shape representations (e.g., water-tight meshes, triangle soups, point clouds), specification of the number of skeletal points, few hyperparameters, and highly efficient computation with improved reconstruction accuracy. Extensive experiments across a wide range of 3D shapes validate the efficiency and effectiveness of Coverage Axis++. \ZY{Our codes are available at \url{https://github.com/Frank-ZY-Dou/Coverage_Axis}.

Contact

  • CYC Building, the University of Hong Kong, Hong Kong,