PH. D.

COMPUTER SCIENTIST - TECH TRANSFER
ADOBE - 3D & IMMERSIVE

ABOUT

Raymond Yun Fei is a Computer Scientist at Adobe 3D&I division (click to join!). He connects research with Adobe Substance 3D products through tech transfers in geometry processing, physics simulation, and machine learning. Raymond published research works of a broad spectrum in computer graphics, including the simulation of complex materials, GPGPU and real-time rendering. Multiple well-known media have covered his research in simulation, and Raymond has also served as a reviewer for top academic conferences such as SIGGRAPH North America/Asia.

PUBLICATIONS

Principles towards Real-Time Simulation of Material Point Method on Modern GPUs
Talks on Game Developers Conference (GDC) and NVIDIA GPU Technology Conference (GTC), 2022
Yun (Raymond) Fei, Yuhan Huang, Ming Gao
Abstract | Project Page
GPU simulation

Physics-based simulation has been actively employed in generating offline visual effects in the film and animation industry. However, the computations required for high-quality scenarios are generally immense, deterring its adoption in real-time applications, e.g., virtual production, avatar live-streaming, and cloud gaming. We summarize the principles that can accelerate the computation pipeline on single-GPU and multi-GPU platforms through extensive investigation and comprehension of modern GPU architecture. We further demonstrate the effectiveness of these principles by applying them to the material point method to build up our framework, which achieves 1.7×--8.6× speedup on a single GPU and 2.5×--14.8× on four GPUs compared to the state-of-the-art. Our pipeline is specifically designed for real-time applications (i.e., scenarios with small to medium particles) and achieves significant multi-GPU efficiency. We demonstrate our pipeline by simulating a snow scenario with 1.33M particles and a fountain scenario with 143K particles in real-time (on average, 68.5 and 55.9 frame-per-second, respectively) on four NVIDIA Tesla V100 GPUs interconnected with NVLinks.

@article{fei2021principles,
  author = {Fei, Yun (Raymond) and Huang, Yuhan and Gao, Ming},
  title = {Principles towards Real-Time Simulation of Material Point Method on Modern GPUs},
  journal = {arXiv preprint},
  volume = {abs/2111.00699},
  year = {2021},
  url = {https://arxiv.org/abs/2111.00699},
  eprinttype = {arXiv},
  eprint = {2111.00699}
}

Revisiting Integration in the Material Point Method: A Scheme for Easier Separation and Less Dissipation
ACM Transactions on Graphics (SIGGRAPH North America), 2021
Yun (Raymond) Fei, Qi Guo, Rundong Wu, Li Huang, Ming Gao
Abstract | Project Page | In Press: Two Minute Papers, GameRes (in Chinese), TechAcute
simulation

The material point method recently demonstrated its efficacy at simulating many materials and the coupling between them on a massive scale. However, in scenarios containing debris, MPM manifests more dissipation and numerical viscosity than traditional Lagrangian methods. We have two observations from carefully revisiting existing integration methods used in MPM. First, nearby particles would end up with smoothed velocities without recovering momentum for each particle during the particle-grid-particle transfers. Second, most existing integrators assume continuity in the entire domain and advect particles by directly interpolating the positions from deformed nodal positions, which would trap the particles and make them harder to separate. We propose an integration scheme that corrects particle positions at each time step. We demonstrate our method's effectiveness with several large-scale simulations involving brittle materials. Our approach effectively reduces diffusion and unphysical viscosity compared to traditional integrators.

Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers
Proceedings of NeurIPS, 2020
Kiwon Um, Robert Brand, Yun (Raymond) Fei, Philipp Holl, Nils Thuerey
Abstract | Project Page | arXiv | GitHub | YouTube
simulation machine learning

Finding accurate solutions to partial differential equations (PDEs) is a crucial task in all scientific and engineering disciplines. It has recently been shown that machine learning methods can improve the solution accuracy by correcting for effects not captured by the discretized PDE. We target the problem of reducing numerical errors of iterative PDE solvers and compare different learning approaches for finding complex correction functions. We find that previously used learning approaches are significantly outperformed by methods that integrate the solver into the training loop and thereby allow the model to interact with the PDE during training. This provides the model with realistic input distributions that take previous corrections into account, yielding improvements in accuracy with stable rollouts of several hundred recurrent evaluation steps and surpassing even tailored supervised variants. We highlight the performance of the differentiable physics networks for a wide variety of PDEs, from non-linear advection-diffusion systems to three-dimensional Navier-Stokes flows.

Multi-Scale Models to Simulate Interactions between Liquid and Thin Structures
Dissertation at Columbia University, honorable mention for Outstanding SIGGRAPH Dissertation Award, 2020
Advisors: Eitan Grinspun, Changxi Zheng
Abstract | Preprint: Original (54.2MB), Reduced (5.1MB) | Columbia Academic Commons
simulation

In this dissertation, we introduce a framework for simulating the dynamics between liquid and thin structures, including the effects of buoyancy, drag, capillary cohesion, dripping, and diffusion. After introducing related works, Part I begins with a discussion on the interactions between Newtonian fluid and fabrics. In this discussion, we treat both the fluid and the fabrics as continuum media; thus, the physical model is built from mixture theory. In Part II, we discuss the interactions between Newtonian fluid and hairs. To have more detailed dynamics, we no longer treat the hairs as continuum media. Instead, we treat them as discrete Kirchhoff rods. To deal with the thin layer of liquid that clings to the hairs, we augment each hair strand with a height field representation, through which we introduce a new reduced-dimensional flow model to solve the motion of liquid along the longitudinal direction of each hair. In addition, we develop a faithful model for the hairs' cohesion induced by surface tension, where a penalty force is applied to simulate the collision and cohesion between hairs. To enable the discrete strands interact with continuum-based, shear-dependent liquid, in Part III, we develop models that account for the volume change of the liquid as it passes through strands and the momentum exchange between the strands and the liquid. Accordingly, we extend the reduced-dimensional flow model to simulate liquid with elastoviscoplastic behavior. Furthermore, we use a constraint-based model to replace the penalty-force model to handle contact, which enables an accurate simulation of the frictional and adhesive effects between wet strands. We also present a principled method to preserve the total momentum of a strand and its surface flow, as well as an analytic plastic flow approach for Herschel-Bulkley fluid that enables stable semi-implicit integration at larger time steps.
We demonstrate a wide range of effects, including the challenging animation scenarios involving splashing, wringing, and colliding of wet clothes, as well as flipping of hair, animals shaking, spinning roller brushes from car washes being dunked in water, and intricate hair coalescence effects. For complex liquids, we explore a series of challenging scenarios, including strands interacting with oil paint, mud, cream, melted chocolate, and pasta sauce.

A Multi-Scale Model for Coupling Strands with Shear-Dependent Liquid
ACM Transactions on Graphics (SIGGRAPH Asia), 2019, selected as the front cover
Yun (Raymond) Fei, Christopher Batty, Eitan Grinspun, Changxi Zheng
Abstract | Project Page | In Press: TechXplore, AAAS EurekAlert!, Two Minute Papers
simulation

We propose a framework for simulating the complex dynamics of strands interacting with compressible, shear-dependent liquids, such as oil paint, mud, cream, melted chocolate, and pasta sauce. Our framework contains three main components: the strands modeled as discrete rods, the bulk liquid represented as a continuum (material point method), and a reduced-dimensional flow of liquid on the surface of the strands with detailed elastoviscoplastic behavior. These three components are tightly coupled together. To enable discrete strands interacting with continuum-based liquid, we develop models that account for the volume change of the liquid as it passes through strands and the momentum exchange between the strands and the liquid. We also develop an extended constraint-based collision handling method that supports cohesion between strands. Furthermore, we present a principled method to preserve the total momentum of a strand and its surface flow, as well as an analytic plastic flow approach for Herschel-Bulkley fluid that enables stable semi-implicit integration at larger time steps. We explore a series of challenging scenarios, involving splashing, shaking, and agitating the liquid which causes the strands to stick together and become entangled.

Addressing Troubles with Double Bubbles: Convergence and Stability at Multi-Bubble Junctions
Technical Report, 2019
Yun (Raymond) Fei, Christopher Batty, Eitan Grinspun
Abstract | ArXiv Page | GitHub
simulation

In this report we discuss and propose a correction to a convergence and stability issue occurring in the work of Da et al.[2015], in which they proposed a numerical model to simulate soap bubbles.

Mechanics-Aware Modeling of Cloth Appearance
IEEE Transactions on Visualization and Computer Graphics (Pacific Graphics), 2019
Zahra Montazeri, Chang Xiao, Yun (Raymond) Fei, Changxi Zheng, Shuang Zhao,
Abstract | Project Page | IEEE Digital Library | Video: YouTube, bilibili | Talk at Pacific Graphics 2019: YouTube
simulation rendering machine learning

Micro-appearance models have brought unprecedented fidelity and details to cloth rendering. Yet, these models neglect fabric mechanics: when a piece of cloth interacts with the environment, its yarn and fiber arrangement usually changes in response to external contact and tension forces. Since subtle changes of a fabric's microstructures can greatly affect its macroscopic appearance, mechanics-driven appearance variation of fabrics has been a phenomenon that remains to be captured. We introduce a mechanics-aware model that adapts the microstructures of cloth yarns in a physics-based manner. Our technique works on two distinct physical scales: using physics-based simulations of individual yarns, we capture the rearrangement of yarn-level structures in response to external forces. These yarn structures are further enriched to obtain appearance-driving fiber-level details. The cross-scale enrichment is made practical through a new parameter fitting algorithm for simulation, an augmented procedural yarn model coupled with a custom-design regression neural network. We train the network using a dataset generated by joint simulations at both the yarn and the fiber levels. Through several examples, we demonstrate that our model is capable of synthesizing photorealistic cloth appearance in a %dynamic and mechanically plausible way.

A Multi-Scale Model for Simulating Liquid-Fabric Interactions
ACM Transactions on Graphics (SIGGRAPH North America), 2018
Yun (Raymond) Fei, Christopher Batty, Eitan Grinspun, Changxi Zheng
Abstract | Project Page | In Press: Deutschlandfunk (in German), Two Minute Papers, 3DVF (in French), Columbia Engineering, CGWorld (in Chinese)
simulation

We propose a method for simulating the complex dynamics of partially and fully saturated woven and knit fabrics interacting with liquid, including the effects of buoyancy, nonlinear drag, pore (capillary) pressure, dripping, and convection-diffusion. Our model evolves the velocity fields of both the liquid and solid relying on mixture theory, as well as tracking a scalar saturation variable that affects the pore pressure forces in the fluid. We consider the porous microstructure implied by the fibers composing individual threads, and use it to derive homogenized drag and pore pressure models that faithfully reflect the anisotropy of fabrics. In addition to the bulk liquid and fabric motion, we derive a quasi-static flow model that accounts for liquid spreading within the fabric itself. Our implementation significantly extends standard numerical cloth and fluid models to support the diverse behaviors of wet fabric, and includes a numerical method tailored to cope with the challenging nonlinearities of the problem. We explore a range of fabric-water interactions to validate our model, including challenging animation scenarios involving splashing, wringing, and collisions with obstacles, along with qualitative comparisons against simple physical experiments.

A Multi-Scale Model for Simulating Liquid-Hair Interactions
ACM Transactions on Graphics (SIGGRAPH North America), 2017
Yun (Raymond) Fei, Henrique Teles Maia, Christopher Batty, Changxi Zheng, Eitan Grinspun
Abstract | Project Page | In Press: 80.lv, Two Minute Papers, CG Channel, PC Gamer, CGWorld (in Chinese), 3Dnchu (in Japanese), GeekiMovie (in Chinese)
simulation

The diverse interactions between hair and liquid are complex and span multiple length scales, yet are central to the appearance of humans and animals in many situations. We therefore propose a novel multi-component simulation framework that treats many of the key physical mechanisms governing the dynamics of wet hair. The foundations of our approach are a discrete rod model for hair and a particle-in-cell model for fluids. To treat the thin layer of liquid that clings to the hair, we augment each hair strand with a height field representation. Our contribution is to develop the necessary physical and numerical models to evolve this new system and the interactions among its components. We develop a new reduced-dimensional liquid model to solve the motion of the liquid along the length of each hair, while accounting for its moving reference frame and influence on the hair dynamics. We derive a faithful model for surface tension-induced cohesion effects between adjacent hairs, based on the geometry of the liquid bridges that connect them. We adopt an empirically-validated drag model to treat the effects of coarse-scale interactions between hair and surrounding fluid, and propose new volume-conserving dripping and absorption strategies to transfer liquid between the reduced and particle-in-cell liquid representations. The synthesis of these techniques yields an effective wet hair simulator, which we use to animate hair flipping, an animal shaking itself dry, a spinning car wash roller brush dunked in liquid, and intricate hair coalescence effects, among several additional scenarios.

Interactive Acoustic Transfer Approximation for Modal Sound
ACM Transactions on Graphics (SIGGRAPH North America), 2016
Dingzeyu Li, Yun Fei, Changxi Zheng
Abstract | Preprint: Compressed, Regular | ACM Digital Library | Video: YouTube, bilibili | Fast Forward | Project Page
simulation

Current linear modal sound models are tightly coupled with their frequency content. Both the modal vibration of object surfaces and the resulting sound radiation depend on the vibration frequency. Whenever the user tweaks modal parameters to adjust frequencies the modal sound model changes completely, necessitating expensive recomputation of modal vibration and sound radiation.

We propose a new method for interactive and continuous editing as well as exploration of modal sound parameters. We start by sampling a number of key points around a vibrating object, and then devise a compact, low-memory representation of frequency-varying acoustic transfer values at each key point using Prony series. We efficiently precompute these series using an adaptive frequency sweeping algorithm and volume-velocity-preserving mesh simplification. At runtime, we approximate acoustic transfer values using standard multipole expansions. Given user-specified modal frequencies, we solve a small least-squares system to estimate the expansion coefficients, and thereby quickly compute the resulting sound pressure value at arbitrary listening locations. We demonstrate the numerical accuracy, the runtime performance of our method on a set of comparisons and examples, and evaluate sound quality with user perception studies.

Computational Design of Metallophone Contact Sounds
ACM Transactions on Graphics (SIGGRAPH Asia), 2015
Gaurav Bharaj, David Levin, James Tompkin, Yun Fei, Hanspeter Pfister, Wojciech Matusik, Changxi Zheng
Abstract | Preprint | ACM Digital Library | Supplementary | Video: YouTube, bilibili | Project page | Industrial Reproduction | In Press: Science 360, Wired, Popular Science, Gizmodo, Inverse, Columbia Engineering, Science Daily, Gizmag, TechRadar, 3DFab+Print
simulation fabrication

Metallophones such as glockenspiels produce sounds in response to contact. Building these instruments is a complicated process, limiting their shapes to well-understood designs such as bars. We automatically optimize the shape of arbitrary 2D and 3D objects through deformation and perforation to produce sounds when struck which match user-supplied frequency and amplitude spectra. This optimization requires navigating a complex energy landscape, for which we develop Latin Complement Sampling to both speed up finding minima and provide probabilistic bounds on landscape exploration. Our method produces instruments which perform similarly to those that have been professionally-manufactured, while also expanding the scope of shape and sound that can be realized, e.g., single object chords. Furthermore, we can optimize sound spectra to create overtones and to dampen specific frequencies. Thus our technique allows even novices to design metallophones with unique sound and appearance.

On the Hessian of Shape Matching Energy
Technical Report, 2016
Yun Fei
ArXiv Page
simulation

SmartGuide: Towards Single-image Building Localization with Smartphone
Proceedings of MobiHoc 2015
Xi Xiong, Yang Zheng, Longfei Shangguan, Yun Fei, Miloš Stojmenović, Yunhao Liu
ACM Digital Library
mobile computing

Parallel L-BFGS-B algorithm on GPU
Computers and Graphics, 2014
Yun Fei, Guodong Rong, Bin Wang, Wenping Wang
Abstract | ScienceDirect | Preprint | Github
GPU optimization

Due to the rapid advance of general-purpose graphics processing unit (GPU), it is an active research topic to study performance improvement of non-linear optimization with parallel implementation on GPU, as attested by the much research on parallel implementation of relatively simple optimization methods, such as the conjugate gradient method. We study in this context the L-BFGS-B method, or the limited memory Broyden–Fletcher–Goldfarb–Shanno with boundaries, which is a sophisticated yet efficient optimization method widely used in computer graphics as well as general scientific computation. By analyzing and resolving the inherent dependencies of some of its search steps, we propose an efficient GPU-based parallel implementation of L-BFGS-B on the GPU. We justify our design decisions and demonstrate significant speed-up by our parallel implementation in solving the centroidal Voronoi tessellation (CVT) problem as well as some typical computing problems.

Towards Photo Watercolorization with Artistic Verisimilitude
IEEE Transactions on Visualization and Computer Graphics, 2014
Miaoyi Wang, Bin Wang, Yun Fei, Kang-Lai Qian, Wenping Wang, Jiating Chen, Jun-Hai Yong
Abstract | Preprint | IEEE Xplore
image rendering

We present a novel artistic-verisimilitude driven system for watercolor rendering of images and photos. Our system achieves realistic simulation of a set of important characteristics of watercolor paintings that have not been well implemented before. Specifically, we designed several image filters to achieve: 1) watercolor-specified color transferring; 2) saliency-based level-of-detail drawing; 3) hand tremor effect due to human neural noise; and 4) an artistically controlled wet-in-wet effect in the border regions of different wet pigments. A user study indicates that our method can produce watercolor results of artistic verisimilitude better than previous filter-based or physical-based methods. Furthermore, our algorithm is efficient and can easily be parallelized, making it suitable for interactive image watercolorization.

Bilateral Blue Noise Sampling
ACM Transactions on Graphics (SIGGRAPH Asia), 2013
Jiating Chen*, Xiaoyin Ge*, Li-Yi Wei, Bin Wang, Yusu Wang, Huamin Wang, Yun Fei, Kang-Lai Qian, Jun-Hai Yong, Wenping Wang (*Joint 1st authors)
Abstract | Preprint, Compressed (1.9MB) , Regular (23.0MB) | Video: YouTube, bilibili | Image Stippling | Saber | GitHub | Project Page
geometry rendering

Blue noise sampling is an important component in many graphics applications, but existing techniques consider mainly the spatial positions of samples, making them less effective when handling problems with non-spatial features. Examples include biological distribution in which plant spacing is influenced by non-positional factors such as tree type and size, photon mapping in which photon flux and direction are not a direct function of the attached surface, and point cloud sampling in which the underlying surface is unknown a priori. These scenarios can benefit from blue noise sample distributions, but cannot be adequately handled by prior art.

Inspired by bilateral filtering, we propose a bilateral blue noise sampling strategy. Our key idea is a general formulation to modulate the traditional sample distance measures, which are determined by sample position in spatial domain, with a similarity measure that considers arbitrary per sample attributes. This modulation leads to the notion of bilateral blue noise whose properties are influenced by not only the uniformity of the sample positions but also the similarity of the sample attributes. We describe how to incorporate our modulation into various sample analysis and synthesis methods, and demonstrate applications in object distribution, photon density estimation, and point cloud sub-sampling.

Point-Tessellated Voxelization
Proceedings of Graphics Interface (GI), 2012
Yun Fei, Bin Wang, and Jiating Chen
Abstract | ACM digital library | Preprint | Video: YouTube, bilibili | Source (Direct3D)
GPU geometry rendering

Applications such as shape matching, visibility processing, rapid manufacturing, and 360 degree display usually require the generation of a voxel representation from a triangle mesh interactively or in real-time. In this paper, we describe a novel framework that uses the hardware tessellation support on the graphics processing unit (GPU) for surface voxelization. To generate gap-free voxelization results with superior performance, our framework uses three stages: triangle subdivision, point generation, and point injection. For even higher temporal efficiency we introduce PN-triangles and displacement mapping to voxelize meshes with rugged surfaces in high resolution.

Our framework can be implemented with simple shader programming, making it readily applicable to a number of real-time applications where both development and runtime efficiencies are of concern.

Fast multi-image-based photon tracing with grid-based gathering
SIGGRAPH Poster, 2012
Yun Fei, Bin Wang
Brief Report | Poster | Technical Report | YouTube
GPU rendering

FILM CREDIT

Alita: Battle Angel
Research & Development. Directed by Robert Rodriguez. 2019; 20th Century Fox, Lightstorm Entertainment and Troublemaker Studios.

DEMO & SHORT FILMS

Affine Particle-in-Cell in 2D
Educational Demo
GitHub | Online Demo

4 Minutes for 40 Years: A Brief Review of Modern Computer Graphics
YouTube

GALLERY