Despite many past efforts to develop haptic experiences of virtual clay modeling, natural interaction with virtual clay remains a hard challenge. In this work, we propose a computational solution for the interactive simulation of clay-like materials with unprecedented realism, coupled with free-air tactile rendering that provides a natural tangible experience. Our solution includes a novel particle-based model of viscoplasticity for efficient interactive simulation, and an optimization-based ultrasound rendering algorithm that takes as input the interaction forces between a virtual hand model and the clay-like material. We demonstrate the effectiveness of our method through expressive creative experiences.
@InProceedings\{BTO21,
author = "Barreiro, Héctor and Torres, Joan and Otaduy, Miguel A.",
title = "Natural Tactile Interaction with Virtual Clay",
booktitle = "Proc. of World Haptics Conference",
year = "2021",
url = "http://gmrv.es/Publications/2021/BTO21"
}
One of the ambitions of virtual reality (VR) is to let people create 3D forms without the constraints of real-world objects, materials and procedures. While VR provides commodity immersive display and hand, simulation models and interaction techniques have not reached the maturity necessary for virtually modeling complex materials such as clay in a natural way.
In this work, we present a simulation and interaction model that enables natural and tangible VR-based modeling of clay. For this end, we present a particle-based model, following the position-based dynamics (PBD) and position-based fluids (PBF) methods, that captures well and efficiently the main features of clay-like materials. We couple this model with existing natural hand simulation model to achieve bidirectional coupling, and use the resulting interaction forces to command a ultrasound-based tactile rendering algorithm.
We characterize clay as an incompressible extremely viscoplastic material. To apply the PBF algorithm to highly viscoplastic materials such as clay, we propose novel constraint formulations that capture the major effects of the material. First, we model viscosity by expressing a constraint on strain rate. To this end, we must turn the velocity-based formulation into position-based constraints. Second, we model elastoplasticity using semi-permanent distance constraints. We achieve plasticity by integrating a hysteresis threshold on the (de)activation of the constraints. And third, we model frictional contact using anchor constraints. All in all, these constraint formulations produce the characteristic clay behavior of extreme viscoplasticity.
Figure 1: Ablation study of the novel constraints in our PBF clay model. We drop a block of clay on an incline, and we show its deformation during the impact (top) and one second later (bottom). From left to right: our full clay model (green); without viscosity constraints, the material flows fast and fractures (red); without elastoplasticity constraints, the material drifts (blue); and without friction constraints the block slides (magenta).
Our ultrasound rendering algorithm follows the clustering-based amplitude-modulation approach of Barreiro et al. to render the interaction of the user with the clay material. However, there are two significant differences in our approach. First, the target pressure field is extracted from the contact forces exerted on ghost particles representing the user's hand. Second, we add a perceptual weight map to the clustering algorithm to favor higher accuracy on more sensitive areas of the hand.
Figure 2: Our rendering algorithm supports perceptual weight maps to favor higher accuracy on more sensitive areas of the hand, such as the finger pads. On the left, we show particles color-coded according to their target pressure; on the right, we show the weight map. The two optimizations indicate the reconstructed pressure with and without weight map.
We demonstrate the effectiveness of our simulation model and rendering algorithm on several examples of creative experiences with complex and rich clay material.
Figure 3: Scenes demonstrating natural user interaction with our clay model. (left) A kindergarten table with colorful blocks of modeling clay. The user deforms, splits and merges the blocks as shown in the images and the video. (right) A potter's wheel with a block of clay. In this case, the user interacts with both hands to create a solid of revolution as the wheel rotates. Both scenes demonstrate the robustness of the clay-like material, which exhibits the extreme viscoplasticity of real-world clay.