When I first watched this video of Daniel Franke & Cedric Kiefer’s soundsculpture project, I was fascinated and watched it again several times.
Not for me to try to breakdown to you how this is done as it is quite complex. I’ll invite you to read the complete explanation from Creative Applications Network. In short, you are looking at a moving sand sculpture from the recorded motion data of a real person.