I will join this project with my Kinect project (link). I will use 3d particles affected by sound and video. One version would also react to persons movement.
Right now I'm trying to research
- how much information I can get from the audio input
- what deformation are possible with Kineme3D
- how to make pixel/area colour/texture detection work
I'm trying to track colour in video input, so I can apply particles of random 3d scrap objects to certain areas (for example black, white or red). These object's movement, scale, deformation and time would be affected by sound input and movement.
I made a quick sketch to illustrate this idea:
