Fusion4D

Back to the top

What is Fusion4D

Fusion4D is an innovative 3D user interface that lets users interact with 3D objects as if the objects were in their hands, allowing them to move, rotate and scale the objects, explode them into its parts, and even navigate in time to see what the objects will look like in the past and future.

Fusion4D is simple to use: the user only has to wear the 3D glasses and use speech commands and hand gestures to manipulate the objects. Besides that, the system uses low cost devices, such as Kinect, and it doesn't require special displays for the 3D images.

The interface was created for Project VIDA – Virtual Interactive Distance Learning on Anatomy, which is developed by Interlab-USP in collaboration with LApIS at EACH/USP and LPAI/Senac.

Back to the top

How it works

How Fusion4D works: Kinect and 3D glasses

Fusion4D uses 3D images and a Kinect device to capture the user's skeleton and voice. Therefore, to interact with the system, all you need to do is wear a pair of glasses and start interacting as if the object were in the real world.

The user interacts with the system using speech commands in English. Those commands allow users to select the object and manipulation mode.

Commands

The "grab" command selects the object and allows users to translate, rotate or scale the object with their hands until the object is released with the "release" command.

The "explode" keyword allows the user to observe the 3D model in details; the "show label" command shows a descriptive label of the objects in the screen, and the "change model" command lets the user to select other models.

Finally, the "time" command allows users to see what the object would look like in the past and future by moving their hands on the timeline. If the user needs help, the "help" keyword shows a list of voice commands.

Back to the top