Game “Cyber Dance” Group Project

Purpose of the Piece

Introduction to the project
Combination of technology and art

• Immersive experience for the audience
• Flexibility and creativity in design
• Increased accessibility for remote viewers
• Interactive and engaging elements

The purpose of VR dance performance is to provide an immersive and engaging experience for the audience, where they can feel like they are present in the performance and interact with the dancers and the environment in real-time.

Enhancing the audience’s experience:

VR technology allows the audience to feel like they are a part of the dance performance and to see it from different angles, providing a more immersive and engaging experience.

Providing new creative opportunities for choreographers:

VR technology enables choreographers to create new dance movements and patterns that would not be possible in traditional performances. For example, dancing in mid-air. Doing movements that obey the gravity law.

Artistic expression:

This project can be a form of artistic expression, allowing dancers and choreographers to convey their ideas and emotions through movement and animation.

Increasing accessibility to dance performances:

VR technology allows for remote viewing, making dance performances accessible to a wider audience that may not be able to attend in-person performances.

Experimenting with different virtual environments:

VR technology allows for the creation of unique and imaginative virtual environments, providing new creative opportunities for choreographers and enhancing the overall dance performance.

This project provides a unique opportunity to combine technology and art, creating a new and exciting way to experience dance.

Creative Process

Process of creating an animated dance performance

Concept and planning:

Before everything started, our group decided to create a VR video dance performance including three different scenes. Each scenes would have a different theme where the VR students decides. I decided to create a scene base on Neon cyborg theme.

Motion Capture Dance move:

I recorded the dance performance using motion capture technology, capturing data on the movements of the dancers.

Conceptualize the environment:

The first step is to decide on the type of environment in which the cyborgs will exist. I decided to build a Cyber Street view as the platform where the dancer would dance on. I got some ideas on what elements and shaders should be included in a cyber scene from the above videos.

Collecting models (character Design):

Then I started collecting prefabs of models. I found two stylish humanoid cyborgs models on sketchfab and decided to use as characters for the two dancers. I also found some usable cyber street view prefabs as the material for building the environment. I also changed some of the textures for example the texture for the goldfish I changed to a hologram type of shader, so it blends into the environment.

In this neon light scene, I tried to make my characters truly unique. To achieve this, I decided to give them masks and wings. These additions gave the dancers an air of mystery and enchantment. The masks became a reflection of their inner selves, expressing a variety of emotions and identities. Meanwhile, the wings add a touch of ethereal grace, symbolizing freedom and transcendence.

Rigging & Animation:

After importing the humanoid model in to maya. I started adding joints and controls. Joints need to be added to create a skeleton that will control the movement of the model. Once the joints and controls are set up, I “skinned” the model to the joints. This involves binding the vertices of the model to the joints so that when the joints move, the skin moves with them. Then I bind the animations created previously by the motion capture Tecnology. Then I started tweaking the timing, easing in and out of movements, and adding secondary motion such as overlapping action, squash and stretch to make the dance move look more natural.

Environment Build Up:

After collecting environment prefabs, I inserted them into unity and started building the street scene.

Texture & Lighting:

The prefab I downloaded for the environment doesn’t includes textures. therefore, I created textures for the buildings and neon lights. I also created a hologram shader for the goldfish. For the lightings, I added three main directional lightings using three colours to create a cyber vibe (Shown in above images). For decoration, I added emission lights to background shops. This could really enhance the theme of a cyber city.

The colors and materials for this project were thoughtfully chosen to create a visually appealing and immersive experience. Vibrant colors were chosen to evoke the energy and passion of the dance, while metallic and reflective materials added a futuristic and technical feel. The textures of the materials were carefully curated to provide a tactile and multi-sensory experience for the audience. These choices were designed to enhance the emotional and visual impact of the project, immersing the audience in a virtual space for a mesmerizing dance experience.

Camera Movement:

The camera transitions in my scenes were a mesmerizing element that enhanced the visual storytelling. Through fluid turns, movements and zooms, it captured the essence of the dancers’ movements and directed the audience’s attention. These deliberate perspective shifts added depth, emphasized highlights, and created an immersive viewing experience that showcased the dancers’ skill and passion.

Connection between scenes (water scene to neon scene):

As the scene change from the water scene to my neon scene, the raindrop system plays a key role in connecting the two environments. The glittering raindrops fall in layers, adding a dynamic, immersive element to the scene. The dancers’ movements are gracefully synchronized with the falling raindrops, creating a harmonious synergy between the natural elements and human expression.

Sound:

The song we chose is a pure music with no lyrics which gives more varieties in choreography. It is a light rhythm song which the dynamic rhythm creates many possibilities for visual creation that could fit in to all three scene themes.

Tool Used

Visual and Choreographic tools

• Motion Capture
  • In this step, I captured the movements of the dancer using motion capture technology. This involves setting up cameras and sensors to record the movements of the performer, and then processing the data to create a motion capture file.
• Maya
  • Once the motion capture data has been captured, the next step is to import it into Maya and use it to create a rig for the character. This involves creating joints and controls that allow me to manipulate the character’s movements. I am able to add additional keyframes and refining the animation.
• Unity
  • I used this software to create the environment of the scene and adding lighting and camera for the final render showcase.

The user/spectator journey

The user/spectator journey of this project begins with a viewing experience of the final animation. As they watch the animation, they are immediately drawn in by the fluidity and realism of the character’s movements, which have been captured through motion capture technology.

As the animation progresses, the viewer is then immersed into the environment which is built up in Unity to create a visually stunning backdrop for the character’s movements. The lighting creates a mood and atmosphere that enhances the emotion and impact of the animation.

Overall, this project is designed to be a captivating and immersive experience that showcases the power and potential of motion capture technology to create stunning and emotionally expressive animations within a virtual environment.

What the product shall looks like

Showcase

Required Equipment

Visual and Choreographic tools

• Motion capture technology: This is used to capture the movements of dancers.
• 3D animation software: This is used to create a rig for the character and to apply the motion capture data to create the animation.
• Unity game engine: This is used to import the character’s 3D model and to create the virtual environment.
• Computer hardware: This includes a powerful computer with a dedicated graphics card to handle the processing and rendering of the animation.
• VR headset: Use to experience the performance in virtual world.

Critical Appraisal

• Introduction
• In general, our group created an animated dance performance taking place in three theme scenes. We included the elements of geometric, water, and Neon. For the first scene (Geometric), it contains technical and angular movement, accentuating the lines of the body. The second scene (Water) has a smooth and fluid movement. Exploration of body and water. For my scene the final Scene (Cyberpunk) uses personification of neon lights. Embodiment of coloured light and emission.
• It is purposely made as a platform for dancers to perform in a way different than reality by including elements cannot achieve in real world. It creates a more immersive experience for the audience and also provides a new creative opportunity for choreographers.

2. Animation Quality

• While the final animation successfully demonstrates the skills of the dancers and the artistic expression potential of VR technology, there are certain aspects of the animation that we could improve. For example, the transitions between dance moves could have been smoother by adding more intermediate frames between key poses. We could also pay more attention to the details of the environment, for example, by adding more props and background elements to make the scenes more immersive. There are also some technical issues, such as one of the character’s hand movements not being accurately captured by the motion capture technology and some movements going through the model, which can be addressed through further testing and refinement. By improving these aspects, we can improve the quality of the animation and make it more attractive and immersive to the audience.

3. Choreography (Do Better) (examples and evidence)

• Although the final animation successfully showcased the dancers’ skills, there were certain aspects of the choreography that we could have improved to make it more dynamic and engaging. First, we could increase the variety of dance movements by incorporating a wider range of movements, including more complex and challenging routines. This would give the dancers more opportunities to showcase their skills and make the animation more interesting to the audience. In addition, we can focus more on the narrative nature of the performance, using dance to tell a story and convey emotion. By adding more storytelling elements, we can make the animation more engaging and relatable. Another area for improvement is the use of space; we could explore more creative ways to use the virtual environment to create unique and visually stimulating dance sequences. Finally, we could pay more attention to the rhythm and rhyme of the dance to ensure it matches the music and creates a cohesive performance.

4. Music and Sound Effects (Do Better) (examples and evidence)

• First, we can use a wider variety of music to create more interest and variety in the performance. This could include experimenting with different genres, rhythms and moods to create a more diverse soundscape that keeps the audience engaged. In addition, we can improve the synchronization between the music and the dance movements to ensure that the two are more closely aligned, allowing the music to enhance the performance. Another area for improvement was the use of sound effects to create a more immersive experience. We could have added more ambient sounds to create a sense of place, such as footsteps, breathing, or other ambient noises, to create a more complete virtual environment. Finally, we could have experimented more with spatial audio, where the sound changes depending on the viewer’s position in the virtual space, creating a more dynamic and realistic audio experience.

5. Visuals (Do Better) (examples and evidence)

• For the visual, I could use more lighting and colour to create a more vibrant and visually appealing experience. For example, I could experiment with different lighting techniques such as coloured filters, spotlights to highlight different areas of the scene and create more dramatic effects. In addition, I could add more visual effects, such as particle systems or motion blur, to add depth and movement to the animation. Another area that needs improvement is the textures used in my model. I could use a more high-quality textures and more detail to make the model look more realistic and appealing. Finally, I could pay more attention to the overall composition of the scene. I could experiment with different camera angles and perspectives to create a more interesting and dynamic shot and used more techniques such as depth of field or framing to direct the viewer’s attention to key areas of the scene.

9. User Feedback (Do Better)

• Some testers noted that they experienced motion sickness or discomfort while using the VR experience. This may be due to factors such as camera movement or the intensity of animations. To address this issue, in future iterations I will adjusting the camera movement or animations to make users more comfortable or providing users with the option to adjust the intensity of the experience to their preferences.

10.difficulties I faced

AI Motion Capture Limitation:

When I first used the AI Motion Capture which only require a video shot of the dance. The dance move become really Stiff and unnatural. The detect of human rig can roughly show the movement with a lot of unnatural joint twists.

model Prefab with no texture:

The prefab I downloaded for the environment doesn’t includes textures. Therefore, I have to create textures for the buildings manually which took me a lot of time to create and assign it each by each.

Humanoid cyborgs not in T-Pose:

The two humanoid prefab I found for the dancers are not in T-Pose as an original form. Therefore, I have to fix and stretch the position manually which took me a long time to manage that. I also didn’t work at the first place when I uploaded the meth to mixamo. The software didn’t recognize the mesh as humanoid. In this case I have to redo the fixing step until it finally worked.

Difficulties to assign bones & joints:

Since I am not familiarity with assigning bones and joints it took me a lot of time and assistance figuring this progress out. Also the maya in my laptop can’t open the files.

Camera movement unused:

At first, we were intended to create a video as the final submission. but when I tried assigning movements to the camera the script doesn’t work so I tried using multi-angle to create a final video. I then realize that the outcome didn’t match what I expected. So we decided to make it a VR experience.

10. Collaboration and Teamwork (What did I learn from the collaboration)

• My work with two dance school students taught me the importance of communication, patience and collaboration in achieving a successful project. We worked closely together, shared feedback, and made decisions as a team. Working with people from different backgrounds and skills has helped me develop interpersonal skills and appreciate the value of teamwork in achieving our common goals. I learned how to effectively incorporate feedback and communicate my ideas. Overall, I am grateful for the opportunity to work with them in creating this successful project.

Additional Add In

After Feed Back

• Why I choose the elements (colours and materials)
  • Read under “Creative Process” “Texture & Lighting”
• Character more different from download (adding mask and wings)
  • Read under “Creative Process” “collecting models (character Design)”
• Camera transition (shake, move, zoom in out, cut)
  • Video Showcase under “what the product shall looks like”
  • Read under “Creative Process” “Camera Movement”
• more connection with scenes
  • Video Showcase under “what the product shall looks like”
  • Read under “Creative Process” “Connection between scenes”