Experiential Design - Final Project

 Name : Ivy Chung Ai Shin

Student ID : 0358429

Course : Bachelor Of Computer Science (Hons) 

Week 11 - Week 15

Final Project 

As for the final task , we will be required to complete our project. Therefore , continuing from the prototype phase , I continued on improving the MVP of the prototype. In this stage , I plan to add a 3D model hand that will imitate the hand of a user touching the car engine parts that will be based on camera raycasting method. I also plan to add tutorial panels that will display a tutorial panel for each engine parts.

Image 1.0 - Hand Model Asset

For the 3D hand model , I had looked up for free source from the internet. However , I encountered some problems with the models having issue with the textures. Therefore , after finding some solutions online , I have found the right solution to fix the problem.

Image 1.1 - Hand Model Implementation

. I just had to create a new material and attach the texture into the material. I then drag and drop the material onto the hand model to show its texture. 

Image 1.2 - Rescale Hand Model

The hand model is rescaled into a viewable position and this is how it will look like. 

Moving on , I plan to add the tutorial panel for each of the engine parts. This panel will show the inspection description for the engine parts as user clicks on the name of the engine parts. 

Image 1.3 - Engine Parts Images 

I had downloaded the realistic images for each of the parts. Initially , I wanted to add 3D models for each parts , however , I failed to find 3D models of them. Therefore , as of now , I will put images and may consider doing 3D models on my own instead when there is sufficient time.

Image 1.4 - Tutorial Panel Implenetation

Here , I added the title , the description and the image onto the panel. As for the panel , I employed the UI that was downloaded earlier.

Image 1.5 - Tutorial Panel Adjustment

Then , I added into a canvas and proceed to write the script to allow it to be displayed as user clicks on the label name of the parts.

Image 1.6 - Circular UI Animation

I also wanted to enhance the design and user experience. Thus , I added animation on the circle UI that will allow it to rotate (like the ironman movie XD). 

Therefore , it ends up looking like this :

Next , I want to add warning signs when users move and direct its camera towards the certain engine parts. Meanwhile , the 3D model hand will also move towards the car engine mimicking the real scenario of user trying to touch the engine part.

Image 1.7 - Warning Panel

I started off with creating the warning panels and it ended up looking like this. 

Image 1.8 - Camera Raycasting

I then created the camera raycasting script as taught by Mr Razif and attached it to the AR camera. Therefore , as user directs the camera towards the certain engine part , the warning signal will show.

Initially , I added the cube as a test object because I thought that the raycasting could not detect my mesh object instead. 

After knowing that the camera raycasting worked , I proceed to write the script for the 3D model hand to move towards the engine part.

 

First Test : 

It worked but the 3D model hand was too big , so I decided to resize it.

Second Test : 

Now , it looks better. Therefore , I went with this size and continued to work for the other engine parts.

Image 1.9 - Warning Panels Variations

Different engine parts will show different signals such that some engine parts should not be touched when hot while some only requires inspection. Thus , I had created different display for each of them using three symbols of warning , inspection , and do not signals. This will enhance a more realistic experience from a user's perspective.

Image 2.0 - Symbols Assets

The final result : 

Most of the panels are downloaded that are of SCI-FI theme from the unity store. 

Image 2.1 - Sci-FI UI Assets

Continuing on , I would want to change the engine parts into 3D models to enhance visual aspect. Therefore , I went to search for some free 3D model engine parts. However , I could only find for car battery , car radiator , and car engine. As for the others , I will be modelling out on my own using blender.

Image 2.2 - OilDipstick 3D Model

This is the oil dipstick that I have modelled out on my own. 

Image 2.3 - Steering Liquid Reservoir 3D Model

This is the power steering oil model. 

Image 2.4 - Radiator Reservoir 3D Model

Lastly , this is the radiator tank reservoir. I then exported all of the model from blender and implement into unity.

These are all the 3D model implemented into the app. 

Engine Parts : 

Image 2.5 - Rotation Implementation

I wanted to create a 3D model that users can rotate freely to see the real model of the engine parts. However , I had no idea on how I could make it at first . Therefore , I had figured out after watching a tutorial video online following the source below. 

I also proceed to render the 3D models onto the tutorial panels and added some animation effect on the enter and exit of the panel.

Image 2.6 - 3D Car Model Implementation

To further enhance user experience and the visual engagement , I also added a 3D car model downloaded online on the "Home" screen where it showcases the car model of the user. Thus , user are able to view their car model at a 360 view.

After successful rounds of testing and debugging , I have just adjusted the canvas and panels to fit within the screen to ensure it is big enough for users to view. Other than that , everything works just fine.

App Scenes : 

OnBoarding Screen 

Image 3.0 - OnBoarding Screen

Home Screen 

Image 3.1 - Home Screen

Main Screen 

Image 3.2 - Main Screen

Inspect Screen 

Image 3.3 - Inspect Screen

ARScene Screen (MVP)

• Car Inspection
• Tutorial Panels
• Warning Panels

Image 3.4 - AR Scene

Image 3.5 - AR Scene (Tutorial)

Image 3.6 - AR Scene (Warning)

After realising that Mr Razif will not be able to employ my application as it only works on my car. Hence , I then implemented the Vuforia session recording which will allow the recording to playback as the application runs.

Image 4.0 - Session Recording

Final Presentation : 

Full App ShowCase: 

Reflection : 

Reflecting on my first AR application journey , I have encountered numerous ups and downs along the way. I'm happy to share my thoughts and opinions on the end result and potential future improvements. Overall, I consider the final product to be a success as the outcome was what I have expected , incorporating challenges without becoming overly difficult to the point of frustration. However , there are definitely space for improvements and further implementation that could be done. 

One area where I see potential for growth is in the animation for the tutorial part. Initially , I wanted to create an animation showcase on how users can perform inspection on each engine part. However , due to the time constraint and also lack of knowledge in animation , I was not able to execute it. Therefore , I find alternative on how I could do to make this work nonetheless. With that , I implemented using my knowledge and skills in 3D modelling instead. I had also proceed to modelled out most of the engine parts on my own to be implemented into the app that will enhance the visual aspect overall. 

The biggest challenge that I have faced during my experience building this app was trying to learn how to extract my car engine using polycam and also model out each of the engine part. One downfall I have faced was when I experienced a crash in my application due to corruption in my file. I soon realise that the problem was when I tried using a library package from unity also known as the Vuforia VFX library.  Unfornuately , I failed to resolve it and had to redo the application again. I definitely felt frustrated in the beginning but learned to deal it with patience and resistance. 

I am also thankful for Mr Razif in always supporting and helping me in the development process. He had given me great advice and feedback on how I could make improvement into my application. I am also grateful for my classmate in cheering for me and consistently checking in for me when I had experienced crash in my application. 

All in all , I have learned so much in developing my very first AR application. I have learned on how to use polycam 3D and also model target generator in bringing real objects into a model. I have also learned how to overcome problems and obstacles by staying resilient and patient.