During an experimental course at WPI our team was given a super open ended project to build whatever we want using the prototyping lab that is available to us. After selling the VR Glove idea to my team of five we hand about three weeks to rapidly prototype the design.
Inspiration – Lucid VR
We took great inspiration for Lucas from Lucid VR as he had come up with the idea of using badge reals to track fingers in a glove. Thanks to his low budget approach and his open-source drivers we were able to quicky iterate upon his design and engineer a force feedback glove using sprockets and solenoids.
Our implementation
We redesigned the badge reel housing and laser cut an acrylic sprocket to attach to it. We attached a solenoid to be used as a locking bar that would fire when the finger needed to stop. The sprocket would catch the solenoid head while the finger was applying pressure, so we only had to power the solenoid once rather than leaving it on.
Assembling the rest of the pieces took a few printings in order to find the right tolerances for the housing.
With about five days left we did not have time to make a PCB so the wiring looks a bit crazy. I also programmed the Arduino with a force feedback piano as a failsafe if we did not get the VR side working in time.
Using divers from Lucid VR helped dramatically as we were able to get the glove working inside of Steam VR with the badge reals without too much trouble. Additionally Lucid VR had already been experimenting with servos to apply force feedback, so refactoring the code to work with solenoids instead turned out to work great!
Whats Next?
With such a short time period there is a lot left to explore. The team is continuing the project through the next academic year. As graduate I will be present to advise their progress and I can’t wait to see where it goes!
The Jazz VR museum was one of the lucky projects to be selected and presented at WPI’s PAX East 2022 Booth!
Jazz History Database (JHDB)
The Jazz History Database is an organization at Worcester Polytechnic Institute (WPI) that stores jazz resources in its archives, including live recordings. The JHDB in collaboration with the Interactive Media and Game Development (IMGD) Department set out to build an interactive 3D environment in which a player can listen to jazz songs and interviews on a record player. We would build this environment for virtual reality (VR), where the player puts on a VR Oculus headset in which they would see and interact with the artifacts in the virtual museum.
Technology
We chose to use the Oculus Quest 2. Aside from having one onhand it is also one of the best plug and play VR headsets currently on the market and makes VR easily accessible. The Oculus can also be plugged into a PC using USB3 which allows for quick testing and a smoother development process.
Unreal Engine 5 was beta released in May of 2021 and it was an ambitious undertaking for this project.
Plugins:
Oculus OpenXR: OpenXR aims to ease VR development across multiple headsets by standardizing the inputs across devices. This project takes advantage of this system but we have yet to test it on headsets other than the Quest 2.
GPU lightmass: Stationary and static lights in the scene can take advantage of the lightmass system by precomputing lighting data and storing the information in lightmap textures. This process usually takes a while to compute; however, this plugin enables the process to be done on the GPU and uses DirectX 12 to speed up the process.
Design
Our initial concept began with brainstorming different spaces the player could visit with each space focusing on a different element of the JHDB. For this project we scoped to focus on one space in particular, the Listening Space, as a proof of concept.
Designing for VR
While we wanted players to browse and play music, using menus in VR can be quite underwhelming or even frustrating. Our solution was to design the environment to be used as an interface and take full advantage of the VR capabilities. While on a website a user may scroll through a list of songs and click on one to select it, in our Listening Space the user browses records on a shelf and can physically pick up a record of their choosing. Additionally the distance of objects from the player also needed to be considered. As This experience was designed to keep the player in a sitting position, the assets in the surrounding environment needed to remain within reach.
Iterations
VR Interaction
Utilizing the Oculus OpenXR and OpenXR plugins to do a lot of the heavy lifting inside the Unreal Engine VR Template, specifically the VRPawn and the GrabComponent Blueprints. The GrabComponent can be attached to any simple objects such as the vinyl record and enables the object to be picked up and thrown by the motion controllers. Additional behavior can then be added to the object for more complex interactions. For example, to create the vinyl records the GrabComponent was attached to a static mesh and a song variable was added so that a specific song could be attached to the object.
Vinyl Folders
The vinyl folder was added to organize and better manage the growing list of songs/records in the space. This blueprint consists of a text label, trigger volume, and placeholder. In addition the VinylFolder requires a reference to the VinylRecord blueprint and for a default song to be selected. The folder can then be placed within the space and on start the blueprint will initialize a vinyl in the location of the placeholder and assign it the selected song. The text label on the side of the folder will always display the name of the song of the last vinyl that it was holding. This means that if a new vinyl is placed into an empty folder the name will update to the correct song.
Skeletal Mesh Interaction
Working with a skeletal model can be really nice as like the Oculus hand models you have access to each bone individually and you can get a lot of complex behavior out of them. For this model the record plate was given a spinning animation while the record head is grabbable. The record player inherits from the GrabComponent but uses a custom grab type to avoid being picked up. Rather it uses the GrabComponent to get access to the MotionController. This way when the player tries to grab the record player the Blueprint sends the MotionController position to the RecordPlayer_AnimBlueprint so that the record head bone can point towards it.
When the vinyl is secured on the plate it will not initially play music, instead to get the vinyl to play the record head must be set in the proper position. Using the object’s forward vector and the record head bone forward vector the cross product can be used to find the range in which the head is over the plate. The same process must also be done with the up and right vectors to make sure the height and quadrant respectively are also within the desired range. A fun trick is to then use the possible forward vector cross product range to select the starting time of the song. As a result the record player will function very similarly to a real record player where the location of the head on the outside of the track will be the start of the song and the inside will be the end of the song.
VR Hand Animation
The Oculus hands are rigged and can be used to make multiple poses using Unreals animation sequence. Combined with a 1D blendspace you can interpolate between the poses with a 0-1 value. For our purpose this creates very satisfying feedback between the amount the player pulls on the grip of the oculus controller and the amount that the hand model closes. This simple system could be expanded into a 3D blendspace with multiple inputs for a variety of poses and in future iterations could make for some unique interactions and experiences.
8ft tall robot, hanging from the ceiling, filled with water, plus RGB lights? Sign me up! Our team of four, designed, constructed, programmed, and composed for this over engineered musical chandelier.
At the start of the Covid-19 pandemic I traveled home safely to Washington State. The school year continued with online classes and I was separated from my team by a 9hr time difference. Even with these unforeseen obstacles we worked together on off hours to ensure a high level of communication and coordination.
Rapid Prototyping
When starting a project, it is important to explore what is
possible. Prototypes are a great way to effectively get your ideas across.
2D side scroller
3D flight sim
3D touch space control
Week 2
Sticking with the 3D flight sim idea from week 1. Began working on orbital mechanics.
Mini solar system
Here we have a small system of only the sun, earth, and
moon. any number of celestial bodies can be added to this system.
Stars planets and moons are on fixed orbits to ensure the player
does not push a planet out of its orbit. However, the objects closest to each celestial
body interact using Newtonian physics.
Orbiting objects
When space objects are spawned, they calculate their initial
velocity to stay in orbit. Afterwards they are free to be interacted with.
spinning rings
Flight Demo
flight demo
Week 3
Updated HUD
With 6 degrees of freedom It is difficult to show the player useful information without overburdening them with numbers. In this demo I am trying out a new HUD element showing the player their current velocities upon their local x, y and z axes.
Satellite Intercept
Simplifying 3D orbital mechanics
It became clear that teaching the player 3D orbital
mechanics may be too much to ask in a museum space game about picking up debris.
Not wanting to leave out the full simulation, giving the player a choice between
normal or simulation difficulties was our next goal.
With this I began trying to simplify the large number of buttons as much as possible.
Eventually landing on one key element Orbital Velocity.
A quick lesson in Orbital Mechanics
Objects in lower orbit maintain a higher velocity to stay in orbit. Objects in higher orbit maintain a lower velocity to stay in orbit. This makes for some counter intuitive mechanics.
Orbital Velocity
As seen in the video, the player in lower orbit has a higher angular velocity than the space debris. when the player increases their Linear velocity, they increase their orbit which reduces the ships angular velocity to match the debris angular velocity.
The best way to simplify this to the player would be to leave them with only one input being their orbit altitude. This way the player in lower orbit can increase their altitude to match the debris angular speed to collect it. Then the player will be able to decrease their altitude, increasing their angular speed to catch up with the next debris piece.
Week 4
Big changes
Testing
With the current build of the game at its multiple difficulty
levels we began testing on various users. Players where enthusiastic about the
game and however the controls where to much of a challenge for many. Even with
the simplification down to just two inputs to change the user’s altitude the
users where confused about how they were supposed to catch up with the debris. With
our focus wanting to be more about space debris and less around 3D orbital
mechanics a large step was taken to change the game.
New controls, new mechanics, new game
The vision for the game changed almost overnight as drastic measures
where taken to simplify the controls and draw out the fun.
new game flight demo
The user once before had access to all six degrees of freedom
as well as full exploration of their environment, is now restricted to an X-Y
plane on a fixed orbit path. This removed all responsibility from the user
about controlling their orbit and trajectories and leaves them to focus on the
task of collecting space debris while avoiding satellites (and soon other obstacles).
Week 5
UI Tweener
Unity has a very in-depth animation system that is a great fit for simple tasks, however a project can get quickly bogged down with multiple animation files. Not to mention that if any of the UI elements have animations on them, unity updates the layout every frame even when nothing is going on.
To avoid using animations and still give UI elements that juicy FX, I created a simple UI tweener using the free tweening library LeanTween.
UI Tweener Component The objectToAnimate is null here as it refers to the object that it is currently attached to, however an object could be responsible for another’s animations if assigned.
This allows for customization over currently four elements (Move, Rotate, Scale, Fade), allows the user to define an ease type (options available from the LeanTween library) and can be stacked to create more complex animations.
The UI Tweener script can also interact with Unity button events allowing for animations to be linked and called easily.
Menus and Scoring
Menus have been added as well as a scoreboard that uses
player prefs to save persistent scores on a device. Here is also a quick flight
demo showcasing the animated UI elements and a change to the targeting/aiming
system.
Week 6
URP
Upgrading the project to us the Universal Render Pipeline (URP),
opens the project to use the post processing stack as well as Shader Graph. Though
no custom shaders have been added yet however, the post processing stack adds a
nice level of polish to game. Even adding a little bit of bloom goes a long
way.
Canvas in World Space
To utilize the cockpit model more than just a nice background the canvas was moved into world space. This allowed for UI elements to be placed over and in the place of buttons and screens.
v4.6.2
Video Player
Unity has a built-in video player that takes care of a lot of the details for you. a small script and few videos made for a quick and easy cinema to play desired videos sutch as the opening intro and the end news report.
Week 7
Assets and Shaders and UI, Oh My!
Short flight demo 4.7.3
Assets
If you pause at (0:44), you can see all the space debris and satellites currently in the game, that are spawned around the earth. A new skybox was also added and lighting was adjusted to help increase visibility.
Shaders
Space debris was difficult to see, and it was difficult to communicate to the player what they should and should not run in to. A quick solution to this was to add a Fresnel shader to surround the debris that the players needed to collect. A shader to the earth was also added, seen in the background with the menus and in game
Input name field
Letter Select
Using the built in Unity Rect Scroll to take care of all the velocity I created a template of RectTransforms to lerp position and scale of the Letter fields. They are still a bit jumpy at the moment.
Stars
Star Animation
Using the UI Tweener, I created earlier creating a star animation system was quick and easy. Just set a tween mode, and delays to appear one after the other.
Week 8
Polish and QA
Audio
An amazing soundtrack and sound FX brought in by our designers completed the experience.
This was a really fun project built in processing. Point and click game with a time limit and multiple puzzles to solve. Switches, Randomly generated pipe puzzle, Keypad.
Free to download, here is our website link if you’re interested
2 week game jam project We build a dance pad for 2 players taking turns in a dance off. The team consisted of 3 artists 2 designers and 1 engineer (myself).
