Table of Content

> back to top

Overview

Our game is based on the desktop based Plants Vs. Zombies Game by PopCap Games. However, we wanted to implemented it as an augmented reality application so that the user would be able to go around the field and actually grab and place plants with their own hands. This, however created some challenges like UI positioning and design, the composition of hardware and the overall feeling and intuitivity of the 3D game. We therefore experimented with the different components until we reached a satisfying result and also included the matter of intuitivity in our user questionnaire. Following we would like to describe and explain our design decisions in more detail.

Game Loop

Our game loop was mostly given by our inspiration of Plants Vs. Zombies. There and in our game the player has to defend their base with plants against the arriving opponents. In our game, however, we choose the opponents to be insects to give it a brighter and cuter look. The player can hereby choose between different types of plants that will help him/her against the different types of enemies.

To prevent the player from just placing plants infinetly a currency system was implemented. The currency hereby are so called sun points which are given by collecting spawned suns. These suns can either spawn randomly in a low frequency or are produced by sunflowers (one of the plant types) in a higher frequency. Every plant costs a certain amount of sun points and can therefore only be placed when the player collected enough.

Since our board in the game can be placed anywhere in the room, we first give the player the possibility to move the board around and then start the game by pressing the "Start" button. Afterwards, the player has short time period where no enemies will spawn yet and where they can prepare themselves for the arriving opponents. The time period differs due to the difficulty level. During this time it is recommended to collect suns and already place some plants for defending the base. After the time period exceeds, the opponents will be spanwed in an area behind the field and start walking towards one of the lanes. In general different types of opponents have different times they will start spawning. This ensures easier opponents to arrive first. We will later describe the intern code structure in the section Program structure in more detail. The player now tries to defeat them. To make the game more challenging, though, a "wave" system was implemented, meaning that after some time a huge amount of enemies will spawn simultaneously. The exact point of time this happens is decided by the number of enemies that have already been defeated.

To make the plant system more interesting, the player, furthermore, has the possibility to spend sun points on upgrading plants. This will make them more powerful. Depending on the single plant it can for example increase the damage or decrease the time the flower needs to spawn items. The plant to upgrade is selected via an energy drink can. We decided us for this desing because we needed an upgrade possibility that is not conducted by clicking on a plant (since our game should be playable with only your bare hands) but still give us a confirmation possibility. The fluid dropping down on the plants gives this confirmation by "watering" the same plant long enough. In a similar apporach we implemented the possibility to delete plants. Only that in this case the player will not use the energy drink can but a beer can. 

The game ends either when the player managed to defeat all enemies or when even just one enemy broke through to the base. We implemented a progress bar at the side of the board to indicate how many opponents the player still has to fight off. This, however, will also be explained in more detail later.

> back to top

Program structure

The game was structured in accordance to the different components we needed. First, we started implementing the main functionalities with extension possibilities in mind. These were:

• the spawning and placement of the plants
• the spawning of the opponents and their movement
• the interaction between the two above
• the instantiation of a suitable field with all necessary components
• possibilities to select and delet plants

We later on extended them further with:

• the display of a progress bar
• updates for the plants with new behaviour
• pause possibility and management

Since our spawning behaviours would change due to the progress of the game, we soon decided to use several manager classes that would handle the sinlge spawning behaviours and a game manager class through which the time was handled indirectly. With this structure we could easily display and handle changes in the game behaviour and ensure the synchronization of our game data.

The overall structure was planned as follow:

The GameManager class was hereby constructed as a singleton which ensures the synchronization of data between the other classes. It only handles data like the remaining number of enemies or the current amount of sunpoints and checks whether player won or not. it also stores if the game is currently paused which can be looked up by all other classes.

> back to top

Plant Selection and Spawning

The interaction with the field pieces and the plants on them happens via so called selector objects.
A selector object checks via raycast whether there is a field piece below it. At first it selects the field by highlighting it and calls further functions on it, if the selection is confirmed.
The field piece manages the spawning, upgrading and deleting of its plant. This way, the plants themselves only perform their gameplay relevant tasks (described in Plants).
When there is a plant on a field that can be modified with a selector, the field piece also manages the correct display of the Plant UI to show its stats and upgrade/deleting progress.

Selectors

There are two categorys of selector objects: the selection plants and the selection cans. They use different techniques to confirm a selection.

Selection Plants

Those objects are attached to the Hand Menu. They are used to spawn new plants on a field. The confirmation is proximity-based. When the selection plant is close enough to the field, a new plant is spawned on the selected field and the selector is destroyed.

Selection Cans

The cans are ancored to a fixed point above the game field. They are used to upgrade or delete plants. The idea is to give the impression of watering the plants. Therfore the selection is confirmed when holding the can in a specific angle for a period of time. The angle is visualized with a particle effect and the time period with a progress bar above the plant. When not deliberately hold by the player, the can is returned to its anchor

Additionally, when a can is grabbed, the stats of all the plants in game are displayed above them. The upgrade can (Energy drink) displays the upgrade costs and the delete can (Beer) displays the HP.

Enemy Spawning and Behaviour

Our enemies are spawned in the EnemySpawnManager. In this class the points of time the enemies enter, spawn, wait until spawn and respawn are handled. These attributes are handled via an array of spawning areas. Each of these spawning areas defines the following for one group of enemies:

• the enemy type
• the enemy's time to start spawning
• the spawning interval
• the maximum number of enemies of this type
• the position the enemy starts spawning

With this we can easily adapt single enemy groups to get a balanced game result. 

The EnemySpawnManager then handles the states of the spawning phase. It differentiates between three states: waiting, spawning and waveSpawning.

At the beginning the state is set to waiting until the time of the spawning start is reached. Afterwards, the state will be set to spawning and the enemy type will be instantiated with its standard time interval between two enemy istances. After a certain amount of spawned enemies, the state will be set back to waiting until the last already spawned enemy on the field was defeated. This triggers the state to be set to waveSpawning in which a certain amount of enemies will be spawned with a much shorter time interval. Then, the spawning will be paused for a few seconds (the state is set back to waiting) until the normal spawning will be done again.

This can be repeated by defining several waveTimes.

> back to top

Game Elements

Plants

There are 3 Types of plants found in the game:

• Sunflowers: providing energy in form of suns to further plant more plants
• Cactuses: shoot projectiles to damage enemies
• Potatoes: Have more HP to block a certain line and hinder enemy movement

The Sunflower and the Cactus are upgradable with the Energy drink can. Initially both should be upgradable 2 times.
For the final build, we decided to cut out the first upgrade, because the visual difference wasn't strong enough when the game got more complex.
Furthermore, by upgrading directly to the second level the mechanic feels more powerful and should encourage the player to use it more.

The potatoes are not upgradable due to their function as a non permanent barrier.

Sunflower

The sunflower generates energy by spawning a new sun after a specific amount of time. The spawned sun has a different color than the ones naturally spawned.

Visuals

Cactus

The cactus shoots projectiles after a fixed amount of time. The upgraded version shoots 2 projectiles to the front and 2 additional ones to its left and right from the red flowers.

Visuals

Potato

The potato's purpose is blocking a path until the shooting plant took care of the enemies on that path. There are no upgrade possibilities for it but whenever a potato is spawned on the field, one of 3 different models is chosen to give some variance in its appearance.

Visuals

> back to top

Enemies

We wanted to implement at least two kinds of enemies: a walking one and a flying one. We decided to use an ant as the walking enemy and a wasp as the flying one.

Every enemy has an Enemy.cs script in which the variables used for balancing (like speed, health points (HP) and attack points (AP)) are handled. For the two enemies we created, it was sufficient to only create one script. It can, however be extended and used as a parent class if further behaviours are needed.

To give them a more natural look we created an animation for walking and attacking. 

Ant

The ant enemy type was meant to be the weakest enemy type. It approaches relatively slowly and can be easily defeated with just 5 hits of the not upgraded ammo.

Following the animations for walking and attacking can be seen.

> back to top

Wasp

The wasp enemy type was maent to be a more challenging enemy. It can take up to 7 hits before being defeated and approaches faster. Originally we also gave it more AP but during balancing we noticed that it would have been too strong.

Following the animations for walking and attacking can be seen.

> back to top

Interaction

The interaction of the player's hands with the game environment is implemented using Leap Motion's Interaction Engine. It provides hand representation and recognition for basic hand gestures like grabbing.
Furthernore it has mechanics for basic objects and button interaction, anchors and hand-based UIs. 

> back to top

UI

As it was our goal to create an immersive experience for the player, an important aspect was the design of the user interface. We tried to use as few classical UI elements as possible and tried to bring the ones we needed in a creative way into the game.

Menus

Regular menus only appear when outside of the main game. The main menus and win/lose screens are anchored to the players head, the pause menu for the game is anchored to the corner of the game field.
By grabbing the red sphere under the pause menu panel, the whole field can be repositioned. The game starts with the pause menu, so the player can adjust the field to their surroundings before playing.

Hand Menu

The menu for placing plants and pause the game is attached to the players left hand. The menu appears depending on the angle of the hand to the camera.

Hand Menu: Planting

When the player faces the palm towards the camera, the planting menu is displayed, showing the available plants and amout of suns the player has at disposal.
If the amount of suns is sufficient, the player can interact with the hologram plants by grabbing them. Whenever one of those selection plants is detached from its anchor, a new one spawns.
When the player lets go of the selection plant without placing it on a field, it fades away after a few seconds.

If there are not enough sunpoints, the hologram plants will glow red instead of green and no interaction will be possible.

Hand Menu: Pause

When looking at the back of your hand, the play/pause button is displayed. With this placement we wanted to resemble a watch.
When pressed, the game will pause and the pause menu will appear. The player can continue the game by pressing the play/pause button on their hand or the continue button of the pause menu.

Plant UI

The first intention was to give all feedback about the plants with visual clues. Like showing a severely damaged plant when it has low HP. Unfortunately these clues were not clear enough and we decided that we needed a minimal UI for the plants. 
The UI is only displayed when one of the selection cans (Selector) is grabbed. The design is compact and modular. Each element works on its own as well as combined with others. It is displayed above the plant itself.
It can show the upgrade costs, HP and upgrade/delete progress of the plant.

Enemy Progress Bar

A progress bar on each side of the field shows how many enemies have already been spawned.

UI Shader

Most of the UI elements use custom shaders to make them stand out from the other elements or eneble their function.

• toon shader: selector plants & suncounter "coin" 
• fill shader: used for the HP and enemy progress bar
• textmesh shader: used for all UI text mesher (the standart textmesh in the UI layer, so it appears even on top of objects that are in the foreground) 
• 2 hologram shader: one for the hologram circles around the selector plants and the other for the field and menu anchor

The circular progress bar uses Unity's transparent cutout shader and an alpha grayscale image.

> back to top

Tutorials

To ease the introduction of the different game mechanics, we implemented 3 short tutorials. 

In the first the very basic grabbing mechanic is introduced. The player has only one line to defend and already enough sun points to place a shooting plant. The random sun spawning as well as the sun flower and the barricade flower selector are removed from the scene, so that the player can only grab the shooting plant. The Selector cans are also not present in the first tutorial. Two texts, explaining that the player should look at their left hand and that plants can be grabbed and placed on the field, provide additional help for the player. Furthermore, there will only spawn one single ant enemy in this scene.

In the second tutorial, the sun catching mechanic and the other two plants are introduced (also via a short text). The player starts without sunpoints this time and can choose between all 3 implemented plants. They will have to defend 3 lines against 15 enemies of the same type.

In the third and last tutorial, the upgrade and delete cans are introduced and the player has to defend 5 lines against two different types of enemies.

> back to top

Visual and Audio Feedback

To alter the game experience we used visual as well as audio feedback for the player.

Visual

Plant Upgrade/Delete

In order to give the user a feedback when upgrading or deleting plants, three visual cues are given:

• the selected fieldpiece is highlited
• a progress bar is visible above the plant (Plant UI)
• plant changes appearance

Plant Shader

To give the player feedback when the plants are being upgraded or deleted the plant shader shows some visual effect. 

*the Potato cannot actually be upgraded.

Fight Feedback

It was important to us to give the user visual feedback whenever the plants and insects interact with each other. This includes:

• shooting particle effects
• plants flash red whenever they are attacked
• enemies have particle effects whenever they are hit by an projectile or die

3D Assets

We could finde some basic 3D assets for the plants and cans, but some we had to create ourself.
The insect models and animations you can see in Enemies, the potatos in the Plants section and the suncounter in UI.

Third Party content

Unity Asset Store

Flower Models: https://assetstore.unity.com/packages/3d/vegetation/plants/lowpoly-flowers-47083

Particle effects: https://assetstore.unity.com/packages/vfx/particles/polygonal-s-low-poly-particle-pack-118355

3D Models

Can model: https://www.turbosquid.com/de/FullPreview/Index.cfm/ID/974986

Arrow model: https://clara.io/view/942deb6f-a75a-4d08-bf20-4bd8dad0bcbe

> back to top

Audio

To give a more relaxing atmoshpere at the beginning of the game we used some copyright free music as background music. The music, however, will change when the enemies start spawning and will be swiched to Wagner's ride of the valkyries, so that the player attention will be focused on the arriving opponents.

In accordance to the former described visual feedback, we also added audio feeback for certain actions and events. An overview is provided hereinafter.

> back to top

Hardware

Components

Oculus Rift: VR headset

ZED mini: Mixed-Reality camera for displaying the surroundings of the player including depth information.

Leap Motion: Hand tracking via infrared

Leap Placement

After several tries (see Challenges: Set Up) we decided to attach the leap motion onto the occulus directly under the ZED mini. This ensured a constant distance between the leap motion' coordinate system and the occulus' coordinate system. The difference was then compensated by an offset within the unity scene.

> back to top

Challenges

During the implementation we faced two major challenges: The set up and the balancing.

Set Up

To get started and implement the basic functions of the game, we first placed the leap motion on the table. With this we could already start implementing the grabbing mechanics within a desktop application. 

Markerless Registration and Synchronization

When we started to use the occulus with the ZED mini we then wanted to keep the table mounted leap motion, so that we would not have to rearrange the game mechanics. However, we first needed to figure out which vr provider we wanted to use. We started with the standard occulus vr set up. Unfortunately, we discovered that the occulus package was not compatible with the ZED mini and the Leap Motion package at the same time. Two of them were fine but whenever we tried to use all three of them, the package added last overwrote functions of at least one of the other two, causing them to throw several exceptions. 

Because of this, we decided to use the steam vr package since it did not cause these issues.

We therefore tried to register and synchronize the different devices and their coordinate systems with Ubitrack using steam vr. Unfortunately there were two problems that could not be resolved. The first was a steam vr/occulus intern problem. For the calibration process we used the occulus controller and rotated them around a fix point that was then also used as a fix point for the leap coordinate system. The controller, however, were constantly rotated in the steam vr scene which led to a wrongly set fix point and therefore to offset between the occulus system and the leap system. Unfortunately we were not able to resolve this offset.

We later on discovered that due to the second problem, the offset did not matter since the occulus coordinate system was constantly changing its position and rotation when the headmounted device was not detected properly or move around relatively fast. Due to this problem, the coordinate system of the leap motion would have shifted apart from the occulus coordinate system over time anyway.

The second problem was the reason why we decided to attach the leap motion to the occulus head mounted device. This would give us a constant offset between the two that could be calculated and compensated within unity itself. This finally gave us the correct hand representation within unity.

Switching Render Pipelines

The project was initially started using Unity's Lightweight Render Pipeline. Unfortunatelly, when we decided to add the Zed mini, we noticed that we could not further use it.
The shader used in displaying the camera image were not compatible with the newer render pipelines.
We had to downgrade the project, which costed us a few visual effects like post-processing and the some shaders had to be rewritten. Due to time constraints some of the previous effects were not restored, like the "melting" effect of the plant shader when deleting a plant.

> back to top

Balancing

A major challenge in defence games is usually the balancing. We therefore implemented our application with several balancing possibilities in mind. 

These were in detail:

• adaptable HP for plants and enemies
• adaptable AP for plants and enemies
• adaptable upgrade possibilities
• adaptable entry and wave times of enemies
• adaptable spawning and wave spawning times
• easily changeable field

Due to these adaption possibilities we were able to find a balanced game experience in our polishing phase.

> back to top