Hi, my name is Susanne Meerwald-Stadler (hello@xrux.at). I am a PhD student, working as a research fellow at the Center for Human-Computer Interaction (University of Salzburg, Austria).
A central question of my scientific work is how to improve human-robot interaction to make it more intuitive and usable for the end-user.
With an industry partner we started to investigate the potential and applicability of augmented reality (AR) in the area of industrial robot control and programming.
I gather first experiences in AR development and held in the hands my first AR research prototype.
Besides my passion for robots, the excitement of augmented and virtual reality has attracted me ever since.
The deeper I get into the matter, the more fascinated am I by the possibilities these technologies create to enrich our working and daily life.
In March 2017 I started the Udacity Virtual Reality Nanodegree program to improve my development skills.
Currently, this site shows my VR work in the Nanodegree program.
October '17: I visited the AWE'17 in Munich read more
October '17: I visited the VR Summit Salzburg 2017 read more
September '17: I visited the Ars Electronica Festival -- Topic: Artificial Intelligence. read more
August '17: My first lecture on Augmented Reality at ditact '17. read more
August '17: My project "Puzzler" received an VRND Outstanding Project Award :).
July '17: In the course of an intensive acrylic painting workshop week Renate Moran, a well known Austrian artist, designed the decoration of the Google Cardboard V1 for me :).
June '17: My project "Build an Apartment" received an VRND Outstanding Project Award :).
VRND Project 1: Your First VR App
Description
This was a nice and simple starter project to get in touch with VR development.
We got first simple insights into Unity, such as change the text of an object, rearrange objects in the scene (position, rotation) and change the functionality of a button.
Furthermore, we learned how to use the Google VR SDK for Unity and deploy the app on the phone.
At the end we were instructed to clean the unity project and delete all nonessential files.
Screenshots
My Challenges
Preparing for the project: My phone lacks the gyroscope. Now I have a phone with a gyroscope.
The task was to build a new apartment scene with nice lighting and custom animation. Then, deploy it to your Cardboard headset.
It was really fun to realize this project. The easy parts of the project were the basic
apartment layout, the arrangement of the objects and the application of different materials. I had minor problems with
the rotation of the animated globe, but looking in the forum provided a solution. One requirement was that objects
do not intersect with each other. From the task description it was not clear for me how strict this will be handled, so
I wrote a script to identify intersections between objects. I am not sure if it works very accurately, however I found
many intersections, for example the two parts of the chimney, the shelves with the wall, and the clocks with the walls.
A much harder task was to set up the lights such that the ambiance was satisfactory. The correct placement, including
the interplay with point, spot and area lights took its time. I hope with more practice I will find an effective workflow
regarding placing and baking the lights and building the project. To build the project with high baked resolution was
really time consuming. To sum up, a great project to learn which I enjoyed very much.
Video
Screenshots
My Challenges
Globe animation: It was unclear for me that I have to add an empty object (coordinate system, empty object) for the visually pleasing rotation of the sphere in the globe holder.
Looking for suggestions in the forum solved that problem.
Lighting: Positioning of point light on Assets downloaded from unity. The point lights were displayed in the Hierachry but no effect in the scene was noticeable (when baked).
The positioning of the point lights a little bit under the lamp models solved that problem.
VRND Project 3: A Maze
Description
Using the unity editor and primitives, create a little labyrinth. It might be a garden, twisting city streets, caves, or an abstraction - use your imagination.
Whatever you decide, the maze must contain the following elements: waypoint navigation, collectables, the gate, the key, and the UI (user interface).
I was surprised by the Wow-effect after building the maze, when first testing it on the phone with the Google Cardboard.
There is a big difference and an other feeling than on the screen :).
To create the maze I followed a MazeGenerator tutorial and wrote an adapted script.
With the script I was able to generate a perfect maze. I created a prefab and used this as basis for the maze in the main scene.
Than I modified the maze (materials, custom models, JarGenerator.cs) such that the look an feel was satisfactory. Afterwards, I placed the coins and the key and wrote the scripts Key.cs, Coins.cs, Door.cs.
As soon as the key has been collected the door can be opened by clicking on it.
Additionally, I extended the scene with a display showing the number of collected coins and when opening the door how long it took the player to achieve the goal.
Video
Screenshots
My Challenges
The workflow using references in the script, going back to the editor assigning GameObjects, was new to me.
Door and Chest animation: I have a better understanding of the Mecanim System. I started with the door. I created the animation in the Animator. So far so good.
For the door I tried using animations (Legacy).
Chest: I was thinking, that I have an idea what a state machine is, but in practice I had troubles when using a bool as trigger and to reset it.
After a while, my solution was to set back the boolean parameter (to false) in the Macanim back in the Exit transition. Doing this via the script was not working.
Update method: I know that the update method is called every frame.
However, as I worked on the animations, it was difficult for me to identify the problem that the animation was started XXX times in the update method.
I had this problem more than one time. Also with the display play state (collected coins, key).
Workflow for debugging: At the moment I am using prints in the scripts.
VRND Project 4: PUZZLER
Project
In the fourth assignment, we developed a VR Puzzler game, targeting our self-chosen user group.
I have chosen elderly adults.
Beside the fact that I have a lot of seniors in my personal surrounding, this decision was based on three considerations:
More and more works describe the potential of VR for elderly adults:
to counteract isolation, to support the therapy of Alzheimer and dementia patients, and distraction from pain [2].
Elderly are the fastest growing segment of the population [1].
VR research and applications have yet to discover this user group [1].
I call this game the ``Mnemonic Puzzler''.
Mnemonic Puzzler
The Mnemonic Puzzler VR game can be used by elderly adults to train their short-term memory.
According to the findings of work done by researchers at Stanford, virtual reality simulations have a direct impact on how people behave in the real world, even after
they took off their headsets [1].
Video
Screenshot
Design Process
Statement of Purpose: Puzzler is a mobile VR application for first time VR users (60+ years) which challenges them to solve a familiar type of puzzle in a new way.
Within this section I describe the steps in the development process of the Mnemonic Puzzler.
An overview of the whole process is given in Fig. 3, including the persona development, the sketching, VR prototyping and the user testing.
Figure 3: Steps in the development process
Persona
At the beginning of the Puzzler project was the question who will use this VR game.
As end-user group I have chosen people with the age of 60+ years, without any or little VR pre-experience and knowledge.
Further, I focus on elderly without heavy physical or visual impairments. The resulting persona is
based on fictional beliefs and observations of seniors. In the following you will be introduced to Helmut Maderer, depicted in Fig. (6),
a senior in the prime of his life.
Figure 6: Persona
Sketches
I made first sketches during the “VR Design” lessons, see Fig. 4 (a & b), but without a user group in mind. After deciding
on seniors (60+ years) as “the” user group for my Puzzler game, I searched for design guidelines and design aspects
for seniors. An interview (see section user testing) with a 66 years old woman helped to make up my mind in terms
of room size, brightness of the room and orbs arrangement. In Fig. 4 (c) you can see the adapted sketch which was the
basis for the first prototype testing.
Figure 4 (a): Puzzler sketch 1
Figure 4 (b): Puzzler sketch 2
Figure 4 (c): Puzzler sketch 3
I sketched the UI elements ``Start'' and ``Restart'', see Fig. 5 (a-c).
Finally, I decided that the ``UI sketch 3'', see Fig. 5 (c), is the best for the puzzler project.
Because Helmut's fingers are not as movable as for young people and the gaze interaction with a reticle pointer is new for him, I opted to
make a huge ``Start'' and ``Restart'' button.
Figure 5 (a): UI sketch 1
Figure 5 (b): UI sketch 2
Figure 5 (c): UI sketch 3
User Testing
Participants: All in all I conducted five user tests with five participants: 2 female -- 66/75 years, 3 male 42/72/76 years.
The VR pre-experience ranged from absolute beginner to some VR experience.
The participants life style varied from a very inactive to a very active life style.
Test Description: I conducted two quick and dirty user tests (quickie), one interview, and four exploratory user tests. All user tests
included a warm welcome of the participants, an introduction to the Mnemonic Puzzler, and the advice that only the
system is tested, not the person themselves.
Interview (UT1): The interview helped to determine information about room size, brightness and orbs arrangement.
Quickies (UT2 & UT4): To identify cock-ups in the design or the game logic, a friend of mine tested the prototypes
in a quick manner. These quickies are preflights and were conducted before each of the exploratory user tests. This
procedure helped to figure out if the adopted prototypes maintained or improved their quality, before testing them
with the “real” participants.
Exploratory User Test (UT3 & UT5): Prototype and user experience tests with the target group.
User Test 1 (Interview)
One person, female 66 years was interviewed.
UT1 Findings:
The outcome of this interview was that the room should be a little bit higher than a normal room
(> 2.5m), but not very wide, because the smaller the room the more cosy it feels. The room should be bright, but
not too bright such that there is enough contrast when the orbs are blinking. Regarding the orbs, I determined that
the arrangement of a five as on a dice was the most comfortable arrangement for playing.
The output was directly used to develop the first version of the puzzler game with Unity.
User Test 2 (Quickie)
One person, male 42 years did this pre-testing of the prototype.
UT2 Findings:
This first preflight revealed that a head mount band should be mounted on the Cardboard, because holding
the Cardboard for a longer period becomes very exhausting, in particular for seniors.
User Test 3 (Dungeon Design & Mechanics)
I conducted four user tests with two subjects: male 76 years, female 66 years.
The participants were told to explain loud what they see and what they will be doing
throughout the test (Think Aloud Method).
All tests were recorded with the AZ Screen Recorder app for later analysis.
Additionally, I made written notes of the observation of the participants.
UT3 Findings:
Visual Design
``I feel comfortable!'' respectively ``I like the room!''.
The layout of the dungeon was OK for both participants regarding room size and lighting.
``The orbs should be more in direction of the door''.
The placement of the orbs was too near to the player position.
The contrast of the orbs with respect to the dungeon was too weak.
``I would prefer that the orbs have no shadows, they distract me''.
The shadows of the orbs were distracting.
``Maybe I have less problems when the orbs are bigger''.
Participants had problems to remember the light up order and stated as reason a combination of light up speed and too small sized orbs.
The participants had problems to recall the light up order, because the light up speed of the orbs in the repeat mode was too high.
For the participants it was hard to distinguish between the repeat and reproduction mode.
The restart canvas is too close to the player's end position.
Acceptance
The participants were confused because for them it was unclear that the light up order remains the same as long they have not solved the puzzler game.
The velocity of the movements from start point to middle point and from middle point to end point was too fast.
Observations revealed that one participant had problems to press the button on the cardboard due to inflexible and thick fingers.
At the beginning the participants made only a few head movements. With more rounds player head movements increased.
One subject suffered from sim sickness.
Both participants felt frustration playing the game. The puzzle was too complex, because of the fast light up of the orbs.
The participants developed ambition to solve the puzzle.
Adjustments for an iterated prototype:
Decrease the speed of motion between play points.
Redesign of orbs: Placement of orbs next to exit; enlarge orbs, avoid shadows on orbs, decrease light up speed in repeat mode
Make the transition between repeat mode and reproduction more clear.
Fix displacement of restart canvas.
User Test 4 (Quickie)
The subject stated that the transition between the repeat mode and reproduction mode is more noticeable as before, however he desired a more direct solution, like a textual link.
I decided to test with the implemented version to explore if changing light could be a supportive way.
User Test 5 (Puzzler)
Two subjects tested the puzzler game: male 72 years. female 75 years.
Because both subject had no prior experience with VR, they started with an other VR Game to get first impressions of
the recticle pointer and how to select an object. The findings presented in the following are based on Think Aloud,
Observation and taken notes. In this session I recorded no AZ Screen Recorder videos, because ”uups“ I have forgotten
to start the screen recorder.
UT5 Findings:
Sound Design: One person mentioned that the sound of wrong and right pressed Orb is imperceptible.
Visual Design:
The basic design of the dungeon was OK for both participants.
Although, one test person stated that a more ''clean`` room setup instead of the dungeon, like only white walls, would be more helpful for the purpose of a mnemonic training in the game.
The other person stated for the purpose of training the memory, the environment is meaningless.
``I don't feel like I am welcomed''. ``The blue color is very cold''
The color cyan for the blinking orbs, had negative influences on the user experience for one test person.
One subject desired a forest or a meadow as environment for the dungeon and to switch from a night scene to a day scene.
The orbs size as well as the contrast with the dungeon was mentioned as good from both participants.
One subject mentioned the blinking color as unsatisfactory.
From the Think Aloud and the discussion afterwards it turned out, when looking straight forward in the reproduction mode the orb in the middle is accidentally selected.
This confused the participants and led to wrong solutions.
The button press mechanism of the cardboard button in combination with the recticle pointer confused and frustrated the subjects.
From the observation and the discussion afterwards it turned out that subjects had problems to select the orb while pressing the button.
Than they got nervous which led to wrong solutions and frustration.
Furthermore, pressing the button with hands up on the button was very exhausting for both participants.
Further, one subject is suffering from arthrosis which made the interaction more difficult.
``When I made a mistake, I watched at the light up order again, and continued from the point of my mistake''.
Participants were confused, at the first tries, what they should do.
The repeat and reproduction mode was communicated too vaguely .
One test person stated that it was unclear to repeat the whole light up order when he made a failure before.
The transition between the repeat and reproduction mode has to be made more visible.
Although I added the glow on the floor to easier identify the repeat and reproduction mode, the participants stated that it was still confusing.
Acceptance:
``Why I have to turn my head?'', ``Why I'am seeing only a part of the scene?'',``The virtual reality confused me.''
Both subjects had no VR pre-experience.
In the discussion afterwards, it turned out that they had problems to understand the concept of a virtual room.
Both test persons stated that they won't play such a game in the VR world, because they would prefer the ''real world``.
Both subjects mentioned that this game is not helpful to train the memory and desired a game with a more ambitious content.
``If I know the solution and I am not able to solve the game because of the non-functional button, that gets on my nerves!''.
Both participants were frustrated, and felt insecure, based on the problems with the selected orb by mistake and the inaccurate button behavior for them.
After playing the Mnemonic Puzzler both participants had weeping eyes.
Immersion: Participants were very ambitious to solve the puzzler.
``I hate the Simpsons'' One participant mentioned that the comic style Skybox is ugly.
Breakdown of final piece (Prototype UT5)
Environment
The dungeon is surrounded with mountains. It is night
and small torches are placed around the dungeon. I used a
comic style Skybox to highlight the game aspect.
UI Canvases
For the Start and Restart Canvas I used the font MorrisRoman-Black to give the text a medieval
aspect. The Start and Restart button is very huge, so that new users can press it easily.
Start Canvas: After a lot of tries I decided to place the
canvas directly in front of the dungeon entrance to ensure
readability. After pressing the start button, the canvas will
be deactivated and after pressing the restart button the
start canvas will be activated again.
Restart Canvas: The canvas is placed further far away.
By default the canvas is deactivated. When the puzzle is
solved, the restart canvas will be activated.
Dungeon
The dungeon was designed in a threshold of ”spooky“ but
also bright atmosphere atmosphere. Therefore, I mixed
white and yellow point lights with the mounted torches on
the wall.
Orbs
The orbs glow and are bigger than in the first prototype, with small gaps in-between. The contrast is very
high to the dungeon.
To distinguish between the repeat and reproduction mode
the color on the floor glows in white (repeat) or cyan (reproduction). In repeat mode the interaction with the orbs
is deactivated. In the reproduction mode the interaction is enabled.
Conclusion
A user testing process uncovers problems overlooked by the designer.
For the ”Mnemonic Puzzler“ the seniors revealed several issues.
Using the Cardboard V2 the subjects stated that the puzzler looked sometimes blurred.
I am not sure how to fix that with the Cardboards.
Playing this game on a Vive or Oculus Rift could fix it.
In the course of the project I equipped the Cardboard with a head mount band to simplify the cardboard interaction for
the seniors. Otherwise the game would have been too exhausting.
For a further iteration of the project, an alternative button mechanism should be found, because it is not working well with seniors.
The participants chosen for the experiments ranged from a very inactive
to a very active life style. One assumption is that the latter are less interested in such a game
than seniors which stay more in their home. Further, it seems that the attitude towards the puzzler increases with
VR pre-experience. Future versions of the Mnemonic Puzzler should take this into consideration and should be more self-explanatory in terms of purpose
and procedure.
A lot of more work has to be done that seniors sense a positive user experience with this game.
To conclude, the current version of the Mnemonic Puzzler is a good basis for further development.
Next Steps
Persona: Include prior game playing behavior, diseases (like arthrosis, visual impairments), and adjust the life style in next iterations.
Cardboard: Equip the cardboard with an alternative button mechanism, for example external button.
Testing of alternative recticle pointer color.
Testing of different approaches to easier distinguish between repeat and reproduction mode, for example darken the room, additional textual sign, different orb colors.
Testing of several environments for the start point, for example forest, day scene, and enlarge or delete torches.
Repositioning of the middle orb, so that it is not selected accidentally.
Testing the influence of visual impairments on the VR experience.
Deactivate the orbs when the puzzle is solved, such that it is impossible to select them from the end point.
Adding different difficulty levels: playing with different orb sizes, number of orbs, space between orbs, colors of orbs, and velocity of blinking.
Include hints for a better understanding of the procedure and the task.
In the 5th Udacity Nanodegree project the task was to build a museum experience with the theme of VR application areas.
I chose the topic Robotics in combination with virtual reality, because I love and worked with robots. In my museum experience the visitor learns how robotics could support
virtual reality and vice versa. All in all the exhibit covers application areas in professional training, robots as simulation tool to improved presence, topics in research,
telepresence robots and machine learning.
After first ideas to combine robots with virtual reality, the museum building had to be made. I have decided that a building in the direction of a warehouse or a factory is ideal.
At this time I visited some museums and exhibits (Museum der Moderne, Salzburg and the MAK in Vienna.
Following the great experiences in these museums I tried to give every station a special characteristic. The visitor can sometimes only look at things, like at the sci-fi posterwall or the improved presence
cylinders, or can actively choose things such as at the definitions station. Especially the entrance area is inspired by the visited museums. All in all in this project I combined images, videos, sounds, animations
to form my museum experience.
Sounds freesound.org
S: park.wav by potok_potoczny | License: Creative Commons 0
I developed the experience with Unity 5.1.1, so only full screen movies were possible on Android.
With Unity >= 5.6 also movies on gameobjects are possible. This I have learned during the development :).
It was very much work to build this experience (around 80h). It started with the initial research. I have now a lot more information. However, the process which information should come to the museum experience took its time. Furthermore, the video and sound search and selection was very time consuming.
Project is on Github: https://github.com/smeerws/VRND-NightattheMuseum
VRND Teamworks Project Sept '17: Theme Color
Description
As part of the recent Udacity VR Teamworks Challenge (Sept. '17), we created a unique VR experience where the user utilizes the microphone to interact with objects in the game.
A detailed description is provided at our medium post .
Hi, my name is Susanne Meerwald-Stadler (hello@xrux.at). I am a PhD student, working as a research fellow at the Center for Human-Computer Interaction (University of Salzburg, Austria).
A central question of my scientific work is how to improve human-robot interaction to make it more intuitive and usable for the end-user.
With an industry partner we started to investigate the potential and applicability of augmented reality (AR) in the area of industrial robot control and programming.
I gather first experiences in AR development and held in the hands my first AR research prototype.
Besides my passion for robots, the excitement of augmented and virtual reality has attracted me ever since.
The deeper I get into the matter, the more fascinated am I by the possibilities these technologies create to enrich our working and daily life.
In March 2017 I started the Udacity Virtual Reality Nanodegree program to improve my development skills.
Currently, this site shows my VR work in the Nanodegree program.
Restart Canvas: The canvas is placed further far away.
By default the canvas is deactivated. When the puzzle is
solved, the restart canvas will be activated.
