Surprise as a Design Strategy in Goal-oriented Mobile Applications

Emotions are important in product experiences. Besides utilitarian aspects in interacting with technological products, recent research has focused on hedonic and affective aspects of product interaction. Designing unexpected product features evoking surprise has proven to be advantageous for product user interaction and user experience. Especially, in classical product design surprising the user is an efficient design strategy to create pleasurable products. However, current research has not extensively focused on the effects of surprise in technological interactive products. It is important to understand users´ experiences in relation to unexpected events in digital products in order to design pleasurable and engaging technological products. Therefore, the focus of this study is to resolve how surprise affects user experience in goal-oriented mobile applications. The product features evoking surprise is conveyed via visual user interface design by manipulating visual characteristics of the user interface elements. The results of this study indicate that introducing surprise in goal-oriented contexts does not have a positive effect on users. It seems that an unexpected interruption of task flow lets satisfaction levels drop. Results of this study provide novel insights how surprise affects user experience in technological products and can be utilized in designing mobile applications.


INTRODUCTION
Usability as a key factor in interactive technological product design has been ruled to no longer be sufficient in determining product´s success.One claim made by the scientific community is that designing for emotional experience in addition to faultless functionality should lie in the developers' focus (e.g.Demir, 2008;Scherer, 2005;Mahlke and Thüring 2007).In addition, it is acknowledged that visual elements in designing technological interactive products possess the potential of eliciting emotions (Cyr, 2013;Zhang, 2013).Visual design is essential for pleasant and meaningful human-technology interaction contributing to positive user experience (UX) (Desmet and Hekkert, 2007;Hassenzahl, 2003;Hassenzahl and Monk, 2010;Tractinsky, Katz, and Ikar, 2000;Tractinsky, 2013).We propose a new way to design for pleasant emotions when interacting with interactive technological products, such as mobile applications, by focusing on users' expectations about the interaction and visual elements.When designing contrary to users´ expectations about a product, an unexpected event (during product interaction) should generally results in surprise (after Reisenzein, 2008).Surprise as a product feature has been proven to be beneficial for product interaction and for UX in classical product design (Ludden et al., 2009).Surprise as a design factor in classical product design has been studied by creating surprising products with visual-tactual incongruities (Ludden et al., 2009).The product experience was reported to include high overall liking, high amusement and fun after interacting with products, which included appropriate level of visual-tactual incongruities and thus surprise.In addition, elevated word-of-mouth has also been reported regarding surprising product design by conflicting information gained through different sensory modalities (Ludden et al., 2008).However, research has mainly focused on this classical product design such as for furniture and other everyday objects.Surprise as a design strategy for mobile applications has rarely been in the research focus.
Related concepts of surprise like for example wow experience have been studied to some extent regarding interaction with technological products.Wow has been defined as a 'strong, positive emotional experience, surpassing basic experience' (Steen, 2005, page 31).Wow consists of several concepts, one of which is pleasant surprise (Desmet et al., 2005).Investigating surprise on its own, on the other hand, has only been done in some product domains.
Previous studies deliberately implementing surprise in digital games confirmed an effect of unexpected events in the game on UX ratings of these games.Unexpected pleasant events like bonus points or motivating messages on the screen elevated UX ratings.Unexpected unpleasant events on the other hand had a negative effect on UX ratings, worsening them compared to control groups (Gross and Thüring, 2013).
Goal-oriented applications on the other hand have not been studied in connection with surprise.Shedding light into the effects of surprise, when interacting with mobile applications, is needed in order to provide novel insights into emotional aspects of UX.Furthermore, with this study affective design approaches for digital interactive products can be scrutinized in more detail.Therefore, this study focuses on the influence of surprise in goal-oriented mobile application.
A touch-based mobile application for information retrieval of public transportation and travel planning was designed to serve as the experimental test environment.To create surprise for users, graphical user interface (GUI) elements of the application were manipulated unexpectedly throughout the completion of a given task.Designing unexpected events was implemented via the manipulation of the following visual characteristics: color, size, shape, and texture (Mullet and Sano, 1995).The visual manipulation of GUI elements also included saturation of colors and animated movements aiming to draw attention and at the same time fit to the context.Appraisal theorists approach emotions as the results of an individual's evaluation and interpretation (e.g.appraisal) of events in an environment (Kleinginna and Kleinginna, 1981).In the case of surprise, these appraisals have been associated with unexpectedness, novelty, unfamiliarity, amazement or pleasantness (Reisenzein, 2008;Roseman and Evdokas, 1996;Scherer, 2005).Therefore, the visual manipulations of unexpected events were designed to contribute to an appropriate level of surprise regarding the product context.For Silvia (2009), the concept of pleasantness and unpleasantness as an indicator of aesthetic feeling is not far fetching enough.He argues for an inclusion of other unusual aesthetic emotions like interest, confusion or surprise.He puts surprise in a family of emotions which he calls the knowledge emotions and defines these emotions as being associated with thinking and comprehending.Particularly surprise is an interesting emotion in the context of product design.While interacting with a product, a surprising episode can shift to other feelings like delight, interest or amusement.
The study follows a between subjects design, including two groups with differing amounts of unexpected events.The first group will encounter surprise only three times during the whole experimental phase.The second group will encounter surprising events all the time during the experimental phase.Before accomplishing tasks, both groups will get to know the application in a training and free exploration phase.After this accommodation phase is over, participants will interact with the application to solve ten tasks.During this baseline phase, no unexpected events will happen in neither of both groups.This baseline phase and the subsequently completed questionnaires will later be compared to ratings of the application after the experimental phase to measure effects of unexpected events on user experience.Results of this study provide novel insights to the influence of surprise in goal-oriented mobile applications.Designers can utilize the results in designing mobile applications for effective and engaging interaction.
The rest of the paper is organized as follows.First, objective of the research is described, including hypotheses, method, stimulus material, participants and the research procedure.Then, results of the study are presented and discussed.Finally, conclusions with practical implications and future research are presented.

RESEARCH OBJECTIVE
This study tries to close the gap between contexts when it comes to research about unexpected behavior in digital interactive products.As has been mentioned above, surprise has been studied extensively in classical product design and has been proven to beneficial for users.A transfer of these findings to the domain of digital interactive products has been attempted when combining a digital interactive game with the element of unexpectedness.Findings showed that pleasant surprise elevated UX ratings and unpleasant surprise lowered them.This experiment wants to transfer these findings even further to a non-gaming, task and goal-oriented context by introducing unexpected product behavior into a mobile application for local public transportation planning.The research questions of this study are:  How do unexpected events created by manipulation of UI elements with visual characteristic affect user experience in a goal-oriented mobile application? How does the frequency of these unexpected events affect user experience? What constitutes as a surprise in this context?Knowing the possible impact of unexpected and thus surprising behavior of goal-directed applications on the UX ratings of these applications valuable to UX research and practice because a) it could give designers a clear directive whether it is useful to deliberately use surprise as a design feature in foal-directed mobile applications and b) it could help elevate the market value of a product by following the recommendations for surprise as a design element depending on context.

EXPERIMENT Method
The study followed a between subjects design.Participants were invited via a bulletin on the university's proband server to test a new mobile application which was in the very first steps of development.The application was presented on a Samsung Galaxy 10.1 Tablet.They had to accomplish several tasks with the application to search and select routes.In between trials users were given several questionnaires to cover UX ratings (see section dependent variables) of the App and reaction times were logged.

Participants
A total amount of 35 participants took part in the study.17 participants were randomly assigned to group A (three surprising events during experimental phase) of which nine were female.18 participants were randomly assigned to group B (ten surprising events during the experimental phase) of which nine were female.The participants were recruited through the Universities proband server.Average age of participants was 25.1 years (SD=4.14).

Stimuli
User interfaces (UIs) communicate to the users through visual elements, such as shapes and colors (Mullet and Sano, 1995).Besides the communicative ability inherent in them, visual elements in UI design are capable of eliciting emotions in users (e.g.Cyr, 2013).Vision is seen as an essential sensory modality in product experience (e.g.Crilly et al, 2004).In the context of mobile applications it is reasonable to design unexpected events focusing on visual design due to the evaluations and impressions of UIs first through vision, and after with other senses.Furthermore, designing unexpected events should not complicate using the app or seem very artificial.By only manipulating some visual characteristic of objects in the application, functionality of the application itself has not been altered.Therefore, the design of unexpected events in the goal-oriented mobile application is based on design conveyed with visual elements.Mullet and Sano (1995) refer to visual UI´s communicative ability with the definition of visual language.Visual language is divided into three categories of visual UI design factors.Visual characteristics are, for instance shape, color, texture, size and orientation in a specific set of design elements, such as, point, line, volume, plane, and the factors by which they relate to each other, such as, balance, structure, proportion and rhythm.
In this study unexpected events in mobile application were designed by manipulating GUI elements with visual characteristics (shape, color, texture and size).The visual manipulation evoking surprise was designed to GUI elements, buttons of the routes, which can be either bus or U-bahn buttons representing different route options.A button, which the user touches when accomplishing the given task changes according to two visual characteristics (Table 1).Every visual characteristic were used in the manipulation five times.In addition, every manipulation of the GUI elements included, besides two visual characteristics, saturation of the color and animated movement.Saturation of the color was included as a design feature due to its ability to draw attention and stand up from the background (Lidwell et al., 2003).Also, the animated movements were included because of attentional effect (Schlatter and Levinson, 2013).Every button manipulation has unique animation of movement to enhance surprising features.The differences of movements are important in order the user´s not to come across animations that would become familiar to them.Animated movements of the buttons were designed on the contrary of general interaction methods of the application.In addition, the position of the manipulated buttons were chosen according to the UI´s spatial areas that draw attention (Galitz, 2007), but still changing the location between different manipulation in order to evoke surprise.
Overall, the manipulations of the GUI elements were designed aiming to the appropriate level of unexpected events, which would be experienced as surprising, but would not lead to bad usability of the product.In addition, the appropriate level of unexpected events was designed regarding the application context as preserving the visual appearance and impression of belonging to the application.In order to understand the effect of visually designed unexpected events and what kind of impacts these events might cause to UX, the visual manipulation of UI elements was controlled according to the design factors described above.The controlled design of manipulations was in focus in order to be able to detect the role of the appearance of the stimulus to the experience.Users accomplished tasks with a transport application to search and select routes.Participants got the information they had to enter into the application in order to complete one task from a Block of 25 pages.On each page was the information for one task.One task consisted of entering start time and date as well as a start and end station into the first screen of the app.On a next screen, participants were presented with four possible connections of which only one was the "right" one.They had to tap that connection to get to the next screen where they were presented with a detailed list of all stations in the chosen connection.One station was marked with an information icon.Participants had to press that icon in order to get additional information about the surroundings of that station.After pressing the information icon, the new information appeared together with a "next" (German: "weiter") button.Participants pressed that button to get to the next task.Figure 1 shows all three screens of the application in normal mode, without any surprising events.

Independent Variables
Group was the first independent variable.Participants were divided into two groups.Group A solved a total of 25 tasks, thereby encountering unexpected visual events in the application during task 18, 22 and 25.Group B also solved 25 tasks with the application but encountered a total of 10 unexpected events of visual manipulation of GUI elements in the application during tasks 16, 17, 18, 19, 20, 21, 22, 23, 24, and 25.Unexpected events appeared when pressing on one of the four shown connections, converting the corresponding rectangle shape into a new shape which started moving around on the screen.See section Stimuli for a more detailed description of the surprise manipulations.
Event was the second independent variable.Participants encountered several unexpected events during interaction with the application.To have an objective indicator for a surprise reaction in the user, reaction times were analyzed for the three first unexpected events in every group (e.g.event 17, 21, and 25 in group A; event 15, 16, and 17 in group B). Figure 2 shows screen two of the application with and without an animation, in this case animation number 8 with manipulations of color and shape.

Dependent Variables
There were four dependent variables, four variables to cover UX and surprise ratings and two variables as objective measures for surprise.To measure UX ratings, following questionnaires were used: the SAM questionnaire which is a 9-point non-verbal instrument for the evaluation of emotions, measuring the dimensions valence and arousal (Bradley & Lang, 1994), and the meCUE questionnaire.The meCUE is a validated modular questionnaire, covering many aspects of UX.The questionnaires structure is based on the Components of User Experience model (CUE model) by Mahlke and Thüring (2007).The model distinguishes between task-related and task-unrelated product qualities.Additionally, users' emotions are integrated into the model as a crucial, mediating factor for the consequences of use, such as acceptance or a global rating.The meCUE has been developed to give researchers an agile tool when investigating the UX of a given product.There are three modules in the meCUE: Product Perception which incorporates/covers the subcategories utility and usability, as well as visual aesthetics, status, and bonding.The module Emotions covers positive as well as negative emotions, and the module Consequences covers product loyalty, intention of use and a global rating (see Minge, Riedel and Thüring 2013 for a more detailed description of the development and validation of the questionnaire).
To explicitly measure subjective surprise, a single item ranging from 1=not surprised at all, to 7=very surprised, was given to participants after completion of all other questionnaires.Reaction times were used as an objective measure for surprise.Following Reisenzein (2008) we use reaction times as an indicator for a surprise reaction in users: prolonged reaction times should indicate an interruption of ongoing processes.This prolongation should vanish over a repeated presentation of unexpected events due to the assimilation of the unexpected new information into the mental model of the situation.Reaction times were defined as the period between first appearance of the third screen of the application and the tapping of the information icon displayed on one of the stations (see figure 1).

Hypothesis
Surprise can have a profound influence on users in classical product design (Ludden et al, 2009).In the domain of digital interactive products there is not much research to be found, except for the domain of digital interactive games.In two studies, negative and positive surprise had an effect on the UX ratings of a digital desktop game (Gross & Thüring, 2013).For the field of goal-oriented digital mobile applications there has not been research on the topic.Taking into account the different contexts between a game and a goal-directed or task-oriented application, our general assumption is that unexpected events during the interaction with a mobile application have an effect on the UX ratings of this application.We assume that surprise conveyed via manipulation of visual design elements will make the task itself surprising, but may have negative influence to the UX.This assumption stems from the fact that surprising unexpected behavior might be beneficial for a game but might interrupt the task-flow for a user when using a goal-oriented application.Additionally, we assume that when the unexpected event occurs in every task it is not seen as surprising but rather annoying and have a strong negative effect on UX.

Procedure
The experiment was conducted in January 2014.The test device was a Samsung Galaxy Note 10.1 tablet (GT-N8000) with an Android version 4.1.2operation system.Completion of the experiment took about 45 minutes per participant.The language spoken was German.Upon arrival, participants were briefed on the purposes of the study, giving them a cover story about the testing of the mobile application.Unexpected events were not mentioned in this introduction.After completing a demographic questionnaire (sex, age, ownership of a smartphone/tablet, knowledge of other public transportation applications), participants got an introduction to the application.Because it was essential that they understood why they needed to complete 20 experimental tasks in total, they were asked to imagine that they had to plan a friend's visit and wanted to find out more about the surroundings of several public transportation stations.
Participants were given five tasks to solve with the application under supervision of the experimenter.This gave them time to ask questions and to accommodate to the data entry format and the design of the application.After this training phase, a block of ten tasks followed after which the SAM and the meCUE were completed (we will call this block the baseline phase).After the baseline phase another block of 10 tasks followed during which unexpected events were encountered, either three or ten times (we will call this block the experimental phase).After the experimental block, the SAM and the meCUE had to be completed a second time.The sequence of presentation of the tasks was random except for tasks 18, 22 and 25 during the experimental phase in both groups.All participants were debriefed to the real purpose of the study after completion.

Reaction times
To properly investigate the effects of the unexpected events on reaction times and thus information processing, a baseline for reaction times was calculated by averaging reaction times per group in overall ten events from the baseline phase.These times were then subtracted from reaction times of the three first unexpected events per group.Figure 3 shows the reaction times for both groups for the first three unexpected events minus baseline reaction time (explaining a negative time for the last event).Because group A experienced only three unexpected events and group B ten unexpected events, only the first three events in group B were compared with the three events in group A. Reaction times were analyzed like this to keep the novelty of the events as comparable as possible.
Comparing event 18 from group A with event 18 from group B would not yield the same amount of information since group B would have seen an unexpected visual event already three times, whereas group A would not have had this experience.A 3(event) x (group) analysis of variance (ANOVA) revealed a significant main effect for event (F(1.622,50.277)=6.798,p=.004, η PART 2 =.180; Greenhouse-Geisser corrected) Contrast analysis revealed the source of this effect being a significant difference of reaction times between event1 and event3 as well as between event2 and event3.ANOVA did not reveal a significant main effect for group (F(1,31)=.548,p=.465,η PART 2 =.017) or the interaction between group and event (F(1.622,50.277)=.490,p=.577,η PART 2 =.016).

SAM ratings
SAM scales were inverted for a better understanding of the ratings.Scales range from 1 to 9. On the valence scale a high rating indicates a more content state of the participants, on the arousal scale a high rating indicates a higher aroused state of the participant and vice versa.Figure 4 shows mean SAM ratings per group and subscales valence and arousal after the baseline phase and after the experimental phase.A oneway ANOVA revealed no significant differences between groups after baseline phase for valence (F(1,31)=2.182,p=.150) and arousal (F(1,31)=1.092,p=.591).After experimental phase, ANOVA revealed a significant main effect on the SAM valence scale (F(1,31)=9.335,p=.005) and no significant main effect on the SAM arousal scale (F(1,31)=.009,p=.926).

MeCUE ratings
MeCUE ratings range from 1 to 7, in which a lower rating indicates disagreement with a statement and a higher rating indicates agreement with a statement.To compare ratings after baseline phase and after experimental phase, differences were calculated between baseline phase ratings and experimental phase ratings.In order to do so, meCUE ratings after the baseline phase were subtracted from ratings after experimental phase.A positive value indicates a higher agreement with the meCUE statements for each scale; a negative value on the other hand indicates a lower agreement with statements of the meCUE scales after being exposed to the unexpected visual manipulations.Figure 5 shows the absolute differences after this procedure ranging from -0.3 to +0.4.MANOVA of all nine subcategories of the meCUE questionnaire revealed no significant multivariate effect (F(9,24)=.799,p=.621).

Surprise item
The surprise item was single-item ranging from 1=not surprised at all to 7=very surprised.Mean surprise indicated in group A was 5.47 and in group B it was 5.16.A oneway ANOVA revealed that this difference was not significant between the groups (F(1,33)=.439,p=.512).

CONCLUSIONS
According to the results of this study, surprise should not be utilized as a design strategy regardless of the product context.In summary, differences between groups after experimental phase could only be observed in terms of SAM valence ratings.Participants who experienced the visual manipulations ten times indicated to be feeling more satisfied than participants who only had three visual manipulations.We argue that for participants in group B, the visual surprises during interaction with the application were experienced as being a new feature of the application, rather than a surprise.One proof for this argument is that after the third appearance of a manipulation, reaction times were back to baseline level.For these participants the manipulations were not surprising anymore and apparently they liked this new feature in their application.
For group A in the other hand the three manipulations did not proof to be beneficial in terms of satisfaction as can be seen by a slight drop on the SAM valence scale and the significantly different ratings compared to group B after the experimental phase.Although reaction times indicate that participants in both group experienced the classical interruption of ongoing processes which indicates a surprise reaction, this did not reflect in most of the questionnaires participants completed, other than the SAM valence scale.Both groups showed significant prolongations of reaction times and also rated their subjective surprise higher than average on the surprise item scale.Nevertheless, for meCUE ratings, differences between ratings after the baseline phase and ratings after the experimental phase only ranged from -0.3 to +0.4, meaning differences of meCUE ratings after the two phases were rather small.None of these differences proofed to be significant, and SAM arousal ratings virtually stayed identical between the two phases, too.
Designing unexpected events in goal-oriented applications does not seem to have a very distinct effect on users.
Although it has been found that surprise has a positive effect on users in classical product design and in the domain of digital games, this does not seem to be the case in digital goal-oriented applications like the one used in this study.However, according to the results of SAM valence scale, it can be stated that designing for surprise is not beneficial in goal-oriented mobile applications, but designing for appropriate level of unexpected events can evoke positive emotions of the application.This applies only if the appropriate level of design is achieved in a way that users perceive it as a part belonging to the application.In designing for an appropriate level of unexpectedness the context of the application and type of technological product regarding design decisions is essential.

Future research
The methodological approach of this study could be implemented in studying the role of surprise in different kinds of technological products, such as mobile applications, websites, or in specific parts of operational systems.In addition, the unexpected events could be designed to involve other sensory modalities to the product experience than the visual domain in technological product context.This could be further extended into studies aiming to resolve which senses dominate the product experience (Fenko, Schifferstein, and Hekkert, 2006) regarding surprise in different use-phases.Moreover, this study could be extended to a longitudinal study in which the effect of surprise could be approached in several use-phases, especially in the retrospective use-phase.Longitudinal study of surprise in product experience would provide more insights into what kind of influence surprise possesses in a product experience life cycle.
The results of this study can be extended to be applied into designing mobile applications in Western cultures.In order to extend the result implications to be applied into other cultures, a cross-cultural research focusing on the role of surprise conveyed through visual UI design would needed to be conducted.
(first to the bottom right corner, then to the bottom left, then top right and then top left) 7. Color & Shape + saturation + flying straight to the top 8to the left and down and then jumping around on the bottom line

Figure 1 :
Figure 1: UI of the application screen 1 to 4 (upper left, right; down left, right)

Figure 2 :
Figure 2: Screen two in task 2 with no manipulation (left) and manipulation (right).

Figure 3 :
Figure 3: Reaction times per event and group.

Figure 4 :
Figure 4: Mean SAM ratings per group after baseline phase (BL) and experimental phase (EXP).

Table 1 :
Visual manipulations of unexpected events in buttons 1. Texture & Size