Emotional Dimensions of User Experience: A User Psychological Analysis

User psychology is a human–technology interaction research approach that uses psychological concepts, theories, and findings to structure problems of human–technology interaction. As the notion of user experience has become central in human–technology interaction research and in product development, it is necessary to investigate the user psychology of user experience. This analysis of emotional human–technology interaction is based on the psychological theory of basic emotions. Three studies, two laboratory experiments, and one field study are used to investigate the basic emotions and the emotional mind involved in user experience. The first and second experiments study the measurement of subjective emotional experiences during novel human–technology interaction scenarios in a laboratory setting. The third study explores these aspects in a real-world environment. As a result of these experiments, a bipolar competence–frustration model is proposed, which can be used to understand the emotional aspects of user experience.


INTRODUCTION
Human-technology interaction can be studied from many different scientific perspectives, such as sociology and social research (Rosenberg, 2004), ethnography (Millen, 2000), and neuroscience (Johnson & Proctor, 2013;Parasuraman & Rizzo, 2006). One important field in the investigation of humantechnology interaction is psychology (Card, Moran, & Newell, 1983), and as a whole, psychological human-technology interaction research is known as user psychology (Moran, 1981;Saariluoma, 2004;Saariluoma & Oulasvirta, 2010). The goal in user psychological research is to explain phenomena typical to interaction between people and technology (Saariluoma, 2004). Such explanation requires explication of the phenomena and their associated problems in the language of psychological theory. The key reason for developing explanatory user psychology for analysis of human-technology interaction is to build a close connection between psychological research and design solutions, in the same way that research in the natural sciences is related to engineering design solutions (Pahl, Beitz, Feldhusen, & Groete, 2007;Ulrich & Eppinger, 2007).
Much attention today focuses on user experience research (Hassenzahl, 2008(Hassenzahl, , 2010Kuniavsky, 2003). Approaches such as affective design (Helander & Khalid, 2006;Khalid, 2004), Kansei engineering (Nagamachi, 2002(Nagamachi, , 2011, emotional design (Norman, 2004), design for pleasure (Jordan, 2000), and funology (Monk, Hassenzahl, Blythe, & Reed, 2002) emphasize investigating users as emotionally experiencing beings. Further, researchers maintain that it should also be possible to consider emotions when designing products and justifying design solutions. For full realization of this agenda, the foundations of user experience research should be built on psychological concepts, principles, and theories. In other words, a user psychological analysis of user experience is required. We move in this direction by grounding the notion of user experience into the psychology of emotion and emotional human-technology interaction.
Emotion plays an indisputable and central position in psychological thinking (Beck, 1976;Ekman, 1999;Lazarus & Lazarus, 1994;Oatley, Keltner, & Jenkins, 2006;Power & Dalgleish, 1997). Rather than understanding emotion as the opposite of cognitive or rational behaviour and mere distraction, emotion can be understood as a critical part of effective decision making and learning (Damasio, 1994;Kahneman, 2011;Scherer, Schorr, & Johnstone, 2001). It is clear that emotion affects user behavior and experience, but the exact role of emotion in user experience and human-technology interaction requires further investigation. How are emotions elicited when people interact with technology? How can we measure or otherwise investigate emotions during this interaction? How can we design for particular emotional user experiences?
Entertainment and art products, such as games, fashion, and films are designed to elicit emotional states, like pleasure, in people. Pleasing the user aesthetically is the goal of many design products like perfume, pottery, and furniture (Jordan, 2000). Achieving high aesthetic value in product design is based on understanding user emotion. From this perspective, it is understandable that the value of emotional research, that is, the research on emotional concepts, theories, and measurement practices, has increased its importance in human-303 304 P. SAARILUOMA AND J. P. P. JOKINEN technology interaction research. However, much of emotional design remains intuitive: designers ground their designs on how they feel and think, rather than on scientifically based objective measurements (Saariluoma & Maarttola, 2003).
Good designers conceive how our games, films, pottery, and clothes should appear and feel, and the emotional aspects of the products depend on their vision (Saariluoma & Maarttola, 2003). However, for some time now, the agenda has been to find a more scientific foundation for design solutions concerning the emotional dimensions of products. As an early example, Kansei engineering studied how one could measure emotional responses to products (Nagamachi, 2011). Domainspecific "Kansei-words" were collected and used to generate measures, which were then used to acquire information about possible affective design goals (Nagamachi, 2011). Kansei engineering was founded on the Charles Osgood (1952) conception of psychology, which did not operate in a cognitivist framework. Osgood, and consequently Kansei engineering, did not use cognitive models of mind; hence this work lacks the power to explain the phenomena under investigation. Here, following the large cognitivist and emotional literature since Osgood, we take emotions and experiences as internal mental and thus representational states (Newell & Simon, 1972).
After Kansei engineering, the next, qualitatively different step in emotional human-technology interaction research was initiated by Donald Norman (2004). This author generated a psychological theory of emotions and wished to ground design argumentation on this description of emotional processes. He suggested that human emotions have three main conceptual levels: visceral (perception level), behavioral (expectation level), and reflective (intellectual level; Norman, 2004;Norman & Ortony, 2003;Ortony, Norman, & Revelle, 2005). We maintain that, although this theory holds merit in deepening analysis of emotion in human-technology interaction, it does not sufficiently consider what emotional experience is like. This kind of analysis necessitates investigation of mental representations and mental content of users; we focus on these elements.
Modern psychology accepted mental representation as its basic analytical concept in the 1960s (Newell & Simon, 1972). Mental representation as a theoretical concept is used to describe how such information as memories, thoughts, ideas, precepts, motor movements, desires, and emotions exist in the human mind as representations (Fodor, 1987(Fodor, , 1990Newell & Simon, 1972). To harmonize user experience research with modern psychology, it is essential to conceptualize emotions in human-technology interaction utilising the concept of mental representation (Moran, 1981;Saariluoma & Oulasvirta, 2010). This kind of analysis allows researchers to associate and to apply the achievements of 150 years of psychological research on the human mind to analyzing and solving design problems (Saariluoma, 2004;Saariluoma & Oulasvirta, 2010).
Mental representation entails at least two layers of mental content. First, mental representation is cognitive and focuses on sensory and memory information. It can, then, contain information about space, objects, colors, movements, systems, laws, or nonperceivable elements. Second, mental representations entail emotional or other types of dynamic information. Emotional content contains such aspects of emotion as valence (positive and negative feelings), arousal (activating or deactivating feelings), mood, and emotional themes, such as fear, envy, rage, or happiness (Power & Dalgleish, 1997;Russell, 1980). Mental representations also have conscious and subconscious parts. We can verbally express and explicate the contents of the conscious part (Allport, 1977), but the subconscious part is beyond our immediate explication. It is possible, however, to become conscious of the subconscious, as in the case of language grammar. Usually a native speaker cannot express the whole system of rules of the language, but these rules can be described by investigating what people speak (Berry & Dienes, 1993). Similarly, although it may not be always possible to explicate emotions during or after an experience, it is possible to study emotional user experience by analyzing the representational systems that concern emotions. It is well known in clinical psychology that consciousness is the gate to subconscious emotional processes (Beck, 1976;Freud, 1917Freud, /2000. Representational systems of emotion can be explicated by experiment, as, for example, in the classic Russell (1980).
In investigating conscious and subconscious aspects of user experience, it is essential to collect information about the mental content of conscious experience (Rauterberg, 2010). This can be accomplished using various methods, such as verbal protocol analysis (Ericsson & Simon, 1984), retrospective interviews, observations, or questionnaires. Each of these methods have advantages and disadvantages, and their role in studying emotional user experience still needs further investigation (Vermeeren et al., 2010). We have previously used protocol analysis (Rousi, Saariluoma, & Leikas, 2010) and semantic differential scales (Saariluoma, Jokinen, Kuuva, & Leikas, 2013) to analyze user experience as mental contents. In the present article, our focus is on questionnaires about basic emotions.
The critical differentiating feature of modern user psychology compared to other approaches lies in explanation and explanatory thinking (Saariluoma, 2004). Basic psychological knowledge, such as the psychology of emotion, is taken as the basis of conceptualisation, operational definitions, interpretation of results, and explanation of outcomes. The classical theory of explanation thus provides us with a solid framework to rely on with respect to thinking about user (Achinstein 1983;Hempel & Oppenheim, 1948;Scriven, 1962;Sellars, 1963). In terms of user thinking, explanation is connected to both the phenomenon to be explained (explanandum) and the phenomenon that explains (explanans). In user psychology, conceptualization and explanation are based on basic psychological concepts, for example, basic emotions, and explanadum is the aspect of the user interaction process under investigation. The underlying structure of researcher thinking takes the schematic form: (a) define relevant interaction phenomenon; (b) conceptualize in terms of basic psychology; (c) respectively construct experiment, that is, operationalise theoretical concepts EMOTIONAL DIMENSIONS OF USER EXPERIENCE 305 and design the testing process; and (d) use basic psychology in interpreting and explaining the outcome. In this way, user psychology allows systematic use of psychological knowledge, not just methodology, in working with interaction problems.

GENERAL METHOD
Operationalization is a central step in any empirical research endeavor (Blalock, 1971;Bridgeman, 1927;Raykov & Marcoulides, 2011). In this process, theoretical concepts are transformed into empirical ones, which can be used in constructing empirical procedures. In our experiments, the focus is on the emotional dimensions of user experience. In the spirit of user psychology, psychological theory and empirical findings on emotion are applied to construct an empirical research approach. Thus, the content validity of the measures is supported by traditional psychological research.
Our goal is to operationalize emotional user experience into questionnaire items of intuitively understandable basic emotions (Ekman, 1999;Power & Dalgleish, 1997;cf. Ortony & Turner, 1990;Scherer, 2005). This way, the participants could report the contents of their firsthand experience without too much interpretation and complexity. In user psychological concepts, human emotional experiences are based on emotional states. Emotional states are combinations of emotions in a representation. Basic emotions are not necessarily atomic but are common and discriminating components of emotional states. It is also well known that the lists of what are basic emotions vary from two to over 15 (Power & Dalgleish, 1997;Spinoza, 1677Spinoza, /1955. In investigating how people experience interaction with technologies, questionnaires allow subjects to report how they experienced the interaction situation. In comparison with open interviews or think-aloud procedures, questionnaires offer a more structured and standardized way to collect data, which can be analyzed with statistical methods. In operationalizing the questionnaire on emotional user experience, we consider emotion items from general sources of psychological studies of emotions (Ekman, 1993;Russell, 1980;Shaver, Schwartz, Kirson, & O'Connor, 1987;Yik, Russell, & Steiger, 2011), and sources in which emotions are studied especially as part of human-technology interaction (Hassenzahl, 2008;Juutinen & Saariluoma, 2007;Picard, 1997). The psychological ground theory relies on the theory of basic emotions (Ekman, 1993(Ekman, , 1999Oatley, 1992;Power & Dalgleish, 1997).
Efficacy and excellence are chosen as emotion-based questionnaire items, to represent participants' reflection of their own skills and capabilities. Efficacy means potency to produce effects and, respectively, the feeling of efficacy or self-efficacy refers to experienced personal efficacy (Bandura, 1982(Bandura, , 1993(Bandura, , 1995. Determination and vigilance are added to allow participants to reflect their autonomy, attentiveness, and exhaustion during interaction with the novel interfaces used in our experiments (Carroll & Thomas, 1988;Hassenzahl, 2008Hassenzahl, , 2010. Pride is included to determine whether successful interaction with novel interfaces is associated with being proud, that is, having joy over one's own performance, as suggested by Juutinen and Saariluoma (2007). To study the connection between pride and success in interaction, we add successful. Finally, we add an item for the amount of control the participants feel during the interaction. However, this item is not considered one of the emotion items and is used only at the end of our analysis, where we combine the data from all experiments.
On the arguably negative side of interaction, we add frustrated (Juutinen & Saariluoma, 2007) and anxious (Liu, Agrawal, Sarkar, & Chen, 2009). Frustration is normally seen as the emotion arising when one cannot reach one's goal, and anxiety can be seen as fear of something unexpected and undefined (Lazarus & Lazarus, 1994). We expect these emotions to be visible after difficult interactions. To reflect the possible "novelty problems" of the interaction, we add confusion and annoyed (Kuniavsky, 2003). To identify especially difficult interactions, struggle is added. Finally, excited is included to determine how arousing an experience the novel interaction would be (Hassenzahl, 2008).
The number of individual questionnaire items varies from 25 to 12 between the three experiments reported, but we choose to report only the emotion-related items previously listed and relevant for our analysis. All questionnaire items for the three experiments are displayed in Appendixes A, B, and C. The questionnaire for emotion-based items was presented to the participants at completion of the experiments, and participants were asked to reflect their experiences during the experiment using the questionnaire. The questionnaire is composed of statement items such as "During the test I felt determined." The items are scaled from 1 (not at all) to 5 (very much).
To study the nature of emotional user experience behind the questionnaire responses, we use multidimensional scaling (MDS). MDS is a statistical method that produces an n-dimensional representation of the similarities between the variables in the analysis. MDS is a good way of eliciting knowledge representations of individuals (Oldon & Biolsi, 1991), but it can also be used to explore more general interindividual commonalities (Kruskal & Wish, 1978). The basic goal of MDS is to analyze proximity of the stimuli, such as questionnaire items (Nunnally & Bernstein, 1994). The S-Stress value from the MDS solution can be used to confirm how many dimensions are suitable for representing the data (Kruskal & Wish, 1978;Russell, 1980). Smaller stress values indicate better fit of the solution (Nunnally & Bernstein, 1994). There are no objective values for a good fit, but the stress values should stop decreasing during the first iterations. For our analyses, we report the first three stress values used to find the best fit of the MDS solution.
MDS has been used to analyze emotional response. In the classic study leading to the valence-arousal circumplex model of emotion, Russell (1980) asked participants to rate different emotions by their similarity and proceeded to generate a twodimensional MDS configuration of the responses. In the same manner, similarities between emotional questionnaire items have been explored using MDS in the studies of, for example, musical expertise (Bigand, Vieillard, Madurell, Marozeau, & Dacquet, 2005), facial expression analysis (Alvarado & Jameson, 2002), and cultural differences (Fontaine, Poortinga, Setiadi, & Markam, 2002;Russell, 1991). Usually in such studies, the participants are asked to directly rate the similarities between two or more stimuli. In our experiments, however, the MDS was conducted to the correlation matrix of the emotionbased items. In the end, the result is the same: a proximity solution for the emotion-based items.
Often in MDS analyses of emotional response, the resulting solution is two-dimensional and is interpreted by either naming the dimensions or creating groups of emotions to represent more general emotional factors. The groups of emotions, that is, emotions that are closely related to each other, can be analyzed by asking what general factors might be behind them. Dimensional analysis, on the other hand, often reveals at least one strong judgment category, valence, and possibly two more, arousal and potency (Desmet, 2012;Russell, 1991;Yik et al., 2011). As shown next, both of these ways of interpreting an MDS configuration can be used to study emotional user experience as mental contents.
In the present study, MDS is used to produce a twodimensional, visualized configuration of the similarities or dissimilarities between the emotional questionnaire items. The MDS solution is obtained using the ALSCAL program in PASW Statistics 18. The program is instructed to create distances from the correlation matrix of the emotion items using Euclidian distance interval, and the transform values are Z score standardized for better visualization. From the observed configuration of the items, we aim to construct sum variables that could represent more general factors of emotional user experience. We use the terms "factor" and "group" to refer to the salient configuration of proximal emotion items. Cronbach's alphas are calculated to confirm a suitable internal consistency of the items and the reliability of the sum variables. The sum variables are calculated as the mean of the emotional questionnaire items included in the sum variable. In this way, the sum variables retain the original scale (from 1 to 5) of the questionnaire.

Method
In the first experiment, the participants used a touch-screenbased interface to control automated features of a miniature crane. A total of 19 participants (10 male, nine female; age range = 20-56, M age = 26.5, SD = 8.9) were recruited for usability tests of a new touch-screen-based user interface. The participants were university students with no prior experience of crane operation. However, the experimental tasks required no prior knowledge, and all participants understood and accomplished the tasks. Although the participants had used touch screen devices before, the combination of crane driving and touch screen controls made the interaction novel and, at times, complex, as was evident in the verbal reports of the participants.
The participants used an ordinary game joystick (Logitech Extreme 3D pro) to control a miniature bridge crane with a working area of 0.6 × 1.5 m. The miniature crane was a model of an overhead bridge crane and was constructed for this experiment. It had 3 degrees of freedom (moving directions): The girder (bridge) could be moved forward of backward, the trolley left or right, and the hook up or down. Each of these directions could be controlled using the single joystick.
The crane was programmed with automated features that were used to position the crane automatically and control operating area limits. The touch screen user interface was implemented on a 7-in. Samsung Galaxy tablet with Android 2.3 operating system. Two differing user interface variations were constructed: one with an interactive map of the miniature crane's operating area and one with only buttons and symbols to achieve similar interaction. The user interface was designed to control the automated features implemented in the crane. The use of a joystick to control the 3 degrees of freedom of the crane, combined with the use of a touch screen to control the automated features, made the control process novel and demanding to the participants.
This experiment was a between-subjects design, with the participants divided equally into two groups of 10 participants. The first group used the map-based touch screen controller and the second group a buttons and symbols-based touch screen controller. The joystick controller was same for both groups. Both groups conducted four tasks that required utilization of both the joystick and the touch screen controller. Each participant conducted the tasks in the same order, as the research interest was not in the difference between tasks but on the participants' whole emotional experience during the experiment.
After the participants had completed given tasks using the miniature crane, they completed a questionnaire with emotion-based items concerning their experiences during the experiment. The emotions included are shown in Figure 1, and introduced in the General Method section. In the data analysis, MDS was conducted on the emotional questionnaire items. Proximities between the items were analyzed using visual inspection and Cronbach's alpha, and sum variables were constructed. The role of analysis in this experiment was exploratory, and we refrain from making hypotheses for the analyses. In the next two experiments, hypotheses created on the basis of the first experiment are stated.

Results
The stress values for the first three iterations of the MDS were .186, .160, and .158. The two-dimensional configuration of the emotional items is set forth in Figure 1. Two main emotional groups are identified from the configuration. The first group consists of feeling successful, efficacious, and vigilant. The second group consists of feeling frustrated, annoyed, confused, and anxious. These two emotional groups are situated on opposite sides of each other on the x-axis, which we interpreted as valence dimension (Russell, 1980(Russell, , 1991. Two sum variables were constructed to represent the two groups or factors of the emotional items. The first factor was designated Competence, and the internal consistency of the three items calculated into the sum variable was α = .768. The mean of the competence sum variable was 3.67 (SD = 0.60). The second factor, consisting of four items, was designated Frustration and had an internal consistency of α = .832. The mean of the sum variables was 2.05 (SD = 0.83), which is considered relatively low (on the scale, 1 = none at all and 2 = a little).
Competence was negatively associated with Frustration, as revealed by Spearman correlation (ρ = -.579, p = .009). This is expected, as the two groups are clearly situated on opposite sides of the MDS configuration. To see whether the two groups of participants (map-or symbol-based touch screen) gave different emotional responses as measured by the two factors, independent-sample t tests were conducted. Somewhat surprisingly, we observed no difference in the means of the sum variables between the two user interface groups (p > .05 for both Competence and Frustration).

Discussion
By asking university students to perform tasks on unfamiliar technology, we aimed to elicit clear emotions associated with novel human technology interaction. The emotional experiences of the participants were collected using a questionnaire containing a collection of emotion-based items. Multidimensional scaling of the chosen questionnaire items revealed two main groups, or factors, of emotions, Competence and Frustration. The main dimension separating the two groups was identified as emotional valence: how pleasant or unpleasant the emotions were.
This finding is in line with classical studies of emotion (Russell, 1980(Russell, , 1991, although the other commonly identified dimension, activation or arousal, was not observed. The lack of the arousal dimension seems, however, to be common in many studies of emotional experience (Desmet, 2012). Emotional arousal is perhaps more difficult to elicit during human-technology interaction than emotional valence: Our questionnaire did not contain the necessary items to reveal such dimension, and our experiment did not create enough emotionally arousing conditions.
Valence is a basic content dimension of human emotion (Frijda, 1986;Russell, 1980Russell, , 1991 and, as such, is compatible with our mental contents approach. Attributing valence to an emotion gives the emotion content-based significance. Another important consideration regarding the mental content of emotions is emotional theme, or narrative. The focus of this concept is the difference between different emotional states with respect to their contents. Anger, for example, differs from happiness, as the former is related to being threatened or offended, whereas the latter refers to a state of well-being and joy. Of the emotion-based items of positive valence, efficacy, success, and vigilance were clustered. Feeling efficacious reflects being successful in producing intended results and controlling events (Bandura, 1982(Bandura, , 1993 and is very important in creating a pleasant user experience in novel interactions. Feeling of success is hence in line with the given emotional state. Feeling vigilant reflects being alert and watchful for risky events, which demands mental effort (Kahneman, 2011). In many cases of complex novel interaction, where safety is of concern, producing vigilance on both the cognitive and emotional levels is a requirement for good user experience. An experience of this sort is identified as one of the main goals of user-experiencefocused design thinking (Hassenzahl, 2008). We feel that the label Competence best describes this emotional user experience factor (Gravill, Compeau, & Marcolin, 2006).
The other cluster of emotions was connected by negative valence. Frustration can be seen as an emotional response to opposition, obstacles, or blockade of action (Juutinen & Saariluoma, 2007). Confusion refers to the inability to get a clear picture of the demands of the interaction situation and is especially critical in novel or complex interactions. Anxiety refers to feelings of uncertainty, fear, and uneasiness, when no definable object is present (Power & Dalgleish, 1997). Annoyance is a state close to aggression and a typical accompaniment to frustration. These feelings are understandably closely linked to both confusion and frustration. Confusion covers a definable target and leads to frustration as well as to anxiety and annoyance. Here, we label this combination of feelings under the common label Frustration and see it as opposite to competence (Juutinen & Saariluoma, 2007).
People who are able to use given technologies evidently see their competence in a positive light. They reach their goals in using the given technology and consequently feel themselves to be skilled, capable, and able to focus on the tasks at hand. The lack of frustration follows from not being confused and angry with a piece of technology and being able to reach personal goals with that technology. The experience of competence creates confidence, and this is based on the experience of smooth performance and interaction.
As the result of the first experiment, we suggest a bipolar competence-frustration hypothesis, which generalizes our earlier pride-frustration thinking (Juutinen & Saariluoma, 2007).
The ability to use technology generates a positive user experience, which can be observed as emotions clustered around the Competence. On the other hand, the inability to successfully interact with technology causes negative user experiences, which can be observed as emotions clustered around the Frustration. We propose that the emotional experience associated with successful use of new technology varies between these two poles. This is evident in the negative correlation between Competence and Frustration.

EXPERIMENT 2
The first experiment resulted in a competence-frustration hypothesis of user experience in a novel interaction scenario. In the second experiment, we tested this hypothesis by again bringing participants into novel interaction and measured their emotional responses. Further, we repeated the same interaction with modifications to the interface, which resulted in two sets of emotional responses by one participant and hence allowed further tests for reliability of the measurement scales of emotional user experience. The hypotheses are as follows: H1: Two emotional dimensions of user experience, separated by their valence, emerge in interaction novel user interface. These are Competence and Frustration. H2: There is a negative correlation between Competence and Frustration.

Method
Ten university students (five male, five female; 19-29 years old, M age = 23.3, SD = 3.7) were recruited to perform interaction tasks using a computer simulation of a remotely controlled port gantry crane. None of the ten students had participated in the first experiment, and operation of cranes was new to all of them. The simulator used two joysticks (Logitech Extreme 3D pro) to achieve five movement directions (east-west, northsouth, up-down, tilt, and skew), which created a complex control situation in an environment unfamiliar to the student participants. The simulator had two remote operating layout designs: a layout without modifications and a modified interface with dynamic layout aids to help the user in controlling the crane. The simulator was designed at Tampere Unit for Computer-Human Interaction. The goal of the user tests was to study whether the dynamic aids help the user to control the crane. However, here our focus is on the emotional questionnaire responses of the participants, not on the dynamic operating aids.
Participants were asked to perform a series of tasks using the simulator in two parts. A single task was either lowering a cargo container onto a truck or lifting a container from a truck. Different container sizes and changing wind conditions were used to introduce variance into the tasks. In the first part of the experiment, the participants performed baseline tasks using the normal interface. In the second part of the experiment, they performed similar tasks using the interface with a dynamic layout, which consisted of automatic camera zooming and augmented reality aids: direction arrows showing the direction and speed of the crane, and arrows suggesting the correct placement of the crane in the task. Ten minutes were reserved to perform as many tasks as possible in both parts of the experiment. After both task blocks, the participants completed a questionnaire with emotion-based items.
We reduced the number of emotion-based questionnaire items to 12. This was done to lessen the participants' load of answering too many questions during the experiment, because the emotion-based questionnaire was completed twice during the experiment and because the experiment involved additional questionnaire scales not reported here (such as feeling of presence, which is important with respect to the remote operation concept). H1 was tested with visual inspection of the MDS solution, created in a similar manner as in the first experiment. Cronbach's alpha was also calculated for items clustered together to confirm the reliability of the scales for the Competence and Frustration factors. H2 was tested with Spearman correlation of the sum variables of these two scales.

Results
The stress values of the first three iterations for the emotional responses for the first part of the experiment were .122, .105, and .105. The MDS configuration of the responses is shown in Figure 2. As in the first experiment, and supporting the first hypothesis, two groups of emotions were observed. The sum variable Competence in the baseline tasks had an internal consistency of α = .874, and included emotional items efficacy, excellence, pride, determination, and vigilance. Its mean was 3.32 (SD = 0.81). The sum variable Frustration in the baseline tasks includes variables annoyance, confusion, anxiety, frustration, struggle, and excitement. It had an internal consistency of α = .908 and its mean was 2.13 (SD = 0.76). Regarding H2, the Spearman correlation between the factors was negative but statistically nonsignificant (ρ = -.346, p = .328).
For the emotion-based items from the second part of the experiment, with the tasks with dynamic layout, the stress values of MDS were .206, .163, and .161. The configuration is shown in Figure 3 and shows a similar grouping of emotionbased items as in the first part of this experiment. The sum variable for the Competence factor had the same five items as in the first part of the experiment, an internal consistency of α = .821, and a mean of 3.40 (SD = 0.67). The sum variable for the Frustration factor also had the same items as in the first part of this experiment, although this time the items were not as close to each other. This is evident in a slightly lower consistency of α = .794. The mean of the sum variable was 2.03 (SD = 0.67). Visual inspection and Cronbach's alphas confirm H1 again. Regarding H2, Spearman correlation revealed no association between feeling competent and frustrated (ρ = .053, p = .884).

Discussion
The second experiment resulted in similar clusters of emotion-based items as the first experiment, confirming H1. Similarly, the emotional groups were separated from each other along a clear emotional valence dimension. Six emotion items of positive valence, and five of negative valence, were identified and included in the summated scales for the Competence and Frustration factors. Contrary to the results of the first experiment and H2, we observed no correlation between feeling competent and feeling frustrated. It is possible that the association between these two emotional user experience factors is dependent on the nature of the interaction. Closer inspection of the items of the factors and their correlations with each other may reveal some details of the relationship between the two factors. This analysis is conducted in the end section of Experiment 3 in this article.
With the addition of determination and pride into the Competence factor, the description of this hypothesized emotional user experience factor becomes clearer. Feeling proud refers to satisfaction elicited from successful technology use (Juutinen & Saariluoma, 2007), and feeling determined shows how competence is associated with autonomy during such use (Hassenzahl, 2008). Especially in the case of novel or complex user interfaces, these emotions are clearly important for the feeling of competence. Regarding the new items of the Frustration factor, excitement and struggle both indicate how frustrating situations can also be accompanied by emotions of high activation or arousal. These emotions are elicited when the interface cannot provide sufficient support for work tasks, and the user must struggle to accomplish them.
The results of the first two experiments show that the factors of emotional user experience can be analyzed using different emotional items. Our contention is that, although different emotions are elicited in different interactions, and although the same interaction can be analyzed with different methods comprising different sets of emotion-based items, in the end the emotional user experience can be analyzed via the same two factors, Competence and Frustration. The reliability of these two scales is supported by the fact that very similar patterns emerged during two distinct interaction tasks. However, these results are from a controlled laboratory environment. To confirm that our results are not artifacts of laboratory experimentation, they need to be replicated in a naturalistic setting.

EXPERIMENT 3
The first and second experiments just reported were conducted in a laboratory environment with user interfaces and tasks, which were novel to the participants. In the third experiment, we used the same basic concept of using questionnaires immediately after a novel interaction scenario to analyse emotional user experience. However, this time we wished to test the ecological validity of our model (Neisser, 1976)  into their existing work routines in an environment familiar to them. The study was conducted as a part of an evaluation of a new gesture-based, loading-station control concept.
For the hypotheses of this experiment, we included the two from the previous experiment, but now the goal was to confirm or reject these hypotheses in the field. In addition, because now there was interest in user acceptance of the new interaction concept, two hypotheses concerning acceptance were added. The logic behind the third and fourth hypotheses is that technologies supporting competence are more easily accepted than technologies causing frustration, as is argued in studies of online-learning tools (Juutinen & Saariluoma, 2007). H1: Two emotional dimensions of user experience, separated by their valence, emerge in interaction novel user interface. These are Competence and Frustration.
H2: There is a negative correlation between Competence and Frustration. H3: Feeling frustrated during the use of novel interaction technologies decreases technology acceptance. H4: Feeling competent during the use of novel interaction technologies increases technology acceptance.

Method
A total of 21 operators and designers (19 male, two female; age range = 18-54, M age = 35.7, SD = 9.4) participated in a usability study of a loading platform controlled via gestures. A loading station is a part of a manufacturing automation system: the user of a loading station attaches raw material to a pallet in the station and sends it to the next step in the manufacturing process. A control panel is used from a safe distance to control the pallet, and the new concept evaluated in this experiment is to replace the manual controllers with gesture-based controls. All participants were familiar with the tested loading station operations, and worked at the site chosen for the user evaluations.
Eight gestures were designed to be used to fully control the normal operations involved with loading station work. Participants were asked to perform gestures in front of a loading station simulator, which consisted of a large computer screen (40 in.) and a gesture detector (Microsoft Kinect). The simulator was created in Tampere Unit for Computer-Human Interaction, and its specifics are reported by Heimonen et al. (2013). Gesture-based interaction was a novel concept in the factory environment, and the participants had no experience of using gestures to control factory automation.
After trying out the simulator with a number of example tasks, the participants completed a questionnaire with the emotion-based items. Further, to test H3 and H4, a scale of acceptance of the gesture-based interaction was included. The scale consisted of the following items used to appraise the gestures: "intelligent," "professional," "safe," and "needful. " We also asked about the physical demand of the gestures, but the responses to this question are not part of the analysis conducted here.
The data were analyzed in a similar manner as in the two experiments just reported. H1 was tested using visual inspection of MDS results, as well as calculating Cronbach's alphas for the two groups of emotions. H2 was tested, as in the second experiment, by taking the Spearman correlation of the two sum variables representing the two factors of emotional user experience. H3 and H4 were tested by correlating the two sum variables with the four items of the acceptance scale.
Finally, to explore first and the second hypotheses in more detail, a combined data set of the three experiments was constructed. Because no participant took part in more than one experiment, the data consist of 50 individual participants: 19 from the first experiment and 21 from the current, third experiment; from the second experiment, we chose the 10 responses from the second task. There are only seven emotional questionnaire items common in all three experiments, and all were included in the dataset. Principal Axis Factoring was used to confirm H1. Promax rotation, allowing correlation of the factors, was used to investigate possible correlations between Competence and Frustration.
In addition, all three experiments had a questionnaire item concerning feeling of control during the experiments. The experienced control potential is an essential part of the emotional appraisal of a situation (Roseman, 2001). Feeling of control has been studied as an important factor in flow experience, for example, and on an emotional level has been argued to be connected to enjoyment (Sweetser & Wyeth, 2005). Although feeling of control was not part of our main investigation of emotional user experience, we analyze, post hoc, the association between the Competence and Frustration factors and feeling of control. Is feeling of control positively associated with Competence and negatively associated with Frustration, and what is the strength of this association?

Results
First, the data from the field experiment alone were analyzed. The stress values for the first three iterations of the MDS of the emotion-based items were .181, .160, and .158. The configuration of the two-dimensional solution is shown in Figure 4. As in the first and second experiments, two groups of emotion-based items are observable. The sum variable for the Competence factor included pride, determination, vigilance, efficacy, success, and excellence, and had a consistency of α = .847; its mean was 2.94 (SD = 0.84). The sum variable for the Frustration factor included frustration, desperation, and anxiety. We excluded feeling excited, because it was not very closely associated with the other three variables. The reliability of the three items were α = .804, and the mean of the sum variable was 1.40 (SD = 0.51). The results confirm H1. As in the second experiment, but unlike in the first, we observed no association between the two emotion-based variables (Spearman correlation ρ = -.292, p = .199). We hence reject H2.
The summated scale for the Frustration factor had high correlation with acceptance of the gesture-based interaction concept. The Spearman correlations of Frustration and the acceptance scale items were following: intelligent, ρ = -.542, p = .011; professional, ρ = -.587, p = .005; safe, ρ = -.671, p = .001; and needful, ρ = -.509, p = .018. We accept H3: feeling frustrated during novel technology use predicts nonacceptance of the technology. However, no statistically significant correlations were observed with the summated scale of the Competence factor and the items in the acceptance scale. We reject H4.
Finally, for a deeper inspection of emotional user experience, especially regarding H1 and H2, the data of the three experiments were combined. The stress values of the MDS for the combined data set for the three experiments were .145, .125, and .123. The resulting two-dimensional configuration is displayed in Figure 5. The emotional valence is once again clearly visible, but it is now also possible to observe the second dimension, arousal. Although arousal creates weaker segregation of emotions than valence, it seems that excellence, for example, is more arousing emotion than efficacy. The dimensions of emotional user experience are now named accordingly in Figure 5.
The Promax-rotated factor matrix of the Principal Axis Factoring is displayed in Table 1. We observed the expected factors of emotional user experience, again supporting H1. These factors explained 62.0% of the total variance of the emotional questionnaire items. This, and sufficient factor loadings of the items, is taken as evidence supporting H1. Because Promax rotation allows correlation between the factors, the results displayed in Table 1 can also be used to analyze H2 in more detail. It seems that efficacy is the only item with strong negative loading on the factor of opposing valence, whereas excitement has positive correlation with both factors.
Based on factor analysis, sum variables for Competence and Frustration were calculated using a regression method, in which individual emotions contribute to the summated scale according to their factor scores. Although some items had cross-factor loadings, no correlation between Competence and Frustration was observed (ρ = -.040, p = .785). In the post hoc analysis of feeling of control, we observed, as expected, a positive correlation between feeling of control and Competence (ρ = .435, p = .002) and a negative correlation between feeling of control and Frustration (ρ = -.472, p = .001).

Discussion
The context of the experiments reported here was that of novel or complex human-technology interaction. The students in Experiments 1 and 2 were faced with unfamiliar crane operation tasks (either with miniature crane or a crane simulator) and had to use a combination of joystick and touch screen (first experiment), or a combination of two joysticks (second experiment). In the third experiment, the industrial work environment and the context of factory automation added an element of complexity, and the use of gestures to manipulate the loading station was a novel control scheme for participants. Hence we limit our discussion to novel interfaces. However, further study of competence and frustration in other contexts is warranted. Note. N = 50. Highest factor loading is in bold.
The third experiment yields the same general result as the first and second experiments by confirming H1 about the bipolar competence-frustration model of emotional user experience. However, this time the study was conducted "on the field" to test the ecological validity of our hypothetical model. Although the experiment involved using a simulated environment, the participants were familiar with the loading station used in the simulation and knew the environment in which the experiment was conducted. This allowed us to confirm that the results were not artifacts of a laboratory environment but reflect the emotional experience of novel interaction situations. In other words, we can now maintain, with strong evidence, that the scales presented for measuring Competence and Frustration have validity.
The two factors of emotional user experience were again separated by their valence. This time, the emotion-based items of the Competence factor were determination, excellence, pride, efficacy, and vigilance. Items for the Frustration factor were anxiety, desperation, and frustration. Because the experiment concerned real users of the proposed novel interaction concept, we were also interested in how the emotional user experience was associated with acceptance of the new technology. The results indicated that negative emotions, summated as the Frustration factor, were strongly associated with not accepting the proposed technology. The Competence factor, on the other hand, was not observed to have such associations. It may be possible that, whereas feeling competent is associated with own skills and capacities, feeling frustrated is more easily attributed to faults in the technology.
Minimizing frustration in interaction with new interfaces is important regardless of how competent the user feels. This is an important notion in relation to much of current user experience research, which advocates the study of mainly positive user experience (e.g., Desmet, 2012;Hassenzahl, 2008). The negative emotional user experience has been shown to have its own relevance, which cannot be reduced to nonpositive emotional user experience. The positive Competence factor and the negative Frustration factor are not necessarily correlated, and they clearly have differing associations, for example, with acceptance of new technology. Negative user experience needs to be taken explicitly into account in design.
By combining data from all three experiments, we were able to analyze H1 , that is, the competence-frustration hypothesis, in more detail. The resulting factor matrix is useful in determining how relevant the individual emotion-based items are for the user experience factors. It seems that vigilance, excellence, and determination all have very similar correlation to feeling competent, whereas the role of efficacy in the formation of selfperceived competence seems to be, although clear, not as strong. Also, H2, that is, the association between the two factors, can now be analyzed. It seems that Frustration is negatively associated with efficacy but not with other items of Competence. This is one more argument for supporting the separate treatment of positive and negative factors of emotional user experience.
Of the emotion items analyzed with the factor analysis, only excitement seems to load positively on both factors. This suggests that feeling excited during novel interaction can be either pleasant or unpleasant, depending on the other emotions experienced. This is understandable but leaves open the question of whether user experience-focused design should aim for exciting interaction. Excitement is often linked to positive user experience (Hassenzahl, 2008;Mandryk, Inkpen, & Calvert, 2006), but it seems that the case is not so straightforward. Our result also differs from the results of Hassenzahl (2008, pp. 13-14), who expect excitement to be associated with competence. Whereas we observed the positive correlation of excitement and competence suggested by Hassenzahl (2008), excitement was also observed to correlate positively with frustration. The role of excitement in emotional user experience remains, hence, to be investigated further. This line of research could also provide more detailed understanding for the notion of arousal as a dimension of emotional user experience.
In interaction with novel or complex interfaces, feeling of control is an important source of positive emotional user experience (Desmet, 2012). Our studies were not designed to investigate feeling of control, but in each of our experiments one questionnaire item was dedicated to it. Post hoc analysis suggests that feeling of control is strongly associated with emotional user experience, but more detailed investigation of this association would require its own experiments and field tests, as well as good and theoretically coherent operationalization of feeling of control. However, once again it is evident that feeling of control is related to both positive and negative user experience, and hence feelings of competence and frustration may have unique associations with it.
In further studies analysing the competence-frustration model of emotional user experience, it becomes important to fix the emotion items to those best suited for analysing the factors. The reliability of the factors in different experiments was good despite the change in emotion-based items between the experiments, but to create a consistent measurement model, the items should be fixed. We suggest that to measure the Competence factor, the four bold items in Table 1 suffice. As for the Frustration factor, anxiety and frustration are evident items. We further suggest completing this scale with annoyance and confusion, as they were most consistently associated with anxiety and frustration. The composition of the two factors of emotional user experience in the three experiments is displayed in Table 2. Further studies confirming the chosen four-item scales in different contexts are warranted, and the next step would be to show that the interface design can cause noticeable variation in emotional user experience as measured by the Competence and Frustration factors. Also of future interest is the relation between the traits of the users (such as the capacity to adapt to new technologies) and the states of emotional user experience (for a discussion on the trait/state problem of the subjective measurement of emotions, see Schorr, 2001).

GENERAL DISCUSSION
User psychology begins with the idea that humantechnology interaction phenomena can be analyzed, conceptualized, tested, understood, and explained using psychological concepts, theories, facts, and methods (Saariluoma, 2004;Saariluoma & Oulasvirta, 2010). As such, user psychology does not differ, but in its topic of research, from such established directions of psychology as child psychology (Piaget, 1970), engineering psychology (Hollands & Wickens, 2000), and clinical psychology (Beck, 1976). In the same way, user psychology sets its questions and answers in the context of general psychological knowledge and research.
On the other hand, user psychology differs from the other paradigms of user experience research, as it begins with psychological concepts, methods, and theories. The operationalization, research, and analysis of results can and should be intimately connected and explainable in terms of modern psychology (Saariluoma, 2004). In the studies reported here, we follow the user psychology methodology by conceptualizing emotional user experience in terms of mental representations. Scales for measuring emotional user experience are operationalized and formulated in concepts of established theories of emotion. As a result, the major concepts of emotional user experience are expressed psychologically and connected to general psychological knowledge.
A central theoretical concept of modern psychology has been mental representation (Newell & Simon, 1972), and its content (Fodor, 1990;Saariluoma, 2003). Experience is always with content, as experience without content could hardly be experienced. Thus, it is logical to investigate emotional user experience using the psychological concept of mental contents. The goal is to open the structure of contents and give clarity to the contents of experience as well as underlying systems of subconscious associations.
As the result of our experiments, we postulate mental representations underlying emotional user experience and concentrate on analyzing the role of basic emotions in user experience (Power & Dalgleish, 1997). The core idea of theories of basic emotions is that people have a set of basic emotions that are used to construct human emotional experience (Ekman, 1999;Oatley, 1992;Power & Dalgleish, 1997). This idea has several times been proposed in history by such thinkers as Aristotle and Darwin, although it also has been criticized as difficult to validate empirically (Ortony & Turner, 1990;Scherer, 2005). However, we contend that asking users to reflect their emotional experience, using basic emotion words, is a valid way of investigating emotional user experience. This helps in understanding the contents of constructed situations and specific emotional states, and provides a way to carry out psychologically valid user experience research.
In the spirit of basic emotion theory, all experiments result in two different clusters of constructed emotional states, which are divided by valence. In the concluding analysis, all clusters were analyzed using factor analysis, resulting in two factors of emotional user experience. The first factor, located on the positive side of the valence dimension of emotional user experience, is labeled Competence. The second factor, organized on the negative side of the valence dimension, is labeled Frustration. If an explicit emphasis on the technological nature of these factors is required, the labels Techno-competence and Techno-frustration can be used.
In our study, techno-competence was a feeling based on such basic emotions and experiences as determination, vigilance, pride, excellence, and efficacy. Together, all these emotions express a mental state that people have when they can use technology to reach what they need to reach. Overall, the valence of such a mental state is positive, because all the involved feelings are positive. Techno-frustration is not the direct opposite of techno-competence, as revealed by different correlation analyses and the concluding factor structure. Nevertheless, it is a negative state of mind characterized by such emotional states as frustration, annoyance, anxiety, confusion, and struggle. This emotional state typically expresses what people feel when they are unable to or uncertain in using technologies in a specific situation and was shown to negatively predict acceptance of new interfaces.
It is important to acknowledge the presence of negative, or unpleasant, emotions in human-technology interaction. Many sources understandably emphasise positive emotions (e.g., Desmet, 2012;Hassenzahl, 2008), but the design of technology should always take into account both pleasant and unpleasant emotions. Our model of frustration, for example, can be used to improve the content validity of studies of negative emotions during technology use. Liu et al. (2009) examined anxiety levels using both subjective measurement scales and physiological data but did not explicate their measurement of the subjective experienced anxiety. Although objective, physiological, and neuropsychological methods are increasingly important in studying emotional user experience (Mandryk et al., 2008;Pecchinenda, 2001), the results of these objective measurements need to be content validated in the context of subjective experience.
To understand the proposed competence-frustration model correctly, it is important to understand that in our experiments the participants interacted with user interfaces that were new to them. The experimental situations were organized around the issue of whether people can use the given new technologies, and what immediate emotional experiences were associated with this question. The "can" questions are currently one of the main issues in usability research and design. If technologies have poor usability, people cannot use them. Hence the extracted emotional mental contents are relevant for usability and subjects' ability to use technologies.
The deeper meaning of the competence-frustration model is that it associates two main concepts and problems of interaction research with each other. The model illustrates what kinds of emotional consequences are associated with difficulties in using technology. The user experience of poor usability activates negative emotional contents of different types, whereas good usability generates positive feelings. Our experiments thus illustrate that poor usability has essential emotional costs, and this explains why people are often poorly motivated in using technologies with poor usability or that are difficult to learn. In sum, we suggest that analysis of emotional interaction can be based on resolving the contents of total emotional states in component emotions and that this is a fruitful way of approaching user experience research. These emotional user experience factors and dimensions can then be connected to what is known about the contents of these emotional states. It is thus possible to gain a more elaborated conception of the emotional state that prevails during a human-technology interaction process.

FUNDING
This research is supported by the Finnish Funding Agency for Technology and Innovation (TEKES) and Finnish Metals and Engineering Competence Cluster (FIMECC) programme UXUS (User experience and usability in complex systems).