Which aspects have to be taken into consideration in choosing elicitation techniques for a product data management system?

III.A Requirements to Elicit

One of the most important goals of elicitation is to find out what problem needs to be solved, and hence identify system boundaries. These boundaries define, at a high level, where the final delivered system will fit into the current operational environment. Identifying and agreeing a system's boundaries affects all subsequent elicitation efforts. The identification of stakeholders and user classes, of goals and tasks, and of scenarios and use cases all depend on how the boundaries are chosen.

Identifying stakeholders—individuals or organizations who stand to gain or lose from the success or failure of a system—is also critical. Stakeholders include customers or clients (who pay for the system), developers (who design, construct, and maintain the system), and users (who interact with the system to get their work done). For interactive systems, users play a central role in the elicitation process, as usability can only be defined in terms of the target user population. Users themselves are not homogeneous, and part of the elicitation process is to identify the needs of different user classes, such as novice users, expert users, occasional users, disabled users, and so on.

Goals denote the objectives a system must meet. Eliciting high-level goals early in the development process is crucial. However, goal-oriented requirements elicitation is an activity that continues as development proceeds, as high-level goals (such as business goals) are refined into lower-level goals (such as technical goals that are eventually operationalized in a system). Eliciting goals focuses the requirements engineer on the problem domain and the needs of the stakeholders, rather than on possible solutions to those problems.

It is often the case that users find it difficult to articulate their requirements. To this end, a requirements engineer can resort to eliciting information about the tasks users currently perform and those that they might want to perform. These tasks can often be represented in use cases that can be used to describe the outwardly visible interactions of users and systems. More specifically, the requirements engineer may choose a particular path through a use case, a scenario, in order to better understand some aspect of using a system.

III.B Elicitation Techniques

The choice of elicitation technique depends on the time and resources available to the requirements engineer, and of course, the kind of information that needs to be elicited. We distinguish a number of classes of elicitation technique:

Traditional techniques include a broad class of generic data gathering techniques. These include the use of questionnaires and surveys, interviews, and analysis of existing documentation such as organizational charts, process models or standards, and user or other manuals of existing systems.

Group elicitation techniques aim to foster stakeholder agreement and buy-in, while exploiting team dynamics to elicit a richer understanding of needs. They include brainstorming and focus groups, as well as RAD/JAD workshops (using consensus-building workshops with an unbiased facilitator).

Prototyping has been used for elicitation where there is a great deal of uncertainty about the requirements, or where early feedback from stakeholders is needed. Prototyping can also be readily combined with other techniques, for instance, by using a prototype to provoke discussion in a group elicitation meeting, or as the basis for a questionnaire or think-aloud protocol.

Model-driven techniques provide a specific model of the type of information to be gathered, and use this model to drive the elicitation process. These include goal-based and scenario-based methods.

Cognitive techniques include a series of techniques originally developed for knowledge acquisition for knowledge-based systems. Such techniques include protocol analysis (in which an expert thinks aloud while performing a task, to provide the observer with insights into the cognitive processes used to perform the task), laddering (using probes to elicit structure and content of stakeholder knowledge), card sorting (asking stakeholders to sort cards in groups, each of which has the name of some domain entity), repertory grids (constructing an attribute matrix for entities, by asking stakeholders for attributes applicable to entities and values for cells in each entity).

Contextual techniques emerged in the 1990s as an alternative to both traditional and cognitive techniques. These include the use of ethnographic techniques such as participant observation. They also include ethnomethodogy and conversation analysis, both of which apply fine-grained analysis to identify patterns in conversation and interaction.

III.C The Elicitation Process

With a plethora of elicitation techniques available to the requirements engineer, some guidance on their use is needed. Methods provide one way of delivering such guidance. Each method itself has its strengths and weaknesses, and is normally best suited for use in particular application domains.

Of course, in some circumstances a full-blown method may be neither required nor necessary. Instead, the requirements engineer needs simply to select the appropriate technique or techniques most suitable for the elicitation process in hand. In such situations, technique-selection guidance is more appropriate than a rigid method.

10.2.3.2.1 Objectives

The requirement analysis phase aims to collate all stakeholders’ requirements. Note that the analysis phase enables us to identify the limits of the product (see Figure 10.9) and characterize interfaces with other products.

Figure 10.9 shows the environment of a system and/or application that comprises three inputs (Ei), two outputs (Sj) and three interfaces (Ik) with existing means (reused systems, power supply, etc.).

The requirements of stakeholders form the basis of whether or not a system is accepted, negotiation and agreement on the project, system development and management of changes of requirements. The requirements define the result as expected by stakeholders; therefore, it is necessary to obtain requirements to the best possible ability.

The following must be identified within the system:

interfaces with the environment (see Figure 10.9); these interfaces can be electrical, mechanical, software-related, pneumatic, etc.

states: stopped, running, degraded state, etc. (see, for example, Figure 10.10). The concept of state introduces partition between proper functioning, fall-back states and hazardous states;

the concept of correct behavior, degraded behavior and dangerous behavior;

the concept of functional and non-functional requirement.

With regard to the system states, Figure 10.10 identifies the correct and incorrect states, but we must go further and introduce all the states attainable by the system that characterize specific behaviors (fall-back, maintenance, degradation, etc.) as shown in Figure 10.11.

During the elicitation phase, it is necessary to consider the non-functional requirements and implement analyses related to dependability, to define the safety requirements but also the availability, reliability and maintainability requirements.

Figure 10.12 shows a process taking into account elicitation of non-functional requirements related to RAMS.

10.2.3.2.2 Elicitation techniques

The objective of elicitation techniques is to help discover conscious, unconscious and subconscious requirements of the stakeholders. These techniques are selected based on risk factors, human and organizational constraints, business sector and the expected level of detail for requirements. Elicitation techniques can also be selected according to the requirement document to be drafted.

Based on the list of stakeholders and sources, it is necessary to establish a requirement acquisition process. There are various elicitation processes such as [ZOW 05]:

investigative techniques: surveys, questionnaires, etc.;

interview techniques;

creative techniques: brainstorming, storyboarding;

animation techniques: brown paper, role-plays, use cases;

observation techniques: field, learning;

prototyping and simulation techniques.

In appendix A of Meinadier [MEI 02], the author presents the methodological aspects related to requirements engineering. Requirements engineering is a tool that system engineering offers to implement as shown in the standard EIA-632 [EIA 98].

The best result is achieved when the analyst implements it along with several of these techniques.

As it is not possible to state all these techniques in an exhaustive manner in this book, in the following subsections we shall introduce only two techniques; however, the identified technology is implemented with respect to the project.

10.2.3.2.3 Interview techniques

An interview consists of the following steps:

Questioning: questions are asked and answers are taken into account.

Pause: during breaks, most people will find something to say and/or explain, which helps to capture additional requirements.

Summary and/or reformulation phase: this phase is important because it enables to verify the understanding of the answers.

Interviewing involves a preparation phase, where a set of questions is prepared. This set of questions should identify the requirements of different stakeholders. This set of questions includes:

Open questions: they require answers other than yes or no;

Closed questions: they elicit yes or no answers. Closed questions are used to obtain a definitive answer (after reformulation, for example).

Note that during the interview, it is possible to add questions based on the responses and after the reformulation of answers step.

All the identified stakeholders must be interviewed and they must be made aware that their requirements must be considered as a whole (system). It is essential to take stakeholders seriously and not to pass judgment on the expressed requirements. During interviews, it is necessary to process all elements as requirements, document the importance of the requirements for each stakeholder, documenting the results of interviews and formalizing the acceptance of stakeholders of the interview results (notes, documents, etc.).

Conducting interviews requires the skill to stimulate and encourage stakeholders to respond. The person in charge of interviews must have strong abilities to manage a discussion and talk while being able to organize pauses.

10.2.3.2.4 Prototyping and simulation techniques

A model is the initial simplified modeling of a problem. A model helps in modeling certain aspects of the problem to be resolved in a more or less accurate manner. When the model handles the actual elements of the system (such as data files, etc.), it is called prototype.

Models and/or prototypes are a way to confront the vision of various stakeholders and expected behaviors (see Figure 10.13).

A prototype can be static, where we try to model the interactions between various elements, including actors, but it can also be dynamic, where simplified behaviors are modeled, and it is possible to run scenarios.

Developing a prototype is a good way to facilitate understanding of requirements, but this involves costs, and often the prototype is considered as the beginning of a solution.

It must be noted that, nowadays, more and more prototypes are used to validate a concept (website, etc.). The challenge with a prototype is not to lose sight that it is a prototype (which can be extremely successful) and that it is not the design of the final product.

2.5.1 Framework for product line practice

The framework for product line practice is based on patterns that have been traditionally used to solve recurrent problems in software engineering. For example, design patterns have been used for designing and implementing various software modules. The framework uses patterns to solve the product line engineering problems at three different levels:

The context is the organizational situation.

The problem is what part of a software product line effort needs to be accomplished.

The solution is the grouping of practice areas and the relations among them that together address the problem for that context.

Overall, the framework uses 12 different patterns (Clements and Northrop, 2002). For example, the Essentials Coverage pattern gives a mapping of each practice area to each of the three essential product line activities: core asset development, product development, and management. The Product Builder pattern consists of practice areas that should be used whenever any product in the product line is being developed. The economic models discussed in the previous section could be applied for relevant patterns. For example, SIMPLE economic model can be applied for calculating ROI for developing new products in the Product Builder pattern.

2.5.2 Feature models enriched with assumptions

Design decisions often determine the development, deployment, and evolution of a system and provide the earliest place for assessing architecture. Lago and van Vliet (Lago and van Vliet, 2005) model assumptions to reason about variability and invariability as well as architectural design decisions. The authors argue that enriching variability with assumptions provides a more complete picture of the SPL. For example, the assumption “modularity is business driven” creates variability by impacting the design of the functionality for common services and service specific functionality.

The approach (Lago and van Vliet, 2005) models relationships between features and assumptions. For example, an assumption can impact a feature. Similarly, a feature can realize an assumption. Modeling assumptions provides several advantages.

Assumptions provide a strong link between requirements and design. Therefore, modeling assumptions improves forward and backward traceability.

Hinders implementation of changes that go against assumptions made at architecting time.

Assumptions are useful for improving management of architectural knowledge. For example, externalizing the documentation assumptions is helpful in their reuse for multiple projects.

2.5.3 Value-based elicitation of variability

Bridging business and technology is receiving increasing attention in the software engineering community. For example, value-based software engineering (VBSE) (Biffl et al., 2005) aims to associate a value (e.g., business value) for artifacts. In this context, value-based variability modeling would mean considering the business value and the associated risks during decision making on variability.

Extracting tacit variability knowledge about product line architecture is a collaborative process (Dhungana et al., 2006). In particular, software engineers/architects involved in developing the reusable assets as well as people from business and marketing (e.g., people involved in selling these assets) are in the process.

The value-based process (Rabiser et al., 2008) for eliciting product line variability aims at integrating three research areas:

SPLE, with a focus on variability modeling and management

VBSE, particularly the question how much is enough in variability modeling?

Collaborative software engineering (Finkelstein et al., 2010) with patterns of collaboration that enable different people working together to produce mutually satisfactory results

The value-based elicitation process (Rabiser et al., 2008) was implemented using structured workshops. Based on the workshops, various lessons were learned. Here are the relevant findings for software architecture:

Different levels of variability decisions are identified, which include decisions on customer and sales, systemwide, subsystemwide, component level, and low-level parameterization

Complementary results with variability recovery tools such as parsers analyzing existing architecture models and configuration files

2.5.4 Issue-based variability management

Issue-based variability management methodology (IVMM) (Thurimella and Bruegge, 2012) supports decision making on variability management based on concepts such as questions, options, and criteria (QOC). In addition, the methodology enables capture of the rationale behind variability decisions. In the IVMM meta-model, a criterion is modeled based on a business goal (e.g., price) or a nonfunctional requirement (e.g., maintainability). Therefore IVMM relates value-based considerations to variability at the modeling level. IVMM supports decision making on variability in three different use cases:

UC1: Variability identification: For deciding on the creation of variation points, stakeholders evaluate possible variations against a set of criteria. In particular, stakeholders collaborate and decide on mandatory, optional, and alternative variations.

UC2: Variability instantiation: Instantiating variability models involves selecting a subset of variants for a new product of a product line. For instantiating variation points, stakeholders evaluate the possible options of a variability model against product-specific quality concerns. For example, stakeholders would be able to reason about the value of a feature for a particular market segment.

UC3: Variability evolution: Stakeholders evaluate alternative change requests for changing assets and reason about the variability changes.

IVMM allows capture of rationale during the decision-making process. The captured rationale information is reused to solve similar issues that occur during variability management. For example, a variation point is instantiated for multiple products of a product line. The tacit knowledge that is captured while instantiating a variation point for a product could be reused for instantiating the same variation point for other products. Similarly, the tacit knowledge captured is also reused for the evolution of variability models.

2.5.5 Value-based portfolio optimization

Value-based portfolio optimization (Muller, 2011) addresses the problem of identifying and prioritizing the most important features to be realized. In particular, the approach portfolio optimization (Muller, 2011) addresses the problem by applying an optimization technique called simulated annealing. The basic idea is to understand how the entities such as features, assets, customers, and market segments affect profits. Based on this idea, the approach models a mathematic utility function for profits with the dependent variables, such as customers, market segments, features, and their costs. Based on the mathematical optimization, the approach suggests

the features that should be implemented and

the starting price for the products.

2.5.6 Discussion and summary

The Framework for Product Line Practice (Clements and Northrop, 2002) addresses product line architecture issues based on patterns. Here, architects have the possibility to apply a set of patterns that give value and economic benefits to the organization.

Identifying the value for creating variation points and making the value explicit are other important topics. Value-based elicitation adds value to artifacts (Biffl et al., 2005) during variability elicitation. Value-based portfolio optimization (Muller, 2011) helps to decide on features to be implemented and their initial price. This technique uses mathematical optimization.

Architectural knowledge management is a related area. Issue-based variability management (Thurimella and Bruegge, 2012) supports decision making and rational management during the identification, instantiation, and evolution of variability. The approach can incorporate value-based criteria. Enriching feature models with assumptions (Lago and van Vliet, 2005) provides value by improving traceability, preventing unanticipated changes, and aiding architectural knowledge management.

20.4.1 Emotion-elicitation methods

The difficulty of obtaining emotionally expressive data is a well-known problem in the field of automatic emotion recognition. Researchers have adopted various methods that allow for different level of control and naturalness for eliciting emotions, such as acting, emotion-induction using images, music, or videos, and interactions (human–machine interactions, human–human conversations).

One of the earliest emotion-elicitation method was to ask actors to perform using stimuli with fixed lexical content [12,37] or to create posed expressions [50,62,133,134]. One of the advantages of this approach is that researchers have more control over the microphone and camera positions, lexical content, and the distributions of emotions. A criticism of acted or posed emotion is that there are differences between spontaneous and acted emotional displays [19,122].

However, many of these criticisms can be mitigated by changing the manner in which acted data are collected. Supporters have argued that the main problem may not be the use of actors themselves, but the methodologies and materials used in the early acted corpora [15]. They argue that the quality of acted emotion corpora can be enhanced by changing the acting styles used to elicit the data and by making a connection to real-life scenarios, including human interaction. Recent acted emotion datasets have embraced these alternative collection paradigms and have moved towards increased naturalness and subtlety [14,17]. One example is the popular audiovisual emotion corpus, Interactive Emotion Motion Capture (IEMOCAP), which was collected by recruiting trained actors, creating contexts for the actors using a dyadic setting, and allowing for improvisation [14]. The same elicitation strategy has been successfully employed in the collection of the MSP-Improv corpus [17].

Various methods have been proposed for eliciting spontaneous emotional behavior. One popular approach is the passive emotion-induction method [54,58,100,111,121,129,131]. The goal is to cause a participant to feel a given emotion using selected images, music, short videos, or movie clips known a priori to induce certain emotions. This method is more suitable for analysis of modalities aside from audio, because the elicitation process often does not involve active verbal responses from the participants. A more active version is induction by image/music/video using emotion-induction events. For example, Merkx et al. used a multi-player first-person shooter computer game for eliciting emotional behaviors [69]. The multi-player setting of the game encouraged conversation between players, and the game was embedded with emotion-inducing events.

Another method often adopted by audiovisual emotion corpora is emotion-elicitation through interaction, including human–computer interaction and human–human interaction. For example, participants were asked to interact with a dialog system [57,59]. The FAU-Aibo dataset collected the interactions between children and a robot dog [6]. The Sensitive Artificial Listener (SAL) portion of the SEMAINE dataset used four agents with different characters played by human operators for eliciting different emotional responses from the participants [67]. Examples of human–human interaction includes interviews of topics related to personal experience [91] and collaborative tasks [88].

An emerging trend is that researchers are moving emotional data collections outside of laboratory settings, a paradigm referred to as in the wild. For example, the Acted Facial Expressions In The Wild (AFEW) dataset selected video clips from movies [26], the Vera am Mittag (VAM) dataset [40] and the Belfast Naturalistic dataset [110] used recordings from TV chat shows. User-generated content on social networks has become another source for creating text-emotion datasets. Researchers curate the contents published on social networks, such as blogs, Twitter, and Facebook for emotion detection [86,87,89].

Finally, researchers use combinations of known “best practices” for collecting behaviors associated with different emotions. For example, in the BP4D-Spontaneous dataset, the spontaneous emotion of the participants was using a series of tasks, including interview, video-clip viewing and discussion, startle probe, improvisation, threat, cold pressor, insult, and smell [139]. These tasks were intended to induce happiness or amusement, sadness, surprise or startle, embarrassment, fear or nervous, physical pain, anger or upset, and disgust, respectively.

20.4.2 Data annotation tools

Emotion annotation is a critical component of emotion recognition. Yet, it is time-consuming and potentially error-prone [136]. Researchers have developed several tools designed to improve the quality of emotion labels and to reduce the annotators' cognitive load. These tools are generally designed to annotate either self-report or perceived emotion (see Table 20.1), but they can be used in either context.

Table 20.1. Common emotion annotations tools. Label type: type of label originally designed to collect (SR: self-report; PE: perceived emotion); Emotion descriptor: the type of emotion descriptor (i.e., categorical or dimensional); Verbal description: if the tool includes verbal description; Graphical illustration: if the tool uses graphical illustration; Continuous: if the tool is used for collect time-continuous labels

Self-Assessment Manikins (SAMs) [55] belong to the most widely used tools for collecting dimensional emotion descriptions. SAMs are a pictorial grounding of N-point Likert scales. The common set is across three dimensions: valence (positive vs. negative), activation/arousal (calm vs. excited), and dominance (dominant vs. submissive). SAMs reduce variability in emotion annotation [39] and make it easier to obtain evaluations across cultures and languages [72].

The Geneva Emotion Wheel (GEW), originally proposed in [104] and updated to the most recent version in [105], is available in several languages. GEW arranges twenty emotion families according to their levels of valence and activation, with an intensity scale for each emotion family. GEW effectively combines categorical and dimensional descriptions of emotion and enables annotation using a wide range of emotions, together with intensity.

The Multi-level Emotion and Context Annotation Scheme (MECAS) is a hierarchical framework for collecting categorical emotion labels [25]. In this scheme, a set of emotion labels is organized into different layers of granularity (e.g., coarse-grained and fine-grained). This hierarchy mitigates challenges due to the scarcity of the fine-grained emotions.

Feeltrace is a tool that permits continuous annotation of emotion on the valence-activation space [22]. It provides visual feedback to the annotators by displaying a trace of their ratings. A more general version of Feeltrace called “General trace” was proposed in [23]. It allows people to create their own dimensions. The EMuJoy [74] is a similar approach to annotation that has been used in the music community.