15. The design and usability testing for first use (out-of-box readiness) need to be included in the early product development.
management and design teams. In my experience, Usability Plan is an efficient tool for developing a well-integrated product entity that meets user needs in the most important areas.[Guidelines 7, 8, 9, 10]
With Usability Risk Management the usability engineer can predict, keep track of and communicate the emerging design problems and product challenges that may cause the failure of product in terms of user acceptance. In the case project it was observed some predicted usability risks emerging, such as too high error rates in the user interface navigation. [Guideline 11]
9.4.2 Effect in product cost
Do the proposed guidelines have an effect in product cost? Product cost is a result of several factors, such as development time and resources, manufacturing, marketing and aftersales activities, and maintenance services. Decreasing any of these areas decreases product costs.
Smart products tend to have functions that are not used, for example, because they are too difficult, or they are not needed. Still, they may be required for marketing purposes, for example. This creeping featurism (Norman 1998, 80) is typical in the introduction of new technologies. Development of any function requires specification, design and implementation, i.e. manual work and resources. By early understanding of user needs, user requirements and user testing it is possible to avoid the implementation of unnecessary functions. [Guidelines 4, 12, 13, 14]
In the case project I could detect decreased project cost elements due to planned usability engineering in:
- product development in the form of better (faster) function and technology selection and design. For example, a complex software feature related to phone set-up was not implemented at all because of the negative user feedback in the early phase of specification.
- aftersales and maintenance services13 in the form of better design of easy first use situation and product configuration. For example, new external interface concepts related to the messaging capabilities were studied and improved in parallel with the user-interface design.
9.4.3 Development time
Do the proposed guidelines help in decreasing development time? Product development time is the major motivation for applying CE. Development time in CE is dependent on the optimal implementation of milestones and success of product integration.
In the case project I was often in the situation when the design work in some design area couldn't proceed without an important selection between different design candidates, and any delay would have delayed the whole project due to design interdependencies. For example, the early selection of phone keypad design and material required user testing in order to find out which is the preferred way (design, technology) to proceed. The test need was foreseen in the Usability Plan and usability testing could be arranged without delays when it was needed for producing data. [Guidelines 7, 9, 10, 14]
13 Lano and Haughton (1992) informed that the costs of maintenance activity have been estimated as being as high as 80% of the long-term cost of developing and maintaining systems; and this proportion is rising.
9.4.4 Cost-efficiency for usability engineering
Do the proposed guidelines enable better cost-efficiency for usability engineering? Design changes in the early development phases are cheaper (time, money, resources needed) than in the late phases. Fewer and smaller changes can be implemented in the late phases than in the early phases.
Compared to earlier development projects, more cost-efficient design changes and improvements could be made in the early development, and less change needs emerged in the late phases due to proper planning and priorisation of usability activities. However, in spite of better planning the product development still contained unexpected usability activities, but which could be managed due to proper planning in other tasks. [Guidelines 7, 10, 12, 15]
9.4.5 Usable mobile phones?
The high-tech industry is in denial of a simple fact that every person with a cell phone or a word processor can clearly see: Our computerized tools are too hard to use. The software engineers who create them have tried as hard as they can to make them easy to use and they have made some minor progress. They believe that their products are as easy to use as it is technically possible to make them. As engineers, their belief is in technology, and they have faith that only some new technology, like voice recognition or artificial intelligence, will improve the user’s experience (Cooper 1999).
Do the proposed guidelines increase the ability to build usable products? In my experience, by following the guidelines it is possible to better adapt usability engineering to support the development of complex products, and to better understand the particular product entity (the user interface, the external interface, the service interface) and the user requirements.
In the case study organisation (Nokia Mobile Phones) the improvement in product usability and usability engineering competence has been very clear. Usability engineering efficiency and effectivity is clearly better in the case project (and test projects 1-3) than in the observed projects (Communicator 1-3). Level of usability engineering ([Coverage, Duration, Visibility], see Section 6.3) has improved at least from [x,y,z] to [x+2, y+1. z+1]. The future of imaging phones will show, whether the products really are usable.
9.4.6 Efficient and effective usability engineering
Do the proposed guidelines increase efficient and effective usability engineering? The terms were defined as follows:
Efficient usability engineering is to work in an effective and competent manner in the current development phase with little wasted effort. Efficiency can be measured by comparing realization (output) against usability engineering goals, plans and amount of work (input).
Effective usability engineering aims to produce an adequate or desired result. Effectiveness can be measured by studying how usable the final product is, or by analysing field feedback. During product development the effectiveness can be measured, for example, by comparing the
performed usability work against the number of usability engineering originated design improvements.
The usability engineering has been efficient in the case project and also in the followed three projects in terms of perceptual measures. It has been possible to execute Usability Plan and to perform the planned tasks. The current situation shows visible improvement to the situation 12 months earlier. Now (May 2002), at NMP many new development projects have a Usability Plan accepted by the management and implemented by usability engineers.
The usability engineering has been effective during the product development. Usability engineering originated design changes at early (before M2) development have been implemented with the rate of 60% (6 proposed changes out of 10 are implemented), and ~30%
for late changes (after M2). The first product that has been developed using these guidelines is Nokia 7650. However, in order to assess the real success of guidelines it has not been possible to evaluate the final product usability with summative testing and real field feedback because the product of this study is not yet on the market place when the writing of this thesis is completed.
9.4.7 Improvements to usability engineering lifecycles
The guidelines improve the existing usability engineering lifecycle definitions in many ways.
The main difference of proposed guidelines to lifecycle descriptions is the emphasis on product development process instead of usability processes.
The lifecycles proposed by Nielsen (1994), Mayhew (1999) and ISO 13407 are based on linear steps executed in temporal order, i.e. phases, but it has also non-linear iterative activities, such as parallel design, prototyping and iterative design. These lifecycles lack the aspect of coordination, concurrent design activities and practical constraints, such as product complexity, prototype availability, time and resources. In addition, they do not take out-of-box readiness issues, important for all communication oriented consumer products, into consideration.
9.4.8 Some thoughts on education
In the development of complex products, such as information appliances and smart phones, it is reasonable to expect that a usability engineer has some knowledge but not full expertise on current and emerging design and technology areas. Hence, the ability to apply existing usability methods and coordinate the work becomes crucial. The former is not anything new, meanwhile the coordination of usability work is not well covered or studied.
As we have seen, CE, especially in the development of innovative and usable mobile phones, requires knowledge and know-how from human-factors and usability disciplines as well as awareness of project management methods. This should be understood in the training and education of usability engineers.