A Gamified Requirements
Inspection Process for Goal Models
João Pimentel Emanuel Santos Tarcisio Pereira Daniel Ferreira Jaelson Castro
that’s me
[ACM SAC 2018 – RE Track]
2
1) Context and Motivation 2) Proposal (gamified process)
3) Evaluation
[agenda]
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models
Inspection is an effective verification approach
Bosu, A., Greiler, M., & Bird, C. (2015, May). Characteristics of useful
code reviews: An empirical study at microsoft. In Mining Software
Repositories (MSR), 2015 IEEE/ACM 12th Working Conference on
(pp. 146-156).
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 4
Bell, T. E., & Thayer, T. A. (1976, October). Software requirements:
Are they really a problem?. In Proceedings of the 2nd international
conference on Software engineering (pp. 61-68). IEEE Computer
Society Press.
2015
Survey with 247 practitioners
55% of participants declared to perform some form of
requirements review
Kassab, M. (2015, August). The changing landscape of requirements engineering practices over the past decade.
In Empirical Requirements Engineering (EmpiRE), 2015 IEEE Fifth International Workshop on.
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
i*
iStar
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 6
Is there an inspection process for i* goal
models?
Is there a inspection
process for goal models?
8
NO *
*to the best of our knowledge
Ring-i
Requirements Inspection Gamified process for i* Models
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 10
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition
link
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHREQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 12
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHOnline Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHREQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 14
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHGOAL
TASK ACTOR
SOFTGOAL
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHREQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 16
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHOnline Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHREQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 18
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHOnline Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISHREQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 20
Online Store
Shopping Cart
Security Internet Orders
Handled
Pay Add Item
Goal Task
Actor Softgoal
Legend
Means-
end link Decomposition link
1
3
2
4
5
6
FINISH123 different cards Based on
Literature Interviews
Protocol analysis
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 22
Some cards are not
actual ‘inspection’
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 24
● Incomplete coverage
● Not every element is inspected
● Not every inspection heuristic is applied
● Does it make sense to continue after a big change?
[agenda]
Empirical Evaluation
26
18 students 5 groups
i*
training i* project Ring-i session
Questionaire
Able to execute the process
without training Perceived as useful
Perceived as fun
Perceived as easy to use
Would recommend
Would use it again?
REQUIREMENTS ENGINEERING TRACK - A Gamified Requirements Inspection Process for Goal Models 28
EASE OF USE
FUN ACCEPTANCE
USEFULNESS
Limitations of this evaluation
● Students as subjects
● Small sample size
● Lack of control group for comparison
30
Future work &
conclusion
Future work
● Expand and update the inspection cards
● Test different game mechanics (e.g. RPG)
● Validate with industry and i* experts
● Create a non-gamified version
32
33
RE goal models inspection Ring-i
Artefacts: www.cin.ufpe.br/~ler/istargame Requirements inspection
[context]
[evaluation]
[proposal]
Thank you!
34
Analyze the Ring-i process;
for the purpose of evaluation and improvement;
with respect to its use by non-experts on i*;
from the point of view of software engineers;
in the context of students applying the process on their
own projects.
Distribution of changes resulting from the Ring-i process
36
Questionnaire answers regarding error discovery
Questionnaire answers regarding improvements to the model
38
Questionnaire answers on whether respondents learned more about i* with the
process
Questionnaire answers regarding usefulness, directly
40
Questionnaire answers regarding process enjoyment
Questionnaire answers regarding ease of use, directly
42
Questionnaire answers regarding ease of use, indirectly
Questionnaire answers regarding acceptance of the proposal
44
