Departmento de Informática e Matemática Aplicada Programa de Pós-Graduação em Sistemas e Computação
Doutorado em Sistemas e Computação
Identifying and Analyzing Java Exception
Handling Guidelines: A Developers
Point-of-View
Hugo Faria Melo
Natal-RN November 2019
Identifying and Analyzing Java Exception Handling
Guidelines: A Developers Point-of-View
Tese de Doutorado apresentada ao Programa de Pós-Graduação em Sistemas e Computa-ção do Departamento de Informática e Mate-mática Aplicada da Universidade Federal do Rio Grande do Norte como requisito parcial para a obtenção do grau de Doutor em Sis-temas e Computação.
Linha de pesquisa: Engenharia de software
Supervisor
Dr
a. Roberta Coelho
PPgSC – Programa de Pós-Graduação em Sistemas e Computação DIMAp – Departamento de Informática e Matemática Aplicada
CCET – Centro de Ciências Exatas e da Terra UFRN – Universidade Federal do Rio Grande do Norte
Natal-RN Novembro 2019
Melo, Hugo Faria.
Identifying and analyzing Java exception handling guidelines: a developers point-of-view / Hugo Faria Melo. - 2019.
143f.: il.
Tese (Doutorado) - Universidade Federal do Rio Grande do Norte, Centro de Ciências Exatas e da Terra, Programa de Pós-Graduação em Sistemas e Computação. Natal, 2019.
Orientadora: Roberta Coelho. Coorientador: Christoph Treude.
1. Computação - Tese. 2. Java exception handling - Tese. 3. Exception handling design - Tese. 4. Qualitative study - Tese. I. Coelho, Roberta. II. Treude, Christoph. III. Título.
RN/UF/CCET CDU 004
Catalogação de Publicação na Fonte. UFRN - Biblioteca Setorial Prof. Ronaldo Xavier de Arruda - CCET
“Identificando e Analisando as Práticas adotadas para Tratamento de Exceções Java: Uma Perspectiva dos Desenvolvedores”
Esta Tese foi julgada adequada para a obtenção do título de doutor em Ciência da Computação e aprovada em sua forma final pelo Programa de Pós-Graduação em Sistemas e Computação do Departamento de Informática e Matemática Aplicada da Universidade Federal do Rio Grande do Norte.
__________________________________________________________ Dra. ROBERTA DE SOUZA COELHO (UFRN)
Orientadora
____________________________________________________________ Profª. Drª. Anne Magaly de Paula Canuto
Coordenadora do PPgSC
Banca Examinadora
_______________________________________________________________ Dr. CHRISTOPH TREUD (UA)
Examinador Externo
_______________________________________________________________ Dr. EIJI ADACHI MEDEIROS BARBOSA (UFRN)
Examinador Externo
_____________________________________________________________ Dr. FERNANDO JOSÉ CASTOR DE LIMA FILHO (UFPE)
Dr. RODRIGO BONIFÁCIO DE ALMEIDA (UnB) Examinador Externo
_______________________________________________________________ Dr. UIRÁ KULESZA (UFRN)
Examinador Interno
____________________________________ Discente: HUGO FARIA MELO
Guidelines: A Developers Point-of-View
Author: Hugo Faria Melo Supervisor: Dra. Roberta Coelho
Co-Supervisor: Dr. Christoph Treude
Abstract
The exception handling mechanism is a feature present in most modern programming languages which helps the development of fault tolerant systems. Despite being an old feature of the Java language, developers still struggle to use exception handling for even the most basic problems. Although the exception handling of a system is essentially a design problem, few works are intended to investigate Java exception handling from the developers’ point of view. In this thesis we explore the decisions made and solutions adop-ted by Java developers for exception handling in their projects. In total we conducadop-ted 6 studies, which consulted a total of 423 developers, including interviews and surveys, and analyzed the source code of 240 Java projects hosted on GitHub. Our results show that decisions regarding Java exception handling are not usually documented, and sometimes not even discussed verbally among the development team; that developers believe their code follows the adopted solutions; that developers learn about exception handling solu-tions through informal meetings and code inspection; that the adopted solusolu-tions in the project are verified in the source code through code review. We analyzed Java source code from 240 projects to verify compliance of 6 of the 31 Java exception handling solutions we identified, and found that the code often fails to deliver what was planned. Our rese-arch reveals a weakness in the design, implementation, and verification of Java exception handling that will help researchers and the community to design tools and other solutions that help developers to apply exception handling effectively.
tratamento de exceções em Java: uma perspectiva dos
desenvolvedores
Autor: Hugo Faria Melo Orientador(a): Dra. Roberta Coelho Co-Orientador: Dr. Christoph Treude
Resumo
O mecanismo de tratamento de exceções é um recurso presente na maioria das linguagens de programação modernas que auxilia o desenvolvimento de sistemas tolerantes à falhas. Apesar de ser um recurso já antigo da linguagem Java, os desenvolvedores ainda sentem dificuldades em usar o tratamento de exceções até para os problemas mais básicos. Apesar do tratamento de exceções de um sistema ser essencialmente um problema de design, pou-cos trabalhos se propõem a investigar o tratamento de exceções Java do ponto de vista dos desenvolvedores. Nesta tese nós exploramos as decisões tomadas e soluções adotadas por desenvolvedores Java para o tratamento de exceções em seus projetos. Ao todo realizamos 6 estudos, que consultaram um total de 423 desenvolvedores, entre entrevistas e surveys, e analisou o código-fonte de 240 projetos Java hospedados no GitHub. Nossos resultados mostram que as decisões relacionadas ao tratamento de exceções Java não costumam ser documentadas, e por vezes não são sequer discutidas verbalmente entre a equipe de de-senvolvedores; que os desenvolvedores acreditam que seus código-fonte seguem as soluções adotadas; que os desenvolvedores aprendem sobre as soluções para tratamento de exceções através de reuniões informais e inspeção de código; que as soluções adotadas no projeto são verificadas no código-fonte através de revisão de código. Analisamos código-fonte Java de 240 projetos para verificar o cumprimento de 6 das 31 soluções para o tratamento de exceções Java que identificamos, e constatamos que muitas vezes o código não cumpre o pretendido. Nossa pesquisa revela uma fragilidade na adoção de orientações para o tra-tamento de exceções em Java, que ajudará pesquisadores e comunidade na elaboração de ferramentas e outras soluções que auxiliem desenvolvedores a aplicarem o tratamento de exceções com eficácia.
1 Research design overview. . . p. 24 2 Java exception handling hierarchy. . . p. 28 3 Example of Java exception handling model. . . p. 29 4 Survey findings related to RQ1. . . p. 43 5 Survey findings related to RQ2. . . p. 44 6 Survey findings related to RQ3. . . p. 45 7 Experience in software development. . . p. 62 8 Agreement with “While developing, I dedicate part of my time to read,
write, or think about exception handling.” . . . p. 62 9 Exception Handling Guidelines. . . p. 64 10 Strategies to disseminate Exception Handling Guidelines. . . p. 76 11 Strategies to verify compliance of Exception Handling Guidelines and
source code. . . p. 77 12 Exception handling guidelines adoption by developers. . . p. 88 13 Explicitness of exception handling guidelines. . . p. 89 14 Benefits of Log the exceptions guideline. . . p. 90 15 Benefits of Handle in a specific layer guideline. . . p. 92 16 Benefits of Delegate to dedicated handlers guideline. . . p. 94 17 Benefits of Define an exception super type guideline. . . p. 96 18 Benefits of Define few exceptions guideline. . . p. 98 19 Benefits of Convert checked exceptions to runtime exceptions guideline. p. 101 20 Benefits of Convert checked exceptions to runtime exceptions guideline. p. 104
22 strategies to disseminate exception handling guidelines. . . p. 107 23 Inspecting your own code manually strategy to check compliance of
ex-ception handling guidelines within source code. . . p. 108 24 Manually peer reviewing other’s code strategy to check compliance of
exception handling guidelines within source code. . . p. 108 25 Writing tests to exercise exception handling code strategy to check
com-pliance of exception handling guidelines within source code. . . p. 109 26 Automated code inspection strategy to check compliance of exception
handling guidelines within source code. . . p. 109 27 Exception Handling Constructs Model. . . p. 121 28 Projects’ conformance rate toward Log the exceptions guideline. . . p. 125 29 Projects’ conformance rate toward Handle in a specific layer guideline. . p. 127 30 Projects’ conformance rate toward Define an exception super type. . . . p. 128 31 Projects’ conformance rate toward Define few exceptions guideline. . . p. 129 32 Projects’ conformance rate toward Convert checked exceptions to
run-time exceptions guideline. . . p. 130 33 Projects’ conformance rate toward Convert external exceptions to
do-main exceptions guideline. . . p. 132 34 Comparison between conformance rate of projects who follow and who
don’t follow the Log the exceptions guideline. . . p. 133 35 Comparison between conformance rate of projects who follow and who
don’t follow the Handle in a specific layer guideline. . . p. 133 36 Comparison between conformance rate of projects who follow and who
don’t follow the Define an exception super type guideline. . . p. 134 37 Comparison between conformance rate of projects who follow and who
don’t follow the Define few exceptions guideline. . . p. 135 38 Comparison between conformance rate of projects who follow and who
don’t follow the Convert external exceptions to domain exceptions
1 Summary of analyzed sources and practices . . . p. 32 2 Best practices for Java exception handling . . . p. 33 3 Developers’ experience in software development . . . p. 36 4 Interview script . . . p. 38 5 Survey questions . . . p. 39 6 Interview questions . . . p. 50 7 Survey questions . . . p. 60 8 Existence of guidelines related to exceptions . . . p. 63 9 Exception handling guidelines used in validation survey . . . p. 85 10 Validation survey questions . . . p. 86 11 Developers’ experience in software development . . . p. 87 12 Summary of analyzed projects . . . p. 115 13 Summary of extracted data to check conformance of Log the exceptions
guideline. . . p. 125 14 Summary of extracted data to check conformance of Handle in a specific
layer guideline. . . p. 126 15 Summary of extracted data to check conformance of Define an exception
super type. . . p. 127 16 Summary of extracted data to check conformance of Define few
excepti-ons guideline. . . p. 128 17 Summary of extracted data to check conformance of Convert checked
1 Introduction p. 21 1.1 Problem . . . p. 22 1.2 Research questions . . . p. 22 1.3 Research Design . . . p. 23 1.4 Thesis Organization . . . p. 26 2 Background p. 27
2.1 Java Exception Handling Mechanism . . . p. 27
3 Study A - Identifying best practices for Java Exception Handling p. 31 3.1 Method . . . p. 31 3.2 Findings . . . p. 32 3.3 Threats to Validity . . . p. 34 3.3.1 Analyzed sources . . . p. 34 3.4 Final considerations . . . p. 34
4 Study B - How developers deal with exceptions: a single company
study p. 35
4.1 Method . . . p. 36 4.1.1 Participants . . . p. 36 4.1.2 Interviews . . . p. 36 4.1.3 Validation survey questionnaire . . . p. 37 4.1.4 Data analysis . . . p. 37
4.2.1 Interviews findings . . . p. 40 4.2.1.1 RQ1: What are the benefits and harms of using the
ex-ception handling mechanism embedded in Java? . . . . p. 40 4.2.1.2 RQ2: What are the obstacles faced by developers while
writing exception handling code? . . . p. 41 4.2.1.3 RQ3: Do developers care about exception handling code? p. 42 4.2.2 Validation survey findings . . . p. 42
4.2.2.1 RQ1: What are the benefits and harms of using the
ex-ception handling mechanism embedded in Java? . . . . p. 42 4.2.2.2 RQ2: What are the obstacles faced by developers while
writing exception handling code? . . . p. 43 4.2.2.3 RQ3: Do developers care about exception handling code? p. 44 4.3 Discussions . . . p. 45 4.3.1 Interviews vs survey findings . . . p. 45 4.3.2 A worthy investment . . . p. 46 4.3.3 Education and training vs knowledge . . . p. 46 4.4 Threats to validity . . . p. 46 4.5 Final considerations . . . p. 46
5 Study C - How developers deal with exceptions: a multi-company
interview-based study p. 48 5.1 Method . . . p. 49 5.1.1 Participants . . . p. 49 5.1.2 Interviews . . . p. 49 5.1.3 Data analysis . . . p. 51 5.2 Findings . . . p. 51 5.2.1 RQ1: What guidelines to exception handling do developers adopt? p. 52
5.2.1.2 Propagate the exception to a specific layer . . . p. 52 5.2.1.3 Wrap exceptions . . . p. 52 5.2.1.4 Throw an exception related to the failure or component p. 52 5.2.1.5 Avoid throwing and catching of generic exceptions . . . p. 52 5.2.1.6 Give good feedback to user . . . p. 53 5.2.1.7 Use only checked exceptions / use only unchecked
ex-ceptions . . . p. 53 5.2.1.8 Use distinct exception to each component or layer . . . p. 53 5.2.1.9 Define one or few exceptions to the project . . . p. 53 5.2.2 RQ2: What are the origin of these guidelines? . . . p. 53 5.2.3 RQ3: What strategies are used to disseminate the
knowledge about these guidelines across the development team? p. 54 5.2.4 RQ4: What strategies are used to verify the conformance of the
source code with the adopted guidelines? . . . p. 54 5.3 Discussions . . . p. 55 5.3.1 Reasons behind adopted guidelines . . . p. 55 5.3.2 About guidelines to exception handling and dissemination and
verification strategies . . . p. 55 5.4 Threats to validity . . . p. 56 5.5 Final considerations . . . p. 57
6 Study D - What are the Java exception handling guidelines GitHub developers adopt in their projects, and how they disseminate and
verify these guidelines. p. 58
6.1 Method . . . p. 59 6.1.1 Participants Selection . . . p. 59 6.1.2 Questionnaire . . . p. 60
6.2 Demographics . . . p. 61 6.3 Findings . . . p. 62
6.3.1 RQ1: Which exception handling guidelines are adopted by Java
developers? . . . p. 63 6.3.1.1 Signaling Guidelines . . . p. 65 6.3.1.2 Catch Guidelines . . . p. 67 6.3.1.3 Handling Actions Guidelines . . . p. 69 6.3.1.4 Remapping Guidelines . . . p. 71 6.3.1.5 Communication Guidelines . . . p. 73 6.3.1.6 Avoid Exceptions . . . p. 75 6.3.1.7 Checked or Unchecked guidelines . . . p. 75 6.3.2 RQ2: How are such guidelines being disseminated among team
members? . . . p. 76 6.3.3 RQ3: How is the compliance between the source code and such
guidelines checked? . . . p. 77 6.4 Discussions . . . p. 77 6.4.1 On the guidelines being used . . . p. 77 6.4.2 On the contribution to the debate about checked and unchecked
exceptions . . . p. 78 6.4.3 On ways used to disseminate the guidelines . . . p. 78 6.4.4 On the compliance checking of guidelines . . . p. 79 6.5 Threats to Validity . . . p. 79 6.5.1 Projects . . . p. 79 6.5.2 Qualitative analysis . . . p. 79 6.5.3 Guidelines . . . p. 79 6.5.4 Project vs personal guidelines . . . p. 80
6.6 Final considerations . . . p. 80
7 Study E - How widespread exception handling guidelines are, and
what benefits they bring to projects. p. 81
7.1 Method . . . p. 82 7.1.1 Participants Selection . . . p. 82 7.1.2 Guidelines Selection . . . p. 83 7.1.3 Questionnaire design . . . p. 84 7.1.4 Data Analysis . . . p. 86 7.2 Demographics . . . p. 87 7.3 Findings . . . p. 87
7.3.1 RQ1: How widespread is the adoption of different exception
han-dling guidelines? . . . p. 88 7.3.2 RQ2: What benefits do developers expect from adopting
excep-tion handling guidelines? . . . p. 89 7.3.2.1 Log the exceptions . . . p. 89 7.3.2.2 Handle in a specific layer . . . p. 92 7.3.2.3 Delegate to dedicated handlers . . . p. 94 7.3.2.4 Define an exception super type . . . p. 96 7.3.2.5 Define few exceptions . . . p. 98 7.3.2.6 Convert checked exceptions to runtime exceptions . . . p. 101 7.3.2.7 Convert external exceptions to domain exceptions . . . p. 103 7.3.3 RQ3: Do developers perceive it is mandatory to follow exception
handling guidelines? . . . p. 106 7.3.3.1 RQ4: How are such guidelines being disseminated among
and such guidelines checked? . . . p. 107 7.4 Discussions . . . p. 110 7.4.1 Actual efficacy in following exception handling guidelines . . . . p. 110 7.4.2 Handle in a specific layer vs Delegate to dedicated handlers . . . p. 110 7.4.3 Combined guidelines benefits . . . p. 111 7.4.4 Extent to follow vs Dissemination strategies vs Verification
stra-tegies . . . p. 111 7.4.5 Open source culture vs Exception handling guidelines . . . p. 111 7.5 Threats to validity . . . p. 112 7.5.1 Qualitative analysis . . . p. 112 7.5.2 Community culture vs Project culture . . . p. 112 7.5.3 Developers . . . p. 112 7.6 Final considerations . . . p. 112
8 Study F - Confronting beliefs and practices: do GitHub developers
actually follow exception handling guidelines? p. 114 8.1 Method . . . p. 115 8.1.1 Projects selection . . . p. 115 8.1.2 Code analysis and data collection . . . p. 116 8.1.2.1 Guidelines heuristics and metrics . . . p. 116 8.1.3 Data analysis . . . p. 120 8.2 Tool . . . p. 120 8.2.1 Tool modules . . . p. 121 8.2.1.1 Exception handling constructs model . . . p. 121 8.2.1.2 Analyzer . . . p. 122 8.2.1.3 Limitations . . . p. 123 8.3 Results . . . p. 124
ception handling guidelines actually conforms to them? . . . p. 124 8.3.1.1 Log the exceptions conformance rate . . . p. 124 8.3.1.2 Handle in a specific layer conformance rate . . . p. 126 8.3.1.3 Define an exception super type conformance rate . . . p. 126 8.3.1.4 Define few exceptions conformance rate . . . p. 128 8.3.1.5 Convert checked exceptions to runtime exceptions
con-formance rate . . . p. 129 8.3.1.6 Convert external exceptions to domain exceptions
con-formance rate . . . p. 131 8.3.2 RQ2: Do the source code of developers who claim to follow
ex-ception handling guidelines conforms to them more than source
code of developers who doesn’t claim to follow guidelines? . . . p. 132 8.3.2.1 Log the exceptions comparison . . . p. 132 8.3.2.2 Handle in a specific layer comparison . . . p. 132 8.3.2.3 Define an exception super type comparison . . . p. 134 8.3.2.4 Define few exceptions comparison . . . p. 134 8.3.2.5 Convert checked exceptions to runtime exceptions
com-parison . . . p. 134 8.3.2.6 Convert external exceptions to domain exceptions
com-parison . . . p. 136 8.4 Discussions . . . p. 136 8.5 Threats to validity . . . p. 137 8.5.1 Conformance rate . . . p. 137 8.5.2 Tool . . . p. 138 8.6 Final considerations . . . p. 138 9 Related Work p. 139
10 Concluding Remarks p. 141 10.1 Contributions . . . p. 142 10.2 Future work . . . p. 143
1
Introduction
The Exception Handling (GOODENOUGH, 1975) is a mechanism present in most mo-dern Object Oriented (OO) languages to help the development of fault tolerant solutions. This mechanism promotes the separation of code which deals with fault detection and their handling from code which implements regular behaviour, the "happy path".
Despite being an old feature of the Java language, the literature shows that developers still find it difficult to use exceptions for even the most basic problems (KERY; GOUES; MYERS, 2016), making the code that should encapsulate and deal with the failures of system in a controlled way ends up being responsible for compromising the robustness of the software (BARBOSA; GARCIA; BARBOSA, 2014; CACHO et al., 2014; EBERT; CASTOR; SEREBRENIK, 2015; SAWADPONG; ALLEN; WILLIAMS, 2012; PáDUA; SHANG, 2018; SENA et al., 2016).
To prevent this from happening, several works (PáDUA; SHANG, 2018; SINHA; ORSO; HARROLD, 2004;ASADUZZAMAN et al., 2016;NAKSHATRI; HEGDE; THANDRA, 2016;KERY; GOUES; MYERS, 2016; CABRAL; MARQUES, 2007; PÁDUA; SHANG, 2017; COELHO et al., 2008;JO et al., 2004) propose the analysis of Java source code or bytecode through static analysis to identify bad practices related to exception handling. Some works even relate these practices to bugs (SENA et al., 2016; BARBOSA; GARCIA; BARBOSA, 2014; EBERT; CASTOR; SEREBRENIK, 2015; FAN et al., 2018; PáDUA; SHANG, 2018), showing empirically which practices are more harmful. These works range from simple data, such as number of throws and catches, to data that require interprocedural analysis, such as the calculation of exceptional flows (PÁDUA; SHANG, 2017;COELHO et al., 2008;SENA et al., 2016;JO et al., 2004). The number of projects analyzed ranges from four (SINHA; ORSO; HARROLD, 2004) to hundreds of thousands (ASADUZZAMAN et al., 2016; NAKSHATRI; HEGDE; THANDRA, 2016;KERY; GOUES; MYERS, 2016).
1.1
Problem
There are several works that investigate exception handling source code. As a result, these works present what they consider to be the best practices, bad practices, bad smells, exception handling patterns and anti-patterns. But these works do not investigate the intentions of the developers to write that code. According to (ROBILLARD; MURPHY, 2000), the exception handling is essentially a design problem. So the solutions that use exceptions are the result of a set of decisions made by developers to manage failures in their projects.
These decisions are valuable knowledge for the project, but few studies are focused on investigating Java exception handling from the origin of the decisions, which are the developers. For this reason, unlike other works that investigates exception handling by looking at source code, our research is aimed at consulting developers about what solutions (practices, guidelines, etc.) they have adopted or developed to handle exceptions in their projects.
After all, do developers care about exception handling code or is this a neglected me-chanism that is used only because the programming language imposes it? If it is something developers care about, specify, design and implement, what guidelines they follow? What are the benefits these guidelines bring to the projects? How do developers enforce them in source code?
In this thesis we investigate Java exception handling, focusing on the guidelines that developers adopt for their projects, how these guidelines are disseminated among other contributors, and how they are verified in source code. With a greater understanding of the solutions adopted, researchers can take a deeper look at what developers are looking for or where they have difficulty writing EH code. In addition, with greater understanding of dissemination and verification strategies, researchers and the community can propose tools or methodologies that seek to ensure the correct implementation of exception handling code, which will ensure the benefits developers expect from designed solutions.
1.2
Research questions
The main goals of the research are to identify and understand the guidelines that Java developers adopt in their projects, and how the knowledge related to these guidelines is managed in their project. To guide our research, we have defined the following research
questions:
• RQ1: What are the exception handling guidelines Java developers adopt in their projects?
• RQ2: What are the perceived benefits related to adopt exception handling guidelines according to the Java developers?
• RQ3: What are the strategies Java developers use to disseminate these guidelines among team members?
• RQ4: What are the strategies Java developers use to verify whether the source code conforms to the these guidelines?
1.3
Research Design
In this work we performed a set of studies which uses different research methods to address the research problem and answer the research questions. The methods used were mainly qualitative, such as literature review, interviews and surveys, but we also used quantitative methods such as surveys and static analysis of source code. The progress of the research was linear, with each new study trying to validate the results of the previous study while reducing the scope of research and deepening discussions. The research design comprises a total of 6 studies. The 1 shows the studies overview. Each study is briefly explained below.
Study A - Identifying best practices for Java Exception Handling.
The first study aimed to identify known best practices to write exception handling code in Java. For this we conducted a review of 7 sources, including commercial tools. As a result we could collect 40 distinct best practices for exception handling, which serves as a knowledge base for the research.
Study B - How developers deal with exceptions: a single company study. The previous study identified a set of best practices for exception handling from litera-ture and from commercial tools, but we still had no information about how developers deal with exceptions on a daily basis. For this we developed a study that conducted interviews with 10 developers and a validation survey with 19 developers of a local organization.
Figura 1: Research design overview.
In this study we investigate how developers deal with Java exception handling by asking questions such as: How often do you deal with exceptions? How well do you know your project exceptions? How could exception handling be better in your project? Where did you learn exception handling? The results showed that the exception handling mechanism promotes more benefits than harms; that developers are neither educated nor trained to deal with exception handling; and that they care about exception handling code, although it’s not a priority when developing software.
Study C - How developers deals with exceptions: a multi-company interview-based study.
The previous study had interesting results on developers’ awareness towards excep-tion handling, but was conducted in a single organizaexcep-tion, limiting the generalizaexcep-tion of the results. For this reason, we developed a new study that interviewed 9 developers from different organizations whose goal was to identify distinct guidelines to exception han-dling, as well as to identify how, in terms of process, exception handling is designed and applied within a team or organization. In the interviews we explore questions like: Does your project have recommendations for writing exception handling code? How did these recommendations come about? How new members are introduced to the
recommendati-ons? How is the compliance with the recommendations verified? Do you think exception handling code are hard to deal? It takes too much of your time? The results showed that 9 distinct guidelines for handling Java exceptions, 8 distinct strategies to disseminate them to new developers, and 4 distinct strategies to verify them in source code. Also, developers doesn’t think that deal with exceptions is a hard or time consuming task.
Study D - What are the Java exception handling guidelines developers adopt in their projects, and how they disseminate and verify these guidelines. In the previous study we performed a set of interviews with developers from different organizations to investigate how Java developers design and use exceptions in their pro-jects. Due to the number of interviewees, we could not generalize the findings. To address that, we conducted a survey with 98 developers from the GitHub community. We desig-ned a questionnaire to investigate the existence of exception handling guidelines and the use of different strategies for the dissemination and verification of these guidelines. As a result we identified 31 exception handling guidelines, which we divided into 7 categories. In addition, we have seen that the exception handling knowledge is mainly disseminated to other developers and verified in source code through code inspection, code review and informal discussions.
Study E - How widespread exception handling guidelines are, and what benefits they bring to projects.
In the previous study we identified a set of 31 guidelines to write exception handling code. Although the sample of that study was significant, it was necessary to validate the results in a larger sample. To do this, we conducted a new survey with 297 GitHub developers, which investigated a smaller set of guidelines, 7, but would also investigate how important the guidelines are for developers and what benefits they bring to the project. The results showed that all investigated guidelines are present in at least 26% of projects, that each guideline promotes a different set of benefits, most of them related to ease bug tracing and fixing, and that 92% of developers believe they follow the guidelines in code in most cases or whenever is possible.
Study F - Confronting beliefs and practices: do GitHub developers actually follow exception handling guidelines?
The previous study showed that developers believe they write source code that con-forms within exception handling guidelines. Also showed that the dissemination of
guide-lines among the team is done mainly through peer code review, informal discussions and meetings, and reading related code of the same project. And the verification of guideli-nes in source code is mainly done through code inspection and peer code review. These results made us question whether dissemination and verification strategies are really effec-tive, thus ensuring the benefits of following exception handling guidelines. To investigate this we designed a study to compare what the developers indicated in the survey with what they practice in the source code. For this we developed a static analysis tool that, fed by a set of metrics, verifies the conformance of projects towards exception handling guidelines used in the last study. The results of this study showed that the conformance rate of projects of developers who claim they follow exception handling guidelines does not differ significantly from projects whose developers claim not to follow the guidelines.
1.4
Thesis Organization
This thesis is organized as following: Chapter 2 present the background; Chapter 3 to 8 present the studies A to F, respectively. Chapter 9 relates our research with other works; and Chapter 10 present the conclusions.
2
Background
In this chapter, we present the basics concepts related to the Java exception handling mechanism, which are needed to understand the research.
2.1
Java Exception Handling Mechanism
Software is a set of components that cooperate with each other to accomplish a prede-termined task. However, the software output does not always match what was specified. In such cases, the software is said to have failed (ANDERSON; LEE, 1985). Every nontri-vial software system has faults that will eventually result in a failure. Developers aim to minimize the number and severity of faults, which can be achieved by a variety of strategies.
One strategy that is supported by most programming languages today is the exception handling mechanism (GOODENOUGH, 1975;GARCIA et al., 2001). Using such a mechanism, developers can encapsulate code that deals with abnormal (exceptional) system states, which promotes a better separation of concerns (CRISTIAN, 1985).
The Java constructs that are related to exception handling are: try, try-with-resources, catch, throw, throws and finally. These constructs and other Java exception handling concepts relevant to this work will be explained below. The Figures 3 and 2 will be used to exemplify the concepts.
Exception. It is the representation of a defect. In Java, it is a class that is part of a predefined hierarchy, whose root is the class java.lang.Throwable. Defining a new excep-tion means defining a new class that is a direct or indirect subtype of java.lang.Throwable.
Exceptional hierarchy. This refers to the hierarchy of classes that are direct or indirect descendants of java.lang.Throwable. Java has a variety of exceptions in its exceptional hierarchy. The exceptional hierarchy can be expanded with new exceptions, such as CustomException and CustomRuntimeException.
Figura 2: Java exception handling hierarchy.
Unchecked exceptions. Exceptions descending of java.lang.RuntimeException. Unchecked exceptions do not need to be declared in the exceptional signature of the methods.
Checked exceptions. Exceptions descending from java.lang.Exception or from java.lang.Error and not descending from java.lang.RuntimeException. Checked ex-ceptions are checked at compile time, causing a method that signal a checked exception to be forced to catch the exception or to signal it in its exceptional interface.
Java exceptions. Exceptions defined by the Java language API, as such java.lang.Exception and java.net.ConnectException.
Project exceptions. Exceptions defined by the system under consideration.
External exceptions. Exceptions defined by any dependency (libraries and fra-meworks) of the system under consideration.
Project exceptions. Exceptions descending from java.lang.Exception or from java.lang.Error and not descending from java.lang.RuntimeException. Checked ex-ceptions are checked at compile time, causing a method that signal a checked exception to be forced to catch the exception or to signal it in its exceptional interface.
Signaler. Method that signals an exception using the throw clause. The term is commonly used for the method that first detects the abnormal state of the program,
Figura 3: Example of Java exception handling model.
instantiates the exception and then signal it (methodC, lines 22 and 25), starting its flow through the application. The term can also be used by any method that throw an exception, as in methodB, line 15.
Handler. Construct that catches an exception and ends its flow. A handler is associa-ted with a single try or try-with-resources block. It is in the handler that the handling actions are encapsulated. The handler is specified with the catch clause (lines 5-7 and 14-16), which indicates which exception types that handler catches. For each exception declared, the handler catches all exceptions that are of that type or any of its direct or indirect subtypes (line 5-7).
Resignaler handler. A handler that, in its scope, explicitly signals exceptions (lines 14-16).
Final handler. A handler that, in its scope, does not explicitly flag exceptions (lines 5-7).
Exceptional flow. The path on which an exception travels until it is caught, or until it escapes the application. The path is represented by a list of methods. In the example, the exception flow of CustomException is formed by the methods methodC and methodB. And the exceptional flow of RuntimeException is formed by the methods methodC, methodB
and methodA.
Handling actions. Code that executes after an exception is caught in the handler (lines 6 and 15). The handling actions are supposed to recover the system from the abnormal state.
Exceptional interface. List of exceptions that can flow from a method. In Java, the declaration of the exceptional interface is mandatory only for checked exceptions. The declaration is made with the throws clause and complements the method signature (line 19).
Protected Region. It is the block of code, associated with one or more handlers, protected from the occurrence of an exception. In Java, the protected region is defined with the try (lines 3-5 and 12-14) or try-with-resources blocks. try-with-resources is a special kind of try block used to close resources automatically. Note that the try-finally construct does not protect the code from the occurrence of an exception and is therefore not considered a protected region.
Exceptional model. Defines how the exception handling works. In Java, once an exception is signaled, the regular execution is interrupted, and it is started the search for a handler. At line 22 of the example, an exception is signaled. The first method which is a candidate to catch it is methodC itself. To catch an exception, two conditions must be met: the code snippet that signaled the exception (usually a method call) must be in a protected region; and the protected region must be protected for the signaled exception type. Because the signaling on line 22 does not meet these conditions, the search goes to the method that invoked the signaler method (methodB). As the calling to methodC in methodB meet the conditions, the flow of CustomException is terminated. If a handler is not found until the exception reaches the program entry point (usually the main method), the exception is said to have escaped from the system.
The exceptional Java model is similar to most modern programming languages. A relevant difference is that Java separates its exceptions into two types, checked and un-checked. Checked exceptions are checked by the Java compiler, and need to be declared in the exceptional interface of the methods. Failing to declare them causes a compila-tion error. Unchecked excepcompila-tions, on the other hand, don’t need to be declared in the exceptional interface. They flow implicitly through the application.
3
Study A - Identifying best
practices for Java Exception
Handling
There are a variety of works (PáDUA; SHANG, 2018; SINHA; ORSO; HARROLD, 2004;
ASADUZZAMAN et al., 2016;NAKSHATRI; HEGDE; THANDRA, 2016; KERY; GOUES; MYERS, 2016;CABRAL; MARQUES, 2007;PÁDUA; SHANG, 2017;COELHO et al., 2008;JO et al., 2004)
that investigate exception handling source code. As a result, these works present what they consider to be exception handling bad practices, bad smells and anti-patterns.
From this, we begin to ask ourselves: What are the best practices to write exception handling code? We conducted a study in which we searched for practices present in li-terature and commercial tools. This study’s goal was to set an initial base of knowledge about exception handling practices. As a result, we could collect 40 distinct best practices for exception handling from 7 different sources.
3.1
Method
At first, we performed a literature review looking for works that present best practices for exception handling, which discuss and justify the rationale behind the practices.
We then look for best practices in commercial coding support tools, also known as “linters”. Linters analyze source code at coding time and notify developers of violations of their rules. The developer can then see details about the rule to understand why their code is violating it, as well as to see code examples that conform to the rule.
The rules contained in the linters address various aspects of source code, such as code style (variables, methods and classes names; comments), code complexity (cyclomatic complexity, excessive nested levels), and other aspects. However, only a few rules are related to exception handling.
Once identified, the practices were analyzed and compared to eliminate duplicates, as different sources sometimes present similar or equal practices. This analysis took into consideration the name of the practice, the context in which it is applied and the actions which represent it.
Finally, we reviewed and changed the names of some practices so that all of them are using uniform EH-related terms and represented positive actions, which improves the coherence and understandability of the findings.
3.2
Findings
Following the method described before we we collected 72 best practices in the fol-lowing sources: the “Effective Java” book (BLOCH, 2008), the “Toward Exception-Handling Best Practices and Patterns” paper (WIRFS-BROCK, 2006), the Exception-Handling
Anti-patterns Blog (ADAMSON, 2018), which is kept as documentation on the Oracle website -and four tools (SonarLint (SONARQUBE, 2018), Robusta (ROBUSTA. . ., 2018), PMD (PMD, 2018) and SpotBugs (SPOTBUGS, 2018)). The table 1 presents the number of practices identified in each source.
Tabela 1: Summary of analyzed sources and practices
Source name # practices ID
Effective Java 9 S1
Toward Exception-Handling Best
Practices and Patterns 9 S2
Exception-Handling Antipatterns Blog 12 S3
SonarLint 12 S4
Robusta 6 S5
PMD 12 S6
SpotBugs 12 S7
After removing 20 duplicates and reviewing each practice to adapt their names, we achieve the final list presented in Table 2.
Tabela 2: Best practices for Java exception handling
Practice name Sources
Use exceptions only for exceptional conditions S1, S2, S6 Use checked exceptions for recoverable conditions and runtime
exceptions for programming errors S1
Avoid unnecessary use of checked exceptions S1, S3, S4 Favor the use of Java standard exceptions S1 Throw exceptions appropriate to the abstraction S1, S2 Document all exceptions thrown by each method S1 Include failure-capture information in detail messages S1 Avoid declaring lots of exception classes S2
Provide context along with an exception S2
Avoid "throws Exception"declaration S3, S6 Exceptions should not be thrown from servlet methods S4 Avoid throwing java.lang.NullPointerException S6
Do not extend java.lang.Error S6
Avoid throw new instance of the same caught exception S6 Avoid create and drop exception rather than thrown S7 Do not catch and ignore exceptions S1, S3, S5,
S6, S7
Do not handle coding errors S2, S6
Handle exceptions as close to the problem as you can S2 Avoid catching java.lang.Exception S3, S4, S6
Avoid log and return null S3
Avoid multi-line log messages S3
Avoid relying on getCause method S3
Avoid jump statements in "finally"blocks S4 Avoid testing of exceptions types by using "instanceof"in
catch blocks S4
Use try-with-resources S4
Combine catch blocks with same handling code S4
Avoid nested try-catch blocks S4, S5
Avoid an unprotected main S5
Dubious catching of IllegalMonitorStateException S7
Don’t ignore exceptional return value S7
Release lock on all exception paths S7
Avoid unreachable catch blocks S7
Do not repeatedly rethrow the same exception S2, S6
Avoid log and throw S3
Preserve original exceptions when rethrowing S3, S4, S6 Avoid throwing exceptions in "finally"blocks S3, S4, S5, S6
Strive for failure atomicity S1
Name an exception after what went wrong, not who raised it S2
Exception classes should be immutable S4
3.3
Threats to Validity
3.3.1
Analyzed sources
To identify sources to the study we performed a literature review of papers that explicitly proposes best practice to exception handling accompanied by the rationale of these practices. This can exclude papers that discuss best and bad practices without explicitly presenting rationale.
3.4
Final considerations
In this study, we compiled a set of 40 distinct best practices over 7 different sources to gather information about the rationale behind the exception handling code. Besides that, according to (ROBILLARD; MURPHY, 2000), the exception handling is essentially a design problem. So the practices that use exceptions in a project are the result of a set of decisions made by developers to manage failures in their projects. Therefore, to investigate the developers’ intentions behind exception handling code, we need to assess them.
4
Study B - How developers deal
with exceptions: a single company
study
The previous study identified a set of good practices for exception handling from lite-rature and commercial tools, but we still had no information about how developers decide which solutions they should adopt to deal with exceptions on an daily basis. For this, we developed a study that conducted interviews with 10 developers and a validation survey with 19 developers of a single company. In this study, we investigate how developers deal with Java exception handling by asking questions such as: How often do you deal with exceptions? How well do you know your project exceptions? How could exception han-dling be improved in your project? Where did you learn exception hanhan-dling? The results showed that the exception handling mechanism promotes more benefits than harms; that developers are neither educated nor trained to deal with exception handling; and that they care about exception handling code, although it’s not a priority when developing software.
The main goal of this study is to give us a better understanding of how professional developers comprehend and care about the exception handling code they write. To guide our research towards this goal, we defined the following research questions:
• RQ1: What are the benefits and harms of using the exception handling mechanism embedded in Java?
• RQ2: What are the obstacles faced by developers while writing exception handling code?
4.1
Method
To answer these questions, we conducted a set of interviews and a survey. The method for participant selection, the design of the interview script, the survey’s questionnaire, and our data analysis process are described next.
4.1.1
Participants
The study was conducted in the context of a development team from an IT sector of a University. The team has a total of 40 developers working on the development and main-tenance of a Java web-based information system responsible for automating university business processes, including academic, administration and human resources processes.
The study was conducted in two phases. In the first phase, we interviewed 8 developers, in a total of 10 contacted. In the second phase, we performed a validation survey to assess if and to what extent the findings observed during the interviews hold for the other developers working in the same IT sector. We sent the survey to the 40 team developers and got 19 responses (47% response rate). As survey participation was anonymous, the 19 respondents could overlap within the interviewed developers.
Table 3 shows the experience of developers in both phases.
Tabela 3: Developers’ experience in software development Interview phase Survey phase
6- months 0 0 6 months to 2 years 2 2 2 to 4 years 0 1 4 to 6 years 1 4 6 to 10 years 4 6 10+ years 1 6
4.1.2
Interviews
The initial script was designed by 3 researchers. The interviews were conducted in person by 2 researchers. After each interview and subsequent data analysis, the script was revised and could be adapted. As one of the researchers knew some of the participants, some extra questions were asked specifically to these participants, taking into account the position he/she held in the organization (novice developer, architect, team leader, etc.).
The questions were organized into 8 sections: professional experience, responsibilities on the project, exception handling usage, Java exception handling mechanism, exception handling design, exceptions and bugs, exception and libraries and checked vs runtime exceptions. Table 4 presents the final version of the script.
The interview began with the researchers presenting themselves, the study, and the participant’s rights, who could give up participating at any time. Then questions were asked about the professional experience (questions 1-4) and the responsibilities of the participant within the project and their communication with other developers (questions 5-10).
Then questions were asked about the frequency of exception handling usage, like signaling and handling (questions 11-17). And then we ask about benefits, problems and how the participants learn about exception handling (questions 18-20).
The next questions ask about the exception handling design in their projects (questi-ons 21 and 22); the relation of excepti(questi-ons and bugs (questi(questi-ons 23 and 24), excepti(questi-ons and libraries (questions 25 and 26); and the preference over checked and unchecked exceptions (questions 27, 28 and 29).
4.1.3
Validation survey questionnaire
The questionnaire consisted of 15 statements directly derived from the findings of the interviews phase and the respondents answered it using a five-point Likert-type scale. All the statements are presented in Table 5. The first column presents the findings of the interviews that relate to the survey question. The findings of the interviews will be explained in section 4.2.1.
4.1.4
Data analysis
We used Grounded Theory (GT) techniques (CHARMAZ, 2014) to iterate through
the interview transcripts. The data analysis in this study consists of three steps: (1) the transcriptions of the interviews, which helps to gather initial insights, (2) open coding, which entails a close reading of the data and (2) axial coding, which combine the most salient themes in the analyzed data to form categories.
The transcriptions were done by two researchers; the open coding and axial coding were done by each researcher separately, and after that, they discussed the codes and
Tabela 4: Interview script
1 Could you talk briefly about your professional experience? 2 How long have you been a professional programmer? 3 Did you have any jobs before the current one? 4 What do you do in your current job?
5 All your time is dedicated to codification? 6 Do you develop new features in the project? 7 Do you maintain features that already exist? 8 Do you fix bugs?
9 Do you participate in project decisions?
10 How do you communicate with your team members, your boss, and other teams? 11 Do you think Java exception handling helps or hinders your work? Why? 12 How often do you implement code that deals with exception handling? Every
day? Rarely?
13 Do you know the exceptions types of your project and in what context should do you use them?
14 When was the last time you had to throw an exception? What was the context? 15 When you throw an exception, do you want to know where it will be handled? 16 In the last time you called a method that threw an exception, how do you handle
that exception?
17 Have you ever created a new exception type in the project?
18 How did you learn about exception handling? Did you have any classes or courses on the subject? Or did you learn in everyday work?
19 Have you had trouble working with Java exceptions? Which?
20 What the benefits of using the exception handling mechanism? Do you see any disadvantages?
21 Have you found any information on how exception handling should be done in your project?
22 What kind of information would help you when handling and throwing excep-tions in your project?
23 Have you fixed any bug reported by users?
24 Every bug report is related to an uncaught exception. Do you think some of these bugs could be prevented? How?
25 When you use a method from a library or framework, do you look up to any exceptions that may be thrown by it?
26 Do you handle exceptions thrown by libraries and frameworks the same as project-specific exceptions?
27 Do you have a preference over checked or runtime exceptions? Why?
28 Do you document the runtime exceptions thrown by a method you are imple-menting?
29 Do you think the "throws"clause in the methods’ signatures helps you during development?
Tabela 5: Survey questions
1 How many years of experience in software develop-ment do you have?
Robustness 2 The exception handling mechanism helps to fix bugs.
Robustness 3 Without exception handling, the system would crash more frequently.
Code Structure 4 The exception handling code is hard to maintain. User Experience 5 The exception handling allows a better interaction
between the system and the user.
Cost of development 6 Dealing with exception handling code consumes a lot of my time.
Training and education 7 The company trained me to deal with the system exceptions.
Training and education 8 I know good and/or bad practices for writing ex-ception handling code.
Documentation 9 The system documentation helps me to unders-tand how to use exceptions in the system.
Negligence and priorities 10 I am comfortable with writing exception handling code of the system.
Negligence and priorities 11 I care about how exceptions are handled.
Negligence and priorities 12 Part of my working time if focused on think about how to handle exceptions.
Negligence and priorities 13 I care about normal code before implementing ex-ception handling code.
Ignore third-party exceptions 14 Exceptions thrown in my project are handled in the same way as third-party exceptions.
Ignore third-party exceptions 15 Before using a method from a third-party library, I try to know what exceptions it can throw.
categories to resolve inconsistencies. The coding process was done until no new insights and questions emerged.
4.2
Findings
This section presents the findings from this study. Section 4.2.1 presents the main themes which emerged from the interviews phase, and Section 4.2.2 presents the findings from the validation survey.
4.2.1
Interviews findings
In this section, we present the findings from the interview transcriptions analysis. To illustrate the findings we provide a selection of quotes. To ensure participants anonymity, they are identified with the convention P#.
4.2.1.1 RQ1: What are the benefits and harms of using the exception han-dling mechanism embedded in Java?
The analysis of the interview transcripts revealed that participants consider that there are both negatives and positives aspects when using the Java exception handling mecha-nism. Robustness and a better code structure are positive aspects, while the increased cost of development is a negative aspect.
Robustness. Developers highlighted that exception handling is useful to fix bugs and prevent system crashes, which means a more robust software.
“I believe that the exceptions are important for enabling the system recovery if it is in an unexpected situation. This way, the system can return to its normal state automati-cally.” P7
Code Structure. The EH mechanisms directly impact the source code structure. Most developers consider that impact on structure a positive factor, as it promotes sepa-ration of concerns.
“[EH] Keeps the code organized and more structured, even for maintenance. Another developer, novice or expert, can look the code and know what to expect.” P5
Some developers, however, revealed that the exception handling mechanism increases code verbosity and criticized those who use checked exceptions improperly.
“Let’s assume a chain of method calls that has five levels. If you use a checked exception in the first level, which will only be handled at the highest level, you will need to change the code in five different points unnecessarily. It turns out to be difficult to maintain and a verbose approach.” P4
Cost of development. Participants considered that the EH mechanism is a burden to software development when they have to make decisions about exception handling. Two participants commented that when there exist rules and standards to be followed, the development becomes easier. The participants also considered the cost of using EH mechanisms acceptable when compared to their benefits.
“We use EH mechanisms a lot. There is a cost for us to detect if some code snippet needs a specific handling action, but it is irrelevant. If there is not a well-structured and abstracted architecture for the developer, productivity drops a lot.” P3
User Experience. Many participants recognized a relation between a good exception handling and a good user experience; because when the exception is handled and a good message is present to the user, he/she can better decide what to do.
“The exception handling should put some coherent message to the user and redirects the system to a screen where he/she can continue his/her work.” P7
4.2.1.2 RQ2: What are the obstacles faced by developers while writing ex-ception handling code?
Our analysis revealed that developers have two main obstacles to write exception handling code: lack of documentation and lack of training.
Lack of training. Most participants said that although the organization conducts training with novices to know the system and its architecture, no guidance is given speci-fically for exception handling. Practices and solutions for exception handling end up being learned in everyday work by inspecting the existing code.
“When I need to catch an exception, I rely on the code of others. I observe how they raise and handle try to do the same way.” P5
“We didn’t have any training. We learn in small steps.” P3
Lack of documentation. Participants mentioned that they do not have any gui-deline to assist them in the implementation of exception handling code. They said that there is a lack of documentation about exception handling design, or they are unaware
of it. For this reason, the EH code is usually implemented without the supporting docu-mentation/guideline.
“Now I realized that we should have documentation explaining the exception handling policies.” P8
4.2.1.3 RQ3: Do developers care about exception handling code?
Our third research question intends to gather information about the care and the priorities of developers towards the exception handling code. In general, we observed that developers tend to adopt a reactive approach, postponing the exception handling implementation until a failure occurs.
Negligence and priorities. Most participants indicated to care only about the im-plementation of the "happy path"of a functionality. When it comes to the imim-plementation of exception handling, they adopt an ignore-for-now approach. In other words, when a bug related to an uncaught exception occurs, they deal with the exceptional situation. One of the participants justified this behavior based on the tight deadlines they have.
“We care about delivering the product and only think about the happy-path. If there is a problem, we take a note of it and think about how to deal with it afterward.” P7
Ignore third-party exceptions. Only one interviewee assumed to care about ex-ceptions raised by external APIs. All the other participants demonstrated to ignore what exceptions could be launched when using an external source code.
“I implement the method call and if it is necessary to deal with some exception, I re-throw it.” P7
4.2.2
Validation survey findings
In this section, we present the findings of the validation survey.
4.2.2.1 RQ1: What are the benefits and harms of using the exception han-dling mechanism embedded in Java?
Five survey questions (Table 5) investigate how the exception handling affects each of four themes related to RQ1 which emerged after interviews analysis. Questions 1 and 2 investigate the system’s robustness. Question 3 investigates the code structure. Question 4 investigates the user experience. Question 5 investigates the cost of development. Figure
Figura 4: Survey findings related to RQ1. 4 summarizes the findings.
• 84% of respondents agreed or strongly agreed that the exception handling mecha-nism is a useful tool for fixing bugs.
• 63% of respondents agreed or strongly agreed that exception handling is directly related to the system’s crashes.
• 62% of respondents disagreed or strongly disagreed that exception handling code is hard to maintain.
• 63% of respondents agreed or strongly agreed that exception handling promotes a better user experience.
• 73% of respondents disagreed or strongly disagreed that implement and maintain exception handling code is an expensive activity, in terms of time.
4.2.2.2 RQ2: What are the obstacles faced by developers while writing ex-ception handling code?
Three survey questions (Figure 5) investigate the two themes related to RQ2 which emerged after interviews analysis. Questions 6 and 7 investigates the lack of training and education about exception handling mechanism and design. Question 8 investigates the lack of documentation in the organization. Figure 5 summarizes the findings.
Figura 5: Survey findings related to RQ2.
• 68% of respondents disagreed or strongly disagreed that they were properly trained by the organization to deal with exceptions.
• 58% of respondents agreed or strongly agreed that they know good and bad practices for implementing exception handling code.
• 68% of respondents disagreed or strongly disagreed that the documentation assists the exception handling code implementation.
4.2.2.3 RQ3: Do developers care about exception handling code?
Six survey questions (Figure 5) investigates the two themes related to RQ3 which emerged after interviews analysis. Questions 9-12 investigate the negligence and priorities of developers towards exceptional handling code. Questions 13 and 14 investigates how they use third-party exceptions. Figure 6 summarizes the findings.
• 75% of respondents agreed or strongly agreed that write exception handling code is a comfortable task.
• 68% of respondents agreed or strongly agreed that they care about how exceptions are handled.
• 74% of respondents disagreed or strongly disagreed that they use a part of their time exclusively to think about how to deal with exceptions.
• 79% of respondents agreed or strongly agreed that they care about the normal code before the code related to exception handling.
Figura 6: Survey findings related to RQ3.
• 37% of respondents disagreed or strongly disagreed that third-party exceptions are handled in the same way as project exceptions. 32% of participants stayed neutral. 31% agreed or strongly agreed with the statement.
• 27% of respondents agreed or strongly agreed that they try to know what exceptions may be thrown by a third-party method.
4.3
Discussions
4.3.1
Interviews vs survey findings
In general, the results obtained in the survey confirm the impressions we had after the interviews. Results such as the use of exception handling to fix bugs, the time spent to write exception handling code, the lack of training and documentation, and the priority of developers to write the "happy path"over exception handling code are already expected. Some results elucidates initial impressions we were not sure about. The low importance given to third-party exceptions and the little effort to maintain exception handling code were results we were not sure of but still expected.
and bad practices for exception handling and that they care about exception handling in general. These results were not expected.
4.3.2
A worthy investment
Our findings shows that developers recognize the importance of the exception handling mechanism to achieve a more robust system. Although there is a higher coding cost, this is a small price to pay, especially since the exception handling makes it easier to find and fix bugs in the code.
4.3.3
Education and training vs knowledge
The findings shows that developers are not trained on how to deal with exceptions, neither in their academic courses nor in their work environment; they do not dedicate much time to exception handling code; prioritize normal code over the exception handling code; their projects have no documentation for exception handling. Still, respondents assumed they know good and bad practices about exception handling code, they care about exception handling code, and mentioned that code is not difficult to maintain.
This apparent contradiction of developers may indicate that exception handling is easy to learn and does not need specific training or education to be used effectively in the project; or developers have a false sense of knowledge and they are not aware of the exception handling code they should be writing.
4.4
Threats to validity
The results of our semi-structured interview and survey-based study may not apply to every Java developer since other populations might add new insights. The population we collected data from was comprised of Java developers of a single IT sector that had time and motivation to answer the interview and survey questions.
4.5
Final considerations
Although the findings that emerged in our study cannot be generalized, they give a view of developers’ perspectives about how developers deal with the EH code and the lack of information and training related to the EH design within a project.
The results are similar to other studies (SHAH; GORG; HARROLD, 2010), which rein-forces that exception handling is sometimes misused because developers lack training and awareness about the exception handling mechanism. This motivated us to investigate how the knowledge related to exception handling, i.e. project solutions, best practices, and pat-terns, are learned by developers and organizations, and especially how this knowledge is disseminated to new team members.
5
Study C - How developers deal
with exceptions: a multi-company
interview-based study
The previous study had interesting results on developers’ awareness towards exception handling but was conducted in a single organization, limiting the generalization of the results. For this reason, we developed a new study that interviewed 9 developers from dif-ferent organizations whose goal was to identify distinct guidelines to exception handling, as well to identify how, in terms of process, exception handling is designed and applied within a team or organization. In the interviews, we explore questions like: Does your project have recommendations for writing exception handling code? How did these re-commendations come about? How new members are introduced to the rere-commendations? How the compliance with the recommendations are verified? Do you think the exception handling code is hard to deal? Does it take too much of your time? The results showed that 9 distinct guidelines for handling Java exceptions, 8 distinct strategies to dissemi-nate them to new developers, and 4 distinct strategies to verify them in source code. Also, developers don’t think dealing with exceptions is a hard or time-consuming task.
The goal of this study is to identify guidelines that developers adopt to Java exception handling in industry projects. Also, identify which strategies developers use to disseminate these guidelines among their team members and how these guidelines are verified in the project source code. To guide the study, the following specific research questions were defined:
• RQ1: What guidelines to exception handling do developers adopt? • RQ2: What are the origin of these guidelines?
• RQ3: What strategies are used to disseminate the knowledge about these guidelines across the development team?
• RQ4: What strategies are used to verify the conformance of the source code with the adopted guidelines?
5.1
Method
To answer these questions, we conducted a set of interviews with industry developers. The method for participant selection, the design of the interview script, and our data analysis process are described next.
5.1.1
Participants
The participants were selected by convenience from the professional contacts of the researchers. The prerequisite for participation was to be working with Java or have wor-ked for the past 2 years. The organization’s names are omitted as requested by some developers.
We interviewed 9 developers from different organizations. They have an average of 8.22 years (median: 6 years) of professional experience in Java development. Each interview lasted on average of 40 minutes.
5.1.2
Interviews
The initial version of the script was designed by 2 researchers. This version contained questions that attempted to identify documented or undocumented guidelines for project’s exception handling, and how these guidelines are disseminated to other developers and how these guidelines are verified in the source code.
While the questions of the main sections of the script remained the same through the study, questions from other sections were modified after each interview. The latest version of the script is presented in Table 6.
The questions were organized into 6 sections: (1) guidelines to exception handling, (2) guidelines life cycle, (3) exceptional types and hierarchy, (4) time and effort, (5) imposition or choice, and (6) tools.
Guidelines to exception handling. Questions to identify the guidelines related to exception handling developers adopted in their projects. The guidelines could be of any kind since it is directly related to exception handling constructs (try, throw, catch,
Tabela 6: Interview questions
Guidelines to exception handling
Are there any documents, formal recommendati-ons, or even verbal guidelines on how to use ex-ceptions, whether you are throwing, catching, or creating new types in your project?
If yes, what are the recommendations? What are the benefits? Could you give me an example? If no, how do you know when you should throw and catch exceptions? And what exceptional type should use?
Guidelines life cycle
What is the origin of the recommendations? When and how their adoption is first proposed? How new team members can learn about the re-commendations?
How the recommendations are checked in source code?
Are you satisfied with these recommendations?
Exceptional types and hierarchy
Do your project define custom exceptions?
They are checked exceptions, runtime exceptions or there are both types?
The custom exceptions are related to some com-ponent or layer?
Is the exceptional hierarchy of you project deeper (types, subtypes, subsubtypes...) or wider (multi-ple siblings types)?
Do you check the exceptional hierarchy when you’re working? Why?
Time and effort
How often do you read, write or think about ex-ception handling code?
How difficult are to deal with exception handling code? How difficult are to throw exceptions? To catch exceptions? To handle exceptions?
Which task is more difficult? Which task do you spend more time with?
Do you deal with exceptions as soon as possible? Or postpone to another moment?
Imposition or choice The usage of exceptions are an imposition of Java or a choice of developers?
Tools Do you use any tool to help you to write exception handling code? Which tools? For what?