Experimentation

9.5 Experimentation

ROSE is used to export the TODOList service as a REST¹¹⁰ endpoint and uses the JSON service for serialization and deserialization. Clients can access the TODOList service through simple http requests¹¹¹. Three types of requests are supported: PUT, GET and DELETE HTTP requests, which respectively add, retrieve and delete TODOs from the TODOList. Table 3 illustrates the REST API that is created.

URL Request type Description

/todolist GET Returns a JSON array of TODOs.

/todolist/{id} GET Returns the TODO item that matches id.

/todolist PUT Adds the content of the request into the TODO list.

/todolist/{id} DELETE Deletes the TODO item that matches id.

Table 3: The TODO list application’s REST API.

The architecture for our application is shown in Figure 78 and consists of 3 components, two provided services, one remote client and a remote HTTP Rest service. The RESTTodo component requires a TODOList service and implements the necessary functionality for ROSE and the REST metadata to export the service. The TODOListImpl component provides the TODOList service and the basic TODO list functionality shown in Table 3. The TODOListImpl component requires a backend to store TODOs. The specific service required is the DataSourceFactory service, which is provided by the SQLite modules that have been packaged in OW2 Chameleon.

Figure 78: Initial TODO List components and modules.

Our initial packaging placed all of our TODO application into a single module, including the definition of both TODO List components. We proceeded to modularize our application around our components using Robusta. The objective was to allow components, the REST TODO Service or the TODOList implementation, to change dynamically, without impacting other components at runtime.

To achieve decoupling, we proceeded to analyze the TODOList Service interface and place it into a separate module that would represent the service contract. Once we found the classes, in this case two classes, we proceeded to separate the component implementations into their own

110 The application doe s not aim to imple me nt the full REST approach. So HATEOS and othe r characte ristics we re voluntary ignore d.

111 For simplicity, TODO ite ms cannot be update d.

packages as well. Because we did not extend classes in our application, there was no need to create contract extensions modules. Robusta properly calculated the dependency graphs each time and facilitated the selection of new classes for packaging. Robusta also found a class dependency that extended from our application to the SQLite Database module. Figure 79 shows the result of our changes to the TODO List application. We decomposed our application into 3 modules, two of which can now evolve independently (the Service Contract cannot change without impacting the other modules).

Figure 79: Resulting TODO List architecture after decoupling analysis and packaging changes.

To test our application and its newly acquired dynamism, we proceeded to update, stop, start and replace our modules at runtime. Robusta verified that no duplicated classes resulted from these operations, meaning that they were properly decoupled. Robusta proved its usefulness and its ability to help architects and developers to modularize their applications.

9.5.2 OW2 JOnAS Java Enter prise Edition Application Ser ver

JOnAS is a Java EE 5 certified open source application server built and hosted by OW2. As a project, JOnAS started in 1998. The current version of JOnAS is built on top of OSGi, and as such, provides and exploits the same levels of dynamism as do other OSGi projects, and allows for the integration of OSGi applications and component models such as iPOJO. JOnAS provides clustering and high availability mechanisms, Web Services, Java EE Connectors, LDAP access, IIOP and many other features.

9.5 Experimentation

According to Ohloh¹¹², JOnAS is a high activity open source project with multiple contributors and consists of 9.87 million lines of code. “ccording to Ohloh s effort analysis using the COCOMO model, JOnAS is a project that accounts for 3,023 years of effort. However, we believe this to be exaggerated given the way Ohloh counts lines of code and the way the OW2 consortium contributes across project boundaries. The JOnAS team reuses and contributes to many OW2 projects (e.g., EasyBeans, JORAM, Shelbie, OW2 Utils) and integrates them into a single application server, making it difficult to estimate its size. Our estimates for active lines of code managed by JOnAS range from +400 thousand lines of code to 1 million, depending on which OW2 projects are included. A final release version of JOnAS undoubtedly holds much more code that is provided by other open source projects and communities. We estimate this to be around 3 million lines of code in a single release.

Table 4: Results obtained when testing Robusta with OW2 JOnAS.

JOnAS is by far the largest application we tested Robusta with and also the most challenging.

We tested Robusta on version 5.3.0-RC1, which consisted of over 300 modules and 120 iPOJO components. Given the sheer size of JOnAS and the effort required to decouple components in JOnAS, our tests were limited to introspection and checking Robustas scalability and overhead.

112 http://www.ohloh.ne t/p/jonas

Criteria tested Result

Startup time (w/o robusta) 43 se conds Startup time (with Robusta) 52 se conds

Runs prope rly with Robusta Ye s. Doe s not show proble ms cause d by class inte rce ption and transformations. Howe ve r, JOnAS was not e xte nsive ly te ste d.

Startup, command line access, and administration console we re te ste d.

Numbe r of classes

inte rce pted and transformed

8324 classe s in total.

Time to calculate Class De pe nde ncy Graph

20 - 163 millise conds.

Time to manipulate a class 0 220 millise conds. Slowe r speeds at startup be cause the system is loade d.

Ave rage is ~9ms.

Time to calculate Service Contract

3 - 20 millise conds in addition to the time ne e ded to calculate the Class De pe nde ncy Graph time .

Time to calculate Exte nde d Se rvice Contract

3 - 64 millise conds in addition to the time ne e ded to calculate the Class De pe nde ncy Graph time .

Our results conclude that Robusta does not inhibit the application server from functioning properly and has little overall overhead. Furthermore Robustas interactive nature ensures there is very little overhead if Robusta is not in use. These data can be seen in Table 4.

In order to execute JOnAS with Robusta enabled, we modified the jonas script that is used to start and stop the server (among other functions), and added the Robusta Agent to the command line arguments. For Robusta Analyzer to function, we manually added the required modules to the internal OSGi framework.

This experiment has proven that the collection of the data and the analysis is fast enough even on large software. The graph computation times are somewhat meaningless in interactive mode because the user is effectively much slower than Robusta. However, for automated analyses, these times become more important and should still be quite acceptable.

9.5.3 Graphical output of Class Dependency G raphs

During the design and development of Robusta, we found it desirable to provide a graphical means of viewing an applications complexity. Complexity often remains an abstract concept and is difficult to grasp. Nevertheless, graphical tools for class dependency analysis are few and far between. Yet, because Robusta has access to such information, we proceeded to create directed dependency graphs and visualize them using graph software. The simple fact of obtaining this information and exporting it to other tools can provide insight into an application

Figure 81: Shows a screenshot of Gephi with a 3000+ class dependency graph.

Robusta can export dependency graphs to files using the robusta:graph command. Such graphs can be visualized using various software, of which we tested three: yEd¹¹³, Graphviz¹¹⁴ and

9.5 Experimentation

Gephi¹¹⁵. Each tool has its advantages and defects, but we largely preferred Gephi. Graphviz provides multiple algorithms and outputs image files (e.g., png) after processing, but few of the algorithms can handle several thousand nodes, and the ones that can appear to be unusable. yEd is quick and light but showed rendering issues. Gephi was the only tool that allowed consistent viewing and manipulation of multi-thousand node class dependency graphs.

Figure 82: Shows a close-up of a class and its dependencies in Gephi.

Disappointingly, none of the graph visualization software provided adequate support for clusters. This means that it was not possible to wrap groups of classes into larger nodes that represented the modules they were contained in. To circumvent this issue, Robusta exports multiple independent graphs for classes, modules and classloaders.

9.5.4 Results & Lessons

The execution overhead of using Robusta is low, as can be seen in the tests using JOnAS. The average graph calculation times are under 100 milliseconds, and the time to instrument a class with the Robusta annotations is often under 10 milliseconds. Furthermore, there is no detectable execution overhead when Robusta is not used, i.e., when there are no dynamic events in the application.

However, memory overhead is much more difficult to calculate because of the complexity of the inner workings of the Java VM and of the operating system. We have pagination, shared libraries, caches, and other features that often make such calculations nonsensical. As such, we have not attempted to estimate this overhead in practice. Be that as it may, the Robusta Agent does cause a permanent memory overhead for each class and each dependency that is encountered.

Table 5 shows Robustas memory overhead.

115 https://ge phi.org/

Permanent Memory Overhead cause by Robusta Agent 1 annotation for each class (@Robusta)

1 annotation per class-dependency (@ClassDependency)

4 attributes per class-dependency annotation (String, String, String, Class)

Table 5: Robusta memory overhead.

Among the lessons we have learned are that Java class loading is lazy and this can impact dynamism and coupling calculations. Loading classes late in the execution can still cause hidden coupling to occur, which can penalize dynamism and result in undesirable behavior. To properly test an application you have to execute it thoroughly in order to cause all classes to be loaded.

Furthermore, garbage collection is also lazy, meaning that there is no guarantee that classes will be timely collected and freed from memory. This makes debugging difficult because, for Robusta, duplicate classes are often a good indicator that something is not working properly. If garbage collection has not occurred, then every update results in duplicated classes.

Interestingly, duplicate classes do occur even if there are no issues with dynamism or with the application. This is because large software tends to repackage libraries of different versions.

Refactoring code and using a single version of a library is often too costly or tedious and provides little immediate benefit to the application. This can lead to issues if not properly managed. Luckily, Robusta helps detect these issues.

Using the root hierarchy object (the Object class in Java) can be very problematic for dependency analysis because it opens the service to being contaminated by any object in the virtual machine. If we are pessimistic in our calculations, we should suppose that such a service is coupled to everything on the platform. If we are optimistic and suppose that it is decoupled, this may lead to memory leaks or undesirable behavior (e.g., class cast exceptions). The use of the root hierarchy object should be avoided.

Decoupling can be costly to developers and to maintenance aspects of software because it adds to the number of modules required. It tends to be expensive because current tools do not make it easy or automate the process sufficiently. Separating provider implementations from interface modules, and consumer implementations, and service extensions, all add more and more modules that need to be maintained. The current state of software development does not sufficiently support the developer when making fine-grained dynamic software.

Finally, there is a tendency to orient coupling from consumer components to provider components. Many projects place the Service Interface inside the provider s modules in order to reduce the number of modules. This makes the consumer s dynamic but changing a provider can have an extensive impact on the running system. We feel that developers should move to implementing fully independent Service Contracts that are maintained separately from both provider and consumer components.