Synthetic Traces

Section 5.1.1 detailed that a Pajé trace file is composed by three sections: the header, the type hierarchy and the timestamped events. The header is the only static part of file, where events are defined with their particular fields. The type hierarchy defines the types that will be present – such as cluster, machine, processor, processes, functions – and the hierarchy among them. The type hierarchy must be followed through the rest of the file in the timestamped events region.

Large-Scale Hierarchies

The first synthetic trace generator tool was created targeting the visual aggregation model. The tool is written in the Python language and receives as parameter a hierarchical structure that configures the generation of the trace. Listing 6.1 shows an example of configuration file that is passed as parameter to the tool. The file is organized hierarchically to reflect the type hier-archy that is generated as output. Each level (eg, Site, Cluster, Machine and Processor) has an attribute container that indicates the number of instances of that type that must be created by the tool. In the example, the configuration tells the tool to create 5 different sites, each one with 3 clusters, each cluster with 100 machines and each machine being composed of 4 processors.

The attributes alias and name are used by the tool to comply with a trace generation required by the Pajé format.

Listing 6.1 – Example of configuration file for the large-scale trace generation tool.

c o n f i g = {

’ c o n t a i n e r ’ : 5 , ’ name ’ : " S i t e " , ’ a l i a s ’ : " S " ,

’ c h i l d ’ : {

’ c o n t a i n e r ’ : 3 , ’ name ’ : " C l u s t e r " , ’ a l i a s ’ : "C " ,

’ c h i l d ’ : {

’ c o n t a i n e r ’ : 1 0 0 , ’ name ’ : " Machine " , ’ a l i a s ’ : "M" ,

’ c h i l d ’ : {

’ c o n t a i n e r ’ : 4 ,

’ name ’ : " P r o c e s s o r " ,

’ a l i a s ’ : " P " ,

’ s t a t e a l i a s ’ : " S " ,

’ s t a t e n a m e ’ : " S t a t e " , }

} } ,

’ a p p d u r a t i o n ’ : 2 0 ,

’ c o s i n e−max−x−a x i s−v a l u e ’ : 7 . 5 ,

6.1. TRACES DESCRIPTION 87

}

Still on Figure 6.1, the last level of the structure – Processor in the example – receives additional configurations: statealias and statename indicating the presence of a state on the containers created in that level. The time duration of the synthetic trace is configured through the appduration. The parameter cosine-max-x-axis-value controls the distribution of state values for the instances of containers in the last level. Its value is used to configure the cosine function from the interval0to the configured value. The tool maps the containers instances of the last level to the xaxis of the cosine to find the amount of time – in percentage from 0 to 1 in the y axis of the function – a given container stays in one of two possible states. The remaining percentage is used to set the amount of time to the other state.

The graph of Figure 6.1 is configured using the data of the example in listing 6.1. The graph is used to define the duration of each of the two states available for every leaf instance container of the hierarchy. Using the configurations, the cosine function varies within the x interval[0,7.5]. There are 6000 processors (result of the multiplication of all container attributes value:4×100×3×5). The Figure also shows the definition of duration for the two states for the container number 4000. The value of the correspondingxvalue in the lower scale is 5. The cosine of 5 is 0.28. Since the values of cosine vary between -1 and 1 in the y-axis, we consider that this value of 0.28 represents 64% of the interval[−1,1]. So, this percentage is used to define the amount of time of the State-0 for the container 4000, which is 12.8 seconds considering the total application duration of 20 seconds. The rest (7.2 seconds) is left to the State-1 of container 4000.

0 1 2 3 4 5 6 7

0 1000 2000 3000 4000 5000 6000

value for cosine equation number of leaf container

State-0 State-0

State-1

State-1

Division for container 4000:

64% for State-0, 36% for State-1

Figure 6.1 – State distribution among leaf containers using the cosine function.

Although the tool is implemented using the cosine function, it could be easily adapted to use other trigonometric functions. The way the state generation is implemented limit the study of different time intervals, as defined by the visual aggregation model. The positive side of the implementation is that it allows the fast generation of traces composed of hierarchies with thousands of nodes. The implemented trace generation tool takes less than 3 seconds to generate a hierarchy with more than 150 thousands leaf containers in a four-level hierarchy. A random state value generation was considered to implementation, but initial tests have shown that the execution time for large-scale hierarchies is too big when using a random number generator.

Typical Communication Patterns and Complex Topologies

The second synthetic trace generator tool targets the three dimensional approach. The main ob-jective is to generate trace files with traditional communication patterns, such as the ones used by master-slave or divide and conquer parallel applications. Listing 6.2 shows the configuration file used by this tool. It earns the basic configurations from the previous script, letting the user configure a hierarchical organization of containers if necessary. We implemented four types of communication patterns: ring, fully connected, star and hierarchical star. The user config-ures the type of pattern used through the option apppattern. If the user uses the hierarchical star communication pattern, an additional option called nchildren is necessary to configure the number of children in the communications. For example, if the nchildren parameter is set to 2, every container will communicate with other 2 containers. Each one of these two containers will communication with other 2 containers, and so on, forming a hierarchical communication pattern. The last options in the bottom of the listing are related to the Pajé links configuration.

The options linkalias and linkname are used to configure the type hierarchy for the Pajé trace file, and the linksource and linkdest indicate which types of container can be used by these links.

Listing 6.2 – Example of configuration file for the synthetic communication pattern trace gener-ation.

c o n f i g = {

# h i e r a r c h i c a l d e f i n i t i o n s e c t i o n

’ c o n t a i n e r ’ : 2 0 , ’ name ’ : " Machine " , ’ a l i a s ’ : "M" ,

’ s t a t e a l i a s ’ : " E " , ’ s t a t e n a m e ’ : " S t a t e " ,

’ a p p d u r a t i o n ’ : 2 0 , ’ c o s i n e−max−x−a x i s−v a l u e ’ : 7 . 5 ,

# ’ c h i l d ’ : {} # h i e r a r c h y w i t h o n l y one l e v e l i n t h i s e x a m p l e

# c o m m u n i c a t i o n p a t t e r n s s e c t i o n

’ a p p p a t t e r n ’ : " r i n g " , # r i n g , o r f u l l , o r s t a r , o r h i e r a r c h i c a l−s t a r

# p a r a m e t e r s t o " h i e r a r c h i c a l−s t a r " a p p p a t t e r n

’ n c h i l d r e n ’ : 2 , # number o f c h i l d r e n p e r n o d e

# l i n k s c o n f i g u r a t i o n

’ l i n k a l i a s ’ : " P " , ’ l i n k n a m e ’ : " L i n k " ,

’ l i n k s o u r c e ’ : "M" , ’ l i n k d e s t ’ : "M" , }

As previously stated, the four types of communication pattern that can be generated by the tool are the ring, the fully connected, the star and the hierarchical star. In the ring pattern, each container communicates exactly with other two containers, forming a single and continuous

6.1. TRACES DESCRIPTION 89 pattern among all nodes. Figure 6.2(a) is an example of this pattern when there are 6 containers participating of the communications.

(a) Ring (b) Fully connected (c) Star (d) Hierarchical Star

Figure 6.2 – Different communication patterns generated by the second synthetic trace genera-tion tool.

Figure 6.2(b) shows the fully-connected communication pattern, where all containers municate with all other containers. Figure 6.2(c) shows the star pattern, where all nodes com-municate with only one node. This type of pattern is typically found in master-slave parallel applications. The last communication pattern, represented in Figure 6.2(d), is a modified ver-sion of the star pattern, but with a hierarchical organization where each node has communica-tions with other two nodes. In the example of Figure 6.2(d), the hierarchy is binary, but other configurations are also possible.