Reports
The JSystem report service provides a broad range of features and functions that enable the user to easily configure the reporting mechanism to the project requirements, they include:
-
HTML based reporting mechanism.
-
Easily add graphs and diagrams.
-
Easily add attachments and links.
-
Hierarchical structure.
-
Rich report UI (html hierarchy, links, graphs and formatting).
-
Pluggable framework.
The Reporting service is an important element required for the visibility of an automation project. The reporting mechanism enables the user to quickly analyze the cause of any test execution failure. Note: It is recommended that the user establishes a reporting convention before the commencement of an automation project.
As is shown in the diagram the test /System Object is run and sends a report message to the “Report Listener Manager”, as a result the “Reports Listener Manager” sends the event to the HTML reporter, XML reporter and the Systout Reporter. The user can add additional reporters as required by implementing the reporter java interface and updating the JSystem properties.
Note: The customizable reporter service enables the programmer to add reporter services to the output of the “Reporter ListenerManager”.
The reporter is a customizable service where by the programmer can customize a single reporter plug-in that enables the sending of report messages to different terminal reporters. Each reporter can then receive the reporter information and process it according to the locally configured requirements. Unlike other reporting systems the JSystem Reporter has the ability to influence the success status of the test. That means that when a fail report message is sent the system will identify the test as a failed test.
When the JSystem framework is run, it automatically starts the report service. The user writes a test/system objects, within the test code the user then activates the Reporter Listeners Manager. Once the test has been written and is complete the user can now implement the test within a scenario. The user now runs the scenario and the report service automatically runs and begins to track report events. The report events arrive to the report service and are transferred to the various Reporters as is shown in the illustration.
The reporter service is used by the framework as a means to connect into test flow and to enable the “pause” and “graceful stop” functions. In addition to the direct benefits of using the JReporter system that provides a rich and informative test log, it is important to use the JReporter service to generate increased behavioral test functionality by implementing the “pause” and “graceful stop” features.
The code example illustrates a simple traffic test that generates traffic and then analysis it. The purpose of this piece of code is to demonstrate how the reporter usage is incorporated into the test code.
/**
* Simple play.
*
*/
public void testSimplePlay() throws Exception {
/*
* Send burst of 100 packets, test that 100 packets received
*/
report.report("Set transmit rate to 100 packets per second");
tg.port[0].ps[0].setRatePPS(100);
tg.port[1].ps[0].setRate(200);
report.step("Set burst size to 100 packets");
tg.port[0].ps[0].setBurstMode(100);
report.step("Set port 0 to send packets");
tg.aPlay.setPlayers(new int[] {0});
report.step("Set port 1 to receive packets");
tg.aPlay.setReceivers(new int[] {1});
report.step("Set modes, play and analyze");
tg.aPlay.setModes(false, false, false, false);
tg.aPlay.play();
tg.aPlay.analyze(new TrafficCounter(1, EthernetPortCounters.RECIEVE_PACKETS, 100, 0.1));
tg.aPlay.analyze(new TrafficCounter(1, EthernetPortCounters.DATA_INTEGRITY_ERRORS, 0, 0));
}In order to configuring the reporter service the programmer must update the “jsystem.properties” file. The following table lists the relevant properties.
| Property | Description | Comments |
|---|---|---|
| logger | Enable/disable the reporter | Possible values are true/false |
| htmlReportDir | Output folder of the reporte****rs | |
| html.old.dir | Html reports backup | |
| html.zip.disable | Enable/disable reports backup | Possible values are true/false |
| warning.color | The color of warning messages | |
| summary.disable | When set to true, signal the system to not aggregate events in JRunner heap memory | Should be |
| reporter.classes | Use this property to add custom reporters | You should point the reader to the section that describes how to extend the reporter |
| browser | The browser that will be used by the JRunner to open the html reports |
The most commonly used reporter application for the analysis of execution results is the “HTML Reporter”. The following section will demonstrate the reporter API functionality and show how the API is reflected in the HTML Reporter within a standard internet browser application.
In order to run the JSystem Report example shown, the programmer writes the following piece of code in “Eclipse” and compiles the code. The user can run the code example from within either the “Eclipse” or “JSystem” environments.
/** * Demonstrates basic reporter methods.
*/
public void testReporterBasics() throws Exception{
//Logs simple reporter message
report.report("message title");
//Logs a report message with title and internal content with success status
report.report("message title","internal message",true);
}The HTML browser screen shot illustrates the result of the basic code example above.
The second layer of the test is accessed by pressing the highlighted “Blue” colored “Title” link as illustrated in the HTML example. The next page illustrated in the screen shot is the second layer of the test example’s output.
The following code example demonstrates the implementation of either a test warning status indicator or a test fail status indicator in the HTML browser application after compilation.
/**
* Demonstrates reporter message which causes test to fail or to be in warning status
*/
public void testReporterWithErrors() throws Exception{
report.report("report with error","internal message",Reporter.FAIL /** Reporter.WARNING */);
}The screen shot illustrates the result of the “Warning” and “Fail” code examples.
Figure 60: Success Status HTML Reporter Example
The following code example demonstrates the implementation of a test fail status indicator with an exception status indicator in the HTML browser application after compilation.
/**
* Demonstrates reporter message which causes test to fail with an exception.
*/
public void testReporterWithException() throws Exception{
report.report("report with exception",new Exception("An error"));
}
By pressing the “report with exception” link the user accesses the second layer of the report that illustrates the exception.
The snapshot shows the HTML report when drilling down into the exception.
The code example demonstrates the use of steps; steps are reflected in two places, they are saved in the JReporter application database, and displayed in the test summary report in the JReporter application.
/**
* Demonstrates step usage.
* Steps are reflected in two places:
* 1. Steps are saved in the Jsystem reports application database, and reports which show a summary
* of all the steps in the test can be generated from this information.
* 2. Steps are showed in bold format in the HTML reporter and Jsystem reporter tab.
*
*/
public void testReporterStep() throws Exception{
report.step("step description");
}The screenshot shows the HTML report that is reproduced by the “step” method.
It is important to note that in order to produce output from the JReporter in the form of a self contained file that the programmer wishes to link, it should be copied as follows. Note: In order to perform this function there is a utility class that copies the file to the reporter folder adding a link to the file.
The code example demonstrates linking a file in the reporter.
/**
*/
public void testReporterAddLinkToAFile() throws Exception{
File f =station.getFile("myFile.txt");
ReporterHelper.copyFileToReporterAndAddLink(report, f, "Link to file");
}The screenshot shows the HTML report of the “testReporterAddLinkToAFile” test.
The screenshot shows the HTML page that opens after pressing the “Link to file” link.
This piece of code demonstrates the integration of an HTML visual graph element to the report structure.
/**
* Demonstrates report with graph.
*/
public void testReportWithAGraph() throws Exception{
Graph graph = GraphMonitorManager.getInstance().allocateGraph(
"Dummy graph of number of recieved packets in ping operation over time", "received packets");
for (int i = 0; i < 10;i++) {
int receive = new Random().nextInt();
graph.add("receive", i, receive);
}
graph.show(report);
}The compiled code executes the following HTML graph within the JReporter.
Figure 66: Graph HTML Report
The report API enables the programmer to create a hierarchical report where by each level contains a link to the next level. By implementing the leveling function the user can build clean focused reports.
Each level of the report contains a link to the next level, using the leveling code example, the user can build cleaner and more focused reports.
/**
*/
public void testReportWithLeveling() throws Exception{
report.startLevel("first level", Reporter.MainFrame);
report.report("message in level 1");
report.startLevel("second level", Reporter.CurrentPlace);
report.report("message in level 2");
report.startLevel("third level", Reporter.CurrentPlace);
report.report("message in level 3");
report.stopLevel();
report.report("another message in level 2");
report.stopLevel();
report.report("another message in level 1");
report.stopLevel();
report.report("message in main report page");
}The screenshot of the HTML report shows the “testReportWithLeveling” test. The main page of the test report has a link in it “first level” by pressing it the user will be directed to the test’s first level page.
The snapshot shows the “first level” web page. Notice that the “first level” page has a link to “second level”. The user has the flexibility to create as many levels as is required.
The code method illustrates the usage of pop-up interactive messages from within the reporter API.
This method shows how to add a pop up interactive messages to the user interface using the reporter API.
public void testInteractiveMessage() throws Exception{
int res = report.showConfirmDialog("Confirm Dialog","Continue test?",JOptionPane.YES_NO_OPTION,JOptionPane.INFORMATION_MESSAGE);
report.step("user reply = " + res);
}The snapshot shows the JRunner when the interactive message is processed.
