Jubula is a kind of tool which is used for automated functional GUI testing. Jubula supports Java and HTML applications, so it's possible to test the 4DIAC-IDE with this software tool. The tests are written from the user perspective and they are keywords-driven. Besides the test case generation is based on dragging and dropping actions from a library, so special programming skills are not necessary. In the following sections the basic principles of "Testing with Jubula" are shown. The term AUT stands for Application Under Test and is the GUI you want to test.
Choose the Jubula Standalone product on the Jubula Download Page and select your operating system. Under Windows:
Before we can get started working with Jubula, some preparative tasks are necessary. You need an installed 4DIAC-IDE. It's not possible to test the 4DIAC-IDE when you are running the IDE in development mode (from source). That means you have to build a binary package. For installation instructions see Building a binary 4DIAC-IDE package from source. Then:
After the configuration we want to start creating a test case. The following picture shows an overview of Jubula.
As an example the creation of the "Create a new system in the 4DIAC-IDE" test case is shown.
In the section before, we have defined some new component names, but at the moment there is no relation to the real object (Jubula calls them technical names) in the AUT, because this is a kind of black box test. So we have to map the component names to the technical names. Therefore we have to the following steps:
You can easily import existing projects. Select the desired project (.xml-document) via Test-->Import. Now you have to adapt the AUT-configuration to your system. Please refer to the "AUT-configuration" chapter. Export your project via Test-->Export. After entering a name a .xml-file is created.
In this chapter some key elements of Jubula, useful hints and also occured problems are described.
A Test Step is the smallest unit and represents one action on one component. A Test Case is the next level in the hierarchy and can contain test steps and other test cases. Then follow Test Suites, Test Jobs and finally the Project is on top of the hierarchy. It is recommended to create a keyword for every repeated action . A keyword is a test case which executes actions. You can reuse keywords and a possible change is only on one place necessary. Keywords can contain parameters. An example could be the Insertation of a Function Block via Type Navigator shown in the following picture. The parameters are the full path name of the FB and the relative coordinates of the editor, where the FB should be dropped. The keyword consists of two actions. The adressed components (tree and graphical component) are fixed.
It's also possible to make the components variable. Then the test case is called Utility module. An example is given in the picture below. You just have to check the box in the Component Name browser.
It's well worth creating an inital state. The IDE saves the last perspective and so you never know the exact state (which windows, tapped panes are open...). I created the following test case to start each use case.
With Event Handlers it is possible to react to errors during test execution. So you can guide your automated test, if an error occours. An Event Handler is a test case with (user-)defined actions. There are basically four types of errors:
For each error you can define the reentry type, return, retry, exit, continue, break, pause. Global Event Handlers should be added to each use case. With local Event Handlers you can handle "expected" errors, without activating the global Event Handlers.
The 4DIAC-IDE is a RCP application and the GEF (Graphical Editing Framework) toolkit can be used to test graphical components. The editor is called figure canvas. There the following items can be displayed (also shown in the graphic below):
All these items can be adressed with Jubula. To get the names of the items, the GEF inspector is used. Activate the GEF inspector via Window-->Show view-->Inspector. In the following picture you can see the inspector and the name of a connection anchor. Click on the marked icon to start the inspection. Then switch to the IDE and click on the desired item. It's not possible to get multiple names during one execution of the inspector.
If you have the names/adresses of your components you can do some special actions with them. You can find these actions in Jubula Test Case Browser --> unbound modules rcp
Annotation: I tried to use the click_connection action but it didn't work, so i used the drag_drop action to connect to graphical components.
It is possible to drag a FB out of the Type Navigator (tree) into the editor (can be seen as graphical component in this case). So use the drag action for the tree type (source) and the drop for the destination, as shown in the following picture.
You can edit properties of a FB in the properties-view. Therefore the FB has to be active and then you set the properties. In Jubula you can select the desired cell in the table. The problem is, if you want to write a text in the cell and the cell is not “visible”(not shown in the window, because you don’t have scrolled down), like for example the “Label/Value”-cell it is not working. My first guess was to maximize the properties window, but then the 4DIAC-IDE doesn’t apply the text input. So I moved the properties window to the right side of the IDE to see more properties. It’s not that easy to resize and move windows in Jubula. To scroll to the desired cell would be the best solution. You can also adress the property-items in a FB directly (Click the desired property). Then you just have to enter text and you don't need to work with tables.
Error description: When deleting an existing system and after that a new system is created with the name of the previously deleted system it will be twice in the system management.
Reproduction: After the inital step it is checked, if the system already exists. If it exists, it's deleted, otherwise nothing happens. After the system is deleted, the check is executed again to be sure there is no second system with the same name. This happens in the DeleteViaSystemManager step. Then the new system is created and afterwards the check function is called again. This procedure is repeated again and so it's possible to detect if a system is twice or not.
Error description: It's not possible to insert FBs from the type navigator into the application.
Reproduction (main steps): A new system is created after it was checked that a system with the same name doesn't already exists. A new application is created. Open the application and the type navigator view. Insert a FB from the type navigator into the application. Check if FB exists in application.
Annotation: The global event handlers are not shown in the graphics.
The defined use cases (test cases), mentioned above, have been executed on two different versions of the 4DIAC-IDE. The test results are:
First of all good knowledge of the IDE which should be tested is a basic thing. That means you should know about the capabilities of the IDE. What’s more structuring of your test is very important. As mentioned above you can create keywords and modules. Basically such small parts should be created for every action being repeated. Using naming conventions and describing actions and parameters is essential too. Once you have built up a set of keywords and modules creating different use cases is getting easier and faster. For each test case an initial step is recommended because the IDE saves the last perspective, but we want a well defined state to start the test. Especially things like moving or resizing windows and other objects is not that easy in Jubula. It can take a lot of time handling such problems. There are basically four types of errors. For each of them a global error handler which performs defined actions should be set. Local error handlers can be used for conditional branches. All in all Jubula is an appropriated kind of tool to reproduce errors, but of course a reproduction of all bugs is not possible.