Testing Your Java Frameworks

RADE	Java Unit Test Manager	Testing Your Java Frameworks How to write, deliver, and run test programs in the CAA V5 environment
Technical Article

Abstract

Like for development tasks, tests activities can take place in your environment that provides an infrastructure for writing, delivering, and running test programs associated with CAA V5 frameworks. This article explains the structure of the test frameworks and how to write test programs, and what are the interests of delivering them in the CAA V5 file tree.

Why Should You Use the CAA V5 Test Infrastructure?
How to Write Test Programs
How to Run Tests
- Inside a Workspace
- Controlling Test Execution
In Short
References

Why Should You Use the CAA V5 Test Infrastructure?

CAA V5 supplies an infrastructure to test your frameworks and applications. This infrastructure is fully integrated in the development environment you use to develop your frameworks and applications and takes advantage of the same tools and mechanisms. In addition, it allows you to automate the test program execution.

[Top]

Tests Are Related to Frameworks

Test programs are written for a specific purpose (i.e. test a specific program!). As a consequence it seems to be obvious that they should be related to, and delivered with the application frameworks they are intended to test. This the CAA V5 file tree provides test frameworks that can contain any kind of programs and that contains, as any framework, an IdentityCard file specifying the tested application frameworks.

[Top]

Tests Can Be Developed and Shared As Well As Programs

Test programs can be as complex as any program, thus the same problems can be encountered: concurrent engineering, parallel development, and so on. Using the test framework, developers can:

Build test programs: the software architecture which is supported for CAA V5 programs is available for test programs as well. You can then use the generic code builder mkmk [1] and structure your programs using modules
Exchange: any user is able to create and to deliver a test framework. Conversely, any user can get a copy of an existing test framework, parallel developments and concurrent engineering are allowed like for application frameworks
Run everywhere: using predefined variables, observing some rules and delivering test programs in test frameworks, you are able to execute them from any workspace.

[Top]

A Predefined Environment Eases the Development of Test Programs

There are often two problems to address when developing test programs: on one hand, you must think of the way you'll test your application, and on the other hand you must set up the runtime environment and automate these tests. This second aspect has been studied and some solutions can be brought in the CAA V5 scope:

Tests are run by a particular command named mkodt [2] that sets up the runtime environment both for CAA V5 applications and for the test programs themselves
Predefined directories and environment variables are available for test programs to find external resources, such as input data, reference output, and the like
The CAA V5 infrastructure can be used whether tests are run on Unix or Windows platforms. This is possible thanks to specific predefined variables that are independent from the operating systems [2] and by following some basic rules [3].

[Top]

Tests Can Be Run Automatically

The fact that test frameworks can be delivered along with application frameworks and that their test programs can be run automatically allow the automation of some test based certifications before code delivery. You will also see that CAA V5 provides a way to record sequences of user interactions in order to replay them afterwards.

[Top]

How to Write Test Programs

The basis of CAA V5 tests is composed of programs that use the interfaces and classes exported by the application frameworks, and shells that are containers for those programs. Shells are used as containers because they can be run with Unix, and withWindows since the Korn Shell is a prerequisite for CAA V5 tools. Test programs and shells are delivered in test frameworks.

[Top]

What Is a Test Framework?

The test framework is one of the three types of CAA V5 frameworks, it provides a predefined structure to store both test programs and related data. A test framework is composed of:

The framework name: it usually is the same as the associated application framework but with .tst as suffix.
The Identity Card: it contains references to application frameworks which are prerequisites for test programs - as these programs are supposed to test a specific application framework, this framework should be one of those references.
Modules: like in application frameworks, a test framework may contain modules which correspond to test programs. Note also that usual interface directories are available in test framework (fw/PublicInterfaces, fw/ProtectedInterfaces, fw/mod.m/LocalInterfaces, ...).
FunctionTests: a specific data module exists in which program container as well as test related data should be stored in (see next figure).

The FunctionTests directory contains predefined directories but, inside them, the user is free to create his/her own structure of files and directories. Here are these directories and what should be stored/find in each of them:

TestCases	This is the directory where shells must be stored. Shells bear .sh as suffix and their purpose is to firstly set up some test specific environment variables, secondly to run the program and finally to return an appropriate error code regarding the program return code and outputs.
InputData	You can store in it any data that could be used as input for your programs. There is no particular suffix defined for those data.
Output	This directory is the repository of all outputs and data that are produced during test program execution. For instance, if a program generates a file, this should be generated in this directory.
OutputRef	This directory is used to store any "reference" data which allows shells to compare what is produced by programs (output messages, files, and so on) with what should be produced (if they worked!). For example, if you want to test a program which automate some modifications on a CATIA model, you can first perform these modifications "by hand" and save the result in a file you store in the OutputRef directory, then make a comparison (in the shell) between this model and the one produced by the test program in the Output/$OS directory.
SwitchTestCases	This is the directory where switch test program shells are stored. These test programs are used to decide whether to switch the current level without replaying all the test objects. They should be representative of the framework
TestCases	This is the directory where common test programs shells are stored. Shells bear .sh as suffix and their purpose is to firstly set up some test specific environment variables, secondly to run the program and finally to return an appropriate error code regarding the program return code and output.

The next section shows you the way to reference these directories and to write portable shells.

[Top]

Referencing External Data

Shell variables are automatically set according to the current test environment (i.e., the workspace from where the tests are run). You can use them to access your data whatever the absolute path to them may be [2].

[Top]

Basic Shell Samples

The minimum that can be written.

#------

SetOdtParam TYPE=JAVABATCH

#-----

JavaReplay MyApplet
exit $rc

Let's detail it:

The first line is a comment (starts with '#').
The next line tells that this ODT is a Java ODT
The line containing the JavaReplay command will launch the applet MyApplet
If the program fails, the shell exits directly using the JavaReplay return code as error code. Otherwise, additional checking may be performed.

In addition to these samples, some rules can help to write shells which can be executed on both Unix and Windows platforms [3].

[Top]

How to Run Tests

As mentioned in previous sections, tests are not directly executed by the user but their execution is controlled by a particular command: mkodt. Its purpose is to prepare the test environment by setting variables needed at runtime according to the local user environment. It is available with "mkmk" commands.

[Top]

Inside a Workspace

Tests always take place in a workspace and are used to check what is developed in this workspace, thus only programs and shared libraries that are visible in a workspace can be tested by test objects in this workspace. However, note that the concatenation feature brings visibility on data managed among multiple workspaces.

Any test can be run either being under a workspace - the current working directory is under the workspace root directory - or by referencing a workspace. Both modes are specified through mkodt options [2]. One test object or all test objects can be run at user's convenience.

We'd just like to give you an idea on how you can put in place some automatic tests in order to certify the quality of your software. Here is an example of an organization where several people are working on the same software, each of them owning a workspace from which he/she delivers pieces of software. Regarding the workspace hierarchy, we can see that all modifications must go through the project workspace. This project workspace is a place where automatic tests should be run each time there are some new modifications to promote outside the scope of this organization.

[Top]

Controlling Test Execution

A shell can include specific commands to control some aspects of its execution. These commands are:

`ODTInit` [4]	Runs an initialization shell script
`SetOdtParam` [5]	Sets ODT parameter values
`WhichDir` [6]	Finds a directory or a file in a workspace

Top]

A shell for a java ODT must include specific command to be launched on every JRE. This command is:

JavaReplay [7] Runs the applet or the java application without taking care of the java environment

In Short

Like program source files are managed in application frameworks, the test framework provides a predefined organization of files and directories (modules, FunctionTests, etc.) in which:

Test applications can be delivered and built using mkmk.
Any useful data can be stored and used at runtime.
A test object consists of at least one shell script which can call either a specific java test program.
Predefined variables are provided to get test objects that can be run on any supported operating system (Unix, Windows).
The runtime environment is automatically set by the mkodt command in order to be able to run the java application with the given JRE.

[Top]

References

[1]	mkmk
[2]	mkodt
[3]	Writing Shell Dos and Don'ts
[4]	ODTInit
[5]	SetOdtParam
[6]	WhichDir
[7]	JavaReplay

[Top]

*History*
Version: 1 [March 2001]	• Document created
[Top]

RADE

Java Unit Test Manager