How to write a Test Suite

This document contains a description on how you start to write your own test suite.

Concepts

This chapter describes some important concepts and classes in JET.

Setups

Setups are used to prepare the environment for a set of test cases. Normally this is to start the software under testing, populate the system with data or create some directory structure.

Test setups run on the client machine (where JET is run), if you want the test setups to do anything on the server machines, you must use clients and MBeans to communicate with the JAGs. If you run both the servers and clients on the same one host, this is not needed, but then JET is maybe overkill.

Test setups must inherit com.sun.jet.framework.TestSetup.

Example:

 package com.example;
 
 import com.sun.jet.framework.TestSetup;
 import junit.framework.Test;
 
 public class SomethingSetup extends TestSetup {
 
   private int pid = 0;
 
   public SomethingSetup(Test test) {
     super(test);
   }
 
   @Override
   protected void setUp() throws Exception {
     String path = getBindings().getBinding("software.path");
     pid = startSoftware(path);
   }
  
   @Override
   protected void tearDown() throws Exception {
     stopSoftware(pid);
   }
 }

To use this in a test xml file, write code like this:

<TestSetup class="com.example.SomethingSetup" ...>
... (test suites, test cases, ...)
</TestSetup>

There are also many examples of test setups in the source code repository, e.g. jet~framework/jet-examples/src/main/java/com/sun/jet/examples/DerbySetup.java

Test Cases

A test case class contains several test case methods that are run with the <TestCase class="..." method="..."> tag in a test XML file.

The test case class with all the test case methods should inherit com.sun.jet.framework.TestCase.

The test case methods take no parameters and throw Exception.

Example:

 import com.sun.jet.framework.TestCase;
 import com.sun.jet.util.TestinfraException;
 
 public class ProductTestCase extends TestCase {
 
   private static final Logger LOG = LoggerFactory.getLogger(ProductTestCase.class);
 
   @public
   public void setUp() {}
 
   @public
   public void tearUp() {}
 
   public void testHelloWorld() throws TestinfraException {
     LOG.info("Hello JET!");
   }
 }

The test case methods can contain any java code, and have access to getBindings() and other helper methods from the TestCase class. The setUp() and tearDown() methods are optional, and will be run before and after each test method in this test class if they exist. Test cases and test setups are constructed a long time before they are used (while the test xml file is parsed), so it is bad practice to do a lot of work in the constructor.

It is normal to create a ProductTestCase class with helper methods which all other TestCase classes for the given product inherits.

There are also many examples of test cases in the source code repository, e.g. jet~framework/jet-examples/src/main/java/com/sun/jet/examples/SimpleJetTestCase.java

Clients and MBeans

When you want to run your code on the server, and not on the client, the TestSetup or TestCase has to communicate with the JAG on the server to run code there.

The code that is needed to deal with this is normally divided in three parts, the MBean, the Ops and the Client.

There is an example in the jet~framework/jet-example-osgi/jet-example-osgi-test/ project.

MBean

The MBean is the JMX interface that is needed for JAG to know which methods to export. In JET it is normal to name this file <Interface>MBean.java. It must inherit com.sun.jet.jag.agent.JAGOperationMBean.

Example:

 import com.sun.jet.jag.JAGException;
 import com.sun.jet.jag.agent.JAGOperationMBean;
 
 public interface EchoMBean extends JAGOperationMBean {
 
   String echo(String source);
 
 }

Ops

The Ops implement the MBean interface. In JET it is normal to name this file <Interface>.java. This class must inherit com.sun.jet.jag.agent.JAGOperation, and implement the MBean interface.

Example:

 import com.sun.jet.jag.agent.JAGOperation;
 
 public class Echo extends JAGOperation implements EchoMBean {
 
   @Override 
   public String echo(String source) {
     return source;
   }
 
 }

Client

The client code runs on the client (JET/JETBatch) side and talks to the MBean over the network. It handles network exceptions and provides a nice interface for test cases and test setups to call the server side code. In JET it is normal to name this file <Interface>Client.java. It must inherit com.sun.jet.jag.client.JAGClient.

Example:

 package com.sun.jet.echo.ops;
 
 import com.sun.jet.jag.client.JAGClient;
 import com.sun.jet.util.JAGInstance;
 import com.sun.jet.util.TestinfraException;
 
 public class EchoClient extends JAGClient {
 	
 	private static final String SERVER_OBJECT = "EchoServer";
        private static final String SERVER_CLASS_NAME = "com.sun.jet.echo.ops.Echo";
 	
 	public EchoClient(JAGInstance jagAddress) throws TestinfraException {
 		super(jagAddress);
 	}
 	
 	@Override
 	public boolean init(String mletloader) throws TestinfraException {
 		return super.initImpl(mletloader, SERVER_OBJECT, SERVER_CLASS_NAME);
 	}
 	
 	public String echo(String source) throws TestinfraException {
 		String res = "";
 		try {
 			res = (String) invoke("echo", new Object[] {source}, 
                                                      new String[] {String.class.getName()});
 		} catch (Exception e) {
 			throw new TestinfraException(e);
 		}
 		return res;
 	}
 	
 }

Using the client

The client must be in the classpath of JET and JATBatch, while the MBean and the Ops must be in the JAG claspath.

To get your MBean and Ops classes to the JAG, you should place them in a separate testsuite-ops.jar, and add that with any dependencies to your jag.jagopsjars property in testsuite_jetdefaults.properties.

A client has a connection to the MBean in JAG. To get which machine (or JAG Instance - since one machine can run several JAGs) to talk to, you can read client and server machines from the bindings object with the getClientMachines() and getServerMachines() methods. To get the bindings, use getBindings() in test setups or test cases.

Configuration Files

JET and JETBatch uses several configuration files which are read from all the JAR files in the classpath. Default values for JET and JETBatch exists in jetdefaults.properties. When you write your own test suite, you can overwrite these default and add more configuration settings by adding a testsuite_jetdefaults.properties in your testsuite.jar file.

Validators

When you try to read or set a binding that does not exist (e.g. are not to be found in either jetdefaults.properties, nor testsuite_jetdefaults.properties), you will get an error. Sometimes you want to dynamically add bindings, and don't want to fail when you do so, in which case you have to write a Validator.

Validators must extend the com.sun.jet.util.bindings.JETBindingsValidator class, and implement the boolean validate(String key) method.

To use the validator, you must define then e.g. in your testsuite_jetdefaults.properties:

 com.sun.jet.util.bindings.<name>.class=some.class.<Software>BindingsValidator 
 com.sun.jet.util.bindings.<name>.name=Software Validator

An example of a validator exists in jet~framework/jet-example-web/jet-example-web-test/src/main/java/com/sun/jet/examples/web/JettyBindingsValidator.java.

Install Files

An installpath will be copied to the machines where the test is run before the tests start. To configure which directories this is, you must add a configuration, e.g. to your testsuite properties, or to the properties you use when you run the test.

 jet.installpath.<name>=<directory>
 jet.installpath.<name>.subdirs=<directory 1>,<directory 2>...
 jet.installpath.<name>.classpath=<jar file>,<directory>...

If you build to a directory structure like this:

 target
 target\bin
 target\doc
 target\data
 driver
 driver\bin
 driver\jars
 driver\javadoc

You might want to set up your installpaths like this:

 jet.installpath.product=null
 jet.installpath.subdirs=bin,data
 jet.installpath.driver=null
 jet.installpath.subdirs=bin,jars
 jet.installpath.classpath=jars/*

Now everybody have to specify jet.installpath.product and jet.installpath.driver when they start a test (becaise they are null, which is understood as not set), to specify which build to test. The doc and javadoc directories will not be copied to the test machines, which save time. The driver jar files will be added to the classpath, so they will be ready to use.

Plugins

Plugins are described on the ExtendingJET page. You can write them and place them in your testsuite.jar, and configure how to use them from your testsuite_jetdefaults.properties

Software

Software plugins let you report information (such as version) about the software you have been using in your test.

The software plugin must inherit com.sun.jet.util.software.SoftwarePlugin, and the getSoftware() method does all the work.

To use software plugins, you must define them in your properties:

 jet.software.general.plugin.<name>.class=some.package.<Something>Software

An example Software plugin exists in jet~framework/jet-example-web/jet-example-web-test/src/main/java/com/sun/jet/examples/web/JettySoftware.java.

Building your test code

If you want to build your test code with Maven, the jet~framework/jet-examples/pom.xml is a good starting point to figure out dependencies. If you don't want to use Maven, you should build your code against jet.jar and jag-api.jar. If you need any load framework, you should also build against jet-load.jar.

Test Files

A test file lets you reuse and combine test setups, test cases and clients. Look at WriteTest for more information about how to write a test file.

Read StartTest for information on how to run a single test with JET.

Batch

A batch file will let you run several test files together with JETBatch. Look at WriteBatch for information about how to write a batch file.

StartTest contains information on how to run a batch.