Getting Started - Writing your First Test Suite
This page should tell you how to get started writing your first test suite. More advanced test suites can be found in TestGearman, TestMemcached, TestWebapp and TestOSGi.
- 1 Getting Started - Writing your First Test Suite
- 1.1 Prerequisites
- 1.2 Directory Layout
- 1.2.1 C-code projects
- 1.2.2 Java projects
- 1.2.3 Example Project
- 1.3 Creating the code
- 1.3.1 Dependencies
- 1.3.2 The Test
- 1.3.2.1 Test XML file
- 1.3.2.2 Run Script
- 1.3.2.3 Test Setup
- 1.3.2.4 Test Case
- 1.3.2.5 Client/MBean/Ops
- 1.3.2.6 Calling client code from test case and test setup
- 1.4 Running the test with JETBatch
- 1.4.1 Start JAG
- 1.4.2 Configuration files
- 1.4.3 Install directory
- 1.4.4 Testsuite default
- 1.4.5 Code Changes
- 1.4.6 Running the batch
- 1.4.7 Inspecting the JAG logs
Prerequisites
You must download and Build JET before you can write your test suite.
Directory Layout
C-code projects
I believe that the test suite should live with the software under testing in the same source code repository. This ensures that the tests are kept up to date with the product.
For a c-code project, I would have a directory structure that looks something like this:
product/ product/Makefile product/src product/docs product/utils product/tests product/tests/pom.xml product/tests/src product/builds product/builds/linux_i386/ product/builds/solaris_sparc/ product/builds/solaris_amd64/
The product directory contains the source code for the software that is tested, and is the root directory in the version control system. It has a Makefile, a src directory for the source code, and some other directories (docs, utills, …).
The product/tests directory contains a normal Maven or Ant java project to build the test suite. The outermost Makefile is responsible to start the test suite build as part of a normal product build.
The product/builds directory stores the builds for the different platforms. This requires a build out of source directory feature of autotools or CMake or similar. For each of the platforms under product/builds one would have the same directory structure, that could look like this:
bin/ bin/product bin/productd docs/ libs/ libs/product.so libs/product-driver.jar etc/ etc/product.conf tests/ tests/testsuite.jar tests/jet.jar tests/jag.jar tests/jag-api.jar
The product/builds/<platform>bin directory contains the binaries of the product, docs contain documentation, lib libraries, and tests the test suite and jar files used to run the tests.
To run the tests with such a layout, the testsuite_jetdefaults.properties in testsuite.jar would contain:
jet.installpath.product=null jet.installpath.product.subdirs=bin,libs,etc,tests jet.installpath.product.classpath=tests/*,libs/*
When somebody StartTest with JETBatch, they would have a property file, e.g. linux_i386.properties, sent to JETBatch with the -props command line option, which would contain something like:
jet.installpath.product=builds/linux_i386/ jet.clientmachines=penguin01 jet.servermachines=penguin02,penguin03 jet.portbase=31337
This says both which software to test, and on which machines to test it, and sets an unique port base that makes sure that the ports this user uses does not collide with ports from other users.
Java projects
If this wasn't a c-code project, but a Java-code project, I would also place the test suite as an individual sub project inside the product. With Maven, I would have product-testsuite be a sub-module of the outermost pom.xml, and have it depend on the parts of the product that it needs to run the test. Examples of this can be found in the jet-example-* modules in the JET source code repository.
Example Project
For this getting started guide, I'll make it as easy as possible - so I will just make test test suite project.
Creating the code
I will use Apache Maven to build the getting started test suite. If you are note familiar to Maven, you can read the Getting Started Guide.
$ cd devel $ mvn archetype:create \ -DarchetypeGroupId=org.apache.maven.archetypes \ -DgroupId=com.sun.jet \ -DartifactId=jetintro … [INFO] BUILD SUCCESSFUL … $ cd jetintro
I will now have a directory structure that looks like this:
pom.xml src/ src/main/ src/main/java src/test src/test/java
I will start by deleting files I do not want
$ rm src/main/java/com/sun/jet/App.java $ rm src/test/java/com/sun/jet/AppTest.java
And then build nothing:
$ mvn install
This gives me a new directory with my jar file:
target/ target/jetintro-1.0-SNAPSHOT.jar
OK, so we have the basic directory structure and build mechanisms, now we want to add the JET tests.
Dependencies
First we will add dependencies to JET in the pom.xml:
I will change the content of the <dependencies> tag to contain dependencies for JET:
<dependencies>
<dependency>
<groupId>com.sun.jet</groupId>
<artifactId>jet</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.sun.jet</groupId>
<artifactId>jet-util</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.1</version>
</dependency>
<dependency>
<groupId>com.sun.jet</groupId>
<artifactId>jag-api</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
If you were not using Maven, you would probably place these jar files in a extlib directory, and make sure they exists in your classpath during build and run.
I'll also specify that I use Java 1.6:
<properties> <compileSource>1.6</compileSource> <compileTarget>1.6</compileTarget> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties>
and this inside the <build> tag.
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>${compileSource}</source>
<target>${compileTarget}</target>
<encoding>${project.build.sourceEncoding}</encoding>
</configuration>
</plugin>
</plugins>
The Test
We want the test to do this:
- Start a Echo server on the server machine
- Test that the echo server works
To run this test, we will need several parts
- A test setup to start the echo server
- A test case to test that the echo server works
- A Client/MBean/Ops combination to work with the echo server software on the JAG on the server machine
- A test XML file to tie the parts together
Test XML file
We will place the test XML files in the src/test/jet/ directory:
mkdir -p src/test/jet
And make sure that the XML files in this directory ends up in our testsuite jar file:
<build>
<resources>
<resource>
<directory>src</directory>
<includes>
<include>test/jet/**/*.xml</include>
</includes>
</resource>
</resources>
</build>
Then we create our test xml file, named src/test/jet/getting_started.xml, with content like this:
<?xml version="1.0" encoding="UTF-8"?>
<Test maxduration="2" xmlns="http://kenai.com/projects/jet" version="1.0">
<TestDescription author="jorgen.austvik@oracle.com" testobject="Echo Server">
<Objective>Test the Echo server</Objective>
<EnvironmentalReq></EnvironmentalReq>
<FailCriteria></FailCriteria>
</TestDescription>
<TestSetup class="com.sun.jet.echo.EchoServerSetup" comment="No test setup">
<TestSuite>
<TestCase class="com.sun.jet.echo.EchoTests"
method="testEcho"
comment="Tests the echo server echo method"/>
</TestSuite>
</TestSetup>
</Test>
The first part of this XML file gives some information about what we are testing, and the maximum duration this test will run for (2 minutes). We then describe the test, that will start up the echo server with a setup, and then run a single test case.
This XML file might seem verbose at first glance, and it is. The reason it is there, is that it lets us stitch elements together to create new tests quickly later. For instance the test setup can be used for many tests that test it, e.g. a load test later.
This file should now be in your JAR when you build:
$ mvn install $ jar -tf target/jetintro-1.0-SNAPSHOT.jar | grep xml test/jet/getting_started.xml ...
Run Script
To experiment with our test, we will create a simple run script, named run.sh:
#!/bin/bash
JAVA="java"
JVM_ARGS="-Xms64m -Xmx512m"
VERSION="1.0-SNAPSHOT"
JET_CLASS="com.sun.jet.framework.engine.JET"
MVN_REP_DIR="${HOME}/.m2/repository/com/sun/jet"
echo "Copy needed jar files:"
cp "${MVN_REP_DIR}/jag-ops/${VERSION}/jag-ops-${VERSION}.jar" jag-ops.jar
cp "${MVN_REP_DIR}/jet/${VERSION}/jet-${VERSION}.jar" jet.jar
echo "Starting example test:"
TESTSUITE_JAR="target/jetintro-1.0-SNAPSHOT.jar"
$JAVA $JVM_ARGS -cp jet.jar:jag-ops.jar:$TESTSUITE_JAR $JET_CLASS src/test/jet/getting_started.xml -v
This script uses JET to run the test, with the current machine as client and server. It has jet.jar and jag-ops.jar in the runtime classpath, in addition to the test suite.
We can now try to run the test with JET:
$ chmod a+x run.sh
$ ./run.sh
JET /Users/austvik/devel/jetintro/jet.jar built on 2011-01-19 10:34:50
JET Running test 'test/jet/getting_started.xml'
…
Jan 19, 2011 11:17:23 AM com.sun.jet.framework.engine.JETTestHandler addFailedCreate
SEVERE: Failed to create test, got the following exception: Unknown class 'com.sun.jet.echo.EchoTests', not found by driver 'jet' ("JET Test Driver"):com.sun.jet.echo.EchoTests
java.lang.ClassNotFoundException: com.sun.jet.echo.EchoTests
…
Jan 19, 2011 11:17:23 AM com.sun.jet.framework.engine.JETTestHandler addFailedCreate
SEVERE: Failed to create test, got the following exception: com.sun.jet.echo.EchoServerSetup
java.lang.ClassNotFoundException: com.sun.jet.echo.EchoServerSetup
…
Jan 19, 2011 11:17:23 AM com.sun.jet.jag.ops.TestManagement subtestResult
INFO: Subtest result. Element is: 'getting_started' Result class is: 'com.sun.jet.util.JETTestResultSerializable' Result value is: 'TEST: getting_started HOST: d-etro01-112-152.norway.sun.com:30000 RUN: 0 FAILED: 0 ERROR: 0 SUCCESS: 0'
We now have a simple test XML file, but as we see from the run, it won't do much, as the test setup and test case does not exist yet.
If you want to look at the logs from the test, you will find several lies under the results_getting_started_2011-01-19_11h15 directories:
jag0.0.log jet0.0.log jetINFO0.0.log jetLOAD0.0.log jetreport.txt jetreport.xml
These files contains logs from the embedded JAG that JET starts and JET itself. jetINFO0.0.log contains only important log messages from JET (while jet0.0.log contains more messages). jetLOAD0.0.log contains messages from the load framework, which we are not using here, so it should be empty.
jetreport.txt shows a report of the test run meant to be read by a human, while jetreport.xml contains the result of the test run in XML format if you want to import it into some reporting system.
Test Setup
We will start with a simple setup filed called src/main/java/com/sun/echo/EchoServerSetup.java, but first we have to create the directory:
$ mkdir -p src/main/java/com/sun/jet/echo/ $ cat src/main/java/com/sun/jet/echo/EchoServerSetup.java
This is the content of the file:
package com.sun.jet.echo;
import com.sun.jet.framework.TestSetup;
import com.sun.jet.util.LoggerFactory;
import java.util.logging.Logger;
import junit.framework.Test;
public class EchoServerSetup extends TestSetup {
private static final Logger LOG = LoggerFactory.getLogger(EchoServerSetup.class);
public EchoServerSetup(Test test) {
super(test);
}
@Override
protected void setUp() throws Exception {
LOG.info("Starting up echo server");
}
@Override
protected void tearDown() throws Exception {
LOG.info("Stopping echo server");
}
}
This setup doesn't do much yet, but it logs when the setUp() and tearDown() methods are called. The setUp() method will be called before any test cases in the test suite inside the test setup in the test XML file is started. The tearDown() method will be called after all test methods inside the test suite is finished running.
We can now test that this builds:
$ mvn install … [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2 seconds [INFO] Finished at: Wed Jan 19 11:35:59 CET 2011 [INFO] Final Memory: 22M/252M [INFO] ------------------------------------------------------------------------ $ jar -tf target/jetintro-1.0-SNAPSHOT.jar | grep class com/sun/jet/echo/EchoServerSetup.class
Test Case
If you try to start the test now, it will still fail. This time because it is missing the EchoTests class:
$ ./run.sh
…
One exception reported for JET Tests
Unknown class 'com.sun.jet.echo.EchoTests', not found by driver 'jet' ("JET Test Driver"):com.sun.jet.echo.EchoTests reported 1 times.
To make the test run, we'll add a test case called src/main/java/com/sun/jet/echo/EchoTests.java
package com.sun.jet.echo;
import com.sun.jet.framework.TestCase;
import com.sun.jet.util.LoggerFactory;
import java.util.logging.Logger;
public class EchoTests extends TestCase {
private static final Logger LOG = LoggerFactory.getLogger(EchoTests.class);
public void testEcho() {
LOG.info("Testing echo function.");
}
}
This is a minimal test class, with just one method that sends a line to the log. Lets build it and see what happens:
$ mvn install … [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5 seconds [INFO] Finished at: Wed Jan 19 12:05:15 CET 2011 [INFO] Final Memory: 22M/252M [INFO] ------------------------------------------------------------------------ $ jar -tf target/jetintro-1.0-SNAPSHOT.jar | grep class com/sun/jet/echo/EchoServerSetup.class com/sun/jet/echo/EchoTests.class $ ./run.sh … Jan 19, 2011 12:07:07 PM com.sun.jet.echo.EchoServerSetup setUp INFO: Starting up echo server JET Executing TC testEcho(com.sun.jet.echo.EchoTests) on client 0, d-etro01-112-152.norway.sun.com:30000 RUN testEcho(com.sun.jet.echo.EchoTests) Jan 19, 2011 12:07:07 PM com.sun.jet.echo.EchoTests testEcho INFO: Testing echo function. Jan 19, 2011 12:07:07 PM com.sun.jet.echo.EchoServerSetup tearDown INFO: Stopping echo server *** RESULTS *** 2 subtests succeeded: 1) testEcho(com.sun.jet.echo.EchoTests): 0 sec 2) com.sun.jet.echo.EchoServerSetup: 0 sec Total time: 0.014 s Success …
As we can see, EchoServerSetup.setUp() is called first, then EchoTests.testEcho(), and then EchoServerSetup.tearDown().
Now we are able to run the test, but the only thing it does is that it logs three lines. We want to expand it to do the actual starting and stopping of the echo server, and to test the echo function.
Client/MBean/Ops
We need to create the code that should run in JAG. First we create some directories for the code to live in:
$ mkdir -p src/main/java/com/sun/jet/echo/client $ mkdir -p src/main/java/com/sun/jet/echo/ops
The MBean is the interface that we are exposing from JAG. For the echo server, it only has one function. Here is the src/main/java/com/sun/jet/echo/ops/EchoMBean.java file:
package com.sun.jet.echo.ops;
import com.sun.jet.jag.agent.JAGOperationMBean;
public interface EchoMBean extends JAGOperationMBean {
String echo(String source);
}
The implementation of the interface in src/main/java/com/sun/jet/echo/ops/Echo.java isn't that advanced either:
package com.sun.jet.echo.ops;
import com.sun.jet.jag.agent.JAGOperation;
public class Echo extends JAGOperation implements EchoMBean {
@Override
public String echo(String source) {
return source;
}
}
We can check that it builds:
$ mvn install $ jar -tf target/jetintro-1.0-SNAPSHOT.jar | grep ops com/sun/jet/echo/ops/ com/sun/jet/echo/ops/Echo.class com/sun/jet/echo/ops/EchoMBean.class
That is all it takes to have a really simple MBean, but to be able to use it, we want a Client for it.
This is the content of src/main/java/com/sun/jet/echo/client/EchoClient.java:
package com.sun.jet.echo.client;
import com.sun.jet.jag.client.JAGClient;
import com.sun.jet.util.JAGInstance;
import com.sun.jet.util.TestinfraException;
public class EchoClient extends JAGClient {
private static final String SERVER_OBJECT = "EchoServer";
private static final String SERVER_CLASS_NAME = "com.sun.jet.echo.ops.Echo";
public EchoClient(JAGInstance jagAddress) throws TestinfraException {
super(jagAddress);
}
@Override
public boolean init(String mletloader) throws TestinfraException {
return super.initImpl(mletloader, SERVER_OBJECT, SERVER_CLASS_NAME);
}
public String echo(String source) throws TestinfraException {
String res = "";
try {
res = (String) invoke("echo", new Object[] {source}, new String[] {String.class.getName()});
} catch (Exception e) {
throw new TestinfraException(e);
}
return res;
}
}
The SERVER_OBJECT constant is the name of this server when it runs on the JAG, while the SERVER_CLASS_NAME is the name of the class that should be instantiated to run with this name. As you see, this is the fully qualified class name of the Echo server. When two clients talk to the same SERVER_OBJECT name, they talk to the same instance of the MBean/Ops class in JAG.
The constructor gets in the JAG we are supposed to run in, and just sends this further. The init() method is an abstract method that we have to implement. We will just use it to tell the parent JAGClient class the server object and class name.
The only code that is special for our interface is the echo() method, which uses the invoke method to execute the echo() function there with source as a String parameter. Since this work over the network, we can get Exceptions (e.g. IOExceptions), and if that happens, we will wrap them in a TestinfraException.
Time image show which classes lives where. EchoClient and EchoClientFactory are utility classes used by EchoSetup and EchoTests, which runs in the JET JVM.
Calling client code from test case and test setup
Now we have the client and MBean/Ops, but we are not using them yet. First we have to start and stop the client from the setup.
To make it easy to create these clients, we will create a factory class in src/main/java/com/sun/jet/echo/client/EchoClientFactory.java:
package com.sun.jet.echo;
import com.sun.jet.echo.client.EchoClient;
import com.sun.jet.util.Bindings;
import com.sun.jet.util.JAGInstance;
import com.sun.jet.util.TestinfraException;
public final class EchoClientFactory {
private EchoClientFactory() {
super(); // Singleton
}
/**
* Gets the property name for the loader.
*
* @param host The host the loader runs on
* @return The property name of the loader
*/
private static String getLoadernameProperty(JAGInstance host) {
return "jag." + host + ".loadername";
}
/**
* Create and initialize a client.
*
* @param bindings JET Bindings
* @param host Which host to connect to.
* @throws TestinfraException If bindings not set or failure of creating the object.
* @return JAGOSOpSys object
*/
public static EchoClient createEchoClient(Bindings bindings, JAGInstance host) throws TestinfraException {
EchoClient echo = null;
try {
echo = new EchoClient(host);
echo.init(bindings.getString(getLoadernameProperty(host)));
} catch (TestinfraException je) {
throw new TestinfraException("Failed to create EchoClient on " + host, je);
}
return echo;
}
}
The getLoadernameProperty() method just returns the property name where we expect to find the loadername iced on a specific JAGInstance. The loadername is the name of the classloader used to load the MBeans for our test.
The createEchoClient() method is why we create this factory. In addition to creating the client, it has to be initiated. We initiate it by telling it which class loader it should load itself into.
We can now call this code from the setup:
package com.sun.jet.echo;
import com.sun.jet.echo.client.EchoClient;
import com.sun.jet.framework.TestSetup;
import com.sun.jet.util.LoggerFactory;
import com.sun.jet.util.JAGInstance;
import com.sun.jet.util.TestinfraException;
import java.util.logging.Logger;
import java.util.Map;
import java.util.HashMap;
import junit.framework.Test;
public class EchoServerSetup extends TestSetup {
private static final Logger LOG = LoggerFactory.getLogger(EchoServerSetup.class);
private Map<JAGInstance, EchoClient> clients = new HashMap<JAGInstance, EchoClient>(1);
public EchoServerSetup(Test test) {
super(test);
}
@Override
protected void setUp() throws Exception {
// Start on all client machines
for (JAGInstance machine : getBindings().getClientMachines()) {
LOG.info("Starting up echo server on " + machine);
EchoClient cli = EchoClientFactory.createEchoClient(getBindings(), machine);
clients.put(machine, cli);
}
}
@Override
protected void tearDown() throws Exception {
for (Map.Entry<JAGInstance, EchoClient> entry : clients.entrySet()) {
LOG.info("Stopping echo server on " + entry.getKey());
entry.getValue().unregister();
}
}
}
Lets look at the setUp() method first. Here we go through all the client machines, create a client for each of them, and save them in a hash map for later. Normally we would use getBindings().getServerMachines() here, because we want to start the echo server on the server machines. I start it on the client machines instead, so that I am able to run this test also in a machine with only a single client machine.
The tearDown() method iterates over all the clients that we started in setUp(), and calls unregister() on them. The unregister() method closes the connection to JAG and deletes the MBean from JAG.
Let run it:
$ mvn clean install $ ./run ... Jan 19, 2011 1:07:23 PM com.sun.jet.echo.EchoServerSetup setUp INFO: Starting up echo server on machine:30000 JET Executing TC testEcho(com.sun.jet.echo.EchoTests) on client 0, machine:30000 RUN testEcho(com.sun.jet.echo.EchoTests) Jan 19, 2011 1:07:23 PM com.sun.jet.echo.EchoTests testEcho INFO: Testing echo function. Jan 19, 2011 1:07:23 PM com.sun.jet.echo.EchoServerSetup tearDown INFO: Stopping echo server on machine:30000
It seems to start.
Lets also call the echo function from the test case:
package com.sun.jet.echo;
import com.sun.jet.echo.client.EchoClient;
import com.sun.jet.framework.TestCase;
import com.sun.jet.util.LoggerFactory;
import com.sun.jet.util.JAGInstance;
import com.sun.jet.util.TestinfraException;
import java.util.logging.Logger;
public class EchoTests extends TestCase {
private static final Logger LOG = LoggerFactory.getLogger(EchoTests.class);
public void testEcho() throws TestinfraException {
JAGInstance machine = getBindings().getClientMachines().get(0);
LOG.info("Testing echo function on " + machine);
EchoClient client = EchoClientFactory.createEchoClient(getBindings(), machine);
try {
assertEquals("Echo", client.echo("Echo")); // The Test!
} finally {
client.close();
}
}
}
This test method now just run against the first client machine. We could iterate against every client machine here too if we wanted. It creates an EchoClient, runs the test where it asserts that the result from the echo function is the same as it's input parameter, and finally calls close() on the client. When you call close() you only close the connection to the client, you do not delete the client from the running JAG.
Now we should be ready to run the test:
$ mvn clean install $ ./run.sh $ results_getting_started_2011-01-19_13h22/jetreport.txt *** RESULTS *** 2 subtests succeeded: 1) testEcho(com.sun.jet.echo.EchoTests): 0 sec 2) com.sun.jet.echo.EchoServerSetup: 0 sec Total time: 0.257 s Success
Cool! Now we will try to change the client code a little to see that it is really executed on the JAG.
We change Echo.java into this:
package com.sun.jet.echo.ops;
import com.sun.jet.jag.agent.JAGOperation;
import com.sun.jet.util.LoggerFactory;
import java.util.logging.Logger;
public class Echo extends JAGOperation implements EchoMBean {
private static final Logger LOG = LoggerFactory.getLogger(Echo.class);
@Override
public String echo(String source) {
LOG.info("The echo of '" + source + "' is '" + source + "'");
return source;
}
}
$ mvn clean install
$ ./run.sh
$ grep "The echo" results_getting_started_2011-01-19_13h26/jag0.0.log
19/01 13:26:29.411 INF Thr{0018} Echo.echo: The echo of 'Echo' is 'Echo'
Congratulation, you have just created your first JET test suite!
Running the test with JETBatch
We now want to run the test with a server on another machine.
Before we can run the test, we must start JAG on the machines where we want to run the test. To keep this as simple as possible, I will use only a single machine.
Start JAG
I will edit jag/config/jag.properties for the machine I am running on. I change these settings:
- jag.logging.path
- /tmp/jag
- jag.localpath.0
- /tmp/jaginstall
The logging path is where JAG stores it's logs. The localpath is where installpaths are copied to. We must now create the paths:
$ mkdir /tmp/jag $ mkdir /tmp/jaginstall
And then we start JAG on this machine:
$ ssh <jag_machine> jag_machine$ cd jag jag_machine$ java -jar target/jag-1.0-SNAPSHOT.jar -config config/jag.properties … JAG seems to have started successfully
You can now point a web browser to http://jag_machine:8184/, and you should be able to browse the JAG and see a list of MBeans that has been registered on it. Take a look at e.g. JAGMachineInfo for information about the machine JAG is running on.
After you have started JAG, you can run many tests on it, so this in something you do only one time for each machine.
This is not the "proper" way to start JAG. For a real deployment, see StartJAG for all the details on how to set up JAG on your platform.
Configuration files
JETBatch support an unlimited number of -propfile options, and will merge settings from all of them, but it is a good practice to create two or three different property files to run the test.
- User
- Platform/Rig
- Batch
In the user property file, we will place settings that are unique to the user:
This is how user.props look like:
jet.portbase=31337 jetbatch.report.plugin.mail.to=user@example.com
We could have one joe.props, another anne.props, etc
In the platform properties, I specify which install to use, and which machines to use. I only use one machine here, if you want to use several machines, you list them up divided by comma.
This is how solaris.props looks like:
jet.installpath.jetintro=builds/solaris/ jet.clientmachines=<machine_name> jet.servermachines=<machine_name>
I also need to specify which tests to run, in the batch property file. Again I will only have one test, but you could add several. See WriteBatch for more details.
This is how my release.props look like:
batch.test.failing=test/jet/getting_started.xml
In a more mature test suite, you might have a directory layout like this:
tests/batches/ tests/batches/release.props tests/batches/night.props tests/batches/week.props tests/batches/performance.props tests/users/ tests/users/joe.props tests/users/anne.props tests/platforms/ tests/platforms/solaris10.props tests/platforms/linux_rhel.props tests/platforms/linux_sles.props
Install directory
In the start of this document, directory layouts are mentioned, and in the solaris.props, an installpath of builds/solaris is used, so we will create that with some sub-directories.
$ mkdir -p builds/solaris $ mkdir -p builds/solaris/doc $ mkdir -p builds/solaris/lib $ mkdir -p builds/solaris/test
Normally, we would have the software we are testing in there, but for this simple example, it will be mostly empty.
To simulate a make install, we will copy some files into build/solaris/test:
$ cp jag-ops.jar builds/solaris/test/ $ cp jet.jar builds/solaris/test/ $ cp target/jetintro-1.0-SNAPSHOT.jar builds/solaris/test/ $ cp ~/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar builds/solaris/test/junit.jar $ cp ~/.m2/repository/com/sun/jet/jet-util/1.0-SNAPSHOT/jet-util-1.0-SNAPSHOT.jar builds/solaris/test/jet-util.jar $ ls builds/solaris/test/ jag-ops.jar jet.jar jetintro-1.0-SNAPSHOT.jar junit.jar jet-util.jar
Testsuite default
For JETBatch to understand which jar files to put in the class path when it starts JET, we will create a testsuite_jetdefaults.properties. This configuration file can also contain other settings with defaults for your project, but we will keep it simple for now.
$ mkdir src/main/resources
This is what src/main/resources/testsuite_jetdefaults.properties look like:
jet.installpath.jetintro.subdirs=test,lib jet.installpath.jetintro.classpath=test jag.jagopsjars=jetintro-1.0-SNAPSHOT.jar,jag-ops.jar,jet-util.jar,junit.jar
The important thing here, is that then jetintro installpath is the same as the jetintro in solaris.props.
The jagopsjars line tell JAG which jar files to load in its classloader, so that we are able to load Ops and MBeans into the JAG classloader.
To get this file into the jar file, we have to edit pom.xml, and add this resource after the test xml resource:
<resource>
<directory>src/main/resources</directory>
<includes>
<include>*.properties</include>
</includes>
</resource>
We build and check that it is included in the jar file:
$ mvn clean install $ jar -tf target/jetintro-1.0-SNAPSHOT.jar … com/sun/jet/echo/ops/EchoMBean.class test/jet/getting_started.xml testsuite_jetdefaults.properties
And copy this to the install directory:
$ cp target/jetintro-1.0-SNAPSHOT.jar builds/solaris/test/
Code Changes
We are good to go, but want to change the code a little to utilize the server machines.
The new EchoServerSetup look like this:
package com.sun.jet.echo;
import com.sun.jet.echo.client.EchoClient;
import com.sun.jet.framework.TestSetup;
import com.sun.jet.util.LoggerFactory;
import com.sun.jet.util.JAGInstance;
import com.sun.jet.util.TestinfraException;
import java.util.logging.Logger;
import java.util.List;
import java.util.Map;
import java.util.HashMap;
import junit.framework.Test;
public class EchoServerSetup extends TestSetup {
private static final Logger LOG = LoggerFactory.getLogger(EchoServerSetup.class);
private Map<JAGInstance, EchoClient> clients = new HashMap<JAGInstance, EchoClient>(1);
public EchoServerSetup(Test test) {
super(test);
}
@Override
protected void setUp() throws Exception {
// Start on all client machines
for (JAGInstance machine : getMachines()) {
LOG.info("Starting up echo server on " + machine);
EchoClient cli = EchoClientFactory.createEchoClient(getBindings(), machine);
clients.put(machine, cli);
}
}
@Override
protected void tearDown() throws Exception {
for (Map.Entry<JAGInstance, EchoClient> entry : clients.entrySet()) {
LOG.info("Stopping echo server on " + entry.getKey());
entry.getValue().unregister();
}
}
private List<JAGInstance> getMachines() {
List<JAGInstance> machines = getBindings().getServerMachines();
if (machines.isEmpty()) {
machines = getBindings().getClientMachines();
}
return machines;
}
}
A new getMachines() method is added, which will return the server machines if it has any, or else the client machines. This method is used in setUp().
The new EchoTests.java test case look like this:
package com.sun.jet.echo;
import com.sun.jet.echo.client.EchoClient;
import com.sun.jet.framework.TestCase;
import com.sun.jet.util.LoggerFactory;
import com.sun.jet.util.JAGInstance;
import com.sun.jet.util.TestinfraException;
import java.util.List;
import java.util.logging.Logger;
public class EchoTests extends TestCase {
private static final Logger LOG = LoggerFactory.getLogger(EchoTests.class);
public void testEcho() throws TestinfraException {
JAGInstance machine = getMachine();
LOG.info("Testing echo function on " + machine);
EchoClient client = EchoClientFactory.createEchoClient(getBindings(), machine);
try {
assertEquals("Echo", client.echo("Echo"));
} finally {
client.close();
}
}
private JAGInstance getMachine() {
List<JAGInstance> machines = getBindings().getServerMachines();
if (machines.isEmpty()) {
machines = getBindings().getClientMachines();
}
return machines.get(0);
}
}
The getMachine() method will return the first server machine, or if there are no server machines, the first client machine. This method is used in testEcho().
Off course, for prettier code, these methods that were added to the setup and test case should be moved to some utility class.
Now build and make install
$ mvn clean install $ cp target/jetintro-1.0-SNAPSHOT.jar builds/solaris/test/
Running the batch
To run the batch, we will create a shell script.
This is run-batch.sh:
#!/bin/bash
JAVA="java"
JVM_ARGS="-Xms64m -Xmx512m"
VERSION="1.0-SNAPSHOT"
JETBATCH_CLASS="com.sun.jet.batch.JETBatch"
MVN_REP_DIR="${HOME}/.m2/repository/com/sun/jet"
JETBATCH_JAR="jet-batch.jar"
TESTSUITE_JAR="target/jetintro-1.0-SNAPSHOT.jar"
echo "Copying needed JAR files"
cp "${MVN_REP_DIR}/jet-batch/${VERSION}/jet-batch-${VERSION}.jar" "${JETBATCH_JAR}"
PROP_PARAMS=""
echo "Generating property parameters"
for i in $*
do
PROP_PARAMS="${PROP_PARAMS} -propfile ${i}.props"
done
echo "Running batch:"
$JAVA $JVM_ARGS -cp $JETBATCH_JAR:$TESTSUITE_JAR $JETBATCH_CLASS -reportdir /tmp/jetbatch_$$ ${PROP_PARAMS}
Now we try to run it:
$ chmod a+x ./run-batch.sh
$ ./run-batch.sh user solaris release
JETBatch version 4.5.1
…
20/01 12:50:52.835 INF Thr{0014} TestManagementClient.notif: Start: khepri02:9928 <className=com.sun.jet.echo.EchoServerSetup>
20/01 12:50:52.864 INF Thr{0014} TestManagementClient.notif: Start: khepri02:9928 <com.sun.jet.echo.EchoTests,testEcho>
20/01 12:50:52.975 INF Thr{0014} TestManagementClient.notif: Ended: khepri02:9928 <com.sun.jet.echo.EchoTests,testEcho>
20/01 12:50:53.286 INF Thr{0014} TestManagementClient.notif: Ended: khepri02:9928 <className=com.sun.jet.echo.EchoServerSetup>
…
Successes:
TEST 0: getting_started
STARTED: 12:50:52.870 ENDED: 12:50:53.547 DURATION: 0 sec
RIG: khepri02:9928austvik:
Successes reported by khepri02:9928:
testEcho(com.sun.jet.echo.EchoTests): 0 sec
com.sun.jet.echo.EchoServerSetup: 0 sec
20/01 12:51:08.786 INF Thr{0010} JETBatch.startJETBatch: JETBatch done.
Inspecting the JAG logs
In the machine where JAG is running, we can check the log:
$ ssh jag_machine
jag_machine$ rep "The echo" /tmp/jag/jag0.0.log
20/01 12:50:52.930 INF The{0149} Echo.echo: The echo of 'Echo' is 'Echo'
You can now start JAG on more machines, and adjust your solaris.props to run the test on all of them. In addition you can make your test case run on all the machines where the server runs.
Oh, and why not adjust your testEcho() method to see what happens when it fails?
Good luck with using JET, and don't be afraid to ask questions on the mailing list.
