Using QTCubed with JMF

Using QTCubed with JMF is as easy as calling QTCubed.usesQTKit() in your code, once, as a fully privileged Java Application. It will update the JMF registry, and QTCubed will be used from then on whenever JMF is loaded, whether by an Application or an Applet.

If your program is a Java Applet, or a Java Webstart Application, (which are viewed by JMF as if they are signed applets) and never has higher privileges than that, call QTCubed.usesQTKit() once at the start of your applet, either as a static initializer, or in your constructor. This is necessary because neither Java Applets, nor Webstart Applications are allowed to modify the JMF registry, and hence will NOT be automatically loaded by JMF.

Required privileges to use QTCubed

Any movie you attempt to construct from a file, you must have the privileges to read from that file on the disk. Capturing data requires the ALL privilege, meaning a signed applet. If you are using JMF, capture needs to be enabled for applets in the JMF registry if you won't be using a signed applet to capture media.

Using a URL to initialize a movie requires appropriate Connect privileges for that URL.

If you are creating a movie from scratch, or from data in memory, no special privileges are required.

Playing Media using Quicktime

Aside from the Cocoa quirks mentioned on the FAQ page, using QTCubed to play media through Quicktime is relatively painless.

First, create a Container, either an Applet or an Application frame will work for this.

Obtain a QTMovieView using the factory method QTCubedFactory.initQTMovieView()

This will give you either a QTKit component under OSX, or a QTJava component under Windows.

Add this component to your container, and make it visible. You MUST complete this step before moving on to initializing a QTMovie or strange behavior will ensue. Check the FAQ to see why this is necessary.

Initialize a QTMovie object using one of the factory methods in QTCubedFactory. There are a variety of methods to load movies, or you can even create one from scratch, and add media to it later. (More on this in later revisions)

Load the movie into the QTMovieView using the setMovie(QTMovie) method.

Let the user start the movie using the controller, or call QTMovieView.play() and playback will begin. There is no way to hide the controller presently, though this will be included in future releases.

Capturing Media from a capture device

QTKit is the basis for capture right now, and I have not abstracted this into QTJava yet, so here's the *TENTATIVE* procedure, but this will break with the first versioned (1.x) release.

Get a list of QTKitCaptureDevice objects using the QTKitCaptureDevice.inputDevices method, OR, get the system specified default device for a particular media type using QTKitCaptureDevice.defaultInputDevice(QTKitMediaType).

Once you have a capture device you'd like to use, create a QTKitCaptureDeviceInput object using the default constructor. This takes the QTKitCaptureDevice as an argument:

QTKitCaptureDevice captureDevice = QTKitCaptureDevice(QTKitMediaType.VIDEO);  // Get the system default video capture device
QTKitCaptureDeviceInput deviceInput = new QTKitCaptureDeviceInput(captureDevice);  // Create a Capture Input from the device

Create a new QTKitCaptureSession, and add the input to it:

QTKitCaptureSession captureSession = new QTKitCaptureSession();
captureSession.addInput(deviceInput);

From here, you need to decide what to do with the output. You can right now, only preview the output, or get the decompressed video frames, or both.

Getting decompressed video frames from the capture device

Implement QTKitOutputBufferStream in one of your classes. This will serve as a callback to receive captured media. If you will be keeping the media around after you return, COPY THE DATA IN THE BUFFER SOMEWHERE ELSE. I cannot emphasize this point enough. In order to keep things simple in the native code, the push buffers are re-used, and will be overwritten with new data. In Cocoa terms, this class will become the delegate of the video output class.

Create a QTKitDecompressedVideoOutput class, and add your class that implements QTKitOutputBufferStream to it using the setDelegate method, then add the QTKitDecompressedVideoOutput to the QTKitCaptureSession as an output:

QTKitDecompressedVideoOutput videoOutput = new QTKitDecompressedVideoOutput();  // Create the video output

videoOutput.setDataDelegate(myOutputClass);  // Wire your class in to receive data here

captureSession.addOutput(videoOutput); // Add the output to the Capture Session you wish to receive data from

A word of warning about setPixelFormat and getPixelFormat

It's possible if you haven't set a pixel format, that the result of getPixelFormat, and the actual format of the output buffer will be different. These are due to some quirks in the way the Cocoa class QTDecompressedVideoOutput works in relation to the QTCaptureDevice class, and at present, I cannot predict with 100% certainty what the output format will be, as the format strings used are actually different between these two classes.

There are also instances where using setPixelFormat results in a Cocoa exception during video decompression that is not made available to the delegate method. (the method I use to retrieve the video bytes) In this instance, I'm not yet sure how to catch the problem, and return the error in a sane manner to the user, as the setPixelFormat methods, and CaptureSession.startRunning methods have both already returned.

This will change in future releases.

Previewing the captured data

QTKitCaptureView works similarly to QTKitMovieView, and has the same Cocoa behavior oddities. For more information, read the FAQ. Create the object, and set it visible in some container before attempting to use any methods.

Once it's visible, assign it a Capture Session to preview:

captureView.setSession(captureSession);

Starting and Stopping capture

Once you have everything wired together, you'll need to start the capture session. It's at this point that the camera will actually be activated, and data will begin to flow.

Capture sessions run until stopped or GCed, so BE SURE TO STOP THEM BEFORE YOU RELEASE THE REFERENCE. There are some circular references used in the classes that I'll be reviewing and setting weak references on as appropriate, but for now, assume that they won't be GCed properly in all cases, or under all JVMs.

captureSession.startRunning();  // Start the capture session

// Do something here

captureSession.stopRunning(); // Stop the capture session

captureSession = null;  // Safe to release the reference since we stopped the session