How to Migrate a JMS Application to Apache Pulsar Using Starlight for JMS API

Starlight for JMS is a powerful API built on Apache Pulsar. It helps developers build reliable, easy-to-scale, lightning-fast applications. Here’s a guide on how to migrate your JMS applications to Starlight.

Starlight for JMS  is a Java Message Service (JMS) 2.0 implementation formerly called  Fast JMS for Apache Pulsar . It’s built on  Apache Pulsar  and offers several benefits to JMS applications like:

• Raw speed:  It easily handles millions of JMS messages per second with 99.9 percentile publish-to-acknowledge latency of less than 10 ms
  
  
• Always online:  It’s fully fault-tolerant with built-in support for asynchronous replication across multiple, geographically separate regions. Even if an entire region goes down, traffic can be rerouted to others
  
  
• Consolidated administration:  It scales out to dozens or even hundreds of brokers and storage nodes. Its multi-tenancy means that a single cluster can be easily shared with thousands of tenants. It also allows you to consolidate your JMS messaging on a single cluster with unified monitoring, management, and security, resulting in dramatically lower operational costs
  
  
• No message loss:  It durably persists messages to disk before accepting them. So once a message is accepted it is guaranteed to be delivered
  
  
• Cost-effective message retention:  It offers flexible message retention policies to allow replay for auditing, testing, and failure recovery. Older messages that are read less frequently can be moved to lower-cost storage like an Apache HDFS or AWS S3

Getting Started with Starlight for JMS

Starlight for JMS implements both the classical API JMS 1.1 and the simplified API JMS 2.0 specifications. You simply include its Java library dependency in your application and replace the legacy JMS dependency in order to use it. As of this writing, the latest version of Starlight for JMS is 1.3.0. Below is an example of what that looks like in a maven pom.xml:

<dependency>
  <artifactId>pulsar-jms-all</artifactId>
  <groupId>com.datastax.oss</groupId>
  <version>1.3.0</version>
</dependency>

Run Apache Pulsar

Starlight for JMS stores messages in an Apache Pulsar cluster. If you’re not running Pulsar, the easiest way to set one up locally is with the  Luna Streaming  distribution of Pulsar – with its  Kubernetes Helm chart  or as a standalone server.

Establish the Connection via PulsarConnectionFactory

As in any JMS application, the first step is to establish connection to the service provider. In our case this is an Apache Pulsar cluster. Starlight for JMS’ implementation of ConnectionFactory, called PulsarConnectionFactory, is the entry point that allows connection. In its simplest form, the way to establish a PulsarConnectionFactory is as follows:

Map<String, Object> configuration = new HashMap<>();

configuration.put("webServiceUrl", "http://localhost:8080"); 

configuration.put("brokerServiceUrl", "pulsar://localhost:6650"); 

PulsarConnectionFactory factory = new PulsarConnectionFactory(configuration)

The only parameter required is a Map that specifies a set of connection parameters. More on that below.

Once we create a PulsarConnectionFactory object, we can get other JMS connection related objects with the standard JMS API calls:

// JMS 2.0 API 
JMSContext jmsContext = factory.createContext(Session.AUTO_ACKNOWLEDGE);

// JMS 1.1 API
Connection queueConnection = factory.createQueueConnection();
QueueSession queueSession = queueConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);

Configure Connection Properties

The biggest difference in using the Starlight for JMS API for an existing JMS application is the connection properties. Compared with traditional JMS providers, Apache Pulsar has a lot more parameters to fine-tune the connection behaviors.

For example, in traditional JMS, username/password login is the only authentication method. But in Pulsar, username/password authentication is not supported. Instead, there are more robust mechanisms such as JSON Web Token (JWT) based authentication.

At a high level, the connection properties used in the Starlight for JMS API fall into several categories:

Related to establishing connection to the Pulsar cluster

• Mandatory
• Available connection properties under this category can be found in the  reference configuration documentation  for Pulsar

Related with Starlight for JMS API behavior itself
(for instance to specify default Pulsar tenant and namespace for JMS topics)

• Optional. When not specified, default values are used
• These properties have the “jms” prefix

Related with fine-tuning Pulsar producer and consumer behaviors

• Optional. When not specified, default values are used
• The available connection properties under this category can be found in the  Configure Producer  and  Configure Consumer  documentation for the Pulsar Java client

You can find more detailed descriptions of these connection properties in  Starlight for JMS configuration documentation .

JMS Destinations (Queues and Topics)

Once the JMSContext or Session an object has been created, we can create a JMS Destination (Queue or Topic) object:

// JMS 2.0 API
Destination destination = jmsContext.createTopic(“persistent://public/default/mytopic”);

 

// JMS 1.1 API
Destination destination2 = queueSession.createQueue(“persistent://public/default/myqueue”);

In Starlight, JMS Destinations are  Pulsar topics  under the hood. So Destination names must conform to valid Pulsar topic names following this form:

persistent://<tenant_name>/<namespace_name>/<topic_name>

Starlight for JMS implements the requested Destination behavior of a JMS queue or a JMS topic via Pulsar’s flexible message processing semantics.

Temporary JMS Destinations are also supported. The corresponding Pulsar topics for temporary JMS queues or topics are deleted by Starlight for JMS when the connection is closed.

Establish Connection and Create Destinations via JNDI Lookup

A typical JMS client application maximizes application portability among different JMS providers by getting ConnectionFactory and Destination objects via Java Naming and Directory Interface (JNDI) lookups. 

These two types of objects are therefore also called administered objects. They’re normally placed in JNDI namespaces.

Figure 1: Process for connection and creating destinations via JNDI lookup.

Starlight for JMS API supports JNDI lookup as well.

Map<String, Object> configuration = new HashMap<>();

configuration.setProperty(Context.INITIAL_CONTEXT_FACTORY, "com.datastax.oss.pulsar.jms.jndi.PulsarInitialContextFactory");

configuration.setProperty(Context.PROVIDER_URL, "pulsar://localhost:6650");

configuration.put("webServiceUrl", "http://localhost:8080"); 

javax.naming.Context jndiContext = new InitialContext(configuration);

ConnectionFactory factory = (ConnectionFactory) jndiContext.lookup("ConnectionFactory");

Topic topic = (Topic) jndiContext.lookup("topics/persistent://public/default/mytopic");

JMSContext jmsContext = factory.createContext(Session.AUTO_ACKNOWLEDGE);

The core processing flow follows exactly the same pattern in the JMS spec.

1. Create the proper JNDI context for administered object lookup
2. From JNDI context, look for the right connection factory from which to establish connection to the Pulsar cluster
3. From JNDI context, look up the correct destination (topic or queue) messages are sent to or received from

When migrating to Starlight for JMS, you will need to update the connection properties as needed to create the JNDI context. In particular, the Context.INITIAL_CONTEXT_FACTORY should be updated to:

“com.datastax.oss.pulsar.jms.jndi.PulsarInitialContextFactory”

And, the Context.PROVIDER_URL needs to be set as the Pulsar broker URL:

"pulsar://localhost:6650"

All other connection properties stay the same as we’ve explored before.

JMS Messages

Starlight for JMS implements JMS message payload types like this:

Figure 2: Table of Payload Types.

Compared with JMS message types, Pulsar has many more  schema types . It includes a native schema registry to simplify dealing with more complex messages. Starlight for JMS internally maps JMS message types to the Pulsar BYTES message type.

Here is an example of creating a text message with a custom property:

Message message = jmsContext.createTextMessage();

message.setStringProperty(“prop_key”, “prop_value”);

JMS message headers and properties are both mapped to Pulsar message properties with some special handling. The details are described in the Starlight for JMS  documentation . Custom, application-specific JMS message properties will be literally passed through as Pulsar message properties with no changes.

JMSXGroupID System Property and Pulsar Message Key

When JMSGroupID – one of the JMS message system properties – is specified, it’s mapped by Starlight for JMS to the message key of a Pulsar message.

This makes it possible to use Pulsar’s  Key_Shared  message consumption semantics so messages with the same JMSXGroupID property value are delivered to the same consumer for processing. For users familiar with ActiveMQ, this is the same semantics as  Message Groups .

JMS Message Producer, Message Consumer, and Queue Browser.

Depending on the API being used (1.1 or 2.0), there can be different names for a message producer like MessageProducer, TopicPublisher, QueueSender, or JMSProducer. They all use a Pulsar  producer  under the hood.

MessageConsumer, TopicSubscriber, QueueReceiver, and JMSConsumer are all also built on the Pulsar  consumer .

The simplest way to create a message producer/consumer and use it to send/receive a message (following JMS 2.0 simplified API) is as follows. Assuming we have jmsContext created following the above examples.

// Message producer

JMSProducer sender = jmsContext.createProducer();

sender.send(destination, message);

// Message consumer (non-durable, non-shared)

JMSConsumer receiver = jmsContext.createConsumer(destination);

Message message = receiver.receive();

From the consumer side, we can also create other types of consumers in a similar way.

• Durable consumer : jmsContext.createDurableConsumer(…)
• Shared Consumer : jmsContext.createSharedConsumer(…)
• Durable Shared Consumer : jmsContext.createSharedDurableConsumer(…)

JMS Queue Browser is also supported via the Pulsar  reader  API. Again, the code to create a queue browser is simple.

QueueBrowser queueBrowser = queueSession.createBrowser(queue);

Enumeration enumeration= queueBrowser.getEnumeration();

while (enumeration.hasMoreElements()) {

Message message = (TextMessage) enumeration.nextElement();

… 

}

It is also worth mentioning that message selector is also supported when creating a message consumer or queue browser. Messages that don’t match the selector condition will stay in the topic or queue.

String msgSelectorStr = “<condition string (SQL92 syntax)>”;

QueueReceiver queueReceiver = queueSession.createReceiver(queue, msgSelectorStr);

Fire-and-Forget vs. Request-Response

The previous producer example follows the  Fire-and-Forget  pattern. The producer sends a message to a destination without worrying about how the message will be processed. Therefore it doesn’t expect any responses.

Starlight for JMS also supports the  Request-Response  message processing mechanism by which it will wait for a response to be returned. This is done via the TopicRequestor or QueueRequestor classes.

Here’s the example code for using Starlight for JMS API to achieve the Request-Response pattern.

QueueRequestor queueRequestor = new QueueRequestor(queueSession, queue);

Message message = queueSession.createTextMessage(“some text”);

Message response = queueRequestor.request(message);

Summary

This article gives examples of using Starlight for JMS and calls out places to pay attention to when migrating existing JMS applications to Starlight.

The code changes required to upgrade for Starlight are minimal. Because Starlight for JMS is fully compatible with the JMS spec 2.0 with complete backward compatibility with JMS spec 1.1, the changes are primarily:

• Using the new PulsarConnectionFactory to establish the connection to an Apache Pulsar cluster with a properly formulated connection property Map object
• Updating Destination names to conform to Pulsar’s topic naming convention

All other existing JMS application code pieces such as creating destinations, sending, or receiving messages do not need changes.

Follow the  DataStax Tech Blog  for more developer stories. Check out our  YouTube  channel for tutorials and here for DataStax Developers on  Twitter  for the latest news about our developer community.

Resources

1. Blog Post: Fast JMS for Apache Pulsar: Modernize and Reduce Costs with Blazing Performance  (now known as Starlight for JMS)
2. Starlight for JMS Documentation
3. Apache Pulsar Documentation
4. DataStax Luna Streaming Documentation
5. Starlight for JMS API Github Repository (source code)
6. Starlight for JMS API GitHub Repository  (example code)