Camel Endpoint DSL

Camel Endpoint DSL

One of the new features of Camel 3.0 is an Endpoint DSL.  This new API aims to provide a type safe replacement for the URLs that are used in Camel to designate the consumer or producer endpoints.  These URLs provide 3 things: the scheme of the URL identifies the component to use, a unique name or id for the endpoint, and a set of parameters to customize the endpoint behavior.

Here is an example of an FTP consumer endpoint definition:

from("ftp://foo@myserver?password=secret&
recursive=true& ftpClient.dataTimeout=30000& ftpClientConfig.serverLanguageCode=fr")

.to("bean:doSomething");

There are several drawbacks with such constructs : no type safety, bad readability and no simple completion.

So we now use the meta model, which is currently extracted from the source using an annotation processor and written in various JSON files, to generate a fluent DSL for each endpoint.  The same java statement can be rewritten in the following way:

from(ftp("myserver").account("foo")
        .password("secret")
        .recursive(true)
        .advanced()
        .ftpClientParameters(
            Collections.singletonMap("dataTimeout", 30000))
        .ftpClientConfig(
            Collections.singletonMap("serverLanguageCode", "fr")))
    .to(bean("something"));

Another route example :

from(file("target/data/files/")
        .initialDelay(0).delay(10).fileName("report.txt")
        .delete(true).charset("UTF-8"))
        .convertBodyTo(String.class)
    .to(mock("result"));

The fluent DSL now provides type safety for parameters.  In addition, it allows leveraging the java editor code completion to access the list of available parameters for the each endpoint.

Now, let's see a few tricks that have been added to the DSL...

Accessing the DSL

The DSL can be accessed in several ways, but the main one is to switch to using an EndpointRouteBuilder instead of the usual RouteBuilder.  It provides access to all endpoint builders which are defined through inheritance on the EndpointRouteBuilder.  

Where to use the DSL

Various parts of the Camel API have been enhanced to allow plugging in the endpoint DSL.  Even though the DSL itself is defined in a separate JAR, the API provides two entry points which are the EndpointConsumerBuilder and EndpointProducerBuilder interfaces.

In short, everywhere where you used to use an endpoint URI defined as a string, you are now able to use one of those interfaces.  So when you were using the following method on the RouteBuilder:

    public RouteDefinition from(String uri);

you can now also use the method:

    public RouteDefinition from(EndpointConsumerBuilder endpointDefinition);

Or, for producer endpoints, the following method on the ProcessorDefinition:

    public Type to(@AsEndpointUri String uri);

which has an endpoint-DSL version:

    public Type to(@AsEndpointUri EndpointProducerBuilder endpoint);

This applies to all methods that were accepting an Endpoint, either using a string URI or an Endpoint object.

Completion

Let's see how the IDE can be leveraged for completion.

The first step is the completion when selecting the component scheme:

The next completion is for selecting properties to configure on the endpoint:

Another example shows here only advanced consumer properties, because the recursive option is a consumer only option and we later restricted to advanced properties using the advanced() call:

Endpoint lists as expressions

There are also methods in the DSL which are using Expression to define a list of endpoints, for example

    public RoutingSlipDefinition routingSlip(@AsEndpointUri Expression expression);

This is usually used for producer endpoints in constructs such as:

    recipientList(constant("seda:b,seda:c"))

Those statements can be rewritten using the endpoint DSL into the following:

    recipientList(endpoints(seda("b"), seda("c")))

Property placeholders

In the XML DSL, we're used to leverage property placeholders to externalize some configuration bits.  This can be done in the Java DSL too but in both cases there are some limitations (a lot of places in the current Java and XML DSL can not leverage placeholders because they are strongly typed, this has been enhanced for Camel 3.1, more on that in a few weeks...)

For the endpoint DSL, each parameter has a typed version and an additional version that accepts a String when the parameter is not of type String.  In the example above, the recursive parameter as a boolean type, but we could have used the String version :

        .password("secret")
        .recursive("{{ftp.recursive}}")
        .advanced()

The value for the recursive property will be extracted from the usual properties component using the ftp.recursive key and transformed into a boolean value.

Consumers and producers

Endpoints can have 2 flavours: consumers and producers.  Sometimes, the configuration of those two endpoints can differ a bit.  For example the moveExisting property is only available for FTP producers while the moveFailed property only apply on consumers. In order to not allow mixing both consumer and producer properties, the DSL introduces typed builders which are specialized in creating consumer or producer endpoints.

So the trick is that the call to ftp("myserver") produces an object that is not typed at this point, so it can be used either as a consumer endpoint or a producer endpoint.  But as soon as a property which is not both a consumer and a producer property is used, the endpoint becomes typed and only properties available on the endpoint type will be available in the DSL.  The exact details on how this is done will be given a bit later.

Common and advanced properties

On the same line, a difference is made between basic and advanced properties.  Some endpoints have dozens of properties that can be configured and as we are relying on the IDE completion to help writing those endpoints, we don't really want to see a hundred of possibilities at each step.  So we leveraged the information from the meta-model which discriminate between basic and advanced properties.  On the basic endpoint, only the common properties are available.  If one wants to configure an advanced property, we need to switch to the advanced builder using the advanced() call as shown in the FTP example above :

        .recursive(true)
        .advanced()
        .ftpClientParameters(
            Collections.singletonMap("dataTimeout", 30000))

It is also possible to get back to basic properties using the related basic() call.

Combining both

It is possible to combine both consumer / producer and basic / advanced properties so that if you work on a consumer endpoint builder and use the advanced() call, only advanced consumer properties will be available.

How that is done

We do play a bit with interfaces in order to achieve the above splits between properties.  For a given endpoint scheme, the builder will have the following structure:

public interface VmEndpointBuilderFactory {

    public interface VmEndpointConsumerBuilder extends EndpointConsumerBuilder {
        default AdvancedVmEndpointConsumerBuilder advanced() {
            return (AdvancedVmEndpointConsumerBuilder) this;
        }
        // [stripped] common basic + consumer basic properties returning a VmEndpointConsumerBuilder

    }
    public interface AdvancedVmEndpointConsumerBuilder extends EndpointConsumerBuilder {
        default VmEndpointConsumerBuilder basic() {
            return (VmEndpointConsumerBuilder) this;
        }
        // [stripped] common advanced + consumer advanced properties returning an AdvancedVmEndpointConsumerBuilder

    }
    public interface VmEndpointProducerBuilder extends EndpointProducerBuilder {
        default AdvancedVmEndpointProducerBuilder advanced() {
            return (AdvancedVmEndpointProducerBuilder) this;
        }
        // [stripped] common basic + producer basic properties returning a VmEndpointProducerBuilder

    }
    public interface AdvancedVmEndpointProducerBuilder extends EndpointProducerBuilder {
        default VmEndpointProducerBuilder basic() {
            return (VmEndpointProducerBuilder) this;
        }
        // [stripped] common basic + producer advanced properties returning a VmEndpointProducerBuilder

    }
    public interface VmEndpointBuilder extends VmEndpointConsumerBuilder, VmEndpointProducerBuilder {
        default AdvancedVmEndpointBuilder advanced() {
            return (AdvancedVmEndpointBuilder) this;
        }
        // [stripped] common basic properties returning a VmEndpointBuilder    

    }
    public interface AdvancedVmEndpointBuilder extends AdvancedVmEndpointConsumerBuilder, AdvancedVmEndpointProducerBuilder {
        default VmEndpointBuilder basic() {
            return (VmEndpointBuilder) this;
        }
        // [stripped] common advanced properties returning an AdvancedVmEndpointBuilder

    }
    default VmEndpointBuilder vm(String path) {
        class VmEndpointBuilderImpl extends AbstractEndpointBuilder implements VmEndpointBuilder, AdvancedVmEndpointBuilder {
            public VmEndpointBuilderImpl(String path) {
                super("vm", path);
            }
        }
        return new VmEndpointBuilderImpl(path);
    }
}

As you can see, 6 interfaces are generated to allow this split in the properties for each component.

Conclusion

This post gives an overview of the new endpoint DSL in Camel 3.0.  Feel free to report to the camel mailing list if you encounter any issue using the DSL.

Happy Camel routing & Merry Christmas !