OSGi is a very popular integration environment. Instead of delivering one big product the products get separated into minor parts and deployed within a general environment. The problem with this concept is to get old, well known concepts up and running in the new environment. In addition tools such as PAX construct allow a better integration into Apache Maven, and extended OSGi runtimes, such as Karaf allow a richer and easier development. Nevertheless, setting up such a system for development means a lot of hard manual work. Using the OpenEngSB such systems can be setup within minutes.

The OpenEngSB introduces a new level of ESB. Development with all typical ESBs mean to start from the ground and develop a complete, own environment, only using existing connectors. Using the OpenEngSB not only connectors but an entire integrated process, workflow and event environment waits for you. In addition connectors to different tools can not only be adapted to the specific needs, but also simply replaced by other connectors, using the Domain concept.

The OpenEngSB delivers a complete management and monitoring environment. While this environment can be added to your project standalone (similar to e.g. Tomcat management console) you also have the possibility to completely integrate the OpenEngSB management environment into your Apache Wicket application.

While typical ESB have to be installed separately from your application the OpenEngSB is delivered with your application. Develop your application in the OpenEngSB environment and scripts to embed your application into the OpenEngSB are provided. In addition easy blending allows to adapt the OpenEngSB visually to your needs and cooperate design.

The OpenEngSB provides the infrastructure for a rich Plug-In and extension system. Using maven archetypes Plug-Ins can be created, uploaded and provided to all other OpenEngSB installations or applications using the OpenEngSB.

TBW(Jira-ISSUE)

To create a new Domain run mvn openengsb:genDomain (or use ../etc/scripts/gen-domain.sh) in the domain folder. You will be asked for the name of your domain. Enter the domain name starting with a lower case letter. For the other questions valid defaults are given.

The new domain project will be added as a submodule. You eventually want to run mvn openengsb:eclipse and import the new project in eclipse.

Add the methods your domain supplies to the domain interface. If your domain raises any events add methods like

void raiseEvent(YourEvent event);

(your event class subtype of Event as single parameter) to the events interface.

To create a new Connector run mvn openengsb:genConnector (or use ../etc/scripts/gen-connector.sh) in the connector folder. You will be asked for the name of the domain you want to implement. Enter the domain name starting with a lower case letter. You may adapt the name of the implemented domain interface if you it does not match the naming convention. Supply the name of the connector staring with a lower case letter.

The new domain project will be added as a submodule. You eventually want to run mvn openengsb:eclipse and import the new project in eclipse.

Implement the domain interface in the supplied class (unfortunately no method stubs are generated).

Unimplemented domain methods should always throw an exception rather than return default value or do nothing. Therefore each domain method without body must throw DomainMethodNotImplementedException to indicate that requested domain functionality is not implemented.

@Override
public void foo() {
	throw new DomainMethodNotImplementedException();
}
          

The ServiceFactory has to supply a ServiceDescriptor that contains all attributes needed to instantiate the Connector. In the methods createServiceInstance and updateServiceInstance use the provided attributes to create a new new instance or update your Connector. The methods updateValidation and createValidation should do the same but try to validate the provided attributes first and return a validation result.

The generated ServiceManager usually does not have to be changed.

One of the principal concepts for the OpenEngSB development is (if possible) to use already existing and proven solutions rather than inventing new ones. In this manner the OpenEngSB is an extension to the ESB concept. Typical ESBs such as Apache Servicemix or other JBI or ESB implementations always have the feeling to be huge and bloated. Complex integration patterns, messaging, huge configuration files and similar concepts/problems lead to this feeling. And those feelings are right. They are bloated. The OpenEngsB tries a different approach. Using Karaf as its base framework the environment is VERY lightweight. Depending on your use case you can use different configurations and packages out of the box.

While Apache Karaf provides a rich environment and functionality we're not done with it. Via the Apache Aries Blueprint-DM extension mechanism, AOP and the OSGi listener model the OpenEngSB directly extends the environment to provide own commands for the console, fine grained security and a full grown workflow model. These extensions are optional and not required if you want to use the platform alone. Add or remove them as required for your use case.

These libraries are the OpenEngSB core. The core is responsible to provide the OpenEngSB infrastructure as well as general services such as persistence, security and workflows. To provide best integration most of these components are tied to the OpenEngSB ESB environment. Nevertheless, feel free to add or remove them as required for your use case.

Although each tool provider gives a personal touch to its product their design is driven by a specific purpose. For example, there are many different issue trackers available, each having its own advantages and disadvantages, but all of them can create issues, assign and delete them. Tool Domains are based on this idea and distill the common functionality for such a group of tools into one Tool Domain interface (and component). Tool domains could be compared best to the concept of abstract classes in in object orientated programming languages. Similar to these, they can contain code, workflows, additional logic and data, but they are useless without a concrete implementation. Together with the ESB, the OpenEngSB infrastructure and the core components the tool domains finally result in the OpenEngSB.

Client Tools in the OpenEngSB concept are tools which do not provide any services, but consume services provided by Tool Domains and Core Components instead. A classical example from software engineering for a client tool is the Integrated Development Environment (IDE). Developer prefer to have the entire development environment, reaching from the tickets for a project to its build results, at hand. On the other hand they do not need to provide any services.

Domain Tools (Service Provider) Domain Tools, compared to Client Tools, denote the other extreme of only providing services. Classically, single purpose server tools, like issue tracker or chat server, match the category of Domain Tools best. Most tools in (software+) engineering environments fit of course in both categories, but since there are significant technically differences between them they are described as two different component types.

Tool Connectors connect tools to the OpenEngSB environment. They implement the respective Tool Domain interface. As Client Tool Connectors they provide a Client Tool with an access to the OpenEngSB services. Again, Domain- and Client Tool Connectors are mostly mixed up but separated because of their technical differences. Additionally it is worth mentioning that tools can be integrated with more than one connector. This allows one tool to act in many different domains. Apache Maven is an example for such multi-purpose tools, relevant for build, as well as test and deploy of Java projects.

A domain model represents an abstraction of data a domain has to work with (e.g. Issue for issue tracking systems). It capsulates all information which is needed for one information unit. Such domain models are defined in the domains and can be used by connectors which use the domain.

This domain models can be saved and versionized with the help of the OpenEngSB and two core components, namely the EDB(Engineering Database) and the EKB(Engineering Knowledge base). Those two components will be explained in more detail in the contributor manual.

A domain model is represented as an interface. This interface has to extend a provided interface which is called "OpenEngSBModel". To work with this interface, we provide a Utils class, (ModelUtils) which is able to proxy the interface and gives you the feeling like you work with a normal class object. Such domain models can be sent via events(EDBInsertEvent, EDBUpdateEvent, EDBDeleteEvent and EDBBatchEvent) to the EDB, where they will be saved, updated or deleted. This events can be thrown from every connector that extends "AbstractOpenEngSBConnectorService", which shall be done by all connector implementations.

An OpenEngSBModel consists only of getter and setter pairs. Those methods describe which fields the model have. Field types which are supported until now are: simple types, Strings, Date, OpenEngSBModel, Lists and Files. Files are a special case, but it is possible to send models with File objects remotely and the File which is represented by the File object will be available at the remote machine as a local copy there.

Every OpenEngSBModel can define an id for itself. This id can be used for easier finding of domain models and to enable the versioning possibility when they are saved in the EDB. Such an id can easily be defined by setting a special annotation over the setter which defines the id for the object. This annotation is called "OpenEngSBModelId".

Domain models can be loaded from the EDB through the EKB. The EKB bundle provides a service called QueryInterface. This service provides all needed functionality to search and load models from the EDB. It also convert the elements from the EDB to a new instance of the domain model you request.

This feature is not yet implemented. There is currently research going on how to accomplish this task the best way.

The context is also used to handle the wiring of services in workflows. Suppose there are two projects that use their own SCM-repositories and for both repositories connector-instances were created to poll them. When executing a workflow contains an action that polls the SCM, the correct service ca be picked by looking up the current thread's context.

In general workflows have references to several domains and other services which they interact with during execution. Each project might have their own tools behind these domains, so these references must be resolved at runtime depending on the current context.

For this to work the workflow-engine declares global variables that are used in rules and processes. A variable is resolved by looking up the service with the same name in the current context. If no service with that name is available in the context it is looked up in the "root"-context.

In detail the wiring is handled via the service-properties. Services contain properties where the key is of the format "location.<contextid>". The value is a list of "locations" represented by an array of strings. So a service may have several locations in several contexts.

When a global variable is accessed during the execution of an action (from a process or rule), the OSGi-context is queried for the corresponding service. The service wired to this variable must have location with the same name as the variable. The service is searched in the current context and the root-context. If no service is found, the action is stalled for 30 seconds. If there is still no service found an Exception is thrown. Internally this is handled using proxies. When the workflow service is started, all globals are populated with proxies, that automatically resolve the service with the corresponding location when a method is invoked.

Example: The auditing-service is registered with the interface AuditingDomain. The service has property "location.root" with value {"auditing"} (array with one element). The workflow engine contains a global named "auditing" and a rule that invokes a method on every Event that is processed. When the rule fires and the consequence is executed, the proxy representing "auditing"-global queries for a service with the location.currentContext or the location.root containing a location-entry "auditing". Since root-services get a service-ranking of "-1" by default, the service current context's would supersede the service located in the root-context.

The OpenEngSB has a central persistence service, which can be used by any component within in the OpenEngSB to store data. The service is designed for flexibility and usability for the storage of relatively small amounts of data with no explicit performance requirements. If special persistence features need to be used it is recommended to use a specialized storage rather than the general storage mechanism.

The persistence service can store any Java Object, but was specifically designed for Java Beans.

The interface of the persistence service supports basic CRUD (create, update, retrieve, delete) mechanisms. Instances of the persistence service are created per bundle and have to make sure that data is stored persistently. If bundles need to share data the common persistence service cannot be used, as it does not support this feature. The persistence manager is responsible for the management of persistence service instances per bundle. On the first request from a bundle the persistence manager creates a persistence service. All later requests from a specific bundle should get the exact same instance of the persistence service.

Queries with the OpenEngSB persistence done via the persistence service. Behind this service an easy query-by-example logic is used to retrieve your results. In some cases the comparison and storage can create some wired problems for your specific use cases. For those cases the IgnoreInQueries annotation had been added. Using this annotation on getters in classes persisted via the persistence service querying them ignores those fields during trying to compare them to stored data.

The persistence solution of the OpenEngSB was designed to support different possible back-end database systems. So if a project has high performance or security requirements, which can not be fulfilled with the default database system used by the persistence service, it is possible to implement a different persistence back-end. To make this exchange easier a test for the expected behavior of the persistence service is provided.

Besides the centralized Java Bean store the OpenEngSB also have a more specialized solution to store configurations. Configurations are basically also Java Beans, but have to extend a ConfigItem. Well, since Configurations are also only Java Beans you may ask: Why not simply store them via the persistence service? The reason is quite simple. We do need to store configurations at various places. Options may be the file system, an object store or any other place. In addition configurations, when you e.g. store them to files, have to be quite specific about their types. Rule, for example, have to be stored as simple strings, flows as xml files and connectors as key-value-pairs. Being so specific the implementations of the backends also have to be specific. Besides, there are kind of regions. Examples are Rules, Flows, and various others. Basically in you code you simply want to ask for a configuration persister for rules and do not care if it is a file persister or something else. In addition rules could be persisted somewhere else than e.g. flows. Therefore those backends have to be configured separate for each type.

Ok, after the need is identified let's take a look at the how. Basically it's quite simple. You register various backend services in the OSGi registry, give them a specific ID, configure how a region is mapped to an idea and request a persister for a specific region or type and retrieve the correct implementation. From a user point of view this system is quite simple. Use the getConfigPersistenceService(String type) method from the OpenEngSBCoreServices class with the type, which is typically stored directly at the configurations, as for example for the RuleConfiguration and retrieve and persist RuleConfigurations. The mapping between the backend and the frontend is defined in the configuration file here. If you want to use another available and compatible backend for rule configurations add the backend id in the configuration file and the service for this region will switch automatically.

Although it is quite simple to configure, change and consume and provide configurations it is mostly not a good idea to simply change the properties, backend or frontend if you're not exactly sure about what you're doing. You can easily take the wrong backend service which will not be able to persist e.g. a RuleConfiguration and throws exceptions. If you swtich the backend during the run everything stored in the old backend would not be available in the new one. Within a client project mostly relay on using those services to read the properties and use the OpenEngSB to store them.

Still the system can easily be extended to your own use. Typically you have to do the following steps to provide a new configuration service. First of all start by providing an own Configuration which extends ConfigItem. Please only use the metadata and content fields and do not add additional variables. They wont get stored. Now add a configuration file into etc with org.openengsb.persistence.config-ANY_NAME_YOU_LIKE.cfg. In this file define the region and the backend id. The exact values and detailed explanation for those fields is available here. If you've not choosen one of the general available services for storage you now can implement your own backend service registered in the OSGi registry with the ID you've configured in the .cfg file before. The interface you have to implement and register as a service is the ConfigPersistenceBackendService.

The OpenEngSB has a central user management service, which can be used for example by an user interface. The service is designed to manage your users. You can create new user and save them to the persistence or retrieve, update and delete them.

The user management needs a back-end database, e.g. the central persistence service of the OpenEngSB.

The interface of the User manager supports basic CRUD mechanisms (create, retrieve, update, delete). The User is the used user model. It holds attributes like a password, username, if the user is enabled, or his account is expired or locked. A user is identified by his username. So the username can not be changed. Another attribute are the authorities. These are the roles granted to the user. These can be for example "ROLE_ADMIN" which defines the user as admin. Depending on the roles, a user can have different rights. For the OpenEngSB-UI a user has to have at least the role "ROLE_USER" which is the default role.

Access control is done on the service level. Core-services and connector-instances are all published as OSGi-services. Other services and components always reference these service instances. We use the approach of AOP to achieve security of these services. The openengsb.core.security-bundle publishes a service that serves as a method-interceptor. When it is attached to a service every method call on the service is preceded with an authorization-check.

A reference to the method-interceptor can be obtained by this line in the spring-context.xml

<osgi:reference id="securityInterceptor" interface="org.aopalliance.intercept.MethodInterceptor" />

In order to attach it to an existing bean, one has to create a ProxyFactoryBean:

<bean id="secureServiceManager" class="org.springframework.aop.framework.ProxyFactoryBean">
  <property name="proxyInterfaces">
    <value>other.ServiceInterface</value>
  </property>
  <property name="interceptorNames">
    <list>
      <value>securityInterceptor</value>
    </list>
  </property>
  <property name="target" ref="<realBean>" />
</bean>

When registering a service in code rather than in a spring context.xml this can be done as seen in the AbstractServiceManager

import org.springframework.aop.framework.ProxyFactory;
//
// ...
//
ProxyFactory factory = new ProxyFactory(serviceObject);
factory.addAdvice(securityInterceptor);
OpenEngSBService securedService = (OpenEngSBService) factory.getProxy();

The decision about the allowing the user access to a service as made by looking at the services instanceId. Therefore, all services that are to be placed under this access control, must implement OpenEngSBservice, and make sure the instanceId is unique enough to ensure security. You may want to derive your service-class from AbstractOpenEngSBService.

The persistence of the security-bundle manages a set of GrantedAuthorities (Roles) for each instanceId. There is one exception: Users with "ROLE_ADMIN" are always granted access.

This chapter describes how to deal with security in internal bundles and client projects

For authentication the OpenEngSB provides an AuthenticationProvider as a service. It's obtainable via blueprint.

        <reference interface="org.springframework.security.authentication.AuthenticationManager" />
      

This service is able to authenticate users (org.springframework.security.authentication.UsernamePasswordAuthenticationToken) and bundles (org.openengsb.core.security.BundleAuthenticationToken). The use of the former is pretty obvious. The latter is used for authentication for internal actions, that require elevated privilages. This authentication should be used with caution, and never be exposed externally for security reasons.

The workflow service is responsible for processing events, and makes sure the rulebase is connected to the environment (domains and connectors). When an event is fired, the workflow-service spawns a new session of the rulebase. The session gets populated with references to domain-services and other helper-objects in form of global variables. A drools-session is running in a sandbox. This means that the supplied globals are the only way of triggering actions outside the rule-session.

The rule manager provides methods for modifying the rulebase. As opposed to plain drl-files, the rulemanager organized the elements of the rulebase in its own manner. Rules, Functions and flows are saved separately. All elements share a common collection of import- and global-declarations. These parts are sticked together by the rulemanager, to a consistent rulebase. So when adding a new rule or function to the rulebase, make sure that all imports are present before. Otherwise the adding of the elements will fail.

In addition to processing Events in global/context-specific rules, it is also possible to use them to control a predefined workflow. The WorkflowService provides methods for starting and controlling workflow-processes.

When the workflow service receives an event, it is inserted into the rulebase as a new fact (and rules are fired accordingly). In addition the event is "signaled" to every active workflow-process. Workflow logic may use specific rules to filter these events.

All workflows started in the OpenEngSB are supplied with the global variable ProcessBag. Inside the workflow you can populate the ProcessBag with your data. As soon as Human Interaction is needed you have to incorporate the sub-workflow "humantask", which wraps the ProcessBag into a Task. You can then query the Taskbox service for open Tasks, and manipulate the data inside of the Task (Not necessarily by Human Interaction). When you are finished, you again call the Taskbox service and supply the changed Task. The changed data gets extracted and is handed back over to the workflow.

The Webtaskbox service provides additional UI Features, if you want to integrate the Taskbox-Concept into a wicket Page. You can query the Webtaskbox service for an Overview Panel that displays all open Tasks. If the default Overview Panel doesn't fit your needs exactly you can develop your own UI-Component using the (Core-)Taskboxservice. If you navigate onto a specific Task the Overview Panel displays a (default) Detail Panel populated with the values of the Task, if there is no custom Panel registered for the supplied tasktype. You can develop your own Detail-Panels and register them for a specific Tasktype via the Webtaskbox service.

Following the "Chain of Responsibility"-Pattern the OpenEngSB uses Filters to modularize the processing and transport of incoming and outgoing messages (see Figure 12.1, “How filters fit in the architecture”). A filter is responsible for one (or more) specific transformation steps. Ideally a Filter should only represent a specific transformation step to increase reuseability.

Figure 12.1. How filters fit in the architecture

A filter is responsible for both ways of a transformation (for example a filter that parses a request is also responsible for marshalling the result). Since it is a chain of filters, every filter has a successor (next) where it passes it's transformed request. After the next filter is done and returns a result the current filter transforms the message to the desired output format. This gives filterchains a pyramid-like architecture (see Figure 12.2, “Pyramid-like architecture of filters”)

Figure 12.2. Pyramid-like architecture of filters

Example: Typically an incoming remote call can be divided into the following steps:

A port realizing such a transport would consist of a task listening for incoming messages and a chain with two Filters: One for marshalling and one handling the request itself. An example for such an implementation is the "jms-json-basic"-port in the "openengsb-ports-jms" project. The incoming port is represented by a single that listens on a specific jms-queue for new requests. If a Text-message is received it is passed on to the filterchain as a string. The filterchain starts with a "JsonMethodCallMarshalFilter". As the name indicates, this filter expects a string containing a json-encoded MethodCall. The string is transformed into a MethodCallRequest object and passed on to the next filter. The next filter is the RequestHandlerFilter. It extracts the MethodCall from the request and passes it on to the RequestHandler and wraps the returned MethodResult into a MethodResultMessage. The MethodResultMessage is then returned to the "JsonMethodCallMarshalFilter". There the MethodResultMessage is encoded in JSON and returned to the MessageHandler which then sends it to the answer-queue. In the JMS-Port the result is sent to a queue named after the "callId" submitted in the Request. The callId however cannot necessarily be extracted from the plain-text message at the beginning of the chain. Therefore a map containing values obtained in the filter chain (propertyContainer) is passed in to each filter during processing.

This is only a specific example of creating a port. Another port behaving similarly but using xml as encoding can easily be configured. It can use the same bean but with a different filterchain. In the filterchain the first element is replaced by an XMLMethodCallMarshalFilter. the RequestHandlerFilter is the same as in the jms-json-port.

An instance of FilterChainElement may only be part of one FilterChain. In order to reuse FilterChainElements in other filterchains new instances must be created. This is because the instances of the filters may contain references to the next element in the chain.

That's why Filterchains are supposed to be configured using a FilterChainFactory. A filterchain is a bean configured with a list of filters. Each element may either be the Classname of a FilterAction-class (which must have a public default constructor) or an instance of a FilterChainElementFactory. The last element in the list may also be an instance of a FilterAction (or other FilterChain). The following example shows how to configure a port via blueprint.

<bean id="incomingFilterChainFactory" class="org.openengsb.core.common.remote.FilterChainFactory">
  <property name="inputType" value="java.lang.String" />
  <property name="outputType" value="java.lang.String" />
  <!-- the list of filters -->
  <property name="filters">
    <list>
      <!-- A class implementing the FilterChainElement-interface -->
      <value>org.openengsb.core.common.remote.JsonMethodCallMarshalFilter</value>
      <!-- instance of a filter-factory -->
      <bean class="org.openengsb.ports.myport.MyFilterFactory">
        <propety name="foo" value="bar" />
      </bean>
      <!-- The last item in the list may be an instance of a FilterAction -->
      <bean class="org.openengsb.core.common.remote.RequestMapperFilter">
        <property name="requestHandler" ref="requestHandler" />
      </bean>
    </list>
  </property>
</bean>
    

When configuring the filter-chain you have to make sure that each filter in the list is compatible with its predecessor. Compatibility is checked when the create-method is invoked. In the above example this would be while processing the blueprint-file.

The filters provided by the OpenEngSB only cover the the requirements for the ports that are provided by the OpenEngSB itself. For custom ports, custom filter-classes may be required.

Filterclasses that are to be used at the end of a chain must implement the FilterAction-interface. In order to be usable anywhere in the filterchain the Classes must implement the FilterChainElement-interface. The interfaces do not use generic parameters because the benefit is really minimal as the information is erased during compilation. There are however abstract classes (with generic parameters) that make it easier to implement new FilterChainElements

Incoming ports receive messages and process them using a filterchain. There are no restrictions on how the implementation of the incoming port actually looks like. Typically an incoming port is an object that spawns a listening thread and uses a filterchain to process incoming messages. This is an example of how the incoming port for JMS could look like.

<!-- example of a bean representing an incoming port -->
<bean id="incomingPortBean" class="org.openengsb.ports.myport.MyIncomingPort" init-method="start" destroy-method="stop">
  <property name="factory">
    <bean class="org.openengsb.ports.jms.JMSTemplateFactoryImpl" />
  </property>
  <property name="filterChain">
    <bean factory-ref="incomingFilterChainFactory" factory-method="create" />
  </property>
</bean>
    

Every filterchain should use make sure to pass the MethodCall to the RequestHandler in the and (using a RequestHandlerFilter.

Outgoing port implementations must follow a few more guidelines than incoming ports, because the OpenEngSB needs to be aware of Outgoing ports present in the system in order to use them in other components (like RemoteEvents).

An outgoing ports is represented as an OSGi-service that implements the OutgoingPort-interface. Also it uses the "id"-property (not to be confused with "service.id" defined in the OSGi-spec) as a unique identification for components that want to interact with remote applications. A reference implementation of the OutgoingPort-interface is provided in the "openengsb-core-common"-project.

<!-- service representing the outgoing port -->
<service interface="org.openengsb.core.api.remote.OutgoingPort">
  <service-properties>
    <entry key="id" value="jms-json" />
  </service-properties>
  <!-- the outgoing port uses a filter-chain to manage the entire calling-procedure -->
  <bean class="org.openengsb.core.common.OutgoingPortImpl">
    <property name="filterChain">
      <bean factory-ref="outgoingFilterChainFactory" factory-method="create" />
    </property>
  </bean>
</service>
      

The actual network-communication is also implemented in a FilterAction. This is an example how a filterchain can be used to handle an outgoing methodCall with a result.

<bean id="outgoingFilterChainFactory" class="org.openengsb.core.common.remote.FilterChainFactory">
  <property name="inputType" value="org.openengsb.core.api.remote.MethodCallRequest" />
  <property name="outputType" value="org.openengsb.core.api.remote.MethodResultMessage" />
  <property name="filters">
    <list>
      <value>org.openengsb.core.common.remote.JsonOutgoingMethodCallMarshalFilter</value>
      <bean class="org.openengsb.ports.jms.JMSOutgoingPortFilter">
        <property name="factory">
          <bean class="org.openengsb.ports.jms.JMSTemplateFactoryImpl" />
        </property>
      </bean>
    </list>
  </property>
</bean>
      

The proxy mechanism allows for any method call to be intercepted.

The connector deployer service creates, updates or deletes instances of connector services.

All files in the "config/" directory with the extension ".connector" are handled by the connector deployer. The .connector files have to be simple property files containing the configuration properies of a certain connector service and their values. Those files have to follow a specific form to be read correctly. First of all they follow the pattern: "domain+connector+instance.connector". Here "domain" stands for the domainId to use (e.g. notification), "connector" for the name of the connector which should be created in the domain (e.g. mail) and "instance" is a unique id per connector. A simple UUID or a unique counter should do here. The content of the file is two-fold. On one hand you can configure the properties of a service directly using property.NAME_OF_THE_PROPERTY=VALUE_OF_THE_PROPERTY (for example property.location.root=notificaiton). Those properties are directly added to the properties of a service. On the other hand you have to configure the connectors themselves. Therefore you have the attribute.NAME_OF_THE_CONNECTOR_ATTRIBUTE=VALUE_TO_ASSIGN. Throwing this together you can end with a connector file as shown next.

property.location.root=notify
attribute.user = user
attribute.password = test
attribute.prefix = [test]
attribute.smtpAuth = true
attribute.smtpSender = test@test.com
attribute.smtpPort = 25
attribute.smtpHost = smtp.testserver.com
      

The context deployer service creates contexts according to any .context files found in the config directory. The context id is the file-name (without the extension). The file content will be ignored. So for example

To use the same dependencies as the OPENENGSB project you have to import the openengsb-bundle-settings project into your dependency management section:

 
     <dependencyManagement>
       <dependencies>
         <dependency>
           <groupId>org.openengsb.build</groupId>
           <artifactId>openengsb-bundle-settings</artifactId>
           <version>Version of OPENENGSB you use</version>
           <type>pom</type>
           <scope>import</scope>
         </dependency>
       </dependencies>
     </dependencyManagement>
      

This will import all the dependencies with the correct versions into the dependencyManagement section. You can now define the dependencies shared between your project and OPENENGSB in your dependencies section without setting the version.

This section describes the setup process required for OpenEngSB development.

If you would like to view a use-case centric tutorial take a look at the continuous integration example.

If you are already familiar with Java EE servers about 15 minutes. We will not be using advanced concepts, so you likely be able to continue with the tutorial even without it.

It is assumed you have basic knowledge of system administration and you are able to set up auxiliary software (i.e. JDK 1.6) yourself.

First of all the JDK6 should be installed on the system and the JAVA_HOME variable should be set. (Java download).

Also, make sure that the java-command is available in the PATH-variable.

Download the latest OpenEngSB release from here.

Extract the archive, and run the openengsb-script (bin/openengsb.bat on windows, bin/openengsb otherwise). Click the following link to open the web interface in your browser http://localhost:8090/openengsb.You should automatically be directed to a page asking for a login. The default login is "admin" with "password" as password.

login

If everything works fine you should be welcomed by the following page presenting you the currently installed domains:

main page

OpenEngSB implements it's functionality in so called features. Each feature contains a number of OSGi bundles. While all features are distributed with the OpenEngSB not all of them are installed to speed up the startup. For the next section (First Steps) it is required to install additional features. Therefore open the console in which you've started the OpenEngSB and enter "list" which should output something like:

karaf@root> list
START LEVEL 100 , List Threshold: 50
   ID   State         Blueprint      Level  Name
[  42] [Active     ] [Created     ] [   60] Apache Karaf :: Shell :: Service Wrapper (2.2)
...
[ 116] [Active     ] [            ] [   60] Jackson JSON processor (1.5.3)
[ 165] [Active     ] [Created     ] [   60] OpenEngSB :: Framework :: API (Version)
[ 166] [Active     ] [Created     ] [   60] OpenEngSB :: Framework :: Engineering Database (Version)
[ 167] [Active     ] [Created     ] [   60] OpenEngSB :: Framework :: Engineering Knowledge Base (Version)
[ 168] [Active     ] [Created     ] [   60] OpenEngSB :: Framework :: Common (Version)
[ 169] [Active     ] [Created     ] [   60] OpenEngSB :: Framework :: Services (Version)
[ 170] [Active     ] [Created     ] [   60] OpenEngSB :: Framework :: Persistence Layer (Version)
...
    

In order to install domains and connectors from other repositories you need to add the corresponding feature-URLs. Note that in order to install a connector you need to install all domains it implements.

In order to install a domain first add the feature-repository using the command features:addurl. When the URL has been added the domain can usually be installed using features:install

root@openengsb>features:addurl mvn:org.openengsb.domain/org.openengsb.domain.notification/[domain version]/xml/features
root@openengsb>features:install openengsb-domain-notification
    

Note that domains and connectors are versioned independently of each other and the OpenEngSB.

Now that the OpenEngSB is up and running (and all required bundles are installed) start with the first integration tutorial.

To shutdown the OpenEngSB, go to the command-window and type shutdown or press Ctrl+D

This section describes a "hello world" use-case for the notification domain using the email connector.

If you are already familiar with setting up services in OpenEngSB about 15 minutes. (see HowTo: Setup

This HowTo assumes you have already a running instance of the OpenEngSB.

Create a new Email Notification Service by clicking the "New..." link for the Email Notification on the Test Client link.

test client

In the following view you have the possibility to configure the Notification Service. The following screen provides an example for a Gmail account. Please use "test1" for the Service Id field.

email notification

When you have finished setting all fields to appropriate values, create the new instance by clicking the "Save" button.

Now create another service with the Service Id "test2". Otherwise you can use exactly the same values again.

You can validate the services open the "Services" page, which should look similar to the following screenshot. All your created services should be available with the state "ONLINE".

overview

Now we're going to validate the created services. First of all start by open the "Test Client" link. Now open the "Notification Domain" tree and choose test1. Next get the notify method from the drop down box. The available fields should change instantly. Let the attachment field free and enter anything into message and subject. The address should be a valid email address (not validated for the moment). After all the view should look similar to the following image:

notification properties

Call the service by using the "Call" button. Some seconds after you've pressed the call button the following message should occur on on your screen:

success

Within the next seconds to minutes the address, specified by you, should receive a mail.

Next validate if the service test2 does the same. Therefore press on test2, choose notify again and enter your values. Click "Call" again and validate if you receive an email.

Till now you've used the services directly. Now one of the OpenEngSB core concepts is presented: the Domains:

To send messages via domains, instead of directly via the connectors a default receiver for a specific project has to be set. A graphical user interface for doing so is the "Context Page":

context

Change the entry "domains/NotificationDomain/defaultConnector/id" to test1 or test2. Do this by clicking on the node (id). This should create a drop down box next to it. Select test1 or test2. Afterwards go back to the "Test Client" page and select "Notification Domain/domains.notification". Now choose the notify method again and try sending a message to yourself.

Again, the call was successful if you receive a message (with the prefix of the notifier you've chosen in the context).

Congratulations, you have just finished the first implementation HowTo of Open Engineering Service Bus.

Now that you've finished the most easy OpenEngSB use case go on with a more complex one: Events

This tutorial shows how the event system in the OpenEngSB can be used. Therefore a log and a domain connector are created and configured. The context system in the OpenEngSB is used to define which connectors should be used and a simple event is used starting a rule.

If you are already familiar with using services in OpenEngSB about 30 minutes. (see HowTo: First steps)

This HowTo assumes you are already familiar with using and configuring services in OpenEngSB.

Now one logging service and one email service should be created. Create one notification service as described in the previous example. Please name it "notification" instead of test1 or test2. Now create a logging service:

logging service

Go to the "Context" page and configure the domains to use the connectors created:

context overview

On the "Send Event Page" you can create and edit Rules. Therefore they have to be edited directly with a text editor. The initial system is empty and does not include any rules. To create a rule choose "new". Enter "hello1" into the rulename input field. Make also sure that "Rule" is selected in the type dropdown box.

As soon as you edit the content of the rule you can save your changes by clicking "save" or revert the changes by clicking "cancel". The name of the rule will automatically be prefixed with "org.openengsb". Please insert the following content into the text box and save the changes:

#
# My notification rule
#
# Sends "Hello World" notification to test recipient.
#

when
	Event(name == "42")
then
	example.doSomething("Hello World");
	Notification n = new Notification();
	n.setSubject("testsubject");
	n.setMessage("testMessage");
	n.setRecipient("testRecipient");
	notification.notify(n);
    

event

Basically this rule reacts on all events (when clause). "log" is a helper class using the default log connector of the log domain to write information to a log file. Notification uses the default notification connector to inform a person. More details about this topic can be found in the user documentation at rules, domains and connectors.

To run a test the n.setRecipient property should be changed to a (e.g.) your email address.

Now we can throw an event and see if the rules work correctly. Stay on the "Send Event Page" enter for the contextId field "foo" and press send:

foo represents the name of the project. For a detailed description about projects and the context see the user documentation. You've should received a email via the rule to the email address configured previously). In addition using the "log:display" command in the OpenEngSB console should present (anywhere in the long log) a logging entry similar to the following (you have to search for the output of the LogService. The other fields can change):

output

Congratulation. You've gained basic knowledge about the OpenEngSB and its functionality. Nevertheless, you've just touched the surface. As a next step it is recommended continue with further tutorials user manual and start exploring the world of the OpenEngSB.

This tutorial describes how to setup a client project for OpenEngSB using maven archetype

If you are already familiar with the OpenEngSB about 30 minutes (This includes only the setup for the project). If you are not familiar with the OpenEngSB please read this manual from the start or check the homepage for further information.

You need to have following tools to be installed

The OpenEngSB provides an maven archetype to create a client project. To use it go into your target directory and type in a shell:

The script generates the result in the directory from where it was started

You will be asked to fill out following values (if no input is provided the default value is kept):

      Project Group Id [org.openengsb.client-project]:
      Project Artifact Id [openengsb-client-project]:
      Project Name [Client-Poject]:
      Project Version [1.0.0-SNAPSHOT]:
      Project Description [This is a client project for the OpenEngSB]:
      Project Url [http://www.openenbsb.org]:
      OpenEngSB version [1.2.0-SNAPSHOT]:
      OpenEngSB maven plugin Version [1.4.0-SNAPSHOT]:
      Plugin Assembly version [2.2-beta-5]:

You will be asked what the groupId, artifactId and a name of your client project should look like. You can also specify the OpenEngSB version you want to use, but it is recommended to use an up-to-date version. To check the current OpenEngSB version have a look at the Download section. It asks you to confirm the configuration and will create the project.

If everything worked as expected you will get a client project having following structure:

.
`-- openengsb-client-project
      |-- assembly
      | |-- pom.xml
      |     `-- src
      |     `-- main
      | |-- descriptors
      | |   `-- bin.xml
      |     `-- filtered-resources
      | |-- etc
      | |   `-- org.apache.karaf.features.cfg
      | |-- features.xml
      |     `-- README.txt
      |-- core
      |   `-- pom.xml
      |-- docs
      | |-- homepage
      | | |-- pom.xml
      | |     `-- src
      | |     `-- site
      | |     `-- ...
      | |-- manual
      | | |-- pom.xml
      | |     `-- src
      | |     `-- ...
      |   `-- pom.xml
      |-- LICENSE
      |-- poms
      | |-- compiled
      | |   `-- pom.xml
      | |-- nonosgi
      | |   `-- pom.xml
      | |-- pom.xml
      |     `-- wrapped
      |     `-- pom.xml
      |-- pom.xml
      `-- README.md

You can find further information about these modules in the OpenEngSB-Manual

To install features to the project have a look at the file org.apache.karaf.features.cfg in assembly/src/main/filtered-resource/etc. Here you can define features to be registered by default or which feature should be installed on startup. To install your own features see the file features.xml in assembly/src/main/filtered-resource. E.G.: You want to add a module called clientproject-ui the core features add this to the features.xml

      <bundle>mvn:org.openengsb.clientproject.ui/clientproject-ui-web$\{project.version\}/war</bundle>

To start the client-project, go to the command-window and type

mvn clean install openengsb:provision

Now you can enter "list" into the karaf console to check what features are installed and running

To shutdown, go to the command-window and type "shutdown" or press "Ctrl+D"

The current JMS Connector allows for internal method calls being redirected via JMS as well as internal services being called.

For resources regarding JMS please take a look at the according Wikipedia Page and for specific language bindings take a look at ActiveMQ

21.1.1. Proxying internal Connector calls

Whenever now a method is sent through the JMS Port the call is marshalled and sent via JMS to a queue named "receive"". The marshalling is done via JSON. The mapping has the parameters methodName, args, classes, metadata and potentially answer and callId. methodName gives the name of the method to call. Args are the serialised parameters of the method. classes are the types of the arguments. This way it is easy to unmarshall the args into the appropriate classes. metadata is a simple Map which stores key value pairs. answer can simply be yes or no and denotes if the methodcall wants an answer to the call. callId gives the return queue the caller will listen to for an answer.

An answer can have the type, arg, className and metaData properties. type can be Object, Exception or Void. arg is the serialised form of the return argument. className is the runtime class of the arg for deserialisation. metadata is a simple key value store.

21.1.1.1. HowTo call an external service via proxies

This section will give a short introduction how to instantiate a proxy and call an external connector

First you have to go to the TestClient to instantiate a new Proxy. Select the Domain you want to have proxied and click New Proxy for that Domain.

Testclient new proxy link

Then you have to set the correct values for the proxy properties. The Service Id is a unique value that identifies the proxy in the OPENENGSB system. The Port Id defines to Port to be used for sending the request. "jms-json" is a currently supported Port that sends the request via a json encoded JMS message. The destination describes the endpoint the message should be sent to. When using jms-json the domain and port of the JMS provider have to be set. When calling a remote connector the unique id of the remote service or connector has to be provided. This way the remote service can identify, load and call a certain service. If the call is not intended to go to another OPENENGSB, or the external service needs no identification of the service to call the remote service id can be omitted.

Create Proxy

After saving the proxy you should be able to test it via the TestClient page. Following is an example of an unsecured call:

{
  "authenticationData": {
    "className":"org.openengsb.core.api.security.model.UsernamePasswordAuthenticationInfo",
    "data": {
      "username":"admin",
      "password":"password"
    }
  },
  "timestamp":42,
  "message": {
    "callId":"xyz",
    "answer":true,
    "methodCall": {
      "classes": [
        "java.lang.String",
        "org.openengsb.core.api.workflow.model.ProcessBag"
      ],
      "methodName": "executeWorkflow",
      "metaData": {
        "serviceFilter": "(objectClass=org.openengsb.core.api.workflow.WorkflowService)",
        "contextId": "foo"
      },
      "args": [
        "simpleFlow",
        {
        }
      ]
    }
  }
}

IF you would like to use security instead you should prefer the following call:

{
{
  "encryptedContent":"encodedMessage", // Base 64 and encryped string of the message above
  "encryptedKey":"encodedKey" // The encoded key
}

Test a proxy

When proxying connector calls you have to provide an answer to every call, as the system blocks until it gets an answer. You have to send a JSON message containing a type string parameter, which can be Object, Exception or Void depending on the return argument of the method, arg where you simply serialise the Return Object, so it can be deserialised into the correct object later and className which gives the exact class that has to be used for deserialisation. The request contains a parameter callId which is the name of the queue the answer has to be sent to.

{"type":"Object","className":"org.openengsb.core.ports.jms.JMSPortTest$TestClass",
          "metaData":{"test":"test"},"arg":{"test":"test"}}

Whenever a call to this proxy is then made a new JMS message will be sent to the "receive" queue on the destination you entered. The exact make up of the message was already described. When implementing an external connector it is best to test the call you want to receive first via the TestClient, so you get the exact message that you will have to work with.

Please always keep in mind security. By default security is turned on. If you want to turn it off please take a look into the etc/system.properties file. While using no security for testing is very interesting we would not advise you to send unencrypted messages in a production environment.

<process xs:schemaLocation="http://drools.org/drools-5.0/process 
  drools-processes-5.0.xsd" type="RuleFlow" name="simpleFlow" id="simpleFlow" 
  package-name="org.openengsb" xmlns="http://drools.org/drools-5.0/process" 
  xmlns:xs="http://www.w3.org/2001/XMLSchema-instance">
 <header>
  <variables>
   <variable name="processBag">
    <type name="org.drools.process.core.datatype.impl.type.ObjectDataType" 
     className="org.openengsb.core.api.workflow.model.ProcessBag"/>
   </variable>
  </variables>
 </header>
 <nodes>
  <start id="1" name="Start" x="16" y="16" width="91" height="48"/>
  <end id="2" name="End" x="21" y="168" width="80" height="40"/>
  <actionNode id="3" name="Action" x="21" y="96" width="80" height="40">
   <action type="expression" dialect="mvel">
    processBag.addProperty("test", 42);
    processBag.addProperty("alternativeName", "The answer to life the universe and everything");
   </action>
  </actionNode>
 </nodes>
 <connections>
  <connection from="3" to="2"/>
  <connection from="1" to="3"/>
 </connections>
</process>

TBW(Jira-ISSUE)

Basically there is nothing special to do to get up a service. Still, since java has no static type information there is no possibility to marshal things like a java.util.List of options. The fix we've introduced for this problem is to annotate parameters and define a custom marshaller. Therefore create a class with a default constructor extending CustomJsonMarshaller. Then annotate the parameter of the implementations you're using with the UseCustomJasonMarshaller annotation adding the Marshaller class. That's it. The framework will automatically create the mapper and call the transform method for you.

Since there is an issue right now with Aries Proxies hiding parameter annotations the CustomMarshallerRealTypeAccess interface provides a workaround for the problem. In case this interface is found the class provided by it is used for searching. Otherwise the class directly is scanned.

There are also other issues with using multiple service in a place where one single service is expected:

All this is specified by a CompositeStrategy. Strategy-implementations must implement the CompositeConnectorStrategy interface and be registered as OSGi-services exporting this interface. Also the service must specify the "composite.strategy.name" property. The strategy is provided with a list of ServiceReferences, and the invocation parameters. The implementation can decide which services to resolve and to invoke. Also the strategy must return a single result.

A composite connector instance can be created like a regular connector-instance. You need to supply the following attributes:

This tutorial explains how to define a domain model for a specific domain. What a domain model is, can be read in the user manual in the semantics section(Chapter 6, Semantics in the OpenEngSB). The structure of a domain model is an interface which extends the OpenEngSBModel.

If you are already familiar with the OpenEngSB about 10 minutes. If you are not familiar with the OpenEngSB please read this manual from the start or check the homepage for further information.

For information about how to get started as contributor to the OpenEngSB project and how to get the current OpenEngSB source please read the contributor section of the manual: Part VI, “OpenEngSB Contributor Detail Informations”.

The first thing to do is to think about the structure of the model you need. Think about which informations are needed and should be included. (e.g. if you want to create a domain model for an appointment domain, the domain model will contain informations like start time, end time, title, ...).

Also give a think about if there exists a field which has the potential to be the id of a model. Such a field has to be unique for a specific domain and connector combination. Such an id can be defined through an annotation with the name OpenEngSBModel.

Writing a model is quite simple. A domain model is an interface, which contains only pairs of getter and setter methods. The method names define the internal names of the fields of the model. Example model:

            interface Appointment extends OpenEngSBModel {
                 @OpenEngSBModelId
                 void setId(Integer id);
                 Integer getId();
                 
                 void setStartTime(Date startTime);
                 Date getStartTime();
                 
                 void setEndTime(Date endTime);
                 Date getEndTime();
                 
                 ...
            }
        

This model defines a part of a domain model for an appointment domain. In this model we also have defined the id of the model, with the name id. You can see that through the set annotation.

The last step is to add the model to the specific domain. For that you simply have to add the model to the model package in the specific domain. Now the model is ready to use in the connectors for the specific domain.

The last step is to use the model. For that you have to use a util class which is placed in the core/common bundle called ModelUtils. An instantiation of a model looks like this:

            Appointment appointment = ModelUtils.createEmptyModelObject(Appointment.class);
        

After that you can use the appointment object like a normal instantiated object.

The service integration tutorial shows how to combine and automate different software tools, services and applications with OpenEngSB. To show OpenEngSB's versatility the use case we will be implementing is a continuous integration (CI) tool for software development processes. The tutorial takes a straight forward approach favoring visible results over architectural details of tool integration. Whether or not you have experience with CI, bear with the tutorial for a moment and you will see how simple it works out.

Before we get started let us lay out the idea of our CI tool and create a step-by-step development plan. The practice of continuous integration aims at improving software quality by frequent (automated) building and testing of a project's source base and by reporting back to the developers. The CI tool must be able to access the source repository, build the project, test the binaries and reports to the developers. And there we have a basic four step plan:

(1) Repository access

(2) Building source

(3) Testing binaries

(4) Notification process

If you would like to take a look at a fully functional CI server built on OpenEngSB check out OpenCIT. It implements a wider range of features, but it's a great reference.

If you are already familiar with the OpenEngSB about 30 minutes. If you are not familiar with the OpenEngSB please read this manual from the start or check the homepage for further information.

It is assumed you have basic knowledge of software development practices and you are able to set up auxiliary software (i.e. JDK 1.6) yourself.

Warning: This section is likely to change in the near future, as the web UI as well as domains and connectors are subject to change.

download openengsb

Getting OpenEngSB is simple. Go to openengsb.org, download the latest stable version to your computer and unpack the archive to a convenient location. Before you fire up OpenEngSB for the first time, please make sure you have a Java Development Kit 1.6+ available and set up.

openengsb console

You can startup OpenEngSB via the openengsb script in the "bin" folder. If you want to explore the web interface yourself before digging into implementing the CI use case, open up your web browser and navigate to http://localhost:8090/openengsb and log on as "admin" with password "password".

openengsb web UI

If you want to take a break or shutdown OpenEngSB in the middle of the tutorial, go ahead and do not worry. All changes made so far are saved and restored upon restart, so you can continue working with the most up-to-date state. Use the shutdown command in OpenEngSB's management console to stop any running services.

check maven and JDK version

It turns out, we actually need a sample project before we can start developing and testing our CI tool. In this tutorial we will be using Apache Maven for project and source management and a small Hello World application written in Java. For this to work flawlessly we need JDK 1.6+ (download) and Maven 3+ (download) to be set up on the computer.

tutorial project package

You may download and extract the openengsb-tutorial (download) project that works out of the box or set up your own sample project via maven archetypes. Put the project files in a memorable location (i.e. "/home/user/Desktop/openengsb-tutorial" or "C:\users\user\desktop\openengsb-tutorial") and that's about it for now.

login

You could certainly build the project using Maven, but we want this to happen from within OpenEngSB. Open up your browser and go to the web interface at http://localhost:8090/openengsb. Authenticate using the Login link as "admin" with password "password".

test client

Switch to the Test client tab and check whether the build domain is available to accept commands.

install connector in console

As the build domain is not shown, we will need to go to OpenEngSB's console. Use the "features:install openengsb-connector-maven" command to load the tool connector for Apache Maven. The tool connector allows OpenEngSB to communicate with an external service or application, Maven in this case. By loading the connector OpenEngSB also loads the domains associated with the connector. A domain is a generic interface that is implemented by specific connectors. For example, the build domain offers a generic function to build a software project. Whether this is done by Apache Maven, an Ant script or a plain compiler depends on the connector chosen by the user, but does not affect the basic model of our CI tool that simply "builds" the source. Using this technique specific tools can be exchanged quickly and transparently while data-flow and process models are completely unaffected.

create new connector instance

Now let us return to the web interface. After a page refresh we are able to create a new Maven connector instance by following the new link next to the connector description.

setup build connector

The service id is an arbitrary name that can be referred to later on, i.e. builder. The project path refers to the local copy of the source repository, i.e. "/home/user/Desktop/openengsb-tutorial". Finally, the maven command is the command line option handed to Apache Maven, i.e. "clean build". Whenever the connector is ordered to build the project it will now execute "mvn clean compile" in the project directory of openengsb-tutorial. Click save to create the connector instance.

test build connector

It is time for some action. Scroll down all the way in the Test Client tab, navigate to the BuildDomain in the tree view and click the builder service. From the drop down menu below select build() and click the "Call" button. When the project is built successfully a "Method called successfully" message appears.

hello world

Let us check the directory of openengsb-tutorial to find the newly created target directory. Using a console we can navigate to the classes sub-folder and run "java org.openengsb.Main". It returns "Hello World!". Congratulations, you have just implemented the core functionality of our CI tool.

In case an error message is returned, we need to check the connector configuration. Please make sure that Maven is able to build the project by manually executing "mvn clean build" in the project directory of openengsb-tutorial. If this does not work out, most likely the setup of either Maven or JDK are incorrect. If it works however, please check the configuration of the connector by navigating to the builder service in the web interface again and clicking the "Edit Service" button.

The next step in building the CI tool is implementation of automated testing. We can use the test domain to achieve this. Conveniently, the Maven connector also exports this type of functionality and we do not to load any additional features in the management console.

setup test connector instance

Use the Test client tab and create a new instance of the Maven connector for the "test" domain. It works the same way as before. The Service Id is an arbitrary name, i.e. tester. The project path points at the location of openengsb-tutorial's base directory, i.e. "/home/user/Desktop/openengsb-tutorial", and the maven command indicates the command line arguments passed to Maven, i.e. "test". Click save to create the connector instance.

test test connector (yes, that's necessary)

We are also going to test the newly created connector. Use the tree view at the bottom of the page to navigate to TestDomain and select the tester service. In the drop down box below choose runTests() and click the "Call" button. If the "method called successfully" message is returned, the Maven connector ran the rigorous unit tests on our sample application and they were passed with flying colors.

send event page

In case you are wondering about the output of the test suite go to the Send Event page. There you will find an audit list of all events processed by OpenEngSB and their corresponding payload, i.e. the output of the test run. The list behaves like a log file with most recent events appended to the bottom. In OpenEngSB any input from connectors is processed by domains and packaged in form of events. Every time an event is raised it can be matched by a rule in a central rule-base and cause a reaction of the system possibly invoking different domains or spawning additional events. By editing the rule-base we become able to link different actions together. Depending on the outcome of an action, we may receive different events, i.e. build success or build failure. By creating separate rules for each case we can react accordingly and by chaining multiple actions and events we could create a longer decision tree or process model.

In the final step we link the build and test stages together and add functionality to output results of the process. For this purpose we will create a small number of rules that react to events generated by the build and test stage. To keep things simple we do not add further connectors and assume that the build process is started by calling the builder's "build()" method in the Test client web UI.

We are going to write the results of the build and test stage to the management console of OpenEngSB. We will notify about the build starting and its outcome. Also, in case the build works out successfully we will automatically start the test process. Hence, we need rules matching "BuildStartEvent" and "BuildSuccessEvent" that write output to the console and potentially activate the test connector.

rule base editor

The easiest way to edit the rule-base of our OpenEngSB instance is the editor area at the bottom of the Send Event page. You can find event types and names by checking the event log displayed on the page and create rules manually in the editor below. The rules are written in plain text, based on Drools (documentation) and Java standards. They are quickly understood, just take a look at the auditEvent rule used to generate the event log displayed on top of the page. You can display the rule by choosing "Rule" in the left-most drop-down box and "org.openengsb.auditEvent" in the second one. The text editor now shows that there is a when section that acts as filter for incoming events and there is a then section that describes the actions to be taken in case of a match.

build started rule

Let us start out and create our own rule. We will inform the user via console when a build process starts. Click the new button next to the drop-down boxes, enter a name for the newly created rule, i.e. "buildStarted", and save it. After selecting the rule in the drop-down box the text editor actually shows the same content as for the audit event. Do not be confused, it's the default template. You can start editing right away. Note: Writing output directly to the console without using a logger service is considered bad practice. Yet, it's simple and sufficient for demonstration purposes.

    
when
  Event(type == "BuildStartEvent")
then
  System.out.println("Build started");
    

This rule matches "BuildStartEvent" and prints a line to the console window. Click save to prepare the rule for testing. Switch to the Test client tab, select the builder and call the "build()" method. If things work out, you'll instantly see the "Build started" notification pop up in the management console.

It's time to push this further. Create another rule, i.e. "buildSuccessful" and edit it to look like this:

    
when
  Event(type == "BuildSuccessfulEvent")
then
  System.out.println("Build successful, running tests");
  tester.runTests();
    

Note: as of OpenEngSB 1.1.2 there is a bug in the editor that prevents connector invocations ("connectorId.doSomething()") from working correctly.

console output

The rule also matches "BuildSuccessEvent" and prints a line to the console window. In addition, however, it calls a method provided by the test connector. Remember the "runTests()" method you called in the tester service by using the test client before? This has exactly the same effect but replaces manual UI interaction with an automated response. Click save and kick off another build using the Test client again. There you go: "Build started" and "Build successful, running tests".

Congratulations, you have created a basic CI tool! The foundations have been extended to allow for easy auditing and extensibility. Of course, at the moment it simply replicates the functionality of existing CI tools, but it can be easily extended using SCM access, reporting and notification tools and work together with project management software and PIM applications. Take a look on the long list of available domains and tool connectors.

If you want to do some more practice you can add more rules, i.e. for "TestSuccessEvent" or "BuildFailureEvent". You can find event types, names and properties by checking the event log displayed on the Send Event page.

There are a number of different HowTo's and tutorials in the online documentation. They describe different scenarios for setup, connectors and domains and event processing. Also, the user manual contains additional information about the topics discussed and numerous OpenEngSB sub projects, i.e. OpenCIT and OpenTicket, can be used for reference.

This tutorial describes how to change the database which is used by the EDB to persist OpenEngSBModels.

The time needed to perform this task depends heavily on which database you want to use instead of the standard H2 database and on your experience with the OpenEngSB. This chapter will list the different possibilities from the shortest time period needed to the most complex one. If you are not familiar with the OpenEngSB please read this manual from the start or check the homepage for further information.

If you want to replace the standard database with another JPA supported database (supported databases listet here), the procedure of changing is rather simple. You have to change the properties for the database connection of the "infrastructure/jpa" bundle. This is done by changing the config file "org.openengsb.infrastructure.jpa.cfg". This file is in the "assembly/src/main/filtered-resources/etc" folder. The file has the following entries:

                  driverClassName=[here shall be the name of your driver class name, 
                     example: org.h2.jdbcx.JdbcDataSource]
                  url=[here shall be the url to the used database file, example: jdbc:h2:openengsb]
                  username=[here shall be the username for authentication at the database]
                  password=[here shall be the password for authentication at the database]
            

After changing of this file, you have to assure that the driver which shall be used by the data source is loaded at runtime. For that you have to load the database driver before the connection to the database is established. Your database bundle dependency should be added to the "parents/shared/pom.xml" file. A dependency entry for the h2 database looks like this (the h2.version is defined as property in the same file:

              <dependency>
                 <groupId>com.h2database</groupId>
                 <artifactId>h2</artifactId>
                 <version>1.3.163</version>
              </dependency>
            

After you added the dependency, you have to make sure the database is loaded before the database connection is established. For that just replace following code line in the file "assembly/src/main/filtered-resources/features.xml" with the new database driver bundle (as you can see you only have to copy and paste the values of the dependency):

               <bundle>mvn:com.h2database/h2/1.3.163</bundle>
            

After this steps, you are done and the EDB uses now the new database as persistence backend.

If you want to replace the standard database with another non JPA supported database you will have to change first the data source like described in the previous section. After that you have to write your own bundle that provides the EngineeringDatabaseService. In this case you also have to reproduce all SQL commands and port it for the new database and replace the standard EngineeringDatabaseService.

In this case you have to wrap the driver for the database. How to do that can be read in the developer manual where wrapping is explained.

Those sections shows what have to be done initally.

26.1.1. Preparing the Server

Before starting the server you have to edit the file OPENENGSB_HOME/etc/system.properties You will find the lines:

# Uncomment to disable encryption for jms-messages
#org.openengsb.jms.noencrypt=true
        
# Uncomment to disable signature and timestamp verification for all remote requests
#org.openengsb.security.noverify=true

Uncomment (or if not existing add) the lines

org.openengsb.jms.noencrypt=true 
org.openengsb.security.noverify=true

This will enable you to send unencrypted messages to the server.

Now start the server and enter the following line:

features:install openengsb-ports-jms

This will open the JMS port of the OpenEngSb server, which is used to receive Json messages.

Installation Check

Have fun testing with Json. :-)

The appointment domain provides the possibility to create, update, delete and retrieve appointments.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

/**
 * This domain is used to integrate different calendar programs. It can create, update, delete and
 * retrieve Appointments.
 */
public interface AppointmentDomain extends Domain {

    /**
     * creates an appointment and returns the generated id
     */
    String createAppointment(Appointment appointment);

    /**
     * updates an appointment
     */
    void updateAppointment(Appointment appointment);

    /**
     * deletes an appointment
     */
    void deleteAppointment(String id);

    /**
     * loads an appointment
     */
    Appointment loadAppointment(String id);

    /**
     * Returns a list of appointments which are lying between the given start and end time.
     */
    ArrayList<Appointment> getAppointments(Date start, Date end);
}

The following interface presents the events an appointment connector can throw:

public interface AppointmentDomainEvents extends DomainEvents {
}

The build domain builds a specific pre-configured project or suite of projects.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

/**
 * This domain can be used to build projects. The affected project is usually configured in the respective tool
 * connector.
 */
public interface BuildDomain extends Domain {

    /**
     * build the currently configured project. This method returns at once with an id. The build is conducted
     * asynchronously. The result can be retrieved using the events raised by this domain, which also contain the id.
     */
    @Raises({ BuildStartEvent.class, BuildSuccessEvent.class })
    String build();

    /**
     * build the currently configured project. This method returns at once with an id. The build is conducted
     * asynchronously. As soon as the build is finished an event is raised. The processId-field of the event must be
     * populated with the supplied processId.
     */
    @Raises({ BuildStartEvent.class, BuildSuccessEvent.class })
    void build(long processId);

}

The following interface presents the events a build connector can throw:

public interface BuildDomainEvents extends DomainEvents {

    void raiseEvent(BuildStartEvent e);

    void raiseEvent(BuildSuccessEvent e);

    void raiseEvent(BuildFailEvent e);
}

The contact domain provides the possibility to create, update, delete and retrieve contacts.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

/**
 * This domain is used to maintain different contact books in different tools like gcontacts or facebook.
 */
public interface ContactDomain extends Domain {

    /**
     * creates a contact on the server and returns the generated id
     */
    String createContact(Contact contact);

    /**
     * updates a contact on the server
     */
    void updateContact(Contact contact);

    /**
     * loads a contact from the server
     */
    Contact loadContact(String id);

    /**
     * deletes a contact on the server
     */
    void deleteContact(String id);

    /**
     * retrieves a list of contacts from the server based on "query by example"
     */
    ArrayList<Contact> retrieveContacts(String id, String name, String homepage,
            Location location, Date date, String comment);
}

The following interface presents the events an appointment connector can throw:

public interface ContactDomainEvents extends DomainEvents {

}

The deploy domain deploys a specific pre-configured project or suite of projects.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

/**
 * This domain can be used to deploy projects. The affected project is usually configured in the respective tool
 * connector.
 */
public interface DeployDomain extends Domain {

    /**
     * Deploy the currently configured project. This method returns at once with an id. The deploy process is conducted
     * asynchronously. The result can be retrieved using the events raised by this domain, which also contain the id.
     */
    @Raises({ DeployStartEvent.class, DeployEndEvent.class })
    String deploy();

    /**
     * Deploy the currently configured project. This method returns at once with an id. The deploy process is conducted
     * asynchronously. The result can be retrieved using the events raised by this domain where the processId-field
     * contains the supplied processId
     */
    @Raises({ DeployStartEvent.class, DeployFailEvent.class, DeploySuccessEvent.class })
    void deploy(long processId);
}

The following interface presents the events an appointment connector can throw:

public interface DeployDomainEvents extends DomainEvents {

    void raiseEvent(DeployStartEvent e);

    @Deprecated
    void raiseEvent(DeployEndEvent e);

    void raiseEvent(DeployFailEvent e);

    void raiseEvent(DeploySuccessEvent e);
}

The issue Domain provides the possibility to create, update, delete and comment issues.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

public interface IssueDomain extends Domain {

    /**
     * creates an issue on the server and returned the generated id
     */
    String createIssue(Issue issue);

    /**
     * add a comment to an issue, specified by his id
     */
    void addComment(String id, String comment);

    /**
     * update an issue, specified by his id, the comment param can be null, changes: key of map is what field has to be
     * changed,
     */
    void updateIssue(String id, String comment, HashMap<IssueAttribute, String> changes);


    /**
     * move all issues from one release ( specified by releaseFromId ) to another release ( specified by releaseToId )
     */
    void moveIssuesFromReleaseToRelease(String releaseFromId, String releaseToId);

    /**
     * close a release specified by his id
     */
    void closeRelease(String id);

    /**
     * generates an report for all closed issues belonging the the specified release
     */
    List<String> generateReleaseReport(String releaseId);

    /**
     * adds a component
     */
    void addComponent(String component);
    
    /**
     * removes a component
     */
    void removeComponent(String component);
}

The following interface presents the events an appointment connector can throw:

public interface IssueDomainEvents extends DomainEvents {

}

The notification domain provides the functionality for sending notifications to a specific recipient.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

public interface NotificationDomain extends Domain {

    void notify(Notification notification);

}

The following interface presents the events an appointment connector can throw:

public interface NotificationDomainEvents extends DomainEvents {

}

The report domain supports basic report generation functionality, like event logging and manual report building. Furthermore it provides basic report management features, like persistent storage of reports and a category system for report storage.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

public interface ReportDomain extends Domain {

    /**
     * Generate the report using the information stored under the given reportId and stop collecting data for this
     * report. The report is added to the specified report category and named as specified by the {@code reportName}
     * parameter. The category is created if it is not already present. If there is already a report with the same name
     * under this category it is overwritten.
     *
     * @return the generated report
     * @throws NoSuchReportException if no report with the given {@code reportId} is currently collecting data.
     */
    Report generateReport(String reportId, String category, String reportName) throws NoSuchReportException;

    /**
     *
     * Generate a report based on the currently available data stored for the given {@code reportId}. The report is
     * generated but not stored. Furthermore the data collection is not stopped for this report. This method can be used
     * if a preview of the report is needed, but the data collection is not finished yet.
     *
     * @return the generated report draft
     * @throws NoSuchReportException if no report with the given {@code reportId} is currently collecting data.
     */
    Report getDraft(String reportId, String draftName) throws NoSuchReportException;

    /**
     * Start a report data collection. If the data collection process is finished the report can be generated by calling
     * the {@link #generateReport(String, String, String)} method specifying the reportId returned by this method.
     *
     * @return the reportId that can later be used to generate the report by calling
     *         {@link #generateReport(String, String, String)}
     */
    String collectData();

    /**
     * Add the given report part to the report data currently collected for the given {@code reportId}. The report part
     * is appended at the end of the report.
     *
     * @throws NoSuchReportException if no report with the given {@code reportId} is currently collecting data.
     */
    void addReportPart(String reportId, ReportPart reportPart) throws NoSuchReportException;

    /**
     * Analyzes the given event and adds all information stored in the event to the report data collection associated
     * with the given {@code reportId}, which was initialized by calling {@link #collectData(IdType, String)}.
     *
     * @throws NoSuchReportException if no report with the given {@code reportId} is currently collecting data
     */
    void processEvent(String reportId, Event event) throws NoSuchReportException;

    /**
     * Get all finished reports of the given category. Reports, which are currently generated and collect data are not
     * included. If the given category does not exist an empty list is returned.
     */
    List<Report> getAllReports(String category);

    /**
     * Store the given report in the report store under the given category. The category is created if it is not already
     * present. If there is already a report with the same name under this category it is overwritten.
     */
    void storeReport(String category, Report report);

    /**
     * Remove the given report from the given category. The report is deleted and cannot be restored if it is not also
     * stored under another category. If no such report exist no operation is performed.
     */
    void removeReport(String category, Report report);

    /**
     * Get all report categories.
     */
    List<String> getAllCategories();

    /**
     * Remove the given category and all reports stored in this category. If no category with the specified name exists,
     * nothing is done.
     */
    void removeCategory(String category);

    /**
     * Creates the given category, which is empty at the start. Reports can later be added by either calling
     * {@link #storeReport(String, Report)} or {@link #generateReport(String, String, String)} specifying the respective
     * category. If a category exists with the given name no operation is performed.
     */
    void createCategory(String category);
}

The following interface presents the events an appointment connector can throw:

public interface ReportDomainEvents extends DomainEvents {

}

The SCM Domain polls external repositories for changes of content under source control and provides functionality to copy/export the repository content for further processing.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

/**
 * ScmDomain is an abstraction for working with SCM tools.
 * 
 */
public interface ScmDomain extends Domain {

    /**
     * Looks up changes in a remote repository and updates the local repository
     * or checks out a new local repository and returns a list of
     * {@link CommitRef} with the revisions produced since the last update or
     * <code>null</code>.
     */
    List<CommitRef> update();

    /**
     * Exports the files and directories of the HEAD revision from a repository
     * without the SCM specific data in a compressed format.
     */
    File export();

    /**
     * Exports the files and directories of a revision identified by the
     * {@link CommitRef} from a repository without the SCM specific data in a
     * compressed format.
     */
    File export(CommitRef ref);

    /**
     * Check if file identified by its {@code fileName} exists in the HEAD
     * revision and returns <code>true</code> if it does.
     */
    boolean exists(String file);

    /**
     * Retrieves a single {@link File} from a repository identified by its
     * {@code fileName} if it exists in the HEAD revision. If the file does not
     * exist in the revision <code>null</code> will be returned.
     */
    File get(String file);

    /**
     * Check if file identified by its {@code fileName} exists in a revision
     * identified by the {@link CommitRef} and returns <code>true</code> if it
     * does.
     */
    boolean exists(String fileName, CommitRef version);

    /**
     * Retrieves a single {@link File} from a repository identified by its
     * {@code fileName} if it exists in the revision identified by the
     * {@link CommitRef}. If the file does not exist in the revision
     * <code>null</code> will be returned.
     */
    File get(String fileName, CommitRef ref);

    /**
     * Returns the {@link CommitRef} of the current HEAD in the repository or
     * <code>null</code>.
     */
    CommitRef getHead();

    /**
     * Adds one or more {@link File} existing in the working directory to the
     * repository and commits them with the passed {@code comment}. Returns the
     * {@link CommitRef} of the commit triggered.
     */
    CommitRef add(String comment, File... file);

    /**
     * Removes one or more {@link File} from the working directory of the
     * repository and commits them with a passed {@code comment}. Returns the
     * {@link CommitRef} of the commit triggered.
     */
    CommitRef remove(String comment, File... file);

    /**
     * Tags the actual HEAD of the repository with the passed {@code tagName}
     * and returns the corresponding {@link TagRef} or <code>null</code>.
     */
    TagRef tagRepo(String tagName);

    /**
     * Tags the commit of the repository identified by the {@link CommitRef}
     * with the passed {@code tagName} and returns the corresponding
     * {@link TagRef} or <code>null</code>.
     */
    TagRef tagRepo(String tagName, CommitRef ref);

    /**
     * Resolves and returns the {@link CommitRef} for a {@link TagRef} or
     * <code>null</code> if the reference does not exist.
     */
    CommitRef getCommitRefForTag(TagRef ref);

}

The following interface presents the events an appointment connector can throw:

public interface ScmDomainEvents extends DomainEvents {

}

The test domain runs all tests for a specific pre-configured project or suite of projects.

The following listing presents the Java Domain Interface. This interface also contains information about events raised by this domain.

/**
 * This domain can be used to run tests for projects. The affected project is usually configured in the respective tool
 * connector.
 */
public interface TestDomain extends Domain {

    /**
     * run all tests for the currently configured project. This method returns at once with an id. The tests are run
     * asynchronously. The result can be retrieved using the events raised by this domain, which also contain the id.
     */
    @Raises({ TestStartEvent.class, TestSuccessEvent.class, TestFailEvent.class })
    String runTests();

    /**
     * run all tests for the currently configured project. This method returns at once with an id. The tests are run
     * asynchronously. The result can be retrieved using the events raised by this domain, with the processId set to the
     * supplied processId
     */
    @Raises({ TestStartEvent.class, TestSuccessEvent.class, TestFailEvent.class })
    void runTests(long processId);
}

The following interface presents the events an appointment connector can throw:

public interface TestDomainEvents extends DomainEvents {

    void raiseEvent(TestStartEvent e);

    void raiseEvent(TestEndEvent e);

    void raiseEvent(TestFailEvent e);

    void raiseEvent(TestSuccessEvent e);
}

The email connector configuration recognizes the following attributes:

TBW

TBW

TBW

TBW

TBW

TBW

TBW

TBW

It is not much needed to get the OpenEngSB console started. Just type in a shell "mvn openengsb:provision" or execute the corresponding shell script (etc/scripts/run.sh)

This section is work in progress, which means that it will be extended every time a new command is available

Global variables are needed to access domain-services and other helper-objects. To manage this variables there is a button in the testclient which brings you to a own site where you can manage this variables. Here you can add, edit and delete them. If some error occur while managing the variables(deleting a variable that is already in a rule, ...), the site tells you what did go wrong.

Imports are important so that global variables can work. If the class of the global variable isn't imported an error occurs because the necessary classes can't be loaded. To manage this imports there is a button in the testclient which brings you to a own site where you can manage this imports. Here you can add, edit and delete them. If some error occur while managing the imports, the site tells you what did go wrong.

The properties of the service will be updated. First, it will try to get the property with the key 'location.' + context from these properties, because there are all locations stored. If there is no such property, a new one will be added. After that, it will insert a new location in that property, which is the name of the given global. As there can be more locations, the new one will be appended excepted such location already exists. Then nothing will be changed, but an info message will appear.

This chapter is a step by step guide on how to create a wrapped JAR.

Although the description above sounds quite simple (and wrapping bundles is simple mostly) still some nasty problems can occur. This section summarizes good tips and ideas how to wrap bundles within the OpenEngSB.

The idea behind the concept of the OpenEngSBModels or domain models is corelated to the domain/connector structure of the OpenEngSB. Models should represent the objects used by a domain in an abstract manner (e.g. the model "Issue" in the issue domain). Since such models are defined on the domain level. All connectors, which belong to a specific domain, use the models defined by this domain. In that way every connector of a specific domain "speaks the same language".

What is a little bit different from the normal approach to develop and design such domain models is, that in the OpenEngSB all domain models are interfaces. This interfaces have to extend the interface "OpenEngSBModel" which is defined in the "core/api" bundle. Which advantage this brings will be explained later on in this chapter.

Another important thing is the structure of such a model interface. Every domain model or OpenEngSBModel interface contains only getter and setter pairs. It is important that every getter has a equivalent setter and the other way round to ensure correct behaviour of this models.

But how to work only with interfaces? Since an OpenEngSBModel is only an interface, we need a mechanism to work in an efficient way with such objects. The way we choose was to create a static model proxy class, which simulates the implementation of a model. The class enabling this, is the ModelUtils class which can be found in the "core/common" bundle.

Every OpenEngSBModel object has three defined functions: getOpenEngSBModelEntries, addOpenEngSBModelEntry and deletOpenEngSBModelEntry. Whenever getOpenEngSBModelEntries is called, the model transform itself into a list of key/value pairs, where every entry have in addition the type of the value saved. This mechanism is used for easier saving of models to the EDB and also to transfer a model to a different machine. With the addOpenEngSBModelEntry you can add additional entries which are not part of the model definition itself to the model. And finally with the removeOpenEngSBModelEntry you can manually remove an entry which has been inserted manually through addOpenEngSBModelEntry.

Currently the models are able to work with primitive types, strings, dates, lists, models and files. All those objects can be set and got through the interface and can also be persisted in the EDB (see the chapter about the EDB for more details).

Special case in the supported field types are file objects. They are quite tricky, especially if the models shall be transfered to another machine. The behaviour of the models are: Whenever a model have to transform itself to a list of OpenEngSBModelEntry, the model is aware of file objects. If it finds a file object, it creates a FileWrapper object out of it. A FileWrapper object contains the name of the file and a byte array which holds the zip compressed content of the file object.

If such a FileWrapper would be accessed by a getter of a file object, the ModelUtils does the conversion work of the FileWrapper to the file object for you. So this feature is completely transparent for the normal user. Note that the conversion of a file object to a FileWrapper and the other way round has only to be done if getOpenEngSBModelEntries is called. For now this is only the case at two points: when a model has to be saved into the EDB or when a model is sent via remote.

For easier maintaining and faster finding of models (and also to enable the versioning possibility), the user is able to define a field to be the id of a model. An important point to consider here is that this id has to be unique for a connector of a domain (this is because the id will be enlarged with the domain id and the connector id).

To define a field to be the id of the model you simply have to add an annotation to the corresponding setter. The annotation is called OpenEngSBModelId. If no id is defined for a model and this model has to be inserted into the EDB, the EDB just take an arbitrary id to save the model.

The EDB concept was introduced with the idea to build a central persisting unit for the domain models of all domains. This central approach offers some interesting advantages, like for example:

Model transformation is the out of the scope of the EDB and will be covered by the EKB.

Another important feature of the EDB are build-in conflict checkers. Until now, there is only one implementation which is based on version numbers. Whenever someone tries to save something into the EDB with the wrong version, the conflict checker tells the user that a conflict has been found and that he has to checkout the newest version of the model before he can save the model.

The EDB is a JPA based implementation of a central database supplier in service orientated architectures, which also have the additional functionality to version data. Currently we are using OpenJPA as JPA implementation.

Since the EDB simulates the functionality of a scm system, the structures of the tables in the EDB are no big surprise. They consist of objects which have a list of key/value pairs bound to them. Also there exist a commit table, with which the EDB is able to keep track of all meta-data of changes.

Until now, the saving/updating of models into/in the EDB is done through specific events which every connector which implements the interface "OpenEngSBConnectorService" can throw. This events hide many details of the saving procedure like the automatically load from where the event is coming and with this, an automatic event enhancement. Also the conflict check is started through such an event. In future the EDB has to be used directly without events to easier enable the possibility to use the EDB functionality in workflows(Jira-ISSUE).

The loading of models from the EDB is in conceptionally possible, but should always be done through the QueryService of the EKB bundle, since this service does the automatic transformation work of elements from the EDB to an OpenEngSBModel. See the detailed explanation of the EKB for more informations.

The conflict detection, as it is implemented now, is a very simple implementation of a conflict checker. The checker is based on simple version numbers. If the version number of the model which has to be saved doesn't fit to the actual version number, the conflict detection throws an error.

In the future there should be more possibilities for conflict detections been found. But for the start, this conflict detection is enough(Jira-ISSUE).

The main idea for the introduction of the EKB is the idea to provide automatic transformations of domain models between tools and the OpenEngSB. But soon it was clear that this will not the only task the EKB should provide.

Some other things the EKB shall provide when it is finished will be the QueryInterface service which represents the loading force of models from the EDB (this is already implemented), the possibility to provide better conflict checkers for the EDB and if possible automatic conflict solving.

This service has the task to load models from the EDB. Since models are no real objects but proxied interfaces, this service has to create a new model instance. This new model instance is initialised with the data which is loaded from the EDB. After all loaded data is inserted, the user can start work with this model. All transformations are done in this service. The user doesn't have to worry about that. In fact, he won't even notice that all this steps were necessary.

The query service provide a list of possibilities how to load models from the EDB (e.g. based on the model id or based on the key value pairs of the model).

In case it isn't known what a tool domain is and how it defines the interface for the tool connector then Section 5.4, “OpenEngSB Tool Domains” is a good starting point. If there's already a matching domain for this tool it is strongly recommended to use it. If the tool requires a new domain to be created relevant information can be found in Chapter 53, How To Create an Internal Domain.

To take burden off the developer of creating the initial boilerplate code and configuration, a Maven archetype is provided for creating the initial project structure. Furthermore we provide the openengsb-maven-plugin (see ???) (or the etc/scripts/gen-connector.sh script, which wraps the invocation of the maven plug-in) which simplifies the creation of a connector project from the archetype. It should be used for assisted creation of a new connector project.

The newly created connector project should have the exact same structure as the following listing:

-- src
|  -- main
|     -- java
|        -- org
|           -- openengsb
|              -- connector
|                 -- [myconnector]
|                    -- internal
|                    |  -- [MyConnector]Connector.java
|                    |  -- [MyConnector]ConnectorProvider.java
|                    |  -- [MyConnector]InstanceFactory.java
|     -- resources
|        -- OSGI-INF
|           -- blueprint
|              -- [myconnector]-[mydomain]-context.xml
|           -- l10n
|              -- bundle.properties
|              -- bundle_de.properties
|           -- bundle.info
-- pom.xml

The MyServiceConnector class implements the interface of the domain and thus is the communication link between the OpenEngSB and the connected tool. To give the OpenEngSB (and in the long run the end user) enough information on how to configure a connector, the MyServiceInstanceFactory class provides the OpenEngSB with meta information for configuring and functionality for creating and updating a connector instances. The MyServiceConnectorProvider class connects connector instances with the underlying OSGi engine and OpenEngSB infrastructure. It exports instances as OSGi services and adds necessary meta information to each instance. Since the basic functionality is mostly similar for all service managers, the MyServiceConnectorProvider class extends a common base class AbstractConnectorProvider. In addition the AbstractConnectorProvider also persists the configuration of each connector, so that the connector instances can be restored after a system restart.

The blueprint setup in the resources folder contains the setup of the service manager. Additional bean setup and dependency injection can be configured there.

The OpenEngSB has been designed with localization in mind. The Maven Archetype already generates two bundle*.properties files, one for English (bundle.properties) and one for the German (bundle_de.properties) language. Each connector has to provide localization through the properties files for service and attributes text values. This includes localization for names, descriptions, attribute validation, option values and more. For convenience the BundleStrings class is provided on all method calls where text is needed for user representation for a specific locale.

The connector provided is responsible for the integration of the connector into the OpenEngSB infrastructure. The correct definition of this service is critical.

In case it isn't known what a domain is and how it defines the interface and events for connectors, then Section 5.4, “OpenEngSB Tool Domains” is a good starting point.

To get developers started creating a new domain a Maven archetype is provided for creating the initial project structure. The openengsb-maven-plugin (see ???) or the etc/scripts/gen-domain.sh script (which only wraps the invocation of the plug-in) simplifies the creation of a domain.

-- src
|  -- main
|     -- java
|        -- org
|           -- openengsb
|              -- domain
|                 -- [mydomain]
|                    -- [MyDomain]Domain.java
|                    -- [MyDomain]DomainEvents.java
|                    -- [MyDomain]DomainProvider.java
|     -- resources
|        -- OSGI-INF
|           -- blueprint
|              -- [mydomain]-context.xml
|           -- l10n
|              -- bundle.properties
|              -- bundle_de.properties
|           -- bundle.info
-- pom.xml

For information regarding the implementation of connectors for the newly created domain see Chapter 52, How To Create an Internal Connector.

This tutorial shows how to extend the OpenEngSB console.

To read this tutorial and get add a first command should not take more than 10 minutes

This HowTo assumes you already have downloaded the OpenEngSB.

It is not much needed to get the OpenEngSB console started. Just type in a shell "mvn openengsb:provision" or execute the corresponding shell script (etc/scripts/run.sh)

This section describes how to add new commands. The project is located in core/console. To add a new command not much is needed. For a finished example have a look at the class org.openengsb.core.console.OpenEngSBInfo. Here a short description:

        import org.apache.felix.gogo.commands.Command;
        import org.apache.karaf.shell.console.OsgiCommandSupport;

        @Command(scope = "openengsb", name = "info", description = "Prints out some information")
        public class OpenEngSBInfo extends OsgiCommandSupport {

            @Override
            protected Object doExecute() throws Exception {
                System.out.println("Here is the information");
                return null;
            }
        }
      

There is just one single other step which has to be done: Go into the core/console/src/main/resources/OSGI-INF/bluepring/shell-config.xml and add the following lines:

        
        <command-bundle xmlns="http://karaf.apache.org/xmlns/shell/v1.0.0">
          <command name="openengsb/info">
            <action class="org.openengsb.core.console.OpenEngSBInfo"/>
OpenEngSBInfo
        </command-bundle>
      
      

This will lead to following command "openengsb:info". To execute this command, start the OpenengSB console in a shell as described above and type in openengsb:info, this will print out the text "Here is the information"

This tutorial describes exemplary for all connectors the implementation of an email connector. The email connector implements the interface of the Chapter 32, Notification Domain, which is already implemented in the OpenEngSB. Therefore, this tutorial describes the implementation of a connector for an already present domain.

If you are already familiar with the OpenEngSB about 30 minutes. If you are not familiar with the OpenEngSB please read this manual from the start or check the homepage for further information.

Warning: This section is likely to change in the near future, as domains and connectors are currently separated from the rest of the OpenEngSB project. Currently connectors are developed together with the core system.

For information about how to get started as contributor to the OpenEngSB project and how to get the current OpenEngSB source please read the contributor section of the manual: Part VI, “OpenEngSB Contributor Detail Informations”.

As the development of a connector is a recurring task the OpenEngSB developer team has prepared Maven archetypes and useful mojos, which provide support for the initial creation of a connector. A new connector can be created by invoking mvn openengsb:genConnector (or using /etc/scripts/gen-connector.sh)

Go into the directory "/connector" and invoke the mojo from there. It generates the result in the directory from where it is started, therefore it is recommended to run it from the "/connector" directory. You can also run it from a different directory and copy the results into the "/connector" directory. Fill in the following values (if no input is provided the default value is kept):

Domain Name (is domainname): notification
Domain Interface (is NotificationDomain):
Connector Name: email
Version (is 1.1.0-SNAPSHOT):
Project Name (is OpenEngSB :: Connector :: Email):
    

Now the maven archetype is executed. It asks you to confirm the configuration:

groupId: org.openengsb.connector
artifactId: org.openengsb.connector.email
version: 1.1.0-SNAPSHOT
package: org.openengsb.connector.email
connectorName: Email
connectorNameLC: email
domainArtifactId: org.openengsb.domain.notification
domainInterface: NotificationDomain
domainPackage: org.openengsb.domain.notification
name: OpenEngSB :: Connector :: Email
 Y: : y
    

A project named "email" is created with the following structure:

email
-- src
|  -- main
|     -- java
|        -- org
|           -- openengsb
|              -- connector
|                 -- email
|                    -- internal
|                    |  -- EmailConnector.java
|                    |  -- EmailConnectorProvider.java
|                    |  -- EmailInstanceFactory.java
|     -- resources
|        -- OSGI-INF
|           -- blueprint
|              -- email-notification-context.xml
|           -- l10n
|              -- bundle.properties
|              -- bundle_de.properties
|           -- bundle.info
-- pom.xml

All these artifacts will be covered during the implementation of the connector and explained in step 2 of this tutorial.

Let's start with the dependencies. As the email connector will be based upon the javax mail libraries, we need to include dependencies for the OSGi versions of these artifacts into the pom file located at "/provision/pom.xml". So we add this dependency to the dependencies section:

    <dependency>
  <groupId>org.apache.servicemix.bundles</groupId>
  <artifactId>org.apache.servicemix.bundles.javax.mail</artifactId>
  <version>1.4.4</version>
</dependency>
    
    

To configure the connector as part of the OpenEngSB two more things are necessary. At first we have to add the connector to the modules section of its parent pom if it is not already present there. If you have run openengsb:genConnector in the "connector" directory this step should have already been performed automatically for you. To check or manually add the entry, open the file "/connector/pom.xml" and add the new connector to the modules section:

  ...
<modules>
  <module>email</module>
...
</modules>
...
  
  

The second step is necessary to configure Karaf correctly. Please open the file "/assembly/pom.xml" and add the following line:

  ...
<profile>
  <id>release</id>
  ...
     <deployURLs>
       ...
       scan-bundle:mvn:org.openengsb.connector/org.openengsb.connector.email/2.4.0,
       ...
     </deployURLs>
...
  
  

Now you can run the following command in the root folder of the OpenEngSB to create an eclipse project for the new connector:

mvn openengsb:eclipse

Now import the connector project into Eclipse and implement the email service by implementing the classes EmailServiceImpl.java and EmailServiceInstanceFactory.java. We won't go into detail about the actual mail implementation here, so we encapsulated the mailing functionality in a mail abstraction. While the class EmailServiceImpl is responsible for the realization of the domain interface, the factory is responsible for creating instances of the email service and for publishing the meta data necessary to configure an instance of the email service. These two classes are now explained in detail.

package org.openengsb.connector.email.internal;

import org.openengsb.connector.email.internal.abstraction.MailAbstraction;
import org.openengsb.connector.email.internal.abstraction.MailProperties;
import org.openengsb.core.api.AliveState;
import org.openengsb.domain.notification.NotificationDomain;
import org.openengsb.domain.notification.model.Notification;
import org.osgi.framework.ServiceRegistration;

public class EmailServiceImpl implements NotificationDomain {

    private final String id;

    private final MailAbstraction mailAbstraction;
    private ServiceRegistration serviceRegistration;
    private final MailProperties properties;

    public EmailServiceImpl(String id, MailAbstraction mailAbstraction) {
        this.id = id;
        this.mailAbstraction = mailAbstraction;
        properties = mailAbstraction.createMailProperties();
    }

    /**
     * Perform the given notification, which defines message, recipient, subject and 
     * attachments.
     */
    @Override
    public void notify(Notification notification) {
        mailAbstraction.send(properties, notification.getSubject(), notification
                .getMessage(), notification.getRecipient());
    }

    /**
     * return the current state of the service,
     *
     * @see org.openengsb.core.api.AliveState
     */
    @Override
    public AliveState getAliveState() {
        AliveState aliveState = mailAbstraction.getAliveState();
        if (aliveState == null) {
            return AliveState.OFFLINE;
        }
        return aliveState;
    }

    public String getId() {
        return id;
    }

    public ServiceRegistration getServiceRegistration() {
        return serviceRegistration;
    }

    public void setServiceRegistration(ServiceRegistration serviceRegistration) {
        this.serviceRegistration = serviceRegistration;
    }

    public MailProperties getProperties() {
        return properties;
    }
}
  

As you can see, without the mail specific stuff the implementation is quite straight forward. Simply implement the domain interface as well as the getAliveState() method, which is used to query to current status of a tool.

 
package org.openengsb.connector.email.internal;

import java.util.HashMap;
import java.util.Map;

import org.openengsb.connector.email.internal.abstraction.MailAbstraction;
import org.openengsb.core.api.ServiceInstanceFactory;
import org.openengsb.core.api.descriptor.AttributeDefinition;
import org.openengsb.core.api.descriptor.ServiceDescriptor;
import org.openengsb.core.api.validation.MultipleAttributeValidationResult;
import org.openengsb.core.api.validation.MultipleAttributeValidationResultImpl;
import org.openengsb.domain.notification.NotificationDomain;

public class EmailServiceInstanceFactory implements
          ServiceInstanceFactory<NotificationDomain, EmailServiceImpl> {

    private final MailAbstraction mailAbstraction;

    public EmailServiceInstanceFactory(MailAbstraction mailAbstraction) {
        this.mailAbstraction = mailAbstraction;
    }

    private void setAttributesOnNotifier(Map<String, String> attributes, 
            EmailServiceImpl notifier) {

        if (attributes.containsKey("user")) {
            notifier.getProperties().setUser(attributes.get("user"));
        }
        if (attributes.containsKey("password")) {
            notifier.getProperties().setPassword(attributes.get("password"));
        }
        if (attributes.containsKey("prefix")) {
            notifier.getProperties().setPrefix(attributes.get("prefix"));
        }
        if (attributes.containsKey("smtpAuth")) {
            notifier.getProperties().setSmtpAuth(Boolean.parseBoolean(attributes.
                    get("smtpAuth")));
        }
        if (attributes.containsKey("smtpSender")) {
            notifier.getProperties().setSender(attributes.get("smtpSender"));
        }
        if (attributes.containsKey("smtpHost")) {
            notifier.getProperties().setSmtpHost(attributes.get("smtpHost"));
        }
        if (attributes.containsKey("smtpPort")) {
            notifier.getProperties().setSmtpPort(attributes.get("smtpPort"));
        }
    }

    /**
     * Called when the {@link #ServiceDescriptor} for the provided service is needed.
     *
     * The {@code builder} already has the id, service type and implementation type
     * set to defaults.
     */
    @Override
    public ServiceDescriptor getDescriptor(ServiceDescriptor.Builder builder) {
        builder.name("email.name").description("email.description");

        builder
            .attribute(buildAttribute(builder, "user", "username.outputMode", 
                    "username.outputMode.description"))
            .attribute(
                builder.newAttribute().id("password").name("password.outputMode")
                    .description("password.outputMode.description").defaultValue("")
                    .required().asPassword().build())
            .attribute(buildAttribute(builder, "prefix", "prefix.outputMode", 
                    "prefix.outputMode.description"))
            .attribute(
                builder.newAttribute().id("smtpAuth").name("mail.smtp.auth.outputMode")
                    .description("mail.smtp.auth.outputMode.description")
                    .defaultValue("false").asBoolean().build())
            .attribute(
                buildAttribute(builder, "smtpSender", "mail.smtp.sender.outputMode",
                    "mail.smtp.sender.outputMode.description"))
            .attribute(
                buildAttribute(builder, "smtpPort", "mail.smtp.port.outputMode",
                    "mail.smtp.port.outputMode.description"))
            .attribute(
                buildAttribute(builder, "smtpHost", "mail.smtp.host.outputMode",
                    "mail.smtp.host.outputMode.description")).build();

        return builder.build();
    }

    private AttributeDefinition buildAttribute(ServiceDescriptor.Builder builder, 
            String id, String nameId, String descriptionId) {
        return builder.newAttribute().id(id).name(nameId).description(descriptionId)
            .defaultValue("").required().build();

    }
    
    /**
     * Called by the {@link AbstractServiceManager} when updated service attributes for
     * an instance are available. The attributes may only contain changed values and 
     * omit previously set attributes.
     *
     * @param instance the instance to update
     * @param attributes the new service settings
     */
    @Override
    public void updateServiceInstance(EmailServiceImpl instance, Map<String,
            String> attributes) {
        setAttributesOnNotifier(attributes, instance);
    }

    /**
     * The {@link AbstractServiceManager} calls this method each time a new service 
     * instance has to be started.
     *
     * @param id the unique id this service has been assigned.
     * @param attributes the initial service settings
     */
    @Override
    public EmailServiceImpl createServiceInstance(String id, 
            Map<String, String> attributes) {
        EmailServiceImpl notifier = new EmailServiceImpl(id, mailAbstraction);
        setAttributesOnNotifier(attributes, notifier);
        return notifier;
    }

    /**
     * Validates if the service is correct before updating.
     */
    @Override
    public MultipleAttributeValidationResult updateValidation(EmailServiceImpl instance,
            Map<String, String> attributes) {
        return new MultipleAttributeValidationResultImpl(true, 
                new HashMap<String, String>());
    }

    /**
     * Validates if the attributes are correct before creation.
     */
    @Override
    public MultipleAttributeValidationResult createValidation(String id, 
            Map<String, String> attributes) {
        return new MultipleAttributeValidationResultImpl(true, 
                new HashMap<String, String>());
    }
}

  

The factory is more interesting with respect to the OpenEngSB. It is used to create and configure instances of the email service. Furthermore it is responsible for publishing which properties a mail notifier needs to be configured in a proper way. The "getDescriptor" method returns a service descriptor, which is created with the help of a builder. This service descriptor contains the properties a mail notifier needs. In this case things like user password, smtp server and so on. The "updateServiceInstance" method updates an already created instance of the mail service. Basically this means setting the properties, which are provided in the attributes map parameter (see "setAttributesOnNotifier" method). The "createServiceInstance" method is responsible for the creation of a new email service. The methods "updateValidation" and "createValidation" are used to check properties before "updateServiceInstance" or "createServiceInstance" are called. As the mail service does not want to check properties beforehand it simply returns that all values are OK.

The Maven archetype already created the blueprint setup for the email service at src/main/resources/OSGI-INF/blueprint. If properties or constructor arguments are needed for the service factory, they have to be defined in the blueprint setup here. In our case the mail abstraction has to be injected as constructor argument on the creation of the email service factory.

With regards to internationalization it is necessary to add a name and a description for each property used in the service descriptor (see email service factory). The properties files for English and German are also already created by the Maven archetype and can be found at "src/main/resources/OSGI-INF/l10n/". In our case the bundle.properties file contains the following entries:

email.name=Email Notification
email.description=This is a Email Notification Service

username.outputMode = Username
username.outputMode.description = Specifies the username of the email account 

password.outputMode = Password
password.outputMode.description = Password of the specified user

prefix.outputMode = Prefix
prefix.outputMode.description = Subject prefix for all mails sent by this connector

mail.smtp.auth.outputMode = Authentification
mail.smtp.auth.outputMode.description = Specifies if the smtp authentication is on or off

mail.smtp.sender.outputMode = Sender Emailadress
mail.smtp.sender.outputMode.description = Specifies the Emailadress of the sender

mail.smtp.port.outputMode = SMTP Port
mail.smtp.port.outputMode.description = Specifies the Port for the smtp connection

mail.smtp.host.outputMode = SMTP Host
mail.smtp.host.outputMode.description = Specifies the SMTP Hostname
    

As you can see each property is defined with name and description. The same entries can be found in the German properties file (bundle_de.properties) with German names and descriptions.

After implementing and testing your connector locally you can try to start up the OpenEngSB with your new connector. Enter the following commands in the root directory of the OpenEngSB to build and start the OpenEngSB in development mode:

mvn clean install
mvn openengsb:provision
    

Now you can enter "list" into the karaf console to check whether your new connector was installed and started.

Now you can use the OpenEngSB administration WebApp (available at http://localhost:8090/openengsb) to test your new connector. For more information about how to use the WebApp see the How-to section} of the OpenEngSB homepage.

This tutorial explains how to create new supported field types for domain models. What a domain model is, can be read in the user manual in the semantics section(Chapter 6, Semantics in the OpenEngSB). In this section is also explained which fields are supported until now. This task is divided in two subtasks, where the second one is optional. The first subtask provides the functionality that the model is working correctly with the new field type. The second subtask is the possibility that this fields are also saved in the EDB and can be loaded from the EDB.

If you are already familiar with the OpenEngSB the first subtask will take about 45 minutes. The second subtask will take about another 45 minutes. If you are not familiar with the OpenEngSB please read this manual from the start or check the homepage for further information.

For information about how to get started as contributor to the OpenEngSB project and how to get the current OpenEngSB source please read the contributor section of the manual: Part VI, “OpenEngSB Contributor Detail Informations”.

All that have to be done is to add a new converter step to the ModelProxy class in the core common bundle. Step by step:

                public interface ModelEntryConverterStep {
                   /**
                    * Checks if the given object is suitable for converting work when "getOpenEngSBModelObjects" of the proxy is
                    * called. (e.g. an OpenEngSBModel)
                   */
                   boolean matchForGetModelEntries(Object object);
                   /**
                    * Does the converting work for the proxy when "getOpenEngSBModelObjects" is called. (e.g. OpenEngSBModel ->
                    * OpenEngSBModelWrapper)
                   */
                   Object convertForGetModelEntries(Object object);
                   /**
                    * Checks if the given object is suitable for converting work when a getter of the proxy is called. (e.g. an
                    * OpenEngSBModelWrapper)
                   */
                   boolean matchForGetter(Object object);
                   /**
                    * Does the converting work for the proxy when a getter is called. (e.g. OpenEngSBModelWrapper ->
                    * OpenEngSBModel)
                   */
                   Object convertForGetter(Object object);
                }
            

To accomplish this task, the EDB and the EKB has to be extended. WARNING: This step is currently under construction and very likely to be changed soon. As example for this subtask, you can check how it was done with the File object.