Integration

Modern IT system is a creative combination of multiple subsystems, working in synergy.

 
 
 

API

API (Application Programming Interface) enables the exchange of information between the application and external software. In the case of web applications, most commonly accessed by a web browser (e.g. Firefox or Internet Explorer), API provides an alternative method to download data or perform operations. Recently, it’s become more popular for mobile applications to communicate through a server’s API.

Properly and carefully designed API may be used by other software vendors to create specialized applications, often just for a single marketing campaign.

Our ActiveForms system, which has been available for several years, is an example of application with publicly available API, which is used for communication with many external systems and applications.

Continuous development of this system, in addition to implementation of new features has resulted in API changes. Previous versions of our API remain available and may be used by already integrated systems, eliminating the need for their adaptation to latermodifications.

Besides API implementation, e-point provides documentation loaded with examples, as well as libraries for various programming languages. This assists new users, who are just beginning integration through our API.

WEB SERVICES

OASIS-based web services, along with REST, is undoubtedly among the most popular technologies for system integration. e-point has been using web services for several years in the following configurations:

  • our system provides services in accordance with WSDL file, prepared by our architects,
  • our system provides services in accordance with WSDL file, delivered by the vendor of the system, with which we integrate,
  • our system uses services of other vendors.

Web services require secure communication. This may be achieved in a multitude of ways, which are depending on local constraints, customer policy and the level of criticality of the information processed:

  • securing of communication channel - SSL, VPN,
  • BasicAuth authorization on the level of web server, or client's SSL certificate,
  • implementation of WS-Security subset,
  • UsernameToken Profile and PKI elements: data signing and encryption in accordance with WS-SecurityPolicy 1.3.

The use of web services with advanced WS-Security features, contrary to what might be thought,depends heavily on implementations used by both communicating systems. When implementing solutions for our customers, we use web services to transfer information through JBoss, WebSphere, and JonAS application servers.

REST

Services conforming to REST (Representational State Transfer) guidelines are also web services, but they are “lighter” than WebService. REST does not involve such complex standards and features, as it is in the case of WebService, but provides only a set of general guidelines, which extends the functionality of HTTP 1.1. Due to the ease of implementation and ability to base integration not only on XML, but also on increasingly popular JSON (JavaScript Object Notation) data format, most APIs of our applications are built in accordance to REST.

REST may be used as reference to build complex solutions, which in terms of security, performance, and stability, are comparable to classic web services (Web Services).

e-point REST-based APIs enable authorization, authentication, and data correctness verification. The data transmission channel is encrypted. In the case of Java, we may use JAXB to obtain the level of type control and ease of use, as characterizes web services defined via WSDL.

OTHER DATA EXCHANGE TECHNOLOGIES

Quite often legacy systems do not support contemporary integration technologies. Communication with some systems may be achieved only by the means of older solutions, such as FTP file exchange, or even direct communication with the database.

We began to fully understand the integration with such systems as we developed software for our customers. For example, using FTP file server for integration, we encountered the problem with transactions which were not supported by FTP server. To solve this problem we used a proprietary implementation of FTP server with data warehouse derived from the database.
When integration is based on direct communication with the database, whether our system communicates with remote database or we provide data from our database to external system, access rights have to be as limited as possible. The transaction isolation level must be appropriate for a given database. This is necessary to prevent deadlocks of application threads operating on a database, resulting from the activity of the external system.

SYNCHRONICITY, ASYNCHRONICITY AND QUEUES

Working as system integrator, we have implemented solutions based both on synchronous and asynchronous data exchange.

Most often the asynchronous, queue-based integration should be preferred (in the case of Java it is provided by JMS and message driven beams). This approach helps us to "isolate" integrated systems and prevent them from adversely impacting one another. Such impact could occur when thread, processor, or memory resources become depleted due to the external system being overloaded, or as a result of business use (e.g. during media campaigns or availability of special offers), or because of a software error.

Using queues, we control the speed of message processing, releasing for integration purposes. Only such resources as can be released without negative impact on the experience of our system end-users.
Using synchronous integration when necessary, we pay particular attention to proper setting of time limits for specific operations and compensation of possible errors.

Every model of integration should include monitoring systems which enables administrators to react in the case of errors and abnormal conditions, resulting for example from increased number of messages or excessively long processing.
In our implementations, applications always provide information about their results to monitoring systems. Information is made available via JMX or files in the format that may be processed by monitoring system (e.g. Nagios or Zabbix).

BATCH JOBS

Another key to integration consists of batch jobs, triggered by external systems, or which run periodically. They may be performed with different frequency, every minute, or every month. We have found that the best way to run such tasks is to use "cron" system service. Our batch job tools prevent overlapping of jobs when preceding task takes longer than expected. They also monitor job execution time and allow execution to be terminated if it extends too long.

In the case of OLTP systems, however, we split large tasks into smaller units, while maintaining sequence and transactional character of operations. This allows us to limit the number of database locks, which could limit system availability to end users.

BROWSER BASED INTEGRATION

Another method of integration uses web browser to communicate with the source system, as opposed to direct communication between two applications. This method is most frequently used for SSO authentication (Single Sign-On). If domains are compatible, we use cookies or generate unique tokens, with additional communication between servers in the case of OAuth schemes.

INTEGRATION FOR AUTHORISATION PURPOSES

Corporate customers often have their own user registers and require their employees to be able to use the same logins and passwords which are used in the case of other systems. For this purpose we often integrate our applications with LDAP or ActiveDirectory servers in customer network.

TEST ENVIRONMENTS

Another challenge in the management of complex systems is posed by application test environments. Effective and valuable application tests scenarios must take into account the integration with external systems. It is obvious that application test environments must be connected with equivalents of external systems. Errors in configuration of such environments, may cause serious business errors, such as placement of a real order, requesting a wire transfer, or erroneous e-mailing of information to end user address.

e-point manages such complex configurations using the versioning system of management, in which every configuration of integration is assigned to a specific system instance, either to production environment or to one of its test environments.
 

 
 
 

Any questions?

 
 
 
Jarosław Błąd
 
Jarosław Błąd

Director of Development Department

jaroslaw.blad at e-point dot pl