Middleware4j Example Configuration

From Commander4j
Revision as of 17:06, 23 August 2024 by Dgarratt (talk | contribs)

The middlware application is predominatly a backgroud service which does not have a user interface. You can run the application and see a status display showing the names of the maps and the number of messages processed - but the actual configuration is done via xml and xsl files.

This page will show the basic set of files required to perform a simple message transformation. In this example we will take a flat ascii text file and parse it so that 2 of the columns of date are combined and put into an xml document.

The config.xml file shown below is found in the <install folder>/xml folder. This example contains only 1 map (transformation) but there is no limit and you can configure a single instance of the program to perform multiple operations which run concurrenly in different background tasks (threads).

Although this example converts from plain text to xml there are many other options which will be covered elsewhere in the wiki.

Configuration

config.xml 

<?xml version="1.0" encoding="UTF-8"?>
<config description="c4jMiddleware Example">
   <logPath/>
   <XSLTPath/>
   <logArchiveRetentionDays>3</logArchiveRetentionDays>
   <retryOpenFileCount>3</retryOpenFileCount>
   <retryOpenFileDelay>1000</retryOpenFileDelay>
   <enableEmailNotifications>true</enableEmailNotifications>
   <statusReportTime>09:00:00</statusReportTime>
   <map id="demo01" enabled="Y" description="GEN ASCII File Fixed Column to XML">
       <input id="in1" description="Read ASCII Flat File">
           <type>ASCII</type>
           <path>./interface/input/demo01 (GEN ASCII File Fixed Column)</path>
           <mask>txt</mask>
           <inputPattern>1-4,18-21</inputPattern>
           <pollingInterval>1000</pollingInterval>
           <XSLT>ASCII_to_XML_Example.xsl</XSLT>
       </input>
       <output id="out1" enabled="Y" description="Write XML">
           <path>./interface/output/demo01 (GEN XML)</path>
           <type>XML</type>
           <XSLT/>
       </output>
   </map>
</config>

Header

Element Value Description
logpath Blank or Path Specifies where log files are saved - leave blank for default.
XSLTPath Blank or Path Specified where XSTL (Style Sheets) used for xml transformations are located.
logArchiveRetentionDays nnn Number of days to keep log files before automatically deleting them.
retryOpenFileCount nnn How many times should the program attempt to gain exclusive access to a file before returning an error.
retryOpenFileDelay nnnn How long to wait before retrying a file operation in milliseconds.
enableEmailNotifications true or false Enable or disable email notifications. Refer to EMAIL Connector

Map

Element Value Description
id id Simple ID which you want to use for the Map.
enabled Y or N Simple way to enable or disable a map without having to delete it. Middleware needs to be restarted after changing.
description string Description of what the map does.

Input Connector

Element Value Description
id id Any single map can only have 1 input but multiple outputs. Suggest 'in1' for input id.
description string Meaningfull description of what the input is.
type ASCII In this example we define the input type to be ASCII aka Flat File.
path string Path to input folder. Can be a network share providing middleware has permissions.
mask txt File extension used to filter files for processing. If blank each type will use a default mask.
inputPattern nn-nn,nn-nn This setting is only appropriate for the ASCII type. It consists of column start and stop positions. This example defines 2 columns.
pollingInterval nnnn Frequency in milliseconds to check for input files.
XLST string Filename of the Style Sheet to use. Only the filename. Path will default. In some scenarios this can be blank.

Output Connector

Element Value Description
id id A map can have more than one output and each output must have a unique id.
description string Meaningfull description of what the output is.
path string Path to output folder. Can be a network share providing middleware has permissions.
type ASCII In this example we define the output type to be XML.
XLST string Filename of the Style Sheet to use. Only the filename. Path will default. In some scenarios this can be blank.

XSLT Style Sheet

ASCII_to_XML_Example.xsl 

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
   xmlns:xs="http://www.w3.org/2001/XMLSchema"
   exclude-result-prefixes="xs"
   version="2.0">
   <xsl:output encoding="UTF-8" indent='yes' method="xml" />
   <xsl:template match='data'>
       <test>
           <xsl:apply-templates select="row"/>
       </test>
   </xsl:template>
   <xsl:template match='row'>
           <barcode><xsl:value-of select='col[1]'/>-<xsl:value-of select='col[2]'/></barcode>     
   </xsl:template>     
</xsl:stylesheet>

The above is the XSLT style sheet that we are going to use to define how the output data should be written. Don't worry about the inner workings at this point. The middleware java code is going to do some of the work and read the ASCII data into memory and represent it as XML. This style sheet is then applied to that temporary xml to generate the output file. 

<xsl:template match='data'>

This line is used to select the intermediate in memory version of the data. 

<test>
     <xsl:apply-templates select="row"/>
</test>

This creates the root element of the output which we are going to call "test" and then calls a template for each input row. 

<xsl:template match='row'>
     <barcode><xsl:value-of select='col[1]'/>-<xsl:value-of select='col[2]'/></barcode>
</xsl:template>

If you recall from the input definition we are extracting 2 colums of data from the input file which were defined by the `<inputPattern>` This statement combines the values of the columns in memory and puts a - delimiter between them. As the input map only read the 2 columns into memory they can be referenced as [1] and [2] in this step.

Email

email.xml 

<?xml version="1.0" encoding="UTF-8"?>
<emailSettings>
   <configuration>
       <property name="mail.smtp.auth" value="true"/>
       <property name="mail.smtp.starttls.enable" value="true"/>
       <property name="mail.smtp.host" value="smtp.gmail.com"/>
       <property name="mail.smtp.socketFactory.port" value="465"/>
       <property name="mail.smtp.user" value="email@gmail.com"/>
       <property name="mail.smtp.password" value="password"/>
       <property name="mail.smtp.socketFactory.class" value="javax.net.ssl.SSLSocketFactory"/>
       <property name="mail.smtp.auth" value="true"/>
       <property name="mail.smtp.port" value="465"/>
       <property name="mail.smtp.from" value="email@gmail.com"/>
       <property name="mail.debug" value="true"/>
   </configuration1>
   <distributionList id="Monitor" enabled="Y" maxFrequencyMins="5">
       <toAddressList>dummy1@gmail.com,dummy2@gmail.com</toAddressList>
   </distributionList>
   <distributionList id="Error" enabled="Y" maxFrequencyMins="5">
       <toAddressList>dummy1@gmail.com,dummy2@gmail.com</toAddressList>
   </distributionList>
   <distributionList id="Excel" enabled="Y" maxFrequencyMins="5">
       <toAddressList>dummy1@gmail.com,dummy2@gmail.com</toAddressList>
   </distributionList>
   <distributionList id="PDF" enabled="Y" maxFrequencyMins="0" description="Email PDF">
       <toAddressList>dummy1@gmail.com,dummy2@gmail.com</toAddressList>
   </distributionList>
</emailSettings>

Example Input

1232     ABCB    5676    EFGF
1233     ABCC    5677    EFGG
1234     ABCD    5678    EFGH
1235     ABCE    5679    EFGI
1236     ABCF    5680    EFGJ
1237     ABCG    5681    EFGK
1238     ABCH    5682    EFGL
1239     ABCI    5683    EFGM
1240     ABCJ    5684    EFGN
1241     ABCK    5685    EFGO

In the above example input file which has no purspose/meaning other than to illustrate how the middleware works, you can see 4 columns of data. In this example we are trying to extract the columns which contain numeric data, columns 1 and 3, and write the output to a xml file.

Example Output

<?xml version="1.0" encoding="UTF-8"?>
<test>
  <barcode>1232-5676</barcode>
  <barcode>1233-5677</barcode>
  <barcode>1234-5678</barcode>
  <barcode>1235-5679</barcode>
  <barcode>1236-5680</barcode>
  <barcode>1237-5681</barcode>
  <barcode>1238-5682</barcode>
  <barcode>1239-5683</barcode>
  <barcode>1240-5684</barcode>
  <barcode>1241-5685</barcode>
</test>

The above shows the output from the middleware transformation. Columns 1 and 3 of the flat file have been extracted and then combined with a "-" between them before writing to the output. Again - there is no significant purpose or meaning to this example, it just illustrates what can be done.

Note that there are more than ASCII and XML types and you can change both input and output types.

Logs and Debug information.

When a file is processed by middleware a number of log files will be created which show the intermediate stages for the message processing. The filenames for these log messages are composed of the date and time followed by the message type and original input filename. 

2024_04_09_18_46_46_700 INPUT_BACKUP_ASCII GEN ASCII File Fixed Column 1.txt
2024_04_09_18_46_46_716 INPUT_IMPORTED_ASCII_in1_GEN ASCII File Fixed Column 1.txt.xml
2024_04_09_18_46_46_757 INPUT_TRANSFORMED_ASCII_in1_GEN ASCII File Fixed Column 1.txt.xml

The above directory listing shows 3 files related to the processing of this sample message.

2024-04-09-18-46-46-700 INPUT-BACKUP-ASCII GEN ASCII File Fixed Column 1.txt
is a unmodified backup of the original input file.

2024-04-09-18-46-46-716 INPUT-IMPORTED-ASCII-in1-GEN ASCII File Fixed Column 1.txt.xml
shows the intermediate stage of the file in which the flat ascii text file has been converted into a XML file. This version is used as input to the XSLT Style Sheet.

{2024-04-09-18-46-46-757 INPUT-TRANSFORMED-ASCII-in1-GEN ASCII File Fixed Column 1.txt.xml
is a copy of the ouput file generated.

Intermediate Stage

    • 2024_04_09_18_46_46_716 INPUT_IMPORTED_ASCII_in1_GEN ASCII File Fixed Column 1.txt.xml**
<?xml version="1.0" encoding="UTF-8"?>

   <row id="1">
       <col id="1">1232</col>
       <col id="2">5676</col>
   </row>
   <row id="2">
       <col id="1">1233</col>
       <col id="2">5677</col>
   </row>
   <row id="3">
       <col id="1">1234</col>
       <col id="2">5678</col>
   </row>
   <row id="4">
       <col id="1">1235</col>
       <col id="2">5679</col>
   </row>
   <row id="5">
       <col id="1">1236</col>
       <col id="2">5680</col>
   </row>
   <row id="6">
       <col id="1">1237</col>
       <col id="2">5681</col>
   </row>
   <row id="7">
       <col id="1">1238</col>
       <col id="2">5682</col>
   </row>
   <row id="8">
       <col id="1">1239</col>
       <col id="2">5683</col>
   </row>
   <row id="9">
       <col id="1">1240</col>
       <col id="2">5684</col>
   </row>
   <row id="10">
       <col id="1">1241</col>
       <col id="2">5685</col>
   </row>
Retrieved from ‘https://wiki.commander4j.com/index.php?title=Middleware4j_Example_Configuration&oldid=975’