

Creating Siddhi Applications

Siddhi applications are files that define the Siddhi logic to process the events sent to the Streaming Integrator. They are written in the Siddhi Query Language using the Streaming Integrator Tooling.

A Siddhi file contains the following configurations:

Configuration	Description
Stream	A logical series of events ordered in time with a uniquely identifiable name, and set of defined attributes with specific data types defining its schema.
Source	This consumes data from external sources (such as TCP, Kafka, HTTP, etc) in the form of events, then converts each event (that can be in XML, JSON, TXT, CSV, binary, etc. format) to a Siddhi event, and passes that to a stream for processing.
Sink	This takes events arriving at a stream, maps them to a predefined data format (such as XML, JSON, TXT, CSV, binary, etc), and publishes them to external endpoints (such as E-mail, TCP, Kafka, HTTP, etc).
Executional Element	An executional element can be one of the following: Stateless query: Queries that only consider currently incoming events when generating an output. e.g., filters Stateful query: Queries that consider both currently incoming events as well as past events when generating an output. e.g., windows, sequences, patterns, etc. Partitions: Collections of stream definitions and Siddhi queries separated from each other within a Siddhi application for the purpose of processing events in parallel and in isolation

A Siddhi application can be created from the source view or the design view of the Streaming Integrator Tooling.

Creating a Siddhi application in the source view

To create a Siddhi application via the source view of the Streaming Integrator Tooling, follow the steps below:

1. Start the Streaming Integrator Tooling by navigating to the <SI_TOOLING_HOME>/bin directory and issue one of the following commands:

   • For Windows: tooling.bat

   • For Linux: ./tooling.sh

2. Access the Streaming Integration Tooling via the http://<HOST_NAME>:<TOOLING_PORT>/editor URL.

   Info

   The default URL is http://localhost:9390/editor.

The Streaming Integration Tooling opens as shown below.

1. Click New to start defining a new Siddhi application. A new file opens as shown below.

   

2. Add the following sample Siddhi application to the file.

       @App:name("SweetProductionAnalysis")
   
       @Source(type = 'tcp', context='SweetProductionData', @map(type='binary'))
       define stream SweetProductionStream (name string, amount double);
   
       @sink(type='log', @map(type='json'))
       define stream ProductionAlertStream (name string, amount double);
   
       from SweetProductionStream
       select *
       insert into ProductionAlertStream;

   Info

   Note the following in this Siddhi application

   Configuration	Description
   Stream	This stream contains two stream configurations: SweetProductionStream define stream SweetProductionStream (name string, amount double); This is the input stream that defines the schema based on which events are selected to be processed by the SweetProductionAnalysis Siddhi application. Events received via the source in this application are directed to this stream. ProductionAlertStream define stream ProductionAlertStream (name string, amount double); This is the output stream from which the sink configured in this application takes events to be published as the output.
   Source	@Source(type = 'tcp', context='SweetProductionData', @map(type='binary')) This source configuration has the following sections: @Source(type = ‘tcp’, context='SweetProductionData' This configuration defines tcp as the transport via which events are received to be processed by the SweetProductionAnalysis Siddhi application. @map(type='binary')) This configuration defines the input mapping. In this scenario, Binary Mapper is used which converts input events into binary events and feeds them into siddhi. The source types and map types are available as Siddhi extensions, and you can find via the operator finder as follows: Click the Operator Finder icon to open the Operator Finder. Move the cursor to the location in the Siddhi application where you want to add the source. Search for the required transport type. Once it appears in the search results, click the Add to Source icon on it. Similarly, search for the mapping type you want to include in the source configuration, and add it. The source annotation is now displayed as follows. You can add the other properties as required, and save your changes.
   Sink	@sink(type='log', @map(type='json')) This sink configuration has the following sections: @sink(type='log') This configuration defines log as the transport via which the processed events are published from the ProductionAlertStream output stream. Log sink simply publishes events into the console. @map(type='json')) This configuration defines the output mapping. Events are published with the json mapping type. Json mapper converts the events in the ProductionAlertStream to the Json format. You can select the sink type and the map type from the Operator Finder.
   Executional Elements	from SweetProductionStream select * insert into ProductionAlertStream; This is where the logic of the siddhi app is defined. In this scenario, all the events received in the SweetProductionStream input stream are inserted into the ProductionAlertStream output stream.

3. To save this Siddhi application, click File, and then click Save. By default, Siddhi applications are saved in the <SI_HOME>/wso2/editor/deployment/workspace directory.

4. To export the Siddhi application to your preferred location, click File, and then click Export File.

5. To see a graphical view of the event flow you defined in your Siddhi application, click Design View.

   

   The event flow is displayed as follows.

   

Creating a Siddhi application in the design view

To create a Siddhi application via the design view of the Streaming Integrator Tooling, follow the steps below:

1. Start the Streaming Integrator Tooling by navigating to the <SI_TOOLING_HOME>/bin directory and issue one of the following commands:

   • For Windows: tooling.bat
   • For Linux: ./tooling.sh

   Streaming Integrator Tooling opens as shown below.

2. Click New to start defining a new Siddhi application. A new file opens as shown below.

   

3. To open the design view, click Design View.

4. To define the input stream into which the events to be processed via the Siddhi application should be received, drag and drop the stream icon (shown below) into the grid.

   Once the stream component is added to the grid, move the cursor over it, and then click on the settings icon as shown below.

   

   As as result, the Stream Configuration form opens as follows.

   Fill this form as follows to define a stream named SweetProductionStream with two attributes named name and amount:

   1. In the Name field, enter SweetProductionStream.

   2. In the Attributes table, enter two attributes as follows. You can click +Attribute to add a new row in the table to define a new attribute.

      Attribute Name	Attribute Type
      name	string
      amount	double

   3. Click Submit to save the new stream definition. As a result, the stream is displayed on the grid with the SweetProductionStream label as shown below.

5. To define the output stream to which the processed events need to be directed, drag and drop the stream icon again. Place it after the SweetProductionStream stream. This stream should be named ProductionAlertStream and have the following attributes.

   Attribute Name	Attribute Type
   name	string
   totalProduction	long

6. To add the source from which events are received, drag and drop the source icon (shown below) into the grid. The source is an input to the SweetProductionStream input stream component. Therefore, place this source component to the left of the input stream component in the grid.
   Once you add the source component, draw a line from it to the SweetProductionStream input stream component by dragging the cursor as demonstrated below. Click the settings icon on the source component you added to open the Source Configuration form. Then enter information as follows.

   1. In the Source Type field, select tcp .

   2. For this example, assume that events are received in the binary format. To indicate that events are expected to be converted from this format, select binary in the Map Type field.

   3. To indicate the context, select the context check box and enter SweetProductionData in the field that appears below.

   4. Click Submit.

7. To add a query that defines the execution logic, drag and drop the projection query icon (shown below) to the grid. The query uses the events in the SweetProductionStream input stream as inputs and directs the processed events (which are its output) to the ProductionAlertStream output stream. Therefore, create two connections as demonstrated below.

8. To define the execution logic, move the cursor over the query in the grid, and click on the settings icon that appears. This opens the Query Configuration form. Enter information in it as follows:

   1. Enter a name for the query in the Name field. In this example, let's enter query as the name.
   2. In order to specify how each user defined attribute in the input stream is converted to generate the output events, select User Defined Attributes in the Select field. As a result, the User Defined Attributes table appears. The As column of this table displays the attributes of the output stream. To derive the value for each attribute, enter required expressions/values in the Expression column as explained below.
      1. The value for name can be derived from the input stream without any further processing. Therefore, enter name as the expression for the name attribute.
      2. To derive the value for the totalProduction attribute, the sum of the values for the amount attribute of input events need to be calculated. Therefore, enter the expression as follows to apply the sum() Siddhi function to the amount attribute. sum(amount)
      3. Leave the default values of the Output section unchanged.
   3. Click Submit to save the information.

9. To add a sink to publish the output events that are directed to the ProductionAlertStream output stream, drag and drop the sink icon (shown below) into the grid.
   Draw an arrow from the ProductionAlertStream output stream to the sink component to connect them.

   Click the settings icon on the sink component you added to open the Sink Configuration form. Then enter information as follows.

   

   1. In this example, let's assume that output needs to be generated as logs in the console. To indicate this, select log in the Sink Type field.

   2. In the Map Type field, select the format in which the output must be generated. For this example, let's select json.

   3. Click Submit to save the information.

10. To align the Siddhi components that you have added to the grid, click Edit and then click Auto-Align. As a result, all the components are horizontally aligned as shown below.

    

11. Click Source View. The siddhi application is displayed as follows.

    

12. Click File and then click Save as. The Save to Workspace dialog box appears. In the File Name field, enter SweetProductionAnalysis and click Save.

    

Top