Spring integration - spillter handling big lists

In my proejct we have a -spring integration poller which gets bunch of records for every five minutes - sends it to the splitter, which splits the records using the taskexector in multi thread fashion (with pool size as 4) at the output channel and send each record for further processing.

This all working fine. However the issue is with splitter blocking until it completely process the passed list.

Our requirement is if another list is passed to splitter then it needs to process it in parallel without waiting for the first list processing is completed. I tried to put another task executor to input channel of the spillter , but it did not resolve the issue.

How to get the splitter work is parellel for two given input lists to them , each inturn run multithreads for its individual passed lists.

Configuration:

using the spring integration 2.2.0.RELEASE and spring 3.1.1.RELEASE versions

<integration:channel id="splitterChannel"/>
    <integration:channel id="serviceRequestChannel"/>
    <integration:channel id="aggregatorChannel"/>

    <integration:channel id="routerChannel" >
        <integration:dispatcher task-executor="taskExecutor"/>
    </integration:channel>

    <task:executor id="taskExecutor" pool-size="4"/>


   <integration:inbound-channel-adapter
            id="inboundAdapter"
            method="retrieveReadyRecords"
            channel="splitterChannel"
            ref="inboundChannelAdapter" auto-startup="true">
       <integration:poller
                trigger="batchtrigger"/>
    </integration:inbound-channel-adapter>


    <integration:splitter
            input-channel="splitterChannel"
            expression="payload"
            output-channel="routerChannel"/>


  <integration:recipient-list-router input-channel="routerChannel">
        <integration:recipient channel="aggregatorChannel"      selector-expression="!payload.state.equals('Y')"/>
        <integration:recipient channel="serviceRequestChannel"  selector-expression="payload.state.equals('N')"/>
    </integration:recipient-list-router>


--- also further configuration for below service requests sending to aggretor

Answers


You put your ExecutorChannel AFTER the splitter. The poller won't be triggered again until the splitter sends its output to routerChannel which is where the handoff happens.

I suggest you remove the dispatcher from the routerChannel and add the taskExecutor to the poller instead.

That way, the poller does the handoff before the splitter and will be available for the next trigger.

You should also use the latest 2.2.x release (2.2.6).


Need Your Help

How to export a data table as different from the UI in JSF Primefaces?

jsf primefaces

I have a data table which I want to export as different from the UI.

Simple Rails 4 migration throwing a flag on Postgres

ruby-on-rails ruby-on-rails-4

I have a simple migration here for my first Rails 4 project. This is the first migration in my project, on the very first model. I also tried doing: