External Servers Sync using Apache NIFI

[keymane]

Currently the tomcat web container that we deploy OpenSRP server to is overworked. It handles user requests, schedule alert generation and periodic sync to external server. These servers are OpenMRS, DHIS2, OpenLMIS and RapidPro.

To scale we need to solve this problem.

One way to solve this is to have two tomcat instances, one acts as a REST API(Rest Api Processing) and another handles External(Third party) services. This could work but tomcat is built to handle client requests (multiple threads) and may not be the best option.

The other way is to have a broker between OpenSRP server and these external services. Apache NIFI can be that broker. It's able to pull data periodically, process it and push it to another service. We've used this in mVacc (OpenSRP RapidPro Integration) and OpenLMIS.

This feature involves creating Apache NIFI flows for syncing between OpenSRP server and the external servers.