Why not multithreading?

[ianni67]

I would suggest to change the socket communication to multithread processing, with a thread that computes the mask and the other that composes the mask with the background.
The overall computation speed and overload should improve dramatically.

[fiblan]

I could use multithreading if the two tasks were internal to the same process. But even then "compose" thread should still wait for the outcome of the "compute mask" thread, unless I calculate the "previous" mask... but I can improve socket communication with a more performant protocol as zerorpc (it builds on top of ZeroMQ and MessagePack), instead of bloated http protocol.

[ianni67]

Yes, indeed I meant a single process with two threads. This would imply porting the bodypix process to python. Not the easiest task, indeed.
Compose would not need to wait for compute_mask to deliver the updated mask: the synchronization between the processes would be non-blocking, and in case the new mask is not available yet, the previous one would be used.
The speedup obtained by multithreading would probably strongly reduce the probability of such a worst case.