Multiple processors

[rafael844]

Hi, we have two processors, 20 cores, 40 logical, each. But when we process with iped it only get one, putting 40 working process. I know its something with java, but you guys know how to encrease it? Im trying -XX:ActiveProcessorCount=2 parameter, but no success.

Thanks.

[lfcnassif]

I have seen this before. I also have two physical processors in my machine and here it works. As a workaround, you can go to LocalConfig.txt and fix numThreads = 80. You will need a good amount of RAM to feed to 80 worker threads (and other auxiliary threads and processes). Possibly your processing could be IO bound and it may not use all processing power anyway.

[wladimirleite]

I think this problem happens when there are more than 64 logical processors, and then "processor groups" are created, making only one group visible to the JVM.
Not sure if there is a simple solution...

Setting more threads may not work properly as only part of the logical processors are visible to the JVM.
When some external process is running, things would be fine (all resources would be used), but when only "internal" threads (of IPED's process) are running, a single group will be used (in the case @rafael844 described, 40 logical processors). So setting 80 workers may be worse than using 40 (unless there is a solution, or external processes dominate the CPU usage).

@lfcnassif, how many logical processors do you have in total?

[lfcnassif]

Thanks @tc-wleite. Just 48. We could use the recently added oshi-core-java11 library to count the number of logical processors correctly, but if JVM just sees 1 processor group, that would be useless...

Maybe for those highly multiprocessed machines it would be better to run 2 (or more) different IPED instances at the same time, if you have available memory and if your IO devices can handle all those parallel requests.

[wladimirleite]

I had this problem a few years ago, using a large Amazon EC2 instance.
I am not sure if there is a simple solution nowadays.

[rafael844]

Thank you, I Will try this tomorrow at work and see what happens. We have 512GB RAM.

[rafael844]

Thank you all, with numThreads = 80 it worked!