IsGrounded Semantic Kernel not working

[kaygu]

Hi,
I tried using the IsGrounded function in the Orchestrator but it does not seem to work for me. If I ask in the chatbox an unrelated question to my data. For example "What is the color of a banana?". GPT will respond something like "The color of a banana is yellow.". And IsGrounded will return "yes" even though my data has no mention of bananas or colors.
Has anyone else experienced the same behavior? Should I change my skprompt.txt file for IsGrounded? I'm not familiar with Semantic Kernel