Simulation versus experimentally derived data

[actionpotential]

Hello, I've modified your Hodgkin Huxley example to run SBI on our (NeuroSim Lab at SUNY Downstate) detailed Pyramidal Tract neuron running in NEURON / NetPyNE. Posterior samples based on stats from simulated Vm samples rock! However, when I try to do the same using actual experimental data (matched sampling rate, etc) I haven't had any luck.

Over the past couple of weeks I've been drilling into the stats calculations to make sure they are the same for both sim and exp voltage traces (would be happy to show).

My questions:

1. is my assumption correct that I should be able to use experimentally derived traces in this way to extract posterior samples?
2. in your collective experiences, what is the best approach for me to try moving forward?

Below is a pic of a sim-based posterior example followed by an experiment-based posterior which is followed by another sim-based posterior example. You can see that samples follow sim-based characteristics. In contrast, exp-based samples do not.

[michaeldeistler]

Hi there! Often, when things work well on simulated data but poorly on experimental data, you have some mismatch between simulation results and experimental results. This can be very tricky to overcome because "all models are wrong".

I recommend you first read this (in particular, I recommend plotting the histograms described in section 1.6):
https://www.mackelab.org/sbi/tutorial/10_crafting_summary_statistics/

Another option might be to add some Gaussian noise to your summary statistics (which makes the posteriors a bit broader, but (at least in our experience) more reliable).

Hope this helps!

Michael