read noise in PCM model

[matifali]

Description

Read noise implementation and documentation are different.

Here g_prog is being used

aihwkit/src/aihwkit/inference/noise/pcm.py

Lines 141 to 142 in 647efaa

	q_s = (0.0088 / ((torch_abs(g_prog) /
	self.g_max) ** 0.65).clamp(min=1e-3)).clamp(max=0.2)

while in the documentation at

aihwkit/docs/source/pcm_inference.rst

Line 157 in 647efaa

Q_s=\min\left(0.0088/g_T^{0.65}, 0.2\right)

g_T is used.

Which one is correct?

I can send a PR if you guide me.

[matifali]

@maljoras

[maljoras]

Hi @matifali ,
indeed, the documentation is correct and there is a slight inaccuracy in the code that the programmed conductance is taken and not the target conductance. The g_target is not given as the input to the function, so it is slightly simpler in the implementation to take just the g_prog. The results will be very similar, however, as the programming noise is typically a couple of percent variation, which will not significantly change the Q_s. But if you want, you could open up a PR to correct for this. The code is indeed not 100% correct.

Thanks!

Best,
Malte

[matifali]

Also, the read noise is applied inside the drift_analog_weights(). So if we are not drifting the weights are we assuming there is no read noise? I mean, if I am using a PCM noise model for inference only with a pre-trained network and I am not drifting the weights. Is there any read_noise being applied to every single inference?

aihwkit/src/aihwkit/inference/noise/pcm.py

Line 123 in 647efaa

def apply_drift_noise_to_conductance(

[maljoras]

If you use the drift analog weights only the long-term and accumulating effects of noise and drift to the weights are modeled. Once you call the drift_analog_weights the weights are fixed for subsequent inference calls. If you want to add shorter-term read-noise that is applied to any mat-vec, then you need to use additionally the rpu_config.forward.w_noise and rpu_config.forward.w_noise_type=WeightNoiseType.PCM_READ setting (see here ). This would model an additional short-term weight noise that is added to the output each time a mat-vec is done with the same characteristics as the 1/f noise in the PCM noise model. Note that in addition there are also out_noise and inp_noise settings available that also effect the MVM noise.