Welcome to Discussions on Forward Learning!

[amassivek]

Welcome!

We use Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions about\
  • development of this library
  • research on forward learning
• Share ideas on forward learning
• Engage with other community members.
• Post with the intent of promoting communication.

[lollopes]

Hi amassivek,

I am a PhD student at TU/e, working on re-implementing Signal-Propagation for a simple MLP trained on MNIST and FMNIST. My ultimate goal is to compare different Forward-Only algorithms in terms of computational efficiency for hardware implementations.

I am having some trouble understanding the local loss used to find the updates of the synaptic matrices. In the case of a simple MNIST classification task, how would this loss look? From Equation 9, it seems that the local loss is a cross-entropy (CE) loss between a metric for the distance between the target activations (t_i) and input activations (h_i), like the dot product or L2 distance, and a reconstructed target (y*).

I am struggling to understand how the reconstructed target (y*) is implemented. Would you mind helping me clarify this?

Also, may I ask you what is the difference of the loss indicated in the paper and the one explained in your website. In the latter, it seems that the loss can simply be the L2 loss between the target activation(t_i) and input activations(h_i) without any CE or reconstructed target.

If I try to use a simple L2 loss I can get only to 92% accuracy on the MNIST dataset with one hidden layer of 256 neurons.

[amassivek]

Hello Lorenzo (@lollopes), your email reached me. I followed up with you. In addition, link me to your code. +Adam