Welcome to MyGrad Discussions!

[rsokl]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[Paul-Pelletier]

Hello,

I just discovered you package and started playing arround. I found out that the followinf input doesn't work :

x = mg.tensor([3,2])
f = np.sum(x**x)
f.backward()
x.grad

whereas the following works

x = mg.tensor([3.0,2.0])
f = np.sum(x**x)
f.backward()
x.grad

that may have something to do with arrays of integers vs arrays of floats e.g this one doesn't work either

x = mg.tensor([int(3.0),int(2.0)])
f = np.sum(x**x)
f.backward()
x.grad

Cheers,
Paul

[rsokl]

Hi Paul, thanks for checking out mygrad!

What is going on here is that integer-dtype tensors are always treated as constants -- mygrad will never compute gradients for them

In [4]: int_tensor = mg.tensor([3,2])

In [5]: int_tensor.constant
Out[5]: True

In [6]: int_tensor.backward()

In [7]: int_tensor.grad is None
Out[7]: True

In [8]: float_tensor = mg.tensor([3.0, 2.0])

In [9]: float_tensor.constant
Out[9]: False

In [10]: float_tensor.backward()

In [11]: float_tensor.grad
Out[11]: array([1., 1.])

This behavior is intended, as it doesn't make sense for integer-valued tensors to be infinitesimally varied.

[omauryIRD]

Hi Ryan,
I just discovered your MyGrad package which is great and does exactly what I need! I'm trying to use it for parameter estimation with the scipy.optimize library but I can't find a way to calculate the hessian matrix (or even better a Hessian-vector product) with MyGrad. Is there a way to do that?
Thanks!
Olivier

[rsokl]

Hi Olivier, I'm glad you're enjoying MyGrad! This library only computes gradients of scalar values. Jax is what you want for Hessians and Hessian vector products. It's quite a bit different than MyGrad in design, but is very powerful and popular.

[omauryIRD]

Thanks so much Ryan for your swift answer, JAX looks fitted for my purpose indeed, I will use it as soon as I manage to install it!