Conv2D #37
Conv2D #37
Conversation
|
@pavanky Added the rest of the Conv2D layer - although it still isn't batched. I need to find a way to confirm it is working properly. I added some more gradients of af functions. |
src/autograd/Functions.cpp
Outdated
| int o2 = a.dims()[2]; | ||
| int o3 = a.dims()[3]; | ||
| printf("Orig: %d,%d,%d,%d -- New: %d,%d,%d,%d",o0,o1,o2,o3,d0,d1,d2,d3); | ||
| } |
There was a problem hiding this comment.
Remove this. Besides you can call a.dims(i);
src/autograd/Functions.cpp
Outdated
| dim4 d = input.array().dims(); | ||
| array res = unwrap(input.array(), wx, wy, sx, sy, px, py); | ||
| int params[] = {wx, wy, sx, sy, px, py, (int)d[0], (int)d[1]}; | ||
| auto tmp = Variable(array(8, params), false); |
There was a problem hiding this comment.
We need a better way to handle this.
There was a problem hiding this comment.
May be check if you can infer the parameters from having access to both input and grad_output
There was a problem hiding this comment.
OK capturing the parameters in the lambda works. Instead of creating tmp, just capture the necessary input params.
There was a problem hiding this comment.
What do you mean by "just capture the necessary input params?"
We can't change the signature of grad_func, right?
There was a problem hiding this comment.
The [] part of the lambda need not be empty. You can specify what variables from the current scope are seen inside the lambda. http://en.cppreference.com/w/cpp/language/lambda#Lambda_capture
This does not change the signature of grad_func.
src/autograd/Functions.cpp
Outdated
| inputs[0].addGrad(moddims(grad_output, p[0], p[1], p[2], p[3])); | ||
| }; | ||
| return Variable(res, {input, tmp}, grad_func); | ||
| } |
There was a problem hiding this comment.
Ok this can be deleted after rebasing from master.
src/autograd/Functions.cpp
Outdated
| inputs[0].addGrad(moddims(grad_output, p[0], p[1], p[2], p[3])); | ||
| }; | ||
| return Variable(res, {input, tmp}, grad_func); | ||
| } |
There was a problem hiding this comment.
Ok this can be deleted after rebasing from master.
src/autograd/Functions.cpp
Outdated
| int *r = inputs[1].array().host<int>(); | ||
| inputs[0].addGrad(reorder(grad_output, r[0], r[1], r[2], r[3])); | ||
| }; | ||
| return Variable(res, {input, reverse}, grad_func); |
src/autograd/Functions.cpp
Outdated
| }; | ||
| return Variable(res, {input, tmp}, grad_func); | ||
| } | ||
|
|
src/autograd/Functions.cpp
Outdated
|
|
||
| int tmp[] = {wx, wy, sx, sy, px, py, c_i, x_o, y_o}; | ||
| auto params = Variable(array(9, tmp), false); | ||
|
|
src/autograd/Functions.cpp
Outdated
|
|
||
| auto d = matmulTN(inputs[2],grad_out_reshape); | ||
| inputs[1].addGrad(moddims(d, p[0], p[1], p[6], d.array().dims()[1])); | ||
|
|
There was a problem hiding this comment.
It should loop over the input images -- still need to do a gradient for join. I'll add that. But the c++ api can only join 4 arrays at a time.
| { | ||
| auto w = nn::weight(wx, wy, n_in, n_out, spread); | ||
| if (bias) { | ||
| auto b = nn::weight(1, 1, n_out, 1, spread); |
There was a problem hiding this comment.
Use the new types. I think the suggested version is lecunNormal.
other usefull gradients are included
Helper functions added - wrap, unwrap, moddims, reorder
|
@pavanky This pr is kind of a mess now. I think I'll abandon this one and move changes to separate commits. One with extra functions and one with conv2d layer. And a third with the alexnet/example.cpp |
|
@plavin sounds good. |
The autograd part still needs to be written.
I also need to be able to join the outputs of multiple inputs, but I'm having trouble getting join to work. This is needed to do batched processing.
Currently arrayfire can join up to 10 arrays at a time. This may or may not be an acceptable batch size. A better solution would be to implement 3d and 4d matmul in arrayfire.