Add Alexnet Example

[dcslin]

No description provided.

[chrishkchris]

Here is a test on alexnet using cifar10:

(singapy36) dcsysh@panda11:~/singa/examples/cnn$ python train.py alexnet cifar10
Starting Epoch 0:
Training loss = 1611.267456, training accuracy = 0.225992
Evaluation accuracy = 0.307091, Elapsed Time = 69.078020s
Starting Epoch 1:
Training loss = 1349.916016, training accuracy = 0.356514
Evaluation accuracy = 0.385917, Elapsed Time = 69.427504s
Starting Epoch 2:
Training loss = 1189.298340, training accuracy = 0.437600
Evaluation accuracy = 0.456430, Elapsed Time = 69.476516s
Starting Epoch 3:
Training loss = 1059.998901, training accuracy = 0.505862
Evaluation accuracy = 0.508614, Elapsed Time = 69.441947s
Starting Epoch 4:
Training loss = 957.258301, training accuracy = 0.561660
Evaluation accuracy = 0.550280, Elapsed Time = 69.805173s
Starting Epoch 5:
Training loss = 870.051025, training accuracy = 0.603773
Evaluation accuracy = 0.553586, Elapsed Time = 69.677023s
Starting Epoch 6:
Training loss = 801.625977, training accuracy = 0.639165
Evaluation accuracy = 0.577123, Elapsed Time = 69.339764s
Starting Epoch 7:
Training loss = 732.376465, training accuracy = 0.670314
Evaluation accuracy = 0.602364, Elapsed Time = 69.410357s
Starting Epoch 8:
Training loss = 676.386963, training accuracy = 0.698183
Evaluation accuracy = 0.660156, Elapsed Time = 69.445957s
Starting Epoch 9:
Training loss = 635.153076, training accuracy = 0.717650
Evaluation accuracy = 0.669772, Elapsed Time = 69.489095s

[chrishkchris]

One point to notice is that:

Currently in SINGA, ReLU is stateless (except we record the input for backward propagation), so we can just use y = autograd.relu(y) without manually initializing a ReLU python object.

On the other hand, if we initialize a ReLU instance manually, we have to the take the element [0] following https://github.com/apache/singa/blob/master/python/singa/autograd.py#L448

All in all, personally I suggest we directly use y = autograd.relu(y) instead

[dcslin]

not able to reproduce issue
#681

thus tried to implement alexnet and it works fine

maybe we should define issue reporting format to better undertanding these kind of the issues:
example of issue reporting format:
https://github.com/pytorch/pytorch/issues/new?template=bug-report.md

[chrishkchris]

I recommend this one is ready for merge, it is good to have Alexnet for research purpose

[chrishkchris]

@dcslin
I suggest you add an line in Alexnet.py

# the code is modified from
# https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py

Please consider follow this example: https://github.com/apache/singa/blob/master/examples/cnn/model/resnet.py#L21