Skip to content

Latest commit

 

History

History
226 lines (186 loc) · 13.3 KB

README.md

File metadata and controls

226 lines (186 loc) · 13.3 KB
Raw

بسم الله الرحمن الرحیم
پیاده سازی پایتورچ سیمپل نت

SimpleNet in Pytorch

SimpleNetV1 architecture implementation in Pytorch

Lets Keep it simple, Using simple architectures to outperform deeper and more complex architectures (2016).

GitHub Logo

This is the pytorch implementation of our architecture SimpleNetV1(2016) .
Pytorch is different from caffe in several sections, and this made it a bit harder to have the architecture properly ported especially since I'm a complete newbie in Pytorch. However, thanks to this great work, I could easily focus on the model and port the architecture and hopefully achieve my reported results in Caffe and also exceed them as well!

The pytorch implementation is also very effieicent and the whole model takes only 1239MB with the batch size of 64! (compare this to other architectures such as ResNet,WRN, DenseNet which a 800K model takes more than 6G of vram!)

Update History:

-- 2023 May 13:
  -- Add support for torch hub.
-- 2023 Apr 14:
  -- update benchmark results.
-- 2023 Apr 13:
  -- new weights for the removed paddings for 1x1 conv layers.
  -- some minor fixes
-- 2023 Feb 12: 
  -- re-structured the repository, moving the old implementation into new directory named 'Cifar` and imagenet into its respective directory
  -- updated the old implementation to work with latest version of pytorch.
  -- updated the imagenet scripts/models compatible with timm and a separate version for pure pytorch uscases
  -- updated pretrained models with the latest results 
-- 2022: Adding initial ImageNet models   
-- 2018: Initial Pytorch implementation (for CIFAR10/100/MNIST/SVHN datasets)    
-- 2016: Initial model release for caffe

The original Caffe implementation can be found here : Original Caffe implementation - 2016

ImageNet Result:

Model #Params ImageNet ImageNet-Real-Labels
simplenetv1_9m_m2(36.3 MB) 9.5m 74.23 / 91.748 81.22 / 94.756
simplenetv1_5m_m2(22 MB) 5.7m 72.03 / 90.324 79.328/ 93.714
simplenetv1_small_m2_075(12.6 MB) 3m 68.506/ 88.15 76.283/ 92.02
simplenetv1_small_m2_05(5.78 MB) 1.5m 61.67 / 83.488 69.31 / 88.195

SimpleNet performs very decently, it outperforms VGGNet, variants of ResNet and MobileNets(1-3)
and its pretty fast as well! and its all using plain old CNN!.

Here's an example of benchmark run on small variants of simplenet and some other known architectures such as mobilenets.
Small variants of simplenet consistently achieve high performance/accuracy:

model samples_per_sec param_count top1 top5
simplenetv1_small_m1_05 3100.26 1.51 61.122 82.988
mobilenetv3_small_050 3082.85 1.59 57.89 80.194
lcnet_050 2713.02 1.88 63.1 84.382
simplenetv1_small_m2_05 2536.16 1.51 61.67 83.488
mobilenetv3_small_075 1793.42 2.04 65.242 85.438
tf_mobilenetv3_small_075 1689.53 2.04 65.714 86.134
simplenetv1_small_m1_075 1626.87 3.29 67.784 87.718
tf_mobilenetv3_small_minimal_100 1316.91 2.04 62.908 84.234
simplenetv1_small_m2_075 1313.6 3.29 68.506 88.15
mobilenetv3_small_100 1261.09 2.54 67.656 87.634
tf_mobilenetv3_small_100 1213.03 2.54 67.924 87.664
mnasnet_small 1089.33 2.03 66.206 86.508
mobilenetv2_050 857.66 1.97 65.942 86.082
dla46_c 537.08 1.3 64.866 86.294
dla46x_c 323.03 1.07 65.97 86.98
dla60x_c 301.71 1.32 67.892 88.426

and this is a sample for larger models: simplenet variants outperform many newer architecures.

model samples_per_sec param_count top1 top5
simplenetv1_small_m1_075 2893.91 3.29 67.784 87.718
simplenetv1_small_m2_075 2478.41 3.29 68.506 88.15
vit_tiny_r_s16_p8_224 2337.23 6.34 71.792 90.822
simplenetv1_5m_m1 2105.06 5.75 71.548 89.94
simplenetv1_5m_m2 1754.25 5.75 72.03 90.324
resnet18 1750.38 11.69 69.744 89.082
regnetx_006 1620.25 6.2 73.86 91.672
mobilenetv3_large_100 1491.86 5.48 75.766 92.544
tf_mobilenetv3_large_minimal_100 1476.29 3.92 72.25 90.63
tf_mobilenetv3_large_075 1474.77 3.99 73.436 91.344
ghostnet_100 1390.19 5.18 73.974 91.46
tinynet_b 1345.82 3.73 74.976 92.184
tf_mobilenetv3_large_100 1325.06 5.48 75.518 92.604
mnasnet_100 1183.69 4.38 74.658 92.112
mobilenetv2_100 1101.58 3.5 72.97 91.02
simplenetv1_9m_m1 1048.91 9.51 73.792 91.486
resnet34 1030.4 21.8 75.114 92.284
deit_tiny_patch16_224 990.85 5.72 72.172 91.114
efficientnet_lite0 977.76 4.65 75.476 92.512
simplenetv1_9m_m2 900.45 9.51 74.23 91.748
tf_efficientnet_lite0 876.66 4.65 74.832 92.174
dla34 834.35 15.74 74.62 92.072
mobilenetv2_110d 824.4 4.52 75.038 92.184
resnet26 771.1 16 75.3 92.578
repvgg_b0 751.01 15.82 75.16 92.418
crossvit_9_240 606.2 8.55 73.96 91.968
vgg11 576.32 132.86 69.028 88.626
vit_base_patch32_224_sam 561.99 88.22 73.694 91.01
vgg11_bn 504.29 132.87 70.36 89.802
densenet121 435.3 7.98 75.584 92.652
vgg13 363.69 133.05 69.926 89.246
vgg13_bn 315.85 133.05 71.594 90.376
vgg16 302.84 138.36 71.59 90.382
vgg16_bn 265.99 138.37 73.35 91.504
vgg19 259.82 143.67 72.366 90.87
vgg19_bn 229.77 143.68 74.214 91.848

Benchmark was done using a GTX1080 on Pytorch 1.11 with fp32, nchw, batchsize of 256, input size = 224x224x3.
For all benchmark results look here

-- The models pretrained weights (pytorch, onnx, jit) can be found in Release section

CIFAR10/100 Results achieved using this implementation :

Dataset Accuracy
CIFAR10 95.51
CIFAR100 78.37

CIFAR10/100 top results (2016):

Method #Params CIFAR10 CIFAR100
VGGNet(16L) /Enhanced 138m 91.4 / 92.45 -
ResNet-110L / 1202L * 1.7/10.2m 93.57 / 92.07 74.84/72.18
SD-110L / 1202L 1.7/10.2m 94.77 / 95.09 75.42 / -
WRN-(16/8)/(28/10) 11/36m 95.19 / 95.83 77.11/79.5
Highway Network N/A 92.40 67.76
FitNet 1M 91.61 64.96
FMP* (1 tests) 12M 95.50 73.61
Max-out(k=2) 6M 90.62 65.46
Network in Network 1M 91.19 64.32
DSN 1M 92.03 65.43
Max-out NIN - 93.25 71.14
LSUV N/A 94.16 N/A
SimpleNet 5.48M 95.51 78.37

Models and logs

-- refer to each dataset directory in the repository for further information on how to access models.

How to use

You can either download the simplenet.py model and use it directly in your projects or you can use torch hub.

Using torch hub you can do something like this:

import torch
# use the latest master
model = torch.hub.load("coderx7/simplenet_pytorch", "simplenetv1_5m_m1", pretrained=True)
# or any of these variants at the moment
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_5m_m2", pretrained=True)
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_9m_m1", pretrained=True)
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_9m_m2", pretrained=True)
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_small_m1_05", pretrained=True)
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_small_m2_05", pretrained=True)
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_small_m1_075", pretrained=True)
# model = torch.hub.load("coderx7/simplenet_pytorch:v1.0.0", "simplenetv1_small_m2_075", pretrained=True)
model.eval()

# Download an example image from the pytorch website
import urllib
url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
try: urllib.URLopener().retrieve(url, filename)
except: urllib.request.urlretrieve(url, filename)

# sample execution (requires torchvision)
from PIL import Image
from torchvision import transforms
input_image = Image.open(filename)
preprocess = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(input_image)
input_batch = input_tensor.unsqueeze(0) # create a mini-batch as expected by the model

# move the input and model to GPU for speed if available
if torch.cuda.is_available():
    input_batch = input_batch.to('cuda')
    model.to('cuda')

with torch.no_grad():
    output = model(input_batch)
# Tensor of shape 1000, with confidence scores over Imagenet's 1000 classes
print(output[0])
# The output has unnormalized scores. To get probabilities, you can run a softmax on it.
probabilities = torch.nn.functional.softmax(output[0], dim=0)
print(probabilities)

# Download ImageNet labels
!wget https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt

# Read the categories
with open("imagenet_classes.txt", "r") as f:
    categories = [s.strip() for s in f.readlines()]
# Show top categories per image
top5_prob, top5_catid = torch.topk(probabilities, 5)
for i in range(top5_prob.size(0)):
    print(categories[top5_catid[i]], top5_prob[i].item())

or you can run the demo online from huggingface simplenetspace: https://huggingface.co/spaces/coderx7/simplenet

Citation

If you find SimpleNet useful in your research, please consider citing:

@article{hasanpour2016lets,
  title={Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures},
  author={Hasanpour, Seyyed Hossein and Rouhani, Mohammad and Fayyaz, Mohsen and Sabokrou, Mohammad},
  journal={arXiv preprint arXiv:1608.06037},
  year={2016}
}