How to use multi-gpu in Keras with shared weights applications model

[UpCoder]

System information

• Linux Ubuntu 16.04)
• TensorFlow backend
• TensorFlow version: 1.10.0

I want to use the keras in multi-gpus with the applications (such VGG16). But there are some error.

I try to use the single-gpus it is correct. But the multi-gpus is wrong.
The code like this:

import keras
    with tf.device('/cpu:0'):
        input1 = keras.layers.Input(config.input_shape)
        input2 = keras.layers.Input(config.input_shape)
        sub_model = keras.applications.VGG16(include_top=False, weights=config.VGG_MODEL_PATH,
                                             input_shape=config.input_shape)
        output1 = sub_model(input1)
        output2 = sub_model(input1)
        model = keras.Model(inputs=[input1, input2], outputs=[output1, output2])
    parallel_model = keras.utils.multi_gpu_model(model, gpus=3)
    parallel_model.compile('sgd', loss=['mse', 'mse'])
    parallel_model.fit((np.random.random([10, 128, 128, 3]), np.random.random([10, 128, 128, 3])),
                       (np.random.random([10, 4, 4, 512]), np.random.random([10, 4, 4, 512])))

The error message is

Traceback (most recent call last):
  File "/data00/home/liangdong.tony/PycharmProject/RetrievalCCWebVideo/AE/demo.py", line 145, in <module>
    parallel_model = keras.utils.multi_gpu_model(model, gpus=3)
  File "/data00/home/liangdong.tony/.local/lib/python2.7/site-packages/keras/utils/training_utils.py", line 177, in multi_gpu_model
    return Model(model.inputs, merged)
  File "/data00/home/liangdong.tony/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
    return func(*args, **kwargs)
  File "/data00/home/liangdong.tony/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1811, in __init__
    'Layer names: ', all_names)
RuntimeError: ('The name "vgg16" is used 2 times in the model. All layer names should be unique. Layer names: ', ['input_1', 'input_2', 'lambda_1', 'lambda_2', 'lambda_3', 'lambda_4', 'lambda_5', 'lambda_6', 'model_1', 'vgg16', 'vgg16'])

In a word, I want to use the VGG16 as backbone. I have to input, which used as the input of VGG16 which shared weights between the two inputs.
Do you have any suggestion about that?
Thank you, looking for your reply!

[bzamecnik]

Hi, these days the multi GPU support is available directly in Keras and TensorFlow. Either use Keras' multi_gpu_model() or TF distribution strategy. https://www.tensorflow.org/beta/guide/distribute_strategy Kind regards, Bohumir Dne ne 9. 6. 2019 12:12 uživatel UpCoder <notifications@github.com> napsal:

…

*System information* - Linux Ubuntu 16.04) - TensorFlow backend - TensorFlow version: 1.10.0 I want to use the keras in multi-gpus with the applications (such VGG16). But there are some error. I try to use the single-gpus it is correct. But the multi-gpus is wrong. The code like this: import keras with tf.device('/cpu:0'): input1 = keras.layers.Input(config.input_shape) input2 = keras.layers.Input(config.input_shape) sub_model = keras.applications.VGG16(include_top=False, weights=config.VGG_MODEL_PATH, input_shape=config.input_shape) output1 = sub_model(input1) output2 = sub_model(input1) model = keras.Model(inputs=[input1, input2], outputs=[output1, output2]) parallel_model = keras.utils.multi_gpu_model(model, gpus=3) parallel_model.compile('sgd', loss=['mse', 'mse']) parallel_model.fit((np.random.random([10, 128, 128, 3]), np.random.random([10, 128, 128, 3])), (np.random.random([10, 4, 4, 512]), np.random.random([10, 4, 4, 512]))) The error message is Traceback (most recent call last): File "/data00/home/liangdong.tony/PycharmProject/RetrievalCCWebVideo/AE/demo.py", line 145, in <module> parallel_model = keras.utils.multi_gpu_model(model, gpus=3) File "/data00/home/liangdong.tony/.local/lib/python2.7/site-packages/keras/utils/training_utils.py", line 177, in multi_gpu_model return Model(model.inputs, merged) File "/data00/home/liangdong.tony/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 87, in wrapper return func(*args, **kwargs) File "/data00/home/liangdong.tony/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 1811, in __init__ 'Layer names: ', all_names) RuntimeError: ('The name "vgg16" is used 2 times in the model. All layer names should be unique. Layer names: ', ['input_1', 'input_2', 'lambda_1', 'lambda_2', 'lambda_3', 'lambda_4', 'lambda_5', 'lambda_6', 'model_1', 'vgg16', 'vgg16']) In a word, I want to use the VGG16 as backbone. I have to input, which used as the input of VGG16 which shared weights between the two inputs. Do you have any suggestion about that? Thank you, looking for your reply! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#10?email_source=notifications&email_token=AADM5LHJVWDV5HXAXUW724DPZTJSXA5CNFSM4HWJDS72YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GYNWVEA>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADM5LEC6HE2CNWNFQVHF7LPZTJSXANCNFSM4HWJDS7Q> .