main principle

[paviddavid]

Hello,

your README is not very long. Therefore, I want to ask on this way. Is this function an implementation of the Early Stopping feature for the Tensorflow Object-Detection? Do I only have to copy this function in the ../models/research/object_detection/ folder and execute it as the provided model_main.py by the official Tensorflow OD API?

How should I use it? As far as I understood it, the function searches for an convergence in the first 100k iterations (default value), am I right? What is the criterion to abort the training process?

Kind Regards

---

EDIT:

If I copy the function inside the folder and execute it, I get the following error message:
TypeError: create_train_and_eval_specs() missing 2 required positional arguments: 'predict_input_fn' and 'train_steps'

Obviously, train_input_fn and train_steps are missing. Please clarify why they are missing.

[paviddavid]

Adding the missing variables (train_input_fn and train_steps) only leads to another error in line 158. TypeError: 'eval_spec' must have type 'tf.estimator.EvalSpec'. Got: <class 'tensorflow.python.estimator.training.TrainSpec'>

[hongym7]

different with Object detection API version or Tensorflow Version

[paviddavid]

@hongym7 Thanks for your response. In my opinion, you missed to add the two mentioned parameters train_input_fn and train_steps when you call the function, as described above. I know you only want to generate the EvalSpec, but the function needs to have all parameters in order to not throw an error.

So, I added them and I saw that the training is really slow (compared to an execution without early stop). For example, without early stop I can train for 110k Iterations while I can only train 42 iterations with early stop. Can you imagine what is the problem why it takes that much time? Before every, lets say 1000th iterations a model got output and now (with early stop) every iteration I obtain a model. The accuracy values are still not good enough.

Thanks a lot in advance for any hints/remarks

[paviddavid]

@hongym7 Currently I am using tensorflow 1.12.0-rc2 and downloaded the TF OD API for one month. So, I think it is the latest version.

[dangmanhtruong1995]

Hi @paviddavid , to fixed the error I looked into the function create_train_and_eval_specs and used the relevant parts, so here is my fix:

train_spec = tf.estimator.TrainSpec(
        input_fn=train_input_fn, max_steps=train_steps, hooks=[early_stopping_hook])

    # eval_specs = model_lib.create_train_and_eval_specs(
    #     train_input_fn,
    #     eval_input_fns,
    #     eval_on_train_input_fn,
    #     predict_input_fn,
    #     train_steps,
    #     eval_on_train_data=False)
    final_exporter_name='Servo'
    eval_specs = []
    eval_spec_names = [str(i) for i in range(len(eval_input_fns))]
    for index, (eval_spec_name, eval_input_fn) in enumerate(zip(eval_spec_names, eval_input_fns)):
        if index == 0:
            exporter_name = final_exporter_name
        else:
            exporter_name = '{}_{}'.format(final_exporter_name, eval_spec_name)
        exporter = tf.estimator.FinalExporter(name=exporter_name, serving_input_receiver_fn=predict_input_fn)
        eval_specs.append(
            tf.estimator.EvalSpec(
                name=eval_spec_name,
                input_fn=eval_input_fn,
                steps=None,
                exporters=exporter))

    # Currently only a single Eval Spec is allowed.
    tf.estimator.train_and_evaluate(estimator, train_spec, eval_specs[0])

I hope that this is the correct way to do it as I am not good at Tensorflow.