How do weak-learners combine?

[jeffltc]

• If I use adanet.AutoEnsembleEstimator to combine weak-learners, do those weak-learners combine side by side on the same layer with weights or one after another on multiple layers?

• I’ve also read Combination of subnetworks #24. Does that mean the side by side combination is a default and if I want to make one after another structure, I need to use custom generator and Subnetwork.shared to pass the tensor to next iteration?

• Any update on displaying the detailed archetecture of the final result? (I’ve tried using method mentioned in [Question] How to retrieve the detailed ensemble architecture? #29 to find out the detailed structure in TensorBoard but as you mentioned, the result is less than ideal. I can only know which weak-learner is in the final model structure in TEXT tab while the graph in GRAPH tab is a little bit too detailed which made me lost in the nest of ops. And the method mentioned in [Question] How to retrieve the detailed ensemble architecture? #29 seemed not compatible with tf 2.0. I'm still working on it.)

[cweill]

If I use adanet.AutoEnsembleEstimator to combine weak-learners, do those weak-learners combine side by side on the same layer with weights or one after another on multiple layers?

AutoEnsembleEstimator use the ensemblers parameter to determine how to ensemble weak learners. By default it uses with adanet.ensemble.ComplexityRegularizedEnsembler which is pretty simple: it just combines the weighted logits of the subnetworks. However, you can subclass adanet.ensemble.Ensembler to do more advanced things if you'd prefer.

I’ve also read #24. Does that mean the side by side combination is a default and if I want to make one after another structure, I need to use custom generator and Subnetwork.shared to pass the tensor to next iteration?

That's correct. Doing so allows you to create more expressive and adaptive search spaces than if you used AutoEnsembleEstimator.

Any update on displaying the detailed archetecture of the final result? (I’ve tried using method mentioned in #29 to find out the detailed structure in TensorBoard but as you mentioned, the result is less than ideal. I can only know which weak-learner is in the final model structure in TEXT tab while the graph in GRAPH tab is a little bit too detailed which made me lost in the nest of ops. And the method mentioned in #29 seemed not compatible with tf 2.0. I'm still working on it.)

Currently it's tough, because we support arbitrary TF 1.0 graphs. So the only source of truth for that kind of connectivity is the TensorBoard "Graph" tab, which as you saw, can be quite noisy and tough to interpret.

That being said, with TF 2.0, we are moving towards a Keras-first world when weak learners will be Keras Models composed of Keras Layers. This should result in us being able to extract a DAG of keras layers that is more human-readable. But that is still a ways away. :)

[le-dawg]

Any update on displaying the detailed archetecture of the final result? (I’ve tried using method mentioned in #29 to find out the detailed structure in TensorBoard but as you mentioned, the result is less than ideal. I can only know which weak-learner is in the final model structure in TEXT tab while the graph in GRAPH tab is a little bit too detailed which made me lost in the nest of ops. And the method mentioned in #29 seemed not compatible with tf 2.0. I'm still working on it.)

Currently it's tough, because we support arbitrary TF 1.0 graphs. So the only source of truth for that kind of connectivity is the TensorBoard "Graph" tab, which as you saw, can be quite noisy and tough to interpret.

That being said, with TF 2.0, we are moving towards a Keras-first world when weak learners will be Keras Models composed of Keras Layers. This should result in us being able to extract a DAG of keras layers that is more human-readable. But that is still a ways away. :)

What is the current level of development on this? There is no way to use plot_model() on the subnetworks?