diff --git a/Experiment_emotions.cfg b/Experiment_emotions.cfg index 8034183..3fd1909 100644 --- a/Experiment_emotions.cfg +++ b/Experiment_emotions.cfg @@ -25,19 +25,9 @@ data/emotions/emotions_test_1.arff data/emotions/emotions.xml - - data/emotions/emotions_train_2.arff - data/emotions/emotions_test_2.arff - data/emotions/emotions.xml - - - data/emotions/emotions_train_3.arff - data/emotions/emotions_test_3.arff - data/emotions/emotions.xml - - + reports/EnsembleMLC summaryEnsembleMLC 1 diff --git a/README.md b/README.md index fab9394..e1698f0 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,15 @@ # EME: Evolutionary Multi-label Ensemble -EME is an evolutionary approach for the automatic generation of ensembles of diverse and competitive multi-label classifiers. It takes into account characteristics of the multi-label data such as the relationships among the labels, imbalance of the data, and the complexity of the output space. The ensemble is based on projections of the label space, considering in this way the relationships among the labels but also reducing the computational cost in cases where the output space is complex. Further, EME takes into account all the labels approximately the same number of times in the ensemble, regardless of their frequency or its ease to be predicted; so that the imbalance of the data is considered and the infrequent labels are not ignored. For that, the fitness function takes into account both the predictive performance of the model and the number of times that each label is considered in the ensemble. Finally, the diversity of the ensemble is not taken into account explicitly, but the ensembles evolve selecting their classifiers based on their overall performance. +EME is an evolutionary approach for the automatic generation of ensembles of diverse and competitive multi-label classifiers. It takes into account characteristics of the multi-label data such as the relationships among the labels, imbalance of the data, and the complexity of the output space. The ensemble is based on small projections of the label space, considering in this way the relationships among the labels but also reducing the computational cost in cases where the output space is complex. Further, EME takes into account all the labels approximately the same number of times in the ensemble, regardless of their frequency or its ease to be predicted; so that the imbalance of the data is considered and the infrequent labels are not ignored. For that, the fitness function takes into account both the predictive performance of the model and the number of times that each label is considered in the ensemble. More information about this algorithm can be find in the following article: > Jose M. Moyano, Eva L. Gibaja, Krzysztof J. Cios, Sebastián Ventura. "An Evolutionary Approach to Build Ensembles of Multi-Label Classifiers". Submitted to Information Fusion. (2018). -If you use EME, please cite the paper. Further, a bibtex cite [] is provided at the end of the description. +If you use EME, please cite the paper. Further, a [bibtex citation file](https://github.com/i02momuj/EvolutionaryMultiLabelEnsemble/blob/master/citation.bib) is also provided. In this repository we provide the code of EME, distributed under the GPLv3 License. EME has been implemented using JCLEC [[Ven08]](#Ven08), Mulan [[Tso11]](#Tso11), and Weka [[Hal09]](#Hal09) libraries. Besides, the last release (v 1.2) [] provides the executable jar to execute EME and also the javadoc. -To execute EME, only have to execute the following command: +To execute EME, the following command have to be executed: ```sh java -jar EME.jar configFile.cfg ``` @@ -51,8 +51,8 @@ The configuration file is a xml file including the parameters of the evolutionar ``` -* The configuration file must start with the `````` tag and then the `````` tag, which indicates the class with the evolutionary algorithm, in our case ```eme.EnsembleAlgorithm```. -* The `````` must determine the seed for random numbers with the ```seed``` attribute. Further, it may indicate the type of the rand-gen-factory, which by default is ```net.sf.jclec.util.random.RanecuFactory```. If several seeds are going to be used, the tag `````` is used, including inside the different seeds, as follows: +* The configuration file must start with the `````` tag and then the `````` tag, the last indicating the class with the evolutionary algorithm, in our case ```eme.EnsembleAlgorithm```. +* The `````` must determine the seed for random numbers with the ```seed``` attribute. Further, it may indicate the type of the rand-gen-factory, which by default is ```net.sf.jclec.util.random.RanecuFactory```. If several seeds are going to be used, the tag `````` may be used, including inside the different seeds, as follows: ```xml @@ -61,16 +61,16 @@ The configuration file is a xml file including the parameters of the evolutionar ... ``` -* The parents selector is determined with the tag ``````. If for example the tournament selector is selected, its size is determined with the sub-tag ``````. +* The parents selector is determined with the `````` tag. If, for example, the tournament selector is selected, its size is determined with the sub-tag ``````. * The size of the population is determined with the `````` tag. * The number of generations of the evolutionary algorithm is determined with the `````` tag. * The `````` tag determines the type of recombinator or crossover operator. In EME, three crossover operators are implemented: ```ModelCrossover```, ```MultiModelCrossover```, and ```UniformModelCrossover```. Further, the probability to apply this operator to each individual is determined with the ```rec-prob``` attribute. -* The `````` tag determines the type of mutation operator. In EME, two crossover operators are implemented: the basic ```IntraModelMutator```, and ```PhiBasedIntraModelMutator```.. Further, the probability to apply this operator to each individual is determined with the ```mut-prob``` attribute. -* The number of classifiers in each ensemble is determined with the `````` tag. -* The number of labels of each classifier, or size of the *k*-labelset, is determined with the `````` tag. +* The `````` tag determines the type of mutation operator. In EME, two crossover operators are implemented: the basic ```IntraModelMutator```, and ```PhiBasedIntraModelMutator```. Further, the probability to apply this operator to each individual is determined with the ```mut-prob``` attribute. +* The number of classifiers in each ensemble is determined by the `````` tag. +* The number of labels of each classifier, or size of the *k*-labelset, is determined by the `````` tag. * The threshold used for the final prediction of the ensemble is determined with the `````` tag. -* The tag `````` determines if the coverage ratio measure is included in the fitness of the individuals. The coverage ratio takes into account the number of times that each label appears in the ensemble. -* With the `````` tag, the datasets used for training (for the evolutionary algorithm) and testing (for testint the final obtained ensemble by EME) are determined with the tags `````` and `````` respectively. The `````` tag indicates the xml file of the dataset. Several datasets, or several partitions of the same dataset may be used, including the tag ``````, including inside the different datasets, as follows: +* The `````` tag determines if the coverage ratio measure is included in the fitness of the individuals. The coverage ratio takes into account the number of times that each label appears in the ensemble. +* With the `````` tag, the datasets used for training (for the evolutionary algorithm) and testing (for testing the final ensemble obtained by EME) are determined with the tags `````` and `````` respectively. The `````` tag indicates the xml file of the dataset (Mulan format, [see more](http://www.uco.es/kdis/mllresources/)). Several datasets, or several partitions of the same dataset may be used, including the tag ``````, and the different datasets inside, as follows: ```xml @@ -91,7 +91,7 @@ The configuration file is a xml file including the parameters of the evolutionar ... ``` -* The `````` tag determines the class used as listener; it is the responsible of creating the different reports during and at the end of the evolutionary process. The `````` tag determines the directory where the reports of the different executions are stored. The `````` tag indicates the filename of the global report file. Finally, the `````` tag indicates the frequency with which the reports for the iterations are created. +* The `````` tag determines the class used as listener; it is the responsible of creating the different reports during and at the end of the evolutionary process. By default, the listener used is the one of the ```eme.EnsembleListener``` class. The `````` tag determines the directory where the reports of the different executions are stored. The `````` tag indicates the filename of the global report file. Finally, the `````` tag indicates the frequency with which the reports for the iterations are created. Then, several more characteristics of the evolutionary algorithm could be modified in the configuration file, but they are optional and default values for them are given if they are not included in this file: * The `````` tag indicates if the training set is divided into training and validation, in order to evaluate the individuals with a different dataset to which was used to train them. By default, its value is ```false```. diff --git a/citation.bib b/citation.bib new file mode 100644 index 0000000..3e63e05 --- /dev/null +++ b/citation.bib @@ -0,0 +1,8 @@ +@article{Moyano_EME, + title = {An Evolutionary Approach to Build Ensembles of Multi-Label Classifiers}, + author = {Jose M. Moyano and Eva L. Gibaja and Krzysztof J. Cios and Sebasti'{a}n Ventura}, + journal = {Submitted to Information Fusion}, + volume = {}, + pages = {}, + year = {2018} +} diff --git a/citation.txt b/citation.txt new file mode 100644 index 0000000..97c4dcc --- /dev/null +++ b/citation.txt @@ -0,0 +1 @@ +Jose M. Moyano, Eva L. Gibaja, Krzysztof J. Cios, Sebastián Ventura. "An Evolutionary Approach to Build Ensembles of Multi-Label Classifiers". Submitted to Information Fusion. (2018). \ No newline at end of file diff --git a/experiment_emotions.cfg_0_0 b/experiment_emotions.cfg_0_0 new file mode 100644 index 0000000..e5d5c44 --- /dev/null +++ b/experiment_emotions.cfg_0_0 @@ -0,0 +1,33 @@ + + + + + + + 2 + + + 50 + 25 + + + + + 12 + 3 + 0.5 + true + + + data/emotions/emotions_train_1.arff + data/emotions/emotions_test_1.arff + data/emotions/emotions.xml + + + + reports/EnsembleMLC + summaryEnsembleMLC + 1 + + + diff --git a/src/main/java/eme/EnsembleAlgorithm.java b/src/main/java/eme/EnsembleAlgorithm.java index 3d6a377..1c83897 100644 --- a/src/main/java/eme/EnsembleAlgorithm.java +++ b/src/main/java/eme/EnsembleAlgorithm.java @@ -287,6 +287,11 @@ private void configureEmeDefaults(Configuration configuration) { if(! configuration.containsKey("parents-selector.tournament-size")) { configuration.addProperty("parents-selector.tournament-size", "2"); } + + //Listener type (only if not provided) + if(! configuration.containsKey("listener[@type]")) { + configuration.addProperty("listener[@type]", "eme.EnsembleListener"); + } } /**