Home  /  Entropy  /  Vol: 21 Núm: 1 Par: January (2019)  /  Article

Simple Stopping Criteria for Information Theoretic Feature Selection


Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite the simplicity of this objective, there still remain several open problems in optimization. These include, for example, the automatic determination of the optimal subset size (i.e., the number of features) or a stopping criterion if the greedy searching strategy is adopted. In this paper, we suggest two stopping criteria by just monitoring the conditional mutual information (CMI) among groups of variables. Using the recently developed multivariate matrix-based Rényi’s a-entropy functional, which can be directly estimated from data samples, we showed that the CMI among groups of variables can be easily computed without any decomposition or approximation, hence making our criteria easy to implement and seamlessly integrated into any existing information theoretic feature selection methods with a greedy search strategy.

 Articles related

Mohsen Nowkarizi,Mahdi Zeynali Tazehkandi    

The aim of the study was to improve Persian search engines’ retrieval performance by using the new measure. In this regard, consulting three experts from the Department of Knowledge and Information Science (KIS) at Ferdowsi University of Mashhad, 192 FUM... see more

Jafar Mehrad,Pegah Tajer    

This paper was aimed at clarifying the links between Uses and Gratification Theory (UGT) and Knowledge and Information Science in both traditional and modern contexts. Uses and Gratification conceptual model were also proposed both for library and inform... see more

Marie-Laure Baron,Herve Mathieu    

Information and knowledge management have become crucial to the development of a competitive edge on the market. This requires the gathering of complete and consistent information in an environment where companies are working increasingly with a vast net... see more

Lesya I. Prokhorenko, Oksana V. Romanenko    

The authors made an attempt to outline the features of classification of information field objects in children with mental retardation, which involved studying the formation of processes and operations that underlie the ability to categorize objects and ... see more

Eduardo Alves Silva,Dalton Lopes Martins    

RESUMO O presente artigo tem por objetivo apresentar a investigação efetuada a partir de objetos digitais, mais propriamente coleções digitais, procurando conceituar como compreende ciência aberta no contexto da pesquisa sobre os acervos museológico... see more

Revista: Liinc em Revista