Author(s)

Liguo Zhang ¹, Jian Zhang ¹, Yan Wang ²

Affiliation(s)

¹ College of Information Science & Technology, Agricultural University of Hebei, Hebei, Baoding, China
² Dept of Computer, North China Electric Power University, Hebei, Baoding, China

Corresponding Author

Liguo Zhang

ABSTRACT

Background and purpose: the central problem in machine learning is identifying a representative set of features from which to construct a classification model and setting the best Hyper-Parameters of the model for a particular task. Materials and Methods: Feature selection aims to reduce the dimensionality of patterns for classificatory analysis by selecting the most informative instead of irrelevant and/or redundant features. Here, RFE is used for selecting the most useful predictive features, PCA extract the characteristics of all the original variables. And also the Grid Search is used for model HPO. Experiments are applied on Breast cancer dataset. Findings and Conclusion: The experiments show that, the number of features is reduced from 30 to 15 features and the classification accuracy increase from 95.6% to 97.02%. And the proposed Feature selection and hyper-parameter optimation method can be used to other pattern classification problems.

KEYWORDS

Feature Selection, PCA, HPO, Recursive Feature Elimination, Random Forest Classification

CITE THIS PAPER

Liguo Zhang, Jian Zhang and Yan Wang, A Novel Classification Model based on Ensemble Feature Selection and Hyper-parameter Optimation. Journal of Artificial Intelligence Practice (2022) Vol. 5: 11-17. DOI: http://dx.doi.org/10.23977/jaip.2022.050103.

REFERENCES

[1] Eid, H. F., Hassanien, A. E., Kim, T. H., & Banerjee, S. (2013). Linear Correlation-Based Feature Selection for Network Intrusion Detection Model. Springer Berlin Heidelberg. Springer Berlin Heidelberg.
[2] Kumaravel., V. ,  Raja., K. ,  Kumaravel., V. , &  Raja., K. . (2014). Feature Subset Selection Algorithm for High-Dimensional Data by using FAST Clustering Approach.
[3] Akadi, A. E., Ouardighi, A. E., & D Aboutajdine. (2008). A powerful feature selection approach based on mutual information. International journal of computer science & network security.
[4] Hall, M. A. (2000). Correlation-based feature selection for machine learning. Phd Thesis Waikato Univer Sity.
[5 Chaves, R., J Ramírez, JM Górriz, M López, & Segovia, F. (2009). Svm-based computer-aided diagnosis of the alzheimer's disease using t-test nmse feature selection with feature correlation weighting. Neuroscience Letters, 461(3), 293-297.
[6] Upadhyay, D., Manero, J., Zaman, M., & Sampalli, S. (2021). Intrusion detection in scada based power grids: recursive feature elimination model with majority vote ensemble algorithm. IEEE Transactions on Network Science and Engineering, PP (99), 1-1.
[7] Kim, J.  "A Comparative Study of Sequential Feature Selection Methods for Support Vector Machine." (2008).
[8] Naing, T. T., & Khaing, K. T. (2010). Enhanced features ranking and selection using recursive feature elimination(rfe) and k-nearest neighbor algorithms in support vector machine for intrusion detection system. International journal of computer science issues.
[9] Chen, X. W., and J. C. Jeong. "Enhanced recursive feature elimination." Machine Learning and Applications, 2007. ICMLA 2007. Sixth International Conference on IEEE, 2008.
[10] Zhang, L., Zhi, L., & Yi, L. I. (2013). Research on diagnosis method of breast tumor based on factor analysis and ga-bp. Journal of Convergence Information Technology.
[11] Karabatak, M., & Ince, M. C. (2009). An expert system for detection of breast cancer based on association rules and neural network. Expert Systems with Applications, 36(2p2), 3465-3469.

Sponsors, Associates, and Links

---

• Power Systems Computation
  
  
• Internet of Things (IoT) and Engineering Applications
  
  
• Computing, Performance and Communication Systems
  
  
• Advances in Computer, Signals and Systems
  
  
• Journal of Network Computing and Applications
  
  
• Journal of Web Systems and Applications
  
  
• Journal of Electrotechnology, Electrical Engineering and Management
  
  
• Journal of Wireless Sensors and Sensor Networks
  
  
• Journal of Image Processing Theory and Applications
  
  
• Mobile Computing and Networking
  
  
• Vehicle Power and Propulsion
  
  
• Frontiers in Computer Vision and Pattern Recognition
  
  
• Knowledge Discovery and Data Mining Letters
  
  
• Big Data Analysis and Cloud Computing
  
  
• Electrical Insulation and Dielectrics
  
  
• Crypto and Information Security
  
  
• Journal of Neural Information Processing
  
  
• Collaborative and Social Computing
  
  
• International Journal of Network and Communication Technology
  
  
• File and Storage Technologies
  
  
• Frontiers in Genetic and Evolutionary Computation
  
  
• Optical Network Design and Modeling
  
  
• Journal of Virtual Reality and Artificial Intelligence
  
  
• Natural Language Processing and Speech Recognition
  
  
• Journal of High-Voltage
  
  
• Programming Languages and Operating Systems
  
  
• Visual Communications and Image Processing
  
  
• Journal of Systems Analysis and Integration
  
  
• Knowledge Representation and Automated Reasoning
  
  
• Review of Information Display Techniques
  
  
• Data and Knowledge Engineering
  
  
• Journal of Database Systems
  
  
• Journal of Cluster and Grid Computing
  
  
• Cloud and Service-Oriented Computing
  
  
• Journal of Networking, Architecture and Storage
  
  
• Journal of Software Engineering and Metrics
  
  
• Visualization Techniques
  
  
• Journal of Parallel and Distributed Processing
  
  
• Journal of Modeling, Analysis and Simulation
  
  
• Journal of Privacy, Trust and Security
  
  
• Journal of Cognitive Informatics and Cognitive Computing
  
  
• Lecture Notes on Wireless Networks and Communications
  
  
• International Journal of Computer and Communications Security
  
  
• Journal of Multimedia Techniques
  
  
• Automation and Machine Learning
  
  
• Computational Linguistics Letters
  
  
• Journal of Computer Architecture and Design
  
  
• Journal of Ubiquitous and Future Networks