Optimizing Hyper Parameters for Enhanced Performance: A Comparative Study on K-Nearest Neighbor and Support Vector Machine Models

Abstract

Numerous applications and areas benefit from the use of machine learning algorithms. The hyperparameters needed to fit a suitable model must be tuned to address different issues. The choice of the best configuration directly affects the performance of the model. User-defined hyper-parameters are those that are set before the training process is carried out. In machine learning, optimizing the hyperparameters is a process that can reduce the cost function. This paper presents a couple of methods that are used to optimize the K-Nearest Neighbor algorithm and the Support Vector Machine model. We have performed several experiments on the different optimization techniques to evaluate their accuracy and complexity.

Index Terms: Hyper parameters, Optimization Techniques, KNN, SVM, Random Search, Grid Search.