How to optimize parameters using genetic algorithms

In such an application, one passes the parameters whose values are to be optimized (in your case, cost, gamma and epsilon) as parameters of the fitness function, which then runs the model fitting + evaluation function and uses a measure of model performance as a measure of fitness. Therefore, the explicit form of the objective function is not directly relevant.

In the implementation below, I used 5-fold cross-validation to estimate the RMSE for a given set of parameters. In particular, since package GA maximizes the fitness function, I have written the fitness value for a given value of the parameters as minus the average rmse over the cross-validation datasets. Hence, the maximum fitness that can be attained is zero.

Here it is:

library(e1071)
library(GA)

data(Ozone, package="mlbench")
Data <- na.omit(Ozone)

# Setup the data for cross-validation
K = 5 # 5-fold cross-validation
fold_inds <- sample(1:K, nrow(Data), replace = TRUE)
lst_CV_data <- lapply(1:K, function(i) list(
    train_data = Data[fold_inds != i, , drop = FALSE], 
    test_data = Data[fold_inds == i, , drop = FALSE]))

# Given the values of parameters 'cost', 'gamma' and 'epsilon', return the rmse of the model over the test data
evalParams <- function(train_data, test_data, cost, gamma, epsilon) {
    # Train
    model <- svm(V4 ~ ., data = train_data, cost = cost, gamma = gamma, epsilon = epsilon, type = "eps-regression", kernel = "radial")
    # Test
    rmse <- mean((predict(model, newdata = test_data) - test_data$V4) ^ 2)
    return (rmse)
}

# Fitness function (to be maximized)
# Parameter vector x is: (cost, gamma, epsilon)
fitnessFunc <- function(x, Lst_CV_Data) {
    # Retrieve the SVM parameters
    cost_val <- x[1]
    gamma_val <- x[2]
    epsilon_val <- x[3]

    # Use cross-validation to estimate the RMSE for each split of the dataset
    rmse_vals <- sapply(Lst_CV_Data, function(in_data) with(in_data, 
        evalParams(train_data, test_data, cost_val, gamma_val, epsilon_val)))

    # As fitness measure, return minus the average rmse (over the cross-validation folds), 
    # so that by maximizing fitness we are minimizing the rmse
    return (-mean(rmse_vals))
}

# Range of the parameter values to be tested
# Parameters are: (cost, gamma, epsilon)
theta_min <- c(cost = 1e-4, gamma = 1e-3, epsilon = 1e-2)
theta_max <- c(cost = 10, gamma = 2, epsilon = 2)

# Run the genetic algorithm
results <- ga(type = "real-valued", fitness = fitnessFunc, lst_CV_data, 
    names = names(theta_min), 
    min = theta_min, max = theta_max,
    popSize = 50, maxiter = 10)

summary(results)

which produces the results (for the range of parameter values that I specified, which may require fine-tuning based on the data):

GA results: 
Iterations             = 100 
Fitness function value = -14.66315 
Solution               = 
         cost      gamma    epsilon
[1,] 2.643109 0.07910103 0.09864132