ValueError: Invalid parameter C for estimator SelectFromModel when using GridSearchCV

Question

I'm using python 3.7.6, and I'm trying to tune some hyperparameters using GridSearchCV

I created a pipeline with the following steps: scaling-> feature selection -> model

But I'm getting error about the C parameter of the feature selection step.

    steps = [('scaler', StandardScaler()),
             ('FeatureSelection', SelectFromModel(LogisticRegression(penalty='l1', solver='liblinear'))),
             ('SVM', SVC())]
    pipeline = Pipeline(steps)  # define the pipeline object.

    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=30, stratify=y)

    parameteres = {'SVM__C': [0.001, 0.1, 10, 100, 10e5],
                   'SVM__gamma':[0.1,0.01],
                   'FeatureSelection__C':['0','0.5']}
    grid = GridSearchCV(pipeline, param_grid=parameteres, cv=5, n_jobs=-1)
    grid.fit(X_train, y_train)
    print("pipeline score: ", grid.score(X_test, y_test))

I'm getting the following error:

ValueError: Invalid parameter C for estimator SelectFromModel(estimator=LogisticRegression(C=1.0, class_weight=None,
                                             dual=False, fit_intercept=True,
                                             intercept_scaling=1, l1_ratio=None,
                                             max_iter=100, multi_class='auto',
                                             n_jobs=None, penalty='l1',
                                             random_state=None,
                                             solver='liblinear', tol=0.0001,
                                             verbose=0, warm_start=False),
                max_features=None, norm_order=1, prefit=False, threshold=None). Check the list of available parameters with `estimator.get_params().keys()`.

What is wrong and how can I fix it ?

desertnaut · Accepted Answer · 2020-03-31 15:02:05Z

1

As is, the pipeline looks for a parameter C in SelectFromModel, can't find one (unsurprisingly, since the module does not have such a parameter), and raises an error. Since you actually want the parameter C of LogisticRegression, you should go a level deeper: change FeatureSelection__C to FeatureSelection__estimator__C in your parameters grid and you will be fine.

edited Mar 31, 2020 at 15:02

answered Mar 31, 2020 at 14:45

desertnaut

60.8k32 gold badges155 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

desertnaut Over a year ago

@user3668129 You sure?? I've actually tried it (different data) and it worked (I even replicated the error). You sure have the correct 2 underscores (__)?

user3668129 Over a year ago

you gave me good hint. I had to remove the "" and it works: 'FeatureSelection__estimator__C': [0.1, 0.5]

desertnaut Over a year ago

@user3668129 I told ya ;) (but you already show single quotes, not double ones)

Collectives™ on Stack Overflow

ValueError: Invalid parameter C for estimator SelectFromModel when using GridSearchCV

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related