Linear regression prediction not matching training data

Question

I am a newbie to machine learning. I am trying a simple prediction using linear regression with "made up" data that follows a specific pattern. For some reason, the prediction is not matching the training data. Can you let me know what I need to fix? The sample code is below

from sklearn import linear_model
import numpy as np

X = np.random.randint(3, size=(3, 1000))
Y = np.random.randint(10, size=(1, 1000))
# f1, f2, f3 - min = 0, max = 2
# f1 = 0 and f2 = 1  then 7 <= Y < 10, irrespective of f3
# f1 = 1 and f2 = 2 Y is 0, irrespective of f3
# f1 = 0 and f2 = 2 if f3 = 2 then 3 <= Y < 7 else Y = 0
for i in range(1000):
    if ((X[0][i] == 0 and X[1][i] == 1) or (X[0][i] == 1 and X[1][i] == 0)):
        Y[0][i] = np.random.randint(7, 10)
    elif ((X[0][i] == 1 and X[1][i] == 2) or (X[0][i] == 2 and X[1][i] == 1)):
        Y[0][i] = 0
    elif ((X[0][i] == 0 and X[1][i] == 2 and X[2][i] == 2) or
         (X[0][i] == 2 and X[1][i] == 0 and X[2][i] == 2)):
        Y[0][i] = np.random.randint(3, 7)
    else:
        Y[0][i] = 0

X1 = X.transpose()
Y1 = Y.reshape(-1, 1)
print zip(X1, Y1)

# create and fit the model
clf = linear_model.LinearRegression()
clf.fit(X1, Y1)

Z = np.array([[0, 0, 0, 0, 1, 1],
              [1, 1, 2, 2, 2, 2],
              [1, 2, 1, 2, 1, 2]])
Z1 = Z.transpose()
print Z1

y_predict = clf.predict(Z1)
print y_predict

lejlot · Accepted Answer · 2016-04-02 09:00:57Z

1

And why would it match the training data? Your X->Y relation is clearly non-linear, and only for perfect linear relation, meaning that Y = AX + b, you can expect linear regression to fit training data perfectly. Otherwise, you can get arbitrary far away from the solution - see for example an Anscombe's quartet (image belowo from wiki).

answered Apr 2, 2016 at 9:00

lejlot

67k9 gold badges138 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user6147957 Over a year ago

Sure what is a better algorithm for this kind of training data?

lejlot Over a year ago

A single decision/regression tree (or a very small random forest) should handle it just fine.

Collectives™ on Stack Overflow

Linear regression prediction not matching training data

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related