How to fix IndexError: invalid index to scalar variable

This code generates error:

IndexError: invalid index to scalar variable.

at the line: results.append(RMSPE(np.expm1(y_train[testcv]), [y[1] for y in y_test]))

How to fix it?

import pandas as pd
import numpy as np
from sklearn import ensemble
from sklearn import cross_validation

def ToWeight(y):
    w = np.zeros(y.shape, dtype=float)
    ind = y != 0
    w[ind] = 1./(y[ind]**2)
    return w

def RMSPE(y, yhat):
    w = ToWeight(y)
    rmspe = np.sqrt(np.mean( w * (y - yhat)**2 ))
    return rmspe

forest = ensemble.RandomForestRegressor(n_estimators=10, min_samples_split=2, n_jobs=-1)

print ("Cross validations")
cv = cross_validation.KFold(len(train), n_folds=5)

results = []
for traincv, testcv in cv:
    y_test = np.expm1(forest.fit(X_train[traincv], y_train[traincv]).predict(X_train[testcv]))
    results.append(RMSPE(np.expm1(y_train[testcv]), [y[1] for y in y_test]))

testcv is:

[False False False ...,  True  True  True]

3 Answers

You are trying to index into a scalar (non-iterable) value:

[y[1] for y in y_test]
#  ^ this is the problem

When you call [y for y in test] you are iterating over the values already, so you get a single value in y.

Your code is the same as trying to do the following:

y_test = [1, 2, 3]
y = y_test[0] # y = 1
print(y[0]) # this line will fail

I’m not sure what you’re trying to get into your results array, but you need to get rid of [y[1] for y in y_test].

If you want to append each y in y_test to results, you’ll need to expand your list comprehension out further to something like this:

[results.append(..., y) for y in y_test]

Or just use a for loop:

for y in y_test:
    results.append(..., y)

Basically, 1 is not a valid index of y. If the visitor is comming from his own code he should check if his y contains the index which he tries to access (in this case the index is 1).

In the for, you have an iteration, then for each element of that loop which probably is a scalar, has no index. When each element is an empty array, single variable, or scalar and not a list or array you cannot use indices.

Leave a Reply

Your email address will not be published. Required fields are marked *