When the number of features p is large, there tends to be a deterioration in the performance of KNN and other approaches that perform prediction using only observations that are near the test observation for which a predi made. This phenomenon is known as the perform poorly when p is large. We will now investigate this curse ction must be curse of dimensionality, and it ties into the fact that non-parametric approaches often a. Suppose that we have e that we have a set of observations , each with measurements on p = 1 feature, X. We assume that X is distributed on [0, 1]. Associated with each observation is a response value. Suppose that we st observ ation's response using only observations that are within 10% of the range of X wish to predict a te closest to that test observation. For instance we will use obs use to make the prediction? , in order to predict the response for a test observation with X 0.6 ervations in the range [0.55, 0.65]. On average, what fraction of the available observations will we
b. Now suppose that we have a set of observations, each with measurements on p- 2 niformly distributed on [O, 1] x [0, 1]. We wish to predict a test observation's response features, X1 and X2. We using only observations that are within 10% of the range of X, and within 10% of the range of test observation. For instance, in order to predict the response for a test observation with X0.6 X, 0.35, we will use observations in the range [0.55,0.65] for Xi and in range [0.3,0.4] for X2.On average, what fraction of the available observations will we use to make the prediction?
c. Now suppose that we have a set of observations on p100 features. Again t he observations are uniformly . distributed on each feature, and again each feature ranges in value from 0 to 1. We wish to predict a test observation's response using observations within the 10% of each feature's range that is closest to that test observation. What fraction of the available observations will we use to make the prediction?
d. Now suppose that we wish to make a prediction for a test observation by creating a p-dimensional hypercube centered around the test observation that contains, on average, 10% of the training observations. For p 1 2 and 100, what is the length of each side of the hypercube?