Machine Learning — Practice

X = dataset[['feature1', 'feature2']].values
y = dataset['label'].values

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = LogisticRegression(C=0.01)
model.fit(X_scaled, y)

predictions = model.predict(X_scaled)
accuracy = (predictions == y).mean()

1   X = dataset.drop(columns=["label"]).values
2   y = (dataset["label"].values == "spam")
3
4   Xtr, Xtest, ytr, ytest = train_test_split(X, y, test_size=0.3)
5
6   best_acc = -np.inf
7   best_clf = None
8
9   for C in [0.1, 1.0, 10.0]:
10      clf = SVC(C=C, kernel="rbf", gamma=0.5)
11      clf.fit(Xtr, ytr)
12      acc = accuracy_score(ytr, clf.predict(Xtr))
13      if acc > best_acc:
14          best_acc = acc
15          best_clf = clf
16
17  print(f"Chosen C: {best_clf.C}, training accuracy: {best_acc:.3f}")
18  print(f"Test accuracy: {accuracy_score(ytest, best_clf.predict(Xtest)):.3f}")

1   from sklearn.linear_model import Lasso
2   from sklearn.model_selection import train_test_split
3
4   X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
5   model = Lasso(alpha=0.1)
6   model.fit(X_train, y_train)
7
8   importances = np.abs(model.coef_)
9   most_important = np.argmax(importances)
10  test_mse = mean_squared_error(y_test, model.predict(X_test))

• If the feature mapping $\phi(x)$ is explicitly known, there is no point in using a kernel (dual) representation.TrueFalse
• The function $k(x, x') := \exp(-\lVert x - x' \rVert^2)$ is a valid kernel.TrueFalse
• The function $k(x, x') := x^\top A x'$ with $A = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ and $x, x' \in \mathbb{R}^2$ is a valid kernel.TrueFalse
• Knowing only the Gram matrix $K$ of the training set is enough to compute predictions on a new test point $x^*$ with a dual kernel method.TrueFalse

• If the design matrix $\Phi$ is singular, Ridge Regression (with $\lambda > 0$) and Ordinary Least Squares (OLS) yield the same solution.TrueFalse
• For unregularized OLS, adding additional features to the model cannot increase the training mean squared error.TrueFalse
• When outputs are corrupted by noise, adding regularization can reduce the MSE on the test set compared to OLS.TrueFalse
• Lasso admits a solution that can be computed by inverting $(\Phi^\top\Phi + \lambda I)$.TrueFalse

• Bagging reduces variance mainly by averaging models trained on bootstrap samples, and is most effective with high-variance, low-bias base learners.TrueFalse
• In boosting, the base learners are trained independently and in parallel.TrueFalse
• In bagging, each base model is trained on a bootstrap sample — a dataset of the same size $N$ obtained by sampling the training set with replacement.TrueFalse
• Boosting can reduce the bias of weak learners, but if run for too many rounds it may overfit the training data.TrueFalse

• Even if a hypothesis space $H$ contains infinitely many hypotheses, its VC dimension is always finite.TrueFalse
• The VC dimension of $H$ is at least $k$ if and only if $H$ shatters at least one subset of the instance space of cardinality $k$.TrueFalse
• The VC dimension of a logistic-regression classifier on features $(x_1, x_2)$ is smaller than that of a logistic-regression classifier on $(x_1, x_2, x_1^2, x_2^2, x_1 x_2)$.TrueFalse
• The VC dimension of a hypothesis space depends on the size of the training set.TrueFalse

• In TD(0) policy evaluation, the update target for $V(S_t)$ uses the one-step bootstrap.TrueFalse
• In Monte Carlo policy evaluation, first-visit and every-visit MC converge to the same value function $V^\pi$ (given sufficiently many visits).TrueFalse
• In Q-learning, updating $Q(S_t, A_t)$ does not require sampling the next action $A_{t+1}$ to compute the TD target.TrueFalse
• SARSA can be applied to non-Markovian tasks, since it does not rely on the Markov property.TrueFalse

• In a soft-margin SVM, removing a training point that is not a support vector leaves the decision boundary unchanged.TrueFalse
• The function $k(\mathbf{x}, \mathbf{z}) = (\mathbf{x}^\top\mathbf{z} + 1)^2$ is a valid kernel.TrueFalse
• The function $k(\mathbf{x}, \mathbf{z}) = \mathbf{x}^\top\mathbf{z} - 1$ is a valid kernel.TrueFalse
• For the Gaussian (RBF) kernel $k(\mathbf{x},\mathbf{z}) = \exp(-\gamma\|\mathbf{x}-\mathbf{z}\|^2)$, decreasing the bandwidth (making $\gamma$ large) tends to increase the risk of overfitting.TrueFalse

initialise w = 0
repeat until no mistakes (or max epochs):
  for each sample (x_i, y_i), with y_i in {-1, +1}:
    if y_i * (w . x_i) <= 0:            # sample is misclassified
      w <- w + y_i * x_i                # perceptron update

1   X = zscore(dataset[['sepal-length', 'sepal-width']].values)
2   t = dataset['class'].values == 'Iris-setosa'
3
4   model = LogisticRegression(penalty='none')
5   model.fit(X, t)

1   Q = np.zeros(nS * nA)
2   Q_old = np.ones(nS * nA)
3   V = np.zeros(nS)
4
5   while np.any(Q != Q_old):
6       Q_old = Q
7       for s in range(nS):
8           V[s] = np.max(Q_old[s*nA:(s+1)*nA])
9       Q = R_sa + gamma * P_sas @ V

X = dataset[['sepal-length', 'sepal-width', 'petal-length']].values
t = dataset['petal-width'].values

N, M = X.shape
lambda_ = 0.1

Phi = np.hstack((np.ones((N, 1)), zscore(X)))
w1 = inv(Phi.T @ Phi) @ (Phi.T @ t)
w2 = inv(Phi.T @ Phi - lambda_ * np.eye(M + 1)) @ (Phi.T @ t)
w3 = (np.abs(w1) > 1 / lambda_) * 1 / lambda_ + (np.abs(w1) <= 1 / lambda_) * w1

SVM_model_1 = svm.SVC(kernel="linear")
SVM_model_1.fit(X_train, t_train)
prediction_1 = SVM_model_1.predict(X_test)

def custom_kernel(X_1, X_2):
    A = np.array([[-0.5, 0], [0, 2]])
    return np.dot(np.dot(X_1, A), X_2.T)

SVM_model_2 = svm.SVC(kernel=custom_kernel)
SVM_model_2.fit(X_train, t_train)
prediction_2 = SVM_model_2.predict(X_test)

print(accuracy_score(t_test, prediction_1), accuracy_score(t_test, prediction_2))

• After pulling an arm, the learner gets feedback on the reward from all the available arms.TrueFalse
• An algorithm designed for the MAB setting should minimize the regret suffered during the learning procedure.TrueFalse
• A problem modeled as an MDP with a single state and an infinite number of actions can be handled with MAB techniques.TrueFalse
• UCB1 and TS are designed to solve the same class of problems.TrueFalse

• If the MDP has stochastic rewards, it may not admit a deterministic optimal policy.TrueFalse
• $V^*(s) \ge Q^*(s, a)$ for all states $s$ and actions $a$, where $V^*$ is the optimal value function and $Q^*$ the optimal state-action value function.TrueFalse
• A larger discount factor assigns more importance to the long-term effects of actions.TrueFalse
• Value iteration is guaranteed to converge to the optimal value function in a finite number of steps.TrueFalse

• When the model of the environment is known, it is never convenient to use sample-based reinforcement learning methods.TrueFalse
• SARSA and Q-learning are both off-policy algorithms for control.TrueFalse
• Q-learning can be considered the sample-based version of value iteration.TrueFalse
• SARSA necessitates playing an $\varepsilon$-greedy policy with a vanishing $\varepsilon$ in order to converge to the optimal policy.TrueFalse

• Backward feature selection evaluates a number of models that is at most quadratic in the number of features.TrueFalse
• Forward feature selection is guaranteed to deliver the best subset of features.TrueFalse
• Wrapper methods should use the training error to guide the selection of the features.TrueFalse
• Filter methods are more appropriate than wrapper methods when training the underlying model is expensive.TrueFalse

• SARSA is a prediction method since it makes use of the Bellman expectation equation for updating the value function.TrueFalse
• Q-learning is based on Temporal Difference learning while SARSA is based on Monte Carlo estimation.TrueFalse
• Both SARSA and Q-learning need to update the policy used to collect samples with the $\varepsilon$-greedy policy improvement.TrueFalse
• Differently from SARSA, Q-learning does not need to wait for the action played in the next state to update the value function.TrueFalse

• Keeping $N$ fixed and increasing $d$, the variance of the model reduces.TrueFalse
• Keeping $d$ fixed and increasing $N$, the bias of the model reduces.TrueFalse
• Increasing both $N$ and $d$, the variance of the model increases.TrueFalse
• Decreasing both $N$ and $d$, the bias of the model increases.TrueFalse

• The learned function is a hyperplane in input space.TrueFalse
• OLS admits a closed-form solution.TrueFalse
• The learned model has 3 parameters.TrueFalse
• The overall significance of the model can be established with a single Student's t-test.TrueFalse

x1 = (2, 3, 4)  y1 = 1      x2 = (0, 1, 2)  y2 = 0
x3 = (1, 2, 5)  y3 = 1      x4 = (1, 4, 3)  y4 = 0
x5 = (0, 3, 1)  y5 = 0      x6 = (1, 2, 2)  y6 = 0
x7 = (3, 1, 4)  y7 = 1      x8 = (4, 2, 5)  y8 = 1
x9 = (1, 3, 3)  y9 = 0      x10 = (1, 2, 4) y10 = 1

1  x = zscore(dataset['petal-length'].values).reshape(-1, 1)
2  y = zscore(dataset['class'].values)
3
4  n_samples = len(x)
5  Phi = np.ones((n_samples, 2))
6  Phi[:, 1] = x.flatten()          # the second column is the feature
7  w = inv(Phi.T @ Phi) @ (Phi.T.dot(y))

1   X = zscore(dataset[['sepal-length', 'sepal-width']].values)
2   t = dataset['class'].values == 'Iris-setosa'
3   X, t = shuffle(X, t, random_state=0)
4
5   c1 = LogisticRegression(penalty='none')
6   c1.fit(X, t)
7
8   c2 = GaussianNB()
9   c2.fit(X, t)
10
11  pred1 = c1.predict(X)
12  pred2 = c2.predict(X)
13  pred3 = pred1 * pred2

1   Q = np.zeros(nS * nA)
2   Q_old = np.ones(nS * nA)
3
4   pi = np.zeros((nS, nS * nA))
5   for s in range(nS):
6       pi[s, s*nA:(s+1)*nA] = 1 / nA
7
8   while np.any(Q != Q_old):
9       Q_old = Q
10      Q = (np.eye(nS * nA) - gamma * P_sas @ pi) @ R_sa
11      pi = np.zeros((nS, nS * nA))
12      for s in range(nS):
13          ga = np.argmax(Q[s*nA:(s+1)*nA])
14          pi[s, s*nA+ga] = 1

• Using Maximum Likelihood Estimation to compute the optimal parameter vector $\mathbf{w}^*$ has a closed-form solution, regardless of the noise distribution on the targets $t_i$.TrueFalse
• The least squares estimator $\mathbf{w}^*$ is the one with the lowest Mean Squared Error among all linear unbiased estimators.TrueFalse
• Introducing nonlinear basis functions increases the model flexibility at the cost of reducing its explainability.TrueFalse
• We can use a regression model even if the targets $(t_1,\dots,t_N)$ form a time series (not independent, distribution depending on the index $i$).TrueFalse

pca = PCA()
pca.fit(X, y)
explained = pca.explained_variance
T = pca.transform(X)
explained_variance = np.cumsum(explained) / sum(explained)
T_tilde = T[:, explained_variance < 0.95]

• The perceptron classifier and the logistic regression classifier are both generalized linear models.TrueFalse
• The logistic regression loss function is convex, therefore it admits a closed-form solution of the optimal weights.TrueFalse
• If the training set is linearly separable, the logistic regression classifier and the perceptron classifier converge to the same solution.TrueFalse
• The logistic regression classifier converges to a solution even when the training set is not linearly separable.TrueFalse

• If the VC dimension of a hypothesis space $\mathcal{H}$ is infinite, then $\mathcal{H}$ contains infinitely many hypotheses.TrueFalse
• The VC dimension of a hypothesis space $\mathcal{H}$ is at least $k$ if and only if $\mathcal{H}$ shatters every subset of cardinality $k$ of the instance space.TrueFalse
• The VC dimension of the logistic regression classifier with features $x_1, x_2$ is smaller than that of the logistic regression classifier with features $x_1, x_2, x_1^2, x_2^2, x_1 x_2$.TrueFalse
• The VC dimension of a hypothesis space is affected by the size of the training set, but not by the size of the test set.TrueFalse

x     1    1.5   3     4.5   5.5   6.5
t     2.5  2.5   6     10    10.5  12
f(x)  2    3     6     9     11    13

1   train_accuracy = []
2   test_accuracy = []
3   for k in np.arange(1, 11):
4       knn = neighbors.KNeighborsClassifier(k)
5       knn.fit(X_train, y_train)
6       accuracy = sum(knn.predict(X_train) == y_train) / len(y_train)
7       train_accuracy.append(accuracy)
8       accuracy = sum(knn.predict(X_test) == y_test) / len(y_test)
9       test_accuracy.append(accuracy)
10  selected = np.argmax(test_accuracy)

• Linear regression can be used to model processes showing a non-linear behaviour.TrueFalse
• In the case we have a small number of samples, the use of Bayesian Linear Regression is advised as a substitute for classical Linear Regression.TrueFalse
• After performing Ridge Regression with regularisation parameter $\lambda=k$, the eigenvalues of the matrix $(\Phi^\top\Phi+\lambda I)$ are smaller than or equal to $k$.TrueFalse
• A Gaussian Process can be reformulated as a specific case of Linear Regression over a specific set of features.TrueFalse

• Monte Carlo Policy IterationTrueFalse
• Value IterationTrueFalse
• SARSATrueFalse
• Q-LearningTrueFalse

initialise w = 0
repeat until no mistakes (or max epochs):
  for each sample (x_i, y_i), with y_i in {-1, +1}:
    if y_i * (w . x_i) <= 0:            # sample is misclassified
      w <- w + y_i * x_i                # perceptron update