Compute metrics of the submissions in the history.

compute_metrics(hist_dir = "history", metrics, y_test, ind_quiz, read_fun)

Arguments

hist_dir
string. directory where the history of the submissions are stored. contains one subdirectory per team.
metrics
named list of functions. Each function in the list computes a performance criterion and is defined as: function(y_pred, y_test)
y_test
character or numeric vector. the test set output.
ind_quiz
indices of y_test in the quiz subset.
read_fun
function that reads a submission file and returns a vector of predictions.

Value

compute_metrics returns a named list with one named member per team. Each member is a data.frame where the rows are the submission files sorted by date and the columns are: