Compute metrics of the submissions in the history.

Compute metrics of the submissions in the history.

compute_metrics(hist_dir = "history", metrics, y_test, ind_quiz, read_fun)

Arguments

hist_dir	string. directory where the history of the submissions are stored. contains one subdirectory per team.
metrics	named list of functions. Each function in the list computes a performance criterion and is defined as: function(y_pred, y_test)
y_test	character or numeric vector. the test set output.
ind_quiz	indices of y_test in the quiz subset.
read_fun	function that reads a submission file and returns a vector of predictions.

Value

compute_metrics returns a named list with one named member per team. Each member is a data.frame where the rows are the submission files sorted by date and the columns are:

date

the date of the submission

file

the file name of the submission

<metric name>.quiz

the score obtained on the quiz subset

<metric name>.test

the score obtained on the test set