Compute metrics of the submissions in the history.
compute_metrics(hist_dir = "history", metrics, y_test, ind_quiz, read_fun)
- string. directory where the history of the submissions are stored.
contains one subdirectory per team.
- named list of functions. Each function in the list computes
a performance criterion and is defined as:
- character or numeric vector. the test set output.
- indices of
y_test in the quiz subset.
- function that reads a submission file and returns a vector of predictions.
compute_metrics returns a named list with one named member per team.
Each member is a
data.frame where the rows are the submission files sorted by date
and the columns are: