In training following function was minimized:

$NLL(p(X), y) = \sum_{i=1}^S y_i \log p(x_i) + (1-y_i) \log (1-p(x_i))$ here ys are outcomes and xs are inputs.

Biases, embeddings follow hyperpriors - $w_k \sim N ( \mu, 1 / \lambda)$ $v_{kf} sim N ( \mu, 1 / \lambda)$ $\mu$, $\lambda$ follow hyperpriors - mu sim N(0, 1) lambda sim Gamma (0, 1)

Due to these hyperpriors the authors don't need finetuning.