1. Knowledge State Encoder: The knowledge state vector is generated as:

$h_t = RNN(x_t)$

$KS_t = W_{h, KS} \cdot h_t + b_{h, KS}$

LSTMs and GRUs were used as RNN variants.

2. Skill Encoder: $e_{t+1}$ is embedded to $s_{t+1}$ as:

$o_{t+1} = ReLU (W_1 \cdot (ReLU(W_0 \cdot e_{t+1} + b_0 )) + b_1)$

$s_{t+1} = L2-normalize(o_{t+1})$

s_{t+1} in U^d = { v in mathbb{R}^d : ||v|| = 1, v^i ge 0 quad i = 1, dots , d }