The distillation loss for relation-based knowledge transfer, based on the relations of feature maps, is calculated as:

L_{RelD}(f_t, f_s) = L_{R^1}(psi_t(acute{f_t}, check{f_t}), psi_s(acute{f_s}, check{f_s}))

Where:

• $f_t$ and $f_s$ are feature maps of the teacher and student models, respectively.
• $\acute{f_t}$ and $\check{f_t}$ are pairs of feature maps chosen from the teacher.
• $\acute{f_s}$ and $\check{f_s}$ are pairs of feature maps chosen from the student.
• $\psi_t(\cdot)$ and $\psi_s(\cdot)$ are similarity functions for pairs of feature maps from the models.
• $L_{R^1}(\cdot)$ is the correlation function between the teacher and student feature maps.