Case Study

Calculating an Output Vector in a Simple Sequence Model

A simple sequence processing model generates an output vector, yiy_i, for each input vector, xix_i, in a sequence. The output yiy_i is a weighted sum of all input vectors up to and including xix_i. The weights are determined by a three-step process:

  1. Scoring: A score is calculated between the current input xix_i and each preceding input xjx_j (where jij \le i) using a dot product: score(xi,xj)=xixjscore(x_i, x_j) = x_i \cdot x_j.
  2. Normalization: These scores are converted into weights, αij\alpha_{ij}, by applying a softmax function across all jij \le i.
  3. Output Calculation: The output yiy_i is calculated as the weighted sum: yi=jiαijxjy_i = \sum_{j \le i} \alpha_{ij} x_j.

Your task is to apply this process to calculate the final output vector, y3y_3, for the given sequence.

0

1

Updated 2025-09-26

Contributors are:

Who are from:

Tags

Data Science

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Application in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science