- Represent input words are x<1>


- Output to match is y<1>


- Tx denotes length of X.


- Ty denotes length of y.


Sequence Models: Notation

Each word is represented as a vector, with value 1 for the word and 0 for all other words in the dictionary (similar to one-hot encoding)

Representing  Words

The task of predicting what word/letter comes next, unlike the FNN and CNN, in sequence modeling the current output is dependent on the previous a=input and the length of the input is not fixed.


Why sequence models? 

For natural language data composed of discrete tokens like words, sequence models are specifically called language models. They provide the capacity to evaluate the likelihood of sentences, sample new text sequences, and optimize for the most likely outputs. By applying the chain rule of probability, language modeling can be reduced to an autoregressive prediction problem, decomposing the joint density of a sequence $$P(x_1, \ldots, x_T)$$ into the product of conditional densities in a left-to-right fashion: $$P(x_1, \ldots, x_T) = P(x_1) \prod_{t=2}^T P(x_t \mid x_{t-1}, \ldots, x_1)$$. For discrete signals, the autoregressive model acts as a probabilistic classifier, outputting a full probability distribution over the vocabulary for the next word given the leftwards context.

Language Model

Sequence labeling is a machine learning methodology used across many Natural Language Processing (NLP) applications. The core idea is to assign a specific class or tag to each token within a given input sequence. The resulting sequence of labels can then be interpreted to extract linguistic annotations. Common examples of this technique include Part-of-Speech (POS) tagging, where each word is labeled with its grammatical role, and Named Entity Recognition (NER), where tokens are tagged to identify entities like names or locations.

Sequence Labeling

https://www.coursera.org/learn/nlp-sequence-models

University of Michigan - Ann Arbor

Coursera's five-course Deep Learning Specialization covers neural networks, hyperparameter tuning, regularization, optimization, machine-learning project strategy, convolutional neural networks, and sequence models. The courses are Neural Networks and Deep Learning; Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization; Structuring Machine Learning Projects; Convolutional Neural Networks; and Sequence Models. https://www.coursera.org/specializations/deep-learning

Coursera: Deep Learning Specialization

https://www.coursera.org/learn/neural-networks-deep-learning?specialization=deep-learning

Neural Networks and Deep Learning

https://www.coursera.org/learn/deep-neural-network?specialization=deep-learning

Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

https://www.coursera.org/learn/machine-learning-projects?specialization=deep-learning

Structuring Machine Learning Projects

https://www.coursera.org/learn/convolutional-neural-networks?specialization=deep-learning

Convolutional Neural Networks

Sequence Models

https://www.coursera.org/professional-certificates/ibm-data-science
Beginner course

IBM Data Science Professional Certificate

https://www.coursera.org/specializations/jhu-data-science
ten course specialization on coursera offered by JHU staff

Learn Before

Related

Learn After