Controllable Text Generation Using Pre-Trained Language Models
Many controllable text generation methods use autoregressive or sequence-to-sequence pre-trained language models to generate text under a task-specific control condition. Given a control condition and output sequence , generation is modeled as . In an autoregressive model, this distribution is factorized as P(Ymid C)=prod_{n=1}^{N}p(y_nmid y_{1:n-1},C), so each token depends on both the preceding tokens and the control condition.
0
1
Contributors are:
Who are from:
Tags
Deep Learning (in Machine learning)
Data Science
Computing Sciences
Related
Representative Transformer-based PLMs
Analysis of Language Model Training Strategies
A startup is developing a system to classify medical research abstracts into different fields of study (e.g., cardiology, oncology, neurology). They have a limited dataset of 10,000 labeled abstracts. Which of the following statements best justifies the decision to use a large, pre-trained language model and fine-tune it, rather than training a new model from scratch on their dataset?
A development team is building a system to classify news articles into categories like 'Sports', 'Technology', and 'Politics'. They are using a modern approach that starts with a large, general-purpose language model. Arrange the following stages of their development process into the correct chronological order.
Traditional Role of Language Models
LLMs as Complete Systems in Generative AI
Controllable Text Generation Using Pre-Trained Language Models
Controllable Text Generation Using Pre-Trained Language Models