1. 导言

自然言语生成 (即文本生成) 是自然言语处理 (NLP) 的核心任务之一。本文将介绍神经网络文本生成范畴当时最先进的解码办法 比照查找 (Contrastive Search)。提出该办法的论文 “A Contrastive Framework for Neural Text Generation” 最初发表于 NeurIPS 2022 ([论文]、[官方完成])。此后， “Contrastive Search Is What You Need For Neural Text Generation” 的作者又进一步证明了比照查找能够用 现有的 言语模型在 16 种言语上生成可媲美人类水平的文本 ([论文]、[官方完成])。

[补白] 对于不熟悉文本生成的用户，请参阅 此博文 了解更多详情。

2. Hugging Face 比照查找演示

目前， transformers 的 PyTorch 和 TensorFlow 后端均支撑比照查找。你能够在 该 Colab notebook 中根据不同的后端挑选相应的部分来探究该办法，文章顶部也有该 notebook 链接。咱们还构建了这个不错的 演示运用，用它能够直观地比较比照查找与其他流行的解码办法 (例如波束查找、top-k 采样 [3] 以及核采样 [4])。

3. 环境装置

在进行后续实验前，咱们要先装置最新的 transformers 库，如下:

pip install torch
pip install "transformers==4.24.0"

4. 现有解码办法存在的问题

解码办法能够分为两类: (i) 确定性办法，(ii) 随机办法。下面咱们分别对两者进行评论！

4.1. 确定性办法

确定性办法，如贪心查找和波束查找，经过在言语模型输出的一切候选补全词中挑选概率最高的词来生成最终文本。但是，正如之前研究 [3][4] 指出的，确定性办法通常会导致 模型退化，即生成的文本不自然且包含不必要的重复。

下面，咱们看一个用 GPT-2 模型和贪心查找生成文本的比如。

from transformers import AutoTokenizer, GPT2LMHeadModel
tokenizer = AutoTokenizer.from_pretrained('gpt2-large')
input_ids = tokenizer('DeepMind Company is', return_tensors='pt').input_ids
model = GPT2LMHeadModel.from_pretrained('gpt2-large')
output = model.generate(input_ids, max_length=128)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: 
Output:
----------------------------------------------------------------------------------------------------
DeepMind Company is a leading AI research company, with a focus on deep learning and deep learning-based systems.
The company's research is focused on the development of deep learning-based systems that can learn from large amounts of data, and that can be used to solve real-world problems.
DeepMind's research is also used by the UK government to develop new technologies for the UK's National Health Service.
DeepMind's research is also used by the UK government to develop new technologies for the UK's National Health Service.
DeepMind's research is also used by the UK government to develop new technologies
----------------------------------------------------------------------------------------------------

[补白] 咱们能够看到，贪心查找生成的成果中有显着的重复。

4.2. 随机办法

为了处理确定性办法带来的问题，随机办法经过在解码过程中引入随机性来生成文本。常用的两种随机办法是 (i) top-k 采样 [3] 和 (ii) 核采样 (也称为 top-p 采样) [4]。

下面，咱们给出用 GPT-2 模型和核采样 (p=0.95) 生成文本的示例。

import torch
from transformers import AutoTokenizer, GPT2LMHeadModel
tokenizer = AutoTokenizer.from_pretrained('gpt2-large')
input_ids = tokenizer('DeepMind Company is', return_tensors='pt').input_ids
model = GPT2LMHeadModel.from_pretrained('gpt2-large')
torch.manual_seed(0.)
output = model.generate(input_ids, do_sample=True, max_length=128, top_p=0.95, top_k=0)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: 
Output:
----------------------------------------------------------------------------------------------------
DeepMind Company is a leading provider of AI-based research, development, and delivery of AI solutions for security, infrastructure, machine learning, communications, and so on."
'AI is not journalism'
Worse still was the message its researchers hoped would reach the world's media — that it was not really research, but rather a get-rich-quick scheme to profit from living forces' ignorance.
"The thing is, we know that people don't consciously assess the value of the others'
information. They understand they will get the same on their own."
One example? Given the details of today
----------------------------------------------------------------------------------------------------

[补白] 虽然核采样能够生成没有重复的文本，但生成文本的语义一致性并不是很好。例如，生成的短语 ‘AI is not journalism’ 与给定的上文即 ‘DeepMind Company’ 不一致。

咱们注意到，这种语义不一致的问题能够经过下降温度 (temperature) 来部分处理。但是，下降温度会使核采样更接近贪心查找，这其实就变成了贪心查找和核采样之间的权衡。一般来讲，要找到一个既能防止贪心查找又能防止核采样圈套的方便且与模型无关的温度适当有挑战。

5. 比照查找

本节咱们来具体介绍一种新的解码办法， _ 比照查找_。

5.1. 解码方针

给定前缀文本 x<tx_{< t}，咱们按如下公式挑选输出词元 xtx_{t}:

在 Transformers 中使用对比搜索生成可媲美人类水平的文本

上式中， V(k)V^{(k)} 是言语模型输出概率散布 p(v∣x<t)p_{\theta}(v|x_{< t}) 中 k 个概率最大的候选词元的集合。榜首项，即 模型置信度 (model confidence)，是言语模型猜测的每个候选词元 vv 的概率。第二项， 退化赏罚 (degeneration penalty)，用于度量 vv 与上文 x<tx_{< t} 中每个词元的相异度，其中函数 s(⋅,⋅)s(\cdot, \cdot) 用于计算每两个词元间的余弦类似度。更具体地说，退化赏罚被定义为 vv 的向量表征 hvh_{v} 与其上文 x<tx_ {< t} 中每个词元的向量表征间余弦类似度的最大值。这儿，候选词元的向量表征 hvh_{v} 是在给定 x<tx_{< t} 和 vv 的条件下将二者连接起来输入给言语模型，然后由言语模型计算出来的。直观上，假如 vv 的退化赏罚较大意味着它与上文更类似 (在表示空间中)，因而更有或许导致模型退化问题。超参数 \alpha 用于在这两项中折衷。当 =0\alpha=0 时，比照查找退化为纯贪心查找。

[补白] 在生成输出时，比照查找一起考虑 (i) 言语模型猜测的概率，以坚持生成文本和前缀文本之间的语义连接性; (ii) 与上文的类似性以防止模型退化。

5.2. 运用比照查找生成文本

下面，咱们运用与 第 4.1 节 和 第 4.2 节 中相同的前缀文本 (即 “DeepMind Company is” )，并运用比照查找生成文本 (取 k=4、=0.6\alpha=0.6)。为了充分展现比照查找的卓越才能，咱们让言语模型生成一个 512 词元的 长文档，如下:

from transformers import GPT2Tokenizer, GPT2LMHeadModel
model_name = 'gpt2-large'
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2LMHeadModel.from_pretrained(model_name, pad_token_id=tokenizer.eos_token_id)
model.eval()
# prepare the prefix
prefix_text = r'DeepMind Company is'
input_ids = tokenizer(prefix_text, return_tensors='pt').input_ids
# generate the result with contrastive search
output = model.generate(input_ids, penalty_alpha=0.6, top_k=4, max_length=512)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')

参数设置如下:

  • --top_k: 比照查找中的超参 kk。
  • --penalty_alpha: 比照查找中的超参 \alpha。
模型输出: 
Output:
----------------------------------------------------------------------------------------------------
DeepMind Company is a leader in artificial intelligence (AI). We have a long history of working with companies such as Google, Facebook, Amazon, and Microsoft to build products that improve people's lives, and today we are excited to announce that DeepMind's AlphaGo program has won the game of Go, becoming the first program to defeat a professional Go player.
The victory is a testament to the power of deep learning, and to the incredible work of our research team, which has been at the forefront of AI research for the past five years. AlphaGo is one of the most advanced Go programs ever created, and its performance is an important step towards the goal of human-level AI.
"This is the culmination of a decade of hard work," said Andy Ng, co-founder and CTO of DeepMind. "We are thrilled to have achieved this milestone and look forward to continuing to develop AI that can be used in a wide range of applications and to help people live better lives."
DeepMind's work on Go began in 2010, when it began to train a neural network to play Go using millions of games played by top Go players around the world. Since then, the team has refined the algorithm, adding more and more layers of reinforcement learning to make it better at recognizing patterns and making decisions based on those patterns. In the past year and a half, the team has made significant progress in the game, winning a record-tying 13 games in a row to move into the top four of the world rankings.
"The game of Go is a complex game in which players have to be very careful not to overextend their territory, and this is something that we have been able to improve over and over again," said Dr. Demis Hassabis, co-founder and Chief Scientific Officer of DeepMind. "We are very proud of our team's work, and we hope that it will inspire others to take the next step in their research and apply the same techniques to other problems."
In addition to the win in Go, DeepMind has also developed an AI system that can learn to play a number of different games, including poker, Go, and chess. This AI system, called Tarsier, was developed in partnership with Carnegie Mellon University and the University of California, Berkeley, and is being used to teach computer vision and machine learning to identify objects in images and recognize speech in natural language. Tarsier has been trained to play the game of Go and other games on a
----------------------------------------------------------------------------------------------------

[补白] 咱们看到生成的文本质量十分高。整个文档语法流畅，语义连接。一起，生成的文本也很好地坚持了事实的正确性。例如，在榜首段中，它正确论述了 “AlphaGo” 作为 “榜首个打败作业围棋选手的程序” 这一事实。

5.3. 比照查找的成果可视化

为了更好地了解比照查找的作业原理，咱们对贪心查找 ( 第 4.1 节 ) 和比照查找进行了直观比较。具体来说，咱们分别将贪心查找和比照查找生成的词元类似度矩阵可视化。两个词元之间的类似性被定义为它们的向量表征 (即最终一个转换器层的隐藏状态) 之间的余弦类似性。贪心查找 (上) 和比照查找 (下) 的成果如下图所示。

在 Transformers 中使用对比搜索生成可媲美人类水平的文本
在 Transformers 中使用对比搜索生成可媲美人类水平的文本

[补白] 从贪心查找的成果中，咱们看到非对角线的类似度很高，这清楚地表明贪心查找产生了重复。相反，在比照查找的成果中，高类似度分数首要出现在对角线上，这证明咱们成功处理了退化问题。比照查找的这一优良特性是经过在解码过程中引入退化赏罚 (参见 第 5.1 节 ) 来完成的。

6. 更多的生成示例

在本节中，咱们提供了更多的生成示例来比较不同的解码办法。

6.1. 示例一: GPT-2

在这部分中，咱们运用 GPT-2 生成文本，其前缀文本来自 OpenAI 发布 GPT-2 的博客文章。

In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.

加载言语模型并预备前缀文本: 
import torch
from transformers import AutoTokenizer, GPT2LMHeadModel
tokenizer = AutoTokenizer.from_pretrained('gpt2-large')
model = GPT2LMHeadModel.from_pretrained('gpt2-large')
prefix_text = r"In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English."
input_ids = tokenizer(prefix_text, return_tensors='pt').input_ids
6.1.1. 运用贪心查找生成文本
代码: [点击打开] 
output = model.generate(input_ids, max_length=512)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: [点击打开] 
Output:
----------------------------------------------------------------------------------------------------
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
The researchers, led by Dr. David R. Williams of the University of California, Santa Cruz, discovered the unicorns in the Andes Mountains of Peru. The area is known for its unique geology and is home to a number of rare species of animals.
The researchers found the unicorns in the Andes Mountains of Peru.
"We were surprised to find that the unicorns were able to communicate with each other," Williams said. "We were also surprised to find that they were able to communicate in English."
The researchers believe that the unicorns are descendants of the ancient Incas, who lived in the area around 2,000 years ago.
"The Incas were the first people to use the Andes Mountains as a refuge from the harsh climate of the Andes," Williams said. "They were also the first people to use the Andes Mountains as a place to hunt and gather food."
The researchers believe that the unicorns are descendants of the ancient Incas, who lived in the area around 2,000 years ago.
"The Incas were the first people to use the Andes Mountains as a refuge from the harsh climate of the Andes," Williams said. "They were also the first people to use the Andes Mountains as a place to hunt and gather food."
The researchers believe that the unicorns are descendants of the ancient Incas, who lived in the area around 2,000 years ago.
"The Incas were the first people to use the Andes Mountains as a refuge from the harsh climate of the Andes," Williams said. "They were also the first people to use the Andes Mountains as a place to hunt and gather food."
The researchers believe that the unicorns are descendants of the ancient Incas, who lived in the area around 2,000 years ago.
"The Incas were the first people to use the Andes Mountains as a refuge from the harsh climate of the Andes," Williams said. "They were also the first people to use the Andes Mountains as a place to hunt and gather food."
The researchers believe that the unicorns are descendants of the ancient Incas, who lived in the area around 2,000 years ago
----------------------------------------------------------------------------------------------------
6.1.2. 运用核采样生成文本
代码: [点击打开] 
torch.manual_seed(0.)
output = model.generate(input_ids, do_sample=True, max_length=512, top_p=0.95, top_k=0)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: [点击打开] 
Output:
----------------------------------------------------------------------------------------------------
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. The study was published in the Journal of Zoology in March 2016.
Polygynous mammals such as unicorns have remained largely unknown to science. Professor Gustavo Giacota, from the University of Oxford who led the study, said that they had been documented as far as Eastern Siberia in Russia, but had only been seen a handful of times in the Gobi Desert.
Tiny animals with pale and shiny coats live in the presence of human beings and are hardly likely to be victims of any cruelty. However, there is some evidence of the condition occurring in both humans and animals in remote regions, which might have similarities to "black moles" that coexist on the skin.
It is thought that Unicorns could be inside themselves, that they have different scents depending on their current environment, or just fall out and there are plenty of legends of how they have survived. Experts speculate that the moths and other animals could be remnants of the Yezidi Isis and Charon, which literally is both the word which means great bird, and the Greek word for sound. It is said that the Isis and Charon taught their young the use of voice in the form of calling out to others.
The scientists think that it could be ancient folklore that has survived and is no longer attributed to a real entity
----------------------------------------------------------------------------------------------------
6.1.3. 运用比照查找生成文本
代码: 
output = model.generate(input_ids, max_length=512, penalty_alpha=0.6, top_k=4)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: 
Output:
----------------------------------------------------------------------------------------------------
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
According to the BBC, a team of scientists led by Dr David MacKay, from the University of Bristol, spent two years searching for the unicorn herd, which they discovered during a survey of the area.
"It's a very rare find," MacKay told the BBC. "There are a few in the Himalayas, but this is the first time we've been able to find one in such a remote area."
The team was surprised to find a herd of unicorns living in a region that has been known to be a hotbed of poaching, with many of the animals poached for their horns, which are used in traditional Chinese medicine to treat everything from rheumatism to cancer.
"We knew that the area was rich in rhino horn, but we had no idea how many there were, or what they were doing there," MacKay said. "This is an area of high poaching pressure, and we wanted to find out what was going on."
In order to do so, the team used GPS collars to track the animals as they moved around the mountain and the surrounding area. The GPS data was then compared with information gathered from local villagers, who had a wealth of information about the animals' movements, including where they were eating, what they were doing at night, and how much time they spent in the mountains each day.
After analyzing the data, the team determined that the herd consisted of at least three species of unicorns, including a male and two females. One of the females was the mother of the male, and the other two were her daughters. All three had the same horn color, which is believed to be a sign of purity in the animal kingdom.
While the discovery is exciting, it's not the first time scientists have discovered an animal that speaks English. Last year, scientists discovered a species of porcupine that can be heard by humans, and has been dubbed "Porcupine Man" for his ability to converse with the human race.
----------------------------------------------------------------------------------------------------

6.2. 示例二: OPT

本节中，咱们运用 Meta 最近发布的 OPT 模型 [5]，并从闻名的 ResNet 论文 [6] 摘要中选取前两个句子来作为前缀文本。

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously.

加载言语模型并预备前缀文本: 
import torch
from transformers import AutoTokenizer, OPTForCausalLM
model_name = r'facebook/opt-1.3b'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = OPTForCausalLM.from_pretrained(model_name)
prefix_text = r"Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously."
input_ids = tokenizer(prefix_text, return_tensors='pt').input_ids
6.2.1. 运用贪心查找生成文本
代码: [点击打开] 
output = model.generate(input_ids, max_length=256)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: [点击打开] 
Output:
----------------------------------------------------------------------------------------------------
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We show that the residual learning framework can be used to train deep neural networks that are significantly more difficult to train than those used previously. We also show that the residual learning framework can be used to train deep neural networks that are significantly more difficult to train than those used previously.
The paper presents a new residual learning framework for deep neural networks that is based on the concept of residuals. The residuals are the residuals of the network that are not used in the training process. The residuals are computed by taking the residuals of the network that are used in the training process and subtracting the residuals of the network that are not used in the training process. The residuals are then used to train the network. The residuals are computed by taking the residuals of the network that are used in the training process and subtracting the residuals of the network that are not used in the training process. The residuals are then used to train the network. The residuals are computed by taking the residuals of the network that are used in the training process and subtracting the residuals of the
----------------------------------------------------------------------------------------------------
6.2.2. 运用核采样生成文本
代码: [点击打开] 
torch.manual_seed(0.)
output = model.generate(input_ids, do_sample=True, max_length=256, top_p=0.95, top_k=0)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: [点击打开] 
Output:
----------------------------------------------------------------------------------------------------
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. The theory focuses on several aspects of learning, including the dynamics of replicative and non-replicative aspects of learning. This framework emphasizes learning by entropy. New randomized algorithms enable training networks with residual learning, so that deep networks can be deployed as reliably and as efficiently as their more conventional counterparts.
----------------------------------------------------------------------------------------------------
6.2.3. 运用比照查找生成文本
代码: 
output = model.generate(input_ids, max_length=256, penalty_alpha=0.6, top_k=6)
print("Output:\n" + 100 *'-')
print(tokenizer.decode(output[0], skip_special_tokens=True))
print("" + 100 *'-')
模型输出: 
Output:
----------------------------------------------------------------------------------------------------
Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously.
In this paper, we propose a model-based residual learning (MBRL) framework that is based on neural networks trained on data that is sparse in terms of dimensionality (e.g., 1, 2, 3, etc.). The network parameters are chosen such that there is a high probability of convergence, i.e., the number of iterations is large enough to minimize the variance of the residuals. This is achieved by training the network on a set of training data, in which the data is sparse in terms of dimensionality, and then discarding the nonparametric part of the data after training is complete.
We show that MBRL outperforms other methods for deep reinforcement learning (RL) and deep convolutional neural networks (CNNs) by a factor of at least 2. In addition, we show that, compared to CNNs, MBRL performs better in two-dimensional (2D) and three-dimensional (3D) cases.
----------------------------------------------------------------------------------------------------

7. 更多资源

有关比照查找的更多具体信息，请查看咱们的论文和代码，如下:

  • A Contrastive Framework for Neural Text Generation: 论文、官方完成
  • Contrastive Search Is What You Need For Neural Text Generation: 论文、官方完成

8. 引用

@inproceedings{su2022a,
   title={A Contrastive Framework for Neural Text Generation},
   author={Yixuan Su and Tian Lan and Yan Wang and Dani Yogatama and Lingpeng Kong and Nigel Collier},
   booktitle={Advances in Neural Information Processing Systems},
   editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
   year={2022},
   url={https://openreview.net/forum?id=V88BafmH9Pj}
}
@article{su2022contrastiveiswhatyouneed,
  title={Contrastive Search Is What You Need For Neural Text Generation},
  author={Su, Yixuan and Collier, Nigel},
  journal={arXiv preprint arXiv:2210.14140},
  year={2022}
}

参考文献

[1] Su et al., 2022 “A Contrastive Framework for Neural Text Generation”, NeurIPS 2022

[2] Su and Collier, 2022 “Contrastive Search Is What You Need For Neural Text Generation”, Arxiv 2022

[3] Fan et al., 2018 “Hierarchical Neural Story Generation”, ACL 2018

[4] Holtzman et al., 2020 “The Curious Case of Neural Text Degeneration”, ICLR 2020

[5] Zhang et al., 2022 “OPT: Open Pre-trained Transformer Language Models”, Arxiv 2022

[6] He et al., 2016 “Deep Residual Learning for Image Recognition”, CVPR 2016

– 本文由 Yixuan Su 和 Tian Lan 撰写

致谢

咱们要感谢 Joao Gante (@joaogante)、Patrick von Platen (@patrickvonplaten) 和 Sylvain Gugger (@sgugger)，感谢他们在咱们将本文中的比照查找集成进 transformers 库的过程中给予的帮助和辅导。

英文原文: hf.co/blog/introd…

原文作者: Tian Lan

译者: Matrix Yao (姚伟峰)，英特尔深度学习工程师，作业方向为 transformer-family 模型在各模态数据上的运用及大规模模型的训练推理。

审校/排版: zhongdongy (阿东)