Stylized Natural Language Generation in Dialogue Systems

This thesis explores approaches of generating natural language in various styles. In addition, it also explores the model's ability to control the strength of style manifestation in a generated sequence. The model for natural language generation was implemented with several aspects of style manifestation, namely poetry, humor, sentiment and specificity. Beam search and Nucleus sampling were used as decoding strategies of language models. The proposed experiments are based on weighted decoding. In particular, the probability mass function computed via language model that generates response is modified via two approaches. The first approach uses manually crafted features, such as NIDF. The second one uses neural probabilistic language models trained on stylistic datasets. The architecture of the model is presented in two versions. The first one is a LSTM-based baseline and the second one uses state-of-the-art pre-trained models BART and GPT-2 for text generation. The experiments have revealed the problem that even current state-of-the-art models suffer from poor trade-off estimation between the style and the context. In other words, the more the style is manifested in the generated sequence, the less it relates to the topic discussed in the dialogue.

Artificial neural network, Natural language generation, Dialogue systems, seq2seq, BART, GPT-2, LSTM