Automatic Stylised Text Generation

Abstract

Platforms such as Google and Facebook make it accessible for small businesses to launch their own advertising campaigns. However, without a well-written ad, a campaign is unlikely to perform well. Many small business owners are not skilled at writing ads, and hiring professional marketers is expensive. To solve this issue, we present a deep learning-based system to produce advertisements automatically, using solely the website of the product or service as an input. We approach this task in a more general setting, namely producing stylistic summaries.

Starting with a piece of text, our framework can be used to summarize it such that the output adheres to a set of desired styles, for example a certain sentiment or a specific vocabulary. These styles can be modified at run-time, meaning that our system is capable of generating various types of output without requiring any retraining. Moreover, it can produce multiple diverse generations for each style requested. At the heart of the model is GPT-2, a pre-trained Transformer model that allows us to leverage transfer learning.

We apply the framework to two tasks: generating product advertisements and customer reviews. A user study we conducted on the former concludes that the participants usually prefer the ads produced by our model over those written by a human marketer. Success in the review generation task shows that our framework is capable of generalizing to a multitude of different applications.

Nicholas Dykeman

Master's Thesis