Pandata Blog

Approachable, Ethical, Human-Centered AI

Welcome (back) to Pandata’s annual Holid.AI Greeting Card generator! Just like in 2018 & 2019, we wanted to bring a bit of absurdist humor to your holiday season, but this year we gave our A.I. three big upgrades:

  1. It has studied really hard since last year. Holid.AI_2020 read & learned from over 8 million documents for a total of 40 GB of text from URLs shared in Reddit submissions.
  2. To be funnier, but still remain coherent, it further customized its outputs on 10,000 “one-liner” jokes across a wide range of topics.
  3. It now has eyes.

Some of the most exciting advances in AI such as understanding and generating human speech require extraordinarily large data sets to “learn” from. This requires extreme processing power resulting in very high cost, limiting development of these tools to only the largest companies or research institutes. However, thanks to “transfer learning”, companies like Pandata can leverage the hard work of other great minds who are willing to share their successes with the world. Transfer learning refers to a very powerful technique that takes an existing model and focuses it to a specific task – for example, taking a general-purpose language generator model and using a much smaller set of data to tailor it to make holiday-themed jokes. More broadly, transfer learning allows us to utilize cutting edge AI advances in a way that is both affordable and customizable to the needs of our clients, be it a marketing team or Santa Claus.

To build the Holid.AI Greeting Card generator, we leveraged transfer learning built upon state-of-the-art speech generation models. We leveraged GPT-2, an open-source language generation model built by OpenAI in 2019 (The most recent version GPT-3 is not licensed for public use). Utilizing transfer learning and our set of 10,000 one-liner jokes, we refined the GPT-2 model from talking about anything and everything on the internet into our Holid.AI that only generates short-format jokes and phrases. Compared to previous years, you might notice less holiday specific language and more jokes & punchlines. This will allow us to improve next year’s version of the generator.

Now, about the eyes. Let’s talk about how the AI can see. We knew that we could prime the joke-generating model so that it would create holiday greetings specifically tailored to a particular subject. That is, we could give our bot a list of subjects {snow, Santa, sleigh}, and it would generate greetings based on those keywords.

However, we didn’t have keywords; we had images. So, the question became how do we generate keywords from our images, which could then feed into our NLP (natural language processing) model? We tackled this by using another powerful tool of the Machine Learning world: computer vision. One of the most popular use cases of computer vision is image categorization & captioning. This means the model can automatically count, label, and categorize the objects within that image and return keywords relevant to it.

Computer vision is already being used across a wide variety of industries. Here are a few examples:

  • Manufacturing – Predictive maintenance can identify an issue before any breakdown occurs
  • Digital Marketing – Bypass traditional demographic research and still target the right online audience; also, ensure ads are not placed near content that is contradictory or problematic for its audience
  • Agriculture – Monitor fields for signs of disease or pests so that swift action can be taken to eradicate it

Now, back to our regularly scheduled program. Given a holiday-themed image, we can pass that image to a pre-trained computer vision model that will generate keywords, which are then passed to our custom-trained language model to generate holiday greetings that relate back to the starting image.

Finishing this pipeline is no trivial task. In fact, so much about getting AI to work well is designing the right process from the start, and then connecting all the pieces. Oh, and on the way out, we would be remiss if we didn’t share this bit of holiday wisdom from our robot friend:

Julie Novic was a Data Scientist and Jack Zhang is an intern at Pandata.