Stand-up comedians were chosen to test LLMs because they are experts in creating comedy, understanding humor, and have experience in engaging audiences. Their skills in writing and performing stand-up routines, which require storytelling, jokes, and humorous situations, make them ideal candidates to evaluate the effectiveness of LLMs in generating funny and engaging content.
The publication of the study on arXiv highlights the growing interest in exploring the capabilities and limitations of language models (LLMs) in creative tasks such as joke writing. It also demonstrates the potential for AI-generated content to serve as a starting point for human creativity, while emphasizing the need for human input to ensure quality and relevance.
The jokes created by LLMs lacked humor because they were generic and lacked surprise elements. The comedians found the AI-generated jokes to be bland and lacking the cutting edge typically needed for a joke to be funny. This is because LLM makers use filters to prevent them from generating output that could be offensive or edgy.