AI-generated news to wake you up instead of a newspaper? TheGen Ponders

Heard of AI-generated news? Imagine a world where your morning news brief is curated by a super-intelligent AI, delivered straight to your inbox, tailored to your interests.   Well, with the rapid advancements in generative AI, this futuristic scenario is becoming a reality.  But should we embrace this AI-driven revolution, or are there concerns that could jeopardize the quality and objectivity of our news?

What's AI-generated news?

Generative AI, like the popular ChatGPT, has shown remarkable capabilities in generating human-quality text. From writing essays to crafting poetry, these AI models are becoming increasingly sophisticated.  Naturally, the news industry has taken notice. News outlets are experimenting with AI to automate tasks such as writing simple articles, generating headlines, and even translating content. It's difficult to overlook the existential threat that large language models, or LLMs, appear to pose to news organizations. Chatbots have the potential to drastically reduce search traffic for news providers if they are able to immediately respond to user inquiries about news without directing them to news websites. Although LLMs are still not trustworthy sources of news and information due to the problem of hallucinations, products such as SearchGPT and Google's AI Overviews are progressively leading the way in this direction.

The industry appears to be preparing for a paradigm shift that would further undermine newsrooms that depend on digital engagement to survive, as evidenced by the recent spike in news publisher litigation, licensing agreements, and scraping limitations. Though tech businesses have a strong motive to promote this change, it's considerably less clear if consumers of news are genuinely interested in using AI chatbots as a source of information. Do people actually ask ChatGPT from OpenAI what the most recent election news is? Are they checking the scores from the NBA games played yesterday night using Google's Gemini app?

Are LLMs really becoming to become the preferred source for news reading in the real world? Whether users and viewers are interested in using these chat interfaces for news and journalism will ultimately determine how much of an influence they have.

AI-generated news workflows

Several news organizations have already integrated AI into their workflows. For instance, the Associated Press uses AI to generate earnings reports for thousands of publicly traded companies.  The Washington Post has experimented with AI-powered tools to assist reporters in their research. And even the BBC has explored the potential of AI for creating personalized news summaries.

Between April 2023 and April 2024, there were approximately one million anonymised interactions on Wildchat involving users from around 200 countries and two OpenAI models, GPT-3.5 and GPT-4. In return for revealing their chat history, users could access these models without any limitations, giving an overview of how LLM is actually used in the real world. Journalists who have examined the Wildchat dataset have discovered an excessive number of sex-related discussions and homework assistance requests.

After searching the dataset for cases of journalists using an LLM to write and report stories, researchers from the University of Washington discovered multiple examples of searches designed to produce article drafts.

Anecdotally, based on my preliminary examination and investigation of the data, additional significant groups consist of: requests for prompts to be utilized with the image generator Midjourney; translation of text; inquiries concerning coding, statistics, and machine learning; and numerous individuals engaged in creative writing tasks, like screenplays and short stories. It's evident that ChatGPT is used for a variety of purposes and behaviors, but in this case, I want to pay particular attention to spotting news-oriented activity. Specifically, I look at news-related searches to address two main questions:

• How often are users inquiring about news from LLMs?

• What kinds of inquiries do people make about news?

Generative AI chatbots via analytical method

I used a multi-step process to find news inquiries in the dataset in order to provide answers to these questions.

• First filtering - I removed all LLM responses from the entire Wildchat dataset and limited it to English-language messages from users residing in the United States in order to concentrate just on the original question. A selection of over 265,000 messages from roughly 146,000 chats were produced as a result.

  
  

• Annotation by hand - By hand, I went over a sample of 1,000 messages at random and categorized them as news queries or non-news inquiries. I defined news queries as inquiries regarding news sources, general news requests, or questions concerning particular occurrences.

In this sample, non-news query examples included:

"Please send me fifty jokes"

Could you demonstrate how to create an application that lets users draw and animate on transparent 3D planes such that their animations seem to be moving in 3D space?

Determine the amount and characteristics of viral pathogens that adsorb on micro- and nano-plastics and that desorb from them.

Moreover, some instances of news questions were:

• What keeps news anchors and reporters on the air during a SAG-AFTRA strike?

• What significant event occurred on January 23, 2023? On a scale of 1 to 10, indicate how certain you are that your response is accurate.

• What has President Biden stated regarding the relationship between the Russian people and Vladimir Putin?

I only identified eight news queries in this sample.

• Targeted search -I ran targeted keyword searches (e.g., "news," "breaking," "New York Times," "Ukraine") to add more pertinent inquiries to the random sample.

  This led to the creation of 58 more questions, which I classified as news queries. I was left with a sample of 1,058 messages after going over the results: 992 non-news inquiries and 66 news queries.

• LLM evaluation - Since users may inquire about the day's headlines, continuing national stories, or particular local occurrences, it was difficult to distinguish messages that were news-related.

I was unable to depend solely on a single set of search terms or manual evaluation due to the magnitude and diversity of the dataset. For a more in-depth review at scale, I consequently depended on LLM annotation.

Queries via generative AI news

It is evident from these annotations of Wildchat users' ChatGPT interactions that news queries are uncommon. Approximately 5,000 messages, or 1.88% of all communications in the sample, were identified as news-related.

Approximately 1,000 distinct IP addresses (or 7.15 percent of the sample's IP addresses) sent these messages.

This scarcity is consistent with the way that most people get their news online; most people spend very little time reading journalism, and very little of what is shared on social media is news.

However, there is a notable difference among the news questions that are present. LLMs are consulted by users who need assistance navigating different aspects of the news consumption experience.

In ways that are difficult for traditional platforms to duplicate, these queries highlight a number of significant trends in the way that news consumers engage with LLMs.

Even though just a small portion of users as a whole presently engage in these behaviors, they point to potential for future news-oriented solutions that might more directly meet user demands.

Active generative AI news support

It's evident from the typology of user behaviors that was found that many users were actively involved in more complicated tasks like summarizing, translating, and analyzing the news rather than just passively consuming it.

This indicates a change in the way some consumers engage with news content: instead of just reading the news, they are utilizing AI to process, understand, and customize it to suit their specific requirements.

Furthermore, some users treated AI as a medium for "conversational news," incorporating news stories into larger discussions about science, politics, or local happenings.

This is a big change from the old one-way news consumption model since it allows readers to interact with news stories through discussion, analysis, and collaboration.

News organizations face concerns around copyright and ownership of their coverage when users wish to include news items into the context of their interactions.

When combined, these actions show how LLMs can help readers by providing context for ongoing tales or summarizing articles.

They can assist in breaking down difficult ideas into simpler ones or address specific follow-up inquiries.

Alternatively, they can provide clarification on terms like "on background" or advice on how to assess the reliability of sources, or they can offer insights into the reporting process itself.

These use cases also show a conflict in many newsrooms' approaches to generative AI from a design standpoint. When publishers experiment with LLM capabilities, like as translation, they frequently fail to customize these outputs for each reader.

However, the questions presented here imply that readers would benefit more from having more discretion over the context and extent of an LLM's aid.

Therefore, newsrooms need to think about where they want to position their generative AI products in relation to publisher and user control.

AI and media literacy

A significant portion of users sought assistance from LLMs in determining the reliability of news sources and comprehending media bias.

This conduct highlights how important a role LLMs may play in promoting media literacy. LLMs might be created to help users assess the reliability of news reports, include background information on various outlets' points of view, and emphasize the qualities of reliable journalism.

This also includes generally assisting users in comprehending the fundamentals of news production as well as the strengths and weaknesses of the model used to assist in delivering it.

LLMs have the potential to let people to make more educated judgments about the information they rely on by assisting users in better understanding the media that they consume.

AI-generated news benefits

• Efficiency: AI can produce content at a much faster pace than human journalists, allowing for quicker news coverage and updates.

• Accessibility: AI can help break down complex information into more digestible formats, making news more accessible to a wider audience.

• Personalization: AI can tailor news content to individual preferences, ensuring that readers receive information relevant to their interests.

AI-generate news challenges

While the benefits of AI-generated news are undeniable, there are also significant challenges to consider:

• Bias: AI models can perpetuate biases present in the data they are trained on. This could lead to the dissemination of biased or inaccurate information.

• Lack of Human Judgment: AI may struggle to understand nuances and context in the same way that a human journalist can. This could result in missed details or inaccurate reporting.

• Job Displacement: The increased use of AI in journalism could lead to job losses for human journalists.

Generative ai chatbots - the future of news?

As AI technology continues to evolve, it is clear that it will play an increasingly important role in the news industry. 

However, it is crucial to approach this development with caution. While AI can be a valuable tool, it should not replace human judgment and oversight. 

The future of news likely lies in a hybrid approach, combining the efficiency of AI with the expertise and critical thinking of human journalists.

Nevertheless, these results point to important potential directions for LLMs in news consumption. The data highlights a compelling use case for more interactive and tailored news experiences, needs that LLMs can effectively address.

Although there is a genuine risk that LLMs present to news organizations, they also present a once-in-a-lifetime chance.

Even as LLMs have the potential to upend conventional news consumption, news organizations should take advantage of this very potential to better serve their consumers.

According to the data, users expect more than simply fast fixes; they also want more customisation, in-depth interaction, and reliable sources.

News companies may fulfill these changing needs by intelligently utilizing generative AI to create tailored, interactive news experiences that entice readers to return.

So, do you want generative AI chatbots to write news for you?  The answer may depend on how you weigh the benefits and risks.  As with any new technology, it is essential to approach it with a critical eye and a commitment to ensuring that it serves the public interest.