Contractors’ Valentine’s Day Fling With ChatGPT: Ben Van Roo

With Valentine’s Day right around the corner, let’s discuss America’s newest love—ChatGPT. It’s been less than three months since this whirlwind romance began, and we can’t escape the media and mania. It had over a million users in five days after launch and has now exceeded over 100 million users. Microsoft recently invested another $10 billion into ChatGPT’s parent company, OpenAI, and launched its integration with Microsoft Bing.

The excitement is palpable, but does ChatGPT really work? And does it work for the public sector?

Courtesy Sam Lau

Some contracting consultants suggest ChatGPT could be useful for the arduous writing and reporting that comes with performing government work.

Unfortunately, for all intents and purposes, ChatGPT and many other Large Language Models (LLMs) can’t do that particularly well. The idea that you can generate contextual documents into a government form conflates the eventual benefits with what exists today. It ignores the practical limitations of LLMs and the changes that would be required to make these technologies truly useful for the private and public sectors.

Reality Bites

Sam Altman, OpenAI founder, tried to warn us of the limitations a week after ChatGPT’ rollout in a tweet, “ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. It’s a mistake to be relying on it for anything important right now.”

So why can’t the public sector rely on it or other LLMs just yet? There are five practical reasons worth understanding:

• ChatGPT represents language but not knowledge, so don’t trust it. ChatGPT can frequently generate believable text and output, it often spits out “hallucinations”—completely non-factual information. This means any claim needs to be double checked, which reduces the time-saving value of using the AI writer in the first place. In the coming years, when LLMs are more integrated with organization knowledge, the output will be more trustworthy and useful.
• It’s devoid of organizational context. “Write my internal memos!” Sorry. ChatGPT doesn’t know anything about the internal workings of organizations, products and services, RFP templates, much less weekly status reports. Simply put, it has no ability to be connected directly to organizations, so how can it draft any worthwhile internal document? A few large companies and startups are working on better connectivity between LLMs and organizational data, and this will bring the technology steps closer to automated memos, RFPs, and status reports.
• The information that fuels ChatGPT is stale. Try the following prompt in Bing or ChatGPT, “Tell me about the war in Ukraine,” you’ll be stuck with a discussion of the 2014 Russian invasion. ChatGPT has information through 2021, so it has no sense of major geopolitical and economic events or recent technological advancements. Therefore, for any memo or briefing requiring recent information, ChatGPT isn’t going to be helpful. ChatGPT’s successor GPT-4 and Google’s competitive answer Bard, both currently in early stages, should greatly improve on these recency issues.
• Third-party apps raise privacy and security questions. It would in general be a bad idea to share all government contracting knowledge over the internet. To date, there’s no private, on-premise, or laptop version of ChatGPT, and it’s not available on any secure networks. Microsoft will change this eventually, but today it will be difficult for most corporate and government security leaders to sign off on ChatGPT.
• This relationship is going to be expensive. For the public sector to embrace ChatGPT-like technologies, companies will need secure, private instances. Estimated costs to run private instances of ChatGPT are in the millions to tens of millions. That will be a tough pill to swallow for many agencies and organizations.

Happily Ever After

OpenAI/Microsoft, Google, and a bouquet of startups (mine included) are tackling these challenges. These issues will likely be sorted out over the next one to three years. At that point, the efficient composition of writing using generative AI will be specific to organizations, teams, and persons in the public and private sectors. Said another way, the AI will be very, very good at helping people tackle a number of written tasks including RFPs, memos, status reports, tweets, and TPS reports.

In the public sector, these tools will enable real change. Organizational and project knowledge will be aggregated and disseminated faster among primes and subcontractors. Moreover, the composition and responsibilities of capture teams, project staffs, and legal assistants will shift to higher order tasks and more productive teams.

We will also see government agencies invest in privately hosted LLMs and deal with the proliferation of content generated by companies using LLMs. Proposals for research dollars and contracts both large and small could go up by one to two orders of magnitude. Therefore, agencies will look to LLMs for support with summarization, and AI detection tools may be used in specific settings where LLM use is discouraged—say, congressional speeches and briefs to the Supreme Court.

So, enjoy your ChatGPT fling. While it may be somewhat short-lived and ready for a long-term commitment, around the corner a deeper relationship is coming for the public sector, LLMs, and other forms of Generative AI.

Subscribers can find related content at Bloomberg Government .

Author Information

Ben Van Roo is the CEO and co-founder of Yurts Technologies Inc, an enterprise platform for Generative AI models. Ben also sits on the Advisory Council for the Global SOF Foundation, a non-profit organization for the global special operations community. Prior to Yurts, Ben built the National Security teams at Primer Technologies Inc. and was a national security policy researcher at The RAND Corporation. Ben has a PhD in Operations Research from the University of Wisconsin-Madison.

Write for us: Email IndustryVoices@bloombergindustry.com