Could An OpenAI Data Breach Expose Your Firm’s Secrets?

The use of generative AI tools like OpenAI’s ChatGPT has exploded across the enterprise in the last two years. From marketing content and coding to customer service and internal comms, these platforms are reshaping how everyday work gets done. But alongside the efficiency gains come growing risks, especially when it comes to data security and privacy.

In many cases, employees are inputting sensitive business information into AI tools with little understanding of where that data goes or how it’s stored. With so much potentially confidential content flowing through external systems, a single data breach at one of these services could expose everything from trade secrets to customer data.

The potential to be impacted by a third-party data breach is exacerbated when individuals are working outside of the IT department’s guidelines. While employees increasingly use tools like ChatGPT to speed up daily tasks such as drafting emails, summarizing meetings and writing code, many are doing so through personal accounts rather than company-managed platforms, with no oversight or approval from IT, which can result in serious privacy concerns.

According to research by Harmonic, for instance, 45.4 percent of sensitive data prompts entered into large language model platforms are submitted via personal accounts, bypassing corporate controls entirely. The study also gave OpenAI – one of the most widely used platforms – a ‘D’ grade for its cybersecurity controls and noted that it accounted for the most reported breaches – 1,140 incidents. These figures highlight the growing security risks posed by ‘shadow AI’. If one of these services is compromised, confidential business data could be exposed, and organizations may never know until it’s too late.

As one of the most widely used generative AI platforms globally, OpenAI has become a high-value target for cybercriminals. With millions of users, including businesses that regularly upload sensitive data, even a small breach could have far-reaching consequences.

The prospect of data breaches affecting generative AI services isn’t hypothetical. In 2023, for instance, OpenAI temporarily took down ChatGPT after discovering a bug that caused some users to view data belonging to other users. The exposed information included chat titles, first messages and, in a small number of cases, first and last names, email addresses, payment addresses and the last four digits of credit cards. While OpenAI fixed the error and issued an investigation summary saying the problem had been resolved, it highlights the risks involved with entrusting sensitive data to these services.

More concerns were raised in February 2025, when an underground forum post alleged that OpenAI had suffered a data breach involving roughly 20 million account credentials, including passwords and email addresses. However, the company stated it found no evidence of an internal system compromise and it was eventually suggested by cybersecurity researchers that login credentials for OpenAI had actually been gathered from other data breaches, where individuals had reused passwords on their ChatGPT accounts.

Despite this, the claim garnered significant attention because ChatGPT’s business‑user base has grown dramatically since 2023, meaning any large‑scale breach could expose vast amounts of corporate data submitted to the platform.

The confirmed and claimed breaches involving OpenAI highlight just how attractive generative AI platforms have become to threat actors. These tools process enormous volumes of user-submitted data, often from enterprise users who may be unaware of the risks. Cybercriminals therefore see these platforms as single points of access to highly valuable corporate intelligence.

Common types of data shared with GenAI platforms that could prove valuable in the event of a breach include:

• Source code or technical documentation
• Customer names and contact details
• Legal contracts or compliance records
• Internal strategy documents or memos
• Credentials or login instructions

Many GenAI tools retain user inputs that include such information by default, unless organizations take specific action to disable this. That means even seemingly low-risk prompts could be stored, reused or potentially accessed during a breach.

Without proper governance, GenAI platforms become blind spots in the cybersecurity stack, housing sensitive data beyond the organization’s visibility or control.

As the use of generative AI grows, businesses must assess the specific risks that come with using third-party tools like ChatGPT, especially when sensitive or regulated data is involved. Without strong policies and controls, even well-intentioned use can lead to serious security and compliance issues. To reduce your exposure, consider the following steps:

• Create clear AI usage policies: Define what can and cannot be shared with GenAI tools.
• Restrict unapproved platforms: Block access to public AI platforms that lack enterprise controls.
• Use privacy-first alternatives: Choose tools with zero-retention settings and enterprise-grade security.
• Train employees regularly: Ensure staff understand the risks of inputting sensitive data.
• Monitor usage across the business: Track how and where GenAI is used to detect shadow activity.

AI can be transformative. But it must be deployed with care, awareness and the right safeguards in place.