On a Friday afternoon, a sales employee pastes a customer complaint, complete with name, address and order history, into his private ChatGPT account so the tool can draft a friendly reply. The reply is good, the complaint is settled. What remains is a chat history full of customer data, held by a provider in the United States, that nobody in the company knows about.

Last week, this column covered the rules: which obligations under the GDPR and the AI Act have applied side by side since 2 August 2026 when staff enter company data into ChatGPT. This second and, for now, final part of our short AI series turns the perspective around and asks where implementation fails in practice. The answer is rarely bad intent. As early as the beginning of 2024, Cisco’s Data Privacy Benchmark Study found that 48 per cent of the professionals surveyed had entered non-public company information into generative AI tools, and 45 per cent had entered information about employees. At the same time, 27 per cent of organisations had banned the tools outright, at least for the time being. Both figures describe the same problem: the tool is being used, just without rules. Five mistakes come up again and again.

Mistake one: customer data in a private or free account

The most expensive mistake is the most everyday one, and it is already in the example above. As soon as customer data lands in a private or free account, three things happen:

• The personal data sits with a provider in the United States, without a data processing agreement and without a legal basis.
• In the consumer products, inputs feed into model training by default unless the user switches that off themselves.
• When the employee leaves, the entire chat history, customer data included, simply leaves with them.

The liability sits not with the employee but with the company. Article 83 GDPR provides for fines of up to 20 million euros or 4 per cent of worldwide annual turnover, whichever is higher, and the people affected can claim damages under Article 82.

Mistake two: business plan bought, contract never concluded

Anyone paying for ChatGPT Business (called “ChatGPT Team” until August 2025) or Enterprise has taken the first step, but does not yet have a contract:

• The Data Processing Addendum that OpenAI provides for these plans and for the API has to be actively accepted by an administrator. Only then does the data processing agreement under Article 28 GDPR exist, which every processing of personal data by a service provider requires.
• Part of this is adding the tool to the record of processing activities under Article 30.
• Activating the EU data residency offered to business customers lowers the risk of the third-country transfer but does not remove it, because the provider remains a US company.

Mistake three: a ban instead of a rule

The obvious reaction of many managing directors is a ban. In practice, it only shifts the use: instead of working in the company account, staff work in a private account on their own phone, with no central settings, no control over deletion and nobody in the building knowing which data goes where. The company trades a manageable risk for an uncontrollable one. The first step is therefore not a technical question but an inventory: who uses which tool, with which data, in which account?

Mistake four: no policy, so no evidence

Article 5(2) GDPR obliges companies to be able to demonstrate compliance with the data protection principles. Without a written AI policy, that evidence is missing, and in proceedings only what is documented counts. A workable policy fits on two pages and sorts data into three classes:

• Public: anything that is on the website anyway. May go into any tool.
• Internal: quotes, process descriptions, drafts. Only in the company account with a data processing agreement, never in private accounts.
• Confidential: personal data, trade secrets, contract contents, credentials. Not entered, no exceptions.

Concrete examples from the company’s own operations carry more weight than any article of law: a customer enquiry, a set of minutes, a quote, each assigned to the right class.

Mistake five: policy distributed, but never trained and never checked

A policy nobody knows stays on paper. Article 4 of the AI Act has in any case required since 2 February 2025 that staff who work with AI systems have sufficient AI literacy. An introductory session of around 90 minutes with real cases from the business, an annual refresher and occasional spot checks on whether the rules are followed day to day are a practical way to meet this duty, and at the same time provide the evidence that mistake four calls for.

The order that works

The sequence follows almost by itself from the five mistakes:

• Inventory: who uses what, with which data, in which account?
• Set up a business plan, accept the data processing agreement, exclude model training, activate EU data residency where available, and document all of it in the record.
• Access through the existing identity management with two-factor authentication; anyone deactivated in the directory loses their AI access with it.
• A policy with three data classes, signed by all staff.
• Training with cases from the company’s own work, repeated annually, with spot checks.

The order matters more than the pace. An account without a contract is worth as little as a policy without training.

Who supervises in Germany and the Netherlands

For GDPR breaches, the data protection authorities of the federal states are responsible in Germany, and the Autoriteit Persoonsgegevens in the Netherlands. The transparency obligations of the AI Act under Article 50 have been supervised in Germany by the Bundesnetzagentur since the end of July 2026; in the Netherlands, under the draft law, they also fall to the Autoriteit Persoonsgegevens, which for a business operating across the border therefore checks both sets of rules from a single desk on the Dutch side.

Outlook

The five mistakes have one thing in common: none of them is a technical question. Account, contract, policy and training are administrative decisions that a management team can take within a few weeks. Once they are taken, the company no longer has to tolerate the use but can permit it, and already has the documents in hand for any review by a data protection authority or the Bundesnetzagentur. This concludes our short series on the AI Act. The next articles return to the topics that shape the daily work of finance and administration departments.

References

European Union (2016). Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 5, 28, 30, 82 and 83. Retrieved from https://eur-lex.europa.eu/eli/reg/2016/679/oj

European Union (2024). Regulation (EU) 2024/1689 (AI Act), Articles 4 and 50. Retrieved from https://eur-lex.europa.eu/eli/reg/2024/1689/oj

Cisco (2024). More than 1 in 4 Organizations Banned Use of GenAI Over Privacy and Data Security Risks, Cisco 2024 Data Privacy Benchmark Study, 25 January 2024. Retrieved from https://newsroom.cisco.com/c/r/newsroom/en/us/a/y2024/m01/organizations-ban-use-of-generative-ai-over-data-privacy-security-cisco-study.html

OpenAI (2026). Enterprise privacy at OpenAI. Retrieved from https://openai.com/enterprise-privacy/

OpenAI (2025). ChatGPT Business Rename FAQ. Retrieved from https://help.openai.com/en/articles/12111915-chatgpt-business-rename-faq

Global Connect Admin (2026). Company data in ChatGPT: what businesses need to know since August 2026. Retrieved from https://globalconnectadmin.com/company-data-in-chatgpt-what-businesses-need-to-know-since-august-2026/

Image: FJ Design Agentur (fjdesign.de)