Uploaded the Wrong File to an AI Model — and Got Fired. How Data Security in AI Really Works

A recent publication on Habr has caused quite a stir. The article describes an incident in which an employee uploaded the wrong file to a neural network service and was subsequently dismissed. While the full details of the case are not disclosed, the authors use it as a launching pad to discuss a much broader issue: the invisible security architecture behind AI products and the risks that arise when users treat these systems like a simple search engine.

This incident highlights a dangerous gap between how users perceive AI services and how these services actually operate. Many people assume that sending a file to ChatGPT or similar tools is as private as sending it to a colleague via corporate email. In reality, the data may pass through multiple servers, be logged, monitored, and even used to improve the model. The article on Habr breaks down the technical and organizational mechanisms of data security in AI, and this piece summarizes the key points while adding practical recommendations.

The Habr case: a cautionary tale that repeats itself

The Habr article opens with a real-world drama: an employee, confident in their knowledge of corporate security, uploaded a file to a popular AI service. The file, containing sensitive information, was not the one intended for analysis. Immediately after the upload, the company's internal security systems detected the transfer. The employee was summoned to the HR department and eventually dismissed. The authors note that the employee had signed a corporate confidentiality agreement and had undergone mandatory security training — yet they still made a fatal mistake.

The key takeaway from the first part of the article is not that AI is inherently evil, but that modern AI systems are equipped with extensive logging and monitoring mechanisms that often work asynchronously. The transfer is recorded, the file is stored on the provider's side, and the company's own security tools (DLP, information security incident detection systems) can analyze the traffic and identify the leak. This is exactly what happened in the described case.

Incidents like this are becoming more frequent. According to the 2025 Verizon Data Breach Investigations Report, a significant portion of data breaches involve internal errors, and the use of AI tools has become a new entry point. While the Habr article does not cite this report, the trend is well documented by security researchers.

Not the first time: Samsung and the cost of carelessness

The story from Habr is not unique. In April 2023, as reported by The Korea Herald, three Samsung employees accidentally leaked sensitive source code and internal meeting minutes to ChatGPT. The company subsequently implemented a ban on the use of generative AI tools on corporate devices, and it even developed its own internal language model.

Samsung's case is a reminder that the damage extends beyond the fired employee or the leaked content. The company faced reputational losses, potential violation of NDA agreements with partners, and the cost of incident response. It also triggered a wave of corporate bans on generative AI across many industries, including banking and legal services.

These incidents share a common pattern: the employee did not intend to steal or leak data; they were simply trying to be productive. But the tool they used did not distinguish between a harmless summary and a confidential document.

How data flows through an AI service

To understand the level of risk, it is useful to know the typical path of a file uploaded to a cloud-based AI:

1. Client-side processing — The file is prepared locally: it may be compressed, resized, or parsed into text. At this point, endpoint security tools (antivirus, sandboxes) can inspect it, but they are not always effective against data exfiltration attempts.
2. Transport — The data is sent via HTTPS (TLS 1.3) to the provider's API endpoint. This encrypts the connection, but the provider can still decrypt the content after receiving it.
3. Server-side temporary storage — The provider stores the data to process the request. This storage can be memory-based (temporary cache) or disk-based (logging). If disk-based, the data may linger for days or months, depending on the provider's data retention policy.
4. Model inference — The model processes the prompt and generates a response. Some providers deploy separate "gatekeeper" models to filter content or check for policy violations. These gatekeepers may retain a copy of the input, sometimes in an unencrypted form so they can be audited by moderators.
5. Persistent retention and deletion — Data is eventually deleted, but deletion is not trivial. In distributed systems, copies exist on multiple nodes, and "deleting" sometimes means removing the ability to access the data, not physically erasing the underlying media. The Habr article emphasizes that the user often has no way to know at which step the data is exposed. The only reliable source of truth is the vendor's official privacy policy and data processing agreement.

Modern DLP systems can inspect the outgoing traffic. For example, a corporate proxy can be configured to block all requests to external AI services that contain files with a certain extension or content pattern. The company in the Habr story likely had such a tool.

The training data trap: why deleted data is never truly gone

One of the most debated points in the Habr material is the use of user data for training neural networks. Many mainstream AI services, especially free tiers, reserve the right to use incoming prompts and files to improve their models. This practice is usually disclosed in the terms of service, but most people do not read those terms.

The problem is that once data is included in a training set, its removal becomes almost impossible. A neural network is not a database table from which you can delete a row; the information is distributed across millions of parameters. Even if a user officially requests deletion, the process of stochastic gradient descent has already left traces in the model's weights.

A well-known example from the past is the GitHub Copilot lawsuit. Developers sued GitHub and OpenAI, claiming that their public code was used to train Copilot without explicit permission, and that the model could reproduce license-protected code. This legal dispute demonstrates that data used for training cannot be easily recalled. As of 2026, the courts have not reached a final unified decision, but the lesson is clear: do not send sensitive data to services that may use it for training.

Consumer vs. Enterprise: what agreements really say

Let's compare how data is handled in a typical consumer plan versus an enterprise plan. The following table is based on publicly available information from major AI providers (OpenAI, Google, Microsoft, Anthropic) as of 2026:

Enterprise plans are a different world: they give the client the right to inspect deletion logs, demand that data is not used for training, and require that prompts are processed in a specific region (data residency). However, such plans are significantly more expensive and require a formal contract negotiated with the vendor.

Encryption, access control, and legal compliance

The Habr article also delves into the technical security layers of AI services. Here are the key elements:

• Data in transit: All traffic between the client and the provider is protected by TLS 1.3. This prevents interception by third parties, but it does not protect against the provider itself.
• Data at rest: Files are encrypted on disk using AES-256 or similar algorithms. The encryption keys are managed by the provider, meaning the provider can technically access the data if required.
• Access control: Internal teams at AI companies have strict access policies. For example, OpenAI uses hardware key management, periodic access reviews, and full audit trails. However, even strict policies do not guarantee immunity to insider threats.
• Compliance: Many enterprise AI services now comply with ISO 27001, SOC 2 Type II, and GDPR. This means the vendor has passed independent audits and must follow specific security and privacy practices. But compliance is not a substitute for technical control; it is only a contractual obligation that can be verified through reports.

One piece of advice from the Habr article: large companies should install an internal proxy system that monitors all prompts to AI services. This proxy can filter out files containing confidential patterns, add corporate headers, and log everything for future review. The publication recommends this as a mandatory element of any corporate AI policy.

Incident response: what to do if the wrong file is already uploaded

If an employee realizes they have uploaded the wrong file, the first rule is not to panic. The second is to report the incident immediately to the security team. Many companies have a formal process for this. In the Habr case, the employee did not report it, and the loss of time became an aggravating factor. A quick report can sometimes mitigate the consequences.

The corporate security team can contact the provider to request expedited deletion, especially if the account is under a business agreement. However, if the data is already stored, the only solution is to limit the damage: change credentials, notify affected parties, and analyze what was exposed.

Even if the employee is not fired, the incident often results in the revocation of access to AI services and the need to pass additional training. In more serious cases, it leads to legal action. The lesson is simple: the moment you realize a sensitive file has left the building, do not hope that "maybe no one will notice."

Practical safeguards for businesses

Based on the Habr article and common security practices, here are concrete steps that can prevent a situation where an employee uploads the wrong file:

1. Create a clear AI usage policy — even a one-page document that describes what can and cannot be sent to public AI services. Include examples of confidential data, such as unreleased financial results, medical records, and personal data of customers.
2. Deploy a DLP (Data Loss Prevention) system — modern DLP tools can scan a file before it is uploaded to an external service and block the transfer if the content matches confidential patterns. Products like Google DLP and Microsoft Defender for Cloud Apps can integrate directly with a corporate network.
3. Implement an AI gateway — instead of allowing direct access to ChatGPT or other services, route all requests through a trusted gateway that adds data sanitization, redaction, and audit logging. This is where platforms like ASI Biont come in: they support integration with AI services such as OpenAI via API and provide a secure layer for corporate users. ASI Biont supports integration with OpenAI via API — learn more at asibiont.com/courses.
4. Prefer self-hosted or private models — open-source models like Llama 3.1 and Mistral can be deployed on your own infrastructure. This ensures that data never leaves the corporate network. The downside is the need for powerful hardware and specialists to operate the model.
5. Train employees effectively — typical security training does not cover the specifics of AI. The training should include interactive examples of what happens to a file after it is uploaded, and how the provider's logging works. For instance, an employee can be shown that their activity is visible in the corporate SIEM system.
6. Anonymize test data — before uploading a file to an AI tool, replace all names, phone numbers, and other identifiers with random placeholders. Even if the data leaks, it will be much less valuable.

The future of AI security

The Habr article concludes with a look at how AI security is evolving. The next few years will bring widespread adoption of confidential computing — a technology where data is processed inside a hardware-secured environment (such as Intel SGX or AMD SEV) that prevents even the cloud provider from seeing the content. Google Cloud and Microsoft Azure are already offering such options for AI workloads.

Another direction is federated learning, where a model is trained across numerous decentralized devices, and raw data never leaves its location. This is still an experimental area, but the first implementations in healthcare and finance have already been demonstrated.

Regulations also play an increasingly important role. The European Union's AI Act, which came into effect in stages, requires companies to assess the security and transparency of high-risk AI systems. Similar initiatives are being developed in the US and China. The result will be that "I didn't know" will no longer be an excuse.

The Habr authors also mention the rise of privacy-preserving machine learning — a set of techniques allowing models to be trained and inferred without exposing raw data. These are exciting prospects, but for now, companies must rely on strict governance and employee vigilance.

Conclusion

The story of the employee who uploaded the wrong file to a neural network is a classic example of the gap between technology's capabilities and human awareness. The technology itself is not the threat; the way we use it can be. The Habr article demonstrates both the full path of data through an AI service and the tools companies already use to track every uploaded file.

To avoid becoming a case study, each of us should start with the basics: treat every AI prompt as if it might be broadcast. Read the privacy policy of the tool you use. If your company doesn't have an AI usage policy, suggest creating one. And most importantly, remember that the delete button in a chat interface does not mean the data is gone — it may already be part of a training set or an audit log.

As the Habr authors rightly note, in the digital world security is not a feature you turn on; it is a habit you practice every time you send data to an AI service.

Source