AI in IT: How to Use ChatGPT Without Leaking Confidential Data

AI in IT: How to Use ChatGPT Without Leaking Confidential Data

ChatGPT has become an everyday tool for developers, DevOps engineers, and IT support teams. It helps with debugging, writing scripts, drafting documentation, and even solving complex infrastructure problems. However, the widespread use of ChatGPT in the workplace has created a new security blind spot. A recent article on Habr Source examines this problem and suggests practical ways to protect confidential data.

The core issue is simple: every prompt you send to ChatGPT is transmitted to external servers, where it may be stored, analyzed, and used for model training. For the default consumer version, conversations are retained unless you explicitly disable history. Even with history off, OpenAI keeps transcripts for up to 30 days for safety monitoring, as stated in the company's API data usage policies. In an enterprise context, this means that proprietary code, customer data, or security credentials could end up in a third-party system without your consent.

Why Data Leaks Are a Growing Risk

The Habr article points out that most leaks are not caused by attackers, but by well-meaning employees who paste sensitive snippets into chat windows. A developer might ask ChatGPT to explain an error log without realizing that the log contains internal IP addresses, database names, or even authentication tokens. Such data, when combined over time, can reveal a company's infrastructure and create a significant attack surface.

Another risk is shadow AI: employees using personal ChatGPT accounts for work tasks without any supervision. The article notes that companies often detect this after an incident, making prevention the priority.

Practical Measures for Safe ChatGPT Use

The article outlines several actionable steps that IT teams can implement:

1. Use enterprise versions. ChatGPT Enterprise and Team plans promise that prompts and conversations are not used for training. For companies that need strict data residency, Azure OpenAI Service offers similar guarantees on Azure's trusted cloud infrastructure.

2. Disable history and training. If your team uses the consumer version, instruct them to turn off chat history. This is not a complete fix, but it limits data retention.

3. Adopt a clear usage policy. Define what can be pasted into AI tools. For example, prohibit source code containing secrets, personal data, or internal architecture details. Provide a flowchart or checklist to help employees decide what is safe.

4. Anonymize data before prompting. Replace real names, IP addresses, and other identifiers with placeholders. The article emphasizes that generic examples often work just as well as real ones.

5. Run local LLMs for highly sensitive tasks. Open-source foundation models (Llama 3, Mistral, etc.) can be deployed on-premises. This way, data never leaves your network. For many IT workflows, self-hosted models now offer accuracy comparable to cloud services.

6. Use the API with contractual safeguards. When building custom internal tools on ChatGPT, use the OpenAI API with a data processing agreement that prevents training and shortens retention. Azure OpenAI also provides this option with additional compliance certifications.

7. Deploy data loss prevention (DLP). Tools like Microsoft Purview can monitor outbound HTTP requests and block uploads to external AI services if they contain sensitive patterns. The article suggests that DLP alongside employee training forms a solid defense.

The table below summarizes which data types require special treatment:

The Integrations Challenge

As IT teams integrate ChatGPT with internal systems—CI/CD pipelines, knowledge bases, and ticketing platforms—the risk increases. These integrations often require API connections that transfer data from corporate databases to external LLM providers. To keep governance under control, it's wise to route these connections through a secure middleware that supports centralized access management. ASI Biont supports connecting to [OpenAI] through API — more at asibiont.com/courses. This way, you can audit every request and enforce data masking rules across the organization.

A Typical Scenario: The Stack Trace

The Habr article describes a typical situation: a developer pastes a stack trace into ChatGPT to get an explanation. The trace contains an internal service name, a database table, and a few IP addresses. While none of these are on their own highly sensitive, together they give a model a "map" of the company's internal architecture. The authors of the article argue that this type of leakage is more common than a direct data breach, and that companies often underestimate its long-term consequences.

To avoid this, the article recommends a simple rule: before pasting anything, ask yourself if the information would be safe in the hands of a stranger. If not, anonymize it or ask a local colleague instead.

The Regulatory Angle

For companies in the EU, GDPR places strict limits on how personal data may be transferred to AI services. Using a US-based LLM without a data processing agreement can result in fines and legal action. The Habr article advises using providers with "data residency" options or hosting an open-source model in your own data center. This is particularly relevant for industries like healthcare, finance, and public administration.

Conclusion

ChatGPT is not a threat in itself; the risk comes from how it's used. The key is to treat every AI prompt as an external communication, not a private thought. By adopting enterprise plans, anonymizing input, writing clear policies, and using the right technical safeguards, IT teams can enjoy the productivity gains without exposing confidential data. As the Habr article concludes, responsible AI usage is a matter of culture, not just technology. Start with one prompt—and make it safe.