Privacy-sensitive information and AI: why responsibility stays with the user

You remain legally responsible for privacy-sensitive data you send to any AI system, whether you built it or not. You must document your lawful basis for processing, obtain fresh consent where required, and verify that every platform you use offers contractual protection before forwarding confidential information.

The prompt is an analysis of 27 July 2026 of how regulators and courts treat responsibility for privacy-sensitive data in AI workflows, which argues that organisations cannot outsource accountability for lawful processing to third-party model providers. Singapore's Personal Data Protection Commission has clarified how the Personal Data Protection Act applies across the entire AI lifecycle, from development through deployment. In our assessment, this signals a shift towards explicit governance requirements: you must now treat AI use as a data handling process subject to the same accountability rules as any other system that touches personal information.

What does the law actually require you to do with service data?

If you have collected personal data from individuals through a product or service, you cannot automatically reuse that data to train or improve a generative AI model. You must obtain fresh consent unless a specific exception under data protection law applies. This applies even to data you already hold. The distinction matters: consent obtained for one purpose does not cover reuse for AI development without explicit notice and a new lawful basis.

Online data presents a particular trap. You cannot treat all publicly available information as freely reusable simply because it exists on the internet. You must define a lawful purpose for any data you supply to an AI system, whether that data came from your own users or from public sources. Responsibility for that lawful purpose stays with you, not with the model provider.

Which concrete controls do you need to demonstrate?

1. Define lawful basis and purpose — document why you are processing each category of personal data and which legal ground permits that processing.
2. Obtain or reassess consent — verify that consent obtained for original service delivery covers AI use, or obtain fresh consent with clear notice of AI purposes.
3. Allocate roles across the AI chain — document which organisation or party is responsible for each step, from data collection through model output, but recognise that you remain accountable as the deploying organisation.
4. Verify platform safeguards before use — confirm contractual terms, data processing commitments and confidentiality guarantees with any AI platform before you send sensitive information to it.
5. Implement access controls and logging — restrict who can input sensitive data, log what is sent and received, and enforce need-to-know principles within your own workflows.

What happens if you send privileged or confidential information to a public AI tool?

A 2026 federal court ruling in New York examined the use of commercially available generative AI platforms to process legally privileged information. The court found that forwarding attorney-client communications or work product to a public platform, without contractual protection, can destroy privilege. The platform's privacy policies and the absence of a protected relationship between you and the platform mean that privilege may lapse. This applies beyond law: any confidential or proprietary information sent to an uncontrolled public AI tool without contractual safeguards is legally comparable to disclosure and may destroy trade secret protection.

What are the failure modes you must guard against?

• Uncontrolled reuse of service data — using personal data collected for one purpose to train AI models without fresh consent or lawful basis.
• Treating public data as freely usable — assuming that online availability means you can supply data to AI systems without defining a lawful purpose.
• Loss of privilege or trade secret protection — forwarding confidential or legally privileged information via platforms without contractual data processing guarantees.
• Unclear responsibility allocation — failing to document which party is responsible for each step in the AI lifecycle, leaving accountability gaps.
• Absence of access controls and audit trails — allowing sensitive data to enter AI workflows without logging, restriction or visibility into what was processed.

How can tooling help you meet these requirements?

Verification layers can make your data handling visible and controllable. A verification system can pre-process and anonymise data before it reaches an AI model, test whether sensitive information meets your policy safeguards before forwarding, and provide logged evidence of what was sent and what safeguards were applied. This creates a traceable record of your compliance steps. However, the final professional judgement—whether to send data to a particular system, whether anonymisation is sufficient for your use case, whether the contractual terms meet your risk tolerance—remains yours. Tooling can enforce the steps you decide on; it cannot replace your assessment of whether those steps are appropriate for your organisation and your data.