Advice

I’m working on an app that leverages Large Language Models (LLMs) to assist professionals in regulated fields like medicine and law. My main concern is ensuring compliance with privacy and secrecy regulations (e.g., Section 203 StGB in Germany or similar).

1. What are the best LLM/cloud providers for building privacy-compliant apps? I believe that directly using OpenAI and Anthropic is a no-go but I think Azure and AWS might have some agreements?

2. Are there any specific privacy-focused settings or features to enable when using these models?

Improve this question

5 Replies 5

jarmod
•

Research the various LLM hoster's compliance and privacy statements. For example on AWS, read How to safeguard healthcare data privacy using Amazon Bedrock Guardrails.

Improve this answer
fersarr
•

Thanks for the reply! So that article mostly discusses guardrails and grounding checks which aim to protect/regulate the output of the LLM. What about protecting your data from the cloud and model providers? I seem to remember Azure had some kind of secrecy agreement like langdock but can't find it

Improve this answer
Korgen
•

For Amazon Bedrock the model providers do not have access to your data:

Amazon Bedrock doesn’t use your prompts and continuations to train any AWS models or distribute them to third parties. Each model provider has an escrow account that they upload their models to. The Amazon Bedrock inference account has permissions to call these models, but the escrow accounts themselves don’t have outbound permissions to Amazon Bedrock accounts. Additionally, model providers don’t have access to Amazon Bedrock logs or access to customer prompts and continuations.

Source: [AWS Blogpost](https://aws.amazon.com/blogs/awsforsap/improve-your-productivity-with-amazon-q-and-bedrock-for-sap-use-cases/)

This concept is also explained (along with other important data control mechanisms) in AWS re:Inforce 2023 - Securely build generative AI apps & control data with Amazon Bedrock (APS208)

Improve this answer
fersarr
•

Thanks Korgen, that is very helpful. I think that's mostly okay for 99% of the cases, but I'm still unsure about the case where you build an app for doctors/lawyers in Germany where section 203 (professional secrets) states that subcontractors must be contractually bound to secrecy. So if my app uses an LLM hosted in AWS, it sounds like we need AWS to be contractually bound to secrecy under § 203 StGB?

Improve this answer
fersarr
•

Just found this from Azure which specifically talks about Section 203 from Germany:
https://assetsprod.microsoft.com/mpn/data-secrecy-amendment-germany.pdf

Microsoft is aware of legal obligations with regard to allowing access to Customer’s clients’ information and that non-compliance with such obligations can entail criminal sanctions for the parties involved (cf. in particular §§ 203, 204 StGB) (imprisonment or fines).

Microsoft shall be obliged to maintain confidentiality in accordanc...

Would be great to find something like this from AWS or GCP

Improve this answer

Your Reply

By clicking “Post Your Reply”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.