PrivateGPT

PrivateGPT is a production-ready AI project that allows you to ask questions about your documents using Large Language Models (LLMs), with complete privacy protection. The platform provides an API that extends the OpenAI API standard, enabling you to build private, context-aware AI applications. Importantly, PrivateGPT operates entirely locally—no data leaves your execution environment at any point, making it 100% private.

PrivateGPT enables you to interact with your documents through Retrieval Augmented Generation (RAG) technology. The system processes documents locally, retrieves relevant information, indexes content securely, and vectorizes text efficiently before analyzing user queries and providing contextualized responses without sharing external data.

You can connect advanced tools to your assistants, including document upload, image analysis, prompt suggestions, source references, and external search tools like Google Search and Webpage Search. Additional features include DataAnalyzer for advanced data processing and artifacts for generating documents, tables, and code.

The API follows and extends the OpenAI API standard, supporting both normal and streaming responses. This means if you can use OpenAI API in one of your tools, you can use your own PrivateGPT API instead with no code changes—and it's free for local setups.

PrivateGPT can be deployed on-premise (data center, bare metal) or in your private cloud (AWS, GCP, Azure). The platform is hosted in your own Azure environment, supporting role-based access management, data encryption, and audit logs.

Setting up PrivateGPT involves cloning the repository, installing required dependencies including the LLM model and embedding model, and configuring the settings.yaml file. You'll need to download language and embedding models, set up the vector database, and test the installation. The LLM model can be installed using the poetry add command, with options like the LLaMA model.

Different regions have different optimal configurations—European users often prefer Mistral-based models due to alignment with EU AI regulations, while North American implementations more commonly use Llama 3 derivatives. PrivateGPT comes with a default language model (gpt4all-j-v1.3-groovy), but you're free to experiment with others like Falcon 40B from HuggingFace.

PrivateGPT allows you to create multiple AI assistants targeted at specific departments or business areas. Each assistant can be configured with its own prompts, access to relevant document sources (RAGs), and specialized tools tailored to the department's needs—such as HR, customer service, or sales.

You can customize the user experience with your own labels, welcome messages, and access rights, ensuring the platform matches your organizational structure and security requirements.

You can set up your own test environment (Testing App) where you can create, try out, and adjust assistants before making them available to the entire organization. This allows you to simulate user scenarios, review chat logs, and collect feedback to ensure quality and relevance before broad rollout.

PrivateGPT achieves privacy through a combination of local processing, zero external connections, and optional encryption—creating multiple layers of protection for sensitive data. You control who has access to which data and functions, and the platform can be configured to follow your company's own policies for data storage, access, and deletion.

The solution is designed to support GDPR and other relevant compliance requirements.