In the first three parts of this series, I focused on running AI models efficiently.
But after living with the platform for a while, I realised something important.
The problem wasn't the models anymore.
The problem was context.
Even the best LLMs only know what you tell them. Every new conversation starts with little or no knowledge about your project. Requirements, architecture documents, API specifications, meeting notes, research papers, and design decisions all sit outside the model's context window.
Copying documents into prompts isn't a long-term solution.
I needed a way for AI to understand my projects without exposing sensitive information to external services.
The Missing Layer
Running local models solves inference.
It doesn't solve project knowledge.
I wanted an AI assistant that could answer questions like:
- Where is this business rule documented?
- Which API specification defines this endpoint?
- Has this requirement already been discussed?
- Which document explains this architecture decision?
- Show me every reference to this feature across the project.
More importantly, I wanted those answers without sending confidential documents to cloud services.
Building a Local Knowledge Platform
Instead of treating documentation as static files, I treat it as structured knowledge.
Every document follows the same processing pipeline.
The same RAG engine powers every interface.
Whether I'm working inside Cursor, Codex, Claude Code, my browser, or a custom application, they're all querying the same knowledge platform rather than maintaining separate indexes.
Why Traditional Search Isn't Enough
There are two common approaches to document search.
Traditional keyword search understands exact matches but struggles with intent.
Semantic search understands meaning and intent but can miss highly specific technical identifiers.
Technical Identifiers
Technical searches often depend on exact matches:
useEffectERR_CONNECTION_REFUSEDMATERIAL_GROUP_ID/api/purchase-ordersQ4_K_M
Business and Semantic Questions
Business requirements are usually expressed as questions whose meaning matters more than exact wording:
- Which procurement requests require finance approval?
- How are invoice mismatches handled?
- What documents are required to onboard a supplier?
- Which business rule defines purchase-order exceptions?
Engineering projects need both kinds of search.
The platform combines semantic retrieval with keyword boosting, allowing it to understand meaning while still prioritising exact technical matches when they matter.
Built for Real Engineering Projects
One challenge I repeatedly encountered while consulting was keeping project knowledge isolated.
Every client has different terminology.
Different business rules.
Different documentation.
Different security requirements.
Instead of creating separate systems for every customer, the platform organises documents into project namespaces.
Each project has its own searchable knowledge base while sharing the same underlying infrastructure.
That makes it practical to support multiple products, multiple teams, and multiple clients from a single deployment.
Local by Design
Privacy wasn't an afterthought.
It was one of the original design goals.
After the initial embedding model is downloaded, everything runs locally.
- Documents remain on your infrastructure.
- Embeddings are generated locally.
- Searches happen locally.
- No recurring embedding API costs.
- No dependency on cloud availability.
For organisations dealing with proprietary source code, engineering documentation, or customer information, that matters.
Where It Fits in My AI Platform
By this point, my home lab has evolved into several specialised layers.
Each component has one responsibility.
The workflow is simple.
The AI model generates answers.
The RAG platform supplies project knowledge.
Together they produce responses that are both intelligent and grounded in the actual documentation.
Instead of repeatedly explaining the project to the AI, the AI retrieves only the relevant context when it needs it.
Looking Ahead
Running models locally solved the compute problem.
Building a project knowledge platform solved the context problem.
Together they transformed AI from a general-purpose assistant into something much closer to an engineering teammate.
In Part 5, I'll bring everything together and show how local models, cloud AI providers, project knowledge, and modern developer tools work as a single AI platform.
Building Private AI Platforms
Many organisations already have thousands of pages of valuable knowledge spread across PDFs, Word documents, technical specifications, internal wikis, and source code repositories.
The challenge isn't creating more documentation.
It's making that knowledge instantly accessible to developers and AI assistants without sending sensitive information to external AI services.
I'm currently building this platform for engineering teams that want:
- Private, on-premise AI knowledge search
- AI assistants that understand your projects
- Local or hybrid LLM deployments
- Multi-project document management
- Integration with existing developer workflows
If your organisation is exploring private AI infrastructure, internal knowledge platforms, or production-ready AI solutions, I'd be happy to discuss your requirements.
Building an AI Home Lab
- Part 1 — Why I Stopped Treating AI as a Cloud Service
- Part 2 — GGUF Switchboard: Bringing Kubernetes Thinking to Local LLMs
- Part 3 — Operating an AI Platform
- Part 4 — Local Project RAG: Giving AI the Context It Actually Needs
- Part 5 — The AI Platform I Use Every Day