With all the hype around AI, I was thinking it is a good time to actually build something simple, that will explain the basic principles of AI systems and LLMs. There are so many notions out there, so many new things, and technology is advancing so rapidly that even senior engineers have a rough time staying up to date.

In order to make this as simple as possible, I will begin with the simplest example of how to integrate an AI API into a .Net app, and then build on that adding more features that will make clear a lot of concepts like : RAG, embeddings, grounding, chunking, guardrails, .Net agents and so on.

I will build this in a public github repository, each step on a particular branch.

Step 1 – From an LLM Call to a Grounded Application

Note, before you try to run the code below, make sure you go over to https://platform.openai.com/ and generate an API key. Before running the application, create an OpenAI API key and configure it using .NET User Secrets or another secure configuration mechanism. You can test with your key in the appSettings.json also but never commit API keys to source control.

Github link: https://github.com/genoiucosmin/AIApp1/tree/Step1

The easiest way to build an AI application is also one of the easiest ways to build a useless one: send a user’s question directly to an LLM and return whatever comes back.

That works for general questions, but enterprise applications usually need something more constrained. The application may have its own documentation, policies, product information or business knowledge, and the model should answer using that information rather than relying solely on its pretrained knowledge.

My first iteration was intentionally simple: a .NET application that calls an LLM, provides it with a set of trusted documents, and instructs it to answer only from those documents.

The architecture is:

HTTP Request
    ↓
ASP.NET Core Controller
    ↓
AI Service
    ↓
Document Service ──→ Application Documents
    ↓
Prompt + Documents
    ↓
OpenAI Client
    ↓
LLM
    ↓
Answer

The important distinction is that the application does not modify or retrain the model. Instead, it supplies additional information as part of the request.

Separating the AI provider from the application

The application exposes a small abstraction:

public interface IOpenAiClient
{
    Task<string> AskAsync(string systemPrompt, string question);
}

The AiService depends on that abstraction rather than directly constructing HTTP requests to OpenAI.

This separation is useful for the same reason provider abstractions are useful elsewhere in software engineering: the application logic should not need to know how a particular external provider is implemented.

The actual OpenAI communication is isolated inside OpenAiClient.

This also gives us an obvious place to handle provider-specific concerns such as authentication, HTTP errors and rate limiting.

Prompting

The application creates a system prompt containing instructions such as:

You are an assistant that answers ONLY using the provided documents.

If the answer is not present, say you don't know.

The documents are then added to the context supplied to the model.

This is more than simply asking the model a question. We are defining the model’s role and constraining how it should use the information supplied by the application.

Grounding

The next important concept is grounding.

Instead of relying entirely on the model’s pretrained knowledge, we provide application-specific information and tell the model to base its answer on that information.

Conceptually:

Application Documents
        +
User Question
        ↓
       LLM
        ↓
Grounded Answer

However, this first implementation has an obvious problem. The application loads all documents and sends all of them to the model. That may be acceptable for a tiny demonstration, but it does not scale. Imagine an application with thousands or millions of documents. Sending everything with every question would create unnecessary token usage, latency and cost. More importantly, the model would receive far more information than it actually needs.

This leads naturally to the next question: How can we find only the information relevant to the user’s question? That is where retrieval begins. The next iteration therefore moves from a simple grounded LLM application toward a retrieval-augmented architecture.

Engineering lessons

Even this very small application already introduces several production concerns.

The OpenAI client uses HttpClient, asynchronous calls and explicit handling for transient failures. Rate limiting is particularly important because an AI API can return HTTP 429 responses when requests exceed available capacity.

The client therefore implements retries with exponential backoff and respects the server’s Retry-After information when available. This is a useful reminder that adding AI to an application does not remove normal software-engineering concerns.

An LLM is still an external dependency. It can fail, become unavailable, rate-limit requests, return unexpected output and introduce latency and cost.

A production AI application therefore still needs the same engineering discipline we apply to other distributed systems.

What this first iteration does — and does not — solve

This application demonstrates:

• calling an LLM from .NET;
• system and user prompts;
• grounding;
• dependency injection;
• provider abstraction;
• external API integration;
• basic rate-limit handling.

It does not yet implement RAG. The application still sends every document to the model. The next iteration will introduce retrieval so that the system can select relevant information before asking the LLM to generate an answer.

That distinction is the key transition from a simple LLM integration to an AI application architecture.

Interview knowledge you should retain from step 1:

If you can answer these naturally, I consider Step 1 learned:

Q: Is this RAG?
→ No. It’s a grounded LLM application, but it doesn’t perform retrieval. It passes all documents to the model.

Q: Why not put the OpenAI call in the controller?
→ Separation of concerns and provider abstraction; it also makes testing/replacing the provider easier.

Q: Why use a system prompt?
→ To establish higher-level behavioral instructions and constrain how the model should use the supplied context.

Q: What is grounding?
→ Supplying external/application-specific information to constrain the model’s response rather than relying solely on pretrained knowledge.

Q: What’s the biggest architectural limitation?
→ We send all documents on every request. This doesn’t scale in context size, latency or cost.

Q: Why does retrieval solve that?
→ Instead of giving the model everything, we first select the information relevant to the query and give the model only that context.

Q: What happens if OpenAI returns 429?
→ Treat it as a transient/rate-limit condition, respect Retry-After where available and retry with bounded backoff.