How Do LLMs Work with SAP?

Have you ever wondered how ChatGPT, Claude, and Gemini understand our questions and provide appropriate answers, even when the topic is complicated?

What is an LLM?

An LLM, or Large Language Model, is an AI model trained to understand and work with human language. It can process text, identify patterns, and generate responses based on the information and instructions it receives.

LLMs can understand and generate human-like text.

What is RAG?

RAG stands for Retrieval-Augmented Generation. In simple words, RAG helps an AI system retrieve relevant information from documents, databases, or other sources before generating an answer.

A Simple Example

For example:

In this example, SAP contains the business data, while the LLM provides a more natural way for people to interact with that information.

Tokenization

Tokenization is the process of breaking text into smaller pieces called tokens so that an LLM can process language.

In a real SAP environment, an LLM should not automatically have access to all SAP information. It would need appropriate authorization and controlled access to relevant data.

Who Develops LLMs?

Some examples of organizations that develop LLMs and AI models include:

• OpenAI → GPT models
• Anthropic → Claude models
• Google → Gemini models
• Meta → Llama models
• DeepSeek → DeepSeek models

My Observation

While learning SAP, I came across the concept of LLMs. I became curious about how ChatGPT, Claude, and Gemini understand our questions and generate responses.

As I explored the topic, I learned about LLMs, tokenization, RAG, and their possible connection with SAP.

It was interesting for me to see how AI and SAP can come together to make business information easier to access and interact with, while still requiring proper authorization and data security.