Agentic AI – with Java (Spring Boot POC)

 

This project is a simple Agentic AI Proof of Concept built using Spring Boot + Spring AI + OpenRouter (LLM).

It demonstrates how an AI Agent can investigate a problem step-by-step (like a human engineer) instead of following a fixed hardcoded workflow.

---

🎯 What Problem Does It Solve?

Given a goal like:

“Why is my Order API slow?”

The agent will:

1. Check system metrics

2. Check database performance

3. Check Kafka lag (optional)

4. Correlate results

5. Find the root cause

6. Stop automatically

---

🤖 What Is “Agentic AI” Here?

This is NOT a chatbot.

This is:

A state machine where an LLM is used only to decide the next step.

The LLM:

• Does NOT access DB

• Does NOT access metrics

• Does NOT execute anything

It only says:

“Check metrics”
“Now check DB”
“Now we are done”

All real work is done by Java code (tools).

---

🏗️ Architecture (Simple View)

User Goal
   ↓
Agent Loop
   ↓
LLM decides next action
   ↓
Java executes tool (DB / Metrics / Kafka)
   ↓
Result stored in memory
   ↓
Loop again until root cause found

---

🔁 Agent Execution Loop (Core Idea)

while(not finished):
    build prompt using (goal + memory)
    ask LLM what to do next
    execute that tool in Java
    store result in memory
    check stop condition

This is called:

Think → Act → Observe → Think → Act → Stop

---

🧩 Components

1. ObservabilityAgent

• The main orchestrator

• Runs the loop

• Talks to LLM

• Calls tools

• Decides when to stop

2. Tools (Simulated)

• MetricsTool → returns API latency

• DbTool → returns slow query info

• KafkaTool → returns consumer lag

(You can replace these with real Prometheus / DB / Kafka later.)

3. MemoryStore

• Stores investigation history

• Passed back to LLM every iteration

• Prevents the agent from “forgetting”

4. LLM (via OpenRouter)

• Used only for decision making

• Returns which action to perform next

---

🛑 Very Important Design Rule

The agent is NOT allowed to run forever.

The code contains hard stop conditions, for example:

• If a slow query is found → STOP

• If max iterations reached → STOP

Never let an LLM control termination in production systems.

---

▶️ How to Run

1. Get a Free API Key

• Go to: https://openrouter.ai

• Create account

• Create API key

2. Configure application.yml

spring:
  ai:
    openai:
      api-key: YOUR_OPENROUTER_KEY
      base-url: https://openrouter.ai/api/v1
      chat:
        options:
          model: mistralai/mistral-7b-instruct:free

server:
  port: 8080

3. Run Spring Boot App

4. Call in Browser

http://localhost:8080/analyze?goal=Why is Order API slow?

or

http://localhost:8080/analyze?goal=Why is my system metrics?

---

✅ Example Output

FINAL DIAGNOSIS:
CHECK_DB => Found slow query: SELECT * FROM orders WHERE status='PENDING' (no index)

---

🧠 What This Demonstrates (Interview Points)

• Agentic AI = LLM + Loop + Tools + Memory

• LLM used as decision engine, not execution engine

• Deterministic execution + probabilistic reasoning

• Safe termination conditions

• Real-world use case: production incident investigation

---

🚫 What This Is NOT

• Not a chatbot

• Not autonomous

• Not replacing real code

• Not suitable for core transaction logic

It is for:

Diagnosis, investigation, exploration, triage, decision support

---

🚀 Future Improvements

• Replace fake tools with:

  • Prometheus client

  • Real DB queries

  • Kafka Admin client

• Use Spring AI Tool Calling instead of parsing text

• Add Verifier Agent

• Add Planner Agent

PHASE 1 — First Agent (NO AUTONOMY, NO LOOP)

Goal: Just understand LLM → Tool call

Step 1.1: Create Spring Boot Project

• Java 17

• Spring Boot 3.x

• Add Spring AI

Step 1.2: Add LLM (Ollama or OpenAI)

spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        model: llama3

Step 1.3: Simple Service

@RestController
public class ChatController {

    private final ChatClient chatClient;

    @GetMapping("/ask")
    public String ask(@RequestParam String q) {
        return chatClient.call(q);
    }
}

Now you have:

❌ NOT an agent
✅ Just LLM integration

PHASE 2 — Tool Calling (This is where “Agent” starts)

Goal: LLM can choose which function to call

---

Step 2.1: Create Tool

@Component
public class DbTool {

    public String checkSlowQueries() {
        return "Top slow query is: SELECT * FROM orders ...";
    }
}

---

Step 2.2: Register Tool in Spring AI

@Bean
public Function<String, String> dbTool() {
    return input -> dbTool.checkSlowQueries();
}

---

Step 2.3: Prompt

If user asks about performance or slowness, call dbTool.

Now:

✅ LLM can call your function
❌ Still NOT a real agent (no loop, no planning)

---

PHASE 3 — Real Agent Loop (THIS IS AGENTIC AI)

Now we implement:

Goal → Think → Act → Observe → Think → Act → Finish

---

Step 3.1: Agent Orchestrator

@Service
public class AgentService {

    public String run(String goal) {
        String state = "START";

        for (int i = 0; i < 5; i++) {

            String prompt = """
            Goal: %s
            Current State: %s

            Decide next action.
            Allowed actions:
            - CHECK_DB
            - CALL_API
            - FINISH

            Respond in JSON:
            { "action": "...", "reason": "..." }
            """.formatted(goal, state);

            String decision = chatClient.call(prompt);

            AgentAction action = parse(decision);

            if (action.action().equals("FINISH")) {
                return "Done";
            }

            state = execute(action);
        }

        return "Max steps reached";
    }
}

---

Step 3.2: Action Execution

private String execute(AgentAction action) {
    if (action.action().equals("CHECK_DB")) {
        return dbTool.checkSlowQueries();
    }
    if (action.action().equals("CALL_API")) {
        return apiTool.checkHealth();
    }
    return "UNKNOWN";
}

---

Now:

✅ You have a REAL AGENT
✅ It can reason, decide, and loop
✅ This is Agentic AI

---

PHASE 4 — Add Memory (Very Important for Interview)

Step 4.1: Memory Store

@Component
public class MemoryStore {
    List<String> history = new ArrayList<>();

    public void add(String info) { history.add(info); }

    public String dump() { return String.join("\n", history); }
}

Inject memory into prompt:

Memory so far:
- previous result...

Now agent:

✅ Remembers past actions
✅ Avoids repeating steps

---

PHASE 5 — Use Case From YOUR PROFILE (Interview Gold)

🎯 Use Case: “Observability Agent”

Goal: “Why is Order API slow today?”

Agent will:

1. Check Prometheus metrics

2. Check DB slow queries

3. Check Kafka lag

4. Correlate

5. Answer

Tools:

• MetricsTool

• DbTool

• KafkaTool

---

Interview Explanation:

“I built an internal observability agent in Spring Boot. The agent receives a goal, plans steps using an LLM, calls internal tools (DB, Prometheus, Kafka), stores intermediate results in memory, and iterates until root cause is found.”

---

PHASE 6 — MCP (VERY HOT IN 2025 INTERVIEWS)

What MCP Solves

Instead of this:

if(action == "CHECK_DB") ...

MCP allows:

• Tools self-register

• LLM discovers them dynamically

• Secure boundary

• Standard protocol

Architecture:

LLM ↔ MCP Server ↔ Tools

You can say:

“MCP is like a service mesh for AI tools.”

---

PHASE 7 — Multi-Agent (Optional but Impressive)

Agents:

• Planner Agent

• Executor Agent

• Verifier Agent

Planner → makes steps
Executor → runs steps
Verifier → validates result

---

PHASE 8 — How You Explain In Interview (SCRIPT)

“I built a Spring Boot based Agentic AI POC. It uses an LLM as a reasoning engine, but all actions are executed via controlled tools. The agent works in a plan-act-observe loop, maintains memory, and can diagnose issues like slow APIs by querying DB, metrics, and Kafka. I also studied MCP which standardizes tool discovery and invocation.

---

 

---

🏁 Final Summary

This project shows how to build a controlled, safe, enterprise-style Agentic AI system using Spring Boot, where an LLM helps in reasoning, but all execution remains in deterministic Java code.

Here is the link for my git hub project Prittz/Spring-Boot-Agentic-AI-with-OPEN-AI-LLM-Connection-POC