Integrating a Self-Deployed LLM Chatbot with IntelOwl

[rshaurya]

Here is a high level architectural diagram describing components and how it integrates with IntelOwl:

Note: This ASCII diagram was created by an AI platform.

                             ┌───────────────────────────────────────────┐
                             │           IntelOwl (Core Stack)          │
                             │                                           │
                             │  ┌─────────────────────────────────────┐  │
 User / Analyst              │  │           IntelOwl Web UI           │  │
┌──────────────────────────┐ │  │ (Django/REST, Durin token auth)     │  │
│         Browser          │◀──┼───────────────HTTP(S)───────────────▶│  │
└──────────────────────────┘ │  └─────────────────────────────────────┘  │
                             │                     ▲                     │
                             │                     │ REST (Durin token)  │
                             │  ┌──────────────────┴──────────────────┐  │
                             │  │           IntelOwl API              │  │
                             │  │   (jobs, observables, reports)      │  │
                             │  └──────────────────┬──────────────────┘  │
                             │                     │                    │
                             │        ┌────────────┴────────────┐       │
                             │        │   IntelOwl Workers      │       │
                             │        │  (Analyzers / Plugins)  │       │
                             │        └────────────┬────────────┘       │
                             │                     │ writes reports     │
                             └─────────────────────┼─────────────────────┘
                                                   │
                                                   ▼
                                      ┌────────────────────────┐
                                      │ IntelOwl DB / ES       │
                                      │ (Postgres + Elasticsearch
                                      │  job + report indices) │
                                      └──────────┬─────────────┘
                                                 │
                                                 │ periodic/stream
                                                 ▼
         ┌─────────────────────────────────────────────────────────────────┐
         │              AI Microservice (FastAPI + PydanticAI)            │
         │          (Runs as its own Docker service in compose)           │
         │                                                                 │
         │  ┌───────────────────────────────────────────────────────────┐  │
         │  │                    FastAPI Web Layer                     │  │
         │  │  - /chat  : conversational interface (UI, API, SOC)     │  │
         │  │  - /ingest: trigger/monitor ingest jobs (optional)      │  │
         │  └───────────────┬───────────────────────────┬─────────────┘  │
         │                  │                           │                │
         │                  │ PydanticAI agents         │                │
         │                  │ (typed orchestration)     │                │
         │   ┌──────────────┴───────────┐   ┌───────────┴────────────┐   │
         │   │        RAG Agent         │   │        Planner         │   │
         │   │ - Talks to Qdrant        │   │ - Parse user intent    │   │
         │   │ - Semantic + cluster     │   │ - Pick analyzers /     │   │
         │   │   search over IOCs       │   │   playbooks            │   │
         │   └─────────────┬────────────┘   └───────────┬────────────┘   │
         │                 │                            │                │
         │                 │                            │                │
         │   ┌─────────────┴────────────┐   ┌───────────┴────────────┐   │
         │   │        Executor          │   │       Summarizer        │   │
         │   │ - Uses pyintelowl        │   │ - Uses Ollama LLM       │   │
         │   │   → IntelOwl API         │   │                         │   │
         │   │ - Starts jobs, fetches   │   │ - IOC/report summaries  │   │
         │   │   results                │   │   for UI or tickets     │   │
         │   └─────────────┬────────────┘   └───────────┬────────────┘   │
         │                 │                            │                │
         │                 │ LLM / embeddings           │                │
         │                 ▼                            ▼                │
         │      ┌──────────────────┐       ┌────────────────────────┐    │
         │      │    Ollama        │       │       Qdrant           │    │
         │      │  (LLM + emb)     │       │  intelowl_reports coll │    │
         │      │  - Llama3/Mistral│       │  - vector: embeddings  │    │
         │      │  - Chat, plans   │       │  - payload: observable,│    │
         │      │  - Summaries     │       │    type, cats, time,   │    │
         │      │  - Embeddings    │       │    es_id, job_id,      │    │
         │      └────────┬─────────┘       │    campaign_id, etc.   │    │
         │               │                 └──────────┬─────────────┘    │
         │               │ embeddings / LLM calls                 │      │
         └───────────────┼────────────────────────────────────────┘      │
                         │                                               │
                         │ ingest job (can be triggered via FastAPI)     │
                         ▼
        ┌────────────────────────────────────────────────────────────┐
        │       Ingest / ETL Job (inside AI microservice)           │
        │  - Periodic task / background worker                      │
        │  - Reads IntelOwl ES / DB reports via pyintelowl   or ES  │
        │  - Builds minimal campaign features (TYPE+OBS+CAT+TIME)   │
        │  - Uses Ollama embeddings                                 │
        │  - Upserts to Qdrant `intelowl_reports` collection        │
        │  - (Optional) recomputes HDBSCAN clusters → campaign_id   │
        └────────────────────────────────────────────────────────────┘

I propose a separate service/container(s) providing AI service due to following advantages:

• These will be part of IntelOwl in the same docker-compose.yml system, same docker network.
• Using IntelOwl's Durin API token auth for user authentication.
  • That enables us to apply user permissions.
  • Apply usage rate limiting per user.
• Minimal changes to IntelOwl's core codebase.
• No additional dependencies for IntelOwl.
• This also makes the AI service independently scalable.
• Fits well with IntelOwl's architectural idea.

A separate chatbot/AI stack will have following components/interfaces:

• FastAPI (Web Layer):

  • Why FastAPI?
    • Works natively with Pydantic and PydanticAI which is needed for multiple AI agents workflow.
    • Simple background tasks plus periodic cron jobs, enough to handle tasks needed.
    • It supports dependency injection and also has auto OpenAPI docs.
      Note: I'm open to working with Django is decided as such.
  • In addition to providing web interface for chat-bot, this container runs:
    • Background tasks triggered by users.
    • Periodic cron jobs that pulls analyzer reports from IntelOwl PostGreSQL/Elasticsearch.
      • Creates vectors using key data points from IOCs.
      • Recomputes HDBSCAN clusters -> campaigns.
      • These are key to answering queries related to "campaigns".

• PydanticAI to be used to define AI agents with specific roles:

  • RAG agent - interacts with Qdrant
  • Planner - Parsing of user queries and selecting analyzers/playbooks etc
  • Executor - Calls IntelOwl plugins using pyintelowl SDK as interface.
  • Summarizer - Generates final response

• Ollama (LLM Inference)

  • Local and isolated LLM using either Llama3 or Mistral
  • Use to generate summaries for IOCs which will then be stored in Qdrant
  • PydanticAI uses Ollama for LLM inference like understanding user queries and running analyzers/playbooks
  • Ollama and PydanticAI together result in typed responses

• Qdrant Vector DB

  • Stores IntelOwl reports/IOCs with embeddings and proximity indexes.
  • Uses Ollama's embedding models to create vectors.
  • Background tasks triggered by FastAPI container pulls new reports from IntelOwl's elasticsearch creates embeddings and upserts to collection.
  • Enables semantic search and cluster search for agents.

[rshaurya]

Hi @mlodic ,

Can you please review this proposal for GSoC 2026 project. I'll be happy to address any questions.