739 lines
33 KiB
TeX
739 lines
33 KiB
TeX
\documentclass[12pt]{article}
|
|
\usepackage[utf8]{inputenc}
|
|
\usepackage[T1]{fontenc}
|
|
\usepackage{lmodern}
|
|
\usepackage[a4paper,margin=1in]{geometry}
|
|
\usepackage{longtable}
|
|
\usepackage{booktabs}
|
|
\usepackage{array}
|
|
\usepackage{graphicx}
|
|
\usepackage{float}
|
|
\usepackage{ulem}
|
|
\usepackage{calc}
|
|
\usepackage[hidelinks]{hyperref}
|
|
\usepackage{tabularx}
|
|
\usepackage{xurl}
|
|
|
|
% Report-style paragraph spacing
|
|
\setlength{\parindent}{0pt}
|
|
\setlength{\parskip}{0.7em}
|
|
|
|
\begin{document}
|
|
|
|
\title{An Agentic Approach to Role-Specific Trainers (Dynavera)}
|
|
\author{Viswamedha Nalabotu\\2402117\\vxn217@student.bham.ac.uk\\University of Birmingham}
|
|
\date{}
|
|
\maketitle
|
|
|
|
\section*{AI Use Declaration}\label{ai-use-declaration}
|
|
|
|
In accordance with the University's academic
|
|
integrity guidelines, I declare that Large Language Models (LLMs) and
|
|
Chat Completion APIs were used in the preparation of this report and for
|
|
assisting with coding the project.
|
|
|
|
\textbf{Scope of AI Usage.} AI was used to assist in the structural organization, grammatical refinement, and syntactic formatting of the prose and technical descriptions.
|
|
|
|
\textbf{Prototyping \& Feasibility Research.} LLMs were employed during the R\&D phase to \textbf{scope technical requirements and perform feasibility checks}. This included generating "throwaway" boilerplate code to test the viability of specific architectural branches (e.g., comparing custom fine tuning against LangGraph API) and validating the compatibility of the Model Context Protocol (MCP) with the existing Django environment.
|
|
|
|
\textbf{Originality of Content.} All core architectural concepts, the design of the \emph{Dynavera} system, the "Distributed Agentic Pattern" logic, and the specific implementation strategies are my own original works.
|
|
|
|
\textbf{Fact-Checking and References.} Any external information or technical claims used to ground the AI\textquotesingle s output have been verified against the primary sources listed in the References section.
|
|
|
|
\textbf{Human Oversight.} I have critically reviewed, edited, and refined all AI-generated suggestions to ensure technical accuracy and alignment with the project's objectives.
|
|
|
|
\section*{Inspector Access Details}\label{inspector-access-details}
|
|
|
|
The public deployment for evaluation is available at:
|
|
\url{https://fyp.viswamedha.com}
|
|
|
|
Use the following credentials for testing:
|
|
|
|
\begin{center}
|
|
\begin{tabular}{p{0.22\linewidth} p{0.46\linewidth} p{0.22\linewidth}}
|
|
\toprule
|
|
Role & Email & Password \\
|
|
\midrule
|
|
Admin & admin@example.com & admin \\
|
|
Manager & haleisaac@example.com & password \\
|
|
User & j.thompson@example.com & password \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{center}
|
|
|
|
\textit{Note: I will try to keep the public website available, but the GPU node
|
|
runs on my home PC and may occasionally go offline. For reliable testing,
|
|
I recommend running the system locally on a machine with a CUDA-enabled GPU.}
|
|
|
|
Manager registration code (for signup): \texttt{MANAGER2026}
|
|
|
|
\section{Introduction}\label{introduction}
|
|
|
|
\subsection{Background: The Corporate Onboarding
|
|
Problem}\label{background-the-corporate-onboarding-problem}
|
|
|
|
When a startup or enterprise hires a new team member, they enter a
|
|
critical induction period requiring exposure to internal tools,
|
|
organizational context, and role-specific responsibilities.
|
|
Traditionally, this process is led by a senior colleague who acts as the
|
|
primary trainer and point of contact.
|
|
|
|
While effective, this model introduces a significant \emph{productivity
|
|
tax}: every hour spent training represents an hour of lost expert-level
|
|
output. This issue is amplified in startups, where teams are small,
|
|
budgets are constrained, and hiring decisions must be highly selective.
|
|
As a result, training often becomes inconsistent, slow, and difficult to
|
|
scale.
|
|
|
|
\subsection{Motivation}\label{motivation}
|
|
|
|
The motivation behind Dynavera is to reduce this productivity loss by
|
|
automating and enhancing role-specific training through AI. By replacing
|
|
static documentation and repeated human-led instruction with intelligent
|
|
agents, organizations can reduce reliance on ad-hoc mentorship while
|
|
preserving access to expert knowledge.
|
|
|
|
I observed this firsthand during my industrial placement at Siemens
|
|
DISW, where onboarding and training five incoming interns required a
|
|
significant investment of time in meetings, planning, and repeated
|
|
knowledge transfer to ensure a graceful handover. Despite careful
|
|
preparation, much of the training depended on individual availability
|
|
and tacit understanding, temporarily diverting effort away from feature
|
|
work. This experience highlighted how difficult it is to scale
|
|
onboarding without imposing a sustained productivity cost on senior
|
|
contributors.
|
|
|
|
By addressing this gap, Dynavera enables organizations to:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Scale Mentorship: Support multiple new hires simultaneously while
|
|
minimising senior staff intervention
|
|
\item
|
|
Standardize Quality: Ensure consistent depth, structure, and
|
|
assessment across all onboarding experiences
|
|
\item
|
|
Reduce Time-to-Productivity (TTP): Provide 24/7 access to contextual,
|
|
role-aware support through AI agents
|
|
\end{itemize}
|
|
|
|
Dynavera is designed as a proof-of-concept platform that transforms
|
|
onboarding into a dynamic, adaptive, and reusable training workflow.
|
|
|
|
\section{Project Background \&
|
|
Context}\label{project-background-context}
|
|
|
|
\subsection{The training bottleneck}\label{the-training-bottleneck}
|
|
|
|
Modern organizations face persistent challenges when standardizing and
|
|
scaling role-specific training:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Skill-Preloading Bias: Limited training capacity forces organizations
|
|
to favor candidates with prior experience in specific tools or
|
|
technology stacks, even when strong general aptitude or learning
|
|
ability may be sufficient.
|
|
\item
|
|
Restricted Talent Pool: By prioritizing immediate productivity over
|
|
trainability, organizations reduce access to diverse candidates who
|
|
could otherwise ramp up quickly with adequate onboarding support.
|
|
\item
|
|
Inflated Hiring Requirements: Role specifications often expand to
|
|
include non-essential tooling experience as a substitute for
|
|
structured training, increasing time-to-hire and cost.
|
|
\item
|
|
Uneven Knowledge Transfer: New hires are expected to ``already know''
|
|
systems and workflows, resulting in fragmented understanding and
|
|
slower integration into team-specific practices.
|
|
\end{itemize}
|
|
|
|
These constraints do more than just increase immediate onboarding
|
|
friction; their accumulation creates a compounding "productivity tax"
|
|
that stifles organizational growth. When left unaddressed, the aftermath
|
|
manifests in three critical areas:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Institutional Fragility: Over-reliance on tribal knowledge and
|
|
senior-led instruction creates single points of failure. If key
|
|
mentors depart, the lack of standardized, automated training leads to
|
|
a permanent loss of institutional memory and a degraded ability to
|
|
upskill replacements.
|
|
\item
|
|
Cultural and Innovation Stagnation: By reinforcing conservative hiring
|
|
through the prioritization of "plug-and-play" candidates over those
|
|
with high learning agility, organizations inadvertently filter out the
|
|
diverse perspectives and outsider logic that drive innovation. This
|
|
results in a homogenized workforce that excels at maintaining the
|
|
status quo but struggles to pivot.
|
|
\item
|
|
Compounded Opportunity Cost: The delay in reaching full productivity
|
|
(TTP) is not a linear loss. It represents a systemic lag in project
|
|
delivery and market responsiveness. For a scaling startup, the
|
|
cumulative effect of several hires operating at 50\% capacity for
|
|
months can be the difference between hitting a product milestone or
|
|
missing a market window entirely.
|
|
\end{itemize}
|
|
|
|
Ultimately, these factors coalesce into a cycle of restricted
|
|
scalability. The cost of adding new talent becomes so high that the
|
|
organization eventually stops growing simply to avoid the sustained pain
|
|
and friction of integration.
|
|
|
|
\subsection{Recent advancements in agentic
|
|
AI}\label{recent-advancements-in-agentic-ai}
|
|
|
|
Recent advances in Large Language Models (LLMs) and multi-agent systems
|
|
offer a viable solution to the onboarding bottleneck. Modern LLMs
|
|
demonstrate strong capabilities in natural language understanding,
|
|
contextual reasoning, and adaptive response generation, making them
|
|
well-suited for interactive, role-aware training scenarios. Unlike
|
|
static documentation, LLM-driven systems can dynamically tailor
|
|
explanations and guidance based on a user's specific role and prior
|
|
knowledge.
|
|
|
|
Rather than relying on a monolithic chatbot, Dynavera employs a
|
|
collection of specialized, collaborating agents. This modular approach
|
|
provides several distinct advantages:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Efficient Resource Allocation: By distributing responsibilities across
|
|
agents, the system maintains clearer reasoning boundaries. This
|
|
architecture reduces the computational overhead and "token bloat"
|
|
often associated with all-in-one prompts, leading to faster response
|
|
times and more efficient use of infrastructure resources.
|
|
\item
|
|
Targeted Maintainability and Explainability: Decoupled agents allow
|
|
for the optimization of specific components, such as the assessment or
|
|
knowledge retrieval modules, without requiring a total system
|
|
redesign. Because each agent has a narrow scope, the system provides
|
|
more transparent reasoning for its guidance, making it easier for
|
|
human supervisors to audit the AI\textquotesingle s logic.
|
|
\end{itemize}
|
|
|
|
Furthermore, agent collaboration enables training workflows that more
|
|
closely resemble human mentorship, where guidance and evaluation occur
|
|
in parallel. This architecture allows Dynavera to serve not only the
|
|
trainee but also the broader organizational stakeholders, including HR
|
|
departments and team leads. By capturing granular interaction data, the
|
|
system creates a comprehensive oversight landscape that includes:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Integral Progress Analytics: Automated reports and charts track
|
|
trainee milestones in real-time, allowing HR to identify exactly where
|
|
a new hire is thriving or stalling without manual check-ins.
|
|
\item
|
|
Continuous Curriculum Optimization: The system can flag specific
|
|
training modules that frequently cause friction or confusion,
|
|
suggesting content updates or sections that require a human-led
|
|
review.
|
|
\item
|
|
Strategic Escalation: By identifying complex, high-friction topics
|
|
that exceed the AI\textquotesingle s current scope, Dynavera can
|
|
pinpoint the exact moments requiring senior staff intervention. This
|
|
ensures that expert time is reserved for nuanced, high-value coaching
|
|
rather than repetitive technical basics.
|
|
\end{itemize}
|
|
|
|
This dual-purpose design ensures that while Dynavera scales the trainee
|
|
experience, it simultaneously provides the data-driven visibility and
|
|
administrative control required for long-term organizational growth.
|
|
|
|
\subsection{Theoretical Foundations}\label{theoretical-foundations}
|
|
|
|
Dynavera is grounded in two complementary system design paradigms that
|
|
enable scalable, context-aware onboarding:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Multi-Agent Systems (MAS): A collection of specialized AI agents
|
|
collaborate through structured communication to achieve complex
|
|
objectives that exceed the capability of a single monolithic model.
|
|
Within Dynavera, this enables separation of instructional delivery,
|
|
contextual reasoning, knowledge retrieval, and evaluation, improving
|
|
modularity, explainability, and system adaptability.
|
|
\item
|
|
Retrieval-Augmented Generation (RAG): Training responses are grounded
|
|
in authoritative, organization-specific documentation rather than
|
|
relying solely on a model's parametric knowledge. This ensures factual
|
|
accuracy, contextual relevance, and rapid adaptability as
|
|
organizational knowledge evolves.
|
|
\end{itemize}
|
|
|
|
To address data privacy and deployment constraints, Dynavera prioritizes
|
|
local inference using quantized open-weight models (e.g., Llama 3 in
|
|
GGUF format). This design choice reduces dependency on external cloud
|
|
APIs, supports offline or air-gapped environments, and aligns with
|
|
enterprise privacy requirements while maintaining acceptable inference
|
|
performance.
|
|
|
|
\subsection{Positioning Against Alternative
|
|
Approaches}\label{positioning-against-alternative-approaches}
|
|
|
|
Dynavera was designed against three practical alternatives. First,
|
|
human-only onboarding preserves expert nuance but does not scale well
|
|
and introduces recurring opportunity cost for senior staff. Second,
|
|
static LMS/document-first onboarding scales distribution but provides
|
|
limited adaptivity, weak context grounding during Q\&A, and little
|
|
operational traceability beyond completion events. Third, a single
|
|
general chatbot can improve interactivity, but it typically blends
|
|
curriculum, retrieval, assessment, and monitoring concerns into one
|
|
prompt surface, making governance and iterative improvement harder.
|
|
|
|
The Dynavera architecture chooses a middle path: specialized agent roles
|
|
within one orchestrated runtime, retrieval-grounded generation, and
|
|
persisted session state for reviewability. This trade-off accepts added
|
|
system complexity in exchange for improved modularity, clearer
|
|
responsibility boundaries, and stronger alignment between training
|
|
delivery and management oversight.
|
|
|
|
\subsection{Learning Origins}\label{learning-origins}
|
|
|
|
The design and implementation of Dynavera synthesize concepts developed
|
|
through university coursework and independent technical exploration:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Software Systems Architecture (CS301): Application of decoupled
|
|
service architectures using Django and Vue.js, alongside the use of
|
|
sidecar-style components to isolate model execution and agent
|
|
coordination.
|
|
\item
|
|
Machine Learning \& NLP: Practical experimentation with LoRA
|
|
fine-tuning and low-bit quantization (e.g., 4-bit inference via
|
|
bitsandbytes) to optimize model performance under local hardware
|
|
constraints.
|
|
\item
|
|
Full-Stack Development: Construction of production-oriented APIs using
|
|
Django REST Framework and responsive front-end interfaces with Vue 3,
|
|
enabling real-world interaction with agent-driven workflows.
|
|
\end{itemize}
|
|
|
|
Together, these learning sources informed both the architectural
|
|
decisions and implementation strategies underpinning Dynavera.
|
|
|
|
\section{Specification}\label{specification}
|
|
|
|
\subsection{System Overview}\label{system-overview}
|
|
|
|
Dynavera is implemented as a Distributed Agentic System, physically
|
|
decoupling the administrative and state management logic from the
|
|
high-latency inference workloads. As illustrated in Figure 1, the
|
|
architecture is split into two primary environments:
|
|
|
|
\begin{enumerate}
|
|
\def\labelenumi{\arabic{enumi}.}
|
|
\item
|
|
The Application Layer: A CPU-optimized environment running Django 5.x.
|
|
This layer handles user authentication, training state, and the MCP
|
|
Server, which acts as a standardized "data bridge" for the AI.
|
|
\item
|
|
The Inference Layer: A dedicated NVIDIA-based node running a FastAPI
|
|
inference engine. This layer handles Large Language Model (LLM)
|
|
execution, semantic chunking, and embedding generation.
|
|
\end{enumerate}
|
|
|
|
The "brain" of the system is the Orchestrator, which lives within a
|
|
Django Channels WebSocket consumer. It maintains a persistent,
|
|
full-duplex connection between the trainee and the distributed AI
|
|
components, ensuring real-time interactivity.
|
|
|
|
\includegraphics[width=\textwidth,keepaspectratio]{diagrams/system-architecture.png}
|
|
|
|
Figure 1: High-level system architecture of Dynavera, illustrating the
|
|
interaction between the user, orchestrator, inference layer and the
|
|
database.
|
|
|
|
\subsection{Technology stack}\label{technology-stack}
|
|
|
|
Dynavera is implemented as a modern full-stack application, with the
|
|
components presented in Table 1.
|
|
|
|
\begin{table}[H]
|
|
\centering
|
|
\begin{tabularx}{\linewidth}{p{0.22\linewidth} p{0.16\linewidth} X}
|
|
\toprule
|
|
Component & Technology & Rationale \\
|
|
\midrule
|
|
Frontend/UI & Vue 3 w/ TS & Typesafe, reactive UI enabling rapid iteration and maintainable component design \\
|
|
State Management & Pinia & Centralized, predictable state management for real-time training progress tracking \\
|
|
Backend/API & Django REST & Secure, mature framework supporting rapid development and scalable API design, informed by prior production experience \\
|
|
Database & PostgreSQL & Reliable, production-grade relational database for organizational and user data \\
|
|
Vector Store & PgVector & Efficient similarity search over embedded training documentation via PostgreSQL \\
|
|
MCP Router & Python & Provides a standardized interface for agents to query data using Model Context Protocol. \\
|
|
\bottomrule
|
|
\end{tabularx}
|
|
\caption{Architectural components of the Dynavera platform, including frontend, backend, and AI integration technologies.}
|
|
\end{table}
|
|
|
|
This stack was selected to balance modularity, rapid iteration, and production readiness.
|
|
A decoupled frontend-backend architecture lets the UI and API evolve independently, while PostgreSQL
|
|
with pgvector provides one ACID-compliant store for both relational state and vector retrieval.
|
|
|
|
To preserve performance and control, orchestration is implemented in native Python rather than heavier
|
|
framework abstractions such as LangChain. This keeps agent state handling explicit, reduces latency in the WebSocket loop,
|
|
and supports local execution, data ownership, and architectural transparency during early-stage development.
|
|
|
|
\subsection{Design Philosophy: The Distributed Agentic
|
|
Pattern}\label{design-philosophy-the-distributed-agentic-pattern}
|
|
|
|
Dynavera leverages the Model Context Protocol (MCP) to solve the
|
|
"context gap" in corporate onboarding. Rather than providing the LLM
|
|
with a static, bloated prompt, the system utilizes a Sidecar Tooling
|
|
approach:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
The MCP Server as a Translator: Integrated directly into the Django
|
|
ecosystem, the MCP layer exposes specific "Tools" (e.g.,
|
|
search\_knowledge, get\_user\_progress) to the AI. This allows the
|
|
model to query the organization\textquotesingle s private data safely
|
|
and efficiently.
|
|
\item
|
|
The Streaming Orchestration Loop: Unlike traditional request-response
|
|
cycles, the system uses an asynchronous loop. The Orchestrator manages
|
|
the "triangle of communication":
|
|
|
|
\begin{enumerate}
|
|
\def\labelenumi{\arabic{enumi}.}
|
|
\item
|
|
Receives user input via WebSockets.
|
|
\item
|
|
Prompts the GPU Layer for a decision.
|
|
\item
|
|
If the AI requests data, the Orchestrator calls the MCP Tool
|
|
internally.
|
|
\item
|
|
Streams the final result back to the user with minimal latency.
|
|
\end{enumerate}
|
|
\end{itemize}
|
|
|
|
This separation ensures that the core application remains responsive
|
|
while the heavy lifting of "thinking" and "embedding" happens on
|
|
specialized hardware. It transforms the onboarding experience from a
|
|
static tutorial into a Streaming Agentic System that adapts to the
|
|
trainee in real-time.
|
|
|
|
\section{Implementation Overview}\label{implementation-overview}
|
|
|
|
\subsection{Backend Realisation}\label{backend-realisation}
|
|
|
|
This section describes how the architecture in Chapter 3 is implemented
|
|
in the current Dynavera codebase. The backend is organized into three
|
|
primary Django app domains: accounts (users, organizations, roles,
|
|
membership), knowledge (training files, ingestion, chunk/embedding
|
|
persistence), and onboarding (sessions, orchestration, generated flows,
|
|
interaction logs). This separation keeps responsibility boundaries
|
|
explicit while allowing shared infrastructure (authentication, ORM, API
|
|
routing, and permissions) to remain centralized.
|
|
|
|
The API surface is intentionally split by interaction pattern. Standard
|
|
management operations are handled through Django REST Framework (for
|
|
example role membership, training file upload, and session endpoints),
|
|
while orchestration-time interaction uses Django Channels over
|
|
WebSockets at /ws/onboarding/\textless session\_uuid\textgreater/. This
|
|
allows the platform to handle both CRUD-style workflows and
|
|
long-running, stateful agent interactions without forcing either pattern
|
|
into the other.
|
|
|
|
For ingestion, the backend follows an asynchronous execution path:
|
|
uploaded files are stored as TrainingFile records, and a post-save
|
|
trigger enqueues background processing through Celery (Redis broker).
|
|
This prevents heavy preprocessing from blocking request-response latency
|
|
on the main web process.
|
|
|
|
Persistence is model-driven and traceable. Session state, progress,
|
|
generated onboarding structures, and interaction events are stored in
|
|
Django models, enabling deterministic recovery and auditability of the
|
|
onboarding lifecycle. In implementation terms, the backend is less a
|
|
single monolith and more a coordinated runtime: REST for management,
|
|
WebSockets for orchestration, Celery for heavy async jobs, and
|
|
PostgreSQL/pgvector as a unified data plane.
|
|
|
|
\subsection{Data Flow}\label{data-flow}
|
|
|
|
\subsubsection{Knowledge Ingestion
|
|
Workflow}\label{knowledge-ingestion-workflow}
|
|
|
|
Figure 2 shows the ingestion data flow between the User/UI, Django REST
|
|
API, Celery worker, PostgreSQL/pgvector database, and GPU endpoint.
|
|
|
|
\includegraphics[width=5.75521in,height=5.14354in]{diagrams/embedding-data-flow.png}
|
|
|
|
Figure 2: Knowledge ingestion data flow diagram, illustrating the
|
|
interaction between the user, REST API, Celery Worker, Pgvector database
|
|
\& GPU endpoint.
|
|
|
|
\underline{Asynchronous processing with Celery (Redis broker)}\\
|
|
When a manager uploads a training file from the UI, the file is sent to
|
|
the Django REST API and stored as a TrainingFile record with an initial
|
|
ingesting status. A post-save hook then enqueues a Celery task via
|
|
Redis, so heavy processing runs outside the request/response cycle. This
|
|
keeps the web server responsive even for large documents.
|
|
|
|
\underline{Semantic chunking on the GPU endpoint}\\
|
|
The Celery task extracts raw text from uploaded files (PDF/DOCX/TXT),
|
|
batches long content, and calls the GPU service at /v1/semantic-chunk.
|
|
The service performs sentence-level semantic breakpoint detection using
|
|
embedding-distance thresholds, then returns coherent chunks with
|
|
embeddings. This avoids naive fixed-size splits that can break context
|
|
mid-concept.
|
|
|
|
\underline{Vector storage and retrieval with pgvector}\\
|
|
Returned chunk embeddings are stored in RoleRagDocument.embedding (768
|
|
dimensions) in PostgreSQL using pgvector, linked relationally to role
|
|
and source file metadata. Retrieval is performed in SQL using
|
|
cosine-distance ranking and top-k selection, allowing role filtering and
|
|
similarity search in one query path.
|
|
|
|
\subsubsection{Agent Orchestration Workflow
|
|
(Simplified)}\label{agent-orchestration-workflow-simplified}
|
|
|
|
\includegraphics[width=6.15132in,height=6.00619in]{diagrams/agent-orchestration-loop.png}
|
|
|
|
Figure 3: Agent orchestration data flow diagram, illustrating the
|
|
interaction between the user/UI, WebSocket Consumer, MCP Router, GPU
|
|
Endpoint \& Pgvector database.
|
|
|
|
Figure 3 presents a simplified view of the orchestration loop. To keep
|
|
the diagram readable, it collapses multiple internal components into a
|
|
single orchestration path and does not show each specialist agent (for
|
|
example curriculum, knowledge, assessment, and monitor agents) as
|
|
separate lifelines.
|
|
|
|
The orchestration layer is implemented in a Django Channels WebSocket
|
|
consumer (/ws/onboarding/\textless session\_uuid\textgreater/). This
|
|
keeps a persistent two-way connection between the frontend and backend
|
|
so the client can receive live status events (for example thinking/tool
|
|
execution/completed) without repeated polling. Once connected, the UI
|
|
sends a query or action payload, and the orchestrator coordinates model
|
|
inference and tool usage.
|
|
|
|
The core loop is tool-aware. The orchestrator sends chat-completion
|
|
requests to the inference endpoint with tool definitions attached. If
|
|
the model returns a tool call, control is passed to an MCP router, which
|
|
executes backend tools such as search\_knowledge and update\_progress.
|
|
For knowledge retrieval, the router generates an embedding for the query
|
|
via the GPU endpoint, performs cosine-distance top-k lookup against
|
|
pgvector-backed role documents, and returns the retrieved context to the
|
|
orchestrator. The tool result is then injected back into the message
|
|
sequence before the next model call.
|
|
|
|
State is persisted through session and flow models (for example
|
|
onboarding session state updates and generated flow storage), while
|
|
interaction events are emitted to the frontend over the same WebSocket
|
|
channel. This allows the system to remain responsive and traceable while
|
|
still supporting retrieval-grounded generation.
|
|
|
|
Note: Per-agent branching logic and detailed phase-specific workflows
|
|
are omitted to keep a simplified diagram. A more detailed diagram is
|
|
available in the repository (TBM).
|
|
|
|
\subsection{Agentic Runtime
|
|
Structure}\label{agentic-runtime-structure}
|
|
|
|
Dynavera implements a multi-agent training workflow through
|
|
role-specialized configurations executed inside a shared orchestration
|
|
runtime. Conceptually, the system uses four agent roles: Curriculum
|
|
Agent (CA), Knowledge Agent (KA), Assessment Agent (AA), and Progress
|
|
Monitor Agent (PMA). In practice, these are represented by agent
|
|
configuration records and invoked by orchestration logic rather than
|
|
isolated microservices, which keeps deployment complexity manageable
|
|
while preserving modular behavior.
|
|
|
|
The Curriculum Agent (CA) defines module order and high-level learning
|
|
path. The Knowledge Agent (KA) generates grounded instructional content
|
|
and relies on retrieval tools when additional context is needed. The
|
|
Assessment Agent (AA) generates evaluation artifacts (for example quiz
|
|
structures) to validate understanding. The Progress Monitor Agent (PMA)
|
|
evaluates learner trajectory and produces concise progress-oriented
|
|
feedback from session context. Together, these roles form a coordinated
|
|
runtime where each stage contributes to structured onboarding output.
|
|
|
|
Tool-mediated grounding is handled through the MCP router. During
|
|
orchestration, model responses may include tool calls; the runtime
|
|
executes approved tools (such as search\_knowledge and
|
|
update\_progress), retrieves contextual evidence from pgvector-backed
|
|
documents, and injects those results back into the message loop before
|
|
final answer generation. This keeps generation anchored in role-specific
|
|
organizational material while preserving a controlled boundary between
|
|
model reasoning and data access.
|
|
|
|
\subsection{Workflow Implementation}\label{workflow-implementation}
|
|
|
|
The implemented training workflow follows a staged operational sequence
|
|
from administrative setup to learner progression. First,
|
|
administrators/managers configure role context and upload role-relevant
|
|
documentation through the application interface. These documents are
|
|
processed through the ingestion pipeline and converted into vectorized
|
|
knowledge records linked to role scope.
|
|
|
|
Next, a trainee enters a role-specific onboarding session. The frontend
|
|
opens a persistent WebSocket connection to the orchestration endpoint
|
|
and submits user prompts/actions as session events. The orchestrator
|
|
resolves the active configuration for that role/session, runs model
|
|
inference, executes retrieval tools when required, and emits structured
|
|
runtime events (status/tool/completion) back to the client.
|
|
|
|
During guided learning, module content generation, context retrieval,
|
|
and assessment output are coordinated in sequence. The curriculum phase
|
|
determines structure, the knowledge phase builds role-grounded
|
|
instructional content, and the assessment phase constructs evaluative
|
|
checkpoints. Final assessment grading follows a mixed strategy: multiple
|
|
choice responses are deterministically compared against configured
|
|
correct options, while non-multiple-choice responses are agent-graded.
|
|
Per-question grading outcomes are persisted in session state for review
|
|
and feedback rendering.
|
|
|
|
Progress monitoring then summarizes current status using persisted
|
|
session state and completed interactions. In the implemented UI path,
|
|
AI monitor inference is only triggered after onboarding completion;
|
|
before completion, the system presents a local progress summary.
|
|
When available, monitor judgements are informed by prior final-quiz
|
|
question/answer evidence and saved grading details. This keeps learning
|
|
flow adaptive without abandoning traceability.
|
|
|
|
Finally, workflow state is persisted throughout execution: user
|
|
responses, progress markers, generated flow structures, and interaction
|
|
logs are stored in backend models. This enables continuity across
|
|
reconnects, supports progress review, and allows the system to advance,
|
|
pause, or remediate onboarding based on recorded outcomes rather than
|
|
transient in-memory state.
|
|
|
|
\section{Results \& Conclusion -
|
|
Draft}\label{results-conclusion---draft}
|
|
|
|
\subsection{System Performance \& Evaluation}\label{system-performance-evaluation}
|
|
|
|
The implementation of Dynavera successfully demonstrates the viability
|
|
of a distributed agentic approach to role-specific training. By
|
|
decoupling the application layer from the inference layer, the system
|
|
maintained a responsive UI even during high-latency LLM reasoning
|
|
phases.
|
|
|
|
Key results observed during testing include:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
\textbf{Retrieval Accuracy:} The use of \textbf{semantic chunking}
|
|
significantly reduced context fragmentation compared to fixed-length
|
|
splitting, allowing the Knowledge Agent to maintain higher grounding
|
|
accuracy during complex RAG queries.
|
|
\item
|
|
\textbf{Orchestration Latency:} The WebSocket-based orchestration loop
|
|
provided a near-instantaneous feedback loop for "thinking" and "tool
|
|
execution" states, which is critical for maintaining user engagement
|
|
in an interactive learning environment.
|
|
\item
|
|
\textbf{Resource Efficiency:} 4-bit quantization enabled the
|
|
deployment of Llama 3 on consumer-grade hardware without a perceptible
|
|
loss in the agent's ability to follow the structured curriculum
|
|
defined by the Curriculum Agent.
|
|
\end{itemize}
|
|
|
|
\subsubsection{Conclusion}\label{conclusion}
|
|
|
|
Dynavera addresses the "Productivity Tax" of corporate onboarding by
|
|
transforming static documentation into a dynamic, role-aware mentorship
|
|
experience. By leveraging the Model Context Protocol (MCP) and a
|
|
distributed architecture, the platform proves that complex AI training
|
|
workflows can be delivered in a private, scalable, and operationally
|
|
practical manner. While this project serves as a proof-of-concept, the
|
|
modular nature of the specialist agents provides a clear path for future
|
|
expansion into more nuanced, multi-modal onboarding scenarios.
|
|
|
|
\begin{figure*}[b]
|
|
\centering
|
|
\includegraphics[width=\textwidth,height=3.2in,keepaspectratio]{diagrams/home-page.png}
|
|
\caption{Home page of Dynavera.}
|
|
\end{figure*}
|
|
|
|
\begin{figure*}[b]
|
|
\centering
|
|
\includegraphics[width=\textwidth,height=3.2in,keepaspectratio]{diagrams/organization-page.png}
|
|
\caption{Organization management view.}
|
|
\end{figure*}
|
|
|
|
\begin{figure*}[b]
|
|
\centering
|
|
\includegraphics[width=\textwidth,height=3.2in,keepaspectratio]{diagrams/onboarding-loading-page.png}
|
|
\caption{Onboarding generation/loading state.}
|
|
\end{figure*}
|
|
|
|
\begin{figure*}[b]
|
|
\centering
|
|
\includegraphics[width=\textwidth,height=3.2in,keepaspectratio]{diagrams/onboarding-content-page.png}
|
|
\caption{Onboarding content delivery view.}
|
|
\end{figure*}
|
|
|
|
\section{References}\label{references}
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Anthropic (2024). Model Context Protocol (MCP) Specification.
|
|
Available at: \url{https://modelcontextprotocol.io} (Accessed: 9 March
|
|
2026).
|
|
\item
|
|
Hugging Face (2024). Introduction to Model Context Protocol (MCP).
|
|
Available at:
|
|
\url{https://huggingface.co/learn/mcp-course/en/unit1/key-concepts}
|
|
(Accessed: 9 March 2026).
|
|
\item
|
|
LangChain (2024). LangGraph: Building Stateful, Multi-agent
|
|
Applications with LLMs. Available at:
|
|
\url{https://docs.langchain.com/oss/python/langgraph/workflows-agents}
|
|
(Accessed: 9 March 2026).
|
|
\item
|
|
Meta AI (2024). Llama 3: Open-weight Large Language Models. Available
|
|
at: \url{https://llama.meta.com/llama3/} (Accessed: 9 March 2026).
|
|
\item
|
|
PostgreSQL Global Development Group (2024). pgvector: Open-source
|
|
vector similarity search for PostgreSQL. Available at:
|
|
\url{https://github.com/pgvector/pgvector} (Accessed: 9 March 2026).
|
|
\item
|
|
Pinecone (2023). Retrieval Augmented Generation (RAG) and Semantic
|
|
Search. Available at:
|
|
\url{https://www.pinecone.io/learn/retrieval-augmented-generation/}
|
|
(Accessed: 9 March 2026).
|
|
\item
|
|
Dettmers, T. (2023). 4-bit Quantization and Bitsandbytes for LLMs.
|
|
Available at:
|
|
\url{https://huggingface.co/blog/4bit-transformers-bitsandbytes} (Accessed:
|
|
9 March 2026).
|
|
\item
|
|
vLLM Team (2024). High-Throughput Serving with PagedAttention.
|
|
Available at: \url{https://vllm.ai} (Accessed: 9 March 2026).
|
|
\item
|
|
Django Software Foundation (2024). Django Channels: Real-time
|
|
WebSockets for Python. Available at:
|
|
\url{https://channels.readthedocs.io/en/stable/} (Accessed: 9 March 2026).
|
|
\item
|
|
Django Software Foundation (2024). Django Documentation.
|
|
Available at: \url{https://docs.djangoproject.com/} (Accessed: 9 March 2026).
|
|
\item
|
|
Encode OSS (2024). Django REST framework Documentation.
|
|
Available at: \url{https://www.django-rest-framework.org/} (Accessed: 9 March 2026).
|
|
\item
|
|
Celery Project (2024). Celery Documentation. Available at: \url{https://docs.celeryq.dev/} (Accessed: 9 March 2026).
|
|
\item
|
|
Redis Ltd. (2024). Redis Documentation. Available at: \url{https://redis.io/docs/} (Accessed: 9 March 2026).
|
|
\item
|
|
FastAPI (2024). FastAPI Documentation. Available at: \url{https://fastapi.tiangolo.com/} (Accessed: 9 March 2026).
|
|
\item
|
|
UKPLab / SBERT (2024). Sentence-Transformers Documentation.
|
|
Available at: \url{https://www.sbert.net/} (Accessed: 9 March 2026).
|
|
\item
|
|
Abetlen (2024). llama-cpp-python Documentation.
|
|
Available at: \url{https://github.com/abetlen/llama-cpp-python} (Accessed: 9 March 2026).
|
|
\item
|
|
ggml-org (2024). llama.cpp Documentation.
|
|
Available at: \url{https://github.com/ggml-org/llama.cpp} (Accessed: 9 March 2026).
|
|
\item
|
|
PyTorch Team (2024). PyTorch Documentation. Available at: \url{https://pytorch.org/docs/} (Accessed: 9 March 2026).
|
|
\end{itemize}
|
|
|
|
\end{document}
|
|
|