EC-RAG new feature update#2445
Open
Yongbozzz wants to merge 1 commit into
Open
Conversation
Signed-off-by: Yongbozzz <yongbo.zhu@intel.com>
Dependency ReviewThe following issues were found:
VulnerabilitiesEdgeCraftRAG/edgecraftrag/requirements.txtLicense IssuesEdgeCraftRAG/cli/requirements.txt
EdgeCraftRAG/cli/setup.py
EdgeCraftRAG/edgecraftrag/requirements.txt
EdgeCraftRAG/ui/vue/package.json
Scanned Files
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates EdgeCraftRAG across the UI, backend, and deployment/tooling to add OVMS-based remote inference support, enhance agent capabilities/UX (including activation toggles and richer config handling), and tighten prompt/citation formatting for RAG outputs.
Changes:
- Added OVMS as a new inference/model type end-to-end (API, pipeline/generator wiring, validation endpoint, UI strings, compose scripts).
- Improved Agent UX and behavior (activate/deactivate endpoint + UI controls, config preview/copy, retrieval context serialization).
- Added/updated deployment and tooling artifacts (Helm chart, new run scripts, docs updates, dependency/version refreshes).
Reviewed changes
Copilot reviewed 99 out of 106 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| EdgeCraftRAG/ui/vue/vite.config.ts | Adds optional local proxy routing for API/chatbot endpoints via env vars. |
| EdgeCraftRAG/ui/vue/src/views/settings/components/Pipeline/components/UpdateDialog/PostProcessor.vue | Lowers default rerank top_n and centralizes it in a constant. |
| EdgeCraftRAG/ui/vue/src/views/settings/components/Agent/components/Table.vue | Adds configs popover (JSON viewer + copy) and activate/deactivate controls. |
| EdgeCraftRAG/ui/vue/src/views/settings/components/Agent/components/DynamicConfigs.vue | Improves long-string editing UX with preview/expand-to-textarea behavior. |
| EdgeCraftRAG/ui/vue/src/views/settings/components/Agent/columnsList.ts | Adjusts agent table columns (widths, titles, operation column width). |
| EdgeCraftRAG/ui/vue/src/views/chatbot/components/KnowledgeBase/UpdateKBDialog/Basic.vue | Refactors formatting; adjusts default vector URL and validation flow. |
| EdgeCraftRAG/ui/vue/src/views/chatbot/components/Chatbot/MessageItem.vue | Tooltip UI tweak (no arrow) and import ordering cleanup. |
| EdgeCraftRAG/ui/vue/src/views/chatbot/components/Chatbot/ChatHistory.vue | Switches agent activation call to new API method. |
| EdgeCraftRAG/ui/vue/src/types/global.d.ts | Extends TableColumns with width. |
| EdgeCraftRAG/ui/vue/src/theme/common.less | Removes unused horizontal-form-item deep selector block. |
| EdgeCraftRAG/ui/vue/src/i18n/zh.ts | Adds OVMS + agent activation/deactivation strings and formatting changes. |
| EdgeCraftRAG/ui/vue/src/i18n/en.ts | Adds OVMS + agent activation/deactivation strings and formatting changes. |
| EdgeCraftRAG/ui/vue/src/auto-imports.d.ts | Regenerates auto-import typings (format/license header changes). |
| EdgeCraftRAG/ui/vue/src/api/pipeline/index.ts | Adds OVMS connectivity check API wrapper. |
| EdgeCraftRAG/ui/vue/src/api/agent/index.ts | Adds new PATCH endpoint wrapper for agent active state. |
| EdgeCraftRAG/ui/vue/package.json | Updates dependency versions (axios, vue-json-pretty). |
| EdgeCraftRAG/ui/vue/components.d.ts | Regenerates component typings (format/license header changes). |
| EdgeCraftRAG/ui/vue/.gitignore | Stops ignoring package-lock.json. |
| EdgeCraftRAG/tools/run_vllm_on_double.sh.old | Adds legacy helper script for multi-container vLLM runs. |
| EdgeCraftRAG/tools/run_ovms_container.sh | Adds compose-based OVMS profile runner with status/log helpers. |
| EdgeCraftRAG/tools/run_ov_container.sh | Adds compose-based local OpenVINO runner with status/log helpers. |
| EdgeCraftRAG/tools/run_ov_baremetal.sh | Adds bare-metal runner managing server/mega/ui processes and ports. |
| EdgeCraftRAG/tools/build_images.sh | Removes header comment lines (script remains functionally similar). |
| EdgeCraftRAG/tests/test_pipeline_local_llm.json | Updates OpenVINO model path in test pipeline config. |
| EdgeCraftRAG/tests/configs/test_pipeline_local_llm.json | Updates OpenVINO model path in test pipeline config. |
| EdgeCraftRAG/tests/common.sh | Minor formatting/newline adjustment. |
| EdgeCraftRAG/README.md | Updates “What’s New” bullets. |
| EdgeCraftRAG/kubernetes/helm/values.yaml | Introduces Helm values for images, ports, env, model paths, GPU groups. |
| EdgeCraftRAG/kubernetes/helm/templates/service-llm-serving-xpu.yaml | Adds service manifest for llm-serving-xpu. |
| EdgeCraftRAG/kubernetes/helm/templates/service-edgecraftrag-ui.yaml | Adds NodePort service for UI. |
| EdgeCraftRAG/kubernetes/helm/templates/service-edgecraftrag-server.yaml | Adds service for pipeline server. |
| EdgeCraftRAG/kubernetes/helm/templates/service-ecrag.yaml | Adds service for mega/reverse-proxy component. |
| EdgeCraftRAG/kubernetes/helm/templates/deployment-edgecraftrag-ui.yaml | Adds UI deployment manifest with hostPath volumes. |
| EdgeCraftRAG/kubernetes/helm/templates/deployment-ecrag.yaml | Adds mega/reverse-proxy deployment manifest with hostPath volumes. |
| EdgeCraftRAG/kubernetes/helm/templates/daemonset-llm-serving-xpu.yaml | Adds daemonset for vLLM serving on GPU nodes. |
| EdgeCraftRAG/kubernetes/helm/templates/daemonset-edgecraftrag-server.yaml | Adds daemonset for pipeline server on GPU nodes. |
| EdgeCraftRAG/kubernetes/helm/templates/configmap-env.yaml | Adds ConfigMap for shared env values across workloads. |
| EdgeCraftRAG/kubernetes/helm/README.md | Adds Helm usage documentation. |
| EdgeCraftRAG/kubernetes/helm/README_zh.md | Adds Chinese Helm usage documentation. |
| EdgeCraftRAG/kubernetes/helm/Chart.yaml | Adds Helm chart metadata. |
| EdgeCraftRAG/edgecraftrag/utils.py | Adds safe prompt template path resolution and model path resolution helper. |
| EdgeCraftRAG/edgecraftrag/server.py | Changes restore error handling to print restore errors. |
| EdgeCraftRAG/edgecraftrag/requirements.txt | Pins/updates backend Python dependencies and adds GenAI-related llama-index packages. |
| EdgeCraftRAG/edgecraftrag/prompt_template/default_prompt.txt | Strengthens citation formatting requirements for DOCUMENT_NODE evidence. |
| EdgeCraftRAG/edgecraftrag/controllers/sessionmgr.py | Extends remote history token sizing logic to OVMS. |
| EdgeCraftRAG/edgecraftrag/controllers/pipelinemgr.py | Adds more aggressive cleanup for pipeline removal and OV inference objects. |
| EdgeCraftRAG/edgecraftrag/controllers/modelmgr.py | Adds OVMS model type; adds GenAI model variants and model path resolving. |
| EdgeCraftRAG/edgecraftrag/controllers/knowledge_basemgr.py | Adds cleanup for indexer model resources on KB delete. |
| EdgeCraftRAG/edgecraftrag/controllers/filemgr.py | Import ordering/formatting tweak. |
| EdgeCraftRAG/edgecraftrag/controllers/agentmgr.py | Clamps agent config to pipeline limits; returns retrievals from agent execution. |
| EdgeCraftRAG/edgecraftrag/config_repository.py | Pipeline persistence captures updated prompt content for ChatQnA generator. |
| EdgeCraftRAG/edgecraftrag/components/retriever.py | Minor formatting/string literal changes. |
| EdgeCraftRAG/edgecraftrag/components/query_preprocess.py | Uses remote_endpoint when available instead of vllm_endpoint. |
| EdgeCraftRAG/edgecraftrag/components/postprocessor.py | Stores default_top_n on rerank processor. |
| EdgeCraftRAG/edgecraftrag/components/ov_llamaindex_helper.py | Adds OpenVINO GenAI embedding and reranking adapters for llama-index. |
| EdgeCraftRAG/edgecraftrag/components/model.py | Adds GenAI model classes, model-path resolution, and benchmark streaming helpers. |
| EdgeCraftRAG/edgecraftrag/components/knowledge_base.py | Minor formatting changes. |
| EdgeCraftRAG/edgecraftrag/components/indexer.py | Import ordering tweak. |
| EdgeCraftRAG/edgecraftrag/components/generator.py | Adds OVMS endpoint support; DOCUMENT_NODE block formatting; GenAI benchmark streaming hooks. |
| EdgeCraftRAG/edgecraftrag/components/benchmark.py | Adds GenAI benchmark insertion methods; adds timeout for vLLM metrics fetch. |
| EdgeCraftRAG/edgecraftrag/components/agents/utils.py | Adds URL-encoding and DOCUMENT_NODE block builder utilities. |
| EdgeCraftRAG/edgecraftrag/components/agents/simple.py | Makes Simple agent configs richer (templated prompts) and tracks retrievals for API output. |
| EdgeCraftRAG/edgecraftrag/components/agents/deep_search/deep_search.py | Moves DeepSearch defaults into code; emits DOCUMENT_NODE blocks for citations. |
| EdgeCraftRAG/edgecraftrag/components/agents/deep_search/config.py | Adds built-in default config dict + getter instead of JSON file. |
| EdgeCraftRAG/edgecraftrag/components/agents/deep_search/cfgs/default.json | Removes JSON default config (moved into Python). |
| EdgeCraftRAG/edgecraftrag/components/agent.py | Adds shared Retrieval model and retrievals tracking on Agent base. |
| EdgeCraftRAG/edgecraftrag/base.py | Adds OVMS to ModelType and InferenceType. |
| EdgeCraftRAG/edgecraftrag/api/v1/system.py | Keeps NPU device in returned device list. |
| EdgeCraftRAG/edgecraftrag/api/v1/pipeline.py | Adds OVMS wiring in generators; clamps top-k; improves restore loop error handling. |
| EdgeCraftRAG/edgecraftrag/api/v1/knowledge_base.py | Refactors imports/formatting; adds safer restore handling and top-k checks. |
| EdgeCraftRAG/edgecraftrag/api/v1/data.py | Import ordering tweak; minor formatting changes. |
| EdgeCraftRAG/edgecraftrag/api/v1/chatqna.py | Adds OVMS connectivity check endpoint; adds agent retrieval serialization in RagQnA output. |
| EdgeCraftRAG/edgecraftrag/api/v1/agent.py | Adds PATCH endpoint to set agent active state; removes pipeline-binding side effects. |
| EdgeCraftRAG/edgecraftrag/api_schema.py | Extends generator input schema with ovms_endpoint. |
| EdgeCraftRAG/docs/Explore_Edge_Craft_RAG.md | Formatting and content tweaks in kbadmin section. |
| EdgeCraftRAG/docs/Explore_Edge_Craft_RAG_zh.md | Formatting tweaks. |
| EdgeCraftRAG/docs/API_Guide.md | Minor formatting tweak. |
| EdgeCraftRAG/docs/Advanced_Setup.md | Updates default paths and adds OVMS deployment instructions. |
| EdgeCraftRAG/docs/Advanced_Setup_zh.md | Updates default paths and adds OVMS deployment instructions (Chinese). |
| EdgeCraftRAG/Dockerfile.server | Adds packages and NPU driver installation; clones openvino.genai for benchmarking tools. |
| EdgeCraftRAG/docker_compose/intel/gpu/arc/README.md | Formatting tweaks; doc cleanups. |
| EdgeCraftRAG/docker_compose/intel/gpu/arc/README_zh.md | Formatting tweaks; doc cleanups. |
| EdgeCraftRAG/docker_compose/intel/gpu/arc/compose.yaml | Adds OVMS profile container; proxy env propagation; adds ACCEL device mapping; image changes. |
| EdgeCraftRAG/cli/setup.py | Adds CLI packaging setup for ecrag console entrypoint. |
| EdgeCraftRAG/cli/requirements.txt | Adds CLI runtime deps. |
| EdgeCraftRAG/cli/README.md | Adds CLI usage documentation. |
| EdgeCraftRAG/cli/quickstart.py | Adds CLI quickstart helper and connection test. |
| EdgeCraftRAG/cli/pyproject.toml | Adds minimal build-system metadata. |
| EdgeCraftRAG/cli/config.py | Adds env-based CLI configuration. |
| EdgeCraftRAG/cli/init.py | Adds package marker and license header. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+116
to
+126
| def insert_llm_data_genai(self, idx, input_token_size=-1, model=None): | ||
| if self.is_enabled(): | ||
| metrics = {} | ||
| metrics["input_token_size"] = input_token_size | ||
| metrics["output_token_size"] = model().perf_metrics.get_num_generated_tokens() | ||
| metrics["generation_time"] = model().perf_metrics.get_inference_duration().mean/1000 | ||
| metrics["first_token_latency"] = model().perf_metrics.get_ttft().mean/1000 | ||
| metrics["other_tokens_avg_latency"] = model().perf_metrics.get_tpot().mean/1000 | ||
|
|
||
| self.llm_data_list[idx] = metrics | ||
|
|
| - ${PIPELINE_SERVICE_PORT:-16010}:${PIPELINE_SERVICE_PORT:-16010} | ||
| devices: | ||
| - /dev/dri:/dev/dri | ||
| - ${ACCEL_DEV:-/dev/dri:/dev/dri} |
Comment on lines
9
to
+17
| RUN echo -e "Types: deb\nURIs: https://ppa.launchpadcontent.net/kobuk-team/intel-graphics/ubuntu/\nSuites: questing\nComponents: main\nSigned-By: /etc/apt/trusted.gpg.d/driver.asc" > /etc/apt/sources.list.d/driver.sources | ||
| RUN apt-get update && apt-get install -y libze-intel-gpu1 libze1 intel-metrics-discovery intel-opencl-icd clinfo intel-gsc && apt-get install -y libze-intel-gpu1 libze1 intel-metrics-discovery intel-opencl-icd clinfo intel-gsc && apt-get install -y libze-dev intel-ocloc libze-intel-gpu-raytracing | ||
|
|
||
| RUN apt-get update && apt-get install -y libze-intel-gpu1 libze1 intel-metrics-discovery intel-opencl-icd clinfo intel-gsc && apt-get install -y libze-intel-gpu1 libze1 intel-metrics-discovery intel-opencl-icd clinfo intel-gsc && apt-get install -y libze-dev intel-ocloc libze-intel-gpu-raytracing libglib2.0-0 libgl1 libmagic-dev poppler-utils tesseract-ocr pandoc | ||
| RUN url=$(curl -s https://api.github.com/repos/intel/linux-npu-driver/releases/latest \ | ||
| | jq -r '.assets[] | select(.name | test("ubuntu2404\\.tar\\.gz$")) | .browser_download_url' \ | ||
| | head -n 1) \ | ||
| && wget -O /tmp/linux-npu-driver.tar.gz "$url" \ | ||
| && tar -xf /tmp/linux-npu-driver.tar.gz -C /tmp \ | ||
| && dpkg -i /tmp/*.deb || apt-get install -f -y \ | ||
| && rm -rf /tmp/linux-npu-driver* |
Comment on lines
+235
to
+237
| flag = pl.check_top_k(ctx.get_knowledge_mgr().get_all_knowledge_bases()) | ||
| if flag == True: | ||
| await save_pipeline_configurations("update", pl) |
Comment on lines
+194
to
+197
| const handleCopyResponses = async (configs: EmptyObjectType) => { | ||
| console.log(configs); | ||
| await copy(JSON.stringify(configs)); | ||
| }; |
| Ten you can select embedding information in 'Indexer' page | ||
|
|
||
| Ten you can select embedding infomation in 'Indexer' page |
Comment on lines
+1
to
+14
| # EdgeCraft RAG Helm Chart | ||
|
|
||
| This doc intrudoces the Helm chart for deploying EdgeCraft RAG (ecrag) on a Kubernetes cluster. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - A running Kubernetes cluster. | ||
| - Helm installed. | ||
| - Required Docker images available in your registry or locally. | ||
|
|
||
| ## Configuration | ||
|
|
||
| Before installing, you should configure the `edgecraftrag/values.yaml` file according to your environment. | ||
|
|
Comment on lines
+4
to
+22
| import io | ||
| import os | ||
| from pathlib import Path | ||
| import asyncio | ||
| from typing import Any, Optional | ||
|
|
||
| import openvino_genai | ||
| import openvino as ov | ||
| import numpy as np | ||
| from edgecraftrag.base import BaseComponent, CompType, ModelType | ||
| from llama_index.embeddings.huggingface_openvino import OpenVINOEmbedding | ||
| from llama_index.embeddings.openai import OpenAIEmbedding | ||
| from llama_index.llms.openvino import OpenVINOLLM | ||
| from llama_index.postprocessor.openvino_rerank import OpenVINORerank | ||
| from edgecraftrag.components.ov_llamaindex_helper import OpenVINOGenAIEmbedding, OpenVINOGenAIReranking | ||
| from llama_index.llms.openvino_genai import OpenVINOGenAILLM | ||
| from pydantic import Field, model_serializer | ||
| from llama_index.core.base.llms.types import CompletionResponse, CompletionResponseAsyncGen, CompletionResponseGen | ||
| from threading import Event, Thread | ||
|
|
Comment on lines
51
to
+59
| return "Create Agent failed. Pipeline id not found." | ||
| if cfgs.type == AgentType.SIMPLE: | ||
| new_agent = SimpleRAGAgent(cfgs.idx, cfgs.name, cfgs.pipeline_idx, cfgs.configs) | ||
| new_agent.configs["max_retrievals"]=min(new_agent.configs["max_retrievals"], self.get_pipeline_by_name_or_id(cfgs.pipeline_idx).max_retrieve_topk) | ||
| elif cfgs.type == AgentType.DEEPSEARCH: | ||
| new_agent = DeepSearchAgent(cfgs.idx, cfgs.name, cfgs.pipeline_idx, cfgs.configs) | ||
| new_agent.configs["retrieve_top_k"]=min(new_agent.configs["retrieve_top_k"], self.get_pipeline_by_name_or_id(cfgs.pipeline_idx).max_retrieve_topk) | ||
| new_agent.configs["rerank_top_k"]=min(new_agent.configs["rerank_top_k"], self.get_pipeline_by_name_or_id(cfgs.pipeline_idx).max_retrieve_topk) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
update EC-RAG new feature
Issues
n/a
Type of change
List the type of change like below. Please delete options that are not relevant.