Using APIMart in AnythingLLM

​

Prerequisites

1. AnythingLLM installed
   Download and install AnythingLLM from official website or visit GitHub
2. APIMart API Key obtained
   Log in to APIMart Console to get your API key (starts with sk-)

​

Step 1: Launch AnythingLLM and Access Settings

​

1.1 Start the Application

1. Launch AnythingLLM desktop application or access the web version
2. Welcome screen will appear on first launch
3. Click the Open settings button in the bottom left corner

​

1.2 Navigate to LLM Configuration

1. Find LLM Preference in the left menu
2. Click to enter LLM configuration page

​

Step 2: Configure APIMart API

​

2.1 Select LLM Provider

1. Find the LLM Provider dropdown menu
2. Select Generic OpenAI

​

2.2 Configure API Information

​

2.3 Adjust Model Parameters (Optional)

​

2.4 Save Configuration

1. Click the Save button at the bottom of the page
2. System will automatically test the connection
3. Success message will appear if configuration is correct

​

Step 3: Configure Embedding Model (Optional)

​

3.1 Navigate to Embedding Settings

1. Find Embedding Preference in the left menu
2. Click to enter embedding model configuration page

​

3.2 Configure Embedding Model

​

Step 4: Create Workspace and Upload Documents

​

4.1 Create Workspace

1. Return to main interface
2. Click + New Workspace
3. Enter workspace name (e.g., “Technical Documentation Assistant”, “Customer Service Knowledge Base”)
4. Click Create

​

4.2 Upload Documents

• 📄 Text Documents - .txt, .md, .pdf, .docx
• 💻 Code Files - .py, .js, .java, .cpp, etc.
• 🌐 Web Pages - Via URL scraping
• 📊 Data Files - .csv, .json, .xml

1. On the workspace page, click Upload Documents
2. Select files or drag and drop files into the upload area
3. Wait for document processing to complete
4. Documents will be automatically vectorized

​

4.3 Manage Documents

1. View all uploaded documents
2. Delete unnecessary documents
3. View document chunking details
4. Edit document metadata

​

Step 5: Start Conversations

​

5.1 Basic Conversations

1. In the workspace, find the conversation input box
2. Enter your question or request
3. AI will generate responses based on your uploaded documents and APIMart models

​

5.2 Using Knowledge Base Features

1. Analyze your question
2. Retrieve relevant content from uploaded documents
3. Generate accurate answers combining retrieved content and AI model

​

5.3 Switch Workspaces

1. Click workspace name in top left corner
2. Select other workspaces or create new ones
3. Each workspace has independent documents and conversation history

​

Advanced Features

​

1. Agent Mode

• 🔍 Search Web - Get real-time information
• 🧮 Perform Calculations - Handle math and data analysis
• 📊 Generate Charts - Visualize data
• 🔗 Call APIs - Interact with external services

1. Find Agent Configuration in workspace settings
2. Select tools and features to enable
3. Save configuration

​

2. Conversation History Management

• Export Conversations - Export conversations as text or JSON format
• Search History - Quickly find historical conversations
• Delete Records - Clean up unnecessary conversation history

​

3. Custom System Prompts

1. Find System Prompt
2. Customize AI’s role and behavior
3. Example:

You are a professional technical support engineer, skilled at answering technical questions about products.
When answering, please:
1. Maintain professionalism and courtesy
2. Provide detailed step-by-step instructions
3. If uncertain, recommend contacting technical support

​

4. Multi-user Management (Docker Deployment Only)

• Create multiple user accounts
• Set different permission levels
• Manage workspace access permissions

​

5. API Access

• Programmatic workspace access
• Upload and manage documents
• Send conversation requests
• Integrate into your applications

​

FAQ

​

Q1: Cannot connect to APIMart service?

1. Check Base URL:
   • Ensure it’s https://api.apimart.ai/v1 (includes /v1)
   • Don’t add extra paths or omit /v1
2. Verify API Key:
   • Confirm API Key starts with sk-
   • Check if key is valid in APIMart Console
3. Check Network Connection:
   • Ensure access to https://api.apimart.ai
   • Check firewall or proxy settings

​

Q2: Documents not retrieving properly after upload?

1. Check Embedding Model Configuration:
   • Confirm embedding model is correctly configured
   • Test embedding model connection
2. Re-process Documents:
   • Delete and re-upload documents
   • Check if document format is supported
3. Adjust Retrieval Parameters:
   • Adjust similarity threshold in workspace settings
   • Increase number of returned document chunks

​

Q3: Slow conversation response?

1. Switch to Faster Models:
   • Use gpt-4o-mini instead of gpt-4o
   • Use gemini-2.0-flash-exp for faster response
2. Optimize Document Quantity:
   • Reduce number of documents in workspace
   • Remove unnecessary large files
3. Adjust Max Tokens:
   • Reduce maximum output length
   • Use more concise prompts

​

Q4: How to view API usage and costs?

• 📊 API call statistics
• 💰 Cost details
• 📈 Usage trend charts
• 🔍 Detailed request logs

​

Q5: What deployment options does AnythingLLM support?

• 🖥️ Desktop Application - Windows, macOS, Linux
• 🐳 Docker - Self-hosted deployment
• ☁️ Cloud Version - AnythingLLM Cloud (coming soon)

​

Use Case Examples

​

1. Enterprise Knowledge Base

• Model: gpt-4o-mini (cost-effective)
• Documents: Internal company documents, manuals, FAQs
• Function: Quick information lookup for employees

• New employee onboarding
• Quick company policy lookup
• Technical documentation retrieval

​

2. Technical Documentation Assistant

• Model: claude-sonnet-4-5-20250929 (excellent for code)
• Documents: API docs, technical specifications, codebase
• Function: Assist developers in finding technical information

• API usage documentation queries
• Code example retrieval
• Technical question answering

​

3. Customer Service Knowledge Base

• Model: gpt-4o (high-quality conversation)
• Documents: Product manuals, FAQs, solutions
• Function: Quick customer question response

• Automatic FAQ answering
• Product usage guidance
• Troubleshooting suggestions

​

4. Research and Learning Assistant

• Model: gpt-5 (powerful understanding)
• Documents: Research papers, textbooks, notes
• Function: Assist in learning and research

• Paper summarization and analysis
• Knowledge point explanation
• Learning path planning

​

Features

• 📚 Private Knowledge Base - Build secure private knowledge base locally
• 🔒 Data Privacy - All data stored locally, protecting privacy
• 🤖 Multi-model Support - Flexibly switch between different AI models
• 📄 Multi-format Support - Support various document formats
• 🎯 Precise Retrieval - Vector-based intelligent document retrieval
• 💬 Contextual Conversation - Maintain context in long conversations
• 🔧 Highly Customizable - Custom prompts, parameters, etc.
• 🌐 Cross-platform - Support Windows, macOS, Linux

​

Data Security and Privacy

​

Local Data Storage

• 📁 Local File System - Documents stored locally
• 🗄️ Local Vector Database - Vector indexes stored locally
• 💾 Conversation History - Conversation records stored locally

​

API Call Security

• 🔐 Encrypted Transmission - All API calls use HTTPS encryption
• 🔑 Key Protection - API Key securely stored
• 🚫 No Data Retention - APIMart does not store your conversation content

​

Best Practices

​

1. Document Management

• Regular Updates - Keep document content current
• Structured Organization - Use clear folder structure
• Naming Conventions - Use meaningful file names
• Delete Outdated - Regularly clean up outdated documents

​

2. Prompt Optimization

You are an assistant

You are a professional technical support assistant specializing in helping users resolve product-related technical issues.

When answering, please follow these principles:
1. Answer based on provided documentation; if information isn't available, clearly state so
2. Provide detailed step-by-step instructions with examples when necessary
3. Use clear, understandable language
4. For complex issues, recommend contacting the technical support team

​

3. Performance Optimization

• Control Document Size - Avoid uploading overly large individual files
• Reasonable Chunking - Use default document chunking settings
• Choose Appropriate Model - Select model based on task complexity
• Monitor Usage - Regularly check API usage

​

4. Workspace Planning

• Divide by Project - Create independent workspaces for different projects
• Permission Management - Set workspace permissions appropriately (Docker version)
• Backup Data - Regularly backup important workspaces

​

Support & Help

• 📚 APIMart Documentation
• 📚 AnythingLLM Official Documentation
• 📚 AnythingLLM GitHub
• 💬 Discord Community
• 🐦 Twitter @APIMart_
• 📧 Technical Support: zhihong@apimart.ai