Skip to content

🛡️ Backup and Recovery

This section provides guidance on how to protect the critical data managed by the uxopian-ai framework. A proper backup strategy is essential to prevent data loss and ensure business continuity.

💾 What to Back Up

There are two primary sources of data you need to protect:

  • 🛠 Configuration Files: All the .yml files that define the service's behavior, including connections, provider settings, and default parameters.
  • 📦 OpenSearch Data: The OpenSearch instance stores all the dynamic data generated by user interactions, which includes conversations, messages, and the centrally managed Prompts and Goals.

🧠 Built-in YAML Backup for Prompts and Goals

The framework includes an automatic backup mechanism for Prompts and Goals. Whenever you create, update, or delete a prompt or goal via the API, the service automatically writes the current state to YAML files in the configured backup directory.

Note: This mechanism is multi-tenant aware, meaning it generates separate files for each tenant to ensure data isolation.

⚙️ Configuration

You only need to configure the directory path where backups should be stored.

For Prompts (prompts.yml):

prompts:
  backup:
    path: ${PROMPTS_BACKUP_PATH:./prompts/}

For Goals (goals.yml):

goals:
  backup:
    path: ${GOALS_BACKUP_PATH:./goals/}

📂 File Naming Convention

The system automatically names backup files based on the tenantId.

Format:

  • Prompts: prompts-{tenantId}.yaml
  • Goals: goals-{tenantId}.yaml

Content Structure: A backup file contains the raw definition of a tenant's configuration.

tenantId: Tenant-development
prompts:
  - id: basePrompt
    role: SYSTEM
    content: "..."
    # ...

🔄 Restoring from YAML Backups

Since backup files contain specific tenant data, restoration involves integrating this data back into your main configuration files or pushing it via the API.

✅ Method 1: Restoration via Configuration (Bootstrap)

This method allows you to restore data by injecting the backup content directly into the main configuration files (prompts.yml or goals.yml) before starting the service.

Steps:

  1. Locate Backup: Open the specific backup file you wish to restore (e.g., prompts-Tenant-development.yaml).
  2. Copy Content: Copy the entire content of the file (including tenantId and the list of prompts/goals).
  3. Edit Main Config: Open your main configuration file (config/prompts.yml or config/goals.yml).
  4. Inject and Configure:
    • Locate the tenants: [] list.
    • Paste the backup content as a new item in this list.
    • Crucial: Add the mergeStrategy field to define how this data should be applied.

Example for Prompts (config/prompts.yml):

prompts:
  globals: [...]

  # Paste your backup here under 'tenants'
  tenants:
    - tenantId: Tenant-development # <--- From Backup
      mergeStrategy: OVERRIDE # <--- ADD THIS MANUALLY
      prompts: # <--- From Backup
        - id: basePrompt
          role: SYSTEM
          content: "..."
  1. Restart: Restart uxopian-ai. The service will process the tenants list and apply the configuration to OpenSearch according to the mergeStrategy.

🧪 Method 2: Manual Restore via API

You can also manually restore a specific tenant's configuration using the REST API.

  1. Open the backup file (e.g., goals-Tenant-development.yaml).
    • Goals: Copy each entry from the goalGroups list.
    • Prompts: Copy each entry from the prompts list.
  2. Send a POST request for each item to the appropriate endpoint.
    • Prompts: POST /api/v1/admin/prompts
    • Goals: POST /api/v1/admin/goals
  3. Headers: You must provide the correct authentication headers to ensure the data is restored to the correct tenant.
    • X-User-TenantId: Tenant-development

🧩 Merge Strategies (Configuration Mode)

When using Method 1, the mergeStrategy field controls how the restored data interacts with existing data in OpenSearch.

🔁 OVERRIDE (Disaster Recovery)

  • Behavior: Deletes ALL existing prompts/goals for this specific tenant in OpenSearch and replaces them with the content of the YAML.
  • Use Case: Restoring a corrupted environment or reverting to a known clean state.

➕ MERGE (Update/Patch)

  • Behavior:
  • Updates items with matching IDs.
  • Creates items that do not exist.
  • Does not delete items that are in OpenSearch but not in the YAML.
  • Use Case: Applying configuration updates without losing data created by users since the backup.

🆕 CREATE_IF_MISSING (Safe Init)

  • Behavior: Only creates items that do not currently exist. Existing items are ignored.
  • Use Case: Deploying default prompts to a new environment without risking overwrite of existing data.

🗂 Backup Strategy Summary

Entity Storage Backup Mechanism Recommended Restore Method
Configuration Filesystem Copy files manually Restore files to ./config/
Conversations OpenSearch OpenSearch Snapshots Elastic/OpenSearch Snapshot Restore
Prompts/Goals OpenSearch Auto YAML Backup Method 1 (Config Merge) for full restore

✅ Key Recommendations

  • 📆 Automate Commits: Since the system automatically updates the YAML files on disk, set up a cron job to commit and push these changes to a Git repository automatically.
  • 📝 Verify Merge Strategy: When restoring via config, always double-check the mergeStrategy. Using OVERRIDE by mistake will wipe out any new prompts created since the backup.