Backup & Recovery

Complete guide to the automated backup system for Lager Guru.

Overview

Lager Guru includes a comprehensive, automated backup system with encryption, retention policies, and multi-tenant support. Backups are stored securely in Supabase Storage (default) or AWS S3, with AES-256 encryption at rest.

Features

Automated Weekly Backups: Configurable cron-based scheduling
Multi-Tenant Support: Per-tenant or global backup settings
Encryption: AES-256-GCM encryption for all backups
Retention Management: Automatic cleanup of old backups
Storage Options: Supabase Storage (default) or AWS S3
Notifications: Webhook and email alerts on success/failure
Super-Admin UI: Complete backup management interface
Manual Triggers: On-demand backup creation
Restore Tools: Safe restore procedures with verification

Backup Types

Database Backups (`db`)

Full PostgreSQL database export
Uses pg_dump when available, falls back to CSV export
Includes all tenant data (filtered by tenant_id when specified)

Storage Backups (`storage`)

Application files and uploaded assets
Configuration files
Customizations

Full Backups (`full`)

Combined database + storage backup
Recommended for complete system recovery

Backup Schedule

Default Configuration

Frequency: Weekly (Monday 03:00 UTC)
Retention: 365 days (1 year)
Max Versions: 52 backups
Encryption: Enabled by default

Customization

Backup schedules can be configured per tenant or globally via:

Super-Admin UI (/admin → Backups)
Edge Function API
Direct database updates (requires service role)

Storage Providers

Supabase Storage (Default)

Configuration:

Bucket: backups (default, configurable)
Path structure: backups/<tenant_id>/<YYYY>/<MM>/<DD>/<type>_<timestamp>.tar.gz.enc
Access: Service role key required

Setup:

Create bucket in Supabase Dashboard → Storage
Set bucket to private
Configure in backup settings

AWS S3 (Optional)

Configuration:

Requires S3 credentials (access key, secret key, region)
Supports SSE encryption
Path structure: Same as Supabase Storage

Setup:

Create S3 bucket
Configure IAM permissions

Set environment variables:

bash

S3_ACCESS_KEY_ID=your-access-key
S3_SECRET_ACCESS_KEY=your-secret-key
S3_REGION=us-east-1
S3_BUCKET=your-bucket-name

Update backup settings to use s3 provider

Encryption

All backups are encrypted using AES-256-GCM:

Key Management: BACKUP_ENCRYPTION_KEY environment variable
Key Storage: Never commit keys to repository
KMS Integration: Optional encryption_key_id field for cloud KMS references
IV + Tag: Stored with encrypted file for secure decryption

Security Best Practices:

Use strong, randomly generated encryption keys (32+ bytes)
Store keys in secure secret management (AWS Secrets Manager, HashiCorp Vault, etc.)
Rotate encryption keys periodically
Never expose keys in logs or UI

Retention Policy

Automatic Cleanup

The retention rotation script (backup:rotate) automatically:

Identifies backups older than retention_days
Moves to backups/archived/ for 30 days (soft delete)
Permanently deletes after archive period
Removes backup_history entries

Manual Cleanup

bash

# Dry run (preview what would be deleted)
npm run backup:rotate -- --dry-run

# Run cleanup for specific tenant
npm run backup:rotate -- --tenant-id=<uuid>

# Run cleanup for all tenants
npm run backup:rotate

Manual Backup Operations

Trigger Backup via UI

Navigate to /admin → Backups
Select tenant (or "Global")
Click "Trigger Backup"
Monitor progress in Backup History table

Trigger Backup via CLI

bash

# Full backup for all tenants
npm run backup:trigger

# Database backup for specific tenant
npm run backup:trigger -- --tenant-id=<uuid> --type=db

# Storage backup
npm run backup:trigger -- --type=storage

Run Scheduled Backups

bash

# Run all enabled backups
npm run backup:run

# Run for specific tenant
npm run backup:run -- --tenant-id=<uuid>

Restore Procedures

Prerequisites

Access to backup file (download from storage)
BACKUP_ENCRYPTION_KEY environment variable
Database connection (DATABASE_URL)
Service role key for storage access

Safety Checklist

Before restoring:

[ ] Stop application workers/background jobs
[ ] Put application in read-only mode (if possible)
[ ] Verify backup file integrity (checksum)
[ ] Test restore on staging environment first
[ ] Document current system state
[ ] Have rollback plan ready

Restore Steps

Download Backup

bash

# Via Supabase Storage API or UI
# Backup path: backups/<tenant_id>/<YYYY>/<MM>/<DD>/<filename>.tar.gz.enc

Restore Database

bash

# Restore with confirmation
npm run backup:restore <backup-id> --confirm-restore

# For production (requires CONFIRM_RESTORE_IN_PROD=true)
CONFIRM_RESTORE_IN_PROD=true npm run backup:restore <backup-id> --confirm-restore

Verify Restore
- Check database connectivity
- Verify data integrity
- Test critical application functions
- Monitor logs for errors

Restart Application

bash

# Restart services
docker compose restart
# or
npm run dev

Test Restore Procedure

Monthly Test Restore (Recommended):

Create test tenant/staging environment
Run restore from latest backup
Verify all data present and correct
Test application functionality
Document results

Notifications

Webhook Notifications

Configure webhook URL in backup settings to receive JSON payloads:

json

{
  "event": "backup_completed",
  "tenant_id": "uuid-or-null",
  "backup_id": "uuid",
  "status": "success|failed",
  "message": "Backup completed successfully",
  "timestamp": "2024-01-01T00:00:00Z"
}

Email Notifications

Configure email address in backup settings. Email includes:

Backup status (success/failed)
Tenant information
Backup ID and timestamp
Error details (if failed)

Email Provider Setup:

Integrate with SendGrid, AWS SES, or SMTP server
Update sendEmailNotification in lib/backupHelpers.ts

Monitoring & Health Checks

Health Endpoint

bash

GET /functions/v1/backup-trigger/health

Returns:

Last successful backup per tenant
Status (healthy/warning) based on backup age
Backup frequency metrics

Monitoring Integration

Prometheus: Export backup metrics
Datadog: Custom dashboard for backup health
Uptime Checks: Monitor health endpoint
Alerting: Set alerts for failed backups or missing backups

Troubleshooting

Backup Fails

Common Issues:

Storage quota exceeded: Check bucket size limits
Encryption key missing: Verify BACKUP_ENCRYPTION_KEY set
Database connection failed: Check DATABASE_URL or Supabase connection
Permission denied: Verify service role key has storage access

Debug Steps:

bash

# Check backup history
# Via UI: /admin → Backups → History
# Via API: GET /functions/v1/backup-trigger/history

# Check logs
# Script logs: Console output from backup-runner.ts
# Edge Function logs: Supabase Dashboard → Edge Functions → Logs

Restore Fails

Common Issues:

Checksum mismatch: Backup file corrupted, try different backup
Decryption failed: Verify encryption key matches backup creation
Database locked: Stop application before restore
Insufficient permissions: Use service role key

Retention Not Working

Check:

Retention script running? (npm run backup:rotate)
Retention days configured correctly?
Backup history entries exist?
Storage permissions for deletion?

Best Practices

Regular Testing: Test restore monthly on staging
Multiple Copies: Store backups in multiple locations
Monitor Health: Set up alerts for backup failures
Document Procedures: Keep restore procedures documented
Key Management: Use cloud KMS for encryption keys
Access Control: Limit backup access to super-admins only
Audit Logging: Review backup operations regularly
Version Control: Keep multiple backup versions (max_versions)

API Reference

Backup operations are available via Edge Function API:

POST /functions/v1/backup-trigger/trigger - Trigger manual backup
GET /functions/v1/backup-trigger/history - Get backup history
GET /functions/v1/backup-trigger/settings - Get backup settings
POST /functions/v1/backup-trigger/settings - Update backup settings
POST /functions/v1/backup-trigger/test-upload - Test storage upload
GET /functions/v1/backup-trigger/health - Health check

See API Integrations for general API documentation.

Backup Setup Guide - Initial configuration
Deployment Guide - Production deployment
Security Overview - Security best practices
Multi-Tenant Architecture - Tenant isolation

Backup & Recovery ​

Overview ​

Features ​

Backup Types ​

Database Backups (db) ​

Storage Backups (storage) ​

Full Backups (full) ​

Backup Schedule ​

Default Configuration ​

Customization ​

Storage Providers ​

Supabase Storage (Default) ​

AWS S3 (Optional) ​

Encryption ​

Retention Policy ​

Automatic Cleanup ​

Manual Cleanup ​

Manual Backup Operations ​

Trigger Backup via UI ​

Trigger Backup via CLI ​

Run Scheduled Backups ​

Restore Procedures ​

Prerequisites ​

Safety Checklist ​

Restore Steps ​

Test Restore Procedure ​

Notifications ​

Webhook Notifications ​

Email Notifications ​

Monitoring & Health Checks ​

Health Endpoint ​

Monitoring Integration ​

Troubleshooting ​

Backup Fails ​

Restore Fails ​

Retention Not Working ​

Best Practices ​

API Reference ​

Related Documentation ​

Backup & Recovery

Overview

Features

Backup Types

Database Backups (`db`)

Storage Backups (`storage`)

Full Backups (`full`)

Backup Schedule

Default Configuration

Customization

Storage Providers

Supabase Storage (Default)

AWS S3 (Optional)

Encryption

Retention Policy

Automatic Cleanup

Manual Cleanup

Manual Backup Operations

Trigger Backup via UI

Trigger Backup via CLI

Run Scheduled Backups

Restore Procedures

Prerequisites

Safety Checklist

Restore Steps

Test Restore Procedure

Notifications

Webhook Notifications

Email Notifications

Monitoring & Health Checks

Health Endpoint

Monitoring Integration

Troubleshooting

Backup Fails

Restore Fails

Retention Not Working

Best Practices

API Reference

Related Documentation