No description
Find a file
Nathan Bland 9667d19a19
All checks were successful
Backend CI / Lint, Type-check, Test (Backend) (push) Successful in 5m3s
```
feat: add archive_events table for tracking archive processing events and errors

Create archive_events table with fields for event level, code, message, details, and context. Add composite index on archiveId and createdAt for efficient event lookups. Include foreign key constraint to archives table with CASCADE delete behavior.
```
2025-12-10 11:30:42 -07:00
.forgejo/workflows ci: use dynamic ports for postgres and minio services in backend CI workflow 2025-09-03 23:18:04 -06:00
.kiro/specs/email-archiving-system docs(specs): add Email Archiving System requirements, design, and tasks specs 2025-08-20 01:47:36 -06:00
.vscode chore: editor settings and Windsurf Qwik app rules 2025-08-20 01:48:00 -06:00
.windsurf/rules chore: editor settings and Windsurf Qwik app rules 2025-08-20 01:48:00 -06:00
backend ``` 2025-12-10 11:30:42 -07:00
docs feat: implement letterparser-based ingestion pipeline with IMAP strategy and MinIO persistence 2025-08-26 10:35:27 -06:00
frontend test: update email viewer test to verify security warning in HTML content view 2025-12-09 13:49:10 -07:00
nginx ``` 2025-12-09 20:01:11 -07:00
scripts feat: implement SafeEmailViewer with URL rewriting and content security 2025-08-24 03:40:06 -06:00
tasks feat: implement letterparser-based ingestion pipeline with IMAP strategy and MinIO persistence 2025-08-26 10:35:27 -06:00
.gitignore chore(git): ignore backend/dist, frontend/.swc and test-results 2025-08-20 01:28:03 -06:00
docker-compose.prod.yml feat: configure persistent storage with local bind mounts and NFS for production volumes 2025-12-09 14:41:52 -07:00
docker-compose.yml feat: implement letterparser-based ingestion pipeline with IMAP strategy and MinIO persistence 2025-08-26 10:35:27 -06:00
Makefile feat: add production deployment workflow with Docker entrypoint scripts and theme refinements 2025-08-24 23:54:16 -06:00
PROJECT_PLAN.md docs: project README, plan, and task docs (PRD/sidebar/login/themes) 2025-08-20 01:47:49 -06:00
README.md feat: implement SafeEmailViewer with URL rewriting and content security 2025-08-24 03:40:06 -06:00

EmailVault - Email Archiving System

A modern, fast, and secure email archiving solution with powerful search capabilities and mobile-optimized interface.

Features

  • 🚀 Lightning Fast - Optimized performance with background processing
  • 🔒 Secure - Enterprise-grade encryption and security
  • 🔍 Powerful Search - Full-text search across all email content
  • 📱 Mobile Optimized - Beautiful responsive design
  • 📧 Universal Support - Works with any SMTP/IMAP email provider
  • 🔄 Background Processing - Non-blocking email archiving
  • 📁 Folder Preservation - Maintains original email folder structure

Tech Stack

Frontend

  • Next.js 15 with App Router
  • React 19
  • Tailwind CSS + shadcn/ui
  • TypeScript
  • React Query (TanStack Query)

Backend

  • Node.js with Fastify
  • Prisma ORM
  • Bull Queue (Redis)
  • TypeScript

Database & Storage

  • PostgreSQL (with full-text search)
  • Redis (queues & caching)
  • MinIO (S3-compatible attachment storage)

Quick Start

Prerequisites

  • Docker and Docker Compose
  • Node.js 20+ (for local development)

Setup

  1. Clone and navigate to the project:

    cd email-vault
    
  2. Start all services:

    docker-compose up -d
    
  3. Initialize the database:

    # Run database migrations
    docker-compose exec backend npm run db:push
    
  4. Access the application:

MinIO setup (automated + manual fallback)

  • Automated (default with Docker Compose):

    • Both docker-compose.yml and docker-compose.prod.yml include a minio-setup one-shot service that:
      • Creates a private bucket named emailvault on first start.
      • Ensures anonymous/public access is disabled for that bucket.
    • Credentials (dev): MINIO_ROOT_USER=emailvault, MINIO_ROOT_PASSWORD=emailvault_password.
    • Endpoint (dev): http://localhost:9000 (console at http://localhost:9001).
  • What the backend expects:

    • The attachment worker stores files in the emailvault bucket (see backend/src/workers/attachment-processing.worker.tsbucketName = 'emailvault').
    • It will also auto-create the bucket at runtime if missing, via ensureBucketExists(). The compose-based setup makes this explicit and deterministic.
  • Manual fallback (if you start MinIO without Compose setup):

    1. Install the MinIO client (mc) or use the minio/mc Docker image.
    2. Set an alias and create the bucket:
      mc alias set local http://localhost:9000 emailvault emailvault_password
      mc mb -p local/emailvault
      mc anonymous set none local/emailvault
      
    3. Alternatively, from the MinIO Console (http://localhost:9001):
      • Create a bucket named emailvault.
      • Keep it private (no anonymous policy).

Production notes

  • In docker-compose.prod.yml set:
    • MINIO_ROOT_USER, MINIO_ROOT_PASSWORD (for the minio service and minio-setup).
    • MINIO_ENDPOINT for the backend (defaults to minio:9000).
    • MINIO_ACCESS_KEY, MINIO_SECRET_KEY for the backend (should match the MinIO root user/pass unless you provision separate users/keys).
    • Optionally override MINIO_BUCKET used by minio-setup (defaults to emailvault).
    • The MinIO console is not exposed by default in prod compose; expose ports if you need console access.

Development

For local development without Docker:

  1. Start infrastructure services:

    docker-compose up postgres redis minio -d
    
  2. Install dependencies:

    # Backend
    cd backend && npm install
    
    # Frontend
    cd ../frontend && npm install
    
  3. Set up environment variables:

    # Backend
    cp backend/.env.example backend/.env
    
    # Frontend  
    cp frontend/.env.example frontend/.env
    
  4. Run development servers:

    # Backend (terminal 1)
    cd backend && npm run dev
    
    # Frontend (terminal 2)
    cd frontend && npm run dev
    

Environment Variables

Backend (.env)

DATABASE_URL=postgresql://emailvault:emailvault_password@localhost:5432/emailvault
REDIS_URL=redis://localhost:6379
MINIO_ENDPOINT=localhost:9000
MINIO_ACCESS_KEY=emailvault
MINIO_SECRET_KEY=emailvault_password
JWT_SECRET=your-super-secret-jwt-key-change-in-production

Frontend (.env.local)

NEXT_PUBLIC_API_URL=http://localhost:3001

Project Structure

email-vault/
├── docker-compose.yml          # Docker services configuration
├── frontend/                   # Next.js 15 frontend
│   ├── src/
│   │   ├── app/               # App Router pages
│   │   ├── components/        # React components
│   │   └── lib/               # Utilities
│   ├── package.json
│   └── Dockerfile
├── backend/                    # Fastify backend
│   ├── src/
│   │   ├── index.ts          # Main server file
│   │   ├── routes/           # API routes
│   │   ├── services/         # Business logic
│   │   └── types/            # TypeScript types
│   ├── prisma/
│   │   └── schema.prisma     # Database schema
│   ├── package.json
│   └── Dockerfile
├── nginx/
│   └── nginx.conf            # Reverse proxy config
└── docs/                     # Documentation

API Endpoints

Authentication

  • POST /auth/login - User login
  • POST /auth/register - User registration

Email Accounts

  • GET /email-accounts - List email accounts
  • POST /email-accounts - Add email account
  • PUT /email-accounts/:id - Update email account
  • DELETE /email-accounts/:id - Remove email account

Archives

  • GET /archives - List archives
  • POST /archives - Start new archive
  • GET /archives/:id - Get archive details
  • DELETE /archives/:id - Delete archive

Emails

  • GET /emails - Search emails
  • GET /emails/:id - Get email details
  • GET /emails/:id/attachments - Get email attachments

Testing

# Backend tests
cd backend && npm test

# Frontend tests  
cd frontend && npm test

# E2E tests
cd frontend && npm run test:e2e

Security & Safe Email Viewer

  • See docs/safe-email-viewer.md for details on the frontend Safe Email Viewer, URL rewriting, and backend proxy endpoints (/api/proxy/image, /api/proxy/link, and attachment proxy routes). It covers sanitization, SSRF protection, allowed content types/size limits, caching, and response security headers.

Production Deployment

  1. Update environment variables for production
  2. Build and deploy:
    docker-compose -f docker-compose.prod.yml up -d
    

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests
  5. Submit a pull request

License

MIT License - see LICENSE file for details