award/README.md

# Ham Radio Award Portal

A web application for amateur radio operators to track QSOs (contacts) and award progress using Logbook of the World (LoTW) data.

## Features

- **User Authentication**: Register and login with callsign, email, and password
- **LoTW Integration**: Sync QSOs from ARRL's Logbook of the World
  - Background job queue for non-blocking sync operations
  - Incremental sync using last confirmation date
  - Wavelog-compatible download logic with proper validation
  - One sync job per user enforcement
- **QSO Log**: View and manage confirmed QSOs
  - Pagination support for large QSO collections
  - Filter by band, mode, and confirmation status
  - Statistics dashboard (total QSOs, confirmed, DXCC entities, bands)
  - Delete all QSOs with confirmation
- **Settings**: Configure LoTW credentials securely

## Tech Stack

### Backend
- **Runtime**: Bun
- **Framework**: Elysia.js
- **Database**: SQLite with Drizzle ORM
- **Authentication**: JWT tokens
- **Logging**: Pino with structured logging and timestamps

### Frontend
- **Framework**: SvelteKit
- **Language**: JavaScript
- **Styling**: Custom CSS

## Project Structure

```
award/
├── src/
│   ├── backend/
│   │   ├── config/
│   │   │   ├── database.js     # Database connection
│   │   │   ├── jwt.js          # JWT configuration
│   │   │   └── logger.js       # Pino logging configuration
│   │   ├── db/
│   │   │   └── schema/
│   │   │       └── index.js    # Database schema (users, qsos, sync_jobs)
│   │   ├── services/
│   │   │   ├── auth.service.js        # User authentication
│   │   │   ├── lotw.service.js        # LoTW sync & QSO management
│   │   │   └── job-queue.service.js   # Background job queue
│   │   └── index.js            # API routes and server
│   └── frontend/
│       ├── src/
│       │   ├── lib/
│       │   │   ├── api.js      # API client
│       │   │   └── stores.js   # Svelte stores (auth)
│       │   └── routes/
│       │       ├── +layout.svelte       # Navigation bar & layout
│       │       ├── +page.svelte         # Dashboard
│       │       ├── auth/
│       │       │   ├── login/+page.svelte   # Login page
│       │       │   └── register/+page.svelte # Registration page
│       │       ├── qsos/+page.svelte    # QSO log with pagination
│       │       └── settings/+page.svelte # Settings & LoTW credentials
│       └── package.json
├── award.db                   # SQLite database (auto-created)
├── drizzle.config.js          # Drizzle ORM configuration
├── package.json
└── README.md
```

## Setup

### Prerequisites
- [Bun](https://bun.sh) v1.3.6 or later

### Installation

1. Clone the repository:
```bash
git clone <repository-url>
cd award
```

2. Install dependencies:
```bash
bun install
```

3. Set up environment variables:
Create a `.env` file in the project root (copy from `.env.example`):
```bash
cp .env.example .env
```

Edit `.env` with your configuration:
```env
# Application URL (for production deployment)
VITE_APP_URL=https://awards.dj7nt.de

# API Base URL (leave empty for same-domain deployment)
VITE_API_BASE_URL=

# JWT Secret (generate with: openssl rand -base64 32)
JWT_SECRET=your-generated-secret-here

# Environment
NODE_ENV=production
```

**For development**: You can leave `.env` empty or use defaults.

4. Initialize the database:
```bash
bun run db:push
```

This creates the SQLite database with required tables (users, qsos, sync_jobs).

## Running the Application

Start both backend and frontend with a single command:

```bash
bun run dev
```

Or start them individually:
```bash
# Backend only (port 3001, proxied)
bun run dev:backend

# Frontend only (port 5173)
bun run dev:frontend
```

The application will be available at:
- **Frontend & API**: http://localhost:5173

**Note**: During development, both servers run (frontend on 5173, backend on 3001), but API requests are automatically proxied through the frontend. You only need to access port 5173.

## API Endpoints

### Authentication
- `POST /api/auth/register` - Register new user
- `POST /api/auth/login` - Login user
- `GET /api/auth/me` - Get current user profile
- `PUT /api/auth/lotw-credentials` - Update LoTW credentials

### LoTW Sync
- `POST /api/lotw/sync` - Queue a LoTW sync job (returns job ID)

### Jobs
- `GET /api/jobs/:jobId` - Get job status
- `GET /api/jobs/active` - Get user's active job
- `GET /api/jobs` - Get recent jobs (query: `?limit=10`)

### QSOs
- `GET /api/qsos` - Get user's QSOs with pagination
  - Query parameters: `?page=1&limit=100&band=20m&mode=CW&confirmed=true`
- `GET /api/qsos/stats` - Get QSO statistics
- `DELETE /api/qsos/all` - Delete all QSOs (requires confirmation)

### Health
- `GET /api/health` - Health check endpoint

## Database Schema

### Users Table
```sql
CREATE TABLE users (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  email TEXT UNIQUE NOT NULL,
  password TEXT NOT NULL,
  callsign TEXT NOT NULL,
  lotwUsername TEXT,
  lotwPassword TEXT,
  createdAt TEXT NOT NULL
);
```

### QSOs Table
```sql
CREATE TABLE qsos (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  userId INTEGER NOT NULL,
  callsign TEXT NOT NULL,
  qsoDate TEXT NOT NULL,
  timeOn TEXT NOT NULL,
  band TEXT,
  mode TEXT,
  entity TEXT,
  grid TEXT,
  lotwQslRstatus TEXT,
  lotwQslRdate TEXT,
  FOREIGN KEY (userId) REFERENCES users(id)
);
```

### Sync Jobs Table
```sql
CREATE TABLE sync_jobs (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  userId INTEGER NOT NULL,
  status TEXT NOT NULL, -- pending, running, completed, failed
  type TEXT NOT NULL,   -- lotw_sync
  startedAt INTEGER,
  completedAt INTEGER,
  result TEXT,          -- JSON
  error TEXT,
  createdAt INTEGER NOT NULL,
  FOREIGN KEY (userId) REFERENCES users(id)
);
```

## Architecture

### Development Mode
- **SvelteKit Dev Server** (port 5173): Serves frontend and proxies API requests
- **Elysia Backend** (port 3001): Handles API requests (hidden from user)
- **Proxy Configuration**: All `/api/*` requests are forwarded from SvelteKit to Elysia

This gives you:
- ✅ Single port to access (5173)
- ✅ Hot Module Replacement (HMR) for frontend
- ✅ No CORS issues
- ✅ Simple production deployment

### Production Mode
In production, the application serves everything from a single port:
- **Backend** on port 3001 serves both API and static frontend files
- Frontend static files are served from `src/frontend/build/`
- SPA routing is handled by backend fallback
- **Single port** simplifies HAProxy configuration

## Production Deployment

This guide covers deployment using **PM2** for process management and **HAProxy** as a reverse proxy/load balancer.

### Architecture Overview

```
Internet
    │
    ▼
┌─────────────────┐
│   HAProxy       │  Port 443 (HTTPS)
│   (Port 80/443) │  Port 80 (HTTP → HTTPS redirect)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   Backend       │  Port 3001
│   Managed by    │  ├─ API Routes (/api/*)
│   PM2           │  ├─ Static Files (/*)
│                 │  └─ SPA Fallback
└────────┬────────┘
         │
         ├─────────────────┬──────────────────┐
         │                 │                  │
         ▼                 ▼                  ▼
    ┌─────────┐      ┌──────────┐     ┌─────────────┐
    │ SQLite  │      │ Frontend │     │  ARRL LoTW  │
    │   DB    │      │  Build   │     │ External API│
    └─────────┘      └──────────┘     └─────────────┘
```

### Prerequisites

- Server with SSH access
- Bun runtime installed
- PM2 installed globally: `bun install -g pm2` or `npm install -g pm2`
- HAProxy installed
- Domain with DNS pointing to server

### Step 1: Build the Application

```bash
# Clone repository on server
git clone <repository-url>
cd award

# Install dependencies
bun install

# Build frontend (generates static files in src/frontend/build/)
bun run build
```

### Step 2: Configure Environment Variables

Create `.env` in the project root:

```bash
# Application URL
VITE_APP_URL=https://awards.dj7nt.de

# API Base URL (empty for same-domain)
VITE_API_BASE_URL=

# JWT Secret (generate with: openssl rand -base64 32)
JWT_SECRET=your-generated-secret-here

# Environment
NODE_ENV=production

# Database path (absolute path recommended)
DATABASE_PATH=/path/to/award/award.db
```

**Security**: Ensure `.env` has restricted permissions:
```bash
chmod 600 .env
```

### Step 3: Initialize Database

```bash
# Push database schema
bun run db:push

# Verify database was created
ls -la award.db
```

### Step 4: Create PM2 Ecosystem Configuration

Create `ecosystem.config.js` in the project root:

```javascript
module.exports = {
  apps: [
    {
      name: 'award-backend',
      script: 'src/backend/index.js',
      interpreter: 'bun',
      cwd: '/path/to/award',
      env: {
        NODE_ENV: 'production',
        PORT: 3001
      },
      instances: 1,
      exec_mode: 'fork',
      autorestart: true,
      watch: false,
      max_memory_restart: '500M',
      error_file: './logs/backend-error.log',
      out_file: './logs/backend-out.log',
      log_date_format: 'YYYY-MM-DD HH:mm:ss Z'
    },
    {
      name: 'award-frontend',
      script: 'bun',
      args: 'run preview',
      cwd: '/path/to/award/src/frontend',
      env: {
        NODE_ENV: 'production',
        PORT: 5173
      },
      instances: 1,
      exec_mode: 'fork',
      autorestart: true,
      watch: false,
      max_memory_restart: '300M',
      error_file: './logs/frontend-error.log',
      out_file: './logs/frontend-out.log',
      log_date_format: 'YYYY-MM-DD HH:mm:ss Z'
    }
  ]
};
```

**Create logs directory:**
```bash
mkdir -p logs
```

### Step 5: Start Applications with PM2

```bash
# Start all applications
pm2 start ecosystem.config.js

# Save PM2 process list
pm2 save

# Setup PM2 to start on system reboot
pm2 startup
# Follow the instructions output by the command above
```

**Useful PM2 Commands:**

```bash
# View status
pm2 status

# View logs
pm2 logs

# Restart all apps
pm2 restart all

# Restart specific app
pm2 restart award-backend

# Stop all apps
pm2 stop all

# Monitor resources
pm2 monit
```

### Step 6: Configure HAProxy

Edit `/etc/haproxy/haproxy.cfg`:

```haproxy
global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

    # Default SSL material locations
    ca-base /etc/ssl/certs
    crt-base /etc/ssl/private

    # Default ciphers to use
    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
    ssl-default-bind-options no-sslv3

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

# Statistics page (optional - secure with auth)
listen stats
    bind *:8404
    mode http
    stats enable
    stats uri /stats
    stats refresh 30s
    stats realm HAProxy\ Statistics
    stats auth admin:your-secure-password

# Frontend redirect HTTP to HTTPS
frontend http-in
    bind *:80
    http-request redirect scheme https

# Frontend HTTPS
frontend https-in
    bind *:443 ssl crt /etc/ssl/private/awards.dj7nt.de.pem
    default_backend award-backend

# Backend configuration
backend award-backend
    # Health check
    option httpchk GET /api/health

    # Single server serving both frontend and API
    server award-backend 127.0.0.1:3001 check
```

**SSL Certificate Setup:**

Using Let's Encrypt with Certbot:

```bash
# Install certbot
apt install certbot

# Generate certificate
certbot certonly --standalone -d awards.dj7nt.de

# Combine certificate and key for HAProxy
cat /etc/letsencrypt/live/awards.dj7nt.de/fullchain.pem > /etc/ssl/private/awards.dj7nt.de.pem
cat /etc/letsencrypt/live/awards.dj7nt.de/privkey.pem >> /etc/ssl/private/awards.dj7nt.de.pem

# Set proper permissions
chmod 600 /etc/ssl/private/awards.dj7nt.de.pem
```

### Step 7: Start HAProxy

```bash
# Test configuration
haproxy -c -f /etc/haproxy/haproxy.cfg

# Restart HAProxy
systemctl restart haproxy

# Enable HAProxy on boot
systemctl enable haproxy

# Check status
systemctl status haproxy
```

### Step 8: Verify Deployment

```bash
# Check PM2 processes
pm2 status

# Check HAProxy stats (if enabled)
curl http://localhost:8404/stats

# Test health endpoint
curl https://awards.dj7nt.de/api/health

# Check logs
pm2 logs
tail -f /var/log/haproxy.log
```

### Updating the Application

```bash
# Pull latest changes
git pull

# Install updated dependencies
bun install

# Rebuild frontend (if UI changed)
bun run build

# Restart PM2
pm2 restart award-backend
```

### Database Backups

Set up automated backups:

```bash
# Create backup script
cat > /usr/local/bin/backup-award.sh << 'EOF'
#!/bin/bash
BACKUP_DIR="/backups/award"
DATE=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR

# Backup database
cp /path/to/award/award.db $BACKUP_DIR/award_$DATE.db

# Keep last 30 days
find $BACKUP_DIR -name "award_*.db" -mtime +30 -delete
EOF

chmod +x /usr/local/bin/backup-award.sh

# Add to crontab (daily at 2 AM)
crontab -e
# Add line: 0 2 * * * /usr/local/bin/backup-award.sh
```

### Monitoring

**PM2 Monitoring:**
```bash
# Real-time monitoring
pm2 monit

# View logs
pm2 logs --lines 100
```

**HAProxy Monitoring:**
- Access stats page: `http://your-server:8404/stats`
- Check logs: `tail -f /var/log/haproxy.log`

**Log Files Locations:**
- PM2 logs: `./logs/backend-error.log`, `./logs/frontend-error.log`
- HAProxy logs: `/var/log/haproxy.log`
- System logs: `journalctl -u haproxy -f`

### Security Checklist

- [ ] HTTPS enabled with valid SSL certificate
- [ ] Firewall configured (ufw/firewalld)
- [ ] JWT_SECRET is strong and randomly generated
- [ ] .env file has proper permissions (600)
- [ ] Database backups automated
- [ ] PM2 stats page secured with authentication
- [ ] HAProxy stats page secured (if publicly accessible)
- [ ] Regular security updates applied
- [ ] Log rotation configured for application logs

### Troubleshooting

**Application won't start:**
```bash
# Check PM2 logs
pm2 logs --err

# Check if ports are in use
netstat -tulpn | grep -E ':(3001|5173)'

# Verify environment variables
pm2 env 0
```

**HAProxy not forwarding requests:**
```bash
# Test backend directly
curl http://localhost:3001/api/health
curl http://localhost:5173/

# Check HAProxy configuration
haproxy -c -f /etc/haproxy/haproxy.cfg

# View HAProxy logs
tail -f /var/log/haproxy.log
```

**Database issues:**
```bash
# Check database file permissions
ls -la award.db

# Verify database integrity
bun run db:studio
```

---

## Features in Detail

### Background Job Queue

The application uses an in-memory job queue system for async operations:
- Jobs are persisted to database for recovery
- Only one active job per user (enforced at queue level)
- Status tracking: pending → running → completed/failed
- Real-time progress updates via job result field
- Client polls job status every 2 seconds

### LoTW Sync Logic

Following Wavelog's proven approach:
1. **First sync**: Uses date `2000-01-01` to retrieve all QSOs
2. **Subsequent syncs**: Uses `MAX(lotwQslRdate)` from database
3. **Validation**:
   - Checks for "Username/password incorrect" in response
   - Validates file starts with "ARRL Logbook of the World Status Report"
4. **Timeout handling**: 30-second connection timeout
5. **Query parameters**: Matches Wavelog's LoTW download

### Pagination

- Default page size: 100 QSOs per page
- Supports custom page size via `limit` parameter
- Shows page numbers with ellipsis for large page counts
- Displays "Showing X-Y of Z" info
- Previous/Next navigation buttons

## Development

### Database Migrations

```bash
# Push schema changes to database
bun run db:push

# Open Drizzle Studio (database GUI)
bun run db:studio
```

### Linting

```bash
bun run lint
```

## License

MIT

## Credits

- LoTW integration inspired by [Wavelog](https://github.com/magicbug/CloudLog)
- Built with [Bun](https://bun.sh), [Elysia](https://elysiajs.com), and [SvelteKit](https://kit.svelte.dev)