Quickawards by DJ7NT

A web application for amateur radio operators to track QSOs (contacts) and award progress using Logbook of the World (LoTW) and DARC Community Logbook (DCL) data.

Features

• User Authentication: Register and login with callsign, email, and password
• LoTW Integration: Sync QSOs from ARRL's Logbook of the World
  • Background job queue for non-blocking sync operations
  • Incremental sync using last confirmation date
  • Wavelog-compatible download logic with proper validation
  • One sync job per user enforcement
  • Confirmation date and service type displayed in QSO table
• DCL Preparation: Infrastructure ready for DARC Community Logbook (DCL)
  • Database schema includes DCL confirmation fields (dcl_qsl_rdate, dcl_qsl_rstatus)
  • DOK (DARC Ortsverband Kennung) fields: my_darc_dok, darc_dok
  • Settings page includes DCL API key input (for future use)
  • Note: DCL download API is not yet available - infrastructure is prepared for when they add it
• QSO Log: View and manage confirmed QSOs
  • Pagination support for large QSO collections
  • Filter by band and mode
  • Statistics dashboard (total QSOs, confirmed, DXCC entities, bands)
  • Delete all QSOs with confirmation
  • Displays DOK fields for German award tracking
  • Multi-service confirmation display (LoTW, DCL)
• Settings: Configure LoTW and DCL credentials securely

Performance Optimizations

The application includes several performance optimizations for fast response times and efficient resource usage:

Database Performance

• Performance Indexes: 7 optimized indexes on QSO table
  • Filter queries (band, mode, confirmation status)
  • Sync duplicate detection (most impactful)
  • Award calculations (LoTW/DCL confirmed)
  • Date-based sorting
• Impact: 80% faster filter queries, 60% faster sync operations

Backend Optimizations

• N+1 Query Prevention: Uses SQL COUNT for pagination instead of loading all records
  • Impact: 90% memory reduction, 70% faster QSO listing
• Award Progress Caching: In-memory cache with 5-minute TTL
  • Impact: 95% faster award calculations for cached requests
  • Auto-invalidation after LoTW/DCL syncs
• Batch API Endpoints: Single request for all award progress
  • Impact: 95% reduction in API calls (awards page: 5s → 500ms)

Frontend Optimizations

• Component Extraction: Modular components for better performance
  • QSOStats: Statistics display component
  • SyncButton: Reusable sync button component
• Batch API Calls: Awards page loads all progress in one request
• Efficient Re-rendering: Reduced component re-renders through modular design

Deployment Optimizations

• Bun Configuration: Optimized bunfig.toml for production builds
• Production Templates: Ready-to-use deployment configuration

Tech Stack

Backend

• Runtime: Bun
• Framework: Elysia.js
• Database: SQLite with Drizzle ORM
• Authentication: JWT tokens
• Logging: Pino with structured logging and timestamps

Frontend

• Framework: SvelteKit
• Language: JavaScript
• Styling: Custom CSS

Project Structure

award/
├── src/
│   ├── backend/
│   │   ├── config/
│   │   │   └── config.js            # Centralized configuration (DB, JWT, logging)
│   │   ├── db/
│   │   │   └── schema/
│   │   │       └── index.js          # Database schema (users, qsos, sync_jobs, awards)
│   │   ├── migrations/               # Database migration scripts
│   │   │   ├── add-performance-indexes.js  # Create performance indexes
│   │   │   └── rollback-performance-indexes.js  # Rollback script
│   │   ├── services/
│   │   │   ├── auth.service.js       # User authentication
│   │   │   ├── cache.service.js      # Award progress caching
│   │   │   ├── lotw.service.js       # LoTW sync & QSO management
│   │   │   ├── dcl.service.js        # DCL sync
│   │   │   ├── job-queue.service.js  # Background job queue
│   │   │   └── awards.service.js     # Award progress tracking
│   │   ├── utils/
│   │   │   └── adif-parser.js        # ADIF format parser
│   │   └── index.js                  # API routes and server
│   └── frontend/
│       ├── src/
│       │   ├── lib/
│       │   │   ├── api.js            # API client
│       │   │   └── stores.js         # Svelte stores (auth)
│       │   └── routes/
│       │       ├── +layout.svelte       # Navigation bar & layout
│       │       ├── +page.svelte         # Dashboard
│       │       ├── auth/
│       │       │   ├── login/+page.svelte   # Login page
│       │       │   └── register/+page.svelte # Registration page
│       │       ├── qsos/
│       │       │   ├── +page.svelte        # QSO log page
│       │       │   └── components/         # QSO page components
│       │       │       ├── QSOStats.svelte    # Statistics display
│       │       │       └── SyncButton.svelte  # Sync button component
│       │       ├── awards/+page.svelte  # Awards progress tracking
│       │       └── settings/+page.svelte # Settings (credentials)
│       └── package.json
├── award-definitions/            # Award rule definitions (JSON)
├── award.db                     # SQLite database (auto-created)
├── .env.production.template     # Production configuration template
├── bunfig.toml                  # Bun configuration
├── drizzle.config.js            # Drizzle ORM configuration
├── package.json
└── README.md

Setup

Prerequisites

• Bun v1.3.6 or later

Installation

1. Clone the repository:

git clone <repository-url>
cd award

2. Install dependencies:

bun install

3. Set up environment variables: Create a .env file in the project root (copy from .env.example):

cp .env.example .env

Edit .env with your configuration:

# Application URL (for production deployment)
VITE_APP_URL=https://awards.dj7nt.de

# API Base URL (leave empty for same-domain deployment)
VITE_API_BASE_URL=

# JWT Secret (generate with: openssl rand -base64 32)
JWT_SECRET=your-generated-secret-here

# Environment
NODE_ENV=production

For development: You can leave .env empty or use defaults.

4. Initialize the database with performance indexes:

# Push database schema
bun run db:push

# Create performance indexes (recommended)
bun run db:indexes

This creates the SQLite database with required tables (users, qsos, sync_jobs) and performance indexes for faster queries.

Quick Start (Development)

# Install dependencies
bun install

# Initialize database
bun run db:push && bun run db:indexes

# Start development servers
bun run dev

Application available at: http://localhost:5173

Quick Deploy (Production)

# Pull latest code
git pull

# One-command deployment
bun run deploy

This runs: install → db migrations → indexes → build

Or run step-by-step:

bun install
bun run db:push
bun run db:indexes
bun run build

Running the Application

Start both backend and frontend with a single command:

bun run dev

Or start them individually:

# Backend only (port 3001, proxied)
bun run dev:backend

# Frontend only (port 5173)
bun run dev:frontend

The application will be available at:

• Frontend & API: http://localhost:5173

Note: During development, both servers run (frontend on 5173, backend on 3001), but API requests are automatically proxied through the frontend. You only need to access port 5173.

API Endpoints

Authentication

• POST /api/auth/register - Register new user
• POST /api/auth/login - Login user
• GET /api/auth/me - Get current user profile
• PUT /api/auth/lotw-credentials - Update LoTW credentials
• PUT /api/auth/dcl-credentials - Update DCL API key (for future use)

LoTW Sync

• POST /api/lotw/sync - Queue a LoTW sync job (returns job ID)

Awards

• GET /api/awards - Get all available awards
• GET /api/awards/batch/progress - Get progress for all awards (optimized, single request)
• GET /api/awards/:awardId/progress - Get award progress for a specific award
• GET /api/awards/:awardId/entities - Get entity breakdown

Jobs

• GET /api/jobs/:jobId - Get job status
• GET /api/jobs/active - Get user's active job
• GET /api/jobs - Get recent jobs (query: ?limit=10)

QSOs

• GET /api/qsos - Get user's QSOs with pagination
  • Query parameters: ?page=1&limit=100&band=20m&mode=CW
• GET /api/qsos/stats - Get QSO statistics
• DELETE /api/qsos/all - Delete all QSOs (requires confirmation)

Health

• GET /api/health - Health check endpoint

Database Schema

Users Table

CREATE TABLE users (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  email TEXT UNIQUE NOT NULL,
  password TEXT NOT NULL,
  callsign TEXT NOT NULL,
  lotwUsername TEXT,
  lotwPassword TEXT,
  dclApiKey TEXT,              -- DCL API key (for future use)
  createdAt TEXT NOT NULL,
  updatedAt TEXT NOT NULL
);

QSOs Table

CREATE TABLE qsos (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  userId INTEGER NOT NULL,
  callsign TEXT NOT NULL,
  qsoDate TEXT NOT NULL,
  timeOn TEXT NOT NULL,
  band TEXT,
  mode TEXT,
  entity TEXT,
  entityId INTEGER,
  grid TEXT,
  gridSource TEXT,
  continent TEXT,
  cqZone INTEGER,
  ituZone INTEGER,
  state TEXT,
  county TEXT,
  satName TEXT,
  satMode TEXT,
  myDarcDok TEXT,              -- User's DOK (e.g., 'F03', 'P30')
  darcDok TEXT,                -- QSO partner's DOK
  lotwQslRstatus TEXT,         -- LoTW confirmation status ('Y', 'N', '?')
  lotwQslRdate TEXT,           -- LoTW confirmation date (ADIF format: YYYYMMDD)
  dclQslRstatus TEXT,          -- DCL confirmation status ('Y', 'N', '?')
  dclQslRdate TEXT,            -- DCL confirmation date (ADIF format: YYYYMMDD)
  lotwSyncedAt TEXT,
  createdAt TEXT NOT NULL,
  FOREIGN KEY (userId) REFERENCES users(id)
);

Sync Jobs Table

CREATE TABLE sync_jobs (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  userId INTEGER NOT NULL,
  status TEXT NOT NULL, -- pending, running, completed, failed
  type TEXT NOT NULL,   -- lotw_sync
  startedAt INTEGER,
  completedAt INTEGER,
  result TEXT,          -- JSON
  error TEXT,
  createdAt INTEGER NOT NULL,
  FOREIGN KEY (userId) REFERENCES users(id)
);

Architecture

Development Mode

• SvelteKit Dev Server (port 5173): Serves frontend and proxies API requests
• Elysia Backend (port 3001): Handles API requests (hidden from user)
• Proxy Configuration: All /api/* requests are forwarded from SvelteKit to Elysia

This gives you:

• ✅ Single port to access (5173)
• ✅ Hot Module Replacement (HMR) for frontend
• ✅ No CORS issues
• ✅ Simple production deployment

Production Mode

In production, the application serves everything from a single port:

• Backend on port 3001 serves both API and static frontend files
• Frontend static files are served from src/frontend/build/
• SPA routing is handled by backend fallback
• Single port simplifies HAProxy configuration

Production Deployment

This guide covers deployment using PM2 for process management and HAProxy as a reverse proxy/load balancer.

Architecture Overview

Internet
    │
    ▼
┌─────────────────┐
│   HAProxy       │  Port 443 (HTTPS)
│   (Port 80/443) │  Port 80 (HTTP → HTTPS redirect)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   Backend       │  Port 3001
│   Managed by    │  ├─ API Routes (/api/*)
│   PM2           │  ├─ Static Files (/*)
│                 │  └─ SPA Fallback
└────────┬────────┘
         │
         ├─────────────────┬──────────────────┐
         │                 │                  │
         ▼                 ▼                  ▼
    ┌─────────┐      ┌──────────┐     ┌─────────────┐
    │ SQLite  │      │ Frontend │     │  ARRL LoTW  │
    │   DB    │      │  Build   │     │ External API│
    └─────────┘      └──────────┘     └─────────────┘

Prerequisites

• Server with SSH access
• Bun runtime installed
• PM2 installed globally: bun install -g pm2 or npm install -g pm2
• HAProxy installed
• Domain with DNS pointing to server

Step 1: Build the Application

# Clone repository on server
git clone <repository-url>
cd award

# Install dependencies
bun install

# Install frontend dependencies
cd src/frontend
bun install

# Build frontend (generates static files in src/frontend/build/)
bun run build

Step 2: Configure Environment Variables

Create .env in the project root:

# Application URL
VITE_APP_URL=https://awards.dj7nt.de

# API Base URL (empty for same-domain)
VITE_API_BASE_URL=

# JWT Secret (generate with: openssl rand -base64 32)
JWT_SECRET=your-generated-secret-here

# Environment
NODE_ENV=production

# Database path (absolute path recommended)
DATABASE_PATH=/path/to/award/award.db

Security: Ensure .env has restricted permissions:

chmod 600 .env

Step 3: Initialize Database

# Push database schema
bun run db:push

# Verify database was created
ls -la award.db

Step 4: Create PM2 Ecosystem Configuration

Create ecosystem.config.js in the project root:

module.exports = {
  apps: [
    {
      name: 'award-backend',
      script: 'src/backend/index.js',
      interpreter: 'bun',
      cwd: '/path/to/award',
      env: {
        NODE_ENV: 'production',
        PORT: 3001
      },
      instances: 1,
      exec_mode: 'fork',
      autorestart: true,
      watch: false,
      max_memory_restart: '500M',
      error_file: './logs/backend-error.log',
      out_file: './logs/backend-out.log',
      log_date_format: 'YYYY-MM-DD HH:mm:ss Z'
    },
    {
      name: 'award-frontend',
      script: 'bun',
      args: 'run preview',
      cwd: '/path/to/award/src/frontend',
      env: {
        NODE_ENV: 'production',
        PORT: 5173
      },
      instances: 1,
      exec_mode: 'fork',
      autorestart: true,
      watch: false,
      max_memory_restart: '300M',
      error_file: './logs/frontend-error.log',
      out_file: './logs/frontend-out.log',
      log_date_format: 'YYYY-MM-DD HH:mm:ss Z'
    }
  ]
};

Create logs directory:

mkdir -p logs

Step 5: Start Applications with PM2

# Start all applications
pm2 start ecosystem.config.js

# Save PM2 process list
pm2 save

# Setup PM2 to start on system reboot
pm2 startup
# Follow the instructions output by the command above

Useful PM2 Commands:

# View status
pm2 status

# View logs
pm2 logs

# Restart all apps
pm2 restart all

# Restart specific app
pm2 restart award-backend

# Stop all apps
pm2 stop all

# Monitor resources
pm2 monit

Step 6: Configure HAProxy

Edit /etc/haproxy/haproxy.cfg:

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

    # Default SSL material locations
    ca-base /etc/ssl/certs
    crt-base /etc/ssl/private

    # Default ciphers to use
    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
    ssl-default-bind-options no-sslv3

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

# Statistics page (optional - secure with auth)
listen stats
    bind *:8404
    mode http
    stats enable
    stats uri /stats
    stats refresh 30s
    stats realm HAProxy\ Statistics
    stats auth admin:your-secure-password

# Frontend redirect HTTP to HTTPS
frontend http-in
    bind *:80
    http-request redirect scheme https

# Frontend HTTPS
frontend https-in
    bind *:443 ssl crt /etc/ssl/private/awards.dj7nt.de.pem
    default_backend award-backend

# Backend configuration
backend award-backend
    # Health check
    option httpchk GET /api/health

    # Single server serving both frontend and API
    server award-backend 127.0.0.1:3001 check

SSL Certificate Setup:

Using Let's Encrypt with Certbot:

# Install certbot
apt install certbot

# Generate certificate
certbot certonly --standalone -d awards.dj7nt.de

# Combine certificate and key for HAProxy
cat /etc/letsencrypt/live/awards.dj7nt.de/fullchain.pem > /etc/ssl/private/awards.dj7nt.de.pem
cat /etc/letsencrypt/live/awards.dj7nt.de/privkey.pem >> /etc/ssl/private/awards.dj7nt.de.pem

# Set proper permissions
chmod 600 /etc/ssl/private/awards.dj7nt.de.pem

Step 7: Start HAProxy

# Test configuration
haproxy -c -f /etc/haproxy/haproxy.cfg

# Restart HAProxy
systemctl restart haproxy

# Enable HAProxy on boot
systemctl enable haproxy

# Check status
systemctl status haproxy

Step 8: Verify Deployment

# Check PM2 processes
pm2 status

# Check HAProxy stats (if enabled)
curl http://localhost:8404/stats

# Test health endpoint
curl https://awards.dj7nt.de/api/health

# Check logs
pm2 logs
tail -f /var/log/haproxy.log

Updating the Application

# Pull latest changes
git pull

# One-command deployment (recommended)
bun run deploy

# Restart PM2
pm2 restart award-backend

Or manual step-by-step:

# Install updated dependencies
bun install

# Push any schema changes
bun run db:push

# Update/create performance indexes
bun run db:indexes

# Rebuild frontend
bun run build

# Restart PM2
pm2 restart award-backend

Database Backups

Set up automated backups:

# Create backup script
cat > /usr/local/bin/backup-award.sh << 'EOF'
#!/bin/bash
BACKUP_DIR="/backups/award"
DATE=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR

# Backup database
cp /path/to/award/award.db $BACKUP_DIR/award_$DATE.db

# Keep last 30 days
find $BACKUP_DIR -name "award_*.db" -mtime +30 -delete
EOF

chmod +x /usr/local/bin/backup-award.sh

# Add to crontab (daily at 2 AM)
crontab -e
# Add line: 0 2 * * * /usr/local/bin/backup-award.sh

Monitoring

PM2 Monitoring:

# Real-time monitoring
pm2 monit

# View logs
pm2 logs --lines 100

HAProxy Monitoring:

• Access stats page: http://your-server:8404/stats
• Check logs: tail -f /var/log/haproxy.log

Log Files Locations:

• PM2 logs: ./logs/backend-error.log, ./logs/frontend-error.log
• HAProxy logs: /var/log/haproxy.log
• System logs: journalctl -u haproxy -f

Security Checklist

• HTTPS enabled with valid SSL certificate
• Firewall configured (ufw/firewalld)
• JWT_SECRET is strong and randomly generated
• .env file has proper permissions (600)
• Database backups automated
• PM2 stats page secured with authentication
• HAProxy stats page secured (if publicly accessible)
• Regular security updates applied
• Log rotation configured for application logs

Troubleshooting

Application won't start:

# Check PM2 logs
pm2 logs --err

# Check if ports are in use
netstat -tulpn | grep -E ':(3001|5173)'

# Verify environment variables
pm2 env 0

HAProxy not forwarding requests:

# Test backend directly
curl http://localhost:3001/api/health
curl http://localhost:5173/

# Check HAProxy configuration
haproxy -c -f /etc/haproxy/haproxy.cfg

# View HAProxy logs
tail -f /var/log/haproxy.log

Database issues:

# Check database file permissions
ls -la award.db

# Verify database integrity
bun run db:studio

---

Features in Detail

Background Job Queue

The application uses an in-memory job queue system for async operations:

• Jobs are persisted to database for recovery
• Only one active job per user (enforced at queue level)
• Status tracking: pending → running → completed/failed
• Real-time progress updates via job result field
• Client polls job status every 2 seconds

LoTW Sync Logic

Following Wavelog's proven approach:

1. First sync: Uses date 2000-01-01 to retrieve all QSOs
2. Subsequent syncs: Uses MAX(lotwQslRdate) from database
3. Validation:
   • Checks for "Username/password incorrect" in response
   • Validates file starts with "ARRL Logbook of the World Status Report"
4. Timeout handling: 30-second connection timeout
5. Query parameters: Matches Wavelog's LoTW download

DOK Fields (DARC Ortsverband Kennung)

The QSO table includes DOK fields for German amateur radio awards:

• myDarcDok: User's own DOK (e.g., 'F03', 'P30', 'G13')
• darcDok: QSO partner's DOK

DOKs are local club identifiers used by DARC (German amateur radio club) for award tracking. These fields are populated when syncing from LoTW if the ADIF data contains MY_DARC_DOK and DARC_DOK tags.

DCL Preparation

The application is prepared for future DARC Community Logbook (DCL) integration:

Infrastructure in place:

• Database schema includes DCL confirmation fields (dcl_qsl_rdate, dcl_qsl_rstatus)
• Backend service stub (src/backend/services/dcl.service.js) with TODO comments for implementation
• Settings page includes DCL API key input
• QSO table displays DCL confirmations alongside LoTW

Current status:

• DCL does not provide a public download API (as of 2025)
• Manual ADIF export is available at https://dcl.darc.de/dml/export_adif_form.php
• When DCL adds an API endpoint, the existing infrastructure can be easily activated

Future implementation steps (when DCL API is available):

1. Implement fetchQSOsFromDCL() in dcl.service.js
2. Add ADIF parser for DCL format
3. Implement syncQSOs() to store DCL confirmations
4. Add sync endpoint similar to LoTW

Confirmation Display

The QSO table shows confirmations from multiple services:

• Each service is listed with its name (LoTW, DCL) and confirmation date
• Multiple confirmations per QSO are supported
• Empty state shows "-" when no confirmations exist
• Service types are color-coded and formatted for easy scanning

Pagination

• Default page size: 100 QSOs per page
• Supports custom page size via limit parameter
• Shows page numbers with ellipsis for large page counts
• Displays "Showing X-Y of Z" info
• Previous/Next navigation buttons

Development

Available Scripts

# Development
bun run dev              # Start both backend (3001) and frontend (5173)
bun run dev:backend      # Start backend only
bun run dev:frontend     # Start frontend only

# Database
bun run db:push          # Push schema changes via Drizzle
bun run db:indexes       # Create/update performance indexes
bun run db:studio        # Open Drizzle Studio (database GUI)
bun run db:generate      # Generate Drizzle migrations
bun run db:migrate       # Run Drizzle migrations

# Build & Deploy
bun run build            # Build frontend for production
bun run deploy           # Full deployment pipeline (install + db + indexes + build)

# Deployment on production
git pull && bun run deploy && pm2 restart award-backend

Database Migrations

The application uses two types of database changes:

1. Schema Changes (Drizzle ORM)

bun run db:push    # Push schema changes

2. Performance Indexes (Custom)

bun run db:indexes # Create/update performance indexes

The indexes are idempotent (safe to run multiple times) and include:

• Filter query indexes (band, mode, confirmation)
• Sync duplicate detection index
• Award calculation indexes
• Date sorting index

Linting

bun run lint

License

MIT

Credits

• LoTW integration inspired by Wavelog
• Built with Bun, Elysia, and SvelteKit