- Delete duplicate getCacheStats() function in cache.service.js
- Fix date calculation bug in lotw.service.js (was Date.now()-Date.now())
- Extract duplicate helper functions (yieldToEventLoop, getQSOKey) to sync-helpers.js
- Cache award definitions in memory to avoid repeated file I/O
- Delete unused parseDCLJSONResponse() function
- Remove unused imports (getPerformanceSummary, resetPerformanceMetrics)
- Auto-discover award JSON files instead of hardcoded list
Co-Authored-By: Claude <noreply@anthropic.com>
After deleting all QSOs, invalidate the stats and user caches so the
QSO page shows updated statistics instead of stale cached data.
Co-Authored-By: Claude <noreply@anthropic.com>
The qso_changes table has a foreign key reference to qsos.id, which
was preventing QSO deletion. Now deletes related qso_changes records
first before deleting QSOs.
Also added better error logging to the DELETE endpoint.
Co-Authored-By: Claude <noreply@anthropic.com>
The db.delete() returns a result object with a 'changes' property
indicating the number of affected rows, not the count directly.
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes frontend freeze during large sync operations (8000+ QSOs).
Root cause: Sequential processing with individual database operations
(~24,000 queries for 8000 QSOs) blocked the event loop, preventing
polling requests from being processed.
Changes:
- Process QSOs in batches of 100
- Single SELECT query per batch for duplicate detection
- Batch INSERTs for new QSOs and change tracking
- Add yield points (setImmediate) after each batch to allow
event loop processing of polling requests
Performance: ~98% reduction in database operations
Before: 8000 QSOs × 3 queries = ~24,000 sequential operations
After: 80 batches × ~4 operations = ~320 operations
Co-Authored-By: Claude <noreply@anthropic.com>
This reverts commit 5b78935 which added:
- Sync type parameter (qsl_delta, qsl_full, qso_delta, qso_full)
- getLastLoTWQSODate() function
- Sync type dropdown on QSO page
- Job queue handling of sync types
Reason: LoTW doesn't provide DXCC entity data for unconfirmed QSOs,
which causes award calculation issues. Going back to QSL-only sync.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add 'any' confirmation type filter showing QSOs confirmed by LoTW OR DCL
- Backend logic: lotwQslRstatus = 'Y' OR dclQslRstatus = 'Y'
- Frontend dropdown option positioned after "All QSOs"
- Shows all QSOs confirmed by at least one service (LoTW, DCL, or both)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add syncType parameter to LoTW sync: 'qsl_delta', 'qsl_full', 'qso_delta', 'qso_full'
- qsl_* = only confirmed QSOs (qso_qsl=yes)
- qso_* = all QSOs confirmed+unconfirmed (qso_qsl=no)
- delta = incremental sync with date filter
- full = sync all records without date filter
- Add getLastLoTWQSODate() for QSO-based incremental sync
- Add sync type dropdown selector on QSO page
- Update job queue service to handle sync types
- Update API endpoint to accept syncType in request body
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implement comprehensive sync job management with rollback capabilities
and real-time status updates on the dashboard.
## Features
### Cancel & Rollback
- Users can cancel failed or stale (>1h) sync jobs
- Rollback deletes added QSOs and restores updated QSOs to previous state
- Uses qso_changes table to track all modifications with before/after snapshots
- Server-side validation prevents cancelling completed or active jobs
### Database Changes
- Add qso_changes table to track QSO modifications per job
- Stores change type (added/updated), before/after data snapshots
- Enables precise rollback of sync operations
- Migration script included
### Real-time Updates
- Dashboard now polls for job updates every 2 seconds
- Smart polling: starts when jobs active, stops when complete
- Job status badges update in real-time (pending → running → completed)
- Cancel button appears/disappears based on job state
### Backend
- Fixed job ordering to show newest first (desc createdAt)
- Track all QSO changes during LoTW/DCL sync operations
- cancelJob() function handles rollback logic
- DELETE /api/jobs/:jobId endpoint for cancelling jobs
### Frontend
- jobsAPI.cancel() method for cancelling jobs
- Dashboard shows last 5 sync jobs with status, stats, duration
- Real-time job status updates via polling
- Cancel button with confirmation dialog
- Loading state and error handling
### Logging Fix
- Changed from Bun.write() to fs.appendFile() for reliable log appending
- Logs now persist across server restarts instead of being truncated
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fix N+1 query, add database indexes, and implement award progress caching:
- Fix N+1 query in getUserQSOs by using SQL COUNT instead of loading all records
- Add 7 performance indexes for filter queries, sync operations, and award calculations
- Implement in-memory caching service for award progress (5-minute TTL)
- Auto-invalidate cache after LoTW/DCL syncs
Expected impact:
- 90% memory reduction for QSO listing
- 80% faster filter queries
- 95% reduction in award calculation time for cached requests
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
QSO stats were only counting LoTW-confirmed QSOs, excluding
QSOs confirmed only by DCL from the "confirmed" count.
Changed getQSOStats() to count QSOs as confirmed if EITHER
LoTW OR DCL has confirmed them:
- Before: q.lotwQslRstatus === 'Y'
- After: q.lotwQslRstatus === 'Y' || q.dclQslRstatus === 'Y'
Fixes stats showing 8317/8338 confirmed when all QSOs were
confirmed by at least one system.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The award page was filtering QSOs by callsign/date/band/mode, which
could return the wrong QSO when multiple QSOs with the same callsign
exist on the same band/mode combination.
Changes:
- Backend: Add qsoId field to award entity breakdown responses
- Backend: Add GET /api/qsos/:id endpoint to fetch QSO by ID
- Backend: Implement getQSOById() function in lotw.service.js
- Frontend: Update openQSODetailModal() to fetch by qsoId instead of filtering
- Frontend: Include qsoId in QSO entry objects for modal click handler
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Critical bug: LoTW sync was missing timeOn in the duplicate detection
query, causing multiple QSOs with the same callsign/date/band/mode
but different times to be treated as duplicates.
Example: If you worked DL1ABC on 2025-01-15 at 10:00, 12:00, and 14:00
all on 80m CW, only the first QSO would be imported.
Now matches DCL sync logic which correctly includes timeOn:
- userId, callsign, qsoDate, timeOn, band, mode
This ensures all unique QSOs are properly imported from LoTW.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
When syncing from LoTW/DCL, only update QSOs if confirmation data
has changed. This avoids unnecessary database updates and makes it
clear which QSOs actually changed.
- LoTW: Checks if lotwQslRstatus or lotwQslRdate changed
- DCL: Checks if dclQslRstatus, dclQslRdate, DOK, or grid changed
- Frontend: Shows skipped count in sync summary
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add import log display that shows QSOs imported via LoTW/DCL sync.
Backend now tracks added/updated QSOs (callsign, date, band, mode)
and returns them in sync result. Frontend displays tables showing
new and updated QSOs after sync completes.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add shared ADIF parser utility (src/backend/utils/adif-parser.js)
- parseADIF(): Parse ADIF format into QSO records
- parseDCLResponse(): Parse DCL's JSON response format
- normalizeBand() and normalizeMode(): Standardize band/mode names
- Implement DCL service (src/backend/services/dcl.service.js)
- fetchQSOsFromDCL(): Fetch from DCL API (ready for API availability)
- parseDCLJSONResponse(): Parse example payload format
- syncQSOs(): Update existing QSOs with DCL confirmations
- Support DCL-specific fields: DCL_QSL_RCVD, DCL_QSLRDATE, DARC_DOK, MY_DARC_DOK
- Refactor LoTW service to use shared ADIF parser
- Remove duplicate parseADIF, normalizeBand, normalizeMode functions
- Import from shared utility for consistency
- Tested with example DCL payload
- Successfully parses all 6 QSOs
- Correctly extracts DCL confirmation data
- Handles ADIF format with <EOR> delimiters
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Frontend now uses @sveltejs/adapter-static for production builds
- Backend serves both API and static files from single port (originally port 3000)
- Removed all throw statements from services to avoid Elysia prototype errors
- Fixed favicon serving and SvelteKit assets path handling
- Added ecosystem.config.js for PM2 process management
- Comprehensive deployment documentation (PM2 + HAProxy)
- Updated README with single-port architecture
- Created start.sh script for easy production start
## Backend
- Add Pino logging framework with timestamps and structured output
- Replace all console.error statements (49+) with proper logging levels
- Fix Drizzle ORM bug: replace invalid .get() calls with .limit(1)
- Remove unused auth routes file (already in index.js)
- Make internal functions private (remove unnecessary exports)
- Simplify code by removing excessive debug logging
## Frontend
- Add navigation bar to layout with:
- User's callsign display
- Navigation links (Dashboard, QSOs, Settings)
- Logout button with red color distinction
- Navigation only shows when user is logged in
- Dark themed design matching footer
## Documentation
- Update README.md with new project structure
- Update docs/DOCUMENTATION.md with logging and nav bar info
- Add logger.js to configuration section
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add backend pagination support for /api/qsos endpoint (page, limit params)
- Return paginated results with metadata (totalCount, totalPages, hasNext, hasPrev)
- Add pagination UI to QSOs page with prev/next buttons and page numbers
- Add back button to QSO Log and Settings pages linking to main menu
- Reset to page 1 when applying filters
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Backend changes:
- Add sync_jobs table for background job tracking with Drizzle schema
- Create job queue service (job-queue.service.js) for async job processing
- Only ONE active sync job per user enforced at queue level
- Refactor LoTW service with Wavelog download logic:
- Validate for "Username/password incorrect" in response
- Check file starts with "ARRL Logbook of the World Status Report"
- Use last LoTW QSL date for incremental sync (qso_qslsince)
- Wavelog-compatible timeouts and error handling
- Add deleteQSOs function to clear all user QSOs
- Fix database path to use absolute path for consistency
- Register job processor for lotw_sync job type
API endpoints:
- POST /api/lotw/sync - Queue background sync job, returns jobId immediately
- GET /api/jobs/:jobId - Get job status with progress tracking
- GET /api/jobs/active - Get user's active job
- GET /api/jobs - Get user's recent jobs
- DELETE /api/qsos/all - Delete all QSOs for authenticated user
Frontend changes:
- Add job polling every 2 seconds during sync
- Show real-time progress indicator during sync
- Add "Clear All QSOs" button with type-to-confirm ("DELETE")
- Check for active job on mount to resume polling after refresh
- Clean up polling interval on component unmount
- Update API client with jobsAPI methods (getStatus, getActive, getRecent)
Database:
- Add sync_jobs table: id, userId, status, type, startedAt, completedAt,
result, error, createdAt
- Foreign key to users table
- Path fix: now uses src/backend/award.db consistently
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add detailed documentation covering architecture, components, code structure
- Document award system with multiple examples (DXCC, WAS, VUCC, Satellite)
- Implement LoTW sync service with ADIF parsing and long-polling
- Add QSO logbook page with filtering and statistics
- Add settings page for LoTW credentials management
- Add API endpoints for LoTW sync, QSO retrieval, and statistics
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
