award/CLAUDE.md at aeeb75c226a32a741dfdab03489a721fbc19f111

Files

fix: count QSOs confirmed by either LoTW or DCL in stats

QSO stats were only counting LoTW-confirmed QSOs, excluding
QSOs confirmed only by DCL from the "confirmed" count.

Changed getQSOStats() to count QSOs as confirmed if EITHER
LoTW OR DCL has confirmed them:
- Before: q.lotwQslRstatus === 'Y'
- After: q.lotwQslRstatus === 'Y' || q.dclQslRstatus === 'Y'

Fixes stats showing 8317/8338 confirmed when all QSOs were
confirmed by at least one system.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

2026-01-19 13:32:45 +01:00

22 KiB

Raw Blame History

Default to using Bun instead of Node.js.

Use bun <file> instead of node <file> or ts-node <file>
Use bun test instead of jest or vitest
Use bun build <file.html|file.ts|file.css> instead of webpack or esbuild
Use bun install instead of npm install or yarn install or pnpm install
Use bun run <script> instead of npm run <script> or yarn run <script> or pnpm run <script>
Use bunx <package> <command> instead of npx <package> <command>
Bun automatically loads .env, so don't use dotenv.

APIs

Bun.serve() supports WebSockets, HTTPS, and routes. Don't use express.
bun:sqlite for SQLite. Don't use better-sqlite3.
Bun.redis for Redis. Don't use ioredis.
Bun.sql for Postgres. Don't use pg or postgres.js.
WebSocket is built-in. Don't use ws.
Prefer Bun.file over node:fs's readFile/writeFile
Bun.$ls instead of execa.

Logging

The application uses a custom logger in src/backend/config.js:

Log levels: debug (0), info (1), warn (2), error (3)
Default: debug in development, info in production
Override: Set LOG_LEVEL environment variable (e.g., LOG_LEVEL=debug)
Output format: [timestamp] LEVEL: message with JSON data

Important: The logger uses the nullish coalescing operator (??) to handle log levels. This ensures that debug (level 0) is not treated as falsy.

Example .env file:

NODE_ENV=development
LOG_LEVEL=debug

Testing

Use bun test to run tests.

import { test, expect } from "bun:test";

test("hello world", () => {
  expect(1).toBe(1);
});

Frontend

Use HTML imports with Bun.serve(). Don't use vite. HTML imports fully support React, CSS, Tailwind.

Server:

import index from "./index.html"

Bun.serve({
  routes: {
    "/": index,
    "/api/users/:id": {
      GET: (req) => {
        return new Response(JSON.stringify({ id: req.params.id }));
      },
    },
  },
  // optional websocket support
  websocket: {
    open: (ws) => {
      ws.send("Hello, world!");
    },
    message: (ws, message) => {
      ws.send(message);
    },
    close: (ws) => {
      // handle close
    }
  },
  development: {
    hmr: true,
    console: true,
  }
})

HTML files can import .tsx, .jsx or .js files directly and Bun's bundler will transpile & bundle automatically. <link> tags can point to stylesheets and Bun's CSS bundler will bundle.

<html>
  <body>
    <h1>Hello, world!</h1>
    <script type="module" src="./frontend.tsx"></script>
  </body>
</html>

With the following frontend.tsx:

import React from "react";
import { createRoot } from "react-dom/client";

// import .css files directly and it works
import './index.css';

const root = createRoot(document.body);

export default function Frontend() {
  return <h1>Hello, world!</h1>;
}

root.render(<Frontend />);

Then, run index.ts

bun --hot ./index.ts

For more information, read the Bun API docs in node_modules/bun-types/docs/**.mdx.

Project: Quickawards by DJ7NT

Quickawards is a amateur radio award tracking application that calculates progress toward various awards based on QSO (contact) data.

Award System Architecture

The award system is JSON-driven and located in award-definitions/ directory. Each award has:

id: Unique identifier (e.g., "dld", "dxcc")
name: Display name
description: Short description
caption: Detailed explanation
category: Award category ("dxcc", "darc", etc.)
rules: Award calculation logic

Award Rule Types

entity: Count unique entities (DXCC countries, states, grid squares)
- entityType: What to count ("dxcc", "state", "grid", "callsign")
- target: Number required for award
- filters: Optional filters (band, mode, etc.)
- displayField: Optional field to display
dok: Count unique DOK (DARC Ortsverband Kennung) combinations
- target: Number required
- confirmationType: "dcl" (DARC Community Logbook)
- filters: Optional filters (band, mode, etc.) for award variants
- Counts unique (DOK, band, mode) combinations
- Only DCL-confirmed QSOs count
- Example variants: DLD 80m, DLD CW, DLD 80m CW
points: Point-based awards
- stations: Array of {callsign, points}
- target: Points required
- countMode: "perStation", "perBandMode", or "perQso"
filtered: Filtered version of another award
- baseRule: The base entity rule
- filters: Additional filters to apply
counter: Count QSOs or callsigns

Key Files

Backend Award Service: src/backend/services/awards.service.js

getAllAwards(): Returns all available award definitions
calculateAwardProgress(userId, award, options): Main calculation function
calculateDOKAwardProgress(userId, award, options): DOK-specific calculation
calculatePointsAwardProgress(userId, award, options): Point-based calculation
getAwardEntityBreakdown(userId, awardId): Detailed entity breakdown
getAwardProgressDetails(userId, awardId): Progress with details

Database Schema: src/backend/db/schema/index.js

QSO fields include: darcDok, dclQslRstatus, dclQslRdate
DOK fields support DLD award tracking
DCL confirmation fields separate from LoTW

Award Definitions: award-definitions/*.json

Add new awards by creating JSON definition files
Add filename to loadAwardDefinitions() file list in awards.service.js

ADIF Parser: src/backend/utils/adif-parser.js

parseADIF(adifData): Parse ADIF format into QSO records
- Handles case-insensitive <EOR> delimiters (supports <EOR>, <eor>, <Eor>)
- Uses matchAll() for reliable field parsing
- Skips header records automatically
parseDCLResponse(response): Parse DCL's JSON response format { "adif": "..." }
normalizeBand(band): Standardize band names (80m, 40m, etc.)
normalizeMode(mode): Standardize mode names (CW, FT8, SSB, etc.)
Used by both LoTW and DCL services for consistency

Job Queue Service: src/backend/services/job-queue.service.js

Manages async background jobs for LoTW and DCL sync
enqueueJob(userId, jobType): Queue a sync job ('lotw_sync' or 'dcl_sync')
processJobAsync(jobId, userId, jobType): Process job asynchronously
getUserActiveJob(userId, jobType): Get active job for user (optional type filter)
getJobStatus(jobId): Get job status with parsed result
updateJobProgress(jobId, progressData): Update job progress during processing
Supports concurrent LoTW and DCL sync jobs
Job types: 'lotw_sync', 'dcl_sync'
Job status: 'pending', 'running', 'completed', 'failed'

Backend API Routes (src/backend/index.js):

POST /api/lotw/sync: Queue LoTW sync job
POST /api/dcl/sync: Queue DCL sync job
GET /api/jobs/:jobId: Get job status
GET /api/jobs/active: Get active job for current user
GET /*: Serves static files from src/frontend/build/ with SPA fallback

SPA Routing: The backend serves the SvelteKit frontend build from src/frontend/build/.

Paths with file extensions (.js, .css, etc.) are served as static files
Paths without extensions (e.g., /qsos, /awards) are served index.html for client-side routing
Common missing files like /favicon.ico return 404 immediately
If frontend build is missing entirely, returns a user-friendly 503 HTML page
Prevents ugly Bun error pages when accessing client-side routes via curl or non-JS clients

DCL Service: src/backend/services/dcl.service.js

fetchQSOsFromDCL(dclApiKey, sinceDate): Fetch from DCL API
API Endpoint: https://dings.dcl.darc.de/api/adiexport
Request: POST with JSON body { key, limit: 50000, qsl_since, qso_since, cnf_only }
parseDCLJSONResponse(jsonResponse): Parse example/test payloads
syncQSOs(userId, dclApiKey, sinceDate, jobId): Sync QSOs to database
getLastDCLQSLDate(userId): Get last QSL date for incremental sync
Debug logging (when LOG_LEVEL=debug) shows API params with redacted key (first/last 4 chars)
Fully implemented and functional
Note: DCL API is a custom prototype by DARC; contact DARC for API specification details

DLD Award Implementation (COMPLETED)

The DLD (Deutschland Diplom) award was recently implemented:

Definition: award-definitions/dld.json

{
  "id": "dld",
  "name": "DLD",
  "description": "Deutschland Diplom - Confirm 100 unique DOKs on different bands/modes",
  "caption": "Contact and confirm stations with 100 unique DOKs (DARC Ortsverband Kennung) on different band/mode combinations.",
  "category": "darc",
  "rules": {
    "type": "dok",
    "target": 100,
    "confirmationType": "dcl",
    "displayField": "darcDok"
  }
}

Implementation Details:

Function: calculateDOKAwardProgress() in src/backend/services/awards.service.js (lines 173-268)
Counts unique (DOK, band, mode) combinations
Only DCL-confirmed QSOs count (dclQslRstatus === 'Y')
Each unique DOK on each unique band/mode counts separately
Returns worked, confirmed counts and entity breakdowns

Database Fields Used:

darcDok: DOK identifier (e.g., "F03", "P30", "G20")
band: Band (e.g., "80m", "40m", "20m")
mode: Mode (e.g., "CW", "SSB", "FT8")
dclQslRstatus: DCL confirmation status ('Y' = confirmed)
dclQslRdate: DCL confirmation date

Documentation: See docs/DOCUMENTATION.md for complete documentation including DLD award example.

Frontend: src/frontend/src/routes/qsos/+page.svelte

Separate sync buttons for LoTW (blue) and DCL (orange)
Independent progress tracking for each sync type
Both syncs can run simultaneously
Job polling every 2 seconds for status updates
Import log displays after sync completion
Real-time QSO table refresh after sync

Frontend API (src/frontend/src/lib/api.js):

qsosAPI.syncFromLoTW(): Trigger LoTW sync
qsosAPI.syncFromDCL(): Trigger DCL sync
jobsAPI.getStatus(jobId): Poll job status
jobsAPI.getActive(): Get active job on page load

Adding New Awards

To add a new award:

Create JSON definition in award-definitions/
Add filename to loadAwardDefinitions() in src/backend/services/awards.service.js
If new rule type needed, add calculation function
Add type handling in calculateAwardProgress() switch statement
Add type handling in getAwardEntityBreakdown() if needed
Update documentation in docs/DOCUMENTATION.md
Test with sample QSO data

Creating DLD Award Variants

The DOK award type supports filters to create award variants. Examples:

DLD on 80m (dld-80m.json):

{
  "id": "dld-80m",
  "name": "DLD 80m",
  "description": "Confirm 100 unique DOKs on 80m",
  "caption": "Contact 100 different DOKs on the 80m band.",
  "category": "darc",
  "rules": {
    "type": "dok",
    "target": 100,
    "confirmationType": "dcl",
    "displayField": "darcDok",
    "filters": {
      "operator": "AND",
      "filters": [
        { "field": "band", "operator": "eq", "value": "80m" }
      ]
    }
  }
}

DLD in CW mode (dld-cw.json):

{
  "rules": {
    "type": "dok",
    "target": 100,
    "confirmationType": "dcl",
    "filters": {
      "operator": "AND",
      "filters": [
        { "field": "mode", "operator": "eq", "value": "CW" }
      ]
    }
  }
}

DLD on 80m using CW (combined filters, dld-80m-cw.json):

{
  "rules": {
    "type": "dok",
    "target": 100,
    "confirmationType": "dcl",
    "filters": {
      "operator": "AND",
      "filters": [
        { "field": "band", "operator": "eq", "value": "80m" },
        { "field": "mode", "operator": "eq", "value": "CW" }
      ]
    }
  }
}

Available filter operators:

eq: equals
ne: not equals
in: in array
nin: not in array
contains: contains substring

Available filter fields: Any QSO field (band, mode, callsign, grid, state, satName, etc.)

Confirmation Systems

LoTW (Logbook of The World): ARRL's confirmation system
- Service: src/backend/services/lotw.service.js
- API: https://lotw.arrl.org/lotwuser/lotwreport.adi
- Fields: lotwQslRstatus, lotwQslRdate
- Used for DXCC, WAS, VUCC, most awards
- ADIF format with <EOR> delimiters
- Supports incremental sync by qso_qslsince parameter (format: YYYY-MM-DD)
DCL (DARC Community Logbook): DARC's confirmation system
- Service: src/backend/services/dcl.service.js
- API: https://dings.dcl.darc.de/api/adiexport
- Fields: dclQslRstatus, dclQslRdate
- DOK fields: darcDok (partner's DOK), myDarcDok (user's DOK)
- Required for DLD award
- German amateur radio specific
- Request format: POST JSON { key, limit, qsl_since, qso_since, cnf_only }
- Response format: JSON with ADIF string in adif field
- Supports incremental sync by qsl_since parameter (format: YYYYMMDD)
- Updates QSOs only if confirmation data has changed

ADIF Format

Both LoTW and DCL return data in ADIF (Amateur Data Interchange Format):

Field format: <FIELD_NAME:length>value
Record delimiter: <EOR> (end of record, case-insensitive)
Header ends with: <EOH> (end of header)
Example: <CALL:5>DK0MU<BAND:3>80m<QSO_DATE:8>20250621<EOR>
Important: Parser handles case-insensitive <EOR>, <eor>, <Eor> tags

DCL-specific fields:

DCL_QSL_RCVD: DCL confirmation status (Y/N/?)
DCL_QSLRDATE: DCL confirmation date (YYYYMMDD)
DARC_DOK: QSO partner's DOK
MY_DARC_DOK: User's own DOK
STATION_CALLSIGN: User's callsign

Recent Commits

[uncommitted]: fix: count QSOs confirmed by either LoTW or DCL in stats
- QSO stats were only counting LoTW-confirmed QSOs (lotwQslRstatus === 'Y')
- QSOs confirmed only by DCL were excluded from "confirmed" count
- Fixed by changing filter to: q.lotwQslRstatus === 'Y' || q.dclQslRstatus === 'Y'
- Now correctly shows all QSOs confirmed by at least one system
233888c: fix: make ADIF parser case-insensitive for EOR delimiter
- Critical bug: LoTW uses lowercase <eor> tags, parser was splitting on uppercase <EOR>
- Caused 242K+ QSOs to be parsed as 1 giant record with fields overwriting each other
- Changed to case-insensitive regex: new RegExp('<eor>', 'gi')
- Replaced regex.exec() while loop with matchAll() for-of iteration
- Now correctly imports all QSOs from large LoTW reports
645f786: fix: add missing timeOn field to LoTW duplicate detection
- LoTW sync was missing timeOn in duplicate detection query
- Multiple QSOs with same callsign/date/band/mode but different times were treated as duplicates
- Now matches DCL sync logic: userId, callsign, qsoDate, timeOn, band, mode
7f77c3a: feat: add filter support for DOK awards
- DOK award type now supports filtering by band, mode, and other QSO fields
- Allows creating award variants like DLD 80m, DLD CW, DLD 80m CW
- Uses existing filter system with eq, ne, in, nin, contains operators
- Example awards created: dld-80m, dld-40m, dld-cw, dld-80m-cw
9e73704: docs: update CLAUDE.md with DLD award variants documentation
7201446: fix: return proper HTML for SPA routes instead of Bun error page
- When accessing client-side routes (like /qsos) via curl or non-JS clients, the server attempted to open them as static files, causing Bun to throw an unhandled ENOENT error that showed an ugly error page
- Now checks if a path has a file extension before attempting to serve it
- Paths without extensions are immediately served index.html for SPA routing
- Also improves the 503 error page with user-friendly HTML when frontend build is missing
223461f: fix: enable debug logging and improve DCL sync observability
27d2ef1: fix: preserve DOK data when DCL doesn't send values
- DCL sync only updates DOK/grid fields when DCL provides non-empty values
- Prevents accidentally clearing DOK data from manual entry or other sources
- Preserves existing DOK when DCL syncs QSO without DOK information
e09ab94: feat: skip QSOs with unchanged confirmation data
- LoTW/DCL sync only updates QSOs if confirmation data has changed
- Tracks added, updated, and skipped QSO counts
- LoTW: Checks if lotwQslRstatus or lotwQslRdate changed
- DCL: Checks if dclQslRstatus, dclQslRdate, darcDok, myDarcDok, or grid changed
3592dbb: feat: add import log showing synced QSOs
- Backend returns addedQSOs and updatedQSOs arrays in sync result
- Frontend displays import log with callsign, date, band, mode for each QSO
- Separate sections for "New QSOs" and "Updated QSOs"
- Sync summary shows total, added, updated, skipped counts
8a1a580: feat: implement DCL ADIF parser and service integration
- Add shared ADIF parser utility (src/backend/utils/adif-parser.js)
- Implement DCL service with API integration
- Refactor LoTW service to use shared parser
- Tested with example DCL payload (6 QSOs parsed successfully)
c982dcd: feat: implement DLD (Deutschland Diplom) award
322ccaf: docs: add DLD (Deutschland Diplom) award documentation

Sync Behavior

Import Log: After each sync, displays a table showing:

New QSOs: Callsign, Date, Band, Mode
Updated QSOs: Callsign, Date, Band, Mode (only if data changed)
Skipped QSOs: Counted but not shown (data unchanged)

Duplicate Handling:

QSOs matched by: userId, callsign, qsoDate, timeOn, band, mode
If confirmation data unchanged: Skipped (not updated)
If confirmation data changed: Updated with new values
Prevents unnecessary database writes and shows accurate import counts

DOK Update Behavior:

If QSO imported via LoTW (no DOK) and later DCL confirms with DOK: DOK is added ✓
If QSO already has DOK and DCL sends different DOK: DOK is updated ✓
If QSO has DOK and DCL syncs without DOK (empty): Existing DOK is preserved ✓
LoTW never sends DOK data; only DCL provides DOK fields

Important: DCL sync only updates DOK/grid fields when DCL provides non-empty values. This prevents accidentally clearing DOK data that was manually entered or imported from other sources.

QSO Page Filters

The QSO page (src/frontend/src/routes/qsos/+page.svelte) includes advanced filtering capabilities:

Available Filters:

Search Box: Full-text search across callsign, entity (DXCC country), and grid square fields
- Press Enter to apply search
- Case-insensitive partial matching
Band Filter: Dropdown to filter by amateur band (160m, 80m, 60m, 40m, 30m, 20m, 17m, 15m, 12m, 10m, 6m, 2m, 70cm)
Mode Filter: Dropdown to filter by mode (CW, SSB, AM, FM, RTTY, PSK31, FT8, FT4, JT65, JT9)
Confirmation Type Filter: Filter by confirmation status
- "All QSOs": Shows all QSOs (no filter)
- "LoTW Only": Shows QSOs confirmed by LoTW but NOT DCL
- "DCL Only": Shows QSOs confirmed by DCL but NOT LoTW
- "Both Confirmed": Shows QSOs confirmed by BOTH LoTW AND DCL
- "Not Confirmed": Shows QSOs confirmed by NEITHER LoTW nor DCL
Clear Button: Resets all filters and reloads all QSOs

Backend Implementation (src/backend/services/lotw.service.js):

getUserQSOs(userId, filters, options): Main filtering function
Supports pagination with page and limit options
Filter logic uses Drizzle ORM query builders for safe SQL generation
Debug logging when LOG_LEVEL=debug shows applied filters

Frontend API (src/frontend/src/lib/api.js):

qsosAPI.getAll(filters): Fetch QSOs with optional filters
Filters passed as query parameters: ?band=20m&mode=CW&confirmationType=lotw&search=DL

DXCC Entity Priority Logic

When syncing QSOs from multiple confirmation sources, the system follows a priority order for DXCC entity data:

Priority Order: LoTW > DCL

Implementation (src/backend/services/dcl.service.js):

// DXCC priority: LoTW > DCL
// Only update entity fields from DCL if:
// 1. QSO is NOT LoTW confirmed, AND
// 2. DCL actually sent entity data, AND
// 3. Current entity is missing
const hasLoTWConfirmation = existingQSO.lotwQslRstatus === 'Y';
const hasDCLData = dbQSO.entity || dbQSO.entityId;
const missingEntity = !existingQSO.entity || existingQSO.entity === '';

if (!hasLoTWConfirmation && hasDCLData && missingEntity) {
  // Fill in entity data from DCL (only if DCL provides it)
  updateData.entity = dbQSO.entity;
  updateData.entityId = dbQSO.entityId;
  // ... other entity fields
}

Rules:

LoTW-confirmed QSOs: Always use LoTW's DXCC data (most reliable)
DCL-only QSOs: Use DCL's DXCC data IF available in ADIF payload
Empty entity fields: If DCL doesn't send DXCC data, entity remains empty
Never overwrite: Once LoTW confirms with entity data, DCL sync won't change it

Important Note: DCL API currently doesn't send DXCC/entity fields in their ADIF export. This is a limitation of the DCL API, not the application. If DCL adds these fields in the future, the system will automatically use them for DCL-only QSOs.

Recent Development Work (January 2025)

QSO Page Enhancements:

Added confirmation type filter with exclusive logic (LoTW Only, DCL Only, Both Confirmed, Not Confirmed)
Added search box for filtering by callsign, entity, or grid square
Renamed "All Confirmation" to "All QSOs" for clarity
Fixed filter logic to properly handle exclusive confirmation types

Bug Fixes:

Fixed confirmation filter showing wrong QSOs (e.g., "LoTW Only" was also showing DCL QSOs)
Implemented proper SQL conditions for exclusive filters using separate condition pushes
Added debug logging to track filter application

DXCC Entity Handling:

Clarified that DCL API doesn't send DXCC fields (current limitation)
Implemented priority logic: LoTW entity data takes precedence over DCL
System ready to auto-use DCL DXCC data if they add it in future API updates

22 KiB Raw Blame History