dj7nt/award
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity
Files
b09e2b3ea297ba26247bfc2a6a119303d7b8a17f
award/CLAUDE.md
Joerg 85d171adc8 docs: document mode groups feature in CLAUDE.md and README.md 
Add documentation for the new configurable mode groups feature:
- CLAUDE.md: Add modeGroups to Award Rule Options section
- CLAUDE.md: Update Award Detail View section with mode group info
- CLAUDE.md: Add to Recent Development Work (January 2026)
- README.md: Add GET /api/awards/:awardId endpoint
- README.md: Add new Mode Groups section in Features in Detail

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-23 07:19:49 +01:00

556 lines
21 KiB
Markdown
Raw Blame History

Default to using Bun instead of Node.js.
- Use `bun <file>` instead of `node <file>` or `ts-node <file>`
- Use `bun test` instead of `jest` or `vitest`
- Use `bun build <file.html|file.ts|file.css>` instead of `webpack` or `esbuild`
- Use `bun install` instead of `npm install` or `yarn install` or `pnpm install`
- Use `bun run <script>` instead of `npm run <script>` or `yarn run <script>` or `pnpm run <script>`
- Use `bunx <package> <command>` instead of `npx <package> <command>`
- Bun automatically loads .env, so don't use dotenv.
## APIs
- `Bun.serve()` supports WebSockets, HTTPS, and routes. Don't use `express`.
- `bun:sqlite` for SQLite. Don't use `better-sqlite3`.
- `Bun.redis` for Redis. Don't use `ioredis`.
- `Bun.sql` for Postgres. Don't use `pg` or `postgres.js`.
- `WebSocket` is built-in. Don't use `ws`.
- Prefer `Bun.file` over `node:fs`'s readFile/writeFile
- Bun.$`ls` instead of execa.
## Logging
The application uses a custom logger that outputs to both files and console.
### Backend Logging
Backend logs are written to `logs/backend.log`:
- **Log levels**: `debug` (0), `info` (1), `warn` (2), `error` (3)
- **Default**: `debug` in development, `info` in production
- **Override**: Set `LOG_LEVEL` environment variable (e.g., `LOG_LEVEL=debug`)
- **Output format**: `[timestamp] LEVEL: message` with JSON data
- **Console**: Also outputs to console in development mode
- **File**: Always writes to `logs/backend.log`
### Frontend Logging
Frontend logs are sent to the backend and written to `logs/frontend.log`:
- **Logger**: `src/frontend/src/lib/logger.js`
- **Endpoint**: `POST /api/logs`
- **Batching**: Batches logs (up to 10 entries or 5 seconds) for performance
- **User context**: Automatically includes userId and user-agent
- **Levels**: Same as backend (debug, info, warn, error)
**Usage in frontend**:
```javascript
import { logger } from '$lib/logger';
logger.info('User action', { action: 'click', element: 'button' });
logger.error('API error', { error: err.message });
logger.warn('Deprecated feature used');
logger.debug('Component state', { state: componentState });
```
**Important**: The logger uses the nullish coalescing operator (`??`) to handle log levels. This ensures that `debug` (level 0) is not treated as falsy.
Example `.env` file:
```
NODE_ENV=development
LOG_LEVEL=debug
```
**Log Files**:
- `logs/backend.log` - Backend server logs
- `logs/frontend.log` - Frontend client logs
- Logs are excluded from git via `.gitignore`
## Testing
Use `bun test` to run tests.
```ts#index.test.ts
import { test, expect } from "bun:test";
test("hello world", () => {
expect(1).toBe(1);
});
```
## Frontend
Use HTML imports with `Bun.serve()`. Don't use `vite`. HTML imports fully support React, CSS, Tailwind.
Server:
```ts#index.ts
import index from "./index.html"
Bun.serve({
routes: {
"/": index,
"/api/users/:id": {
GET: (req) => {
return new Response(JSON.stringify({ id: req.params.id }));
},
},
},
// optional websocket support
websocket: {
open: (ws) => {
ws.send("Hello, world!");
},
message: (ws, message) => {
ws.send(message);
},
close: (ws) => {
// handle close
}
},
development: {
hmr: true,
console: true,
}
})
```
HTML files can import .tsx, .jsx or .js files directly and Bun's bundler will transpile & bundle automatically. `<link>` tags can point to stylesheets and Bun's CSS bundler will bundle.
```html#index.html
<html>
<body>
<h1>Hello, world!</h1>
<script type="module" src="./frontend.tsx"></script>
</body>
</html>
```
With the following `frontend.tsx`:
```tsx#frontend.tsx
import React from "react";
import { createRoot } from "react-dom/client";
// import .css files directly and it works
import './index.css';
const root = createRoot(document.body);
export default function Frontend() {
return <h1>Hello, world!</h1>;
}
root.render(<Frontend />);
```
Then, run index.ts
```sh
bun --hot ./index.ts
```
For more information, read the Bun API docs in `node_modules/bun-types/docs/**.mdx`.
## Project: Quickawards by DJ7NT
Quickawards is a amateur radio award tracking application that calculates progress toward various awards based on QSO (contact) data.
### Award System Architecture
The award system is JSON-driven and located in `award-definitions/` directory. Each award has:
- `id`: Unique identifier (e.g., "dld", "dxcc")
- `name`: Display name
- `description`: Short description
- `caption`: Detailed explanation
- `category`: Award category ("dxcc", "darc", etc.)
- `rules`: Award calculation logic
### Award Rule Types
1. **`entity`**: Count unique entities (DXCC countries, states, grid squares)
- `entityType`: What to count ("dxcc", "state", "grid", "callsign")
- `target`: Number required for award
- `allowed_bands`: Optional array of bands that count (e.g., `["160m", "80m", "40m", "30m", "20m", "17m", "15m", "12m", "10m"]` for HF only)
- `satellite_only`: Optional boolean to only count satellite QSOs (QSOs with `satName` field)
- `filters`: Optional filters (band, mode, etc.)
- `displayField`: Optional field to display
2. **`dok`**: Count unique DOK (DARC Ortsverband Kennung) combinations
- `target`: Number required
- `confirmationType`: "dcl" (DARC Community Logbook)
- `filters`: Optional filters (band, mode, etc.) for award variants
- Counts unique (DOK, band, mode) combinations
- Only DCL-confirmed QSOs count
3. **`points`**: Point-based awards
- `stations`: Array of {callsign, points}
- `target`: Points required
- `countMode`: "perStation", "perBandMode", or "perQso"
4. **`filtered`**: Filtered version of another award
- `baseRule`: The base entity rule
- `filters`: Additional filters to apply
5. **`counter`**: Count QSOs or callsigns
### Current Awards
- **DXCC**: HF bands only (160m-10m), 100 entities required
- **DXCC SAT**: Satellite QSOs only, 100 entities required
- **WAS**: Worked All States award
- **VUCC SAT**: VUCC Satellite award
- **SAT-RS44**: Special satellite award
- **73 on 73**: Special stations award
- **DLD**: Deutschland Diplom, 100 unique DOKs required
### Key Files
**Backend Award Service**: `src/backend/services/awards.service.js`
- `getAllAwards()`: Returns all available award definitions
- `calculateAwardProgress(userId, award, options)`: Main calculation function
- `calculateDOKAwardProgress(userId, award, options)`: DOK-specific calculation
- `calculatePointsAwardProgress(userId, award, options)`: Point-based calculation
- `getAwardEntityBreakdown(userId, awardId)`: Detailed entity breakdown
- `getAwardProgressDetails(userId, awardId)`: Progress with details
- Implements `allowed_bands` and `satellite_only` filtering
**Database Schema**: `src/backend/db/schema/index.js`
- QSO fields include: `darcDok`, `dclQslRstatus`, `dclQslRdate`, `satName`
- DOK fields support DLD award tracking
- DCL confirmation fields separate from LoTW
- `satName` field for satellite QSO tracking
**Award Definitions**: `award-definitions/*.json`
- Add new awards by creating JSON definition files
- Add filename to `loadAwardDefinitions()` file list in awards.service.js
**ADIF Parser**: `src/backend/utils/adif-parser.js`
- `parseADIF(adifData)`: Parse ADIF format into QSO records
- Handles case-insensitive `<EOR>` delimiters (supports `<EOR>`, `<eor>`, `<Eor>`)
- Uses `matchAll()` for reliable field parsing
- Skips header records automatically
- `normalizeBand(band)`: Standardize band names (80m, 40m, etc.)
- `normalizeMode(mode)`: Standardize mode names (CW, FT8, SSB, etc.)
- Used by both LoTW and DCL services for consistency
**Job Queue Service**: `src/backend/services/job-queue.service.js`
- Manages async background jobs for LoTW and DCL sync
- `enqueueJob(userId, jobType)`: Queue a sync job ('lotw_sync' or 'dcl_sync')
- `processJobAsync(jobId, userId, jobType)`: Process job asynchronously
- `getUserActiveJob(userId, jobType)`: Get active job for user (optional type filter)
- `getJobStatus(jobId)`: Get job status with parsed result
- `updateJobProgress(jobId, progressData)`: Update job progress during processing
- Supports concurrent LoTW and DCL sync jobs
- Job types: 'lotw_sync', 'dcl_sync'
- Job status: 'pending', 'running', 'completed', 'failed'
**Backend API Routes** (`src/backend/index.js`):
- `POST /api/lotw/sync`: Queue LoTW sync job
- `POST /api/dcl/sync`: Queue DCL sync job
- `GET /api/jobs/:jobId`: Get job status
- `GET /api/jobs/active`: Get active job for current user
- `DELETE /api/qsos/all`: Delete all QSOs for authenticated user
- `GET /*`: Serves static files from `src/frontend/build/` with SPA fallback
**SPA Routing**: The backend serves the SvelteKit frontend build from `src/frontend/build/`.
- Paths with file extensions (`.js`, `.css`, etc.) are served as static files
- Paths without extensions (e.g., `/qsos`, `/awards`) are served `index.html` for client-side routing
- Common missing files like `/favicon.ico` return 404 immediately
- If frontend build is missing entirely, returns a user-friendly 503 HTML page
- Prevents ugly Bun error pages when accessing client-side routes via curl or non-JS clients
**DCL Service**: `src/backend/services/dcl.service.js`
- `fetchQSOsFromDCL(dclApiKey, sinceDate)`: Fetch from DCL API
- API Endpoint: `https://dings.dcl.darc.de/api/adiexport`
- Request: POST with JSON body `{ key, limit: 50000, qsl_since, qso_since, cnf_only }`
- `cnf_only: null` - Fetch ALL QSOs (confirmed + unconfirmed)
- `cnf_only: true` - Fetch only confirmed QSOs (dcl_qsl_rcvd='Y')
- `qso_since: DATE` - QSOs since this date (YYYYMMDD format)
- `qsl_since: DATE` - QSL confirmations since this date (YYYYMMDD format)
- `parseDCLJSONResponse(jsonResponse)`: Parse example/test payloads
- `syncQSOs(userId, dclApiKey, sinceDate, jobId)`: Sync QSOs to database
- `getLastDCLQSLDate(userId)`: Get last QSL date for incremental sync
- `getLastDCLQSODate(userId)`: Get last QSO date for incremental sync
- Debug logging (when `LOG_LEVEL=debug`) shows API params with redacted key (first/last 4 chars)
- Fully implemented and functional
- **Note**: DCL API is a custom prototype by DARC; contact DARC for API specification details
### DLD Award Implementation
The DLD (Deutschland Diplom) award:
**Definition**: `award-definitions/dld.json`
```json
{
"id": "dld",
"name": "DLD",
"description": "Deutschland Diplom - Confirm 100 unique DOKs on different bands/modes",
"caption": "Contact and confirm stations with 100 unique DOKs (DARC Ortsverband Kennung) on different band/mode combinations.",
"category": "darc",
"rules": {
"type": "dok",
"target": 100,
"confirmationType": "dcl",
"displayField": "darcDok"
}
}
```
**Implementation Details**:
- Function: `calculateDOKAwardProgress()` in `src/backend/services/awards.service.js`
- Counts unique (DOK, band, mode) combinations
- Only DCL-confirmed QSOs count (`dclQslRstatus === 'Y'`)
- Each unique DOK on each unique band/mode counts separately
- Returns worked, confirmed counts and entity breakdowns
**Database Fields Used**:
- `darcDok`: DOK identifier (e.g., "F03", "P30", "G20")
- `band`: Band (e.g., "80m", "40m", "20m")
- `mode`: Mode (e.g., "CW", "SSB", "FT8")
- `dclQslRstatus`: DCL confirmation status ('Y' = confirmed)
- `dclQslRdate`: DCL confirmation date
**Frontend**: `src/frontend/src/routes/qsos/+page.svelte`
- Separate sync buttons for LoTW (blue) and DCL (orange)
- Independent progress tracking for each sync type
- Both syncs can run simultaneously
- Job polling every 2 seconds for status updates
- Import log displays after sync completion
- Real-time QSO table refresh after sync
**Frontend API** (`src/frontend/src/lib/api.js`):
- `qsosAPI.syncFromLoTW()`: Trigger LoTW sync
- `qsosAPI.syncFromDCL()`: Trigger DCL sync
- `jobsAPI.getStatus(jobId)`: Poll job status
- `jobsAPI.getActive()`: Get active job on page load
### Adding New Awards
To add a new award:
1. Create JSON definition in `award-definitions/`
2. Add filename to `loadAwardDefinitions()` in `src/backend/services/awards.service.js`
3. If new rule type needed, add calculation function
4. Add type handling in `calculateAwardProgress()` switch statement
5. Add type handling in `getAwardEntityBreakdown()` if needed
6. Update documentation
7. Test with sample QSO data
### Award Rule Options
**allowed_bands**: Restrict which bands count toward an award
```json
{
"rules": {
"type": "entity",
"allowed_bands": ["160m", "80m", "60m", "40m", "30m", "20m", "17m", "15m", "12m", "10m"]
}
}
```
- If absent or empty, all bands are allowed (default behavior)
- Used for DXCC to restrict to HF bands only
**satellite_only**: Only count satellite QSOs
```json
{
"rules": {
"type": "entity",
"satellite_only": true
}
}
```
- If `true`, only QSOs with `satName` field set are counted
- Used for DXCC SAT award
**modeGroups**: Define mode groups for filtering in award detail view
```json
{
"modeGroups": {
"Digi-Modes": ["FT8", "FT4", "MFSK", "PSK31", "RTTY", "JT65", "JT9"],
"Classic Digi-Modes": ["PSK31", "RTTY", "JT65", "JT9"],
"Mixed-Mode w/o WSJT-Modes": ["PSK31", "RTTY", "AM", "SSB", "FM", "CW"],
"Phone-Modes": ["AM", "SSB", "FM"]
}
}
```
- Optional field at award definition level (not in `rules`)
- Key is the display name shown in the mode filter dropdown
- Value is an array of mode strings to include in the group
- Used to create convenient mode filters that combine multiple modes
- Awards without `modeGroups` work as before (backward compatible)
**filters**: Additional filtering options
- `eq`: equals
- `ne`: not equals
- `in`: in array
- `nin`: not in array
- `contains`: contains substring
- Can filter any QSO field (band, mode, callsign, grid, state, etc.)
### Confirmation Systems
- **LoTW (Logbook of The World)**: ARRL's confirmation system
- Service: `src/backend/services/lotw.service.js`
- API: `https://lotw.arrl.org/lotwuser/lotwreport.adi`
- Fields: `lotwQslRstatus`, `lotwQslRdate`
- Used for DXCC, WAS, VUCC, most awards
- ADIF format with `<EOR>` delimiters
- Supports incremental sync by `qso_qslsince` parameter (format: YYYY-MM-DD)
- **DCL (DARC Community Logbook)**: DARC's confirmation system
- Service: `src/backend/services/dcl.service.js`
- API: `https://dings.dcl.darc.de/api/adiexport`
- Fields: `dclQslRstatus`, `dclQslRdate`
- DOK fields: `darcDok` (partner's DOK), `myDarcDok` (user's DOK)
- Required for DLD award
- German amateur radio specific
- Request format: POST JSON `{ key, limit, qsl_since, qso_since, cnf_only }`
- Response format: JSON with ADIF string in `adif` field
- Syncs ALL QSOs (both confirmed and unconfirmed)
- Updates QSOs only if confirmation data has changed
### ADIF Format
Both LoTW and DCL return data in ADIF (Amateur Data Interchange Format):
- Field format: `<FIELD_NAME:length>value`
- Record delimiter: `<EOR>` (end of record, case-insensitive)
- Header ends with: `<EOH>` (end of header)
- Example: `<CALL:5>DK0MU<BAND:3>80m<QSO_DATE:8>20250621<EOR>`
- **Important**: Parser handles case-insensitive `<EOR>`, `<eor>`, `<Eor>` tags
**DCL-specific fields**:
- `DCL_QSL_RCVD`: DCL confirmation status (Y/N/?)
- `DCL_QSLRDATE`: DCL confirmation date (YYYYMMDD)
- `DARC_DOK`: QSO partner's DOK
- `MY_DARC_DOK`: User's own DOK
- `STATION_CALLSIGN`: User's callsign
### QSO Management
**Delete All QSOs**: `DELETE /api/qsos/all`
- Deletes all QSOs for authenticated user
- Also deletes related `qso_changes` records to satisfy foreign key constraints
- Invalidates stats and user caches after deletion
- Returns count of deleted QSOs
### QSO Page Filters
The QSO page (`src/frontend/src/routes/qsos/+page.svelte`) includes advanced filtering capabilities:
**Available Filters**:
- **Search Box**: Full-text search across callsign, entity (DXCC country), and grid square fields
- **Band Filter**: Dropdown to filter by amateur band (160m, 80m, 60m, 40m, 30m, 20m, 17m, 15m, 12m, 10m, 6m, 2m, 70cm)
- **Mode Filter**: Dropdown to filter by mode (CW, SSB, AM, FM, RTTY, PSK31, FT8, FT4, JT65, JT9)
- **Confirmation Type Filter**: Filter by confirmation status
- "All QSOs", "LoTW Only", "DCL Only", "Both Confirmed", "Not Confirmed"
- **Clear Button**: Resets all filters
**Backend Implementation** (`src/backend/services/lotw.service.js`):
- `getUserQSOs(userId, filters, options)`: Main filtering function
- Supports pagination with `page` and `limit` options
- Filter logic uses Drizzle ORM query builders for safe SQL generation
**Frontend API** (`src/frontend/src/lib/api.js`):
- `qsosAPI.getAll(filters)`: Fetch QSOs with optional filters
- Filters passed as query parameters: `?band=20m&mode=CW&confirmationType=lotw&search=DL`
### Award Detail View
**Overview**: The award detail page (`src/frontend/src/routes/awards/[id]/+page.svelte`) displays award progress in a pivot table format.
**Key Features**:
- **Summary Cards**: Show total, confirmed, worked, needed counts for unique entities
- **Mode Filter**: Filter by specific mode, mode group, or view "Mixed Mode" (aggregates all modes by band)
- Awards can define `modeGroups` to create convenient multi-mode filters
- Example groups: "Digi-Modes", "Classic Digi-Modes", "Phone-Modes", "Mixed-Mode w/o WSJT-Modes"
- Visual separator (`─────`) appears between mode groups and individual modes
- **Table Columns**: Show bands (or band/mode combinations) as columns
- **QSO Counts**: Each cell shows count of confirmed QSOs for that (entity, band, mode) slot
- **Drill-Down**: Click a count to open modal showing all QSOs for that slot
- **QSO Detail**: Click any QSO to view full QSO details
- **Satellite Grouping**: Satellite QSOs grouped under "SAT" column instead of frequency band
**Column Sorting**: Bands sorted by wavelength (longest to shortest):
160m, 80m, 60m, 40m, 30m, 20m, 17m, 15m, 12m, 10m, 6m, 2m, 70cm, SAT
**Column Sums**: Show unique entity count per column (not QSO counts)
**Backend Changes** (`src/backend/services/awards.service.js`):
- `getAllAwards()`: Returns award definitions including `modeGroups`
- `getAwardById(awardId)`: Returns single award definition with `modeGroups`
- `calculateDOKAwardProgress()`: Groups by (DOK, band, mode) slots, collects QSOs in `qsos` array
- `calculatePointsAwardProgress()`: Handles all count modes with `qsos` array
- `getAwardEntityBreakdown()`: Groups by (entity, band, mode) slots
- Includes `satName` in QSO data for satellite grouping
- Implements `allowed_bands` and `satellite_only` filtering
### DXCC Entity Priority Logic
When syncing QSOs from multiple confirmation sources, the system follows a priority order for DXCC entity data:
**Priority Order**: LoTW > DCL
**Rules**:
1. **LoTW-confirmed QSOs**: Always use LoTW's DXCC data (most reliable)
2. **DCL-only QSOs**: Use DCL's DXCC data IF available in ADIF payload
3. **Empty entity fields**: If DCL doesn't send DXCC data, entity remains empty
4. **Never overwrite**: Once LoTW confirms with entity data, DCL sync won't change it
**Important Note**: DCL API currently doesn't send DXCC/entity fields in their ADIF export.
### Critical LoTW Sync Behavior
**⚠️ IMPORTANT: LoTW sync MUST only import confirmed QSOs**
LoTW ADIF export with `qso_qsl=no` (all QSOs mode) only includes:
- `CALL` (callsign)
- `QSL_RCVD` (confirmation status: Y/N)
**Missing Fields for Unconfirmed QSOs:**
- `DXCC` (entity ID) ← **CRITICAL for awards!**
- `COUNTRY` (entity name)
- `CONTINENT`, `CQ_ZONE`, `ITU_ZONE`
**Result:** Unconfirmed QSOs have `entityId: null` and `entity: ""`, breaking award calculations.
**Current Implementation (CORRECT):**
```javascript
// lotw.service.js - fetchQSOsFromLoTW()
const params = new URLSearchParams({
login: lotwUsername,
password: loTWPassword,
qso_query: '1',
qso_qsl: 'yes', // ONLY confirmed QSOs
qso_qslsince: dateStr, // Incremental sync
});
```
### Recent Development Work (January 2026)
**Award System Enhancements**:
- Added `allowed_bands` filter to restrict which bands count toward awards
- Added `satellite_only` flag for satellite-only awards
- DXCC restricted to HF bands (160m-10m) only
- Added DXCC SAT award for satellite-only QSOs
- Removed redundant award variants (DXCC CW, DLD variants)
- Added `modeGroups` for configurable multi-mode filters in award detail view
- Per-award configuration of mode groups (Digi-Modes, Phone-Modes, etc.)
- Visual separator in mode filter dropdown between groups and individual modes
- DXCC and DLD awards include: Digi-Modes, Classic Digi-Modes, Mixed-Mode w/o WSJT-Modes, Phone-Modes
**Award Detail View Improvements**:
- Summary shows unique entity progress instead of QSO counts
- Column sums count unique entities per column
- Satellite QSOs grouped under "SAT" column
- Bands sorted by wavelength instead of alphabetically
- Mode removed from table headers (visible in filter dropdown)
- Mode groups allow filtering multiple modes together (e.g., all digital modes)
**Backend API Additions**:
- Added `GET /api/awards/:awardId` endpoint for fetching single award definition
- `getAllAwards()` now includes `modeGroups` field
**QSO Management**:
- Fixed DELETE /api/qsos/all to handle foreign key constraints
- Added cache invalidation after QSO deletion
Reference in New Issue View Git Blame Copy Permalink