Joerg
223461f536
- Fix logger bug where debug level (0) was treated as falsy - Change `||` to `??` in config.js to properly handle log level 0 - Debug logs now work correctly when LOG_LEVEL=debug - Add server startup logging - Log port, environment, and log level on server start - Helps verify configuration is loaded correctly - Add DCL API request debug logging - Log full API request parameters when LOG_LEVEL=debug - API key is redacted (shows first/last 4 chars only) - Helps troubleshoot DCL sync issues - Update CLAUDE.md documentation - Add Logging section with log levels and configuration - Document debug logging feature for DCL service - Add this fix to Recent Commits section Note: .env file added locally with LOG_LEVEL=debug (not committed) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
366 lines
14 KiB
Markdown
366 lines
14 KiB
Markdown
|
|
Default to using Bun instead of Node.js.
|
|
|
|
- Use `bun <file>` instead of `node <file>` or `ts-node <file>`
|
|
- Use `bun test` instead of `jest` or `vitest`
|
|
- Use `bun build <file.html|file.ts|file.css>` instead of `webpack` or `esbuild`
|
|
- Use `bun install` instead of `npm install` or `yarn install` or `pnpm install`
|
|
- Use `bun run <script>` instead of `npm run <script>` or `yarn run <script>` or `pnpm run <script>`
|
|
- Use `bunx <package> <command>` instead of `npx <package> <command>`
|
|
- Bun automatically loads .env, so don't use dotenv.
|
|
|
|
## APIs
|
|
|
|
- `Bun.serve()` supports WebSockets, HTTPS, and routes. Don't use `express`.
|
|
- `bun:sqlite` for SQLite. Don't use `better-sqlite3`.
|
|
- `Bun.redis` for Redis. Don't use `ioredis`.
|
|
- `Bun.sql` for Postgres. Don't use `pg` or `postgres.js`.
|
|
- `WebSocket` is built-in. Don't use `ws`.
|
|
- Prefer `Bun.file` over `node:fs`'s readFile/writeFile
|
|
- Bun.$`ls` instead of execa.
|
|
|
|
## Logging
|
|
|
|
The application uses a custom logger in `src/backend/config.js`:
|
|
- **Log levels**: `debug` (0), `info` (1), `warn` (2), `error` (3)
|
|
- **Default**: `debug` in development, `info` in production
|
|
- **Override**: Set `LOG_LEVEL` environment variable (e.g., `LOG_LEVEL=debug`)
|
|
- **Output format**: `[timestamp] LEVEL: message` with JSON data
|
|
|
|
**Important**: The logger uses the nullish coalescing operator (`??`) to handle log levels. This ensures that `debug` (level 0) is not treated as falsy.
|
|
|
|
Example `.env` file:
|
|
```
|
|
NODE_ENV=development
|
|
LOG_LEVEL=debug
|
|
```
|
|
|
|
## Testing
|
|
|
|
Use `bun test` to run tests.
|
|
|
|
```ts#index.test.ts
|
|
import { test, expect } from "bun:test";
|
|
|
|
test("hello world", () => {
|
|
expect(1).toBe(1);
|
|
});
|
|
```
|
|
|
|
## Frontend
|
|
|
|
Use HTML imports with `Bun.serve()`. Don't use `vite`. HTML imports fully support React, CSS, Tailwind.
|
|
|
|
Server:
|
|
|
|
```ts#index.ts
|
|
import index from "./index.html"
|
|
|
|
Bun.serve({
|
|
routes: {
|
|
"/": index,
|
|
"/api/users/:id": {
|
|
GET: (req) => {
|
|
return new Response(JSON.stringify({ id: req.params.id }));
|
|
},
|
|
},
|
|
},
|
|
// optional websocket support
|
|
websocket: {
|
|
open: (ws) => {
|
|
ws.send("Hello, world!");
|
|
},
|
|
message: (ws, message) => {
|
|
ws.send(message);
|
|
},
|
|
close: (ws) => {
|
|
// handle close
|
|
}
|
|
},
|
|
development: {
|
|
hmr: true,
|
|
console: true,
|
|
}
|
|
})
|
|
```
|
|
|
|
HTML files can import .tsx, .jsx or .js files directly and Bun's bundler will transpile & bundle automatically. `<link>` tags can point to stylesheets and Bun's CSS bundler will bundle.
|
|
|
|
```html#index.html
|
|
<html>
|
|
<body>
|
|
<h1>Hello, world!</h1>
|
|
<script type="module" src="./frontend.tsx"></script>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
With the following `frontend.tsx`:
|
|
|
|
```tsx#frontend.tsx
|
|
import React from "react";
|
|
import { createRoot } from "react-dom/client";
|
|
|
|
// import .css files directly and it works
|
|
import './index.css';
|
|
|
|
const root = createRoot(document.body);
|
|
|
|
export default function Frontend() {
|
|
return <h1>Hello, world!</h1>;
|
|
}
|
|
|
|
root.render(<Frontend />);
|
|
```
|
|
|
|
Then, run index.ts
|
|
|
|
```sh
|
|
bun --hot ./index.ts
|
|
```
|
|
|
|
For more information, read the Bun API docs in `node_modules/bun-types/docs/**.mdx`.
|
|
|
|
## Project: Quickawards by DJ7NT
|
|
|
|
Quickawards is a amateur radio award tracking application that calculates progress toward various awards based on QSO (contact) data.
|
|
|
|
### Award System Architecture
|
|
|
|
The award system is JSON-driven and located in `award-definitions/` directory. Each award has:
|
|
- `id`: Unique identifier (e.g., "dld", "dxcc")
|
|
- `name`: Display name
|
|
- `description`: Short description
|
|
- `caption`: Detailed explanation
|
|
- `category`: Award category ("dxcc", "darc", etc.)
|
|
- `rules`: Award calculation logic
|
|
|
|
### Award Rule Types
|
|
|
|
1. **`entity`**: Count unique entities (DXCC countries, states, grid squares)
|
|
- `entityType`: What to count ("dxcc", "state", "grid", "callsign")
|
|
- `target`: Number required for award
|
|
- `filters`: Optional filters (band, mode, etc.)
|
|
- `displayField`: Optional field to display
|
|
|
|
2. **`dok`**: Count unique DOK (DARC Ortsverband Kennung) combinations
|
|
- `target`: Number required
|
|
- `confirmationType`: "dcl" (DARC Community Logbook)
|
|
- Counts unique (DOK, band, mode) combinations
|
|
- Only DCL-confirmed QSOs count
|
|
|
|
3. **`points`**: Point-based awards
|
|
- `stations`: Array of {callsign, points}
|
|
- `target`: Points required
|
|
- `countMode`: "perStation", "perBandMode", or "perQso"
|
|
|
|
4. **`filtered`**: Filtered version of another award
|
|
- `baseRule`: The base entity rule
|
|
- `filters`: Additional filters to apply
|
|
|
|
5. **`counter`**: Count QSOs or callsigns
|
|
|
|
### Key Files
|
|
|
|
**Backend Award Service**: `src/backend/services/awards.service.js`
|
|
- `getAllAwards()`: Returns all available award definitions
|
|
- `calculateAwardProgress(userId, award, options)`: Main calculation function
|
|
- `calculateDOKAwardProgress(userId, award, options)`: DOK-specific calculation
|
|
- `calculatePointsAwardProgress(userId, award, options)`: Point-based calculation
|
|
- `getAwardEntityBreakdown(userId, awardId)`: Detailed entity breakdown
|
|
- `getAwardProgressDetails(userId, awardId)`: Progress with details
|
|
|
|
**Database Schema**: `src/backend/db/schema/index.js`
|
|
- QSO fields include: `darcDok`, `dclQslRstatus`, `dclQslRdate`
|
|
- DOK fields support DLD award tracking
|
|
- DCL confirmation fields separate from LoTW
|
|
|
|
**Award Definitions**: `award-definitions/*.json`
|
|
- Add new awards by creating JSON definition files
|
|
- Add filename to `loadAwardDefinitions()` file list in awards.service.js
|
|
|
|
**ADIF Parser**: `src/backend/utils/adif-parser.js`
|
|
- `parseADIF(adifData)`: Parse ADIF format into QSO records
|
|
- `parseDCLResponse(response)`: Parse DCL's JSON response format `{ "adif": "..." }`
|
|
- `normalizeBand(band)`: Standardize band names (80m, 40m, etc.)
|
|
- `normalizeMode(mode)`: Standardize mode names (CW, FT8, SSB, etc.)
|
|
- Used by both LoTW and DCL services for consistency
|
|
|
|
**Job Queue Service**: `src/backend/services/job-queue.service.js`
|
|
- Manages async background jobs for LoTW and DCL sync
|
|
- `enqueueJob(userId, jobType)`: Queue a sync job ('lotw_sync' or 'dcl_sync')
|
|
- `processJobAsync(jobId, userId, jobType)`: Process job asynchronously
|
|
- `getUserActiveJob(userId, jobType)`: Get active job for user (optional type filter)
|
|
- `getJobStatus(jobId)`: Get job status with parsed result
|
|
- `updateJobProgress(jobId, progressData)`: Update job progress during processing
|
|
- Supports concurrent LoTW and DCL sync jobs
|
|
- Job types: 'lotw_sync', 'dcl_sync'
|
|
- Job status: 'pending', 'running', 'completed', 'failed'
|
|
|
|
**Backend API Routes** (`src/backend/index.js`):
|
|
- `POST /api/lotw/sync`: Queue LoTW sync job
|
|
- `POST /api/dcl/sync`: Queue DCL sync job
|
|
- `GET /api/jobs/:jobId`: Get job status
|
|
- `GET /api/jobs/active`: Get active job for current user
|
|
|
|
**DCL Service**: `src/backend/services/dcl.service.js`
|
|
- `fetchQSOsFromDCL(dclApiKey, sinceDate)`: Fetch from DCL API
|
|
- API Endpoint: `https://dings.dcl.darc.de/api/adiexport`
|
|
- Request: POST with JSON body `{ key, limit: 50000, qsl_since, qso_since, cnf_only }`
|
|
- `parseDCLJSONResponse(jsonResponse)`: Parse example/test payloads
|
|
- `syncQSOs(userId, dclApiKey, sinceDate, jobId)`: Sync QSOs to database
|
|
- `getLastDCLQSLDate(userId)`: Get last QSL date for incremental sync
|
|
- Debug logging (when `LOG_LEVEL=debug`) shows API params with redacted key (first/last 4 chars)
|
|
- Fully implemented and functional
|
|
- **Note**: DCL API is a custom prototype by DARC; contact DARC for API specification details
|
|
|
|
### DLD Award Implementation (COMPLETED)
|
|
|
|
The DLD (Deutschland Diplom) award was recently implemented:
|
|
|
|
**Definition**: `award-definitions/dld.json`
|
|
```json
|
|
{
|
|
"id": "dld",
|
|
"name": "DLD",
|
|
"description": "Deutschland Diplom - Confirm 100 unique DOKs on different bands/modes",
|
|
"caption": "Contact and confirm stations with 100 unique DOKs (DARC Ortsverband Kennung) on different band/mode combinations.",
|
|
"category": "darc",
|
|
"rules": {
|
|
"type": "dok",
|
|
"target": 100,
|
|
"confirmationType": "dcl",
|
|
"displayField": "darcDok"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Implementation Details**:
|
|
- Function: `calculateDOKAwardProgress()` in `src/backend/services/awards.service.js` (lines 173-268)
|
|
- Counts unique (DOK, band, mode) combinations
|
|
- Only DCL-confirmed QSOs count (`dclQslRstatus === 'Y'`)
|
|
- Each unique DOK on each unique band/mode counts separately
|
|
- Returns worked, confirmed counts and entity breakdowns
|
|
|
|
**Database Fields Used**:
|
|
- `darcDok`: DOK identifier (e.g., "F03", "P30", "G20")
|
|
- `band`: Band (e.g., "80m", "40m", "20m")
|
|
- `mode`: Mode (e.g., "CW", "SSB", "FT8")
|
|
- `dclQslRstatus`: DCL confirmation status ('Y' = confirmed)
|
|
- `dclQslRdate`: DCL confirmation date
|
|
|
|
**Documentation**: See `docs/DOCUMENTATION.md` for complete documentation including DLD award example.
|
|
|
|
**Frontend**: `src/frontend/src/routes/qsos/+page.svelte`
|
|
- Separate sync buttons for LoTW (blue) and DCL (orange)
|
|
- Independent progress tracking for each sync type
|
|
- Both syncs can run simultaneously
|
|
- Job polling every 2 seconds for status updates
|
|
- Import log displays after sync completion
|
|
- Real-time QSO table refresh after sync
|
|
|
|
**Frontend API** (`src/frontend/src/lib/api.js`):
|
|
- `qsosAPI.syncFromLoTW()`: Trigger LoTW sync
|
|
- `qsosAPI.syncFromDCL()`: Trigger DCL sync
|
|
- `jobsAPI.getStatus(jobId)`: Poll job status
|
|
- `jobsAPI.getActive()`: Get active job on page load
|
|
|
|
### Adding New Awards
|
|
|
|
To add a new award:
|
|
|
|
1. Create JSON definition in `award-definitions/`
|
|
2. Add filename to `loadAwardDefinitions()` in `src/backend/services/awards.service.js`
|
|
3. If new rule type needed, add calculation function
|
|
4. Add type handling in `calculateAwardProgress()` switch statement
|
|
5. Add type handling in `getAwardEntityBreakdown()` if needed
|
|
6. Update documentation in `docs/DOCUMENTATION.md`
|
|
7. Test with sample QSO data
|
|
|
|
### Confirmation Systems
|
|
|
|
- **LoTW (Logbook of The World)**: ARRL's confirmation system
|
|
- Service: `src/backend/services/lotw.service.js`
|
|
- API: `https://lotw.arrl.org/lotwuser/lotwreport.adi`
|
|
- Fields: `lotwQslRstatus`, `lotwQslRdate`
|
|
- Used for DXCC, WAS, VUCC, most awards
|
|
- ADIF format with `<EOR>` delimiters
|
|
- Supports incremental sync by `qso_qslsince` parameter (format: YYYY-MM-DD)
|
|
|
|
- **DCL (DARC Community Logbook)**: DARC's confirmation system
|
|
- Service: `src/backend/services/dcl.service.js`
|
|
- API: `https://dings.dcl.darc.de/api/adiexport`
|
|
- Fields: `dclQslRstatus`, `dclQslRdate`
|
|
- DOK fields: `darcDok` (partner's DOK), `myDarcDok` (user's DOK)
|
|
- Required for DLD award
|
|
- German amateur radio specific
|
|
- Request format: POST JSON `{ key, limit, qsl_since, qso_since, cnf_only }`
|
|
- Response format: JSON with ADIF string in `adif` field
|
|
- Supports incremental sync by `qsl_since` parameter (format: YYYYMMDD)
|
|
- Updates QSOs only if confirmation data has changed
|
|
|
|
### ADIF Format
|
|
|
|
Both LoTW and DCL return data in ADIF (Amateur Data Interchange Format):
|
|
- Field format: `<FIELD_NAME:length>value`
|
|
- Record delimiter: `<EOR>` (end of record)
|
|
- Header ends with: `<EOH>` (end of header)
|
|
- Example: `<CALL:5>DK0MU<BAND:3>80m<QSO_DATE:8>20250621<EOR>`
|
|
|
|
**DCL-specific fields**:
|
|
- `DCL_QSL_RCVD`: DCL confirmation status (Y/N/?)
|
|
- `DCL_QSLRDATE`: DCL confirmation date (YYYYMMDD)
|
|
- `DARC_DOK`: QSO partner's DOK
|
|
- `MY_DARC_DOK`: User's own DOK
|
|
- `STATION_CALLSIGN`: User's callsign
|
|
|
|
### Recent Commits
|
|
|
|
- **Uncommitted**: fix: logger debug level not working
|
|
- Fixed bug where debug logs weren't showing due to falsy value handling
|
|
- Changed `||` to `??` in logger config to properly handle log level 0 (debug)
|
|
- Added `.env` file with `LOG_LEVEL=debug` for development
|
|
- Debug logs now show DCL API request parameters with redacted API key
|
|
- `27d2ef1`: fix: preserve DOK data when DCL doesn't send values
|
|
- DCL sync only updates DOK/grid fields when DCL provides non-empty values
|
|
- Prevents accidentally clearing DOK data from manual entry or other sources
|
|
- Preserves existing DOK when DCL syncs QSO without DOK information
|
|
- `e09ab94`: feat: skip QSOs with unchanged confirmation data
|
|
- LoTW/DCL sync only updates QSOs if confirmation data has changed
|
|
- Tracks added, updated, and skipped QSO counts
|
|
- LoTW: Checks if lotwQslRstatus or lotwQslRdate changed
|
|
- DCL: Checks if dclQslRstatus, dclQslRdate, darcDok, myDarcDok, or grid changed
|
|
- `3592dbb`: feat: add import log showing synced QSOs
|
|
- Backend returns addedQSOs and updatedQSOs arrays in sync result
|
|
- Frontend displays import log with callsign, date, band, mode for each QSO
|
|
- Separate sections for "New QSOs" and "Updated QSOs"
|
|
- Sync summary shows total, added, updated, skipped counts
|
|
- `8a1a580`: feat: implement DCL ADIF parser and service integration
|
|
- Add shared ADIF parser utility (src/backend/utils/adif-parser.js)
|
|
- Implement DCL service with API integration
|
|
- Refactor LoTW service to use shared parser
|
|
- Tested with example DCL payload (6 QSOs parsed successfully)
|
|
- `c982dcd`: feat: implement DLD (Deutschland Diplom) award
|
|
- `322ccaf`: docs: add DLD (Deutschland Diplom) award documentation
|
|
|
|
### Sync Behavior
|
|
|
|
**Import Log**: After each sync, displays a table showing:
|
|
- New QSOs: Callsign, Date, Band, Mode
|
|
- Updated QSOs: Callsign, Date, Band, Mode (only if data changed)
|
|
- Skipped QSOs: Counted but not shown (data unchanged)
|
|
|
|
**Duplicate Handling**:
|
|
- QSOs matched by: userId, callsign, qsoDate, timeOn, band, mode
|
|
- If confirmation data unchanged: Skipped (not updated)
|
|
- If confirmation data changed: Updated with new values
|
|
- Prevents unnecessary database writes and shows accurate import counts
|
|
|
|
**DOK Update Behavior**:
|
|
- If QSO imported via LoTW (no DOK) and later DCL confirms with DOK: DOK is added ✓
|
|
- If QSO already has DOK and DCL sends different DOK: DOK is updated ✓
|
|
- If QSO has DOK and DCL syncs without DOK (empty): Existing DOK is preserved ✓
|
|
- LoTW never sends DOK data; only DCL provides DOK fields
|
|
|
|
**Important**: DCL sync only updates DOK/grid fields when DCL provides non-empty values. This prevents accidentally clearing DOK data that was manually entered or imported from other sources.
|