Commit Graph
100 Commits
Author SHA1 Message Date
ask 6ee1146173 fix(chdb): scan ClickHouse log_scores into signed model fields
ci/woodpecker/push/woodpecker Pipeline was successful
The PostgreSQL model migration regenerated LogScore.ID, MonitorID and
ServerID as signed/pgtype fields, but the ClickHouse log_scores columns
are unsigned (id UInt64, monitor_id/server_id UInt32). clickhouse-go
refuses to scan UInt64 into *int64, so every score row failed to parse
and the scores endpoints returned empty.

Scan each row into locals matching the ClickHouse column types, then
convert into the model, mirroring the existing leap handling. Factor the
shared scan into scanLogScore used by both Logscores and
LogscoresTimeRange.
2026-05-23 23:52:09 -07:00
ask ab2c853fd7 feat(db): migrate from MySQL to PostgreSQL
ci/woodpecker/push/woodpecker Pipeline was successful
Replace MySQL driver with pgx/v5 and pgxpool:
- Update sqlc to use postgresql engine
- Convert query.sql to PostgreSQL syntax ($1 params, CASE WHEN,
  ANY() arrays)
- Replace sql.DB with pgxpool.Pool throughout
- Change nullable types from sql.Null* to pgtype.*
- Update ID types from uint32 to int64 for PostgreSQL compatibility
- Delete MySQL-specific dynamic_connect.go
- Add opentelemetry.gowrap template for tracing
2026-05-24 01:51:42 -07:00
ask 4834d2153a build(ci): pin docker-tags-plugin to 0.2.1-abh
ci/woodpecker/push/woodpecker Pipeline was successful
2026-05-24 01:34:37 -07:00
ask 37204a5dda fix(cors): correct malformed askdev wildcard origin
The askdev origin pattern was missing a slash (https:/* instead of
https://*), so requests from *.askdev.grundclock.com never matched the
CORS allowlist and got no Access-Control-Allow-Origin header.
2026-05-24 01:34:12 -07:00
ask bf27d87891 fix: support /api/data/ path prefix as ingress workaround
ci/woodpecker/push/woodpecker Pipeline was successful
Requests from the web frontend use /api/data/... paths but the
data-api routes are registered under /api/.... Add a Pre middleware
rewrite rule to strip the /data segment before route matching.
2026-03-18 23:06:05 -07:00
ask 2a3429b594 fix(chdb): add connection name to ClickHouse error messages
ci/woodpecker/push/woodpecker Pipeline was successful
ci/woodpecker/tag/woodpecker Pipeline was successful
Wrap logs/scores connection errors with their name so it's
clear which connection failed on startup.
2026-03-07 22:10:05 -08:00
ask 7d66b87387 build: use Alpine 3.23 2026-03-07 21:25:04 -08:00
ask c9b8348ead fix(ci): use milliCPU units for goreleaser resource requests
ci/woodpecker/push/woodpecker Pipeline was successful
CPU values 6000/10000 were interpreted as whole cores (not millicores),
making the pod unschedulable on any node.
2026-03-07 21:17:03 -08:00
ask 27add1fb3d refactor: move config import from api to common module
Replace go.ntppool.org/api/config with go.ntppool.org/common/config,
removing the api module dependency entirely.

Also ignore vendor/ directory in .gitignore.
2026-03-07 21:16:11 -08:00
ask 5695ebb3fd fix(ci): use environment for secrets in v2
ci/woodpecker/push/woodpecker Pipeline failed
Replace deprecated `secrets` keyword with
`environment`/`from_secret` syntax for
Woodpecker v2 compatibility.
2026-03-07 20:55:21 -08:00
ask 989b20fedd fix(ci): add vendor step for private Go module authentication
Add a vendor step that uses gitea_token to fetch private modules
from gitea.develooper.com, then subsequent steps use the vendored
dependencies without needing credentials.
2026-03-07 17:09:31 -08:00
ask 0dfa41da8e build(ci): migrate from Drone to Woodpecker
ci/woodpecker/tag/woodpecker Pipeline failed
Replace .drone.yml with .woodpecker.yaml and update
scripts/run-goreleaser to use CI_COMMIT_TAG instead
of DRONE_TAG.
2026-03-07 16:18:53 -08:00
ask e4f6d8cafb fix(chdb): rename geodns references to dns
continuous-integration/drone/push Build is passing
ClickHouse DNS tables moved from geodns/geodns3 to
a single dns database.
2026-03-07 16:07:20 -08:00
ask 1b1413a632 build: Use go 1.26
continuous-integration/drone/push Build is passing
2026-03-07 16:05:28 -08:00
ask 85d86bc837 build: update go and dependencies
continuous-integration/drone/push Build is passing
2025-09-27 08:17:04 -07:00
ask 196f90a2b9 fix(db): use int for netspeed_active to prevent overflow
continuous-integration/drone/push Build is passing
GetZoneStatsData and GetZoneStatsV2's netspeed_active values can
exceed 2 billion, causing 32-bit integer overflow. Changed from
int32/uint32 to int (64-bit on modern systems) to handle large
network speed totals.

- Update sqlc column overrides to use int type
- Fix type compatibility in dnsanswers.go zoneTotals map
- Regenerate database code with new types

Fixes https://community.ntppool.org/t/error-message-displayed-on-the-monitoring-score-page/4063
2025-09-21 00:08:21 -07:00
ask 02a6f587bb Update schema 2025-09-20 10:29:53 -07:00
ask 2dfc355f7c style: format Go code with gofumpt
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
Apply consistent formatting to Go source files using gofumpt
as required by pre-commit guidelines.
2025-08-03 16:06:59 -07:00
ask 3e6a0f9e63 fix(api): include deleted monitors in name-based lookups
Remove status filter from GetMonitorByNameAndIPVersion query to allow
historical score data for deleted monitors to be accessible when
querying by monitor name/TLS name, making behavior consistent with
ID-based queries.
2025-08-03 14:53:21 -07:00
ask 9c6b8d1867 fix(api): handle score monitors in name-based lookups
Score monitors have type='score' and ip_version=NULL, but the
GetMonitorByNameAndIPVersion query required ip_version to match.
This broke monitor lookups by name for score monitors.

Modified query to match either:
- Regular monitors with specified ip_version
- Score monitors with NULL ip_version

Fixes issue reported by Ben Harris at:
https://community.ntppool.org/t/monitor-recentmedian-no-longer-works/4002
2025-08-04 20:43:53 -07:00
ask 393d532ce2 feat(api): add relative time support to v2 scores endpoint
- Add parseRelativeTime function supporting "-3d", "-2h", "-30m" format
- Update parseTimeRangeParams to handle Unix timestamps and relative times
- Add unit tests with comprehensive coverage for all time formats
- Document v2 API in API.md with examples and migration guide

Enables intuitive time queries like from=-3d&to=-1h instead of
Unix timestamps, improving developer experience for the enhanced
v2 endpoint that supports 50k records vs legacy 10k limit.
2025-08-03 12:12:22 -07:00
ask 267c279f3d Update dependencies
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-08-02 19:48:42 -07:00
ask eb5459abf3 fix(api): protocol-aware monitor filtering for multi-protocol monitors
continuous-integration/drone/push Build is passing
Servers with monitor filtering returned incorrect results when monitors
have same names but different protocols (v4/v6). Monitor lookup now
considers both name and IP version to match the correct protocol.

- Add GetMonitorByNameAndIPVersion SQL query with protocol matching
- Update history parameter parsing to use server IP version context
- Fix both /scores/{ip}/log and Grafana endpoints
- Remove unused GetMonitorByName query

Fixes abh/ntppool#264
Reported-by: Anssi Johansson <https://github.com/avijc>
2025-07-27 00:37:49 -07:00
ask 8262b1442f feat(api): add Grafana time range endpoint for scores
- Add /api/v2/server/scores/{server}/{mode} endpoint
- Support time range queries with from/to parameters
- Return data in Grafana table format for visualization
- Fix routing pattern to handle IP addresses correctly
- Add comprehensive parameter validation and error handling
2025-07-27 02:18:32 -07:00
ask d4bf8d9e16 feat(api): add Grafana test endpoint for table format
Add `/api/v2/test/grafana-table` endpoint to validate Grafana
table format compatibility before implementing the full time
range API.

- Create server/grafana.go with table format structures
- Add structured logging and OpenTelemetry tracing
- Include realistic NTP Pool sample data with null handling
- Set proper CORS and cache headers for testing
- Update implementation plan with Phase 0 completion status

Ready for Grafana JSON API data source integration testing.
2025-07-26 09:03:46 -07:00
ask 6c5b762a57 Update dependencies
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-05 12:29:51 -07:00
ask fd6e87cf2d fix(history): sanitize NULL bytes in CSV error output 2025-07-05 12:27:35 -07:00
ask a22d5ebc7e feat(api): add RTT data to history endpoints
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
- Add RTT column to CSV output (before leap column)
- Add RTT field to JSON ScoresEntry
- Add avg_rtt field to JSON MonitorEntry
- Convert RTT from microseconds to milliseconds
- Calculate average RTT per monitor from history data
2025-07-04 09:41:17 -07:00
ask 42ce22e83e adjust cache-control for history api
continuous-integration/drone/push Build is passing
it seems like there's a bug in the data calculations so many
servers get the too long maximum cache time; make it shorter
while we debug
2025-06-27 17:48:40 +08:00
ask 087d253d90 Update schema for monitors v4; use go tool
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-06-21 03:49:02 -07:00
ask ae7acb4111 Update schema dump from development
continuous-integration/drone/push Build is passing
2025-04-08 00:30:52 -07:00
ask bd4e52a73b Update Go to 1.24 + dependencies
continuous-integration/drone/push Build is passing
2025-03-08 10:21:44 -08:00
ask 118e596098 build: update goreleaser to 2.7.0
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-02-23 09:49:39 -08:00
ask e6f39f201c dns queries: set cache-control header 2025-02-23 09:48:56 -08:00
ask 962839ed89 add dns query count endpoint 2025-02-23 09:28:30 -08:00
ask f8662fbda5 Support DSN config and auth for clickhouse connection
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-02-22 22:41:06 -08:00
ask a5b1f9ef08 Upgrade Go & dependencies
continuous-integration/drone/push Build is passing
2025-01-17 21:03:31 -08:00
ask e316aeee99 db: less logging when opening a database connection
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-01-06 19:33:45 +01:00
ask 3a9879b793 ch: add healtcheck for logs database
continuous-integration/drone/push Build is passing
2025-01-06 19:21:17 +01:00
ask 9fb3edacef data-api: fix health check shutdown; adjust db idle reset feature
continuous-integration/drone/push Build is passing
2025-01-03 14:28:36 +01:00
ask d206f9d20e history: fix the more nuanced cache-control max-age logic
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2024-12-30 17:33:29 -08:00
ask dc8adc1aea health: fix noisy logs
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2024-12-28 00:52:11 -08:00
ask 35ea262b99 go lint tweaks; update common
continuous-integration/drone/push Build is passing
2024-12-27 18:45:28 -08:00
ask 058531c362 Update Go and dependencies
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2024-12-27 17:38:58 -08:00
ask 801d685940 health: ping databases for health checks 2024-12-27 17:37:16 -08:00
ask 904f4b1df5 remove slightly less debugging 2024-11-03 07:15:25 +00:00
ask 78432679ec Update Go & dependencies
continuous-integration/drone/push Build is passing
2024-10-05 01:31:02 -07:00
ask 397dd13c38 Fix 500 errors when requesting with an invalid or unknown monitor parameter
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2024-07-21 04:31:42 -07:00
ask 79eea1d0f8 scores: improve error handling for invalid monitor parameters
continuous-integration/drone/push Build is passing
2024-07-20 00:29:36 -07:00
ask 8dfd7c8a4e Update Go and dependencies
continuous-integration/drone/push Build is passing
2024-07-20 00:24:17 -07:00
ask 574c7cfbf0 Fix lint warnings
continuous-integration/drone/push Build is passing
2024-03-09 09:03:41 -08:00
ask 3cbef93607 Update schema, Go 1.22.1, and dependencies
continuous-integration/drone/push Build is passing
2024-03-09 09:59:29 -08:00
ask 675e993353 scorer: fix parsing leap column
continuous-integration/drone/push Build is passing
2024-01-20 23:43:22 -07:00
ask e1398e7472 scores: full_history option for internal clients
continuous-integration/drone/push Build is passing
(somewhat inefficient, but for now rarely used ...)
2024-01-20 23:21:21 -07:00
ask b786ed6986 metrics: add echo metrics
continuous-integration/drone/push Build is passing
2024-01-20 21:35:49 -07:00
ask 2f2a407409 scorer: configurable default source 2024-01-20 21:35:18 -07:00
ask 6df51fc19f scores: clickhouse support
continuous-integration/drone/push Build is passing
2024-01-20 19:41:02 -07:00
ask 5682c86837 Set environment in tracing, minor dependency updates 2024-01-13 21:49:10 -08:00
ask 9428c1a227 Minor logging tweak 2024-01-12 22:39:19 -08:00
ask 6b84bbe5e1 Update Go and dependencies
continuous-integration/drone/push Build is passing
2024-01-12 22:14:15 -08:00
ask 47b96cd598 zones: per zone server counts API migrated
continuous-integration/drone/push Build is passing
2023-12-23 01:54:21 -08:00
ask 19c02063e9 Make text/csv inline in the browser
continuous-integration/drone/push Build is passing
... by using text/plain
https://bugs.chromium.org/p/chromium/issues/detail?id=152911
2023-12-22 09:33:14 -08:00
ask 84523661e2 scores: Allow 10000 rows 2023-12-22 09:22:26 -08:00
ask 6553b4711b scores: allow specifying the monitor by name
continuous-integration/drone/push Build is passing
2023-12-22 08:59:13 -08:00
ask 9280668d28 scores: support requesting logs by monitor name 2023-12-22 08:15:13 -08:00
ask ccc2fd401f scores: better error when monitor parameter is invalid 2023-12-22 08:11:43 -08:00
ask f2e4530023 Update dependencies 2023-12-22 08:10:42 -08:00
ask 8f333354d2 Register build_info metric 2023-12-15 00:10:39 -08:00
ask 41e7585637 scores: redirect POST requests
continuous-integration/drone/push Build is passing
2023-12-14 23:57:36 -08:00
ask 36f695c146 trace tweaks
continuous-integration/drone/push Build is passing
2023-12-14 23:17:44 -08:00
ask 404c64b910 Return not found for more recently deleted servers, too 2023-12-14 23:17:44 -08:00
ask 2cd4d8a35a Fix url.path in trace 2023-12-14 23:14:00 -08:00
ask bae726dba6 scores: handle IPs with no current history
continuous-integration/drone/push Build is passing
2023-12-14 20:00:09 -08:00
ask f6b0f96a34 scores: json handler
continuous-integration/drone/push Build was killed
2023-12-10 21:42:15 -08:00
ask 61245cc77c scores: csv handler 2023-12-10 21:02:04 -08:00
ask adab600e26 Function to get server from IP or ID parameter
continuous-integration/drone/push Build is passing
2023-12-10 20:55:44 -08:00
ask 9ef534eafa server: add url.path to traces
continuous-integration/drone/push Build is passing
2023-12-10 20:49:44 -08:00
ask 9c6ea595f1 dnsanswers: tracing adjustments 2023-12-10 15:11:26 -08:00
ask e824274998 Add png graph handler
continuous-integration/drone/push Build is passing
2023-12-09 13:54:13 -08:00
ask 69cc4b4e80 Update dependencies and Go 2023-12-08 22:02:49 -08:00
ask 37d66b073e Fix 'unsupported value: +Inf' error when the zone doesn't have active servers
continuous-integration/drone/push Build is passing
2023-11-21 15:10:42 -08:00
ask 954d97f71d Log version on startup
continuous-integration/drone/push Build is passing
2023-11-21 12:24:02 -08:00
ask 4d2ff44b18 Update dependencies; better logging and tracing
continuous-integration/drone/push Build is passing
2023-11-21 12:15:19 -08:00
ask fa86be0310 Use Fastly IPs 2023-11-12 15:53:34 -08:00
ask 86ad4a76b7 Use logger instance instead of default log/slog
continuous-integration/drone/push Build is passing
2023-10-22 16:33:54 -07:00
ask f85f09ebcf Small error handling updates
continuous-integration/drone/push Build is passing
2023-10-22 16:28:24 -07:00
ask 1f5fae973f Graceful shutdown
continuous-integration/drone/push Build is passing
2023-10-22 15:47:53 -07:00
ask 41c8fffcb0 Add trace_id to logs
continuous-integration/drone/push Build is failing
2023-10-22 13:02:53 -07:00
ask ceddc2564d Update tracing API
continuous-integration/drone/push Build is passing
2023-10-22 12:01:43 -07:00
ask f9b521664d Add tracing 2023-10-14 04:01:43 -07:00
ask 42dccd28a7 Go 1.21.3; dependencies 2023-10-14 00:01:44 -07:00
ask ca2cdaf458 Go 1.21.1, log/slog, dependencies
continuous-integration/drone/push Build is passing
2023-09-17 23:12:33 -07:00
ask 27c8bc4776 Calculate netspeed points, too
continuous-integration/drone/push Build is passing
2023-07-16 02:40:50 -07:00
ask 1f7973e885 Disable clickhouse debug logging
continuous-integration/drone/push Build is passing
2023-07-15 23:49:39 -07:00
ask 80ef36ae1c More CORS headers
continuous-integration/drone/push Build is passing
2023-07-15 22:35:41 -07:00
ask b10134b3da basic cache-control headers for the dns answers api
continuous-integration/drone/push Build is passing
2023-07-15 20:36:45 -07:00
ask ada531e561 Add basic request log
continuous-integration/drone/push Build is passing
2023-07-15 20:31:41 -07:00
ask 1963aae0b7 Database config parameter 2023-07-08 22:15:03 -07:00
ask 6583a36514 Adjust port numbers
continuous-integration/drone/push Build is failing
2023-07-08 21:05:22 -07:00
ask aeac7f1a39 Build tools
continuous-integration/drone/push Build is passing
2023-07-08 22:09:31 -07:00