diff --git a/README.md b/README.md index a442d5e..4c8e5fc 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,25 @@ # Nerd Monitor ๐Ÿ“Š -A lightweight, cross-platform system monitoring solution written in Go. Monitor CPU, memory, and disk usage across multiple machines with a beautiful web dashboard. +A lightweight, cross-platform system monitoring solution written in Go. Monitor CPU, memory, and disk usage across multiple machines with a beautiful web dashboard featuring instant real-time updates powered by Server-Sent Events and HTMX. ![Go](https://img.shields.io/badge/Go-1.23-blue?logo=go) +![Real-time](https://img.shields.io/badge/real--time-SSE%20%2B%20HTMX-orange?logo=lightning) ![License](https://img.shields.io/badge/license-MIT-green) ![Platforms](https://img.shields.io/badge/platforms-Linux%20%7C%20macOS%20%7C%20Windows-important) ## Features - ๐Ÿ–ฅ๏ธ **Multi-platform Support** - Deploy agents on Linux, macOS, and Windows (AMD64 & ARM64) -- ๐Ÿ“ˆ **Real-time Monitoring** - Track CPU, RAM, and Disk usage with 15-second refresh intervals +- โšก **Instant Real-time Updates** - Stats update immediately when agents report, not on fixed intervals - ๐ŸŸข **Live Status Indicators** - See which machines are online/offline at a glance - ๐Ÿ”’ **Secure Authentication** - Session-based admin authentication for the dashboard - ๐Ÿงน **Stale Agent Management** - Automatically detect and remove agents inactive for 6+ months -- ๐Ÿ“ฑ **Responsive Dashboard** - Beautiful, modern UI with auto-refresh -- ๐Ÿš€ **Minimal Dependencies** - Only Chi router and Templ templating engine (plus Go stdlib) +- ๐Ÿ“ฑ **Responsive Dashboard** - Beautiful, modern UI with seamless HTMX-powered updates +- ๐Ÿš€ **Minimal Dependencies** - Only Chi router, Templ templating, and HTMX (plus Go stdlib) - โš™๏ธ **Auto-Generation** - Agent IDs automatically generated from hostname - ๐Ÿ’พ **In-Memory Storage** - Fast, zero-database setup (perfect for small to medium deployments) +- ๐Ÿ“ก **Server-Sent Events** - Real-time push notifications from server to web clients +- ๐Ÿ“ **Structured Logging** - Comprehensive slog-based logging throughout the application ## Quick Start @@ -54,7 +57,7 @@ Binaries are created in the `bin/` directory. -password securepassword ``` -Then access the dashboard at `http://localhost:8080` +Then access the dashboard at `http://localhost:8080`. The interface provides real-time updates - stats refresh instantly when agents report new data! ### Running Agents @@ -87,7 +90,7 @@ nerd-monitor-agent.bat --server 10.0.20.80:9090 --interval 30s ### Overview Page -Shows all connected agents with real-time metrics: +Shows all connected agents with instant real-time metrics: - **Status Badge** - Green (Online) or Red (Offline) - Online: Last report within 15 seconds @@ -96,16 +99,18 @@ Shows all connected agents with real-time metrics: - **Memory Usage** - Used / Total with visual progress bar - **Disk Usage** - Used / Total with visual progress bar - **Last Seen** - Human-readable timestamp +- **Live Updates** - Stats refresh immediately when agents report new data ### Agent Detail Page -Click any agent hostname to see detailed statistics: +Click any agent hostname to see detailed statistics with live updates: -- Large CPU usage display with progress bar +- Large CPU usage display with progress bar (updates in real-time) - Memory and disk breakdowns - Agent ID and detailed metadata - Last exact timestamp - Delete button for removing the agent +- **Instant Updates** - CPU stats refresh immediately when agent reports ### Stale Agent Management @@ -122,10 +127,15 @@ Agents inactive for 6+ months: - **HTTP Router**: Chi v5 for efficient routing - **Authentication**: Session-based auth with 24-hour expiry - **Storage**: In-memory concurrent-safe store (sync.RWMutex) +- **Real-time Broadcasting**: Server-Sent Events (SSE) for instant UI updates +- **Structured Logging**: slog-based logging with debug/info levels - **API Endpoints**: - - `POST /api/report` - Agent stats reporting + - `POST /api/report` - Agent stats reporting (triggers real-time broadcasts) - `GET /api/agents` - List all agents - `GET /api/agents/{id}` - Get specific agent stats + - `GET /api/dashboard/table` - HTML fragment for HTMX dashboard updates + - `GET /api/agents/{id}/stats` - HTML fragment for HTMX agent detail updates + - `GET /api/events` - Server-Sent Events for real-time notifications ### Agent (`cmd/agent/`) @@ -138,8 +148,9 @@ Agents inactive for 6+ months: ### Views (`views/`) - **Templ Templates**: Type-safe HTML templating +- **HTMX Integration**: Smooth, instant updates without page refreshes +- **Server-Sent Events**: Real-time push notifications from server - **Responsive Design**: Works on desktop, tablet, and mobile -- **Auto-Refresh**: Dashboard refreshes every 5 seconds - **Color-coded Status**: Visual indicators for system health ## Project Structure @@ -148,28 +159,28 @@ Agents inactive for 6+ months: nerd-monitor/ โ”œโ”€โ”€ cmd/ โ”‚ โ”œโ”€โ”€ agent/ -โ”‚ โ”‚ โ””โ”€โ”€ main.go # Agent executable +โ”‚ โ”‚ โ””โ”€โ”€ main.go # Agent executable with slog logging โ”‚ โ””โ”€โ”€ server/ -โ”‚ โ””โ”€โ”€ main.go # Server executable +โ”‚ โ””โ”€โ”€ main.go # Server executable with slog logging โ”œโ”€โ”€ internal/ โ”‚ โ”œโ”€โ”€ api/ -โ”‚ โ”‚ โ””โ”€โ”€ api.go # API handlers +โ”‚ โ”‚ โ””โ”€โ”€ api.go # API handlers + real-time broadcaster โ”‚ โ”œโ”€โ”€ auth/ -โ”‚ โ”‚ โ”œโ”€โ”€ middleware.go # Auth middleware +โ”‚ โ”‚ โ”œโ”€โ”€ middleware.go # Auth middleware with detailed logging โ”‚ โ”‚ โ””โ”€โ”€ errors.go # Error definitions โ”‚ โ”œโ”€โ”€ stats/ โ”‚ โ”‚ โ””โ”€โ”€ stats.go # System stats collection โ”‚ โ”œโ”€โ”€ store/ -โ”‚ โ”‚ โ””โ”€โ”€ store.go # In-memory agent storage +โ”‚ โ”‚ โ””โ”€โ”€ store.go # In-memory agent storage with logging โ”‚ โ””โ”€โ”€ ui/ -โ”‚ โ””โ”€โ”€ handlers.go # Dashboard handlers +โ”‚ โ””โ”€โ”€ handlers.go # Dashboard handlers + HTMX endpoints โ”œโ”€โ”€ views/ -โ”‚ โ”œโ”€โ”€ layout.templ # Base layout template -โ”‚ โ”œโ”€โ”€ dashboard.templ # Dashboard page -โ”‚ โ”œโ”€โ”€ agent_detail.templ # Agent detail page +โ”‚ โ”œโ”€โ”€ layout.templ # Base layout with HTMX + SSE +โ”‚ โ”œโ”€โ”€ dashboard.templ # Dashboard with real-time updates +โ”‚ โ”œโ”€โ”€ agent_detail.templ # Agent detail with live stats โ”‚ โ”œโ”€โ”€ login.templ # Login page โ”‚ โ”œโ”€โ”€ components.templ # Reusable components -โ”‚ โ””โ”€โ”€ generate.go # Templ code generation +โ”‚ โ””โ”€โ”€ *_templ.go # Generated Templ code โ”œโ”€โ”€ Makefile # Build system โ”œโ”€โ”€ go.mod / go.sum # Go dependencies โ”œโ”€โ”€ AGENTS.md # Agent guidelines @@ -231,9 +242,10 @@ Options: - **Agent Memory**: ~8-10 MB (depending on platform) - **Server Memory**: Scales with connected agents (~1 MB per 1000 agents) -- **Network**: Minimal bandwidth (~1 KB per report) -- **Dashboard Refresh**: 5 seconds (configurable) +- **Network**: Minimal bandwidth (~1 KB per report + SSE connections) +- **Real-time Updates**: Instant UI updates when agents report stats - **Agent Reporting**: 15 seconds (configurable) +- **SSE Connections**: Lightweight persistent connections for real-time notifications ## Building for Specific Platforms @@ -261,6 +273,8 @@ make build-all - `github.com/go-chi/chi/v5` - HTTP router - `github.com/a-h/templ` - HTML templating - `github.com/shirou/gopsutil/v3` - System statistics +- **HTMX** (CDN) - Frontend interactivity and real-time updates +- **Server-Sent Events** (Go stdlib) - Real-time push notifications ### Go Standard Library @@ -270,6 +284,7 @@ make build-all - `time` - Timestamps and durations - `os` - Hostname, signals, I/O - `crypto/rand` - Token generation +- `log/slog` - Structured logging throughout application ## Troubleshooting @@ -279,6 +294,15 @@ make build-all 2. Check agent logs for connection errors 3. Ensure correct server address and port 4. Check firewall rules allow outbound connections +5. Check server logs for SSE broadcasting activity + +### Stats not updating in real-time + +1. Verify browser supports Server-Sent Events (SSE) +2. Check browser developer tools for JavaScript errors +3. Ensure `/api/events` endpoint is accessible (requires authentication) +4. Check server logs for SSE connection/disconnection messages +5. Verify HTMX library loaded correctly ### Agent shows "Offline" immediately @@ -286,6 +310,7 @@ make build-all 2. Verify server address is reachable: `ping 10.0.20.80` 3. Check if server is listening on the correct port 4. Review server logs for errors +5. Note: Agents are marked offline if no report received in 15 seconds ### Can't login to dashboard @@ -293,6 +318,7 @@ make build-all 2. Try the default: `admin` / `admin` 3. If changed, restart server with original credentials to reconfigure 4. Check browser cookies are enabled +5. Clear browser cache if SSE connections seem stuck ### Windows agent window appears then closes @@ -312,6 +338,18 @@ make templ go run github.com/a-h/templ/cmd/templ@latest generate ``` +### Viewing Logs + +The application uses structured logging with `slog`. Set log level with: + +```bash +# Debug logging (shows all activity) +export GOLOGLEVEL=debug + +# Info logging (default, shows important events) +export GOLOGLEVEL=info +``` + ### Running Tests ```bash @@ -324,20 +362,42 @@ go test ./... gofmt -w ./cmd ./internal ./views ``` +### Testing Real-time Features + +To test SSE and HTMX functionality: + +```bash +# Start server +./bin/nerd-monitor-server + +# In another terminal, send test stats +curl -X POST "http://localhost:8080/api/report?id=test-agent" \ + -H "Content-Type: application/json" \ + -d '{"hostname": "test", "cpuUsage": 25.5, "ramUsage": 1000000, "ramTotal": 8000000, "diskUsage": 50000000, "diskTotal": 100000000}' +``` + ## Contributing Contributions are welcome! Please follow the code style guidelines in `AGENTS.md`. +## Recent Updates + +- โœ… **Real-time Updates**: Instant UI updates via Server-Sent Events and HTMX +- โœ… **Structured Logging**: Comprehensive slog-based logging throughout +- โœ… **Enhanced Monitoring**: Better debugging and operational visibility + ## Future Enhancements - [ ] Database persistence (SQLite/PostgreSQL) - [ ] Alerting system (email/Slack notifications) -- [ ] Historical data / graphing +- [ ] Historical data / graphing with time-series storage - [ ] Agent grouping and tagging - [ ] Custom metric collection - [ ] TLS/HTTPS support - [ ] Multi-tenancy -- [ ] API documentation (Swagger) +- [ ] API documentation (Swagger/OpenAPI) +- [ ] Webhook integrations +- [ ] Mobile app companion ## License diff --git a/cmd/server/main.go b/cmd/server/main.go index 0481ce4..ff95b3c 100644 --- a/cmd/server/main.go +++ b/cmd/server/main.go @@ -43,11 +43,19 @@ func main() { r.Get("/login", uiHandler.Login) r.Post("/login", uiHandler.Login) + // Public routes (no auth required) + r.Post("/api/report", apiHandler.ReportStats) + r.Get("/api/agents", apiHandler.ListAgents) + r.Get("/api/agents/{id}", apiHandler.GetAgent) + // Protected routes (auth required) r.Group(func(protectedRoutes chi.Router) { protectedRoutes.Use(authMgr.Middleware) protectedRoutes.Get("/", uiHandler.Dashboard) protectedRoutes.Get("/agents/{id}", uiHandler.AgentDetail) + protectedRoutes.Get("/api/dashboard/table", uiHandler.GetDashboardTable) + protectedRoutes.Get("/api/agents/{id}/stats", uiHandler.GetAgentStats) + protectedRoutes.Get("/api/events", uiHandler.Events) protectedRoutes.Post("/agents/{id}/hostname", uiHandler.UpdateAgentHostname) protectedRoutes.Post("/logout", uiHandler.Logout) protectedRoutes.Post("/api/agents/remove-stale", uiHandler.RemoveStaleAgents) diff --git a/internal/api/api.go b/internal/api/api.go index cb1da24..3792b21 100644 --- a/internal/api/api.go +++ b/internal/api/api.go @@ -4,12 +4,57 @@ import ( "encoding/json" "log/slog" "net/http" + "sync" "github.com/go-chi/chi/v5" "nerd-monitor/internal/stats" "nerd-monitor/internal/store" ) +// Broadcaster manages Server-Sent Events clients. +type Broadcaster struct { + clients map[chan string]bool + mu sync.RWMutex +} + +var apiBroadcaster = &Broadcaster{ + clients: make(map[chan string]bool), +} + +// Broadcast sends a message to all connected SSE clients. +func (b *Broadcaster) Broadcast(message string) { + b.mu.RLock() + defer b.mu.RUnlock() + + for clientChan := range b.clients { + select { + case clientChan <- message: + default: + // Client channel is full, skip + } + } +} + +// AddClient adds a new SSE client. +func (b *Broadcaster) AddClient(clientChan chan string) { + b.mu.Lock() + defer b.mu.Unlock() + b.clients[clientChan] = true +} + +// RemoveClient removes an SSE client. +func (b *Broadcaster) RemoveClient(clientChan chan string) { + b.mu.Lock() + defer b.mu.Unlock() + delete(b.clients, clientChan) + close(clientChan) +} + +// GetAPIBroadcaster returns the API broadcaster instance. +func GetAPIBroadcaster() *Broadcaster { + return apiBroadcaster +} + // Handler manages HTTP requests. type Handler struct { store *store.Store @@ -40,6 +85,12 @@ func (h *Handler) ReportStats(w http.ResponseWriter, r *http.Request) { slog.Debug("Updating agent stats", "agentID", agentID, "hostname", stat.Hostname, "cpu", stat.CPUUsage) h.store.UpdateAgent(agentID, &stat) + + // Broadcast update to all connected SSE clients + message := "event: stats-update\ndata: {\"type\": \"stats-update\", \"agentId\": \"" + agentID + "\"}\n\n" + slog.Info("Broadcasting stats update", "agentID", agentID) + apiBroadcaster.Broadcast(message) + w.Header().Set("Content-Type", "application/json") json.NewEncoder(w).Encode(map[string]string{"status": "ok"}) slog.Debug("Stats report processed successfully", "agentID", agentID) diff --git a/internal/ui/handlers.go b/internal/ui/handlers.go index 44afa25..2a10518 100644 --- a/internal/ui/handlers.go +++ b/internal/ui/handlers.go @@ -2,11 +2,13 @@ package ui import ( "context" + "fmt" "log/slog" "net/http" "time" "github.com/go-chi/chi/v5" + "nerd-monitor/internal/api" "nerd-monitor/internal/auth" "nerd-monitor/internal/store" "nerd-monitor/views" @@ -14,15 +16,20 @@ import ( // Handler serves UI pages. type Handler struct { - store *store.Store - auth *auth.Manager + store *store.Store + auth *auth.Manager + broadcaster interface { + AddClient(chan string) + RemoveClient(chan string) + } } // New creates a new UI handler. func New(s *store.Store, a *auth.Manager) *Handler { return &Handler{ - store: s, - auth: a, + store: s, + auth: a, + broadcaster: api.GetAPIBroadcaster(), } } @@ -212,3 +219,177 @@ func (h *Handler) getStaleAgents() []*store.AgentStats { return stale } + +// GetDashboardTable returns HTML fragment for the agent table. +func (h *Handler) GetDashboardTable(w http.ResponseWriter, r *http.Request) { + slog.Debug("HTMX dashboard table request", "method", r.Method, "path", r.URL.Path, "remoteAddr", r.RemoteAddr) + + agents := h.store.GetAllAgents() + + w.Header().Set("Content-Type", "text/html") + w.Header().Set("Cache-Control", "no-cache") + + if len(agents) == 0 { + w.Write([]byte(`
+ โ„น๏ธ No agents connected +

+ Start an agent to see its statistics appear here. +

+
`)) + return + } + + w.Write([]byte(`
+
Active Agents
+ + + + + + + + + + + `)) + + for _, agent := range agents { + // Determine online status (agent is online if reported within last 15 seconds) + isOnline := time.Since(agent.LastSeen) < 15*time.Second + statusClass := "status-red" + statusText := "Offline" + if isOnline { + statusClass = "status-green" + statusText = "Online" + } + + row := fmt.Sprintf(` + + + + + + `, + statusClass, + statusText, + agent.ID, + agent.Hostname, + agent.CPUUsage, + formatBytes(agent.RAMUsage), + formatBytes(agent.RAMTotal), + formatBytes(agent.DiskUsage), + formatBytes(agent.DiskTotal), + agent.LastSeen.Format("2006-01-02 15:04:05")) + w.Write([]byte(row)) + } + + w.Write([]byte(`
HostnameCPU UsageMemoryDiskLast Seen
+
+ %s + %s +
+
%.1f%%%s / %s%s / %s%s
`)) +} + +// GetAgentStats returns HTML fragment for agent statistics. +func (h *Handler) GetAgentStats(w http.ResponseWriter, r *http.Request) { + slog.Debug("HTMX agent stats request", "method", r.Method, "path", r.URL.Path, "remoteAddr", r.RemoteAddr) + + agentID := chi.URLParam(r, "id") + if agentID == "" { + slog.Warn("Missing agent ID in stats request", "remoteAddr", r.RemoteAddr) + http.Error(w, "missing agent id", http.StatusBadRequest) + return + } + + agent := h.store.GetAgent(agentID) + if agent == nil { + slog.Warn("Agent not found for stats request", "agentID", agentID) + http.Error(w, "agent not found", http.StatusNotFound) + return + } + + w.Header().Set("Content-Type", "text/html") + w.Header().Set("Cache-Control", "no-cache") + + // Return just the stats content that should be updated + statsHTML := fmt.Sprintf(`
%.1f%%
+
+
+
+
+ Last updated: %s +
`, + agent.CPUUsage, + agent.CPUUsage, + agent.LastSeen.Format("2006-01-02 15:04:05")) + + w.Write([]byte(statsHTML)) +} + +// Events provides Server-Sent Events for real-time updates. +func (h *Handler) Events(w http.ResponseWriter, r *http.Request) { + slog.Info("SSE connection established", "remoteAddr", r.RemoteAddr) + + // Set headers for Server-Sent Events + w.Header().Set("Content-Type", "text/event-stream") + w.Header().Set("Cache-Control", "no-cache") + w.Header().Set("Connection", "keep-alive") + w.Header().Set("Access-Control-Allow-Origin", "*") + + // Create a channel for this client + clientChan := make(chan string, 10) + h.broadcaster.AddClient(clientChan) + + // Clean up when client disconnects + go func() { + <-r.Context().Done() + slog.Debug("SSE client disconnected", "remoteAddr", r.RemoteAddr) + h.broadcaster.RemoveClient(clientChan) + }() + + // Send initial connection event + fmt.Fprintf(w, "event: connected\ndata: {}\n\n") + w.(http.Flusher).Flush() + + // Send a test event after 2 seconds + go func() { + time.Sleep(2 * time.Second) + select { + case clientChan <- "event: test\ndata: {\"message\": \"SSE working\"}\n\n": + default: + } + }() + + // Listen for broadcast messages + for { + select { + case <-r.Context().Done(): + return + case message := <-clientChan: + slog.Debug("Sending SSE message to client", "remoteAddr", r.RemoteAddr, "message", message) + fmt.Fprintf(w, "%s\n", message) + w.(http.Flusher).Flush() + } + } +} + +// formatBytes converts bytes to human-readable format. +func formatBytes(bytes uint64) string { + const ( + kb = 1024 + mb = kb * 1024 + gb = mb * 1024 + ) + + switch { + case bytes >= gb: + return fmt.Sprintf("%.1f GB", float64(bytes)/float64(gb)) + case bytes >= mb: + return fmt.Sprintf("%.1f MB", float64(bytes)/float64(mb)) + case bytes >= kb: + return fmt.Sprintf("%.1f KB", float64(bytes)/float64(kb)) + default: + return fmt.Sprintf("%d B", bytes) + } +} diff --git a/views/agent_detail.templ b/views/agent_detail.templ index 5c0c2d2..db30d66 100644 --- a/views/agent_detail.templ +++ b/views/agent_detail.templ @@ -9,9 +9,13 @@ templ AgentDetail(agent *store.AgentStats) { @BaseLayout(agent.Hostname, agentDetailContent(agent)) } +templ AgentDetailContent(agent *store.AgentStats) { + @agentDetailContent(agent) +} + templ agentDetailContent(agent *store.AgentStats) { โ† Back to Dashboard - +

{ agent.Hostname }

@AgentStatusBadge(agent.LastSeen) @@ -21,14 +25,16 @@ templ agentDetailContent(agent *store.AgentStats) {
CPU Usage
-
- { FormatPercent(agent.CPUUsage) } -
-
-
-
-
- Last updated: { FormatTime(agent.LastSeen) } +
+
+ { FormatPercent(agent.CPUUsage) } +
+
+
+
+
+ Last updated: { FormatTime(agent.LastSeen) } +
diff --git a/views/dashboard.templ b/views/dashboard.templ index a0155d4..5155a3a 100644 --- a/views/dashboard.templ +++ b/views/dashboard.templ @@ -8,74 +8,80 @@ templ Dashboard(agents []*store.AgentStats, staleAgents []*store.AgentStats) { @BaseLayout("Dashboard", dashboardContent(agents, staleAgents)) } +templ DashboardContent(agents []*store.AgentStats, staleAgents []*store.AgentStats) { + @dashboardContent(agents, staleAgents) +} + templ dashboardContent(agents []*store.AgentStats, staleAgents []*store.AgentStats) {

Agent Status Overview

- - @StaleAgentAlert(len(staleAgents), nil) - - if len(agents) == 0 { -
- โ„น๏ธ No agents connected -

- Start an agent to see its statistics appear here. -

-
- } else { -
-
Active Agents
- - - - - - - - - - - - for _, agent := range agents { - @AgentRow(agent.ID, agent.Hostname, agent.CPUUsage, agent.RAMUsage, agent.RAMTotal, agent.DiskUsage, agent.DiskTotal, agent.LastSeen) - } - -
HostnameCPU UsageMemoryDiskLast Seen
-
- } - - if len(staleAgents) > 0 { -
-
- Stale Agents ({ len(staleAgents) }) -
-
- - - - - - - - - - for _, agent := range staleAgents { - - - - - - } - -
HostnameLast Seen
{ agent.Hostname }{ FormatTime(agent.LastSeen) }
-
- + + @StaleAgentAlert(len(staleAgents), nil) + +
+ if len(agents) == 0 { +
+ โ„น๏ธ No agents connected +

+ Start an agent to see its statistics appear here. +

- + } else { +
+
Active Agents
+ + + + + + + + + + + + for _, agent := range agents { + @AgentRow(agent.ID, agent.Hostname, agent.CPUUsage, agent.RAMUsage, agent.RAMTotal, agent.DiskUsage, agent.DiskTotal, agent.LastSeen) + } + +
HostnameCPU UsageMemoryDiskLast Seen
+
+ }
- + } diff --git a/views/generate.go b/views/generate.go deleted file mode 100644 index be187ec..0000000 --- a/views/generate.go +++ /dev/null @@ -1,3 +0,0 @@ -//go:generate templ generate - -package views diff --git a/views/layout.templ b/views/layout.templ index faed30e..dd5df73 100644 --- a/views/layout.templ +++ b/views/layout.templ @@ -6,8 +6,31 @@ templ BaseLayout(title string, content templ.Component) { - { title } - Nerd Monitor + +