The Disciplines Plate VI Monitoring & Observability

Monitoring &
Observability.

So you hear first.

Dashboards, alerts and telemetry set up so that when something goes wrong the first person to hear about it is you, and you have time to fix it before anybody else notices. Prometheus, Grafana, Zabbix, Checkmk.

Return to the workshop

Some businesses cannot send monitoring data to the cloud for compliance or security reasons and need it kept on site, sometimes fully air-gapped, while others are happy with a hosted service, and most end up somewhere in between.

I design the monitoring around your setup, whether that is a handful of servers or hundreds of endpoints, and weigh what you need to see against security and cost. Wherever I can I use open-source tools (Prometheus, Grafana, Uptime Kuma, Checkmk), which do the same job as the big commercial platforms without the licence fees, so the money goes on setting it up and none of it on software.

§ What I monitor

  1. I.

    Infrastructure health

    CPU, memory, disk and network. The alerts fire before a server runs out of room, and the history shows you when an upgrade is due, a long time before it becomes an emergency.

  2. II.

    Uptime & availability

    Your site checked from several locations, with warnings before SSL certificates expire, DNS checks, and response times tracked for SLA reporting.

  3. III.

    Application & database

    Query performance, connection pools, replication lag and slow queries, plus the metrics that are specific to your application, and where the standard tools don't cover something I add custom instrumentation.

  4. IV.

    Security events

    Failed logins, firewall blocks and unusual network traffic, fed into a SIEM if you have one, and correlated across systems so a pattern shows up even when each event looks harmless on its own.

  5. V.

    Log aggregation

    Logs from every system in one place, searchable across months in seconds and structured so that problems are quicker to trace. How long they are kept depends on what compliance requires and what the storage costs.

  6. VI.

    Alerting

    Escalation policies and on-call schedules, with alerts delivered by Slack, Teams, email or SMS. The thresholds are tuned to cut down false alarms, because people stop reading alerts that are wrong too often.

§ The process

  1. Step 1

    Discovery

    I look at what you have and what matters most: which systems are critical, how much downtime you can live with, what compliance applies and whether it should run in the cloud, on site or air-gapped.

  2. Step 2

    Design

    You get a proposal that fits your requirements and your budget and says which tools to use, where they run and which metrics are worth tracking.

  3. Step 3

    Deploy

    I install the agents, set up the dashboards and alerts and connect everything to the tools you already use, and you get notes on what was deployed and why.

  4. Step 4

    Tune

    The first month is for adjusting: thresholds moved, noisy alerts cut, metrics I missed added and useless ones dropped.

  5. Step 5

    Support

    If you want it, there is ongoing support, which means adding new systems as they arrive and helping when something goes wrong, and if you would rather run it yourself you get a clean handover.

§

Questions, answered

Before you
have to ask

We already get alerts and ignore them. How is this different?

If people ignore the alerts, the alerts are badly designed. Most of the work is deciding what is worth waking someone for and deleting the rest, which usually leaves you with fewer alerts than you have now.

Do we need to buy a monitoring platform?

Usually not. For a business of this size, the open-source tools named above do everything the licensed products do, and the money is better spent on having them set up well.

What will it cost?

The prices on this page are starting points, so that you can tell in a minute whether you are in the right range. I then quote each job once we have talked it through, because the price depends on the shape of the job, and at this point nobody knows that yet. The first half hour is free, and usually it settles whether this is a small piece of work or a large one before any money is discussed.

Who owns the code when it is finished?

You do, outright, including the source. I hand it over with the runbooks you need to keep it running, and there is no licence to renew and no seat to pay for. Where this page quotes a monthly figure, that is for hosting or ongoing support, which is optional and can be stopped without taking anything away from you. When I say commissions, not subscriptions, I mean it literally.

What happens if you are unavailable?

I build everything so that somebody else could pick it up: common technologies, readable code, written runbooks and nothing proprietary. It is a fair question to ask of a one-person business, and the answer lies in how the work is built.

Do you work outside Suffolk and Cambridgeshire?

Yes. With local work you get the option of having me in the room, which is worth more than it sounds for the first conversation and for the handover, and for most clients everything after that is done remotely anyway.