Posted by Recently somebody asked me if it was possible to see the current status for individual NSX-T load balancer server pool members. This information is indeed available in the NSX Manager simplified UI as you can see below:
The same info can be found under Advanced Networking & Security:
It’s nice that we can find this info in the NSX Manager UI, but it got me thinking that it would be even better if we could get notified on pool member status changes. After all, nobody has time to hang around in the NSX Manager UI all day long. It turns out that this is pretty easy to accomplish.
Log Insight
To make this happen I’m turning to one of my favorite tools namely vRealize Log Insight. No environment should be without it if you ask me. It’s a simple yet powerful tool which is why I like it so much. Receiving events, querying for events, and acting on query results. That’s about it most of the time.
So in the case of the load balancer server pool member status I create a Log Insight query that is looking for events containing the text obj.type: ‘poolmember’ and status.newstatus:
Seconds after I shut down one of the web servers in my load balancer server pool, the query above shows me following result:
Each of the NSX Edge nodes involved with the load balancer instance (i.e. the Edge nodes hosting the Tier-1 gateway constructs) generates the same event which is why we receive two identical events.
The event itself contains a lot of relevant information. A quick look at the key pieces of information in this event:
- Obj.Ip: ‘172.16.12.20’ – The IP address of the pool member.
- Obj.Port: ’80’ – The configured port for the pool member.
- Pool.Name: ‘web-pool’ – The server pool name.
- Lb.Name: ‘lb-01’ – The load balancer instance name.
- Vs.Name: ‘web-01’ – The name assigned to the pool member.
- Status.NewStatus: ‘Down’ – The new/current status of the pool member.
- Status.Msg: ‘Connect to Peer Failure’ – The reason for the status change.
A very similar event will be generated once I start the web server again:
This time the event contains:
- Status.NewStatus: ‘UP’
- Status.Msg: ‘pool member is up’
Alerting
Log Insight can send alerts based on query results.
Alerts can be send using email or made available via a webhook for third party integrations (like with Slack). Here I’m configuring an email alert for my pool member status change query:
I’m triggering the event once more by shutting down the web server:
I’ve got mail!
From now on I will receive an email alert each time the status of a pool member changes. Simple and easy.
Summary
Although most organizations have systems in place for service availability monitoring and alerting, it can’t hurt to have an extra little eye watching things from the NSX-T perspective. Especially when it’s this easy to set up.
A final note. To set up event forwarding from NSX-T to Log Insight you should have a look at the NSX-T content pack. Installing this content pack extends Log Insight with dashboards and queries especially for NSX-T. It also provides detailed instructions on how to configure event forwarding on the different NSX-T platform components.
rutgerblom.com 