Debug F5 monitor response from the server

It is quite simple to see if a pool member failed it’s health check by checking the pool status via GUI/CLI and the ltm logs also give you more information on the time lines when the pool went down/up;

cd /var/log
cat ltm | grep

But what if you’ve configured a custom health monitor for a particular pool and now that pool is down and you know it’s the monitor that is failing it. You’ve verified that the F5 is indeed sending the F5 monitor traffic to the nodes. You run some captures on the interface or on an intermediary firewall and see the node is sending replies as well. Now, what if you want to check the contents of the server’s response during that time from the F5 itself?

So here’s the step-by-step instructions you need to follow to effectively get that information;

1. Enable the debug on F5

tmsh modify sys db bigd.debug value enable

2. Check if debug is enabled

tmsh list sys db bigd.debug

3. Check the debug logs from bigdlog file  for particular node

tail -f /var/log/bigdlog | grep

4. Disable debug! The file size gets huge pretty quickly!

tmsh modify sys db bigd.debug value disable

5. Navigate to the log folder

cd /var/log

6. This will list the bigdlog log file with details like date, time and size

ls –ltr

7. Remove the log file if you’ve copied the information you wanted or you can keep it there if you’ve a good size of flash on the device.

rm bigdlog

Here’s an example of a custom HTTPS monitor;

ltm monitor https Custom_HTTPS_Monitor {
       cipherlist DEFAULT:+SHA:+3DES:+kED
compatibility enabled
       defaults-from https
destination *:*
interval 5
recv "Site is Up"
send "GET /is_the_site_up?/ \\r\\n\\r\\n"
       time-until-up 0
timeout 16}

In the debugs you capture, you should be able to see similar output as below when everything is working fine (will need to go through a lot of crap before you actually find this stuff that is related to your monitor).

2014-11-04 23:31:08.010366: ID 764   :(_send_active_service_ping): writing [ addr=::ffff: srcaddr=::ffff: ] send=GET /is_the_site_up?/ \x0d\x0a\x0d\x0a
2014-11-04 23:31:08.486884: ID 766   :(_main_loop): rfd selected [ addr=::ffff: srcaddr=::ffff: fd=99 pend=0 ]
2014-11-04 23:31:08.486972: ID 766   :(_recv_active_service_ping): reading [ addr=::ffff: srcaddr=::ffff: ]
2014-11-04 23:31:08.487153: ID 766   :(_recv_active_service_ping): rcvd 5120 bytes: -->\x0a\x09
! Lots of crap. Omitted for brevity. Just included the actual header line that matches our recv string.
<td class="Heading1">Site is Up</td>
2014-11-04 23:31:08.487276: ID 766   :(_recv_active_service_ping): Response matched regex [ addr=::ffff: srcaddr=::ffff: ] send=GET /is_the_site_up?/ \x0d\x0a\x0d\x0a

If your send string gets you the desired output that your receive string is expecting to see, your monitor shouldn’t fail. If you don’t see the receive string in the output of the bigdlog file, your server is sending something else or your receive string on the F5 is wrong.

Hope this helps!!!

4 thoughts on “Debug F5 monitor response from the server

  1. Thanks for the post its very informative.what if you have a bulky F5 with alot of nodes.
    Is it non-impacting when you debug run-time traffic of all the nodes?

  2. Hey Vaibhav, glad you liked this. I tried this on a production load balancer with around 60 pools probably and didn’t give up on me. I don’t think it will have much of a performance impact but the only concern here would be to ensure you stop the debugs or the file it writes the logs to can get pretty huge and overwhelm the flash storage.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s