"There's a dashboard for that!" (TM)
Fifemon is the FIFE project monitoring system developed and operated by the User Support for Distributed Computing group at Fermilab. Primarily this is monitoring the HTCondor pools, data handling and storage systems, and other related systems.
For an overview of Fifemon and many screenshots of the full deployment at Fermilab please see Kevin Retzke's talk from HTCondor Week 2016.
We have made the generic probe scripts and generalized dashboards available for use by the HEP, Open Science Grid, and wider HTCondor community on GitHub. Scripts and dashboards are released only as time and effort permits on an 'as available' basis, and we look to the community for help in improving and expanding the application for general use.
Questions, comments, and open discussion are encouraged and should be directed to the firstname.lastname@example.org email list.
You may subscribe to this list by sending an email with
SUBSCRIBE FIFEMON <your name> in the body to email@example.com.