LogicMonitor Year in Review: Architecture and Release Processes

This is Part I of a four-part Year in Review blog series. Check out what else the LogicMonitor team was up to in 2016: Automation and Extensibility, Data Visualizations, and Actionable Data. 

The LogicMonitor team is already moving full steam ahead into 2017! We are well underway with our first release of the year and are excited for the litany of new features in store for our product. But before we get too deep in the trenches, we will be publishing a series of blog posts examining the major milestones our business, customer base, and product experienced in 2016.

Let’s kick-off this series by highlighting some changes to our core technology architecture and release processes. Because these changes are generally made behind-the-scenes, they often do not get the fanfare they deserve. Nevertheless, they have helped LogicMonitor make tremendous strides in both the granularity and dynamicity of our product.

TSDB

Near the forefront of LogicMonitor’s 2016 accomplishments, we replaced our round-robin database (RRD) with a new Time-Series Database (TSDB). Data is king at LogicMonitor and our platform is evidence of that: we collect approximately 100 billion individual metrics per day across our customer base. To ensure that we are able to efficiently store every last one of these datapoints for future issue remediation and performance evaluation, we built our own proprietary TSDB that is capable of handling well over 800,000 insertions per node per second, versus just 11,200 with the premier existing time-series database. For our customers, this means that you can drill down to individual raw-data points for up to two years after they were collected. Such granular storage has also enabled us to begin designing advanced event correlation and data visualization features…more info on these to come in 2017!

Service-Oriented Architecture

As our product gets increasingly sophisticated with the likes of TSDB and an ever-growing roster of features, we implemented a Service-Oriented Architecture (SOA) to ensure our backend could keep up. This meant compartmentalizing the LogicMonitor platform’s individual components rather than building them into a single monolithic application. This change is a huge step forward in two ways:

1. It allows us to be extremely flexible when deploying new features while simultaneously minimizing–and in some cases eliminating–downtime. Instead of repackaging, SDTing, and upgrading an entire monolithic application each time we release a new set of code, dedicating each service to its own host (or container) allows us to avoid downtime for our general application and isolate code deployments to the relevant feature. Additionally, by using a queued management system, even the upgraded features will remain accessible to clients during the deployment.  
2. By having dedicated resources for each component of our software, there is an inherent transparency as to where any potential technical issues are emanating. This significantly reduces the resources we have to dedicate to troubleshooting and time-to-resolution. As a performance monitoring software company, we want to follow our own monitoring best practices: the simpler you can make your components, the easier life will be for your DevOps team. 

Collector Deployments

Last, but certainly not least, we overhauled our process for releasing new Collector versions. Our eagle-eyed customers may notice that we are cheating slightly by including this in a “2016 Year in Review” blog since we implemented this change at the tail-end of 2015. Nevertheless, its wide-ranging implications for both our product and customer base merits it a spot on this list.

In previous years, LogicMonitor only had one active Collector version at a time, to which all customers had to upgrade. Since then, we have broken our Collector versions into three distinct categories:

1. Required General Releases: stable Collector versions to which all customers must upgrade in order to adequately support changes in our application.
2. Optional General Releases: Collectors that have stable, post-beta features but are not required for upgrades.
3. Early Releases: Collectors containing beta features.

Using this new model of deployment, we can roll out beta Collectors supporting highly-requested new features for customers who want them, while simultaneously maintaining stable versions for customers who do not wish to continuously upgrade. Overall, this offers our customers more flexibility and choice when it comes to our Collector releases.

The opportunity to implement these improvements has already been incredibly gratifying for everyone on the LogicMonitor team. But, we especially like to hear from customers about how these changes have benefited their performance monitoring. So drop us a line using the Feedback button to share your stories! Also be sure keep an eye out for Part II of our 2016 Year in Review blog series!