TRAC's recent article "Application Performance Management - The Journey of a Technology Label" talked about how diluted the APM space has became and that it is getting more difficult for end-user organizations to distinguish between vendors' marketing messaging and the true value that these solutions can deliver. Sometimes, we would get vendor requests for a call to discuss their latest customer win, selection process that an end-user organization went though and a type of IT environment that they are managing. The most impressive part about some of these calls is not necessarily how the vendor won that deal, but how some vendors they were competing against even got to be considered in that type of usage scenario.

Recently, we had two application monitoring vendors referring to the same competitor as a company that is "having more success against us than ever before" and "gets involved in some of our deals, but we very rarely see them win". On the other hand, when talking with end-users about the same product, the feedback that we get can range anywhere from them raving about the product to complaining about all of the "blind spots" that it leaves. So, is the solution that this vendor provides good or bad? It is actually very good when used for addressing problems that it was designed to address.

The application performance monitoring market has been growing very fast and some vendors are trying to grow their presence by going after nearly any environment that lacks visibility into application performance. This typically results in them sometimes getting invited to the table in usage scenarios where they have very little chance of winning, while it makes it more difficult for companies that could actually add value in those environments to fight through marketing noise and even be considered. The reality is that even though there are a lot of vendors that play in this space, due to new market dynamics, "shortlists" of vendors that can be truly effective in certain usage scenarios are sometimes very short.

To shed light on this issue, we have been interviewing end-user organizations and trying to find out what they really care about when evaluating technology vendors. The findings came down to three key areas: 1) fit for their IT environment; 2) problems that they are trying to solve; 3) product attributes that they are looking for. We also found significant differences between preferences of different job roles and companies that fall into different categories based on the size of the organization and the industry sector that they compete in. The list of individual criteria that impacts their selection processes was fairly long and SOME of them are listed below.

Our goal was to publish a study that would help end-user organizations make the correct decisions about selecting solutions for monitoring application performance that would be the best fit for their needs. However, the variety of their goals, selection criteria, types of IT environments and company profiles created a challenge of coming up with recommendations that would be truly relevant and actionable and not just a superficial overview of the market.

We realized that the complexity of the application performance space calls for a new report format and the idea for it came straight from our research. Many organizations are reporting that a Web browser is becoming the center of their IT performance efforts, as technology initiatives such as SaaS, IaaS and PaaS are changing how computing resources are being used and information is delivered to business users. As a result, we built a Web application for publishing the research findings that will allow end-user organizations to access information they really care about in a more interactive fashion.

With all due respect to good old PDF, there is too much complexity in the application performance monitoring market to fit research findings in a "one size fits all" publishing format and still make them actionable and relevant for different profiles of end-users. We have been advocating that information only has value if it is actionable and relevant so now it is time for us to "walk the walk".

If you are an end-user of IT performance technologies, we are interested in hearing your thoughts about the new report format before it gets published. TRAC will be hosting a series of demo presentations of the new report format and If you are interested in attending, please send a note to