Video Summary

Although IT teams understand the value of network and video conferencing monitoring, it’s not uncommon to run into roadblocks when trying to communicate the value and details to other departments and executives. This video will cover how to get areas of the company outside of the IT organization to understand the value that effective monitoring brings to the company so that you can get your project and initiatives off the ground. Robert Hock (aka “Rob Boss”) spent 6+ years working for SolarWinds IT department, transitioned to SpareFoot to become their Director of IT, and now Hock is Vyopta’s Director of Product Management. He’s a rare medley of front-line experience and “executive speak” making him the perfect resource to help you figure out a way to “talk-the-talk” so you can cut through the red take and GSD for your IT department.

Watch and Rob Hock, Patrick Hubbard and Lean Adato tackle the topic of, “Making Network Monitoring a Priority.”

Video Transcript

[Music 00:00:00]
Leon Adato: Welcome. At last year’s THWACKcamp, we sat down with SolarWinds’s CTO, Joel Dolisy, to talk about ways that IT professionals could level up their skills when it comes to getting new software and hardware to help us do our job better. We focused on the issues that motivate business leaders, to make decisions on how IT pros can speak to those motivations.
Patrick Hubbard: We wanted to keep that line of conversation going by focusing on another area that can be a disconnect, and that’s especially how to get areas of the company outside of IT, whether that’s management or other technical teams, to understand the value that monitoring brings to the table.
Leon Adato: Joining us today in this discussion is my fellow head geek, Patrick Hubbard, along with another familiar face in the SolarWinds community, Rob Hock. After six years at SolarWinds, Rob transitioned to Vyopta as their Director of Product Management.
Robert Hock: Good day, everyone.
Patrick Hubbard: Yes, we’re really happy to have you here. You’re still hugely popular on THWACK and not just because they like Rob Boss, but we thought you’d be a great guest for this session because much like The Stig, the tame racing driver, you were a tame IT executive.
Robert Hock: You never saw me in my prime.
Leon Adato: Right, but you can’t seem to stay away from monitoring, even though you spent time on both sides of the fence. For example, now you are managing the product team for Vyopta, right?
Robert Hock: Yes, that’s correct. Vyopta is a cloud-based solution to monitor your video conferencing infrastructure. We provide monitoring and analytics around your endpoints and MCU bridges. Soup to nuts. Multi-product. One view. Single pane of glass.
Patrick Hubbard: Exactly, it’s familiar. Before that you were a superstar in IT here at SolarWinds for a long time and then you left and became the Director of IT for SpareFoot. What was interesting about that was you went from being in IT and products over to the other side of being an executive who had to juggle all the technology decisions for technology buying, manage the teams and then translate all that into something the C-level executives could understand.
Robert Hock: That’s correct. I became very versed in buzz words and fixing coffee.
Leon Adato: Right. Again, what we want to do is help the viewers understand what’s going on in the minds of management and also external technical teams when we’re talking about monitoring to make it matter to them as more than just this extra thing that they don’t know what’s going on. I think the first thing I would like to do is really frame up the issues that are at hand.
From the technical standpoint, I believe that there is a view from technical teams … Again, application developers or specific silo teams that monitoring is noise, that it really can destabilize the system, that your things is actually is what pushed over my thing or that you, the monitoring folks, couldn’t possibly understand the special snowflake that is my area of affect. I mean, is that accurate? That’s my view, but-
Robert Hock: Yes, from a technical team perspective, I’ve heard that a lot. The Heisenberg’s Theory of Monitoring, where, “Why is you monitoring breaking my system? Let’s turn it off and see if that thing happens again.” Well, you won’t know because you turned off the monitoring. Ergo, we’ve solved the problem.
Patrick Hubbard: Monitoring uncertainty principle?
Robert Hock: Yes, very much so. That helps frame the noise debate, where were getting words in the middle of the night or the monitoring is triggering on some counter that we’ve never seen before and the technical team may not necessarily even understand what that counter does or the downstream consequences of that counter. Let’s take an exchange mail server, for example. Out of the box, a toolbox SolarWinds will apply several dozen different templates to it and then be able to monitor each and all those counters. An Exchange Admin may not necessarily know what all those counters mean and then when you apply those templates out of the box, all of a sudden things go red, things go yellow. Now, that may seem like noise, initially, but as a monitoring professional, that provides a good segue to having that conversation with the admin. Say, “Hey, I don’t necessarily know what this means and you’ve already researched it so you know what it means, but I think we might want to dig into this a bit more and see why the cue latency is X and Microsoft recommends Y. So, maybe we should look at it. Sure, everything is looking great. Email’s working fine, but, you know, as a IT professional, maybe you might want to take a, take a peek at this.”
Leon Adato: Right and we’ll talk more about specific ways to address things in a little bit, but the problem again is that observer buys us the Heisenberg theory. All of that. That’s from the technical standpoint.
Patrick Hubbard: Yes. Once the team really does buys in and they have to discover the value first, but then you’ve come up against a management bias in a lot of cases that IT is already viewed as a cost center. This is one more cost that has to be justified and they don’t know whether it’s maybe a compliance issue or a CYA that’s driving it and a lot of times it’s hard for them to even understand that it’s working. They may not know the technology enough or you come at them with a whole bunch of acronyms to say, “Hey, look at my beautiful charts and graphs.” “I still don’t get why, why I need this.”
Robert Hock: Yes and “What do you have today and how is this different and why is it still working?” Going after a bunch of Best of Breed point solutions and trying to roll those through your CFO is never going to be a winning strategy for sure.
Patrick Hubbard: What’s the first step in starting that conversation? I mean, you can go at it from the, you try to get them to read maybe or send them white papers or something else, but when you just walk into the office and help them focus, what’s a great way to begin that conversation with them?
Robert Hock: Yes, throwing a Best Practices document in front of a senior exec is a non-starter.
Patrick Hubbard: It’s like, invite them to a SCRAM webinar while you’re at it.
Leon Adato: Right.
Robert Hock: Maybe someone’s lab. Maybe you ant to stop by?
That’s going to be a non-starter, but achieving visibility of monitoring is a discipline within your business unit, be it at the corporate level, division level, wherever that happens to be, but get your corporate leader invested in monitoring. Raise your level of visibility. That can happen a variety of ways. Alerting, although noise, potentially can raise the level of visibility. At least, initially, there may be a morale impact. Maybe you don’t alert the end tech guys that need to go take action, but you alert yourself and report and be able to report to that executive and say, “Hey, here’s all the issues that I’ve found over this week period with this tool. Do we want to talk about re-mediating it and what are the downstream consequences if we don’t take action on it?”
Patrick Hubbard: You recommend fire hosing a whole bunch of alerts at another uninitiated executive and say-
Leon Adato: Don’t do this at home.
Robert Hock: Exactly. That’s what we’re getting at. No, report on it yourself, alert yourself, package it nicely. This is where PowerPoint skills come in handy, I’m sorry.
Patrick Hubbard: It is unfortunate to say that. That if you want to speak to management, you’re going to have to learn their language which apparently is PowerPoint.
Robert Hock: Yes, but-
Leon Adato: No, that’s okay. I was going to say, one thing that you mentioned that really got me was the idea of morale. When earlier we were talking, you mentioned there were the 3 points that, I think, IT pros don’t really glom onto in terms of the pressure points or the mental views. Morale is one. There were 2 other ones that you had mentioned that I wanted to make sure we got out on the table.
Robert Hock: In terms of morale, do we want to dig into that a bit more?
Leon Adato: Yes.
Robert Hock: Our typical IT guy. We’re the guys in the trenches. We’re the firefighters. We see ourselves with just unlimited bandwidth. The problem is our wives may not or the organization as a whole realizes that your time is finite.
Patrick Hubbard: As IT we want to be helpful and that drives a tendency for over-commitment. No matter how much we say, “We’re going to manager our time,” or you walk in and say, “I’m only going to spend 80 percent of my time fighting fires and the other 20 percent is going to be strategic.” No, no, the firefighting feels good because we try to be helpful.
Robert Hock: Absolutely, you’re getting stuff done. You’re managing alerts, you’re managing your environment, but you may be running on a consistent treadmill and answering alerts late in the night on a Saturday night is not how you should be spending your time and if you find yourself in that position, you should find a way to not make that happen.
Patrick Hubbard: Or, I was going to say, or fighting fires during the day time and finding that 20 percent of the time, on the weekend to work on a PowerPoint presentation for management? That’s the part that is not second nature to us.
Leon Adato: Remember, what we’re talking about is that we’re the monitoring expert, but we’re talking about getting other teams to understand it. As much as we appreciate those alerts and we know that they’re meaningful or that their indicative, that the other team, that other group that’s getting that message at 2 o’clock in the morning, like you said, that’s the morale detractor. That’s the one that kills morale, because they didn’t necessarily ask for it even if they asked for it.
Patrick Hubbard: You could always set them up with a risk conversation, right? There are certain executives that respond really well to that.
Robert Hock: That may be a good wedge to drive the cost conversation. When we talk about risk versus cost and being able to show the risks that you’ve identified in the environment. Say, “Hey here are these 20 things that we should be monitoring, should be alerting on, that these teams should be taking action on and if we’re okay accepting that risk, great. If not, well, here’s what it’s going to cost to get it re-mediated or provide additional monitoring so we can take some sort of proactive, um, measures to address it.”
Patrick Hubbard: Then you have a risk ORI factor of, “This is the potential damage and isn’t monitoring a inexpensive way to reduce our potential cost?
Robert Hock: Exactly. I mean, 90 percent of statistics are made up on the spot.
Leon Adato: Okay.
Robert Hock: If you want to-
Leon Adato: And you can quote him on that.
Robert Hock: Exactly. If you want to take a downtime … If you want to be able to quantify your downtime. Say, if my messaging server goes down, we’re looking at $1,000 per minute of downtime. Make it up. Nobody can dispute that and as long as you have some-
Patrick Hubbard: I feel that.
Robert Hock: Yes, as long as you have some justification behind those numbers that is halfway plausible, it will be accepted. It’s order of magnitude off? That’s fine, but at least you’ll have a number that you can work off and at least have the conversation as opposed to no basis to go on.
Leon Adato: I’ll take a step back from making up the numbers completely fabricated is to say that, what I’ve said in the past is, “Based on what I’m seeing, it could be costing up to, you know, blank in terms of downtime or whatever. If the other team has a problem with that number, please, by all means, bring me your data.” Again, the idea is that you’re engaging in a conversation. It’s not that you were going to have the answer immediately, but that you’ve started the conversation going. “I think it’s $100,000 a year.” “That’s ridiculous.” “Well, I’m looking at this much downtime based on my monitoring tools and, you know, at this total loaded cost of staff and so on and that’s how I’m getting this number.” “Yes, but you missed the fact that-” Good. We’re now talking.
Robert Hock: In all seriousness, at this point, we’ve framed the conversation from a technical conversation into dollars, cents, business risk and that’s where your executive team is going to think about this and that’s how they’re going to address these issues. You’re not going to explain to them BGP or OSPF. You’re going to explain to them how downtime effects their business, their line of business and how that translates into dollars and sense. That’s the bottom line.
Patrick Hubbard: I want to argue … I want to say that if you’re worried about risk that you’re an old timer, but there are a certain of the new kids, let’s say, who may not respond to that. What’s another way to start that conversation? Could you talk about innovation or agility or-
Robert Hock: I think agility is a good one. With the rise of DataOps, now, IT organizations are almost expected to be more agile and certainly the larger the organization becomes, the less agile it’s expected to be, but your CIO’s going to these conferences and hearing about continuous integration, continuous deployment and he’s going to want your organization to be able to accommodate those.
Patrick Hubbard: DataOps in a box. It’s easy. You just buy it and-
Leon Adato: Yes, you just buy some DataOps.
Robert Hock: Yes, it’s a product I can pull off the shelf. You’re going to progressively hear more and more of that and your IT organization needs to be able to have an answer for that question and monitoring plays, I believe, a key role in that.
Patrick Hubbard: The thing about with agile environments and DataOps and anytime you have automated process, that’s the dream is always the thing we try to avoid is … We want quality. We want QA but we think that can never, ever happen but the key to Quality Assurance is actually measurement. Your monitoring systems are the ultimate bible of the quality of the DataOps process and whether your agile processes are working as expected, because that is either providing data that says it is working or it’s not. The quality of the changes we’re making are poor and it’s affecting our overall service delivery or it’s not.
Robert Hock: I couldn’t have said it any better myself.
Leon Adato: Even with the DataOps conversation, what I’ve seen out in social media and other places is the phrase is that, “DataOps without monitoring is a fail.” That’s also when you have executives who come to you and say, “Yes, but we need to be agile and, you know, we’re building a DataOps culture.” Yes, monitoring is part and parcel of that DataOps culture. Absolutely.
Okay, I think we’ve done a good job of framing the mindset and we’d started to touch on some of the ways to address it but I want to really start to dig in to some of the ways that people can do that. I want to really focus on the answers. We’ve got the problems sorted out. I know that for a lot of our viewers, the management angle is what really is the nut to crack. They’re very comfortable talking to technical teams, they’re very comfortable pulling out the metrics and statistics for, again, the storage team or the virtualization team, but I think when they walk into what I affectionately call, Mahogany Row or Oak Avenue, and they have to start talking-
Patrick Hubbard: Where all the white noise generators are?
Leon Adato: Right, exactly. Yes.
Patrick Hubbard: When you get a ticket. “Come fix the white noise generator.” You’re just, “Ahh.”
Leon Adato: Yes. Yes.
Robert Hock: “Sir, you’re going to need an escort.”
Leon Adato: Yes, right. Exactly. That area.
All of a sudden there’s a whole different language going on. How do we talk to management and help them understand that monitoring is not just the checkbox for compliance? That it actually serves a purpose?
Robert Hock: We started down that road a few minutes ago. So one is visibility and if you walk into Mahogany Row blind with no ammunition, it’s going to go nowhere. If you try to hand over a PO for a brand new monitoring tool, let’s say, to do APM on this brand new system that’s going to generate a bunch of money for the organization, even though you know it’s a sound investment, unless you can raise the visibility of monitoring as a discipline within the organization to the measurability aspect, they’re not necessarily going to care.
Patrick Hubbard: You can snap those individual requests into a larger story that they’re aware of and feel that it’s a valuable part of the organization.
Robert Hock: Very much so. Although old school, you have knock views. Knock boards. Great way to go. Stick them outside your CIO’s office. Green is good, red is bad. To the parable before about setting up a new revenue-generating application, what is 2 milliseconds of response time mean versus 4 milliseconds of response time? What is the dollar value of that and can you display that in big, bold letters outside his office, because that’s going to translate into something he cares about.
Patrick Hubbard: Especially, where you have those other program owners at the point of interacting with other technical teams where they’re going to stand up a new application. If they want to get some glory for their app, it needs to be on that dashboard that lets you also start that conversation with this other team to say, “Yes, would you like us to monitor your application and include it on our dashboard?” Then they will say, “Of course, we would love that.” That opens that channel to then be able to get some of the administrative access that you’re going to need to be able to extend monitoring to something that otherwise was a black box.
Robert Hock: Yes.
Leon Adato: Part of that conversation with the application team is, “Can you help me quantify this?” I don’t want to show if it’s up or down or good or bad or slow or whatever. We want to talk about the financial impact so that we can have that on the dashboard. It’s just math, right? Again, when the exec says, “That’s impossible. That doesn’t cost us-” “Well, actually, let’s talk about that.” That’s definitely one way to get it going.
Also, we talked about understanding how monitoring fits into the organization and why wasn’t what I had before doing it now? I think there is a case to be made for what I call, a Feature Function Matrix, but it’s a list of, “Here’s all the tools that we have. Here’s all the features that we need done. Here are the gaps. Here’s the overlaps.” My question is whether that’s compelling to management or if that’s just my own technical checklist. Do I have all my bases covered?
Robert Hock: No, I think that’s entirely valuable because as everyone knows well when they walk into ask for money, the first thing they’re going to hear is, “Well, what do we have today and why isn’t it working?” Then the second question is going to be, “How does this overlap with the tools we have today?” And being to justify that overlap.
Patrick Hubbard: What can we get rid of to fund this?
Robert Hock: Precisely.
Leon Adato: Brick in. Brick out.
Robert Hock: Very much so. I think something like that is entirely valuable to frame the conversation. Then it goes from technical, which is the Venn diagram of overlap into the dollars and cents and what does this actually mean from a positive perspective of the organization.
Leon Adato: Great. I think there’s an opportunity right now, and we started to edge into it with DataOps, but I think there’s an opportunity to frame the conversation about monitoring, because it’s actually not all the same. You mentioned before, Patrick, about old timers. I don’t think it’s old timers anymore. I think that the landscape.
Patrick Hubbard: Or just conservative. If you’re IT for a financial services company, you might be a very young person with a lot of great ideas and energy and excitement about new technology and clean sheet re-implementation and you want to start down that path but the requirements of the business are going to make you operate in a little bit more traditional manner.
Leon Adato: Right, but with all of that said, I think that the nature of what we’re seeing in technology allows you to go in with a, “It’s not same old business.” I don’t know if that-
Robert Hock: The landscape is certainly is changing. Obviously, the technology refresh cycle in, let’s say, 2 years max. What’s old is new again, potentially, in the mainframe world and those problems aren’t going to go away. You could be a burgeoning, young professional that does have a slightly more conservative, we won’t say stodgy, environment that does have compliance concerns, change management. These things aren’t going to go away and then on the other end of the spectrum, you have the hot, young startup web-based application, containerization where APM is going to be a bit more important than monitoring your ZOS.
Patrick Hubbard: Right.
Robert Hock: Somewhere in between there is the traditional enterprise and I would say that that is actually shifting more towards the hot startup agility, in terms of the integration continuous deployment and the paradigms, not to abuse a phrase, of monitoring are shifting.
Patrick Hubbard: Do you find that it’s easier in some cases to start that conversation if an IT executive comes to you and they just got back from a conference or read some blog article about the hot, new thing and they’re like, “Hey, we heard this thing Cloud is really great. It’s going to save us a ton of money or we’re going to move everything to SalesForce and I’m sure it’s going to be just fine. We don’t need to upgrade our internet connections.” Is that a chance to take their enthusiasm for technology and steer it back to, “Yes, but you still have this huge investment in infrastructure to take care of?”
Robert Hock: Absolutely. I call that the Squirrel Theorem. That squirrel, right? Harness that energy. Subscribe to CIO Quarterly. They have an application that you can put on your phone if you don’t read magazines. Read what your leadership does so you can be prepared to have those conversations ahead of the leader driving it. You can actually, practically, go to them and say, “Hey, how was Microsoft Ignite? Anything interesting there? By the way, there’s these few things I’d like to talk to you about that may be of interest to our organization.”
Patrick Hubbard: “How did you know I went to Ignite?” “I read your email.”
Leon Adato: Oops!
Patrick Hubbard: You can also do little things like send them a link every now, right? To let them know that you’re looking at the same sorts of things that they are, to assume that you are in the mood to have those sorts of conversations and you’re not just down there doing all that technical stuff, whatever it is that you do.
Robert Hock: Absolutely, you can be seen as a business leader and a force within the organization as opposed to the guy who sends alerts out and wakes people up out of bed.
Leon Adato: Right and at least business interested and I think that’s another message that we’ve gotten over the course of these types of conversations is that no one’s asking the IT pro, the monitoring engineer to suddenly be interested in ROI and EBITDA and all those financial terms. Nobody wants that but understanding that the business drivers and being visibly engaged in those things. Understanding that stability is more important at end of month when there’s closings or this application which runs hot in the middle of the month, you’ve got the data to actually know that and being sensitive to it shows that you are sensitive to the business without having to make the transition to the dark side and get into management or something like that.
Robert Hock: Along those lines, having all that data does put you in the catbird seat to have that conversation since many of the other technical teams in the organization may not. IT is generally seen as a cost center, obviously, but by having the data, it enables you to potentially have the conversation of translating into revenue generation. IT as a service. Being able to do chargeback. Being able to do showback. All of that’s impossible in the absence of having the data. As we see IT organizations progress, that’s, I don’t want to say the holy grail, but maybe the next step in the evolution of IT departments where it seems an internal service. Obviously, cloud adoption is ramping rapidly, so, being able to position yourself as internal IT, getting ahead of that and saying, “Okay, well, AWS is interesting to us and here’s what it would cost if we were transitioning these services, but internally, we can quantify what it would cost us today and then be able to show the cost savings over time.” Having the data is really critical [crosstalk 00:22:08].
Patrick Hubbard: That’s a really interesting point because if you think about it, you’re really … If you don’t have the data, you’re at a disadvantage to all of the other parts of the organization, right? Marketing, sales, operations, facilities, production. Any of those other parts of the business are going to be generating piles and piles of data to measure their effectiveness and if you can’t come to the same meetings. If your executives don’t go to their E-staff meetings armed with the same types of data for IT, they’re automatically going to have a squishy feeling about what it is that you do.
Robert Hock: By data we don’t necessarily just mean CAMP reachability, right?
Patrick Hubbard: No.
Robert Hock: Let’s do-
Patrick Hubbard: It needs to be rolled up and aggregated a little more than that with some treads.
Robert Hock: One might say. Just ping status, not going to cut it in this day and age. Being able to look at monitoring from a service delivery perspective. It’s interesting that the web server is up but is it actually delivering the application to the end users and what does their experience look like? Response time translates to dollars in this day and age.
Patrick Hubbard: Remember, you as IT have numbers that nobody else has access to. You knew when you added and removed hardware. You have an eye on capacity. You knew when your spend hit the budget. With a little bit of research, you can do things like show capacity planning, actual capacity and performance charted together. You can say, “This investment actually resulted in improvement and here’s an area where maybe we had a leaner quarter and we pushed off a purchase and watched our performance for an application actually decline and then expand back up again with investment.” You can show that continuous investment results and improved performance.
Robert Hock: To that end, we’re talking about going to the business armed with data to justify purchases. The feedback loop there is also equally important. To be able to close loop and say, “Hey, we made this investment in monitoring hardware, whatever the case may be, and here’s what we forecast it was going to result in. How did we do? How accurate were we?” That will buy you credence with executives going on to be able to justify a future purchase. “Well, we were accurate the past 2 times, what makes us think we’re not going to be accurate now?”
Patrick Hubbard: Is it frustrating when you are in the exec seat of you get constant requests for hardware or for services? You’re writing a lot of checks and then something just goes off and you never get any feedback of whether that was a pre-” I mean, no ones first thing going to say, come and say, “Thanks, that was great. Really appreciate it,” but it’s got to be really frustrating if you’re not getting that feedback loop to say, “I’m continually investing in this thing and I never hear anything back.” Monitoring is a great way to prove the value of those checks.
Robert Hock: Yes, it’s a direct feedback loop. I cut a check. What does it buy me? Was it implemented? How are we using it? Did we get out of it what we thought we were going to get out of it? By monitoring … Having that visibility into the infrastructure and providing that feedback loop, the infrastructure to management- [crosstalk 00:24:54].
Patrick Hubbard: I was going to say, to close that thought off, the other nice thing is that it gives you a chance to show up and say, “Hey, Boss, thanks so much for, um, for supporting our infrastructure.” You, actually, instead of always being the apologist or always being the one called on the carpet when something breaks that you had no responsibility for. They were working with a Front Loader out in the yard-
Leon Adato: Backhoe.
Patrick Hubbard: Back, back throws-
Robert Hock: Bring in the guy I yell at, again.
Patrick Hubbard: You get to now start showing up and saying, “Hey, thanks so much. Appreciate it. By the way, here’s the upside for the organization.”
Leon Adato: The other thing that I think IT pros miss is that when you are reporting up to management, even C-Level executives, they want to be able to go tell other people about this thing. “Hey, you know, my, my, my, my staff here told me this is a really good investment. I believe them because I’m, because their my staff and, you know, I really love their technical expertise and we went with it. We did it and here is what the result is.” They want to be able to talk to either their upper management or to the rest of the crew because that’s a morale booster.
Robert Hock: To that end, a really great data set, if you can get it, is, let’s say, a baseline of data from other organizations that maybe in your particular vertical. If you can get that data for your boss, you’ll be golden. To be able to say, “Here’s how we look from an adoption perspective compared to, uh, other financial services companies or here’s how we look for web response time compared to, uh, other health providers.” Whatever the case may be. He can take those middle level metrics and socialize them with his peers so when he does go to those C whatever meetings, I can say, “Well, Jim, looks like your response times a little slow this week. What’s going on?” Give some ammunition when he’s talking to his peers.
Patrick Hubbard: That’s always popular, but shiny and new. Always a good thing to talk about, right? What are you seeing in terms of trends now? We’ve talked about for our customers, they’re increasingly having a challenge with managing Hybrid IT environments. What are you seeing really changing from the executive side in IT, let’s say, over the next 12 to 18 months?
Robert Hock: I think the tide has shifted with DataOps and Hybrid IT and that’s no longer on the horizon. That’s a here and now. When we talk about that spectrum of organizations, the middle part of the bell curve right now is trying to figure out what Hybrid IT means to them. Various disciplines in the business maybe working outside of IT. So called shadow IT organization. Just spinning up services that may be SASS. They may actually be platforms. You may have marketing going off, spinning up AWS in essences.
Leon Adato: [inaudible 00:27:35]
Robert Hock: That’s a terrifying thought, but if IT isn’t seen as a driver of the business or at least an enabler, you will see that behavior occur more and more. That’s something to watch for.
Leon Adato: It seems that understanding that parts of the business have the wherewithal, they have the budget and whatever to suddenly spin up their own whatever it is system on AWS or Azure or what have you. Being prepared for that means that you say, “My monitoring tools,” because we’re here to talk about monitoring,. “My monitoring tools are able, are ready to address that or ready to fill in that gap.” They don’t have to go off into the wild in that shadow IT method and that way you socialize that.
Robert Hock: Yes.
Patrick Hubbard: Is that an opportunity to start a conversation? Give you an example. Marketing, you gave this example before, in particular, likes to find new solutions based on something that they have heard or can prove have worked for other organizations and a lot of the times they’re SASS. It’s not even I’m going to stand up AWS or Azure [inaudible 00:28:33] and build it myself. I’m just going to subscribe to this SASS service. Now, you have moved mission critical services off to a SASS provider. You need to assure that that’s working for the business, but that person may not even know that they need to have monitoring on that service to make sure that it’s going to work. Is there an opportunity to propose that as a way of managing that random IT spend or people who are going off and investing in services and not talking about it, because that’s the risk to the CIO.
Robert Hock: Yes, it’s a good foothold to have that conversation. Certainly, IT is going to be the one to get the call if there’s an issue with that service. Whether or not you can affect change is a different story, but you’re going to get the call. Pivoting that into a monitoring conversation saying, “Is this provider living up to their SLAs, um, so we can at least go after them and, by the way, in order for me to have full visibility, I need some administrative credentials for this application so I can sign into it, be able to come up with scripts to pull metrics out of it. Whatever the case may be.” I think it’s a good segue into having that conversation.
Patrick Hubbard: To address the risk, or their concern about risk, you can say, “Hey, I know you’re worried about these things. If someone adopts a service, I can monitor it. Just send them to me and I’ll take care of it.”
Robert Hock: Yes. The leader in your organization is going to be much more comfortable if you were able to go to them preemptively and say, “Don’t worry. We’re already set up with X, Y and Z to handle this when marketing comes to us with this request,” versus getting hit sideways in a meeting where you’re just not prepared.
Patrick Hubbard: They’re the ones who are going to have access to that shadow IT. They’re going to be aware of that shadow IT purchase, because they’re in the meetings where that was funded but you in IT may not know anything about it.
Leon Adato: Right and this is a good place to start to dovetail into the second half which is we’ve talked about addressing managements concerns or helping management become more aware of how monitoring matters to the business, but what we are speaking to here is to a level of confidence that external teams whether it’s marketing or storage or what have you, but making it our monitoring. Making us matter to them so they’re not ignoring it. That they are coming to us and saying, “Hey, I really need this capability.” It could be a SASS, could be a hybrid answer. “I need this capability. What can you do to help me make sure that it’s reliable and stable?” Coming to us first. Let’s talk about that.
I’m just going to start off an idea which is I see a lot IT pros, a lot of monitoring professionals jump in and they turn on the whole solution … We’ve talked about this before. There’s that fire hose of alerts and they’re relatively generic, they don’t really-
Patrick Hubbard: Monitor all the things.
Leon Adato: Monitor all the things. Alert all the things, all the time and I would caution that you want to deal with that stuff one alert at a time. Just start with one and say, “How’d that work for you?” That’s the first thing.
Patrick Hubbard: How did it work for you? How do you want to consume it? What’s sort of notification is effective?
Leon Adato: Right.
Patrick Hubbard: What kind of triage working for you?
Leon Adato: Right. You mentioned this earlier and I just wanted to reemphasize it. You said, “Don’t send the alert to them first.”
Robert Hock: Yes, you definitely don’t want to alert the end practitioner right out of the gate. In fact, that will do nothing but assure a rift between the teams. When you’re testing alerts, when you’re testing new technologies, whatever the case may be, alert yourself first. Be able to maybe even report ahead of that for issues and then have the conversation during normal business hours. Maybe informal, maybe structured. “Here’s the things I’ve found.” It’s not accusatory. It’s just, “Here’s what’s going on with the environment. Should I be concerned about these things? Should you be concerned about these things? I don’t know,” but I do know because I researched it ahead of having this conversation and it helps you break down the silos between technical disciplines and finds you more visibility, more credence as a monitoring professional.
Leon Adato: Right and I also think that when you come to them and say … They were up at 2 o’clock in the morning because the system failed. You know that because you got the alert. Okay. You didn’t send them the alert first because they don’t know it, but I have had wonderful conversations where they said, “Where did you get that? Where are you getting that information from? Oh, well I have the system that I’m doing for my own stuff or my whatever it is and I just happened to see-” “Really? Could you hack some of that stuff for me?”
Patrick Hubbard: You said something the other day that was interesting about having the wrong data sometimes is a great start because at least it facilitates that conversation.
Robert Hock: Yes, absolutely. Even if the data you’re pulling happens to be incorrect. It’s the wrong SMNP counter because the template was wrong or whatever the case may be, at least opens that door and you can go to them and say, “Hey, I see fan speed on this is zero, that seems odd.” “Well, no, it’s working just fine.” Okay, well, let’s dig in from a technical perspective. You find out the firmware on this thing is different or whatever the case may be. It ingratiates you with that team. Starts to break down those silos and then moreover starts to establish your monitoring system as the system of record. You’re able to look across silos, across teams between storage, database, networking systems and you become the system of record. If it exists, it should exist in monitoring and people start to come to you for monitoring requests, data requests.
Leon Adato: Right. There’s also the, “Hey, I noticed the fan speed was zero.” “Wait, you can get fan speed? Wait a minute? What else can you see?”
Patrick Hubbard: “I have a whole lot of other things I’d like you to see.”
Leon Adato: “If you can get that then maybe you can-” They didn’t even realize that that metric was possible to be collected. I also see that wrong number. It doesn’t have to be inaccurate. It can be the wrong one and we talk a lot on Lab about this that in Linux, there’s 2 ways to calculate RAM and there’s the one way that’s built into a lot of systems and then there’s the other way that a lot of Syst Admins want. Coming to them and saying, “You know, this is the RAM value.” “Oh, that’s the wrong one.” “Oh, do you want me to switch it? Because I can switch it.” As long as your tools are able to pivot like that. To be flexible. Again, I think that starts the conversation technically.
Patrick Hubbard: Yes, and if you want that fix, vote it up on THWACK.
Leon Adato: Right and there’s some templates and right. Exactly. Thank you. Gone but haven’t forgotten. That’s great.
Patrick Hubbard: We talked about something the other day and that was how much executives just love receiving lots and lots and lots and lots of help desk tickets. Wherever possible don’t use any automation. Don’t use your monitoring to streamline your processes.
Robert Hock: [inaudible 00:34:53] sense of sarcasm.
Leon Adato: Again. There’s-
Robert Hock: Don’t try this-
Patrick Hubbard: Instead of having 10 people … Don’t have 2 help desk specialists. Have 10. That’s much, much better.
Leon Adato: Yes. Hire lots of eyeballs.
Patrick Hubbard: Yes. No. That automated action is important.
Robert Hock: In terms of, let’s say, automated remediation? Yes, obviously another value that you as a monitoring professional can provide to the organization. Let’s say, something goes bump in the middle of the night and you can automatically reboot a server or, moreover, alert a different team if you’re able to scale it horizontally, that can take care of that in the middle of the night that doesn’t necessarily need to be your tier 3 person. Yes, there are plenty of ways that you as a monitoring professional with a properly configured alertive infrastructure can save a bunch of money.
Patrick Hubbard: What else can you automate? Baselining?
Robert Hock: Baselining and not just baselining. Baselining’s great for sculpting the alerts, right? To prevent those things that go bump in the middle of the night from going bump in the middle of the night, but also from a capacity perspective. To be able to alert proactively on things that will become a problem. You can alert during normal business hours about this thing that will become a problem in a week form now or whatever the case may be and being able to report on those instances and be able to show that back to the organization and say, “Here’s what we prevented in terms of, let’s say, downtime, because remember what happened last time this occurred? It cost us X amount of money.” That’s very tangible to the leaders in IT.
Leon Adato: I think that’s important as monitoring professionals to understand the difference and help you consumers, once you engage them, understand the difference between an alert and a report and a screen and I think a lot of times everyone goes straight to the ticket, because that’s the trackable thing, but making sure people understand that I can tell you at 2 o’clock in the morning when it broke but I can tell you at 2 o’clock in the afternoon on every Thursday where it’s at so you avoid that 2 o’clock in the morning … Because you’ve gotten the report to know what’s going on.
Patrick Hubbard: I was going to say, the one thing that we definitely want to, before we wrap up, we want to get to and that is a big don’t which is, it goes to the fear that a lot of people in IT have, which is do not suggest monitoring for a second is going to replace their management tools, because a lot of times they would and they should push back on that, because the management tools are working at managing infrastructure. They actually work together.
Robert Hock: Yes, I would say that’s absolutely true, particularly, as you’re trying to ingratiate yourself in the various silos that exist technically in your organization. They have their own tools. They have their own vendor tools that they use to great affect to effect change on various devices. You’re not going to replace that and if those tools had built-in monitoring capabilities that you have on your side, they wouldn’t need that. However, it’s a heterogeneous environment and you need those solutions.
Leon Adato: Yes and even if their management tools have monitoring, that’s a sanity check. The management tools, given the forensics, the deep dive, the more intensive scrutiny and not only that it shouldn’t replace it but you want to make it clear that you have no intention of replacing it.
Patrick Hubbard: It’s also showing them how to use those tools more effectively, right? It’s showing whether the quality of the changes made by those tools much more quickly and comprehensively than if you have to do it manually.
Leon Adato: We talked earlier about single pane of glass. Not only with their management tools they’re seeing that particular area with a good, sophisticated monitoring tool. They’re seeing not only seeing that, but the impact for other areas.
I just want to wrap up by pointing out that at the beginning of the session I talked about how monitoring is seen as noise and how it’s viewed as an interruption or possibly a disruption. When done right and I’ve seen this and it’s wonderful, that monitoring goes from a “Do we have to?” That teams are like, “If I have to install, if I have to enable that, fine.” To, the teams coming back and saying, “Your high vilability … Your disaster recovery level in our corporate structure isn’t high enough.”
Patrick Hubbard: I think it’s where they can’t live without it.
Leon Adato: Yes. They absolutely … “You need to, like, bring it up.” That is, even though it’s a little bit of a challenge for monitoring professionals. “How do I make this HA? How do I whatever?” That is a compliment and that means that you’ve done all of what you’re talking about right.
Robert Hock: Very much so. Monitoring is not a destination, it’s a journey and if done appropriately, you’re going to get to the point where something really obscure is going to go bump in the middle of the night. Somebody’s going to come to you and say, “Why weren’t we monitoring that?” That’s actually a really great place to be because it means you covered all your bases and now you’re just chopping down that long tail of issues to go monitor. You’ll never be done.
Leon Adato: Rob, thank you so much for joining us.
Robert Hock: It’s an absolute pleasure, gentleman.
Patrick Hubbard: It was great having you and we hope you enjoyed this session. Please keep your feedback coming.

Pin It on Pinterest