Manage Telepresence Server Overload As Seasonality Forces Employees to Work Remotely

It’s hard to forget the icy slap the “Polar Vortex fo 2015″ gave the north-eastern areas of the United States. In fact, this storm made nationwide news as temperatures smashed 120-year-old records and trapped thousands inside of their homes. Although weather radars indicated a “snowmageddon” of sorts, most companies didn’t even encounter a blip on their radar. 

After all, business must always go on. Right?

How You See It (a.k.a. Reality)

 How Your Boss Sees It (a.k.a Insanity)

Although many employees wish for hot cocoa, warm fires, and pajamas during major snowstorms they’re required to work remotely. However, what most people forget is that “working remotely” means a whole different thing for IT and Network teams. These teams have to work even harder when the weather gets rough. Not only are IT and Network folks forced to work from home, but they’re now required tirelessly to manage huge spikes in video conferencing traversal calls made by an influx of employees who are trapped in their houses due to poor weather conditions.

Their worries are not as simple as making sure to clean out grocery stores of milk, eggs, and bread like the rest of us. Instead, they need to make sure business must continue. 

How An Unexpected Influx in Remote Workers Can Max Out Telepresence Server Capacity

Liam McDonough, the Senior Collaboration Engineer at Cimpress (formerly Vistaprint), tells the Polar Vortex of 2015 story like none other. Since Cimpress is a global company, it didn’t matter than four feet of snow fell overnight and the rest of his state had closed. The Cimpress IT team had to stay up and monitor their video networks because of the horrible weather, Business couldn’t stop just because entire offices were working from home.

In our recent webinar, Liam McDonough recounts the morning of the Polar Vortex quite vividly.  “The first thing I realized at about 8:00am, after made myself a cup of coffee and looked at the wonderful falling snow outside, was once I logged into Vyopta and found 100% traversal and license registration usage for video calling.

Generally, we had never gotten up about the 65% mark until this, very moment.” He, and his counterpart Nick Parece then took a moment to kick back and admire the screenshot McDonough grabbed of their Telepresence Service Peak Capacity chart from their Vyopta control panel.

Screenshot: Cimpress’ Server Capacity During “Polar Vortex 2015”

McDonough went on to explain, “MeetMe, which is conductors and telepresence servers together, and this is us maxing out every one of those telepresence servers throughout the day. This graph right here was literally a lifesaver. This gave me a huge amount of detail on which server is going when, I was able to go and look at who was using our booked bridges, who was using what.”

By having the right information, the Cimpress IT team was able to drill down into the data and move Cisco Jabber-based video meeting resources into cloud-based resources, such as Cisco WebEx) so end-users could get their meeting off-premise and open up telepresence licenses for some bigger, schedule video conferencing meeting that were scheduled and the team knew would require a specific amount of capacity.  

Reallocating ACTIVE Telepresence Resources

Parece continued by explaining exactly how the team was able to quickly and efficiently reallocate telepresence resources without disturbing end-users.

1. Skim Real-Time Meetings: at each individual meeting taking place in real-time and identify the different types.
2. 1:1 Video Meetings: If it’s a 1:1: video meeting, you can quickly message the participants or specific meeting room using chat and ask for the video call participants to quickly hang up and call directly between two, specific endpoints.
3. Multi-Participant Meetings: If you’re encountering meetings with more than two participants, you can jump-in and message them and ask for them to please move to Cisco WebEx (or whatever video conferencing solution you prefer) to try and maintain capacity.

BONUS: Encountering Employee Resistance?

If you’re feeling resistance, there is something to be said for placing your face on the screen and asking for a favor when $%^& is hitting the fan. It may feel like a long-shot, but more often than not you’ll be able to showcase how attentive you are at doing your job and making sure that business can continue as normal for everyone. It’s also much easier to brush off someone over chat than it is to say “no” to their face.   

Reallocating IN-ACTIVE Telepresence Server Resources to Maintain Performance

By reallocating active telepresence endpoints/codecs, the Cimpress IT team was able to quickly able to get capacity spikes under control. However, this wasn’t enough to ensure the video network was able to support all employee activity. McDonough continued to explain still had a huge amount of video meeting registrations and was encountering capacity peaks from time to time throughout the morning.

As the day progressed, McDonough and Pearce noted that their T3 servers trend data in Vyopta’s vAnalytics (if you have this available) were not being used at all. So, they were able to manually move their T3 servers, change software, and moved it under the conductor to expand capacity. Although he did have other resources they could have leveraged, they would have caused issue somewhere else for the team to deal with later in the day.

BONUS: Long-term Telepresence Capacity Win

After this instance, the team moved their T3 servers for about two months only to put them back again because they learned just how successful this tactic.

Pearce continued, “We had more painful ways of increasing capacity. We had VM infrastructure that was running secondary and tertiary servers that we could have brought down to expand out our on-premise video bridging pool. This would have put us at a risk-point at a time where we already had a lot of risk with snow and power and people doing “snow things.”

One of the key things for us was that we’re able to track things because this wasn’t just for a day, it was over a few of weeks. We never had to use the painful option “C”, but were also very comfortable that we didn’t have to use it because we had the numbers to say we had 15%-20% capacity free with our initial reallocation of telepresence resources. We were able to not do the painful things and in the past, we would have started to bleed a little bit.”

Benefits of Telepresence Capacity Monitoring and Management

In short, had they not had the insights they had, “my initial response would have been to move everyone off of Cisco Jabber so those high-profile meetings to go” says McDonough. As we say at Vyopta, “more video endpoints, more problems!” By having the right tools and insight at your fingertips you can often bypass Network teams and fix video conferencing issues on your own.

Also, by actively managing your video environment in real-time during high-influx times like this, you’ll likely gain more credibility and respect for the future. In this instance of the “Polar Vortex 2015” The Cimpress IT team was able to monitor their video conferencing

• Happy CEO!
• No (or less) video call drops for everyone
• No (or less) user complaints and support troubles tickets
• Your sanity remains intact

Let us know if you have questions, or feel free to check out McDonough and Pearce’s webinar session to learn more about how they manage their global video conferencing environment. This two-man IT team was able to save their company $160,000 and improve communication with their Network team and resolve issues faster for a better QOS.