What exactly does downtime mean for a capital markets firm? For voice trading systems, it translates to an average loss of $9.3 million per hour. A hefty sum, yet that’s just the immediate impact. Consider the hit you’ll take in customer loyalty and potential long-term reputational damage. Especially if the system goes down during peak hours.

Uptime is particularly critical for Fixed Income, Commodities and Currency (FICC) investment services, which are largely based on voice trading. From an IT perspective, this requires a reliable Unified Communications (UC) platform and advanced trading tools — dealerboards or turret voice systems.

Reliability is the operative word here, as FICC trading is conducted globally for 23 hours out of each work day. Plus, FICC trading has tighter regulations due to the financial instruments traded and the risk of speculation. That means firms need solutions to record email, text and voice to meet strict compliance requirements.

There is no room for error with voice trading, either. If compliance requirements cannot be met — perhaps due to a malfunction of the recording system – trading must be halted, meaning big losses. Any issues with transaction traceability, such as misplaced voice recording of calls, can result in hefty fines. And further risk of reputational damage if it becomes public.

In short, downtime is costly, painful and has negative long-term effects on investment firms. Unfortunately, the solution has been a manual verification process that is time-consuming, laborious and fairly unreliable. IT staff physically checks all components of the UC platform, including the Private Branch Exchange (PBX) as well as each dealerboard or turret system. Remember there is only a one-hour window when there is no FICC trading.

The Case for Automation

This manual process is not effective. All too often, investment firms first discover an issue when the voice system goes down during trading. Dead in the water, the IT team races to get the system back up and running. As a result, investment firms are looking to digital solutions that can automate verification to ensure uptime and compliance. They need to preempt the disconnect.

Let’s break this down by the numbers. UC voice systems offer a base reliability of no more than 99.5 percent guaranteed uptime under industry standards. That’s 29 hours of downtime per year – at an average of $9.3 million per hour for a whopping $269.7 million loss. More than a quarter billion dollars. Ouch!

Manual testing boosts UC voice system reliability up to 99.8 percent and outage time down to 12 hours per year. While that cuts the losses by more than half, it’s still $111.6 million in the red.

Automating the test and verification process brings reliability up to 99.99 percent minimum, resulting in less than an hour of downtime. In fact, it’s about 36 minutes, which comes to $5,580,000. That’s nearly 50 times less than losses from no testing and 20 times less than manual testing.

A proactive automated approach that increases the frequency and includes after-hours testing can bring voice trading system reliability up to ‘five nines,’ or just 5 minutes of downtime per year. It’s worth the extra effort, bringing downtime losses below $1 million to $775,000. That’s still not a small sum, but a mere fraction over what you’d lose with manual testing or simply relying on the service level agreements for your UC and voice trading systems with no testing.

Testing     Voice System Reliability   /   Downtime   /   Est. Loss

No testing     99.5%     /     29 hours     /     $269,700,000

Manual testing     99.8%     /     12 hours     /     $111,600,000

Automated testing     99.99%     /     36 minutes     /     $5,580,000

Proactive Automated     99.999%     /     5 minutes     /     $775,000

Further Automation Advantages

The case for an automated verification approach is compelling based on the numbers alone. Beyond that, automated solutions typically cost 60 to 70 percent less than manual testing. It frees IT staff to focus on more strategic tasks rather than the laborious and time-sensitive job of physically checking each device, feature and technology.

Automated verification is also more accurate, reducing human error. One of the biggest benefits is the ability to verify regularly and frequently. In fact, the industry will move toward a continuous verification model that can address issues before they become problems that can cause outage. That 99.999 percent reliability can go up.

Keeping Your System onPOINT

The dramatic losses investment firms experience during downtime all but demand a better solution. As our teams at tekVizion talked to customers in the financial services industry, we kept hearing the need to reduce this costly downtime and help ensure that firms could meet compliance requirements. Beyond testing interoperability for UC platforms and turrets, companies really needed more insight into their voice trading systems with a way to boost reliability.

At tekVizion, we’ve developed an automated solution to verify system performance called onPOINT. It’s become a popular product with service providers to help them meet the contractual uptime commitments in their Service Level Agreements. Based on the feedback we heard from investment firms, tekVizion developed an automated solution specific to the financial services industry called onPOINT FS.

A number of investment firms are using onPOINT FS to boost reliability for their voice trading systems and gain performance insight in near real-time to manage uptime. In all cases, they’ve been able to increase the frequency of verification and preempt potential issues that might have been costly to the firm.

Automation is rapidly changing the tech landscape, providing more efficient and cost-effective solutions. For investment firms, it makes tremendous financial sense to automate verification of voice trading systems. The automated solution costs just a fraction of what companies pay to manually test – and more importantly the results are much better. Given the hundreds of millions of dollars the solution will save, it’s worth a closer look.

To learn more about onPOINT FS or schedule a demo, please contact me at info@tekvizion.com

Israel Hersh is SVP of business development for tekVizion, a firm that automates verification of voice & UC systems. He has more than 30 years of high tech experience with a focus on financial services and voice trading solutions.