In Gordon Ramsay’s latest reality TV show, “24 Hours to Hell and Back,” chef Ramsay and his team pick a failing restaurant and, over the course of 24 hours, use a combination of training, physical changes and tough love in an attempt to get them back on track and save them from themselves. The theme of every episode is teamwork and the distinct roles everyone plays from owner to head chef to server; those not pulling their weight are often asked to leave. Why talk about a cooking show? Because like these restaurants, many in the security industry have suddenly realized — almost overnight — they are in trouble when it comes to how they handle the issue of cyber security. And while they may have started out by pointing fingers at others, such as the end users or “IT,” they have now come to realize that the only way forward is teamwork and making sure each participant in the security chain plays their role to the best of their ability.

“Cyber threats can no longer be ignored,” says Morgan Harris, senior director of enterprise solutions, ADT, Boca Raton, Fla. (SDM’s 2017 Dealer of the Year). “Typically we hear about those breaches affecting millions of consumers’ personal data or financial records; but more and more we are finding common security appliances, such as in-home cameras, being subject to hackers. … Attacks are being launched through unprotected or under-protected networks that host security appliances such as IP cameras, access control card readers and even IP-enabled intrusion detection panels. The security industry has taken notice; but we are still at the beginning stages of addressing the problem.”

That is a common theme across the industry, where descriptions like “in its infancy,” “on the right track,” “lagging behind,” and “overwhelmed,” are commonly uttered phrases, even amongst the most cyber-savvy. But one person who is more optimistic than most, ironically, is Bill Bozeman, CPP, president and CEO of PSA Security Network, Westminster, Colo., who has been pushing the importance of cyber security to the industry for several years now. “I think the attitude is much better. The fact that it is a huge problem for all of us is generally accepted industry-wide, which is a nice change.

“Maybe I am a little more optimistic simply because we were so early to the game, Andrew [Lanning] and I. I feel a little bit better about it. I spoke with someone on the insurance side and we are up to 70 percent of integrators that have cyber security insurance. Last year it was probably 45 to 50 percent. It depends on your expectations and how quickly you want to get things done, but I think everyone is trying like crazy. I wouldn’t call it the infancy. That was the denial stage. We are way past denial. We do still have a whole lot of ‘What the [heck] should I do?’ but at least they want to do something,” Bozeman says.

Others, such as Brad Hedgepeth, manager, technical services, G4S Secure Integration, Omaha, Neb., describe a feeling of being overwhelmed, in part because of the perception that the security industry is lagging behind the IT industry and struggling to catch up. “There are no shortage of vectors in which the industry is being braced with messaging on risks, best practices, technologies and services. While the larger IT industry had years to gradually absorb increasingly sophisticated technologies and adapt a cultural shift in information security, the physical security industry has found itself trying to navigate a rapidly evolving landscape.”

Until recent years, security systems lived in their own bubble (and often on their own network). But now that everything is IP-enabled, cloud-based and/or residing on shared networks, security industry players are learning that even if their technology doesn’t contain anything worth stealing — such as access control cards that only have a random number attached and no personal information — that doesn’t mean they aren’t worth hacking. 

“The threat landscape has really changed … as the devices we use in the industry, in our offices and in our homes have increasingly relied on internet connection,” says Andrew Jamieson, director, security and technology, Underwriters Laboratories (UL), Northbrook, Ill. “Many people still consider they are not a target as they have no assets worth the effort to compromise; but the processing power and functionality of these systems is often an asset in and of itself. … It’s no longer a case of ‘I’m not worth hacking,’ as every device is an asset to some malicious party. These threats can often be external to both the manufacturers and users of the devices.”

This was the case with one of the most famous DDOS (distributed denial of service) attacks to hit the industry to-date: the Mirai Botnet in 2016. Mirai didn’t happen as a result of a malicious actor, or nation-state, targeting a weak security system. It was, according to a December 2017 article in Wired Magazine, the result of a couple of college kids trying to take down a competing Minecraft server. In doing so they unknowingly unleashed an army of bots that went searching the internet for any devices that still had the manufacturer’s default settings. Unfortunately for the security industry, many did.

“In the past, the attitude of many vendors was that the customer was responsible for securing their network,” says David Brent, network video and cyber training engineer, Bosch Security and Safety Systems, Fairport, N.Y. “The Mirai DDOS attack was a wakeup call for vendors who had not yet taken the issue of cyber security seriously. The attack leveraged over 540,000 IoT devices — mostly IP cameras and DVRs. Now, customers are making choices based on security, and as they become aware of the risk that their security system could potentially be turned against them, they are making changes.”

These more sophisticated end users, who are all fearful of being the next Target or Sony, are pushing for change from their security vendors and security integrators. “It’s on everyone’s mind, and those that are embracing it most seem to be all-in,” says Henry Hoyne, CTO, Northland Controls, Milpitas, Calif. “Today’s clients have sophisticated IT infrastructures and demand that devices that reside on them adhere to their strict requirements.”

Yet, Hoyne adds, every link in the chain matters when it comes to cyber security. “Everybody must do their part, and the onus cannot be placed on any one entity.” 

Jim Hoffpauir, president, Zenitel Americas, Kansas City, Mo., agrees. “The ‘security industry’ is an ecosystem of product and service vendors, consultants … and integrators. The entire ecosystem is often at the mercy of the weakest link, which includes the end user.”  

If it takes a village to ensure a cohesive and cyber-secure environment, let’s take a closer look at what steps are being taken at each level of this security ecosystem, along with some of the challenges each has along that path.

The Manufacturers

The main cyber security efforts major security manufacturers of hardware and software have implemented are procedures and processes to harden their systems, and — critically — publishing hardening guides and offering training to security integrators on how to install and set up these systems to be as cyber-secure as possible. 

Additionally, as systems get more open, partnerships are formed, and there is generally more sharing of information across the board, many manufacturers are reaching out to each other to coordinate efforts. 

“As part of our commitment to cyber security, Mercury takes a number of additional steps to help partners and end user organizations create cyber security systems,” says Matt Barnette, president, Mercury Security, part of HID Global, Long Beach, Calif. These steps include third-party penetration testing, hardening guides as well as intelligence gathering. “With the industry’s largest installed base of four million panels deployed … Mercury leverages our extensive market coverage to collaborate with leading access control manufacturers, systems integrators, consultants and end users for intelligence-gathering. This comprehensive reach throughout the global access control community provides Mercury with broad insight across nearly every vertical market and geography as it relates to cyber security best practices, potential threats, reporting and communication methods.”

Vulnerability testing is something that is critical for manufacturers, says Eric Widlitz, vice president – North America sales, Vanderbilt, Parsippany, N.J. “Vulnerability testing puts a product through its paces, and once weaknesses are exposed, they can be patched up and the cycle of attack-and-defense can take place again until eventually a watertight ship is in place and ready for market.”

But the release to market doesn’t end the process, he adds. “Testing must continue long after a product is introduced to the market and constantly updated through manufacturer firmware updates.”

Jamieson also stresses the need for post-release testing. “When you buy a computer or mobile phone, you generally know about the security posture of that system and how long the operating system in that will be supported, and it’s part of your purchase decision. We need similar thought to go into purchase of security and life-safety products and systems. Therefore, we recommend that manufacturers ensure that their products are developed securely and have a process for maintaining/updating cyber security utilizing best practices and that they make this clear to their customers so they can understand the value that this brings.”

Manufacturers should, at a minimum be making sure what they provide to security integrators is as hardened as possible; but ideally it goes further than that, says Joseph Gittens, director of standards, Security Industry Association (SIA). “There is a chain of cyber-accountability throughout the security ecosystem. From the time a network-enabled product leaves the factory it should be secure on all layers and include clear instructions on how the devices should be provisioned for network security. Manufacturers should also have resources available for bugs to be reported and, subsequently, the latest firmware and patches to be downloaded.” 

This kind of attention to the manufacturing process doesn’t go unnoticed by security integrators. Ryan Loughin, president, NextGen Security, Exton, Pa. (SDM’s 2017 Systems Integrator of the Year), recounts going to a recent conference where the CEO of a major software manufacturing company spoke. “[He] spoke one time during the four-day event. What do you think he talked about the entire 90 minutes? Cyber security and what they have done to make their solution set the most protected in the industry. That is a good sign that it’s getting attention at the highest level of their organization. … If they are relying on the integrator to have the awareness and protective measures then it’s not a good business practice in today’s market. This is why we work with the manufacturers we do.” 

Christine Lanning, president, Integrated Security Technologies Inc., Honolulu (featured on this month’s cover), agrees. “As the industry developed technology at a crazy pace the emphasis was on how we have to get out the latest and greatest, and nobody was taking the time to test the systems from a cyber security perspective. It’s harder to add cyber security after the fact.”

For Ryan Zatolokin, business development manager, senior technologist, North America, Axis Communications, Chelmsford, Mass., the key is to have good communication between the manufacturer, the integrator and ultimately the end user. “We need to educate on best practices, make documents available [and] be transparent if we have a vulnerability … We see our role as making it easier to maintain systems that are cyber secure and that will help drive integrators and end users to making those changes. Essentially, we have eliminated the amount of time it takes to harden them and published a variety of additional information that goes through step-by-step what you can turn off and on in terms of services, best practices and using the tools we have to quickly implement that across hundreds of different devices.”

Genetec, Montreal, is also very focused on bringing awareness and education to the market, both to security integrators and other manufacturers, says Christian Morin, vice president integrations and cloud services, chief security officer. “We need to lead by example, and as a manufacturer be more responsible and show the way to other manufacturers. In all fairness, if you look at Genetec five years ago cyber security wasn’t a big thing for us. We didn’t crack down on it and we made mistakes. We shipped equipment with default passwords or open ports. But once we realized the issue, we started changing our philosophy. We have decreed that products have to be secure by default and have an opt-out process rather than an opt-in.”

Genetec and other manufacturers are also starting to do what they can to make the integrator’s job easier, Morin says. Whether that means shipping products with the highest security settings, forcing integrators and end users to turn on or off things that would make it less cyber-secure, or making it easier to identify and correct firmware and software bugs and push updates, manufacturers are recognizing that integrators are overwhelmed and asking themselves how they can make things easier. “With each iteration of our software the bolts are being tightened more and more so you don’t get into a situation where a systems integrator installs software and leaves a big gaping hole because he forgot to do something,” Morin describes. 

The Security Dealers & Integrators

While manufacturers are doing a better job than in the past hardening their systems, security integrators are still in a tough spot. Why? Not all manufacturers are doing the same level of cyber-hardening and some aren’t as far along as others. What’s more, integrators are often dealing with multiple systems from different manufacturers and — even worse — legacy systems that aren’t hardened at all. Then there are the end user customers; many are pushing for more cyber security, but those that aren’t may not want to pay more for hardened systems and the maintenance and lifecycle upkeep required to be cyber secure. It can be confusing, but many integrators and dealers are stepping up to the challenge.

Morin says being an integrator is not unlike being a chef. “You have to have the right ingredients; buy fresh and not frozen. One will be more expensive. In security the software that you use, the camera you use, you need to pick the right thing because there is usually a reason why it’s cheaper. Then you need the basic skills to put together a meal. You need a recipe. If you take that recipe in terms of what those vendors have, with your basic skillset you can have a very secure environment. But it does take the proper ingredients and skills, and it is not easy.” And just like a chef, if the customer is not happy you will hear about it. 

“The first thing we did was educate ourselves,” Lanning says of her and her husband Andrew’s efforts to get on top of the cyber issue. “We spent the last several years making connections with local law enforcement; we joined several industry associations to elevate the cyber conversation. Andrew is on the PSA Cyber Committee and the SIA Advisory Council.”  

Lanning says her company has gotten more particular about which manufacturers they work with in the process. “You have to do your due diligence, but that is also part of the opportunity if you can stop thinking of it as a cost. We have vetted these manufacturers and don’t deal with low-level companies; we deal with those that implement [cyber security] standards.” 

Integrator Hank Monaco, vice president of marketing, Johnson Controls Building Solutions North America, Milwaukee, sees the role of the integrator or security dealer as a holistic one. “It starts with the fundamentals around training and passwords and software upgrades and patches we deploy. You need holistic management of those elements to make sure customers are protected. … It is important to ensure we are taking every step possible to ensure there is a really rich security protocol for any networks we are connected to.”

But all of that comes at a price. From additional training, to additional steps at installation and more follow-up on the back end to monitor for vulnerabilities, the expanded role of the security integrator is indeed an opportunity; but it is also one that not all end users will sign on for. 

“It has dramatically changed the way a security integrator should look at a network deployment,” says Michael Ficco, director of engineering at NextGen Security. “There are many more factors to take into account when designing and estimating a project and these additional factors all have costs associated with them. In bid situations we’ve found that low bids typically have not considered the extra labor steps needed to properly secure devices nor specified more secure networks.”

Existing systems are yet another challenge, Monaco says. “A lot of the work we do is adding onto and integrating with legacy systems. We are taking every step we can to ensure that where a legacy system would require an upgrade we are recommending that.” He credits choosing the right manufacturing partners who are creating backwards-compatible integrations that are secure with helping mitigate that unique challenge.

It pays to be equally choosy when it comes to the customer themselves, Lanning says. If the customer is not concerned about cyber security she says they often walk away. “It is like the difference between McDonald’s and Mortons. Do you want to be about convenience or quality? You can’t be everything to everybody. There are a lot of customers we walk away from because we won’t fight that low fight. If the customer is not concerned with cyber security, we don’t want to deal with them because we don’t want to be responsible for that.” 

Zatolokin advises integrators to have an in-depth discussion with end users and their IT departments around expectations and what they are willing to do around cyber-hardening their security systems. “Best practices can all be done easily at the time of installation, which is baseline. Anything above the baseline, there should be a discussion about it and if the end user chooses not to implement those additional features I would personally have a little document saying they have declined to implement these.”

The integrator is in a position where they need to be both educated and educator, Hoyne says. “Security integrators carry the most trust amongst manufacturers and end users. They are expected to be trained on product deployment and have an understanding of threats in the IT landscape. Our responsibility is performing due diligence when deploying based on the manufacturer’s guidelines and to the client’s requirements. We’re also responsible for educating the end users on current and emerging threats and what can be done to mitigate them.”

And just like manufacturers have to be aware their products could develop a vulnerability after release, integrators too need to stay on top of that, whether it is monitoring manufacturer’s sites for published vulnerabilities, signing up for email alerts on products they have installed or keeping up with the latest standards and regulations. “A product that is secure today may not necessarily be secure tomorrow,” Jamieson says. “The threats or legislation that applies to some areas today may be changing in the future.” (For more on this see “Cyber Laws & Standards” online at https://www.sdmmag.com/cyber-laws-standards.)

End Users & Consultants

All the efforts being made by manufacturers and integrators are good news for the industry going forward. But it is ultimately the end user who reaps the benefits — or not, if they don’t do their part, as well.

At the enterprise end of the market end users are actually pushing cyber security measures forward but that is not the case across the board, Zatolokin says. “Where we are seeing the most progress is where the end users are driving integrators to implement cyber security. The integrator can make end users aware of cyber, but if they are not interested it won’t go very far.”

Fortune 500 companies, tech companies, and those with strong IT backgrounds are all aware of the issue, Morin says. “They are very, very cautious about information security and the bar is raising very fast. But I do fear on the lower end of the spectrum … the level of awareness is not there.”

Lanning says end-user awareness definitely depends on the customer. “Everyone talks about Target and cyber hacks. They understand what phishing is; but I don’t know if they all think of the security systems being installed from a cyber security standpoint. We often get looks when we say, ‘You can hack Wiegand.’ That is on the commercial side. On the DOD side, they have cyber regulations all laid out. So it definitely runs the gamut and it is part of our job to educate the end user [where necessary] as much as possible.”

Anjené Abston, vice president security and life safety, SL Green Realty Corp., New York, N.Y., is one end user who is actively investing in cyber security. “I am getting my masters in information security so it is a daily thing for me to think about,” she says. “I realize in the process of getting our physical security system it is my responsibility to know as much as I can because … the same vulnerabilities that exist in a corporate network can exist in a physical security system. As more end users push to have these conversations, the market will get there.”

Abston is a proponent of getting what you pay for, although she also notes they are of a size to get economies of scale, something she feels benefits not just her company but also the integrator.

“I think if you actually get what you ask for and you are getting quality, yes to a certain extent we will pay a little more. … We don’t want to be the only ones asking for something because it doesn’t give the integrator as much breadth of experience if they are only looking at our system.”

Sometimes end users may have to be willing to trade features for security, Bozeman adds. “That end user needs to make sure his company is protected as best as possible, even to the extent where every single feature set he or she might desire may not be available. But if one product is more cyber-hardened than another with bigger feature sets there is that much risk and it is that much more important [to sway the decision].

“Features are still important, but cyber now has a seat at the head of the table. A lot of this is now not only up to the CISOs but they direct report to the C-Suite level. It is that big of a deal.”

William Plante, senior principal consultant, enterprise risk group for Aronson Security Group, an ADT Company, Renton, Wash., agrees. “There is very much a heightened awareness of the risks, issues and rising expectations by the enterprise to improve the cyber security of physical security systems … CIOs and CISOs are taking notice and injecting their interests into the physical security systems architecture.”

Next Steps

At all levels of the security chain there is not only awareness but action now. So what comes next? 

Keep doing more and stay on top of the problem, Morin says. “We are all in this boat together and at the end of the day we are sharing a common resource, which is the internet. We have to be good citizens if we want to keep it clean and secure and safe moving forward.”

Lanning encourages security integrators and others to take heart in what other industries and sectors are facing. While it may feel like the security industry is way behind when you look at IT, she recounts a recent conversation with a Coast Guard admiral, who admitted they didn’t pay as much attention as they should have originally, either. “We are not alone as an industry. [Others] are going through this as well.”

She also says there is room for everyone at the table, even if a security dealer or integrator chooses not to go “all in” on cyber security; but they may have a different clientele if they take that route. “As an integrator you have to decide what you want to be. Do you want to become a cyber security integrator? It is expensive. Or do you want to sell the cheaper stuff? There are people in need of those services, too. But if you don’t do anything I do think that is a recipe for disaster because businesses either grow or shrink. If you are not constantly educating yourself and being important to your customers, you will be out of business.”

Plante echoes that advice. “Determine what kind of business you are in and who your ideal client is. If you want to be in this industry, then you need to know your craft. And cyber security of physical security devices now must be part of your craft.” 

Once you are on that path, you need to keep going, advises David Miller, IT manager, LVC Companies (SDM’s 2018 Systems Integrator of the Year). “Cyber security is not going away and is a never-ending battle,” he says. “New threats are showing up every day. Be aware of the constant stream of vulnerabilities in the products that you sell, install and service. Increase the knowledge of your entire team of people. If you haven’t already, implement a cyber security awareness training program for all your employees.”

ADT’s Harris says it is time now to go to the next level. “We have embraced the basics like updating firmware, changing passwords and deploying firewalls and anti-virus software to protect end points such as cameras. We now need to advance the conversation to include true hardening of our appliances, risk analysis and design better tools, systems and automated detection processes … There needs to be a constant monitoring of the state of the industry, of ever-changing threats and evolving technology. This is a new form of terrorism that shows no sign of going away any time soon.”