How One Town Said Goodbye to Legacy Infrastructure and Welcomed IT Modernization
Before its incorporation in 1988, the Town of Apple Valley, California had a colorful start. In 1946, two enterprising partners marketed the area as "The Golden Land of Apple Valley,” a destination resort and residential community, and invited Hollywood celebrities to visit. Within ten years, some 180 businesses and organizations grew up in this fruitful desert valley.
Today, the Town of 73,000 residents is run by the Town Council, Town Manager and 120 employees. It’s still a thriving business-friendly community and was recently named in the top 3% of schools nationwide. They’ve also emerged as a leader in advanced health care systems, from pediatric services to radiology.
The Town had previously outsourced its IT operations before bringing IT back in house in 2017. Innovation & Technology Officer Brandon Cales joined the Town in 2019 to lead the IT team and help the Town meet residents’ growing needs.
Within days of starting his new role, Cales found multiple issues:
- Aging technologies — As with many municipalities, the Town’s technologies were about 10 years old and the IT team was simply keeping the lights on.
- Inconsistent power protection for UPS systems — In 2018, the Town nearly shut down, having to provide manually processed services for two weeks or more due to a power surge. When he came onboard, Cales found that the backup batteries lacked proper electrical grounding.
- Inconsistent/missing backups — Finally, and most critically, Cales and his team discovered that, for quite some time, backups had been done inconsistently or not at all.
This last point is particularly salient as recent research indicates that 44% of global ransomware attacks in 2020 targeted municipalities. While targeted cities, such as Atlanta and Baltimore, make headlines, smaller municipalities—New Bedford, MA, Pensacola, FL and even tiny Wilmer, TX—have also been subject to costly ransomware attacks.
Not only do cybercriminals know that governments, in general, have at their disposal a seemingly endless supply of money, governments are also responsible for managing vital services and critical infrastructure—the loss of any of which can cripple its population.
Thus began a race against the clock for Cales and his team—including Chris Johnson and Gene Whiteside—to restore power protections and unravel the backup issues before disaster struck.
Admittedly, Cales is not a networking guy. “The town doesn’t need someone who is an expert in every area,” he said, “they just need someone who knows how to get and manage that expertise accordingly to get the job done. With Logicalis, I knew I could reach out to them and they’d be able to help out.”
Having worked with Logicalis previously, it was an easy call. Logicalis Solution Architect Todd Thienngern came to the rescue, spending several days onsite to help them get backups running—in essence, buying time until they could implement a more modern solution.
“We originally went in there to help with the backups and we talked about what the future could look like,” said Thienngern. “We gave Brandon a demo, never knowing that it would be a ‘this year’ project.”
Phase 1 Implementation: Data Center Refresh and Backup Upgrades
With the Town’s help, Cales was able to secure enterprise funding and the Logicalis team went to work. They executed their data center refresh plan by first updating their power systems and then modernizing the storage and backup system with HPE Nimble Storage dHCI and StoreOnce Systems. Like hyperconverged infrastructure, dHCI enables storage, compute and networking to be managed from a single management plane, yet storage and compute can be scaled independently to prevent stranded resource issues. In addition, dHCI enables much more streamlined data center management for Cales and his team.
StoreOnce provides one high-performance backup architecture that spans the entire environment and, through deduplication, enables customers to reduce the amount of backup data they need to store by up to 95%.
With the infrastructure in place, they then upgraded the Town’s VMware installation and upgraded and re-installed Veeam Backup & Replication. Logicalis also implemented storage snapshot integration from HPE Nimble to enable the Town to have fast backups and replicas made directly from VM and storage snapshots, reduce the impact of backups on their production environment and empower them to initiate backups at any time or frequency with little to no impact on operations.
Veeam health check was then used to identify installation and configuration issues. With their data protection up to date, Cales and his team can control the flow of deduped data through StoreOnce and reduce backups from hours to minutes. Logicalis also leveraged an existing server to serve as a backup server.
Having dealt with ransomware in the past and knowing that cities are a frequent target, Cales understood all too well the value of having good backups. Backups protect organizations from data loss not only through physical disaster, stolen equipment and corrupted files, but also from loss due to cyberattack. The ability to instantly recover valuable data and, in the case of ransomware, prevent data deletion and make it recoverable at any time is key to fighting back. Fortunately, Veeam also provides immutable backups that secure data by making it unchangeable for nearly bulletproof ransomware protection.
Ironically, just as they were finishing the implementation, a server failed. “Luckily, with the new system, we managed to get it back online,” said Cales. “It was the same thing with the phone system…our current system started dying. Luckily, the Town has a great relationship with the IT team and with Logicalis to help us out with everything.”
Phase 2 Implementation: CARES Act Funding
When the pandemic struck, the IT team—which had already deployed Office 365—seamlessly transitioned about 25 employees to remote work. But with their offices closed to the public, the business of the Town still had to get done.
Cales sought—and eventually received—CARES Act funding, along with matching enterprise funds from the Town, to have Logicalis update their legacy network including switches, voice and wireless. With these considerable improvements, the Town can now hold virtual meetings between departments and—with an integration between their audio-visual system and their new voice system—stream public meetings allowing the public to participate remotely.
“We’re able to bring our resources because the Town has a small IT staff,” said Account Executive Lisa Waelde, “but even in large IT teams, not everyone is going to know everything, so you have to rely on resources outside your organization.”
To date, Cales and his team have modernized the Town of Apple Valley’s environment to:
- Improve the performance of older applications.
- Support new online payment systems and GIS-based software that enables public works crews to work more efficiently and remotely.
- Protect the Town from cyberattacks, particularly ransomware, and ensure anytime recovery through immutable backups.
- Have the ability to test and validate backups.
- Significantly reduce backup times from hours to minutes.
- Streamline data center management, allowing Cales and his team to focus more on other IT and Town initiatives.
- Better support the Town’s infrastructure.
- Enable the Town to keep employees informed through virtual meetings and keep residents informed through streaming meetings.
When asked why he would recommend Logicalis to other municipalities, Cales said that, of course, their knowledge and experience were key. But it was something else, too. “It was their ability to understand the issues we were facing and, instead of pushing any one solution…they wanted to make sure that we were happy with the decisions we were making based on the recommendations they provided. They are definitely my go-to resource for any type of new project.”
Customer: Local municipality
Partners: Logicalis, Veeam, HPE and Cisco
Challenge: The town had aging infrastructure that was well past end of life, creating an environment conducive to outages, downtime and risk.
Solution: Logicalis Professional Services, Veeam Backup & Replication, APC UPS Battery Solution, HPE Nimble Storage dHCI, HPE StoreOnce Systems and Cisco Network Infrastructure, Cisco Wireless and Cisco Unified Communications.