Resolving the “Trouble” with RTUs
PUBLISHED ON Dec 15, 2021
A major Midwest utility embarks on a multi-year replacement of all their Remote Terminal Units.
At utility substations, Remote Terminal Units (RTUs) are critical components used for monitoring circuit breaker positions, alarms, voltages/currents, temperatures, and additional data from a variety of wired sensors. They can also control breakers, tap changers, and capacitor banks. However, for many utilities, RTUs remain a trouble spot. As RTUs age, reliability issues, unforgiving software, lack of vendor support for older product lines, increased training time, and diminishing availability of replacement parts can make RTUs an ongoing and time-consuming task for substation technicians.
“I get more trouble calls related to RTUs than I should,” says the senior supervisor at a Fortune 500 electric and natural gas utility serving the power needs of seven states in the Midwest. “What I need is for my RTUs to work reliably because my field engineering team doesn’t have the time to keep coming back to resolve trouble calls and investigate why an RTU isn’t functioning properly or stopped working.”
Despite utilizing RTUs from several known providers, the reliability issues became so frequent that the engineering team decided the best course of action was to start upgrading all the RTUs across their entire network.
“Some of our RTUs started having software and hardware failures,” says their senior supervisor. “They were not very old, maybe ten years. We had one unit where the screen froze up, and the software and the configuration were corrupted. The RTU vendor told us the only thing we could do was to rebuild the database, which we did completely.”
While the engineering team had the knowledge and experience to resolve the issues, it was another task for already thin resources. The team at the utility already has extensive responsibilities that go beyond substation installation to include programming and telecommunication systems. “We manage basically everything that goes from the main control center all the way to the handoff at the end device in the substation,” he adds.
Contributing to their RTU replacement decision was the fact that another one of their RTU vendors announced they would no longer support their existing software. The outcome was going to require the utility to replace the software and train the team. “The training for the new software was going to cost $30,000 for one week of in-person training,” says the senior supervisor.
As a result, they started to explore other options on the market. This led them to an RTU manufacturer that sold them on the promise of a newly redesigned platform. The vendor brought in a new RTU for an initial test. “It just didn’t work,” says the senior supervisor. “For an installation job that was supposed to take about two and a half hours, we ended up putting the old RTU back in thirteen hours later because the vendor could not get the replacement unit running.”
The disappointing experience increased the scrutiny of the claims made by other RTU vendors. At this time, utility personnel began discussions with Pennsylvania-based, NovaTech Automation, a leading substation automation provider. The company offered its OrionLX automation platform as an RTU replacement. Configuration of the NovaTech OrionLX-based RTU, including the I/O system, the Alarm Tile Annunciator, Math & Logic routines, and IED data access, is accomplished through NovaTech Configuration Director, a license-free tool used for Orion models. NCD eliminates most configuration efforts by providing pre-configured pick lists for over 250 commonly applied intelligent electronic devices (IEDs).
After testing to fully validate the new RTUs, the utility chose to replace their existing units with the NovaTech Orion line (LX, LX+, LXm, and now Orion I/O) together with the company’s Distributed Discrete I/O (DDIO) and Combination I/O (DCIO) modules and advanced I/O’s.
As part of the project, NovaTech worked with the utility to custom build and configure an OrionLX unit to function identically to one of the brands of RTUs they were replacing.
The team used the NovaTech Configuration Director to configure the new RTU to essentially mirror the performance of the units they were replacing, including communicating using the competitor’s proprietary protocol. The replacement units have the same number of serial ports, so installing the unit only required plugging the serial cables into the corresponding port number on the new OrionLX once the configuration was completed.
“The wiring requirement was minimal, so it was as close to plug-and-play as you could ask for and the unit came online quickly and worked as expected,” says the senior supervisor, adding that the utility has now replaced all six of their units in this manner.
As for the RTUs with the software that was no longer being supported, the utility is currently phasing out all those units as well.
“We still have probably a dozen of those left,” said the senior supervisor. “In the next few years, we will have completely converted over to the new RTUs across our entire system.”
He adds that early in the NovaTech RTU evaluation, one issue did arise when lightning struck a couple of the new units. According to the senior supervisor, “the lightning blew them apart. We received an alarm from the unit saying the power supply was dead and needed to be replaced, but we were running a redundant power supply, and the Orion units remained online and functional.”
Ease of training and maintenance
In addition to improving reliability and performance, the utility also sought a solution that eliminated the other “trouble” associated with RTUs: the training time required for technicians to program and install the new units. In this regard, the NovaTech RTUs were “technician friendly,” and training time was minimal.
“I was able to take apprentices and show them how the software works,” says the senior supervisor. “After an hour, they were able to take an assignment sheet with all the items needed and program the RTU. That doesn’t mean they have a full understanding of everything the unit is doing yet, but they can program it.”
Ease of maintenance figures highly too. “It probably took me a year to figure out all the little bugs about the software of our prior RTUs. It was very unforgiving,” said the senior supervisor. “Troubleshooting required going through multiple tables and layers. The RTU would not indicate why it was offline, only that it was offline. What we appreciate about the new RTUs is you can review the error log to diagnose the specific problem that is occurring.
Based on the positive experiences to date, the utility continues with its rollout of replacement RTUs. Other utilities have contacted them to understand how they addressed issues with an RTU conversion.
“I’m glad I didn’t resist change when we were considering replacing the RTU units that were causing us trouble,” says the senior supervisor. “With our confidence in the reliability of our new RTUs, and standardizing on solutions from a single provider, our engineering team is able to focus on improving our network infrastructure in other ways without being constantly drawn back to performance issues.”
Click here to download PDF of article
