The Federal Reserve said an outage of its key financial services on Wednesday was caused by a maintenance mistake and it is taking steps to prevent a recurrence.
“The incident was caused by an operational error involving an automated data center maintenance process that was inadvertently triggered during business hours,” a Fed spokeswoman said. Such tasks are normally performed after-hours, she said, adding, “This was human error.”
The disruption shut down all the Fed’s financial services Wednesday for about four hours, halting electronic systems used by banks, businesses and government agencies to move money. While the problem was resolved after the Fed extended its business hours to clear a backlog of payments, it prompted banks to initiate emergency protocols and raised questions about the resilience of critical financial plumbing.
“When things like this happen it undermines public confidence,” said Andrew Levin, a Dartmouth College economics professor who served at the Fed from 1992 to 2012.
One person familiar with the Fed’s systems said they rely on a geographically dispersed network of data centers, servers and mainframes. Problems that occur in a key location at the wrong time can require the systems to be rebooted, which is what the Fed did Wednesday.
The timing of the incident—the middle of the day, between 11:15 a.m. and 3 p.m. Wednesday—likely limited its impact, as the heaviest volume of financial transfers occurs earlier in the morning and later in the afternoon, that person said. Had the outage lasted overnight, banks wouldn’t have known their cash position at the end of the day, and workers’ paychecks could have been delayed, the person said.
That’s because the Fed’s systems can take days to settle transactions. Other central banks, including those of Europe, Brazil and Mexico, do so instantly, something the Fed doesn’t anticipate doing until 2023.
The systems affected Wednesday included the Fedwire Funds Service, which the Fed describes as “the premier electronic funds-transfer service that banks, businesses and government agencies rely on for mission-critical, same-day transactions.” Also affected were FedACH, an automated clearinghouse network that enables debit and credit transactions, and Fedwire Securities, which provides transfer and settlement services for securities issued by the U.S. Treasury, government agencies and government-sponsored housing enterprises.
In 2019, an unspecified technical issue took down parts of the Fed system on separate occasions, including Fedwire Funds System and FedACH. In a meeting shortly after the first outage in April 2019, the New York Fed’s Treasury Market Practice Group noted that there “would have almost certainly been very significant ramifications for the market” if the Fedwire service hadn’t resumed within the same day.
But while individual systems have experienced problems, an outage affecting all the Fed’s financial services appeared to be without precedent, current and former officials said.
“Yesterday’s disruption to the Fed’s payment systems is concerning given the size and scope of these operations,”
Sen. Pat Toomey
(R., Pa..), the top Republican on the Senate Banking Committee, said in an emailed statement. “Fortunately, the U.S. has a robust and resilient financial system, and the Fed was able to restore the services quickly. It is my understanding that the real-world impact has been limited.”
—Michael S. Derby contributed to this article.
Write to Paul Kiernan at [email protected]
Copyright ©2020 Dow Jones & Company, Inc. All Rights Reserved. 87990cbe856818d5eddac44c7b1cdeb8