Sudhakar Ramakrishna understands that people are angry. They have a right to be. “You can’t be dismissive of that, you have to accept it,” he tells Verdict.
Ramakrishna is the CEO of SolarWinds. Just weeks before he stepped into the role in January 2021, the software vendor became ground zero for one of the biggest cyberattacks ever recorded. Over 18,000 organisations were compromised in the so-called Sunburst attack where cyber raiders had created a backdoor in a SolarWinds update. The update was then downloaded by both private and governmental organisations. All of them had one thing in common: they had trusted SolarWinds to keep them safe.
Go deeper with GlobalData
When the Texas-headquartered firm broke that trust, it provoked outrage among its clients. In the 20 months since he stepped into the role, Ramakrishna has dealt with the fallout of the hack, including answering calls from furious customers.
“The most important thing for them was us acknowledging [the breach] and facing them,” the SolarWinds CEO says. “Because, oftentimes in situations like this, the temptation is to hide, the temptation is to spin, the temptation is to throw so much fire and PR at it that – no disrespect – it drowns the real issue.”
At the same time, customers demanded answers. How had it happened? How had one of the world’s most trusted software companies missed that Kremlin-backed digital thugs had compromised its systems and tainted one of its updates? How had SolarWinds’ own cybersecurity experts missed it? And what were they going to do about it?
These were questions that Ramakrishna himself wanted answers to. However, for him the Sunburst chaos started before he’d even become the CEO of SolarWinds.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataHow the SolarWinds hack unfolded
On the evening of the 13th of December 2020, Ramakrishna was enjoying a late birthday dinner with family and friends. “We were just sitting down to eat,” he recalls.
Then his phone rang. He looked at the number. It was Jason Bliss, the chief administrative officer at SolarWinds, the company Ramakrishna was set to join in the new year.
“I did not pick up the phone because there were a bunch of people around me, but I did text him back saying ‘I’m the middle of dinner. If it’s urgent, I’ll step out and call you. If not I call you after this.’ And he responded, ‘call me after you’re done.'”
Ramakrishna finished the meal. He stepped out afterwards and returned Bliss’ call. That was when Bliss told him that SolarWinds had been hacked.
“He honestly did not know all the details at the time,” Ramakrishna remembers. “He simply said ‘I’m giving you a heads-up because we’ve been made aware of this issue.’ At that point [he didn’t know] a lot of the specifics.”
Ramakrishna thanked him for the courtesy, finished the conversation and went back to the festivities. At this stage, he didn’t think much of it. As a former executive at a cybersecurity firm, Ramakrishna was aware that businesses are hacked every hour of every day. It was only a few days later when the story hit the news, that he realised just how bad things were getting.
“‘Til the 12th, all I got was congratulations and then, on the morning of the 14th, all I got was commiserations,” Ramakrishna says.
How did the Sunburst attack unfold?
It is unclear when the Sunburst attack really started. SolarWinds’ own research has suggested that the cybercriminals gained access to its systems at some point before September 2019.
US and UK cyber agencies have linked the attack to a subgroup within the Russian Foreign Intelligence Service (Sluzhba Vneshney Razvedki, the SVR) known in popular parlance as Nobelium, APT29, The Dukes and Cozy Bear.
SolarWinds wasn’t the main target, but a means to an end. When Cozy Bear injected a security backdoor into network management software made by the IT vendor, it provided it with access to any computer in the world that installed the malicious update.
During 2020, the code drip-fed into more and more computers – first in the US and Europe and then across more continents. Silent like cancer, the contagion spread as more organisations downloaded the tainted update.
Cybersecurity firm FireEye unearthed the contagion almost by chance in December 2020. FireEye found evidence of the hack when it was investigating a breach of its own, which it also believed was linked to the SVR. When examining roughly 50,000 lines of code, FireEye discovered the SolarWinds backdoor. The firm then reported the breach to the US National Security Agency (NSA), which uses SolarWinds’ software itself.
On December 13, Reuters broke the story about the hack, reporting that several US Treasury and Commerce departments had been breached. It also revealed that the cyberattack was so serious that it had prompted an NSA meeting at the White House.
While Kremlin-backed hackers had already been fingered as the culprits behind the attack, the Russian foreign ministry denied the accusations, describing the allegations as “another unfounded attempts of the US media to blame Russia for hacker attacks on US governmental bodies.”
For the incoming SolarWinds CEO, the reporting about the attack was the first sign that this was going to get serious.
To be or not to be the next CEO of SolarWinds
The news split Ramakrishna’s family and friends down the middle. They started to argue with him about whether or not he should take the new job.
His wife told him she would support him no matter what he decided to do. Ramakrishna’s children were worried. As the severity of the hack became public knowledge, they pleaded with him to do something else, to go and teach somewhere. They couldn’t understand why he’d want the headache of dealing what was rapidly becoming known as one of the worst cyberattacks in history.
“My friends [were] in two camps; one [said] you have nothing to prove, don’t take this job. There are five others waiting for you,” Ramakrishna says. “The [other] said ‘you’re perfect, you’ll fix this problem. If anyone can fix it, it’s you.'”
Both camps had a point. He had the right experience for the job. Over the past three decades, Ramakrishna has built up an impressive resume. He’s clocked up time in executive roles at places like Motorola Networks, media collaboration company Polycom and digital workplace platform Citrix.
Most recently, Ramakrishna had sold the security venture Pulse Secure to Ivanti. He later said the deal meant that he’d never need another pay cheque. So if it wasn’t for the money, why did he decide to still join Solarwinds? Ramakrishna says it was a matter of principle.
“Let’s say I’d joined the company already,” Ramakrishna says. “And then, two days after joining this company, I come to know about it. What would I do? Would I turn around and run at that point?”
The idea of tucking tail didn’t sit well with him. Moreover, he welcomed the challenge of navigating SolarWinds through the crisis. That proved the deciding factor for him.
“Why not go and do something where I can probably create a different impact in the industry, in the customer base, in the employee base and so on,” he says.
With that in mind, Ramakrishna decided to honour his commitment and to step into the role as the new CEO of SolarWinds on January 4, 2021.
Sudhakar Ramakrishna is the CEO of SolarWinds.
Becoming the CEO of SolarWinds
Ramakrishna later found out that SolarWinds’ employees had made bets about whether or not he would actually walk through the doors or not. “Let’s put it this way: I’m sure some people lost quite a bit of money,” he laughs.
However, disappointing some Solarians who’d bet against him taking up the mantle to lead SolarWinds was the least of his worries as CEO.
“The company was shell shocked,” Ramakrishna recalls. “The company had never really focused on grabbing attention in the marketplace and they worried about their jobs, [about the] business and they didn’t like the spotlight, at least not in this way. [There] was genuine pride and anger that somebody could do this to us.”
The first thing the new SolarWinds CEO did to tackle this was to display a sense of calm. At the same time, Ramakrishna says he made it clear that there was a huge problem that needed a solution.
He addressed these woes in a public blog on January 7, saying that the company was working on improving SolarWinds’ security practices in principles he referred to as “secure by design”.
At first, this boiled down to deploying additional threat hunting technologies, keep investigating the breach and to rebuild the development environment from scratch. Later, it would result in a total overhaul in SolarWinds’ design work.
Designing for defence
SolarWinds’ engineers were hard at work during most of 2021, working on introducing new development infrastructures to avoid the company being exploited the same way in the future.
Their efforts included basing the new system on ephemeral operations. Basing the design on the cloud-based solutions of CircleCI and GitHub Actions, SolarWinds new design would be put into a container and spun up on demand. It would then be destroyed when the task had been completed. That way, SolarWinds aimed to reduce the number of long-lived environments hackers could use to establish home bases in.
SolarWinds have also introduced “consensus-attested builds” that means that each system is duplicated and built in parallel with each other. The idea is that this will make it easier to make integrity checks and to boost the security of the system.
It has also overhauled different developers’ control over the process. While they still have control over what they create, they have no say in how those things are validated and secured.
“By doing so the software becomes more secure,” Ramakrishna says.
Upping its digital defences has also enabled SolarWinds to update customers about the measure the company was taking to avoid another breach.
“We only established one goal for 2021 and that was customer retention – [that we’d] do everything possible to support customers to get back on their feet,” Ramakrishna says.
In an attempt to mitigate damage to its tarnished brand, SolarWinds enlisted PR crisis management firm Goldin Solutions in 2021. Part of the agency’s endeavours manifested in sending testy missives to journalists, including Verdict, in which they attempted to dissuade referring to Cozy Bear/Nobelium/APT29/the SVR as the “SolarWinds hackers”.
When we breached the topic of Goldin’s efforts with SolarWinds’ PR handlers, we were told that Ramakrishna “is unlikely to comment on the performance of a supplier.”
While Ramakrishna claims that SolarWinds’ retention level are back to its historic levels, the company’s stock has not recovered from the breach.
On December 11 2020, IT vendor’s shares traded at $24.83 a pop. By the 18th, they’d fallen below $15. Even though it did recover slightly during 2021, SolarWinds has continued to tumble in the public market. At the time of writing, its shares are hoovering just above the $10 mark. SolarWinds has a market cap of $1.66bn.
It is, however, difficult to say whether that is due to the hack or if the company has been caught in the same slump as other public tech ventures have suffered in 2022.
Not fighting back against Russia
Even though Russia has denied being behind the Sunburst hack most of the world’s security experts believe Kremlin gremlins were behind the digital assault.
When Ramakrishna first became CEO of SolarWinds, he noted how several engineers wanted to give the hackers some payback. Earlier this year, they got a golden opportunity.
When Vladimir Putin’s troops invaded Ukraine, several western tech companies stepped up to offer Ukraine their support. Microsoft, for instance, offered its support and guidance to protect the nation against Russian cyberattacks. However, SolarWinds is not joining the war effort, not even to give Cozy Bear a good licking.
“Nation state sponsored cyber attacks have been going on for quite some time,” Ramakrishna says. “I know our attack has been attributed to the Russian SVR. But, to be honest with you, that is not our area of expertise. We focused on our technology, our products, our people and our customers, and then let governments deal with what they have to deal with.”
For now, he says, the company is focusing on enriching the lives of the people it serves, not to join the cyberwar against Russia.
GlobalData is the parent company of Verdict and its sister publications.
