WASHINGTON — Ever since the Supreme Court agreed to hear a case challenging the constitutionality of the Federal Housing Finance Agency, many observers have expected the justices ultimately to rule that a president should be able to fire a sitting FHFA director at will.

But comments by some of the justices during oral arguments Wednesday hinted that the court may stop short of such a decision.

At issue is whether the powers afforded to Senate-confirmed directors such as FHFA chief Mark Calabria, who can only be fired for cause and do not answer to a board, are unconstitutional. Plaintiffs in the case argue the agency erred in its controversial step ordering Fannie Mae and Freddie Mac to sweep most of their profits into the U.S. Treasury, because the FHFA’s leadership structure is illegal.

But justices signaled that that constitutionality question may be moot because the “net worth sweep” was implemented by former acting FHFA Director Ed DeMarco, who lacked Senate confirmation and therefore any protection from presidential firings.

“If we agree with you … that the only relevant action was one taken by the acting director, would we have any reason to address the question whether the restriction on the removal of a confirmed director is unconstitutional?” Justice Stephen Breyer asked Hashim Mooppan, deputy assistant attorney general at the Justice Department.

The high court on Wednesday heard arguments in Collins v. Mnuchin, a case brought by shareholders of Fannie and Freddie who argue that the 2012 profit sweep agreement between the FHFA and Treasury violates the law. Specifically, they say the provision in the 2008 law establishing the FHFA, saying a president can only fire the director for cause, was unconstitutional.

The case resembles one the Supreme Court decided this summer in which the justices ruled that the president has the authority to remove the director of the Consumer Financial Protection Bureau at will. Because the leadership structure of the two agencies resemble each other, many experts said it would be difficult for the FHFA to avoid a similar fate.

While that ruling eliminated the so-called for-cause provision in the Dodd-Frank Act, which created the CFPB, it kept the rest of the law and the agency intact, meaning there was no impact on the agency’s past nine years of rulemakings, decisions and enforcement actions.

But several justices indicated that the circumstances were different enough in the FHFA case to distinguish it from Seila Law v. CFPB, in part because DeMarco hadn’t been Senate-confirmed.

Mooppan defended the legality of the net worth sweep but not the FHFA’s leadership structure. The court appointed Aaron Nielson, a law professor at Brigham Young University, to defend the constitutionality of the agency’s single-director structure.

Mooppan told Justice Brett Kavanaugh that “the easy solution” would be to “simply say that under this statute, the acting director, who is the official who took this decision on behalf of the FHFA, is in fact removable at will by the president.”

But David Thompson, managing partner at Cooper & Kirk who represented the plaintiffs in the case, said that finding would not go far enough. He poined out that the sweep was later administered by Senate-confirmed directors, who he maintained were serving illegally because they were insulated from being removed by the president.

“So your theory is that even if an acting director approved the instrument under which payments are going to be made, that when those payments are made, if there’s an unconstitutional director, that they are invalid?” Chief Justice John Roberts asked.

“Even if there’s an acting director, the president can’t put the person that he wants in there, he had to pick one of the three deputy directors who were in turn picked by the prior director,” Thompson responded, referring to stipulations in the Federal Vacancies Reform Act that govern how a president can name acting officials.

While the court appeared much more focused on the legality of the net worth sweep than the constitutionality of the agency’s structure, justices were split over whether they should apply the Seila Law ruling about the CFPB equally to the FHFA case.

“I see vast differences between the FHFA and the CFPB,” said Justice Sonia Sotomayor. “The FHFA’s most notable power … is that they can put certain government-affiliated companies under conservatorship. Conservatorships are never thought of, in my experience, as an executive power. It’s historically been an adjunct to the judicial power.”

But Kavanaugh suggested that he thought the ruling the court handed down regarding the independence of the CFPB director would also apply to the FHFA director.

“My understanding of the principle that would be applicable here would be that single-director independent agencies are not historically rooted as the court said in Seila Law, and that’s all we would … [be] saying … here,” he said.

Several justices expressed doubt in Thompson’s argument that the net worth sweep harmed investors, or even was related to the for-cause provision.

“It does seem counterintuitive … to say that assuming you’re right that the FHFA director must be removable at will, why you should get anything more than a declaratory judgment to that effect,” said Sotomayor.

Roberts also pushed back on the assertion that shareholders’ investments in the government-sponsored enterprises were worthless because of the sweep.

“I checked this morning and Fannie Mae was trading the morning at $2.69 and Freddie Mac at $2.56 and your shares are not worthless, they’re worth something,” he said to Thompson. “Doesn’t that render you nationalization rhetoric just that — rhetoric?”

Thompson responded no, and contended “there’s no scenario” under the Third Amendment to the preferred stock purchase agreements, which instituted the sweep, “in which we will be able to recover any economic value.”

The case has attracted more attention since Joe Biden was declared the winner of the presidential election last month. If the court were to rule that the president does have the authority to remove the FHFA director at will, Biden would be able to replace Calabria, who was appointed by President Trump.

Calabria has made clear his intentions to free Fannie Mae and Freddie Mac from conservatorship, and has set the companies on a path to privatization. He has devised a post-conservatorship capital framework and allowed the companies to retain more of their earnings. But some of his moves have come under fire from Democrats, who have expressed concern about the impact on taxpayers and homeowners.

It is unclear when the Supreme Court will issue a ruling for the FHFA case, although the general expectation is that it could be handed down before next summer. (The court’s term ends in July.)

Unlike some cases in front of the high court, Collins v. Mnuchin did not appear to divide the justices along partisan lines, making the outcome more challenging to predict.

But given the attention the justices paid to the acting director question, it’s possible the court could decline to rule on the constitutionality issue entirely, Ashwin Phatak, appellate counsel for the Constitutional Accountability Center, said in a briefing after the oral arguments.

“It did seem like the more likely way that they might end up dealing with this case is to just say that at the time that the FHFA entered into this agreement, the director of the FHFA had been appointed in an acting capacity … and the court may say that acting officials actually don’t have any for-cause removal protection,” he said.