Prime Minister Narendra Modi appointed Subrahmanyam Jaishankar, India's ambassador to the United States, as foreig secretary replacing Congress party-appointed Sujatha Singh, a day after a successful visit by US President Barack Obama.
Modi's appointment of Jaishankar as foreign secretary, the top bureaucrat at the foreign ministry, underscores the prime minister's control over foreign affairs.
Jaishankar, credited with helping Modi turn around the relationship, said his job was to carry out the government's priorities.
"A big responsibility. I am honoured that I have been assigned this responsibility," he told reporters as he took charge.
Jaishankar has previously been ambassador to China and Singapore.
His abrupt sacking of Singh was criticised by the Congress party, under whom she was appointed.
Since he swept to power in 2014, Modi has virtually taken over the reins of foreign policy, beginning with an unprecedented invitation to regional leaders to attend his inauguration and followed by high profile meetings with the leaders of United States, China, Japan and Australia.
Singh was asked to step down from the post, the government said late on Wednesday, with seven months to go before retirement. The last time the head of the elite foreign service was removed was in the late 1980s.
"Why have you replaced the head of the foreign service so unceremoniously," Congress party leader Manish Tewari said, adding, "The government needs to explain."
During Singh's tenure under the previous administration, the government and the US were embroiled in a diplomatic row over the arrest of a junior Indian diplomat in New York for an alleged visa fraud. Ties hit their lowest level in a decade.
Since then, PM Modi has moved rapidly to rebuild ties with Washington, putting behind his own embarrassment at being denied a visa for a decade for religious violence in Gujarat when he was Chief Minister of the state.
The Prime Minister went on a state visit in September 2014, building a relationship with Obama and then hosting him as the chief guest at the country's 66th Republic Day Parade on January 26. Obama became the first US president to do so.
FULL COVERAGE:Barack Obama's India visit 2015