Boris Johnson has refused to apologise for saying that women wearing burkas look like letter boxes.
Earlier, a source close to Mr Johnson reportedly said it's "ridiculous" that he's being criticised for Islamophobia.
Conservative Party chairman Brandon Lewis said he had asked Johnson to apologize, though didn't say if there would be any sanctions if the request was ignored.
Alistair Burt, minister for the Middle East, told BBC Radio 4's Today programme: "I would never have made such a comment, I think there is a degree of offence in that, absolutely right".
She said: "I've written to @PennyMordaunt and @BrandonLewis to ask what action will be taken against Boris Johnson for his ugly and naked Islamophobia".
Mr Johnson said full-face veils should not be banned but looked "ridiculous".
Lady Warsi also echoed calls for an independent inquiry into alleged Islamophobia in the Conservative Party, claiming the party's Old Etonians were crassly using Muslim women as "a political battleground and it must be condemned by the leadership right from the Prime Minister down".
Some suspected Mr Johnson's burka comments were meant to boost his appeal among right-wing members of the party.
In March there was a spate of suspicious packages delivered to Muslim MPs, thought to relate to the "punish a Muslim day" letters that were debated in the Commons.
Critics accused him of "bigotry" and "dog-whistle politics", while even his own party members raised questions over his choice of words.
Theresa May has backed calls for Boris Johnson to apologise for his comments about Muslim women who wear burkas.
The same piece said the city had made scapegoats of the police in the Hillsborough disaster while refusing to acknowledge the part played by "drunken" fans "mindlessly" trying to fight their way in - comments which caused outrage in the city.
But Conservative backbench MP Andrew Bridgen said Mr Johnson had raised an important subject in a "light-hearted way".
Mr Johnson's father, Stanley, came to his defence, telling ITV1's Good Morning Britain: "I think we are just picking holes here".
Mohamed Sheikh, founder of the Conservative Muslim Forum, said Johnson's article had been "totally out of order".
He suggested to the BBC that the reaction to the column "says a lot about internal Conservative Party politics".
Another Conservative MP, Nadine Dorries, said the government should apologise and that "Boris didn't go far enough".
"Any clothing a woman is forced to wear which hides both her beauty and her bruises should be banned and have no place in our liberal, progressive country", she said.
However, Johnson said he did not support a blanket ban the face veil in the UK.