Senior Tories have voiced fears that Boris Johnson may have damaged his campaign to be re-elected London mayor next year after delaying his return to Britain from a family holiday in North America.

Tory MPs criticised Johnson after he was heckled in Battersea on Tuesday by local residents who shouted that he should have come back sooner.

A woman told Johnson he should resign because he is racist. The mayor responded by turning to a man who had challenged him on Mark Duggan to say: "It is time that people who are engaged in looting and violence stopped hearing economic and sociological justifications for what they..."

As the man asked whether his remarks were aimed at him, Johnson walked away from the crowd. "Why are you here now?" another man shouted. The man added: "Three days too late." Tory MPs were scathing. "Boris should have come back from his holiday earlier, he should have come back as soon as Tottenham erupted," a senior MP said. "It is fine that the prime minister did not come back until it was clear that London was burning.

"People have been talking about whether this is a [Hurricane] Katrina moment. It is not David Cameron's Katrina moment. But I would not be surprised if it becomes Boris Johnson's Katrina moment. I hope that this does not damage his re-election chances. But I fear it will.

"Boris's only advantage at the moment is that Ken Livingstone has gone completely over the top and blamed government cuts for causing the riots. Every London Labour MP has been measured."

Theresa May, the home secretary who cut short her holiday, also faced criticism for a highly cautious interview on Radio 4's Today programme on Tuesday. Tim Montgomerie, the founder of the ConservativeHome website, tweeted: "I don't think a single @ConHome contributor thinks Theresa May got response right earlier. Cameron will need to show much more steel."

May later faced criticism from Labour after she quietly abandoned Johnson while he was being harangued by the crowd in Battersea. Labour MP Tom Watson tweeted: "It's the way the home secretary sneaks off at about one minute 55 seconds that says it all."

But a Tory MP said: "Theresa May is a consistent performer. She is a no-nonsense politician who is as tough as old boots."

Posted in: Senior Tories have voiced fears that Boris Johnson may have damaged his campaign to be re-elected London mayor next year after delaying his return to Britain from a family holiday in North America.