The call comes from a new Tory campaign group, Managed Migration, which claims the support of up to 20 Tory MPs.
The group’s organiser, Conservative MP Mark Field, warned Mr Cameron against getting into “a Dutch auction” over immigration numbers and said he could not “out-UKIP UKIP”.
But Mr Cameron said the target was still “important”.
“It is right to target a reduction in immigration,” he said on a visit to Hull.
“If you look at immigration from outside the European Union, it is down by a third and it is at its lowest level since 1998 and we have to keep working towards that important target.”
However, Mr Field, the MP for the Cities of London and Westminster, said the target was “impractical”, harmed Britain’s global competitiveness and was clearly not going to be met.
“Very few voters out there believe we can deliver on it and, indeed, all the evidence suggests it is now moving in the wrong direction,” he told BBC Radio 4’s Today programme.
“And, of course, very few businesses – and I feel that very close to my heart, representing the sort of seat I do – believe it is practical.
“And it risks, in my view, sending out the wrong signals about our openness – and Britain’s traditional openness – as a trading nation, to the world at large.”
He said recent big increases in net migration were a sign that the UK’s economy was recovering strongly.