The Migration Advisory Committee said there were 23 fewer UK jobs for every 100 migrants from outside the EU.
But a separate report from the National Institute of Economic and Social Research (NIESR) says immigration has had little impact.
It follows Migrationwatch UK saying there was probably a link.
The Migration Advisory Committee estimates that 160,000 British-born workers’ jobs have been “displaced” following non-EU immigration between 1995 and 2010.
Between those years, the total working-age migrant employment rose by 2.1 million and currently displaces 160,000 British-born workers, it said.
But the impact and displacement of British workers does not last forever, the Mac report found.
“Those migrants who have been in the UK for over five years are not associated with displacement of British-born workers,” it said.
The committee also looked at the effect of migration on salaries, and found overall wages for the most well-paid people went up, while those at the bottom went down.
The report also added that EU migration had had “little or no impact” on the native employment rate.
Dr Scott Blinder, Senior Researcher at the Migration Observatory at the University of Oxford, said: “This report highlights the need to decide and articulate more clearly whose needs government is prioritising when developing immigration policy.
“Trade-offs need to be confronted head on. Without more debate and clarity about whose interests policy is trying to maximise we cannot hope to reach more agreement about the costs and benefits of specific policies.”
‘Strong work ethic’
Earlier, the National Institute of Economic and Social Research (NIESR) said there was “no association” between migration and the numbers of people claiming unemployment.
The study looked at the number of migrants given National Insurance numbers between 2002-3 and 2010-11 and compared them with the number claiming unemployment benefits.
The NIESR said: “The results show a very small negative and generally insignificant correlation between the migrant inflow rate and the change in the claimant count rate.
“For all practical purposes, these results suggest that migration has essentially no impact on claimant count unemployment.”
But it said it was still not known whether an increase in the number of migrants coming to the UK leads to a fall in the number of low-skilled jobs for British workers which is masked by more jobs for highly skilled Britons.
It said migrants from Poland, the Czech Republic, Hungary, Slovakia, Slovenia, Estonia, Latvia and Lithuania – which joined the EU eight years ago – “tended to be disproportionately young, well-educated, prepared to work for low wages and imbued with a strong work ethic”.
On Monday, Migrationwatch UK, which campaigns for tighter immigration controls, said there probably was a link between rising levels of youth unemployment in Britain and an increase in migration from eastern Europe.
It said that in the third quarter of last year there were 600,000 more workers in the UK from eight former Soviet bloc countries than in 2004, when they joined the EU – and that over the same period, UK youth unemployment rose by almost 450,000.
It would be “a very remarkable coincidence if there was no link at all” between the figures, it said.
A Home Office spokeswoman said: “We have already made sweeping changes to tackle the uncontrolled immigration of the past.
“We have limited non-EU workers coming to the UK, with latest figures showing a year on year fall in work visas issued.
“We will shortly announce reforms of the family migration and settlement routes.”
Source: BBC © 2012