David Lammy faced calls to resign if Epping sex attacker Hadush Kebatu is not deported by the end of the week.

Shadow Justice Secretary Robert Jenrick goaded his Labour counterpart, declaring the Government has failed to remove the “only Channel migrant who wants” to leave the UK.

Mr Jenrick branded the bungled release of Hadush Kebatu a “national embarrassment”, adding that the Ethiopian sex offender returned to HMP Chelmsford five times after mistakenly being let out due to “human error”.

Mr Lammy, in an extraordinary row in the House of Commons, revealed former NCA Director General Lynne Owens will lead an independent investigation into the scandal and claimed release checks will be intensified.

There will now be more senior oversight on ensuring checks are correctly applied, including a clear checklist for prison governors to determine that every step has been followed the evening before any release takes place.

But Mr Jenrick said: “The Justice Secretary could not deport the only small boat migrant who wanted, no tried, to be deported having been mistakenly released.

“On the subject of the inquiry itself, you won’t be surprised to hear I am, how should I put it, sceptical about the Government’s ability to conduct inquiries with any competence?

“Why is it limiting itself to this particular security farce and not the other glaring issues, like the doubling of drone sightings above prisons or the rampant extremism we’re now seeing in our jails?

“Shocking as this accidental release is, it is not a one-off blunder. The number of prisoners mistakenly let out early has more than doubled. Can the Justice Secretary tell the House how many of the 262 prisoners let out mistakenly in the year to March were violent or sexual offenders and how many are still at large?

“Can the Justice Secretary give this House his cast iron assurance that this man will be deported from our country by the end of the week, as he promised. And if he fails, will he take responsibility and resign?”

Mr Jenrick added: “His officials, briefing the press, called it the mother of all – yeah, they’re not wrong, are they?”

Mr Lammy laughed as his Tory rival continued: “Calamity Lammy strikes again. It’s a national embarrassment and today the Justice Secretary feigns anger at what happened.”

The Justice Secretary told MPs that it “appears to have been human error” that led to Kebatu being mistakenly released and triggering a manhunt.

He said that “there must and there will be accountability” for the mistaken release of Hadush Kebatu from prison.

Mr Lammy told MPs: “I’ve been clear from the outset that a mistake of this nature is unacceptable. We must get to the bottom of what happened and take immediate action to try and prevent similar releases in error to protect the public from harm.”

Hitting back at Mr Jenrick, the senior Cabinet minister said: “This is a serious issue and that’s why there will be a full independent investigation. The shadow justice secretary, I’ll give him this, he’s smooth. But as my mother would have said, if he was chocolate, he’d lick himself.

The Deputy Prime Minister said that he ordered an “urgent review” into the checks that take place when someone is released from prison, and that new safeguards have been added that amount to the “strongest release checks that have ever been in place”.

The Justice Secretary said: “I instructed the CEO of His Majesty’s Prison and Probation Service (HMPPS) to carry out an urgent review to look at the checks that take place when a prisoner is released and identify immediate changes that can be made to the process in order to mitigate against the risks of release and error.

“As a result, HMPPS have taken steps to make these processes more robust. There will now be more direct senior accountability for ensuring protocols and checks are correctly applied, including a clear checklist for governors to determine that every step has been followed the evening before any release takes place.

“These are the strongest release checks that have ever been in place, and they will apply to every release from custody, and are effective immediately.”

Chelmsford MP Marie Goldman has said that “scapegoating a single prison officer for systemic failure is unacceptable”, adding that any prison governors found to be at fault should resign.

The Liberal Democrat MP said: “The trial, sentencing and mistaken release of Hadush Kebatu all took place in the heart of my Chelmsford constituency, and it is entirely unacceptable.

“I welcome the announcement of an independent investigation, but the Government must ensure that any recommendations are implemented in full, without delay, and not just left to gather dust on a bookshelf.

“Turning to prison leadership, does the Justice Secretary agree with me that scapegoating a single prison officer for systemic failure is unacceptable and that if leaders, including prison governors, are found to be at fault, they should resign.”

Responding, Mr Lammy said: “I know that this has been shocking for her constituents, and I know that the reports and the scenes of this prisoner wandering around Chelmsford will have been incredibly disturbing, and particularly disturbing for the victims.

“I know that we have suspended the officer involved pending an investigation, that must be right and proper. But I hope that she will recognise, in asking Dame Lynne Owens to look at this closely that, of course, we will take seriously her recommendations, and I will ensure that this House has full scrutiny of those recommendations when they come forward.”

Asked on Monday when Kebatu would be removed from the country, the Prime Minister’s official spokesperson said “you have us on record from this morning that we expect that to happen imminently”, and suggested that would be within a few days.

He said prison release errors “are never acceptable” and “this is another symptom of the justice system crisis inherited by this Government having suffered cuts to staffing, failure to build prison places” and “chronic underinvestment”.

It comes as the chief inspector of prisons said that mistakes over prisoner releases are happening “all the time” and are symptomatic of the chaos within the system.

Charlie Taylor said prisoners being released early, in error or even late is an “endemic problem” now that needs to be fixed by prison service leaders.