They will swear by Allāh to you when you return to them that you would leave them alone. So leave them alone; indeed they are evil; and their refuge is Hell as recompense for what they had been earning.
Those who stay behind will swear by Allah when you, O believers, return to them, in order to strengthen their false oaths, so that you do not blame them. Therefore leave them alone and ignore them. They are filthy and impure from the inside. They will end up in Hell, as a repayment to them for their hypocrisy and the sins they committed.