The answer is that you may do. If your ex-partner has Parental Responsibility for the children then he will need to give permission for you to permanently remove them from this country to live elsewhere.
You should check first with a Solicitor to see if your ex has Parental Responsibility. If he does then speak to him and see if you can get an agreement. You will need to think about the contact he will have with the children once you have left. This may be less regular but for longer periods. You will also need to arrange telephone and internet contact and if you are able to set up Skype so that he can both see and talk to the children then that is even better.
If you can't get him to agree then you may need to apply to the Court for leave to remove the children. This can be a lengthy process and you should therefore leave as much time as possible to deal with this. As long as adequate contact is arranged and there is no risk of harm to the children in the country they are going to then usually the Court would agree leave to remove.