should have

Use "should have" and then the main verb is in the form of a past participle when talking about mistakes or things that you regret.