He is scared, but also feels lust for her. It's his wedding day, but the first chance he gets, he's going to run and hide! He's not king yet, so he can't banish her to the Maidenvault until then. Oh Baelor!
Okay, I get that. The realm just wasn't lucky when it came to Baelor, if he had done his duty (like a pious man should in the image of the Father) then Daemon Blackfyre probably wouldn't have been born and we all know how that turned out.