I don’t want to ask him about this because I afraid he will get ashamed of this.

But the key to any good, strong, healthy relationship is comunication. All we can do is take our best guess as what’s going on, you won’t actually know though, until you talk to him. If you want this relationship to work, you(both) need to get comfortable communicating even when it’s awkward or dificult.