Was my dad really justified in being annoyed about this, all things considered?

It is definitely overbearing especially if it is your own money. I think he is trying to teach you responsibility but might be taking too seriously.