Awesome new job, but bosses management style is a challenge

I started a new job about 3 months ago. I love everything about it, except for my bosses management style. He seems to be a perfectionist who is resentful that he now has to be a manager. I am his only direct report. He has been extremely critical of any errors, even as minor as 1 letter being incorrectly capitalized in my work. To be fair to him, it’s never personal. It’s always about the work. Still, I spent a day in tears because he snapped at me that he “didn’t care why” when I tried to explain that he’s holding me to a standard no one else in the company is consistently meeting, not even our head of engineering. Since I started, I’ve learned a new programming language and have really been proactive in my work. The customers are happy with what I am doing. Because he’s so critical of small mistakes, I’m afraid every day that if I make a rookie mistake that actually is a big deal that he won’t have my back and will just fire me on the spot. He doesn’t seem interested in taking about anything, so I hope someone has ideas how I can earn the trust of Mr. Uptight and get him to back off of me a little so that I have a chance to succeed in my new role. I do not want to quit or go to HR because I believe this is just a difference of style and normal growing pains. I hope someone here is a Type A boss dealing with an employee who is less detail oriented and can give me some tips.