You are correct. The "crown" is the dome-like section of the hat that directly covers your head. The height of the un-shaped crown (commonly known as "open crown"), measured from the brim up, varies from style to style depending on how the crown will ultimately be shaped. It's quite simple, really--the taller the crown, the more material you have to work with to achieve the shape you want, or, in your case, to accommodate the wearer's head.
I never thought I had a particularly "tall" head myself, but once I started wearing hats regularly I learned rather quickly that I also need hats with higher/taller crowns.
Kings and Queens always have nice crowns....weeee doggies! Haha!