Thanks for sharing your thoughts John.
Imagine the frustration of a freshly starting musician who first encounters this clash of opinions. Someone new to the field likely has no opinion on the importance of music theory yet. He runs into it and seeks to understand it, but then comes across all these different people (including teachers) who have a very strong stance on it in either extreme. He may then adopt such a person's opinion as his own, without really having weighed all the arguments. This certainly could have happened to me.
When it comes to the debate Is music theory necessary? I would first like to ask: necessary for what? And of course it would help to define what we mean by music theory. You seem to go with music theory as analysis, but then also as form/structure. When I started learning about music and working through musictheory.net my understanding of music theory was that it provides the basic building blocks of music (note lengths, rests, keys, chords etc.). You could draw the analogy with the difference between analyzing a painting by Picasso and learning the names of different colors, introduction to materials and how to set up your canvas.
So with my definition, I would say the theory is fundamental. As analysis I would find it more of an advanced tool that is helpful to achieve more advance purposes. But in terms of necessity? We can agree on the fact that people can play music and write music without necessarily having the vocabulary to go with what they're doing. In the same way, people can draw nice pictures or make good photographs without having the vocabulary or insight into color and composition theory, or aesthetics. I do think that someone who wants to be serious about his craft would do well to immerse himself in the principles, theories and even "rules" that come with it in order to create more effectively and to more consistently "nail it" with his creative endeavors. Knowing what you're doing is generally helpful in every area of life.
Personally, I'm very analytically oriented and I like to have a grasp of basics, definitions, principles and structures. My teacher will teach me some music theory if I ask for it and if it is relevant to what I'm trying to do, but for the most part she seems to believe that my creativity will be stifled if we get too much into forms and all that. I feel kind of shut down by that kind of attitude. She fears that I will take too much of a mathematical approach and end up "programming" music, rather than writing it. I don't think that's what I'd do at all. I just want to understand what's going on and use specific existing forms as stepping stones to more original music. I'm doing something now, but without more direction I'll likely be stuck doing the same thing over an over. I want to explore the field of music and see what is out there and what I can learn from it. That doesn't have to mean I'll copy everything and lose originality (if that even exists anymore).
Lastly, I think music theory in general suffers from a bad reputation as being unpractical, which may explain its de-emphasis in a school setting and the low expectations teachers have from their students. If it is not meaningfully connected to something tangible (as John described with his guitar class) it will stay in the unpractical. And if unpractical, why bother with it?
It probably also doesn't help when a teacher is not a composer/musician himself/herself. I'm trying to work my way through Paul Hindemith's works on composition (part 1 and 2). In the introduction he laments the state of music theory education and names the lack of composer-teachers as part of the problem. If the teacher is not immersed in the craft himself, how is he expected to meaningfully transmit the knowledge to others?
Unfortunately for music theory, it does tend to get a little dry and technical, even a little esoteric when you get to things like harmonics and get the math and physics involved. I find it not as accessible as other subjects, though I imagine you could get esoteric with color theory too if you bring in the frequencies of colors 0_0;