We read the story how the Jews complained about not having water to drink. Moshe was told to speak to the rock in front of the nation and water would come forth to satisfy the thirst of the people. Moshe hit the rock, the people drank and Moshe got punished for hitting rather than speaking.
In 20:12 Hashem punishes Moshe for hitting the rock, He says, “Because you did not have faith to sanctify Me in the eyes of Bnei Yisrael, etc..”. Rashi explains this verse that had Moshe spoken to the rock, the people would have extrapolated from the event that just as the rock was spoken to and did as it was told despite the fact that it receives no punishment for not listening, how much more so do I have to listen to Hashem’s word when I do get punished for not listening.
I was thinking about this “kal v’chomer” that the people would make to learn the lesson Hashem wanted them/us to learn. It seems that had Moshe spoken they would have naturally come up on their own with the above extrapolation. That means, I think, that now that Moshe hit the rock, they are coming up naturally with a different extrapolation. That kal v’chomer must have been – “Moshe hit the rock and it listened, so Hashem will hit us and we will listen (I.e. if we do not listen Hashem will hit us).”
If my thoughts are correct that the lesson we were supposed to learn was avoided and instead we learned a different lesson, it is understandable why Moshe was punished so harshly for a seemingly slight mistake (after all, hitting a rock to draw water is also a pretty big miracle). That would be because look at the lesson he taught them. He taught them (or at least made them think so) by his actions that God is a harsh God and quick to hit to have people act in accordance with His will. That is not the lesson Hashem wanted conveyed, because it is not true.