I feel healthier and better in general when I have access to milk and other milk products, as well as meat, whole grains, fruits and vegetables, etc. As a child I had a very restricted diet because my mother was fascinated with nutrition and tried to make sure we ate "perfectly", thus ensuring that we were missing variety in our diets... When I left home, I grew two inches, probably making up for lost time. Nowadays, I can think better and deal with sensory stuff better, because I can buy meat and milk whenever I like. (We had enough fruit and vegetables growing up.) I'm also physically more energetic--both meat and milk are high energy foods.
A restricted diet can lead to euphoria and better focus, in the early stages, because your body taps into its reserves to try to give you energy to find more food. But in the long term, you'll run out of energy and your body will have to cut corners on things like growth, attention, concentration, and higher-level sensory and cognitive skills. Unless you cannot digest an otherwise nutritious food, or are allergic to it, I see little reason to exclude it from your diet.