I found my skin is better without dairy

I was gonna lose weight, so I stopped eating dairy altogether, and I had a little cholesterol, my doctor said. Even protein-powder I avoided, because it was milk. I found that my skin looked better without it actually, yes, it looks kinda drier, leaner. it looks less shiny also from less milk-fats, have you experienced the same?