Outside of the athletic dept. it really wasn't a bad place. However, their insane desire to be relevant amongst the ginormous public schools it is in the Big XII with led to some absolutely disastrous decisions that have painted a permanent black mark on the school as a whole. And you think the Briles stuff was bad, just go back a few years further and remember that basketball players were murdering other players while Dave Bliss was coach.
I did get chastised once for asking why Housing and Residential Life was looking at retention numbers at OU, Texas, and A&M and not places like Rice, SMU, TCU, TU. I was firmly told that they only compared numbers to other Big XII schools and I sort of hit back with that the student body at Baylor had very few similarities to other Big XII institutions outside of regionality. Even doing cost comparisons to those other schools was pretty futile because they're not alone at all.
Anyway, I worked with a lot of good people at Baylor and I had almost no contact with anyone from the athletic dept or even any athletes as they all were sort of isolated from the general student body. My first contact with the athletic dept was probably the meet and greet they arranged for student life when Bliss was hired and he was trying to generate interest in students going to games b/c in those days attendance at games outside of baseball was woeful.