Baby Vegeta at his peak is definitely above Buuhan Goku doesn’t have to fight him to know, he’s already felt Buuhan’s ki before. It’s not a fighting thing, it’s a “he’s already seen the power” thing if you’re referring to Baby’s weaker forms (anything below SBV1), then I can see what you mean if you want to follow the manga route