I actually disagree. I think it added to his character. Sure, dick move, but as a Yonko crew It was necessary to show they're not to be fucked with by anyone, that they mean business and you will never challenge their authority and definitely not encroach upon their territory and get away with it because you're a friend of. They did burn their flag which is practically a declaration of war. I also believe he knew they'd survive since he didn't seem to have any intentions of killing them. And sure, that explosion and narration seem like he didn't give a shit but we all know how One Piece is when it comes to these type of moments.
I see it as Yasopp upholding the standard to an extent which shows you how much he values their crew and flag, and what it means to him. Just because you've proven loyalty to a friendly crew of theirs doesn't mean you get to piss on their flag and get away with it.
Also, it appears that Barto got the wrong idea that they're "soft" because of that "pass" he got. So him going around running his mouth on how he burned the Red Hair flag and got away with it free of charge could give other punks the wrong idea.