As someone who watched the anime as a kid, but has only recently started to read through the manga, I don't think it really matters but I would advise going through the manga first and then watch the anime.
Not only does the manga give you more insight into each of the characters, making them even more likable than their anime counterparts, but the anime elected to skip passed the first 58 chapters of the manga. This is because the manga was originally much darker than the anime and was focused on games in general with the idea of Yugi punishing bad people by putting them through shadow games and hitting them with often fucked up penalties.
Basically you miss several key events if you start with the anime or when they are shown (such as episode 1's duel with Kaiba, which is actually adapted from their second duel in the manga) they don't have all the build up and gravitas that went into them originally.
Of course I still heavily recommend the anime after you finish the manga, because the other Yugioh series (GX, 5D, ZEXAL etc.) all follow after the ANIME as opposed to the manga. Never mind the fact the anime is still great in it's own right, with its two filler arcs ranging from pretty good (virtual world arc) to one of the best arcs in the entire franchise (waking the dragons).