He's saying it based on the nature of how HF Shirou performed against a few Servants... the problem is that he ignores context on HOW HF Shirou got those wins:
He states that Shirou defeated Heracles... but if you read the VN(As the movie doesn't show the context of how absolutely fucked up Heracles was then), Heracles did not have:
- God Hand
- Hearing
- SightYes really)
- Cognition(Believed he was still fighting Saber Alter)
- Healthy(was still heavily injured)
Hell, Nasu has stated in the Blu-Ray Set Manual that if Heracles was not in that condition, not only would have Nine Lives Blade Works not happened, Heracles would have instinctively countered all of it and considering the bullshit Heracles has done in the franchise? I fully believe it. And that's not counting if Illya wasn't there, Heracles would have STILL killed Shirou there.
Otherwise, we do see Shirou vs. Medusa in HF... if he tries to kill Sakura in her sleep, Medusa effortlessly kills him before he even registers she's there.
Again, not really sure why they have this battle as again, Shiki Ryougi is the worst nightmare for any character in Frieren.