robots.txt olmalı hocam.
Sitenizin robots.txt'si;
https://www.literalwise.com/robots.txt

Sizin eklediğiniz robot;
https://www.literalwise.com/robot.txt