You should utilize a robots.txt file to dam resource files like unimportant image, script, or design and style data files, should you believe webpages loaded with no these resources will not be significantly afflicted by the reduction.large-ranging troubles like who really should be chargeable for robots, what legal rights they must have, And exact