Robots.txt Not Cumulative

Yesterday, gs1md wrote a meaty post on WMW regarding how Googlebot responds to robots.txt containing directives for both User-Agent:* and User-Agent: Googlebot. Both Googleguy and Vanessa Fox dropped by to clarify the situation: When you use both specifications, Googlebot will go with User-Agent: Googlebot.

Related links:

Related Posts

What's Your Take?