Uses of Class
crawlercommons.robots.SimpleRobotRules

Packages that use SimpleRobotRules

Package	Description
crawlercommons.robots	The robots package contains all of the robots.txt rule inference, parsing and utilities contained within Crawler Commons.

Uses of SimpleRobotRules in crawlercommons.robots

Methods in crawlercommons.robots that return SimpleRobotRules

Modifier and Type	Method	Description
SimpleRobotRules	SimpleRobotRulesParser.failedFetch(int httpStatusCode)
SimpleRobotRules	SimpleRobotRulesParser.parseContent(String url, byte[] content, String contentType, String robotNames)