Package | Description |
---|---|
crawlercommons.filters.basic | |
crawlercommons.sitemaps |
Sitemaps package provides all classes relevant to focused sitemap parsing,
url definition and processing.
|
Modifier and Type | Class and Description |
---|---|
class |
BasicURLNormalizer
Code borrowed from Apache Nutch.
|
Modifier and Type | Method and Description |
---|---|
void |
SiteMapParser.setURLFilter(URLFilter filter)
Use
URLFilter to filter URLs, eg. |
Copyright © 2009–2021 Crawler-Commons. All rights reserved.