Package | Description |
---|---|
crawlercommons.fetcher |
The main fetching package within Crawler Commons, this package defines base fetching
and encoding classes, Enum's to determine reasoning behind typical fetching behaviour
as well as the base Exceptions which may be used.
|
crawlercommons.fetcher.file |
SimpleFileFetcher code which extends the BaseFetcher . |
crawlercommons.fetcher.http |
This package concerns the fetching of files over the HTTP protocol:
Extending from
BaseHttpFetcher (which itself extends BaseFetcher ) the
SimpleHttpFetcher provides the Crawler Commons HTTP fetching implementation. |
Modifier and Type | Method and Description |
---|---|
Payload |
FetchedResult.getPayload()
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
abstract FetchedResult |
BaseFetcher.get(String url,
Payload payload)
Deprecated.
Get the content stored in the resource referenced by the 'url' parameter.
|
void |
FetchedResult.setPayload(Payload payload)
Deprecated.
|
Constructor and Description |
---|
FetchedResult(String baseUrl,
String redirectedUrl,
long fetchTime,
org.apache.tika.metadata.Metadata headers,
byte[] content,
String contentType,
int responseRate,
Payload payload,
String newBaseUrl,
int numRedirects,
String hostAddress,
int statusCode,
String reasonPhrase)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
FetchedResult |
SimpleFileFetcher.get(String url,
Payload payload)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
FetchedResult |
SimpleHttpFetcher.fetch(org.apache.http.client.methods.HttpRequestBase request,
String url,
Payload payload)
Deprecated.
|
FetchedResult |
SimpleHttpFetcher.get(String url,
Payload payload)
Deprecated.
|
Copyright © 2009–2016 Crawler-Commons. All rights reserved.