crawlercommons.fetcher.http

Class SimpleHttpFetcher

java.lang.Object

crawlercommons.fetcher.BaseFetcher

crawlercommons.fetcher.http.BaseHttpFetcher

crawlercommons.fetcher.http.SimpleHttpFetcher

All Implemented Interfaces:

Serializable

Deprecated.

As of release 0.6. We recommend directly using Apache HttpClient, async-http-client, or any other robust, industrial-strength HTTP clients.

@Deprecated
public class SimpleHttpFetcher
extends BaseHttpFetcher

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SimpleHttpFetcher.IdleConnectionMonitorThread Deprecated.
static class	SimpleHttpFetcher.MyConnectionKeepAliveStrategy Deprecated.

Nested classes/interfaces inherited from class crawlercommons.fetcher.http.BaseHttpFetcher

BaseHttpFetcher.RedirectMode

Field Summary

Fields inherited from class crawlercommons.fetcher.http.BaseHttpFetcher

_acceptLanguage, _maxConnectionsPerHost, _maxRedirects, _maxThreads, _minResponseRate, _redirectMode, _userAgent, DEFAULT_ACCEPT_LANGUAGE, DEFAULT_MAX_CONNECTIONS_PER_HOST, DEFAULT_MAX_REDIRECTS, DEFAULT_MIN_RESPONSE_RATE, DEFAULT_REDIRECT_MODE, NO_MIN_RESPONSE_RATE, NO_REDIRECTS

Fields inherited from class crawlercommons.fetcher.BaseFetcher

_defaultMaxContentSize, _maxContentSizes, _validMimeTypes, DEFAULT_MAX_CONTENT_SIZE

Constructor Summary

Constructors

Constructor and Description
SimpleHttpFetcher(int maxThreads, UserAgent userAgent) Deprecated.
SimpleHttpFetcher(UserAgent userAgent) Deprecated.

Method Summary

Modifier and Type	Method and Description
void	abort() Deprecated. Terminate any async request being processed.
FetchedResult	fetch(org.apache.http.client.methods.HttpRequestBase request, String url, Payload payload) Deprecated.
FetchedResult	fetch(String url) Deprecated.
protected void	finalize() Deprecated.
FetchedResult	get(String url, Payload payload) Deprecated. Get the content stored in the resource referenced by the 'url' parameter.
int	getConnectionRequestTimeout() Deprecated.
int	getConnectionTimeout() Deprecated.
org.apache.http.HttpVersion	getHttpVersion() Deprecated.
int	getMaxRetryCount() Deprecated.
int	getSocketTimeout() Deprecated.
void	setConnectionRequestTimeout(int _connectionRequestTimeoutInMs) Deprecated.
void	setConnectionTimeout(int connectionTimeoutInMs) Deprecated.
void	setHttpVersion(org.apache.http.HttpVersion httpVersion) Deprecated.
void	setMaxRetryCount(int maxRetryCount) Deprecated.
void	setSocketTimeout(int socketTimeoutInMs) Deprecated.

Methods inherited from class crawlercommons.fetcher.http.BaseHttpFetcher

getAcceptLanguage, getMaxConnectionsPerHost, getMaxRedirects, getMaxThreads, getMinResponseRate, getRedirectMode, getUserAgent, setAcceptLanguage, setMaxConnectionsPerHost, setMaxRedirects, setMinResponseRate, setRedirectMode

Methods inherited from class crawlercommons.fetcher.BaseFetcher

addValidMimeType, addValidMimeTypes, get, getDefaultMaxContentSize, getMaxContentSize, getMimeTypeFromContentType, getValidMimeTypes, setDefaultMaxContentSize, setMaxContentSize, setValidMimeTypes

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleHttpFetcher

public SimpleHttpFetcher(UserAgent userAgent)

Deprecated.

SimpleHttpFetcher

public SimpleHttpFetcher(int maxThreads,
                         UserAgent userAgent)

Deprecated.

Method Detail

getHttpVersion

public org.apache.http.HttpVersion getHttpVersion()

Deprecated.

setHttpVersion

public void setHttpVersion(org.apache.http.HttpVersion httpVersion)

Deprecated.

getSocketTimeout

public int getSocketTimeout()

Deprecated.

setSocketTimeout

public void setSocketTimeout(int socketTimeoutInMs)

Deprecated.

getConnectionTimeout

public int getConnectionTimeout()

Deprecated.

setConnectionTimeout

public void setConnectionTimeout(int connectionTimeoutInMs)

Deprecated.

getConnectionRequestTimeout

public int getConnectionRequestTimeout()

Deprecated.

setConnectionRequestTimeout

public void setConnectionRequestTimeout(int _connectionRequestTimeoutInMs)

Deprecated.

getMaxRetryCount

public int getMaxRetryCount()

Deprecated.

setMaxRetryCount

public void setMaxRetryCount(int maxRetryCount)

Deprecated.

get

public FetchedResult get(String url,
                         Payload payload)
                  throws BaseFetchException

Deprecated.

Description copied from class: BaseFetcher

Get the content stored in the resource referenced by the 'url' parameter.

Specified by:

get in class BaseFetcher

Parameters:

url - a string url for which to get content

payload - a populated Payload

Returns:

the FetchedResult associated with the URL

Throws:

BaseFetchException - if an error results from fetching the url.

fetch

public FetchedResult fetch(String url)
                    throws BaseFetchException

Deprecated.

Throws:

BaseFetchException

fetch

public FetchedResult fetch(org.apache.http.client.methods.HttpRequestBase request,
                           String url,
                           Payload payload)
                    throws BaseFetchException

Deprecated.

Throws:

BaseFetchException

abort

public void abort()

Deprecated.

Description copied from class: BaseFetcher

Terminate any async request being processed.

Specified by:

abort in class BaseFetcher

finalize

protected void finalize()

Deprecated.

Overrides:

finalize in class Object