freenet.client.async

Class ClientGetter

java.lang.Object

freenet.client.async.ClientRequester

freenet.client.async.BaseClientGetter

freenet.client.async.ClientGetter

All Implemented Interfaces:

ClientRequestSchedulerGroup, FileGetCompletionCallback, GetCompletionCallback, WantsCooldownCallback, java.io.Serializable

public class ClientGetter
extends BaseClientGetter
implements WantsCooldownCallback, FileGetCompletionCallback, java.io.Serializable

A high level data request. Follows redirects, downloads splitfiles, etc. Similar to what you get from FCP, and is used internally to implement FCP. Also used by fproxy, and plugins, and so on. The current state of the request is stored in currentState. The ClientGetState's do most of the work. SingleFileFetcher for example fetches a key, parses the metadata, and if necessary creates other states to e.g. fetch splitfiles.

See Also:

Serialized Form

Field Summary

Fields inherited from class freenet.client.async.ClientRequester

blockSetFinalized, cancelled, client, creationTime, failedBlocks, fatallyFailedBlocks, latestFailure, latestSuccess, minSuccessBlocks, priorityClass, realTimeFlag, sentToNetwork, successfulBlocks, totalBlocks

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ClientGetter() Required because we implement Serializable.
	ClientGetter(ClientGetCallback client, FreenetURI uri, FetchContext ctx, short priorityClass)
	ClientGetter(ClientGetCallback client, FreenetURI uri, FetchContext ctx, short priorityClass, Bucket returnBucket)
	ClientGetter(ClientGetCallback client, FreenetURI uri, FetchContext ctx, short priorityClass, Bucket returnBucket, BinaryBlobWriter binaryBlobWriter)
	ClientGetter(ClientGetCallback client, FreenetURI uri, FetchContext ctx, short priorityClass, Bucket returnBucket, BinaryBlobWriter binaryBlobWriter, boolean dontFinalizeBlobWriter, Bucket initialMetadata)
	ClientGetter(ClientGetCallback client, FreenetURI uri, FetchContext ctx, short priorityClass, Bucket returnBucket, BinaryBlobWriter binaryBlobWriter, boolean dontFinalizeBlobWriter, Bucket initialMetadata, java.lang.String forceCompatibleExtension) Fetch a key.
	ClientGetter(ClientGetCallback client, FreenetURI uri, FetchContext ctx, short priorityClass, Bucket returnBucket, BinaryBlobWriter binaryBlobWriter, Bucket initialMetadata)

Method Summary

Modifier and Type	Method and Description
protected void	addKeyToBinaryBlob(ClientKeyBlock block, ClientContext context) Add a block to the binary blob.
void	cancel(ClientContext context) Cancel the request.
boolean	canRestart() Can the request be restarted?
void	clearCooldown(ClientGetState state) The request has unexpectedly left cooldown.
protected void	clearCountersOnRestart()
protected boolean	collectingBinaryBlob() Are we collecting a binary blob?
void	enterCooldown(ClientGetState state, long wakeupTime, ClientContext context) The request has gone into cooldown for some period.
java.lang.String	expectedMIME()
long	expectedSize()
boolean	finalizedMetadata() Are we sure the expected MIME and size won't change?
Bucket	getBlobBucket()
SnoopBucket	getBucketSnoop() Get the intermediate data snoop callback
protected ClientBaseCallback	getCallback()
ClientGetCallback	getClientCallback()
byte[]	getClientDetail(ChecksumChecker checker)
java.io.File	getCompletionFile() Get the final location of the downloaded data.
SnoopMetadata	getMetaSnoop() Get the metadata snoop callback
FreenetURI	getURI() What was the URI we were fetching?
protected void	innerNotifyClients(ClientContext context) Notify clients listening to our ClientEventProducer of the current progress, in the form of a SplitfileProgressEvent.
void	innerOnResume(ClientContext context) Called for a persistent request after startup.
protected void	innerToNetwork(ClientContext context) Notify clients that some part of the request has been sent to the network i.e.
boolean	isCurrentState(ClientGetState state)
boolean	isFinished() Has the fetch completed?
void	onBlockSetFinished(ClientGetState state, ClientContext context) Called when no more blocks will be added to the total, and therefore we can confidently display a percentage for the overall progress.
void	onExpectedMIME(ClientMetadata clientMetadata, ClientContext context) Called when we know the MIME type of the final data
void	onExpectedSize(long size, ClientContext context) Called when we have some idea of the length of the final data
void	onExpectedTopSize(long size, long compressed, int blocksReq, int blocksTotal, ClientContext context) Called when we know the size of the final file, and the number of blocks needed etc.
void	onFailure(FetchException e, ClientGetState state, ClientContext context) Called when the request fails.
void	onFailure(FetchException e, ClientGetState state, ClientContext context, boolean force) Internal version.
void	onFinalizedMetadata() Called when we are fairly sure that the expected MIME and size won't change
void	onHashes(HashResult[] hashes, ClientContext context) Called when we know the HashResult of the final file.
void	onShutdown(ClientContext context) Called just before the final write when shutting down the node.
void	onSplitfileCompatibilityMode(InsertContext.CompatibilityMode min, InsertContext.CompatibilityMode max, byte[] customSplitfileKey, boolean dontCompress, boolean bottomLayer, boolean definitiveAnyway, ClientContext context) Called when we know the settings for the splitfile.
void	onSuccess(java.io.File tempFile, long length, ClientMetadata metadata, ClientGetState state, ClientContext context) Call when the download has completed and the tempFile contains the downloaded data, but may be too long.
void	onSuccess(StreamGenerator streamGenerator, ClientMetadata clientMetadata, java.util.List<? extends Compressor> decompressors, ClientGetState state, ClientContext context) Called when the request succeeds.
void	onTransition(ClientGetState oldState, ClientGetState newState, ClientContext context) Called when the current state creates a new state and we switch to that.
boolean	restart(FreenetURI redirect, boolean filterData, ClientContext context) Restart the request.
boolean	resumedFetcher()
boolean	resumeFromTrivialProgress(java.io.DataInputStream dis, ClientContext context)
SnoopBucket	setBucketSnoop(SnoopBucket newSnoop) Set a callback to snoop on buckets (all intermediary data - metadata, containers) during fetches.
SnoopMetadata	setMetaSnoop(SnoopMetadata newSnoop) Set a callback to snoop on metadata during fetches.
boolean	start(boolean restart, FreenetURI overrideURI, ClientContext context) Start the request.
void	start(ClientContext context)
java.lang.String	toString()
boolean	writeTrivialProgress(java.io.DataOutputStream dos) If the request is simple, e.g.

Methods inherited from class freenet.client.async.ClientRequester

addBlock, addBlocks, addMustSucceedBlocks, addRedundantBlocksInsert, blockSetFinalized, cancel, completedBlock, failedBlock, failedBlock, fatallyFailedBlock, getAll, getClient, getClientDetail, getLatestFailure, getLatestSuccess, getPriorityClass, getSchedulerGroup, getTotalBlocks, hashCode, isCancelled, notifyClients, onResume, persistent, realTimeFlag, resetBlocks, setPriorityClass, toNetwork

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ClientGetter

public ClientGetter(ClientGetCallback client,
                    FreenetURI uri,
                    FetchContext ctx,
                    short priorityClass)

ClientGetter

public ClientGetter(ClientGetCallback client,
                    FreenetURI uri,
                    FetchContext ctx,
                    short priorityClass,
                    Bucket returnBucket)

ClientGetter

public ClientGetter(ClientGetCallback client,
                    FreenetURI uri,
                    FetchContext ctx,
                    short priorityClass,
                    Bucket returnBucket,
                    BinaryBlobWriter binaryBlobWriter)

ClientGetter

public ClientGetter(ClientGetCallback client,
                    FreenetURI uri,
                    FetchContext ctx,
                    short priorityClass,
                    Bucket returnBucket,
                    BinaryBlobWriter binaryBlobWriter,
                    Bucket initialMetadata)

ClientGetter

public ClientGetter(ClientGetCallback client,
                    FreenetURI uri,
                    FetchContext ctx,
                    short priorityClass,
                    Bucket returnBucket,
                    BinaryBlobWriter binaryBlobWriter,
                    boolean dontFinalizeBlobWriter,
                    Bucket initialMetadata)

ClientGetter

public ClientGetter(ClientGetCallback client,
                    FreenetURI uri,
                    FetchContext ctx,
                    short priorityClass,
                    Bucket returnBucket,
                    BinaryBlobWriter binaryBlobWriter,
                    boolean dontFinalizeBlobWriter,
                    Bucket initialMetadata,
                    java.lang.String forceCompatibleExtension)

Fetch a key.

Parameters:

client - The callback we will call when it is completed.

uri - The URI to fetch.

ctx - The config settings for the fetch.

priorityClass - The priority at which to schedule the request.

clientContext - The context object. Used for round-robin query balancing, also indicates whether the request is persistent.

returnBucket - The bucket to return the data in. Can be null. If not null, the ClientGetter must either write the data directly to the bucket, or copy it and free the original temporary bucket. Preferably the former, obviously!

binaryBlobWriter - If non-null, we will write all the keys accessed (or that could have been accessed in the case of redundant structures such as splitfiles) to this binary blob writer.

dontFinalizeBlobWriter - If true, the caller is responsible for BlobWriter finalization

ClientGetter

protected ClientGetter()

Required because we implement Serializable.

Method Detail

start

public void start(ClientContext context)
           throws FetchException

Throws:

FetchException

start

public boolean start(boolean restart,
                     FreenetURI overrideURI,
                     ClientContext context)
              throws FetchException

Start the request.

Parameters:

restart - If true, restart a finished request.

overrideURI - If non-null, change the URI we are fetching (usually when restarting).

container - The database, null if this is a non-persistent request; if this is a persistent request, we must be on the database thread, and we will pass the database handle around as needed.

context - The client context, contains important mostly non-persistent global objects.

Returns:

True if we restarted, false if we didn't (but only in a few cases).

Throws:

FetchException - If we were unable to restart.

clearCountersOnRestart

protected void clearCountersOnRestart()

Overrides:

clearCountersOnRestart in class ClientRequester

onSuccess

public void onSuccess(StreamGenerator streamGenerator,
                      ClientMetadata clientMetadata,
                      java.util.List<? extends Compressor> decompressors,
                      ClientGetState state,
                      ClientContext context)

Called when the request succeeds.

Specified by:

onSuccess in interface GetCompletionCallback

Parameters:

state - The ClientGetState which retrieved the data.

onSuccess

public void onSuccess(java.io.File tempFile,
                      long length,
                      ClientMetadata metadata,
                      ClientGetState state,
                      ClientContext context)

Description copied from interface: FileGetCompletionCallback

Call when the download has completed and the tempFile contains the downloaded data, but may be too long. The callback must truncate the file, check the hashes on the file and complete the request.

Specified by:

onSuccess in interface FileGetCompletionCallback

Parameters:

tempFile - A file in the same directory as the completion file, containing the downloaded data.

length - The length of the downloaded data.

metadata - The MIME type of the downloaded data.

state - The calling ClientGetState.

context - Contains run-time support structures such as executors, temporary storage factories etc. Not static because we want to be able to run multiple nodes in one VM for tests etc.

onFailure

public void onFailure(FetchException e,
                      ClientGetState state,
                      ClientContext context)

Called when the request fails. Retrying will have already been attempted by the calling state, if appropriate; we have tried to get the data, and given up.

Specified by:

onFailure in interface GetCompletionCallback

Parameters:

e - The reason for failure, in the form of a FetchException.

state - The failing state.

onFailure

public void onFailure(FetchException e,
                      ClientGetState state,
                      ClientContext context,
                      boolean force)

Internal version. Adds one parameter.

Parameters:

force - If true, finished may already have been set. This is usually set when called from onSuccess after it has set finished = true.

cancel

public void cancel(ClientContext context)

Cancel the request. This must result in onFailure() being called in order to send the client a cancel FetchException, and to removeFrom() the state.

Specified by:

cancel in class ClientRequester

Parameters:

context - The ClientContext object including essential but non-persistent objects such as the schedulers.

isFinished

public boolean isFinished()

Has the fetch completed?

Specified by:

isFinished in class ClientRequester

getURI

public FreenetURI getURI()

What was the URI we were fetching?

Specified by:

getURI in class ClientRequester

innerNotifyClients

protected void innerNotifyClients(ClientContext context)

Notify clients listening to our ClientEventProducer of the current progress, in the form of a SplitfileProgressEvent.

Specified by:

innerNotifyClients in class ClientRequester

innerToNetwork

protected void innerToNetwork(ClientContext context)

Notify clients that some part of the request has been sent to the network i.e. we have finished checking the datastore for at least some part of the request. Sent once only for any given request.

Specified by:

innerToNetwork in class ClientRequester

onBlockSetFinished

public void onBlockSetFinished(ClientGetState state,
                               ClientContext context)

Called when no more blocks will be added to the total, and therefore we can confidently display a percentage for the overall progress. Will notify clients with a SplitfileProgressEvent.

Specified by:

onBlockSetFinished in interface GetCompletionCallback

onTransition

public void onTransition(ClientGetState oldState,
                         ClientGetState newState,
                         ClientContext context)

Called when the current state creates a new state and we switch to that. For example, a SingleFileFetcher might switch to a SplitFileFetcher. Sometimes this will be called with oldState not equal to our currentState; this means that a subsidiary request has changed state, so we ignore it.

Specified by:

onTransition in interface GetCompletionCallback

Specified by:

onTransition in class ClientRequester

Parameters:

oldState - The old ClientGetState.

newState - The new ClientGetState.

context - The database handle. Must not be used by other threads.

canRestart

public boolean canRestart()

Can the request be restarted?

restart

public boolean restart(FreenetURI redirect,
                       boolean filterData,
                       ClientContext context)
                throws FetchException

Restart the request.

Parameters:

redirect - Use this URI instead of the old one.

filterData -

context - The database. We must be on the database thread! See ClientContext for convenience methods.

Returns:

True if we successfully restarted, false if we can't restart.

Throws:

FetchException - If something went wrong.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

addKeyToBinaryBlob

protected void addKeyToBinaryBlob(ClientKeyBlock block,
                                  ClientContext context)

Add a block to the binary blob.

collectingBinaryBlob

protected boolean collectingBinaryBlob()

Are we collecting a binary blob?

onExpectedMIME

public void onExpectedMIME(ClientMetadata clientMetadata,
                           ClientContext context)
                    throws FetchException

Called when we know the MIME type of the final data

Specified by:

onExpectedMIME in interface GetCompletionCallback

Parameters:

clientMetadata - The MIME type, possibly including parameters, as a String. E.g. "text/html; charset=ISO-8859-1".

context - Utility object containing helpers, mostly not persistent, such as the Ticker, temporary storage factories etc.

Throws:

FetchException

onExpectedSize

public void onExpectedSize(long size,
                           ClientContext context)

Called when we have some idea of the length of the final data

Specified by:

onExpectedSize in interface GetCompletionCallback

Parameters:

size - The expected size of the final data.

context - Utility object containing helpers, mostly not persistent, such as the Ticker, temporary storage factories etc.

onFinalizedMetadata

public void onFinalizedMetadata()

Called when we are fairly sure that the expected MIME and size won't change

Specified by:

onFinalizedMetadata in interface GetCompletionCallback

finalizedMetadata

public boolean finalizedMetadata()

Are we sure the expected MIME and size won't change?

expectedMIME

public java.lang.String expectedMIME()

Returns:

The expected MIME type, if we know it.

expectedSize

public long expectedSize()

Returns:

The expected size of the returned data, if we know it. Could change.

getClientCallback

public ClientGetCallback getClientCallback()

Returns:

The callback to be notified when we complete the request.

getMetaSnoop

public SnoopMetadata getMetaSnoop()

Get the metadata snoop callback

setMetaSnoop

public SnoopMetadata setMetaSnoop(SnoopMetadata newSnoop)

Set a callback to snoop on metadata during fetches. Call this before starting the request.

getBucketSnoop

public SnoopBucket getBucketSnoop()

Get the intermediate data snoop callback

setBucketSnoop

public SnoopBucket setBucketSnoop(SnoopBucket newSnoop)

Set a callback to snoop on buckets (all intermediary data - metadata, containers) during fetches. Call this before starting the request.

onExpectedTopSize

public void onExpectedTopSize(long size,
                              long compressed,
                              int blocksReq,
                              int blocksTotal,
                              ClientContext context)

Description copied from interface: GetCompletionCallback

Called when we know the size of the final file, and the number of blocks needed etc. For recent metadata, this is known at the time of handling the top block.

Specified by:

onExpectedTopSize in interface GetCompletionCallback

Parameters:

size - The final size of the data.

compressed - The size of the data after compression / before decompression.

blocksReq - The number of blocks needed to decode the file.

blocksTotal - The total number of blocks available.

context - Utility object containing helpers, mostly not persistent, such as the Ticker, temporary storage factories etc.

onSplitfileCompatibilityMode

public void onSplitfileCompatibilityMode(InsertContext.CompatibilityMode min,
                                         InsertContext.CompatibilityMode max,
                                         byte[] customSplitfileKey,
                                         boolean dontCompress,
                                         boolean bottomLayer,
                                         boolean definitiveAnyway,
                                         ClientContext context)

Description copied from interface: GetCompletionCallback

Called when we know the settings for the splitfile.

Specified by:

onSplitfileCompatibilityMode in interface GetCompletionCallback

Parameters:

min - The lowest CompatibilityMode that appears to be valid based on what we've fetched so far.

max - The highest CompatibilityMode that appears to be valid based on what we've fetched so far.

customSplitfileKey - The fixed byte[] encryption key used on insert. On anything recent, we generate a single key, randomly for an SSK, or based on the content for a CHK, and use it for everything. This saves metadata space and improves security for SSKs.

dontCompress - Whether the content is compressed. If false, the dontCompress option was used.

bottomLayer - Whether this report originates at the bottom layer of the splitfile pyramid. I.e. the actual file, not the file containing the metadata to fetch the file (this can recurse for several levels!)

definitiveAnyway - Whether this report is definitive even though it's not from the bottom layer. This is true of recent splitfiles, where we store all the data in the top key.

context - Utility object containing helpers, mostly not persistent, such as the Ticker, temporary storage factories etc.

onHashes

public void onHashes(HashResult[] hashes,
                     ClientContext context)

Description copied from interface: GetCompletionCallback

Called when we know the HashResult of the final file. This will be checked when we actually fetch it, so is guaranteed to be correct. For recent metadata this is known at the top layer/block.

Specified by:

onHashes in interface GetCompletionCallback

Parameters:

hashes - A set of hashes for the final file content.

context - Utility object containing helpers, mostly not persistent, such as the Ticker, temporary storage factories etc.

enterCooldown

public void enterCooldown(ClientGetState state,
                          long wakeupTime,
                          ClientContext context)

Description copied from interface: WantsCooldownCallback

The request has gone into cooldown for some period.

Specified by:

enterCooldown in interface WantsCooldownCallback

clearCooldown

public void clearCooldown(ClientGetState state)

Description copied from interface: WantsCooldownCallback

The request has unexpectedly left cooldown.

Specified by:

clearCooldown in interface WantsCooldownCallback

getBlobBucket

public Bucket getBlobBucket()

getClientDetail

public byte[] getClientDetail(ChecksumChecker checker)
                       throws java.io.IOException

Overrides:

getClientDetail in class ClientRequester

Parameters:

checker - Used to checksum and isolate large components where we can recover if they fail.

Returns:

A byte[] representing the original client, to be written to the file storing a persistent download. E.g. for FCP, this will include the Identifier, whether it is on the global queue and the client name.

Throws:

java.io.IOException

innerOnResume

public void innerOnResume(ClientContext context)
                   throws ResumeFailedException

Called for a persistent request after startup.

Overrides:

innerOnResume in class ClientRequester

Throws:

ResumeFailedException

getCallback

protected ClientBaseCallback getCallback()

Specified by:

getCallback in class ClientRequester

writeTrivialProgress

public boolean writeTrivialProgress(java.io.DataOutputStream dos)
                             throws java.io.IOException

If the request is simple, e.g. a single, final splitfile fetch, then write enough information to continue the request. Otherwise write a marker indicating that this is not true, and return false. We don't need to write the expected MIME type, hashes etc, as the caller will write them.

Throws:

java.io.IOException

resumeFromTrivialProgress

public boolean resumeFromTrivialProgress(java.io.DataInputStream dis,
                                         ClientContext context)
                                  throws java.io.IOException

Throws:

java.io.IOException

resumedFetcher

public boolean resumedFetcher()

onShutdown

public void onShutdown(ClientContext context)

Description copied from class: ClientRequester

Called just before the final write when shutting down the node.

Overrides:

onShutdown in class ClientRequester

isCurrentState

public boolean isCurrentState(ClientGetState state)

Overrides:

isCurrentState in class ClientRequester

getCompletionFile

public java.io.File getCompletionFile()

Description copied from interface: FileGetCompletionCallback

Get the final location of the downloaded data. If this returns non-null, the caller may create a temporary file in the same directory and use it for e.g. storing the downloaded splitfile blocks. When complete, the caller should truncate the file to the correct length, and then call onSuccess().

Specified by:

getCompletionFile in interface FileGetCompletionCallback

Returns:

The final target File, or null if the download isn't to a (non-temporary) file, the target file can't be used for temporary storage etc. The returned file must be absolute.