freenet.client.async

Class ClientLayerPersister

java.lang.Object

freenet.client.async.PersistentJobRunnerImpl

freenet.client.async.ClientLayerPersister

All Implemented Interfaces:

PersistentJobRunner

public class ClientLayerPersister
extends PersistentJobRunnerImpl

Top level of persistence mechanism for ClientRequest's (persistent downloads and uploads). Note that we use three different persistence mechanisms here: 1) Splitfile persistence. The downloaded data and all the status for a splitfile is kept in a single random access file (technically a LockableRandomAccessBuffer). 2) Java persistence. The overall list of ClientRequest's is stored to client.dat using serialization, by this class. 3) A simple binary fallback. For complicated requests this will just record enough information to restart the request, but for simple splitfile downloads, we can resume from (1). The reason for this seemingly unnecessary complexity is: 1) Robustness, even against (reasonable) data corruption (e.g. on writing frequently written parts of files), and against problems in serialization. 2) Minimising disk I/O (particularly disk seeks), especially in splitfile fetches. Decoding a segment takes effectively a single read and a couple of writes, for example. 3) Allowing us to store all the important information about a download, including the downloaded data and the status, in a temporary file close to where the final file will be saved. Then we can (if there is no compression or filtering) simply truncate the file to complete. Also, most of the important global structures are kept in RAM and recreated after downloads are read in. Notably ClientRequestScheduler/Selector, which keep a tree of requests to choose from, and a set of Bloom filters to identify which blocks belong to which request (we won't always get a block as the direct result of doing our own requests). Please don't shove it all into a database without serious consideration of the performance and reliability implications! One query per block is not feasible on typical end user hardware, and disks aren't reliable. Losing all downloads when there is a one byte data corruption is unacceptable. And blocks should be kept close to where they are supposed to end up... Note that inserts may be somewhat less robust than requests. This is intentional as inserts should be relatively short-lived or they won't be much use to anyone as the data will have fallen out. SCHEMA MIGRATION: Note that changing classes that are Serializable can result in restarting downloads or losing uploads.

Nested Class Summary

Nested classes/interfaces inherited from interface freenet.client.async.PersistentJobRunner

PersistentJobRunner.CheckpointLock

Field Summary

Fields inherited from class freenet.client.async.PersistentJobRunnerImpl

serializeCheckpoints

Constructor Summary

Constructors

Constructor and Description
ClientLayerPersister(Executor executor, Ticker ticker, Node node, NodeClientCore core, PersistentTempBucketFactory persistentTempFactory, TempBucketFactory tempBucketFactory, PersistentStatsPutter stats) Load everything.

Method Summary

Modifier and Type	Method and Description
void	deleteAllFiles()
void	disableWrite()
java.io.File	getWriteFilename()
protected void	innerCheckpoint(boolean shutdown)
boolean	newSalt() For persistent requests, return true if the bloom filter salt has changed when loading the requests.
void	panic()
protected void	save(boolean shutdown)
void	setFilesAndLoad(java.io.File dir, java.lang.String baseName, boolean writeEncrypted, boolean noWrite, DatabaseKey encryptionKey, ClientContext context, RequestStarterGroup requestStarters, java.util.Random random) Set the files to write to and set up encryption

Methods inherited from class freenet.client.async.PersistentJobRunnerImpl

checkpointOffThread, delayedCheckpoint, getClientContext, handleCompletion, hasLoaded, isKilledOrNotLoaded, killAndWaitForNotRunning, killAndWaitForNotWriting, lock, onLoading, onStarted, queue, queueInternal, queueInternal, queueNormalOrDrop, setCheckpointASAP, shutdown, shuttingDown, start, updateLastCheckpointed, waitAndCheckpoint, waitForIdleAndCheckpoint, waitForNotWriting

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClientLayerPersister

public ClientLayerPersister(Executor executor,
                            Ticker ticker,
                            Node node,
                            NodeClientCore core,
                            PersistentTempBucketFactory persistentTempFactory,
                            TempBucketFactory tempBucketFactory,
                            PersistentStatsPutter stats)

Load everything.

Parameters:

persistentTempFactory - Only passed in so that we can call its pre- and post- commit hooks. We don't explicitly save it; it must be populated lazily in onResume() like everything else.

Method Detail

setFilesAndLoad

public void setFilesAndLoad(java.io.File dir,
                            java.lang.String baseName,
                            boolean writeEncrypted,
                            boolean noWrite,
                            DatabaseKey encryptionKey,
                            ClientContext context,
                            RequestStarterGroup requestStarters,
                            java.util.Random random)
                     throws MasterKeysWrongPasswordException

Set the files to write to and set up encryption

Parameters:

noWrite - If true, don't write the data to disk at all, and delete existing client.dat*.

Throws:

MasterKeysWrongPasswordException - If we need the encryption key but it has not been supplied.

innerCheckpoint

protected void innerCheckpoint(boolean shutdown)

Specified by:

innerCheckpoint in class PersistentJobRunnerImpl

save

protected void save(boolean shutdown)

newSalt

public boolean newSalt()

Description copied from interface: PersistentJobRunner

For persistent requests, return true if the bloom filter salt has changed when loading the requests. In which case all the Bloom filters will be invalid and will need to be recomputed.

getWriteFilename

public java.io.File getWriteFilename()

panic

public void panic()

deleteAllFiles

public void deleteAllFiles()

disableWrite

public void disableWrite()

Overrides:

disableWrite in class PersistentJobRunnerImpl