net.pakl.rl
Class ValueFunctionResidualAlgorithmPerceptron

java.lang.Object
  net.pakl.rl.ValueFunctionPerceptron
      net.pakl.rl.ValueFunctionResidualAlgorithmPerceptron

All Implemented Interfaces:

java.io.Serializable, ValueFunction

Direct Known Subclasses:

ValueFunctionResidualAlgorithmBairdPerceptron, ValueFunctionResidualAlgorithmPatrykNetwork

public class ValueFunctionResidualAlgorithmPerceptron
extends ValueFunctionPerceptron
implements ValueFunction

See Also:

Serialized Form

Field Summary

boolean	CROSS_PRODUCT
int	hiddenUnits
boolean	INCREMENTAL_UPDATE
double	learningRate
double	multiplyAndAdd
protected java.lang.String	name
protected int	numPresynaptics
boolean	USE_REAL_VECTORS

Fields inherited from class net.pakl.rl.ValueFunctionPerceptron

momentum

Constructor Summary

ValueFunctionResidualAlgorithmPerceptron()

ValueFunctionResidualAlgorithmPerceptron(World w)

Method Summary

protected double	dotProduct(double[] u, double[] v)
double	forNetwork(double value)
double	fromNetwork(double value)
java.lang.String	getName() It is important for value functions to have names for the patchwork approach, so that states can identify to which value function patch they belong by the PolicyExtractor.
net.pakl.neuralnet.Perceptron	getNetwork()
protected double	getValue(HasVectorRepresentation state)
double	getValue(State state) Retrieve the value associated with a state (which may be different for non-stored states depending on the actual class implementing this value function).
protected void	initializeNetworkForStatesOfType(HasVectorRepresentation thisStateObject)
protected double[]	intsToDoubles(int[] x)
void	setAllowExpansionOfStateBounds(boolean trueOrFalse)
void	setHiddenUnits(int newValue)
void	setLearningRate(double newValue)
void	setMaxMinValue(double newValue)
void	setName(java.lang.String name)
void	setRandomSeed(long x)
void	setResidualWeighting(double newValue)
protected void	setValue(HasVectorRepresentation thisStateObject, HasVectorRepresentation nextStateObject, double newValue, double discountFactor)
void	setValue(State state, double newValue)
void	setValue(State thisState, State nextState, double newValue, double discountFactor)
void	setValueOfOutOfBoundsStates(double newValueOfOutOfBoundsStates)
void	setValueOfTerminalStates(double newValue) Many problems define a value of 0 for their terminal states, where the agent can remain.
void	storeWeightChangesIfNonIncremental()
protected double	sum(double[] v)
protected double	sum(double[][] v)
java.lang.String	toText()
protected double[][]	vadd(double[][] v1, double[][] v2)
protected double[]	vadd(double[] v1, double[] v2)
protected double[]	vmul(double scalar, double[] vector)
protected double[][]	vmul(double scalar, double[][] vector)
protected double[][]	vsub(double[][] v1, double[][] v2)
protected double[]	vsub(double[] v1, double[] v2)
protected void	zeroOutTotalWeightChangeForEpoch()

Methods inherited from class net.pakl.rl.ValueFunctionPerceptron

setMomentum, setWorld

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface net.pakl.rl.ValueFunction

setWorld

Field Detail

hiddenUnits

public int hiddenUnits

learningRate

public double learningRate

multiplyAndAdd

public double multiplyAndAdd

numPresynaptics

protected int numPresynaptics

INCREMENTAL_UPDATE

public boolean INCREMENTAL_UPDATE

CROSS_PRODUCT

public boolean CROSS_PRODUCT

USE_REAL_VECTORS

public boolean USE_REAL_VECTORS

name

protected java.lang.String name

Constructor Detail

ValueFunctionResidualAlgorithmPerceptron

public ValueFunctionResidualAlgorithmPerceptron(World w)

ValueFunctionResidualAlgorithmPerceptron

public ValueFunctionResidualAlgorithmPerceptron()

Method Detail

setName

public void setName(java.lang.String name)

Specified by:

setName in interface ValueFunction

Overrides:

setName in class ValueFunctionPerceptron

getName

public java.lang.String getName()

Description copied from interface: ValueFunction

It is important for value functions to have names for the patchwork approach, so that states can identify to which value function patch they belong by the PolicyExtractor.

Specified by:

getName in interface ValueFunction

Overrides:

getName in class ValueFunctionPerceptron

setValueOfOutOfBoundsStates

public void setValueOfOutOfBoundsStates(double newValueOfOutOfBoundsStates)

Overrides:

setValueOfOutOfBoundsStates in class ValueFunctionPerceptron

setAllowExpansionOfStateBounds

public void setAllowExpansionOfStateBounds(boolean trueOrFalse)

Overrides:

setAllowExpansionOfStateBounds in class ValueFunctionPerceptron

storeWeightChangesIfNonIncremental

public void storeWeightChangesIfNonIncremental()

Overrides:

storeWeightChangesIfNonIncremental in class ValueFunctionPerceptron

zeroOutTotalWeightChangeForEpoch

protected void zeroOutTotalWeightChangeForEpoch()

getNetwork

public net.pakl.neuralnet.Perceptron getNetwork()

Overrides:

getNetwork in class ValueFunctionPerceptron

initializeNetworkForStatesOfType

protected void initializeNetworkForStatesOfType(HasVectorRepresentation thisStateObject)

setValue

protected void setValue(HasVectorRepresentation thisStateObject,
                        HasVectorRepresentation nextStateObject,
                        double newValue,
                        double discountFactor)

getValue

public double getValue(State state)

Description copied from interface: ValueFunction

Retrieve the value associated with a state (which may be different for non-stored states depending on the actual class implementing this value function).

Specified by:

getValue in interface ValueFunction

Overrides:

getValue in class ValueFunctionPerceptron

getValue

protected double getValue(HasVectorRepresentation state)

sum

protected double sum(double[] v)

Overrides:

sum in class ValueFunctionPerceptron

sum

protected double sum(double[][] v)

Overrides:

sum in class ValueFunctionPerceptron

vsub

protected double[] vsub(double[] v1,
                        double[] v2)

Overrides:

vsub in class ValueFunctionPerceptron

vsub

protected double[][] vsub(double[][] v1,
                          double[][] v2)

Overrides:

vsub in class ValueFunctionPerceptron

vadd

protected double[] vadd(double[] v1,
                        double[] v2)

Overrides:

vadd in class ValueFunctionPerceptron

intsToDoubles

protected double[] intsToDoubles(int[] x)

Overrides:

intsToDoubles in class ValueFunctionPerceptron

vadd

protected double[][] vadd(double[][] v1,
                          double[][] v2)

Overrides:

vadd in class ValueFunctionPerceptron

vmul

protected double[] vmul(double scalar,
                        double[] vector)

Overrides:

vmul in class ValueFunctionPerceptron

vmul

protected double[][] vmul(double scalar,
                          double[][] vector)

Overrides:

vmul in class ValueFunctionPerceptron

dotProduct

protected double dotProduct(double[] u,
                            double[] v)

Overrides:

dotProduct in class ValueFunctionPerceptron

setValue

public void setValue(State state,
                     double newValue)

Specified by:

setValue in interface ValueFunction

Overrides:

setValue in class ValueFunctionPerceptron

setResidualWeighting

public void setResidualWeighting(double newValue)

Overrides:

setResidualWeighting in class ValueFunctionPerceptron

setHiddenUnits

public void setHiddenUnits(int newValue)

Overrides:

setHiddenUnits in class ValueFunctionPerceptron

setLearningRate

public void setLearningRate(double newValue)

Overrides:

setLearningRate in class ValueFunctionPerceptron

setMaxMinValue

public void setMaxMinValue(double newValue)

Overrides:

setMaxMinValue in class ValueFunctionPerceptron

toText

public java.lang.String toText()

Overrides:

toText in class ValueFunctionPerceptron

setValueOfTerminalStates

public void setValueOfTerminalStates(double newValue)

Description copied from interface: ValueFunction

Many problems define a value of 0 for their terminal states, where the agent can remain.

Specified by:

setValueOfTerminalStates in interface ValueFunction

Overrides:

setValueOfTerminalStates in class ValueFunctionPerceptron

setValue

public void setValue(State thisState,
                     State nextState,
                     double newValue,
                     double discountFactor)

forNetwork

public double forNetwork(double value)

Overrides:

forNetwork in class ValueFunctionPerceptron

fromNetwork

public double fromNetwork(double value)

Overrides:

fromNetwork in class ValueFunctionPerceptron

setRandomSeed

public void setRandomSeed(long x)

Overrides:

setRandomSeed in class ValueFunctionPerceptron