net.pakl.rl
Class ValueFunctionHashMap

java.lang.Object
  net.pakl.rl.ValueFunctionHashMap

All Implemented Interfaces:

java.io.Serializable, ValueFunction

Direct Known Subclasses:

ValueFunctionHashMapZipped

public class ValueFunctionHashMap
extends java.lang.Object
implements ValueFunction, java.io.Serializable

A ValueFunction maps states, which are positions in a World, to values, and may be replaced with a neural network. When the value function is optimal then the optimal policy can be extracted from it.

See Also:

Serialized Form

Constructor Summary

ValueFunctionHashMap()

ValueFunctionHashMap(World sourceWorld)
While the initial ValueFunction is arbitrary, it must be based on a World's states.

ValueFunctionHashMap(World sourceWorld, boolean SAFE_MODE)

Method Summary

void	addTimer(double time)
void	clear()
java.util.Set<State>	getKeySet()
java.util.Iterator	getKeySetIterator()
java.lang.String	getName() It is important for value functions to have names for the patchwork approach, so that states can identify to which value function patch they belong by the PolicyExtractor.
int	getSize()
double	getTimer()
double	getValue(State state) The value of a state is defined as the sum of the terinforcements received when starting in that state and following some fixed policy to a terminal state; the optimal policy would map states to actions that maximizes the sum of reinforcements received when starting in an arbitrary state and performing actions until the terminal state is reached
void	init(World sourceWorld)
void	loadFrom(java.lang.String filename)
void	resetTimer()
void	saveTo(java.lang.String filename)
void	setAllowExpansionOfStateBounds(boolean trueOrFalse)
void	setName(java.lang.String name)
void	setSafeMode(boolean b)
void	setValue(State state, double newValue)
void	setValueOfNonStoredStates(double x)
void	setValueOfOutOfBoundsStates(double newValueOfOutOfBoundsStates)
void	setValueOfTerminalStates(double newValueOfTerminalStates) Many problems define a value of 0 for their terminal states, where the agent can remain.
void	setWorld(World w)
int	size()
java.lang.String	toText()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ValueFunctionHashMap

public ValueFunctionHashMap(World sourceWorld)

While the initial ValueFunction is arbitrary, it must be based on a World's states. The initial random values should probably be a function of the number of states in the world, but I don't yet know what that function should be.

ValueFunctionHashMap

public ValueFunctionHashMap(World sourceWorld,
                            boolean SAFE_MODE)

ValueFunctionHashMap

public ValueFunctionHashMap()

Method Detail

setValueOfNonStoredStates

public void setValueOfNonStoredStates(double x)

setName

public void setName(java.lang.String name)

Specified by:

setName in interface ValueFunction

getName

public java.lang.String getName()

Description copied from interface: ValueFunction

It is important for value functions to have names for the patchwork approach, so that states can identify to which value function patch they belong by the PolicyExtractor.

Specified by:

getName in interface ValueFunction

setSafeMode

public void setSafeMode(boolean b)

getSize

public int getSize()

setWorld

public void setWorld(World w)

Specified by:

setWorld in interface ValueFunction

init

public void init(World sourceWorld)

setValueOfOutOfBoundsStates

public void setValueOfOutOfBoundsStates(double newValueOfOutOfBoundsStates)

setAllowExpansionOfStateBounds

public void setAllowExpansionOfStateBounds(boolean trueOrFalse)

getKeySetIterator

public java.util.Iterator getKeySetIterator()

getKeySet

public java.util.Set<State> getKeySet()

clear

public void clear()

size

public int size()

getValue

public double getValue(State state)

The value of a state is defined as the sum of the terinforcements received when starting in that state and following some fixed policy to a terminal state; the optimal policy would map states to actions that maximizes the sum of reinforcements received when starting in an arbitrary state and performing actions until the terminal state is reached

Specified by:

getValue in interface ValueFunction

getTimer

public double getTimer()

resetTimer

public void resetTimer()

addTimer

public void addTimer(double time)

setValue

public void setValue(State state,
                     double newValue)

Specified by:

setValue in interface ValueFunction

saveTo

public void saveTo(java.lang.String filename)
            throws java.lang.Exception

Throws:

java.lang.Exception

loadFrom

public void loadFrom(java.lang.String filename)
              throws java.lang.Exception

Throws:

java.lang.Exception

toText

public java.lang.String toText()

setValueOfTerminalStates

public void setValueOfTerminalStates(double newValueOfTerminalStates)

Description copied from interface: ValueFunction

Many problems define a value of 0 for their terminal states, where the agent can remain.

Specified by:

setValueOfTerminalStates in interface ValueFunction