org.eyelanguage.rl.reading
Class ReadingMain

java.lang.Object
  org.eyelanguage.rl.reading.ReadingMain

Direct Known Subclasses:

OptimalActionsLister

public class ReadingMain
extends java.lang.Object

A class configured by a textfile called "reading.prop" that can be run from the command line to initiate a reinforcement learning simulation on a reading problem; all variables below can be specified in the reading.prop file, e.g. saccadicError = true

Field Summary

protected int	acuityLimit The maximum allowed distance in the state space between fovea and center of attention.
protected boolean	acuityLinearPenalize reading.prop parameter: If true, one extra timestemp of processing is required for each character space from the center of the word being processed for word identification.
protected int	acuityLinearPenaltySlope Amount of cost added per character eccentricity for linear acuity penalty.
protected int	agentParallelNumThreads reading.prop parameter
protected java.lang.String	agentType reading.prop parameter
protected java.lang.String	aidTimes Specified the amount of time it will take the agent to recognize each word, if that word if foveated in its center.
protected boolean	ALLOW_SIMULTANEOUS_ATTNEXT_AND_IDENT Specifies in generic policy if attention shifts and word identification can occur at the same time.
protected boolean	ALLOW_SIMULTANEOUS_SACREQ_AND_ATTNEXT Specifies in generic policy if saccade request actions and attention shift actions can occur at the same time.
protected java.lang.String	attention reading.prop parameter: Serial attention means attend one word at a time, semiparallel means multiple but with a serial sliding window, and parallel means fully parallel processing of words in any order.
protected boolean	bitPacker reading.prop parameter
protected int	boundedBufferSize reading.prop parameter
protected int	centerOfAttention If attention is not serial, The number of words the agent can attend (process lexically) in parallel to the left of attention focus.
protected double	defaultReinforcement reading.prop parameter
protected double	discountFactor Represents how much future reward is weighted in comparison to the immediate reward.
protected boolean	distributedJavaParty reading.prop parameter
protected boolean	DYNAMICALLY_LOAD_VALUE_FUNCTION_PATCHES Affect PolicyExtractor to load value functions if the state does not reside in the current value function; if the state does not support SubdivisionIdentification the PolicyExtractor will communicate this.
protected double	epsilon Amount by which to incrementally update current predictions, useful in the face of stochasticity.
protected double	greed This number is used in the TrajectorySampler mode, and specified how likely the agent is to take the best known action.
protected boolean	handCraftedVectors (Experimenal) Engages special handcrafted code for function approximator state vector representation.
protected int	hiddenLayerSize reading.prop parameter
protected double	identifyReinforcement The reward received by the agent when entering a state where the current word is newly identified.
protected boolean	incrementalUpdate Function Approximators: update incrementally (on each input-output mapping) if true, else epoch-wise.
protected double	learningRate Determines the learning rate of the neural network or function approximator that is being used.
protected java.lang.String	loadWorldCache reading.prop parameter
protected double	momentum Determines how recent changes in the neural network or function approximator continue to affect the network.
protected double	netOutputRange For a neural network or function approximator, specifies the meaning of the maximal output and minimal output values.
protected boolean	PARALLEL_DIVIDED_ATTENTION_GRADIENT
protected boolean	PARALLEL_LINEAR_ATTENTION_GRADIENT
protected boolean	PARALLEL_UNIFORM_ATTENTION
protected int	parallelAttendableWords If attention is not serial, The number of words the agent can attend (process lexically) in parallel to the right of attention focus.
protected java.lang.String	predictabilities reading.prop parameter: Space-separated real numbers specifying the predictability of each word, starting with the second word in the sentence?
protected long	randomSeed reading.prop parameter
protected double	reachTerminalReinforcement The reward received by the agent when entering a terminal state.
protected java.lang.String	readingStateType reading.prop parameter
protected boolean	residualAlgorithm reading.prop parameter
protected double	residualWeighting reading.prop parameter
protected java.lang.String	rlAlgorithm reading.prop parameter
protected int	saccadeTimePenalty reading.prop parameter: How much time (negative) should be removed during a saccadic eye movement?
protected boolean	saccadicError reading.prop parameter: True if saccadic motor error should be enabled, false otherwise.
protected boolean	saccadicErrorGaussian reading.prop parameter: Should the Gaussian saccadic error be used -- otherwise, a uniform saccadic error will be used.
protected double	saccadicErrorSpread reading.prop parameter: The amount of "spread" of this particular kind of saccadic error (e.g.
protected int	saccPrgTime Amount of time it takes the agent to program a saccade.
protected int	saveNetworkEvery reading.prop parameter
protected java.lang.String	sentence
protected java.lang.String	startState
protected boolean	stateActionCache reading.prop parameter
protected boolean	stateVectorsComplement reading.prop parameter
protected boolean	stateVectorsCrossProduct reading.prop parameter
protected int	testAgentEvery
protected java.lang.String	testAidTimes reading.prop parameter
protected Corpus	testCorpus
protected int	testIterations reading.prop parameter
protected java.lang.String	testPredictabilities reading.prop parameter
protected java.lang.String	testSentence reading.prop parameter
protected boolean	testTrainSameVf reading.prop parameter
protected java.lang.String	testWordLengths
protected boolean	testWordLengthsOverridden
protected int	totalIterations reading.prop parameter: How many interations to train before stopping and testing.
protected Corpus	trainCorpus
protected boolean	updateOnlyWhenGreedy
protected java.lang.String	valueFunctionName Stores a name in the value function that should correspond to the patch name identifying the patch name; perhaps in the future value function patches should have more than one name.
protected java.lang.String	valueFunctionToLoad reading.prop parameter
protected java.lang.String	valueFunctionToSave reading.prop parameter
protected double	valueOfNonStoredStates
protected double	valueOfTerminalStates reading.prop parameter
protected java.lang.String	wordLengths
protected boolean	wordLengthsOverridden
protected boolean	wrapSerialValueFunctionWithParallelAdapter reading.prop parameter

Constructor Summary

ReadingMain()

Method Summary

protected void	loadProperties()
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

totalIterations

protected int totalIterations

reading.prop parameter: How many interations to train before stopping and testing.

testAgentEvery

protected int testAgentEvery

incrementalUpdate

protected boolean incrementalUpdate

Function Approximators: update incrementally (on each input-output mapping) if true, else epoch-wise.

attention

protected java.lang.String attention

reading.prop parameter: Serial attention means attend one word at a time, semiparallel means multiple but with a serial sliding window, and parallel means fully parallel processing of words in any order.

PARALLEL_UNIFORM_ATTENTION

protected boolean PARALLEL_UNIFORM_ATTENTION

PARALLEL_LINEAR_ATTENTION_GRADIENT

protected boolean PARALLEL_LINEAR_ATTENTION_GRADIENT

PARALLEL_DIVIDED_ATTENTION_GRADIENT

protected boolean PARALLEL_DIVIDED_ATTENTION_GRADIENT

saccadicError

protected boolean saccadicError

reading.prop parameter: True if saccadic motor error should be enabled, false otherwise.

saccadicErrorSpread

protected double saccadicErrorSpread

reading.prop parameter: The amount of "spread" of this particular kind of saccadic error (e.g. could be the standard deviation of a normal distribution, or the width of a uniform distribution.)

acuityLinearPenalize

protected boolean acuityLinearPenalize

reading.prop parameter: If true, one extra timestemp of processing is required for each character space from the center of the word being processed for word identification. If false, a constant penalty (default 1) is added no matter how eccentric the eyes are.

acuityLinearPenaltySlope

protected int acuityLinearPenaltySlope

Amount of cost added per character eccentricity for linear acuity penalty.

saccadeTimePenalty

protected int saccadeTimePenalty

reading.prop parameter: How much time (negative) should be removed during a saccadic eye movement?

saccadicErrorGaussian

protected boolean saccadicErrorGaussian

reading.prop parameter: Should the Gaussian saccadic error be used -- otherwise, a uniform saccadic error will be used.

identifyReinforcement

protected double identifyReinforcement

The reward received by the agent when entering a state where the current word is newly identified.

reachTerminalReinforcement

protected double reachTerminalReinforcement

The reward received by the agent when entering a terminal state.

defaultReinforcement

protected double defaultReinforcement

reading.prop parameter

saccPrgTime

protected int saccPrgTime

Amount of time it takes the agent to program a saccade.

parallelAttendableWords

protected int parallelAttendableWords

If attention is not serial, The number of words the agent can attend (process lexically) in parallel to the right of attention focus.

centerOfAttention

protected int centerOfAttention

If attention is not serial, The number of words the agent can attend (process lexically) in parallel to the left of attention focus.

acuityLimit

protected int acuityLimit

The maximum allowed distance in the state space between fovea and center of attention.

ALLOW_SIMULTANEOUS_SACREQ_AND_ATTNEXT

protected boolean ALLOW_SIMULTANEOUS_SACREQ_AND_ATTNEXT

Specifies in generic policy if saccade request actions and attention shift actions can occur at the same time.

ALLOW_SIMULTANEOUS_ATTNEXT_AND_IDENT

protected boolean ALLOW_SIMULTANEOUS_ATTNEXT_AND_IDENT

Specifies in generic policy if attention shifts and word identification can occur at the same time.

DYNAMICALLY_LOAD_VALUE_FUNCTION_PATCHES

protected boolean DYNAMICALLY_LOAD_VALUE_FUNCTION_PATCHES

Affect PolicyExtractor to load value functions if the state does not reside in the current value function; if the state does not support SubdivisionIdentification the PolicyExtractor will communicate this.

valueFunctionName

protected java.lang.String valueFunctionName

Stores a name in the value function that should correspond to the patch name identifying the patch name; perhaps in the future value function patches should have more than one name.

sentence

protected java.lang.String sentence

predictabilities

protected java.lang.String predictabilities

reading.prop parameter: Space-separated real numbers specifying the predictability of each word, starting with the second word in the sentence?

aidTimes

protected java.lang.String aidTimes

Specified the amount of time it will take the agent to recognize each word, if that word if foveated in its center.

wordLengths

protected java.lang.String wordLengths

wordLengthsOverridden

protected boolean wordLengthsOverridden

testSentence

protected java.lang.String testSentence

reading.prop parameter

testIterations

protected int testIterations

reading.prop parameter

testPredictabilities

protected java.lang.String testPredictabilities

reading.prop parameter

testAidTimes

protected java.lang.String testAidTimes

reading.prop parameter

hiddenLayerSize

protected int hiddenLayerSize

reading.prop parameter

testWordLengths

protected java.lang.String testWordLengths

testWordLengthsOverridden

protected boolean testWordLengthsOverridden

agentType

protected java.lang.String agentType

reading.prop parameter

agentParallelNumThreads

protected int agentParallelNumThreads

reading.prop parameter

boundedBufferSize

protected int boundedBufferSize

reading.prop parameter

valueOfTerminalStates

protected double valueOfTerminalStates

reading.prop parameter

testTrainSameVf

protected boolean testTrainSameVf

reading.prop parameter

stateActionCache

protected boolean stateActionCache

reading.prop parameter

bitPacker

protected boolean bitPacker

reading.prop parameter

valueFunctionToLoad

protected java.lang.String valueFunctionToLoad

reading.prop parameter

wrapSerialValueFunctionWithParallelAdapter

protected boolean wrapSerialValueFunctionWithParallelAdapter

reading.prop parameter

valueFunctionToSave

protected java.lang.String valueFunctionToSave

reading.prop parameter

loadWorldCache

protected java.lang.String loadWorldCache

reading.prop parameter

distributedJavaParty

protected boolean distributedJavaParty

reading.prop parameter

handCraftedVectors

protected boolean handCraftedVectors

(Experimenal) Engages special handcrafted code for function approximator state vector representation.

rlAlgorithm

protected java.lang.String rlAlgorithm

reading.prop parameter

greed

protected double greed

This number is used in the TrajectorySampler mode, and specified how likely the agent is to take the best known action. Lower values encourage more of the random exploration behavior.

discountFactor

protected double discountFactor

Represents how much future reward is weighted in comparison to the immediate reward. If your environment has loops (and the agent can return to the same state it was before) you may wish to set this to a value less than 1 to avoid exploding value functions (i.e. infinite predicted reward).

epsilon

protected double epsilon

Amount by which to incrementally update current predictions, useful in the face of stochasticity.

learningRate

protected double learningRate

Determines the learning rate of the neural network or function approximator that is being used.

momentum

protected double momentum

Determines how recent changes in the neural network or function approximator continue to affect the network.

netOutputRange

protected double netOutputRange

For a neural network or function approximator, specifies the meaning of the maximal output and minimal output values.

saveNetworkEvery

protected int saveNetworkEvery

reading.prop parameter

readingStateType

protected java.lang.String readingStateType

reading.prop parameter

randomSeed

protected long randomSeed

reading.prop parameter

valueOfNonStoredStates

protected double valueOfNonStoredStates

startState

protected java.lang.String startState

updateOnlyWhenGreedy

protected boolean updateOnlyWhenGreedy

residualAlgorithm

protected boolean residualAlgorithm

reading.prop parameter

residualWeighting

protected double residualWeighting

reading.prop parameter

stateVectorsCrossProduct

protected boolean stateVectorsCrossProduct

reading.prop parameter

stateVectorsComplement

protected boolean stateVectorsComplement

reading.prop parameter

testCorpus

protected Corpus testCorpus

trainCorpus

protected Corpus trainCorpus

Constructor Detail

ReadingMain

public ReadingMain()

Method Detail

main

public static void main(java.lang.String[] args)

loadProperties

protected void loadProperties()