Alright, here is the first in a series of detailed explanations of ACT-R/S from a modeler’s perspective. This will cover the current state of the implementation.
As you may be aware ACT-R/S is a spatial extension to ACT-R, specifically jACT-R. Why is it only available for jACT-R? Two reasons: 1) before 6.0, the addition of modules was a bit of a challenge in ACT-R. 2) Java contains a much wider variety of libraries that make interfacing 3D simulations/applications much easier (although, none are currently implemented).
The current implement only deals with the configural system which is used to support navigation and localization. Unfortunately, the current implementation only supports XML based static environments (through CommonReality, the embodiment layer for jACT-R). More dynamic sensors are slated for the future, but I have only so much time.
What does the configural system currently support?
- attending to configural reps (angular vectors to bounding edges of an object) in space through the visual-location system
- automatic linking of token information from visual-objects (i.e. that’s a configural rep of BOB, not the configural rep of the specific visual percept of BOB)
- semantic comparisons of configural reps (which is above,below,left,right, etc)
- the automatic or intentional encoding of configural-locations (i.e. linking of two configural reps to establish a unique location in space relative to the two)
- configural buffer with multiple chunk capacity (configurable, but a hard, # of chunks, limit)
- intentional mental transformations of representations (i.e. simulated walking and turning)
What is still pending?
- nique configural-location identifiers (i.e. “am i in the same spot that I was previously?”)
- navigation queries (i.e. I’m at configural-location-A, I want to get to configural-location-B, where do I turn to face and how far do I move)
- encoding of configural reps from auditory information (auditory-location)
- automatic path-integration updating of representations (I still need to build a hack ambulatory system)
- more useful CommonReality sensors other than the static xml one
Now that those caveats are out of the way, let’s take a peak at how things work.
Configural Representations
At the heart of the configural system is the configural rep. This is a symbolic representation of the spatial information of an object in space. It contains just enough information to navigate around and to identify. Basically, it’s a bunch of egocentric angular vectors to the bounds of an object, plus some extra tidbits.
The extra tidbits are pulled from the visual system (in the future it will support information from the auditory module). The configural system does not do any object identification - rather when a new configural rep is encoded, it listens to the visual buffer. If a visual-object is (or was) encoded that refers to the same object, it snags the token information from the visual-object.
The slots of the configural rep are left-bearing, left-distance, right-bearing, right-distance, top-pitch, top-distance, bottom-pitch, bottom-distance, center-bearing, center-pitch, center-distance, screen-pos (bad name, but used for consistency with the visual-objects), identifier.
The coordinate system used is polar, but slightly tweaked so that it makes more sense for modelers (but less so for 3D graphics people). Bearings are measured in degrees, -180 - 180, left to right, with 0 being straight ahead (since there is no head in ACT-R yet, retinal coordinates are the same as configural angles.. but technically configural is reckoned from the body, not the head or eye). Pitch is -90 - 90, bottom to top. Distances are currently in meters.
Attending
Much like the visual system, configural reps are attended by the configural buffer which needs to know the location of the object to attend to. Currently, it only supports location information from the visual-location buffer, but auditory is planned. Take a peak at this production (yes, I know it’s in the jACT-R format and not the canonical Lisp format, but you can figure it out quite easily) :
<production name="search-for-configural">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="starting"/>
</match>
<query buffer="visual">
<slot name="state" equals="free"/>
</query>
<query buffer="configural">
<slot name="state" equals="free"/>
<slot name="buffer" not="full"/>
</query>
</conditions>
<actions>
<add buffer="visual-location" type="visual-location">
<slot name=":attended" equals="new"/>
<slot name="nearest" equals="current"/>
<slot name="kind" equals="configural"/>
</add>
<modify buffer="goal">
<slot name="stage" equals="searching"/>
</modify>
</actions>
</production>
This asks the visual-location buffer for any unattended object that matches the properties. You will notice the use of the kind slot to specify that we are only interested in objects that could be encoded in the configural buffer. If you don’t do this, you’ll get any visual object. When you try to encode it as a configural rep, the configural system will do its best to accommodate you, but you might get an error. The next production requests that the configural system encode a configural at the location.
<!-- found a visual location, let's try to encode -->
<production name="search-succeeded">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="searching"/>
</match>
<match buffer="visual-location" type="visual-location">
<slot name="kind" equals="configural"/>
</match>
</conditions>
<actions>
<output>"Found a configural at =visual-location attempting to encode"
</output>
<add buffer="visual" type="move-attention">
<slot name="screen-pos" equals="=visual-location"/>
</add>
<add buffer="configural" type="move-attention">
<slot name="screen-pos" equals="=visual-location"/>
</add>
<modify buffer="goal">
<slot name="stage" equals="encoding"/>
</modify>
</actions>
</production>
Notice that this is making two requests, one from the visual system to attend to the visual-object the other from the configural system. I routinely do this so that the configural rep will have a valid identifier. The two need not complete at the same time. So long as a newly created configural rep is in the buffer, if it’s corresponding visual-object is encoded, the identifier information will be extracted and assigned to the configural rep. However, this only works with newly created configural reps, copies of previously encoded reps might not work.
Encoding
Like the visual-location buffer, the encoding of the contents of the configural buffer is not handled by the core system. This was done because of the high likelihood of many, many, many configural reps being created over the course of the simulation, most of which will never be used (e.g. intermediate transforms). As a result, chunks in the configural system are not always encoded on their removal from the configural buffer. That is the event that will trigger the encoding, but they will only be encoded if they were previously matched by a fired production.
Basically, it’s a use-it-or-loose-it encoding policy. I will be parameterizing this in the future.
:buffer Status
If you were to check the :buffer state after this operation, unlike most other ACT-R buffers, it won’t say full (at least not by default). Because the configural buffer can contain more than one chunk, :buffer will actually return the number of chunks in the buffer until the limit is reached, at which time it will be full.
<!-- we were able to encode both the visual object and the configural-rep
note that configural :buffer it might be 1..full
-->
<production name="encoding-succeeded">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="encoding"/>
</match>
<match buffer="visual" type="visual-object">
<slot name="screen-pos" equals="=visLoc"/>
<slot name="token" equals="=identifier"/>
</match>
<match buffer="configural" type="configural">
<slot name="screen-pos" equals="=visLoc"/>
<slot name="identifier" equals="=identifier"/>
<slot name=":buffer" equals="=reps"/>
</match>
</conditions>
<actions>
<output>"Sweet, found both =visual and =configural refering to =identifier at =visual-location =reps total"</output>
<output>"Lets keep searching until Ive attended to them all"</output>
<modify buffer="goal">
<slot name="stage" equals="starting"/>
</modify>
</actions>
</production>
During the encoding of configural reps, the buffer’s :state will be busy. If an error occurs, it will be error, however, that does affect the current contents in anyway.
Semantic Comparisons
We could iterate over the previous productions a few times until we fill up the buffer. Now it’s time to look at what kind of comparisons we can make. Basically speaking, the contents of the configural reps aren’t really for normal consumption. Most of the time you will just use configural-query. This chunk-type allows you to compare two configural reps (query and reference). Take a look at this production and its harvester:
<!-- no location was found, but there is something in the configural buffer,
which is full. Let's try a comparison -->
<production name="configural-full">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="starting"/>
</match>
<query buffer="configural">
<slot name="buffer" equals="full"/>
</query>
</conditions>
<actions>
<output>"We have a full configural buffer, lets try a comparison"</output>
<add buffer="configural" type="configural-query"/>
<modify buffer="goal">
<slot name="stage" equals="comparing"/>
</modify>
</actions>
</production>
<production name="comparison-succeeded">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="comparing"/>
</match>
<match buffer="configural" type="configural-query">
<slot name=":state" equals="free"/>
<slot name="query" equals="=query"/>
<slot name="reference" equals="=ref"/>
<slot name="bearing" equals="=bearing"/>
<slot name="pitch" equals="=pitch"/>
<slot name="distance" equals="=distance"/>
</match>
</conditions>
<actions>
<output>"Awesome, we know something about =query relative to =ref"</output>
<output>"=query is =bearing =pitch and =distance than =ref"</output>
<output>"lets explicitly establish a location"</output>
<add buffer="configural" type="configural-location"/>
<modify buffer="goal">
<slot name="stage" equals="locating"/>
</modify>
</actions>
</production>
The first makes an empty query request, but it makes sure that the buffer has at least two reps in it (although, an error might still occur if there aren’t actually two configural reps). The second snags the results. As you can see, the configural-query lets you know whether query is to the right/left, above/below, closer or further than reference. It also provides magnitudes, which are currently precise.. and will later be fuzzy.
Two things to note: 1) you may specify query and reference either fully or partially before making the request, however, they must be in the buffer otherwise an error will be reported. 2) Partial specifications (as above) make no promises regarding which will be query or reference.
Mental transformations
Until the ambulatory system is in place, currently the only way to update representations is to use intentional transformations. There are two kinds: transform-translation and transform-rotation. They do exactly what you think. They each take distance which for translation is in meters, rotation is in degrees (+ to the right, - to left). Translation also takes a heading (default 0), which specifies the direction to move in degrees. Rotation also permits imagining rotations in pitch by setting axis to pitch (heading is default).
How long the transform will run, depends upon the module parameters ImaginedDistanceRate, ImaginedHeadingRotationRate, and ImaginedPitchRotationRate.
These transformations will work with the automatic location encoding (below) - but can produce some crazy results when it comes to illusionary locations. Below are a handful of productions that deal with translations and rotations. Note the use of the integrator buffer state slot. This will be busy whenever there is spatial updating taking place (via motion or volition).
<!-- location suceeded, lets do some mental transforms
specifically, pretend to walk forward 10 m
the remove below will remove the match contents
of configural, i.e. the configural-loction
-->
<production name="locating-succeeded">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="locating"/>
</match>
<match buffer="configural" type="configural-location">
<slot name=":state" equals="free"/>
<slot name="query" equals="=query"/>
<slot name="reference" equals="=ref"/>
<slot name="angle" equals="=angle"/>
<slot name="reference-distance" equals="=rDistance"/>
<slot name="query-distance" equals="=qDistance"/>
</match>
</conditions>
<actions>
<output>"I am =rDistance from =ref and =qDistance from =query"</output>
<output>"=angle degrees separates =query and =ref"</output>
<output>"lets walk forward 10m"</output>
<add buffer="configural" type="configural-location"/>
<modify buffer="goal">
<slot name="stage" equals="moving"/>
</modify>
<add buffer="configural" type="transform-translation">
<slot name="distance" equals="2"/>
</add>
<remove buffer="configural"/>
</actions>
</production>
<!-- ok, we're moving...note the use of
the integrator state slot
-->
<production name="translation-running">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="moving"/>
</match>
<query buffer="configural">
<slot name="state" equals="busy"/>
<slot name="integrator" equals="busy"/>
</query>
</conditions>
<actions>
<output>"Were still walking"</output>
</actions>
</production>
<!-- done walking, lets turn around -->
<production name="translation-complete">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="moving"/>
</match>
<query buffer="configural">
<slot name="state" equals="free"/>
<slot name="integrator" equals="free"/>
</query>
</conditions>
<actions>
<output>"Done pretending to walk, lets turn around"</output>
<modify buffer="goal">
<slot name="stage" equals="turning"/>
</modify>
<add buffer="configural" type="transform-rotation">
<slot name="distance" equals="180"/>
</add>
</actions>
</production>
<production name="rotation-running">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="turning"/>
</match>
<query buffer="configural">
<slot name="state" equals="busy"/>
<slot name="integrator" equals="busy"/>
</query>
</conditions>
<actions>
<output>"Still turning"</output>
</actions>
</production>
<production name="rotation-complete">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="turning"/>
</match>
<match buffer="configural" type="configural">
<slot name="center-bearing" equals="=bearing"/>
<slot name="identifier" equals="=id"/>
<slot name=":state" equals="free"/>
<slot name=":integrator" equals="free"/>
</match>
</conditions>
<actions>
<output>"All done!! =id is at =bearing"</output>
<modify buffer="goal">
<slot name="stage" equals="finished"/>
</modify>
</actions>
</production>
Configural locations
Using to configural representations you can establish a unique location in space (well, planar space). This can be accomplished both implicitly and explicitly in ACT-R/S. By default, implicit location encoding is disabled because it requires some careful balancing of the ejection policy (see below) and capacity (otherwise you can get illusionary locations during updating). It can be enabled by setting the AutoLocationEncoding parameter to true of the module.
Explicit location encoding is just a simple request. Much like the configural-query, configural-location has two settable slots: query and reference that can be partially or completely specified at request. Similarly, the configural reps must be in the buffer or an error will be logged and the request will be aborted. The two productions below illustrate the explicit request and harvest of a location.
<production name="comparison-succeeded">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="comparing"/>
</match>
<match buffer="configural" type="configural-query">
<slot name=":state" equals="free"/>
<slot name="query" equals="=query"/>
<slot name="reference" equals="=ref"/>
<slot name="bearing" equals="=bearing"/>
<slot name="pitch" equals="=pitch"/>
<slot name="distance" equals="=distance"/>
</match>
</conditions>
<actions>
<output>"Awesome, we know something about =query relative to =ref"</output>
<output>"=query is =bearing =pitch and =distance than =ref"</output>
<output>"lets explicitly establish a location"</output>
<add buffer="configural" type="configural-location"/>
<modify buffer="goal">
<slot name="stage" equals="locating"/>
</modify>
</actions>
</production>
<production name="locating-succeeded">
<conditions>
<match buffer="goal" type="test">
<slot name="stage" equals="locating"/>
</match>
<match buffer="configural" type="configural-location">
<slot name=":state" equals="free"/>
<slot name="query" equals="=query"/>
<slot name="reference" equals="=ref"/>
<slot name="angle" equals="=angle"/>
<slot name="reference-distance" equals="=rDistance"/>
<slot name="query-distance" equals="=qDistance"/>
</match>
</conditions>
<actions>
<output>"I am =rDistance from =ref and =qDistance from =query"</output>
<output>"=angle degrees separates =query and =ref"</output>
<add buffer="configural" type="configural-location"/>
<modify buffer="goal">
<slot name="stage" equals="finished"/>
</modify>
</actions>
</production>
The configural-location also has another slot, location, which should be a chunk referencing a unique location identifier. Currently it is null because the system is not implemented. The idea here is that if you are in (roughly) the same place, but encoding locations using difference configural reps, they should all link up to the same location in space. So, you could recognize a location using different sets of configural representations.
Another request is pending completion and is by far the most useful. It is the navigation-query request. This will take two configural-locations and return the bearing and distance needed in order to get to it (assuming they both reference configural reps that have the same identifiers)
Capacity buffer behavior
I’m not going to get into why we have a capacity buffer, only point to two tidbits. The capacity can be set using the buffer’s ChunkCapacity parameter. Now, because the buffer can contain more than one chunk, we need to be concerned with which chunk gets the boot when the capacity is exceeded. That’s where the EjectionPolicy parameter comes in. Currently there are three values: LRM (least recently matched), LRA (least recently added), LRU (least recently used). LRU is the combination of LRM and LRA. LRM orders the chunks based on when they were matched by a production that fired. LRA orders the chunks based on when they were added.
Normal left hand side queries will work as you’d expect them too - however, if there is more than one chunk in the buffer that can match the query, then the least recent one is matched against.
The benefit of this is that it allows you to iterate over the contents of the configural buffer and know that you will get all the contents by the time you match a chunk the second time.
The buffer’s activation is spread out equally amoung the chunks in the buffer. Later implementations will use different proportions based on similar constraints as the ejection policy.
