How do I create two mutual producer/consumers with internal state in Haskell?

Question

I've got an agent that takes in states and returns actions, while keeping an internal representation of the utility of state/action pairs. I've also got an environment that takes in actions and returns state/reward pairs.

I need to be able to set the agent up with a start state and then continuously go from agent -(action)-> environment -(state, reward)-> agent -(action)->... However, the internal states (which need to be updated every iteration) need to stay private (that is, within the agent or the environment). This means that I can't simply call environment as a function within the agent using state and action as arguments.

I'm somewhat of a Haskell noobie, so I'm not even sure if this is possible.

Answer 1

You'll need to decide which of the agent and the environment sits "on top" - let's suppose for the rest of this answer that it's the environment on top calling into the agent since that generally makes most sense.

You can keep data representations of the Agent private using the module system - just export the datatype name without any of its internals.

module Agent (AgentState, other_stuff) where

as opposd to

module Agent (AgentState(..), other_stuff) where

If the agent also needs to be passed the environment's state (though I can't see any reason this would be necessary as the environment can keep track of it for itself), then make the agent functions polymorphic so that they can be passed any state type - then the environment can pass whatever it likes in without its representation being exposed.

It should also be possible to use state monads to achieve more control over what happens to the state, for example preventing the environment from duplicating a state the agent gives it and calling the agent repeatedly with that same state, but if you're new to Haskell it's probably best to get a bit of experience without monads first. (Not that monads are particularly scary or anything, but they do hide detail from you so it's harder to see what's going on.)

Answer 2

Two questions:

• If the agent must use a state to compute an action, then how do you expect to keep the representation of states secret from the agent?

• If the environment expects to produce a state (and a reward) given an action, how do you expect to keep the representation of states secret from the environment?

Both of these are possible, but each must involve some sort of abstraction for querying states or creating states. I don't have a good feeling about this design.

It would help to clarify the question by

• Providing type signatures for the functions of interest

• Identifying to which functions you want the representation of states not to be exposed.

P.S. These difficulties are quite separable from Haskell and would be at issue regardless of the choice of implementation language (provided that the implementation language supports some form of privacy).