State Machines and User Interface work -- any examples/experience?

Question

Hello

I'm looking for ways to de-spaghttify my front-end widget code. It's been suggested that a Finite State Machine is the right way to think about what I'm doing. I know a State Machine paradigm can be applied to almost any problem. I'm wondering if there are some experienced UI programmers who actually make a habit of this.

So, the question is -- do any of you UI programmers think in terms of State Machines in your work? If so, how?

thanks, -Morgan

Answer 1

Each interface item that's presented to the user can go to another state from the current one. You basically need to create a map of what button can lead to what other state.

This mapping will allow you to see unused states or ones where multiple buttons or paths can lead to the same state and no others (ones that can be combined).

Answer 2

Its not really a UI problem, to be honest.

I'd do the following:

1. Define your states
2. Define your transiations - which states are accessible from which others?
3. How are these transitions triggered? What are the events?
4. Write your state machine - store the current state, receive events, and if that event can cause a valid transition from the current state then change the state accordingly.

Answer 3

I'm currently working with a (proprietary) framework that lends itself well to the UI-as-state-machine paradigm, and it can definitely reduce (but not eliminate) the problems with complex and unforeseen interactions between UI elements.

The main benefit is that it allows you to think at a higher level of abstraction, at a higher granularity. Instead of thinking "If button A is pressed then combobox B is locked, textfield C is cleared and and Button D is unlocked", you think "Pressing button A puts the app into the CHECKED state" - and entering that state means that certain things happen.

I don't think it's useful (or even possible) to model the entire UI as a single state machine, though. Instead, there's usually a number of smaller state machines that each handles one part of the UI (consisting of several controls that interact and belong together conceptually), and one (maybe more than one) "global" state machine that handles more fundamental issues.

Answer 4

Hey Morgan, we're building a custom framework in AS3 here at Radical and use the state machine paradigm to power any front end UI activity.

We have a state machine setup for all button events, all display events and more.

AS3, being an event driven language, makes this a very attractive option.

When certain events are caught, states of buttons / display objects are automatically changed.

Having a generalized set of states could definitely help de-spaghttify your code!

Answer 5

A state machine is something that allows code to work with other state machines. A state machine is simply logic that has memory of past events.

Therefore humans are state machines, and often they expect their software to remember what they've done in the past so that they can proceed.

For instance, you can put the entire survey on one page, but people are more comfortable with multiple smaller pages of questions. Same with user registrations.

So state machine have a lot of applicability to user interfaces.

They should be understood before being deployed, though, and the entire design must be complete before code is written - state machine can, are, and will be abused, and if you don't have a very clear idea of why you're using one, and what the goal is, you may end up worse off than other techniques.

Answer 6

It's not the UI that needs to be modeled as a state machine; it's the objects being displayed that it can be helpful to model as state machines. Your UI then becomes (oversimplification) a bunch of event handlers for change-of-state in the various objects.

It's a change from:

DoSomethingToTheFooObject();  
UpdateDisplay1();  // which is the main display for the Foo object  
UpdateDisplay2();  // which has a label showing the Foo's width,
                   // which may have changed  
...

to:

Foo.DoSomething();  

void OnFooWidthChanged() { UpdateDisplay2(); }  
void OnFooPaletteChanged() { UpdateDisplay1(); }

Thinking about what changes in the data you are displaying should cause what repainting can be clarifying, both from the client UI side and the server Foo side.

If you find that, of the 100 UI thingies that may need to be repainted when Foo's state changes, all of them have to be redrawn when the palette changes, but only 10 when the width changes, it might suggest something about what events/state changes Foo should be signaling. If you find that you have an large event handler OnFooStateChanged() that checks through a number of Foo's properties to see what has changed, in an attempt to minimize UI updates, it suggests something about the granularity of Foo's event model. If you find you want to write a little standalone UI widget you can use in multiple places in your UI, but that it needs to know when Foo changes and you don't want to include all the code that Foo's implementation brings with it, it suggests something about the organization of you data relative to your UI, where you are using classes vs interfaces, etc.... Fundamentally, it makes you think more seriously about what is your presentation layer, more seriously than "all the code in my form classes".

-PC

Answer 7

State machines are generally too low-level to help you think about a user interface. They make a nice implementation choice for a UI toolkit, but there are just too many states and transitions to describe in a normal application for you to describe them by hand.

I like to think about UIs with continuations. (Google it -- the term is specific enough that you will get a lot of high quality hits.)

Instead of my apps being in various states represented by status flags and modes, I use continuations to control what the app does next. It's easiest to explain with an example. Say you want to popup a confirmation dialog before sending an email. Step 1 builds an email. Step 2 gets the confirmation. Step 3 sends the email. Most UI toolkits require you to pass control back to an event loop after each step which makes this really ugly if you try to represent it with a state machine. With continuations, you don't think in terms of the steps the toolkit forces upon you -- it's all one process of building and sending an email. However, when the process needs the confirmation, you capture the state of your app in a continuation and hand that continuation to the OK button on the confirmation dialog. When OK is pressed, your app continues from where it was.

Continuations are relatively rare in programming languages, but luckily you can get sort of a poor man's version using closures. Going back to the email sending example, at the point you need to get the confirmation you write the rest of the process as a closure and then hand that closure to the OK button. Closures are sort of like anonymous nested subroutines that remember the values of all your local variables the next time they are called.

Hopefully this gives you some new directions to think about. I'll try to come back later with real code to show you how it works.

Update: Here's a complete example with Qt in Ruby. The interesting parts are in ConfirmationButton and MailButton. I'm not a Qt or Ruby expert so I'd appreciate any improvements you all can offer.

require 'Qt4'

class ConfirmationWindow < Qt::Widget
  def initialize(question, to_do_next)
    super()

    label = Qt::Label.new(question)
    ok = ConfirmationButton.new("OK")
    ok.to_do_next = to_do_next
    cancel = Qt::PushButton.new("Cancel")

    Qt::Object::connect(ok, SIGNAL('clicked()'), ok, SLOT('confirmAction()'))
    Qt::Object::connect(ok, SIGNAL('clicked()'), self, SLOT('close()'))
    Qt::Object::connect(cancel, SIGNAL('clicked()'), self, SLOT('close()'))

    box = Qt::HBoxLayout.new()
    box.addWidget(label)
    box.addWidget(ok)
    box.addWidget(cancel)

    setLayout(box)
  end
end

class ConfirmationButton < Qt::PushButton
  slots 'confirmAction()'
  attr_accessor :to_do_next
  def confirmAction()
    @to_do_next.call()
  end
end

class MailButton < Qt::PushButton
  slots 'sendMail()'
  def sendMail()
    lucky = rand().to_s()
    message = "hello world. here's your lucky number: " + lucky
    do_next = lambda {
      # Everything in this block will be delayed until the
      # the confirmation button is clicked. All the local
      # variables calculated earlier in this method will retain
      # their values.
      print "sending mail: " + message + "\n"
    }
    popup = ConfirmationWindow.new("Really send " + lucky + "?", do_next)
    popup.show()
  end
end

app = Qt::Application.new(ARGV)

window = Qt::Widget.new()
send_mail = MailButton.new("Send Mail")
quit = Qt::PushButton.new("Quit")

Qt::Object::connect(send_mail, SIGNAL('clicked()'), send_mail, SLOT('sendMail()'))
Qt::Object::connect(quit, SIGNAL('clicked()'), app, SLOT('quit()'))

box = Qt::VBoxLayout.new(window)
box.addWidget(send_mail)
box.addWidget(quit)

window.setLayout(box)
window.show()
app.exec()

Answer 8

Hi,

There is a book out there about this topic. Sadly its out of print and the rare used ones available are very expensive.

Constructing the User Interface with Statecharts
by Ian Horrocks, Addison-Wesley, 1998

Answer 9

I got a prezi-presentation about a pattern that I call "State First".

It is a combination of MPV/IoC/FSM and I've used it successfully in .Net/WinForms, .Net/Silverlight and Flex (at the moment).

You start by coding your FSM:

class FSM
    IViewFactory ViewFactory;
    IModelFactory ModelFactory;
    Container Container; // e.g. a StackPanel in SL
    ctor((viewFactory,modelFactory,container) {
        ...assignments...
        start();
    }

    start() {
        var view = ViewFactory.Start();
        var model = ModelFactory.Start();
        view.Context = model;
        view.Login += (s,e) => {
            var loginResult = model.TryLogin(); // vm contains username/password now
            if(loginResult.Error) {
                // show error?
            } else {
                loggedIn(loginResult.UserModel); // jump to loggedIn-state
            }
        };
        show(view);
    }


    loggedIn(UserModel model) {
        var view = ViewFactory.LoggedIn();
        view.Context = model;
        view.Logout += (s,e) => {
            start(); // jump to start
        };
        show(view);
    }

Next up you create your IViewFactory and IModelFactory (your FSM makes it easy to see what you need)

public interface IViewFactory {
    IStartView Start();
    ILoggedInView LoggedIn();
}

public interface IModelFactory {
    IStartModel Start();
}

Now all you need to do is implement IViewFactory, IModelFactory, IStartView, ILoggedInView and the models. The advantage here is that you can see all transitions in the FSM, you get über-low coupling between the views/models, high testability and (if your language permits) a great deal of type safely.

One important point in using the FSM is that your shouldn't just jump between the states - you should also carry all stateful data with you in the jump (as arguments, see loggedIn above). This will help you avoid global states that usually litter gui-code.

You can watch the presentation at http://prezi.com/bqcr5nhcdhqu/ but it contains no code examples at the moment.