Microblogging demo - google i/o talk, for java?

Question

Hi,

I'm looking to implement a pub/sub feature in my web app as Bret Slatkin described in his 2008 google i/o presentation, "creating scalable web apps". If UserA posts a message, I want all their followers to see that they have a message waiting for them next time they visit the web app.

I'm not familiar with python and am not sure if I translated his example correctly here:

class Message {
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    Long id;

    String sender;
    Text body;
}

class MessageIndex {
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    Long id;

    @Persistent // Puts us under same entity group as parent Message.
    @Extension(vendorName="datanucleus", key="gae.parent-pk", value="true")
    private Long idParent;


    List<String> receivers;
}

public void userPostsMessage(String username,
                             String body)
{
    PersistenceManager pm = PMF.get();

    // We have to load all friends of 'username' to define the recipients
    // of the message. This could be catastrophic if the user has lots of
    // friends!
    List<String> friendIds = datastore.getAllFriendsForUser(username);

    // Create the message.
    Transaction tx = pm.currentTransaction();
    try {
        tx.begin();

        Message msg = new Message();
        msg.sender = username;
        msg.body = body;

        pm.makePersistent(msg);

        // When creating the MessageIndex instance, we are putting it under
        // the same entity group as the parent Message instance.
        MessageIndex mi = new MessageIndex();
        mi.idParent = msg.id;

        // This will only handle 5,000 followers. I'm not sure if even 
        // attempting to write anywhere near that amount in a single 
        // operation is even ealistic either!
        for (String it : friendIds) {
            mi.receivers.add(it);
        }

        pm.makePersistent(mi);

        tx.commit();
    }
    finally {
        if (tx.isActive()) {
            tx.rollBack();
        }
    }
}

public void doIHaveAnyMessages(String username) {
    PersistenceManager pm = PMF.get();

    // Create the message.
    Transaction tx = pm.currentTransaction();
    try {
        tx.begin();

        Query query = pm.newQuery(Message.class); 
        query.setFilter("receivers == '" + username + "'"); 
        List<MessageIndex> results = (List<MessageIndex>) query.execute();

        // We have the list of MessageIndex instances matched for us,
        // but how do we do the parallel fetch of their parent Message
        // instances as in the presentation?:
        List<Message> messages = pm.getObjectByKeys(results?);

        tx.commit();           

        return messages; 
    }
    finally {
        ...
    }
}

What I'm not sure about:

1. How am I supposed to get the list of the user's friends [efficiently] to write each of them into the MessageIndex.receivers list? If a user has one million friends, this is going to require a great deal of resources?
2. Since each List property can only hold 5,000 indexed entries, what do you do when the user has one million friends? Do I have to just create multiple MessageIndex instances for the one Message instance?

3. In the doIHaveAnyMessages() method, once I get the list of MessageIndex results, how do I efficiently get their parent Message instances? In the python example, he does this:

   indexes = db.GqlQuery( "SELECT key FROM MessageIndex " "WHERE receivers = :1", me) keys = [k.parent() for k in indexes]
   messages = db.get(keys)

not sure how statement #2 (keys = [k.parent() for k in indexes]) translates to java.

Sorry for the massive question, I hope I've explained the misunderstood parts clearly,

Thank you

http://code.google.com/events/io/2009/sessions/BuildingScalableComplexApps.html