maybe you can use traer physics to simulate it...
http://www.cs.princeton.edu/~traer/physics/

there is a smoke example http://www.cs.princeton.edu/~traer/smoke/
maybe if you give them some repulsive force they behave similar to what you are looking for.
I guess the only thing you have to figure out os how to keep it within a container... but it should be possible. Seems to be not that complicated.

The real issue is going to be optimizing it in a way that it will run at a real time speed.

I would probably try to create a simple 2D spacial partitioning system like a grid to store references to the particles, as particles move between grid cells they should notify the spacial grid of their cell change

Then it will be easy to create a list of neighbors for each particle  sorted closest first using an insertion sort

(probably want to use the 4 or 9 closes cells depending on the size of your cells)

you may want to limit the neighbors to the top x number of particles in the neighbor list to reduce the number of neighbors you calculate forces for, then finally iterate through all the particles applying force from each of the neighboring particles.

You'll have to find the sweet spot where when the particles relax they're efficiently using up the grid. If your grid is to dense you'll take a performance hit for iterating over a lot of empty cells, if it's too sparse you'll end up with too many clusters of particles that end up creating a large performance penalty.

A tricky but interesting problem for sure!

Jack