Let’s look at how to use Mosami to build an interactive display triggered by the viewer’s actions – a video output of a presenter+slide with two different layouts. When the viewer is far away, it will emphasize the presenter over the slides, but swap focus to the slides for better readability when the viewer leans in for a closer look.

What’s under the hood? Let’s take a look.

Code for this example is from our examples library at https://launchpad.net/mosami/+download in the file nearfar.py

The main thing is that we want our application to respond to the viewer’s position. To do that, we use the DetectFace module, which outputs a stream of messages with the bounding box (x,y,w,h) of the detected face(s), and attaching a callback function to handle those messages. Whenever a new DetectFace message arrives, it checks to see whether the (single) face is big or small compared to a threshold. To make it less jumpy, we accumulate those results, and add a slow drift to “far” if no face is detected.

if msg['num'] == 1:
     if msg['width'] > self.defaults['face_width_threshold']:
          self.state['tracker'] += 1.0
     else:
          self.state['tracker'] -= 1.0
else:
     self.state['tracker'] -= 0.05

After that, the callback thresholds the recent activity and chooses a “near” or “far” layout accordingly

if self.state['tracker'] > 20:
     self.state['layout'] = 'near'
     self.state['tracker'] = 10
if self.state['tracker'] < -20:
     self.state['layout'] = 'far'
     self.state['tracker'] = -10
self.mix.msg('switch_layout', layout=self.state['layout'])

Finally, we want that ‘switch_layout’ message to change the display. First, we need the basic display elements. For the presenter, we remove the background with CutoutPerson. For the slides, we use ViewPDF to convert the slides into a video stream. Then, we composite them together using LayoutMix, which allows one to preconfigure multiple layouts.