A guest post from Thomas Voss’ blog. Thomas is the Technical Architect (Client) at Canonical:

Some time ago, Canonical started internal discussions about our convergence strategy, clearly spelling out the distant target of shaping and developing a single computing platform and operating system that is able to power the cloud, classic desktop machines, laptops, TV sets, phones and tablets (fridges anyone?). More to this, we stated that we want a single mobile device, a bundle of portable computing power together with the respective operating system, that seamlessly adapts to different form-factors and use-cases. Or to put it a little more catchy:

Your phone is your TV, is your desktop, is your … you name it.

And yes, we were aiming for the moon. We worked hard to draft a system that would allow us to reach the moon, while at the same time catering towards our other central goal of providing a beautiful and lean user experience. It became apparent in conversations and discussions that one of the cornerstones of the future system we were designing will be the graphics stack, including the Unity shell. After much discussions about existing solutions (X & Weston), and how we could leverage them, we took a step back and distilled down our (technical) requirements for a future graphics stack:

• Tailored towards an EGL/GL(ES) world.
• Minimal assumptions regarding the underlying driver model.
• Ability to leverage existing drivers implementing the Android driver model.
• Ability to leverage existing hardware compositors.
• Efficient, in terms of memory-, CPU- and GPU-consumption/usage.
• Tightly integrated with the Unity shell, fulfilling the shell’s requirements while at the same time not dictating any sort of semantics up the stack.
• An efficient and secure input subsystem supporting demanding mobile use-cases.
• Fully tested from the ground up.
• Adaptable to future requirements.

With these priorities in mind, we revisited and carefully evaluated existing solutions again and found that neither of them satisfies our requirements. In particular, X and its driver model is way too complicated and feature-laden, resulting in a less efficient system and a driver model that is unlikely to be widely supported on mobile platforms. In the case of Weston, the lack of a clearly defined driver model as well as the lack of a rigorous development process in terms of testing driven by a set of well-defined requirements gave us doubts whether it would help us to reach the “moon”. We looked further and found Google’s SurfaceFlinger, a standalone compositor that fulfilled some but not all of our requirements. It benefits from its consistent driver model that is widely adopted and supported within the industry and it fulfills a clear set of requirements. It’s rock-solid and stable, but we did not think that it would empower us to fulfill our mission of a tightly integrated user experience that scales across form-factors. However, SurfaceFlinger was chosen as our initial solution for getting started with the overall Ubuntu Touch project, planning to replace SurfaceFlinger with Mir as soon as possible.

In summary, our evaluation of existing solutions led us to the conclusion that neither of them fits our requirements and that adjusting/adapting them would require substantial efforts, too. For this, we decided to go for our own solution, catering directly towards our goals and our vision, effectively saying: Yes, we are going to do our own display server.

The project was born and we decided to name it Mir (as in the space station), as it would be our outpost that finally enables us to reach our goal of arriving at the moon, providing a seamless and beautiful user experience spanning multiple form-factors. The team set off to work on Mir while large parts of Canonical started working on the Ubuntu Touch project (relying on SurfaceFlinger). A lot of knowledge and experience was and is transferred from the Ubuntu Touch project to the Mir team, with the work on the phone and tablet revealing more and more technical requirements and subtleties that heavily influenced the Mir architecture and implementation. Thus, one of the earliest technical decisions that has been taken by the team concerned the protocol that applications rely upon to communicate to Mir. We evaluated Wayland and compared it to the SurfaceFlinger approach, realizing that neither of them fits our needs. Wayland is a very promising and sensible attempt at standardizing the way that applications talk to a display server, however, it exposes privileged sections like the shell integration that we planned to handle differently, both for security reasons and as we wanted to decouple the way the shell works on top of the display server from the application-facing protocol. In the end, we decided to implement Mir’s core in a protocol-agnostic way, considering the communication protocol a very important part of the overall story, but not the driving one. We have chosen Google’s protocol buffers as our data and interface description language (DDL & IDL) and implemented a lean RPC layer that abstract away transport-specific details (relying on an ordinary socket by default). It is well tested and the protobuf IDL and DDL provide us with forward- as well as backward-versioning capabilities. However, the communication component of the overall system is adaptable and can be adjusted to account for future requirements and developments.

A final word on timelines: At the time of this writing, we are preparing to deploy the next version of the Unity shell tightly integrated with Mir in the not-too-distant future on the phone and the tablet. However, regarding the desktop use-case: We have integrated X, leveraging the prior XWayland work, to run on top of Mir (XMir) and started initial porting efforts for toolkits with a focus on Qt5. With our X-integration in place, you can run Mir on your desktop machine if your system runs a GPU that supports the free driver stack. For the closed-source desktop drivers: We are in active conversations with GPU vendors to enable Mir on those drivers/GPUs, too. More to this, we are working towards a more unified driver model sitting on top of EGL.

As we are moving along with the development of Mir, we will work on integrating existing toolkits with the new display stack to allow application developers a seamless transition between different form-factors and prevent them from the burden of supporting multiple different display servers. As noted before, Qt5 is our first target here, with GTK3 and XUL being in the pipeline. To support legacy X applications that cannot be adjusted to work against Mir, we will provide a translation-layer that leverages the existing XMir implementation and allows legacy X apps to use Mir transparently.

And that’s pretty much it. Thanks for reading. I hope I could give you some insight into the history of Mir, its motivation and its trajectory. If you want to dive even further into the Mir world, head over to wiki.ubuntu.com/MirSpec. A lot of work has already been done, but more of it lies ahead of us. So if you are curious, have questions, want to help us or simply want to get to know the team, join the Mir sessions at the upcoming virtual UDS. You might also want to read Olli’s description of the bigger picture, including our overall convergence strategy for Unity.

To the moon,

Thomas

A guest post from Olli Ries’s blog. Olli is the Engineering Director for Unity and Display Server at Canonical:

It all sounds good in theory…

Not too long ago, Mark communicated the vision for Ubuntu and Unity for 2013 as “[…] Unity in 2013 will be all about mobile – bringing Ubuntu to phones and tablets […]” and my team is responsible for taking Unity to these hardware platforms.

What you should expect to see during this year is an overhaul of Unity in order to power a wide variety of display sizes (think phone to tablet to desktop to TV to…), input methods (touch screens & on screen keyboards, traditional keyboards & pointer devices, voice, and whatever else Tony Stark makes us think of [*]), CPUs & GPUs, external peripherals and everything else we expect from a modern OS.

Looking closer at the problem ambitious goal, we had to take a few interesting decisions how we possibly would get to where we want to be.

Today Unity (the rendering part of it) runs as a plugin in Compiz which sits on top of X and is a recurring source of frustration on the developer-, design- and finally also on the user side as a lot of our ideas (handling a multi-monitor setup in 12.04 or our plans for menu bars come to mind) require quite some intrusive changes to the underlying system. Regardless of how carefully crafted the solution, it is bending a stack to something that it wasn’t necessarily designed for. Bend over backwards too much and you will fall on your back, bugging users with nasty bugs, regressions, unexpected behavior and plenty more that drives the frustration level up.

[*] I’d really love to see motion gestures, accelerometer and camera based ones- volunteers welcome

Enter Mir

While evaluating our options, looking at extending the current stack to our needs, using the Wayland protocol (or any of its implementations) and comparing that with our designs & ideas we concluded that neither approach would allow us to do what we want in the quality that we would like to see for Ubuntu & Unity (at cost and in time).

Plus, there is the rather sizable challenge of pulling Wayland/X onto a mobile device, working with SoCs on driver support, tuning this stack for power consumption and performance and dealing with other issues of a stack that hasn’t been designed for a convergence setup as we envision it. A lot of distraction from the actual goal, to provide an outstanding experience across all the supported devices – from consumer electronics to desktop computing devices to enterprise devices.

The chosen approach was to develop Mir, our own Display Server which is engineered driven by the designs and requirements that our larger vision dictates – no compromises, no crude hacks, fully testable & tested, performance in mind, support for legacy X applications, developed by Ubuntu for Ubuntu. We have put together our thoughts in the Unity & Mir specs and will introduce the topic for discussion at UDS this week.

In accordance with Ubuntu Touch’s SDK (http://developer.ubuntu.com/2013/02/releasing-the-ubuntu-touch-developer-preview-and-sdk-alpha/) we have also decided to transition the Unity Shell from Nux towards a QML based implementation. The new proposed monthly release cadence would allow us to iterate quickly and we are planning to tackle the transition component by component. Having one single code base that serves all variants of input & display types will help greatly to deliver the unified User Experience we are committing to.
Unity’s APIs are at the heart of your (Desktop) Shell and are constantly being evaluated and improved. In 2013 we will be looking at these APIs with a specific focus on resource efficiency, performance and actual API design to help increase the adoption with 3rd party developers.

Thomas Voss is giving you some more insight into Mir over at his blog.

…but in practice

Despite the large scope of our grand plan, it still looks good in practice and I am very excited about the change that we are going to drive.

While criticized as being full of mocks, placeholders and the like, all the code in Ubuntu Touch is what we deem production code and we have planned to remove stubs & mocks in the next little bit. A look at the published sources and following the development will show you our commitment to deliver high quality software. For Unity, this means there is one code base that has already demoed strong with the industry at CES and MWC and will be build upon – there is no distinction between demo & production code, in other words: what you see is the real thing and it’s improving every day

As Ubuntu Touch shows, Qt/QML has proven to be a rapid development “framework” that allows us to implement our user experience and more importantly to iterate over existing implementations with feedback from User Research and other drivers. The already mentioned proposed shorter release cycles will help us to land code faster and in better quality.

The “100 Scopes project” which was initially targeted for 13.04 is the first initiative to review our APIs and is well on it’s way – it is continuing to show us potential limitations of the current design  which are going to be discussed at upcoming UDSes.

Mir, despite being released only now has had some internal head start and is in certain areas progressing very well, to the extent that we are confident for Mir to start the discussion about landing Mir in an Ubuntu release soon.

Now, I am not trying to understate the problem we are will be solving and I can share with you that there are still some nights with less sleep (not quite sleepless though) about the unknown unknowns that are ahead of us.

I am certain that we will hit limitations in QML sooner rather than later which will have us look more closely into Qt and work with the large Qt community to address these.

Mir is an ambitious project with lots of parameters to it and we appreciate that we are trying to replace a stack that was maturing for almost 3 decades (no pressure, team .

Finally, designing a user experience that adopts itself seaminglessly between all the various devices and combinations thereof is no easy task and there are plenty of discussions how to close the gap between the Unity that you know today (up to Ubuntu Raring) and what we showed as part of Ubuntu Touch.

If it was easy it wouldn’t be fun! (famous last words)

2D or not 2D (or: learn, adapt, win)

12.04 shipped “Unity” (Nux) and “Unity 2D” (Qt/QML) , in 12.10 “Unity 2D” (Qt/QML) was deprecated in favor of “Unity” (Nux) and now Unity will be moved to Qt/QML? What the…

One thing I enjoy about Canonical is that we are not shy in providing leadership and will take controversial decisions if and when we believe they will help Ubuntu as a project to achieve the vision. Success however is only reached if you don’t rest on a made-decision but constantly evaluate where your decisions took you and what is happening around you. If needed you will have to go back and adapt to something new. A very basic concept of evolution in my books.

Back at UDS Q and prior to that, we discussed in length where Unity should be headed. Are we betting on Qt/QML, which at that time was going through a hair-raising transition or should we play safe and keep full control of what is dear to us? There were lots of technical evaluations, measurements, discussions, arguments and everything else you would do to make an influential decision like that. The outcome was that the implementation of Unity known as Unity 2D would be deprecated in order to not having to maintain 2 code bases with differing levels of feature completion, bug incompatibility and so on.

Fast forward 9 months, we have learned that our concerns have not been justified, Qt 5 has come out nicely and we have full confidence in using it as one core technology in our offerings.

Invitation to participate

Starting today we are opening up the development of this new exciting next step of Unity and invite interested developers to participate.

We will run sessions regarding Mir and Unity at this weeks UDS and would be happy to have you there. If you can’t make it to UDS this time then feel free to reach out to the team on the unity-dev mailing list (mailto: unity-dev@lists.launchpad.net). You can of course also find us on IRC (freenode, #ubuntu-unity).