When I first started working in internet development in the late 90s the landscape was very different from what we see today. Broadband in the home was unheard of, and mobile phones were still toys of the rich. People used the internet solely on computers, where the average screen resolution was 800x600 pixels, and the browsers of the time were basic in capabilities.
Whilst managing the file size of a page was crucial, the constraints did make things simple in a way. The general approach was to build to a template, taking into account the interface of the browser to give us the remaining screen space. If you stuck to a limit of 800x600 you could be confident the page would look the same to the majority of visitors.
Things are very different today. The internet is increasingly accessed by mobile devices with access speeds exceeding those which were common place in the home only a few years ago. For many, the mobile phone (or tablet) is their primary internet device.
Whilst this increase in speed, combined with modern browsers on both the desktop and mobile device, enables us to build increasingly complex designs, it also presents us with a problem.
In an ideal situation content would only be built once so that it works across multiple devices. But today we are targeting a hugely diverse range of devices and with them come a huge range of different sized displays. A basic smartphone will have a resolution of 320x240 pixels, the iPhone 5 at 1136x640 has nearly 10 times as many pixels. The pixel density (the number of pixels in an inch) varies greatly from as little as 130ppi to over 340ppi. Combine the two together and designing an interface for a fixed resolutions becomes a problem. This page gives you some idea of the problem: http://en.wikipedia.org/wiki/List_of_displays_by_pixel_density#Devices_sorted_by_size
In a web development this can be overcome in a number of ways. We can use multiple templates to display content differently on different devices. We can use CSS to create fluid layouts which adapt to the page size creating a look and feel more suited to the device. A lot of sites may use a combination of the two. We can build our services and create our content only once, but tailor what the user sees dependant on their chosen platform. But whichever approach or combination is taken, complexity and development time is increased.
Having moved away from HTML development and solely into Flash development fairly early on in my career, these things traditionally have been of no concern to me. Flash has always been a vector based environment and as such whole interfaces can be scaled to any size without distortion. As everything was built for computers I would simply build a fixed ratio design, which would stretch to fill the users screen without distortion. Whilst Flash is generally not needed for traditional website development anymore, it is a great tool for developing mobile apps. With the introduction of AIR for mobile, one code base can be used to create Flash content for both the Android and iOS devices whilst also targeting traditional desktop computers. But, there is a problem.
Mobile devices like bitmaps, images which can be loaded into a graphics processing unit (GPU). They do not like vector assets which Flash has traditional used as they cannot be loaded into the GPU. As a result performance becomes an issue when using Flash in its traditional way. Using bitmaps is greatly desired. But bitmaps in themselves do not scale as cleanly as vectors.
Mobile devices are held in the hand, and operated with fingers. As such a button needs to be big enough to be easily selected, with enough space around it such that an accidental press of an adjacent button does not occur. We are talking physical sizes, in millimetres, not pixels. Whilst it may be appropriate to have two different layouts for phones and tablets where the physical size allows for more information on screen, two similarly sized screens should display the interface elements at the same size regardless of pixel density. A button that is 1x1cm on one device should be 1x1cm on another.
I have been following Starling, a popular and rapidly expanding 2D game framework for AIR for a while. It taps into the hardware accelerated graphics of mobile devices, by presenting the GPU accelerated 3D functionality of Flash in a way that works for 2D content. It works with bitmaps assets and provides some methods to get round this scaling issue. It allows a developer to create content using a fixed display size with a set of bitmap assets, our SD version, and a second set of HD assets. Starling can pick the most appropriate assets depending on the devices pixel density. It then scales the whole application to the correct relative size for the device. You can read more about it here: http://wiki.starling-framework.org/manual/multi-resolution_development
This basic approach (strategy 2 in the Starling guide) works to an extent, and perfectly well if you are only developing for a couple of different displays (such as the iPhone and iPad) but increasingly has issues when we start supporting Android. The sheer range of resolutions and pixel densities means that whilst some devices will be displaying the assets (either SD or HD) without any scaling, many devices will. As a result on those we will have slightly blurred elements, which is particularly noticeable on text. We could make huge (super HD) assets, and always scale them down, but this would decrease performance and increase file sizes. It is possible in Starling to combine multiple asset sets with a fluid layout (strategy 3 in the guide), but it becomes increasingly more complex the more screens you attempt to target.
Another drawback of Starling is that it uses its own implementation of the basic display objects found within a Flash file, and as such converting existing code takes some time. We have built up an extensive library of reusable code which speeds up our development cycles and so I looked at ways to get these performance boosts without being dependant on Starling.
I tried to look at things from a slightly different angle. If we could combine scalable vector graphics with a fluid design, and convert those graphics to bitmaps once scaled we can have the best of both worlds. The approach is to develop to a target resolution, with a target pixel density. When the application runs we can calculate the users resolution and pixel density, and by comparing it to our target we end up with a scale factor. We know now how many pixels make up a physical size on the users device. We can therefore scale our vector graphics so they are exactly the size we want them to be, reposition them based on this scale factor, and finally convert them to bitmaps.
We still have to consider the varying display sizes, such that we build interfaces where content is neither lost off the edges on a smaller device, or appears too isolated on a larger one, but we know that buttons will be functional without being unnecessarily large, and that our text will always be clear and a sensible size.
Over the last few weeks I have been experimenting with several of the ideas and trying to build a framework that allows me to create applications that are optimised for mobile devices whilst continuing to take advantage of the power of vector assets. I have found Starling to deliver great performance and excellent results if iOS is the only target (though the new iPhone 5s resolution throws up an extra layer of complexity). Add Android to the mix however and it becomes a lot more complicated to get clean assets. My own approach of taking this scaling approach but applying it to vectors appears to have promise.
It's not without issue, as certain objects like being scaled exactly by a whole factor (such as a 1pixel outline) and I am looking at ways to overcome this with certain assets, but this problem exists when using Starling (or any other system reliant on scaled bitmaps) also. I am continuing to see where it leads, and whether it is indeed practical. It may turn out that the additional work required in considering the design and layout is too much, and a combination of my methods and Starlings may yield the best results. In fact after starting on my own libraries I discovered Feathers (http://wiki.starling-framework.org/feathers/start), recently merged into the main Starling branch. It's approach is very similar with the goal being fixed sized objects across all devices. It uses fixed bitmaps as its base, but these could be generated from the vectors when the application first runs.
Whilst doing this research I came across several approaches similar to my own, thinking in physical units is not a new concept. Perhaps I am trying to reinvent the wheel, but everybody’s workflow is different and exploring new ideas is the best way to understand the power, and weaknesses, of the existing ones.
What I can be sure of is that mobile devices are only going to grow in popularity, and with them the range of displays. Designing content in pixels just isn’t ideal anymore, as even with the ability to scale, the cleanest content is only possible with assets tailored to that display. If we can combine the versatility of the vector and the speed of the bitmap however we can create content which not only looks great, but runs great too.
