For more details go to:  http://developer.android.com/sdk/ndk/index.html

Graphics Performance is the most important component in defining any user interface (UI) experience. The smoothness of the UI interactions and the crispness of the images are all directly dependent on the graphics capabilities of the underlying platform. Customers expect rapid visual feedback when buttons and keys are pressed.  Having a dedicated graphics engine and an operating system that correctly utilizes that engine will significantly reduce the workload on the CPU, vastly improving user experience.

Here at MIPS we support many open source and commercial operating systems with versions that are optimized to take full advantage of the MIPS processor architecture.  Android is one of those open source operating system that we have put significant work into porting to our architecture.  Early in the life of Android many cell phone companies shipped “bare bones” products running Android.  Many of those products did not have the responsiveness and crisp graphics customers were expecting because the products did not have a complete graphics engine and relied on the CPU to “emulate” graphics operations.  Since then almost all smart phones now have graphics engines that fully support the requirements of Android which takes a large burden off the CPU.

Moving onto the specifics of efficient graphics processing, the hardware and software have to work in tandem to enable this. This post aims to briefly explain a few related topics so that engineers can have a better understanding of Android’s Graphics subsystem. I will cover some of these topics in more detail in my future posts.

The Android Graphics Software Components

The following chart shows the software components in the Android operating system responsible for graphics.  I will go through a discussion of most of these and provide some insight into their purpose.

Approximate Relationship between various graphics components in a typical Android application

Role of GPUs:

A Graphics Processing Unit (GPU) is a specialized hardware engine which greatly accelerates graphics operations. They are different from CPUs in that they are designed to do highly parallel work – which is typical of most graphics operations.

When Android first started, GPUs were somewhat optional, but with newer releases, GPUs became a mandatory requirement. Systems without GPUs use a software OpenGL ES stack consisting of libagl & pixelflinger, sometimes with hardware support from CopyBIT. Unfortunately software emulation of OpenGL on Android does not support the OpenGL ES 2.0 standard.  Today large parts of the Android operating system use OpenGL ES 2.0 through components like HWUI, Renderscript, SurfaceTextures, etc.  Tablets devices all have high resolution and software simply cannot keep up with the fill rates required to provide a smooth user experience.  Manufacturers who plan to launch new devices that support Android Ice Cream Sandwich (ICS) or later releases must have an OpenGL ES 2.0 GPU.

Canvas:

Canvas is the Android class which application developers would use to draw widgets, pictures etc. In Android versions Froyo and Gingerbread Canvas would do the drawing using Skia.  Android Honeycomb and onward, HWUI was added as an alternate GPU-accelerated option. Android Ice Cream Sandwich (ICS) and onward, HWUI is the default.

Skia:

Skia is a 2D drawing API which is used by applications and works completely in software.  For performance reasons Skia is slowly being replaced by HWUI.

HWUI

The HWUI library enables UI components to be accelerated using the GPU. HWUI was introduced in Honeycomb to make the user interface fast, responsive and smooth.  Since tablets had very high resolutions, using Skia for effects like animation would have been too CPU intensive and slow. HWUI requires an OpenGL ES 2.0 compliant GPU which cannot be emulated by software on Android.

Renderscript

Renderscript was a new API introduced in Honeycomb* to address portability and performance at the same time. The application developer writes the code in the Renderscript language (which is based on C99), and an LLVM cross compiler on the host converts it to machine-independent IR called bit code, the developer then packages the bit code files into the Android application (APK).  When the user downloads the APK, an on-device compiler (LLVM based, located in /system/lib/libbcc.so) compiles it from bit code to the actual machine instructions for the target platform.

* Renderscript was available in Froyo & Gingerbread but was not a published API. Only a few wallpapers in the Android source used it. Its implementation was also fairly trivial & constrained.

Surface:

A Surface in Android corresponds to an off screen buffer into which an application renders the contents. An application might be a game which uses OpenGL to draw 3D objects directly into a surface or a normal application which uses Skia to draw widgets, text, etc. It could even use HWUI library to enable a GPU accelerated user interface. From Android ICS, surfaces are implemented with a SurfaceTexture backend which means instead of a buffer, a texture is used.

Graphics stack in an Android Platform

SurfaceFlinger:

SurfaceFlinger is the compositor that manages multiple surfaces from various apps. For example, there could be many running applications with independent Surfaces that are being rendered. SurfaceFlinger determines what will be shown on the screen and does overlays as needed.

SurfaceFlinger uses only the functions in the OpenGL ES 1.1 standard.  If SurfaceFlinger used functions in OpenGL ES 2.0, it would require GPU hardware and drivers to be working – which would complicate bring-up on new devices and the Android emulator.

HW Composer:

HW Composer is a Hardware Abstraction Layer (HAL) introduced in Honeycomb that is used by SurfaceFlinger to efficiently perform composition using hardware resources like a 3D GPU or a 2D graphics engine.

CopyBit:

CopyBit was a hardware abstraction layer (HAL) which allowed the use of special hardware that can speed up some graphics operations like blitting etc. It was used to accelerate software rendering on systems without 3D GPUs. CopyBit support was removed in ICS, since GPUs became mandatory for supporting the newer user interface.

Libagl/PixelFlinger:

libagl is the library which implements the OpenGL ES 1.0 & 1.1 API in software. It uses PixelFlinger to implement the OpenGL calls. To speed up the rendering using PixelFlinger, a just in time compiler (JIT) was introduced called CodeFlinger. CodeFlinger generates direct machine code which drastically speeds up many types of pixel operations.

Conclusion

I hope this provided you with a general summary of all the different graphics components in Android.  In my next article I will go into more details about the operation of some of these components.  MIPS continues to lead the industry in highly optimized versions of the Android operating system.  For more information about Android on MIPS go to http://developer.mips.com or http://www.github.com/mips

Acknowledgments:

Romain Guy & Chet Haase’s presentation
http://www.curious-creature.org/2010/12/02/android-graphics-animations-and-tips-tricks/

Cemil Azizoglu’s presentation
http://www.vivantecorp.com/Khronos.pdf

Jim Huang’s presentation
http://0xlab.org/~jserv/android-graphics.pdf

fatal: Unable to look up github.com (port 9418) (Name or service not known)

Chances are you are having a DNS server error, the link below provides a temporary solution to the problem.

http://www.question-defense.com/2009/03/25/fatal-unable-to-look-up-githubcom-port-9418-name-or-service-not-known

There is also a new release of Dalvik Turbo for Ice Cream Sandwich available at:  http://developer.mips.com/login/dalvik-turbo/ This release addresses an issue with the Dalvik test suite.

MIPS’ MT technology boosts performance 43 percent; combined MT & MC boosts performance 150 percent

The use of multi-core technology to deliver more CPU horsepower is one of the increasingly common methods to providing higher system performance in hardware. This is true even for high volume consumer applications where cost and power can be very important. However, upgrading to a multi-core system doesn’t automatically guarantee performance improvements or an enhanced user experience. It’s not just a hardware problem – software must be written in a way that can make use of parallel hardware resources. But software is adapting – systems are getting much more complex, such that multiple processes and threads are running simultaneously in many cases, and applications are being written to take better advantage of multiprocessing hardware trends.

With that in mind, we recently used the BrowsingBench™ benchmark from EEMBC to evaluate the performance benefits of MIPS’ multi-core (MC) and multi-threading (MT) technologies. Our objective was to find out how these technologies enhance the user experience of a very popular and very real consumer application – web browsing on the Android™ software platform.

BrowsingBench is a credible and widely used tool that is trusted and cited by leading technology companies. It measures web page loading and rendering time for a large set of web pages with diverse content, and it does this in a reliable way which leads to repeatable and meaningful results. It will run on any connected device with a web browser. And rather than performing a synthetic test, BrowsingBench performs the same operations which a human would perform on the device. We’ve used several other benchmarks in the past that were suitable for evaluating MC/MT system performance; however none represented the real-world user experience on connected devices as well as BrowsingBench.

We ran BrowsingBench on a system based on the MIPS32® 1004K™ Coherent Processing System (CPS). In its maximum implementation, the 1004K CPS can support up to four cores and two hardware threads (also known as Virtual Processing Elements or VPEs) per core. To keep things simple for this test however, the configuration we used was dual core with two VPEs per core, for a total of 4 VPEs. VPEs are essentially logical CPUs that share one physical pipeline in each 1004K core, based on MIPS’ multi-threading technology.

The software platform running on our system was Android, and the web browser we used was the Android browser which comes with every Android-based system.

To evaluate the benefits of multiple cores and VPEs on web browser performance, we ran BrowsingBench using the 4 different configurations listed in the table below. In all cases, the tests were executed on the same dual-core 1004K system; however we used the operating system to enable and disable cores and VPEs.

The big question we wanted to answer was whether Android would be able to take advantage of these multiple processing resources to load and render web pages faster, thereby enhancing the user experience. To do that, Android would have to use parallel processes and threads while executing the browsing workload.
The results, which are shown in the table and chart below, leave no doubt: Android based web browsing performance is greatly enhanced by MC and MT technologies.

The main observation is that browsing performance improves more than 2.5x when comparing the full configuration to the basic configuration. With a great deal of parallel execution under Android, the browser can truly benefit from the combination of MT and MC. A closer look at what’s happening under the hood in the Android system indeed shows that a lot of processes are running in parallel. The two main processes in the system are the Android Browser itself and another process called “system server,” which manages many components of Android including the display system, and is kept very busy during the BrowsingBench run.

Even if we limit the system to a single core, the MIPS MT technology gives us a BrowsingBench performance boost of 43 percent. One of the primary attributes of MT is to improve performance efficiency of a core, which it does by increasing the pipeline utilization of that core when multiple processes and/or threads are running. So for systems where silicon real estate is at a premium, choosing a multithreaded core can be a great way to boost system performance.

When multi-core and multi-threading systems were first introduced into the marketplace, most existing software was not optimized to make good use of these technologies. Today this is changing. Android is a complex software platform, and a perfect example of a high volume consumer platform that is quickly evolving and being optimized for optimal user experience in a Web-connected world.

At MIPS we are very pleased with the benchmark results because they demonstrate that our MC and MT technologies deliver much higher performance than the standard hardware used only a few years ago, and make a significant impact to end users of many connected devices from smart phones and tablets to connected DTVs.

http://developer.mips.com/login/dalvik-turbo/