Development

Apple TV: How to Design and Develop An App for the Big Screen

This is part two of our Apple TV series. In the first post, we explored whether you should build an app for this platform. Now we’ll explain how to design and develop an Apple TV app.

As we explained in a previous post, the Apple TV has been around for quite some time. In fact, it was introduced back in 2007, when the very first iPhone was released.

But, unlike the iPhone, the Apple TV had a back-seat status of being more of an Apple hobby. And despite numerous incremental improvements over the years, this little black box had still not received much attention from Apple.

This all changed, however, in fall 2015. Amidst the excitement of Apple’s announcements of the new iPhone 6S and iPad Pro in September, the company finally put Apple TV in the spotlight.

The device has been reborn with an array of hardware enhancements, including a fast A8 processor. And the new Apple TV will be controlled through a cleaner interface with deep searching, a gesture-friendly remote control, and the ability to hook up gaming controllers — all powered by Siri.

But the most notable improvement is the all-new tvOS framework, which opens up the platform for the first time to third-party developersNo longer is the Apple TV a second class citizen in Apple’s app-driven ecosystem.

In this article, we’ll explain the process of developing and designing an Apple TV app, starting with the different approaches developers can take when building for tvOS, followed by some design considerations.

Apple TV Apps: TVML vs Custom

Being a completely new platform for third-party developers, Apple has provided two different approaches for creating tvOS apps.

The first utilizes Apple’s new Television Markup Language (TVML), a JavaScript-centric framework that is well suited to video consumption apps like Hulu or Ted TV.

The second option is a custom app, which essentially is an iOS app that conforms to various tvOS SDK features and requires extensive use of Swift or Objective-C.

Next we will explain each of these approaches in greater detail, including their benefits and limitations.

TVML

TVML acts as a bridge or gateway between native iOS code and the UI for Apple TV.

If you decide to go with this approach, you would build your app through a series of JavaScript pages, without having to rely too heavily on Swift or Objective-C. Instead, you would utilize TVMLKit, consisting of TVML and TVJS.

Each TVML template represents a single page and is made up of a series of simple and compound elements, which contain content information or images that are displayed on screen.

The sequence of orchestrating and displaying TVML template pages is managed via a TVJS JavaScript file called by the tvOS binary app, which handles user input in addition to loading various pages.

Apple TV App: TVML

In the client-server paradigm, your JavaScript file orchestrates loading assets (images, videos, text content) from the server and overlays them into the UI elements, which sit within the TVML files.

Apple TV App: TVMLKit

TVML makes use of traditional web networking technologies such as HTTPS, XMLHttpRequest, DOM, and other JavaScript libraries to acquire content to display TVML pages.

Templates

As mentioned in part one of this series, Apple provides 18 TVML templates. The list includes an alertTemplate, catalogTemplateproductTemplate, ratingTemplate, searchTemplate, and more.

Each of these templates serves a specific role, while allowing developers to add their own simple and compound elements to customize each screen.

(We will discuss various UX considerations when designing screens in the second half of this article.)

Apple TV App: TVML Templates

A template in code/script format would resemble something like the alertTemplate below:

<document>

   <alertTemplate>

      <title>Update Available</title>

      <description>Get the latest tvOS version</description>

      <button>

         <text>Update Now</text>

      </button>

      <button>

         <text>Cancel</text>

      </button>

   </alertTemplate>

</document>

You can check out the complete list of TVML templates that Apple provides here.

We expect to see third-party templates gradually become available, and Apple will also most likely continue adding new templates in the future.

Simple and Compound Elements

TVML pages contain both simple elements and compound elements, which together make up the template.

Simple Elements

Simple elements represent a singular item, be it a badge, text, image, etc.

Apple TV App: Simple Elements

Apple TV App: Simple Elements Example

Some of the simple display elements that Apple has made available so far include:

Badge a link to a small image that is usually displayed on top of another image (similar to the badges we see on iOS app icons)

DecorationImage — an image that is usually displayed inside of another image

FullscreenImg — displays the image over an entire element

HeroImg — a hero image, usually displayed inside of a banner

Img — a standard image, usually a preview image of a product

RatingBadge — a different type of badge that is displayed inside a review card, which is a compound element

SeasonBadge — an image to indicate the season count, which is also usually displayed inside a review card

Apple also provides simple multimedia elements for streaming audio and video stored on a server, as well as an array of simple text elements for creating headings, monograms, ordinal elements, and so forth.

Compound Elements

Compound elements are higher-level groupings of simple elements, consisting of compound background elements that display images prominently in the back of elements, as well as play audio in the background of a template page.

Compound banner and header elements contain buttons, heroImg, row, stack, and other types of simple elements. There is also a footer compound element and itemBanner, which displays information along the top of the element and consists of heroImg, row, and Img.

Compound information elements provide user-specific group elements such as activity indicators and placeholders, as well as ratingCard and relatedContent elements. Information elements could also include menuItem for labeling an item inside of a menuBar and reviewCard for displaying abbreviated review information.

Compound layout elements help organize and display elements they contain as a shelf, collectionList, grid, imgDeck, infoList, list, menuBar, overlay, section, and more.

Compound lockup elements are a combination of other elements treated as a single element, usually presented in grouped elements like a shelf.

For instance, you might incorporate a badgeLockup, which is a combination of badge and title, or listItemLockup, which is a list item consisting of various elements, such as decorationImage, Img, ordinal, relatedContent, and more.

Additional Notes about Elements

In addition to simple and compound elements and working with predefined templates, Apple also provides guidance on how you can style elements with colors, attributes, font weights and sizes, and margins.

There are tons of elements and templates available, and it can be overwhelming.

That’s why it’s a good idea for you to download the TVML Catalog sample app, which will allow you to play with all the different options and find the right fit and feel to meet your app needs.

Benefits and Limitations

You’ve now had a brief introduction to TVMLKit, which is a completely new framework for creating tvOS apps.

The most striking feature is how simple it is to get started, with a rich tapestry of simple and compound elements and the ability to promote conformance through templates. For developers already familiar with JavaScript and Node.js, this is a perfect framework to create a rich, compelling client-server app.

However, there are some distinct limitations to TVML apps vs. custom apps.

The most notable limitation is that TVML is only suited to simple, lightweight apps that support accessing a list of videos or audio on a server, but nothing too complex or fancy like creating games or iOS-style apps.

If you want to create a more iOS-style app, such as an AirBnB-like app or 3D game, you will need to build a custom Apple TV app. We will discuss that approach in greater detail in the next section.

Custom Apps

In the context of Apple TV, custom apps are basically just iOS apps that are extended to conform to the tvOS SDK.

In most cases, seasoned Swift and Objective-C iOS developers are already accustomed to most of the libraries that are part of tvOS, including AVKit, AVFoundation, UIKit, CoreGraphics, CloudKit, and so forth.

For iOS developers, this process should be rather straightforward. However, developers do need to be mindful of the differences in designing and developing for tvOS as opposed to iOS.

The Focus Engine

Apple TV developers don’t have the luxury of having a screen where they can use touch gestures. Instead they must rely on a remote that uses the Focus Engine to focus on a single item at any given time, using remote swipe gestures.

Of course, users can also use Siri to control their element interactions on the Apple TV or plug in a bluetooth gaming controller for playing games.  

This means developers need to design their storyboard and elements in a manner that is conducive to users navigating with singular focus, clearly and consistently, in a layout that is either grid-like or stacked logically.

(We will discuss the UX aspects further in the next section of this article.)

As Apple explains in its documentation:

“Focus refers to the effect onscreen of external, indirect user input from a remote or another input device. In a focus-based interaction model, a single view onscreen is considered focused, and the user can move focus to other views by navigating through different UI items onscreen, which triggers a focus update.

“The focused view is used as the target of any user actions. For example, if an onscreen button is focused, the button’s action is triggered when a select event is sent from a remote.”

Conforming Existing iOS Elements to tvOS

Most of the existing standard Apple frameworks, such as UIKit, will automatically support the Focus Engine, meaning they inherently support the behavior of being controlled by an Apple TV remote for focus, without any additional programming needed.

It’s just a matter of understanding which UIKit elements can receive focus and which can’t. For example, UIButtons can receive focus, while UILabels obviously cannot.

The standard UITableView and UICollectionView elements allow developers to use the standard delegate objects they are familiar with in order to receive UIFocusEnvironment behavior events.

You can also obtain focus updates through delegates programmatically, but you cannot control or set focus explicitly and programmatically, such as to move focus to a certain element.

UITextField, UITableView, UICollectionView, and UISearchBar also have inherit focus support, which developers should be able to understand by referring to Apple’s reference documentation.

Gestures and touches are observed synonymously to iOS apps, through UIGestureRecognizer delegate methods and the UIResponder class, with the added methods to respond to buttons pressed via the Siri remote, as well as responding to pan and swipe gestures via the remote glass pane.

Custom Support for Focus Engine

For custom UI elements that don’t have inherent focus support, you will need to implement your own custom behavior for focusing. 

This would entail receiving delegate callbacks to update the app state based on the focus change. Alternatively, the developer can create a new kind of custom focus interaction interface design.

Through UIFocusEnvironment, you can dictate how each UI element reacts to changes in focus by having your UI control conform to the UIFocusEnvironment Protocol.

This will allow you to gain access to the custom focus behaviors, including where the focus is allowed to move and how to respond to the focus. You also gain access to focus updates as they occur, through a callback delegate method called didUpdateFocusInContext:withAnimationCoordinator:.

Benefits and Limitations

The benefits of custom apps pick up where TVML apps left off, as you can use this approach to create truly robust iOS apps for the big screen.

If you are a seasoned Swift or Objective-C developer, you will feel very much at home. And, by conforming to certain design principles (which we will discuss shortly), as well as certain protocols, your standard frameworks will work seamlessly on tvOS in most cases.

Of course, not every framework has been ported to tvOS, which is why developers need to consult Apple’s documentation in regard to their specific use cases.

If you already have an iOS app, it makes sense to leverage existing iOS code when making the transition to a tvOS app. Also, if your business needs require creating a game or iOS-style app, it makes sense to develop a custom app for Apple TV.

However, if you are only going to showcase video or audio content and/or your available talent pool mostly consists of web developers, it would make sense to take the TVML app approach instead.

Now that you are well-versed on the two approaches for building an Apple TV app, we will focus the remainder of this article on design and UX considerations for tvOS apps.

Design and UX Considerations

We’ve just covered development considerations for building an Apple TV app. Now we will look at the design and UX considerations that go into creating such apps.

Unlike developing for the iPhone and iPad — or even the Apple Watch, which provides a more intimate user experience — designing for the Apple TV is the exact opposite.

Unlike other Apple devices, where Apple provides the complete hardware, the Apple TV is a box that connects to third-party televisions, which each have different screen sizes and resolutions.

tvOS apps are meant to be shared in a room with others, so the user experience is less private but potentially more engaging (and certainly apps can take advantage of greater visual real estate), in comparison to other Apple devices.

With that in mind, there are quite a few considerations worth noting as a designer, when working on your tvOS app.

Focus and Control

Unique to the Apple TV, users control and interact with the platform through their Apple TV remote.

With the new Siri-powered remote, users are able to use gestures to navigate the home screen icons in a grid, as well as navigate specific apps horizontally and vertically.

Apple TV App: Focus and Control

Emphasizing Focus

As obvious as this might sound, when you go through each UI element within your app, you need to emphasize focus.

The other iOS platforms don’t show a cursor or any focused element because you don’t have a mouse, but rather use your fingers. In contrast, on the Apple TV, users need to keep track of their active or current selection.

Apple has decided the best UX approach is their Focus Engine, which we discussed above.

An icon or image is in focus when the current item is all of the following:

  • highlighted;
  • elevated in depth; and
  • sways in response to a light circular gesture on the touch interface of the remote.

Apple TV App: Emphasizing Focus

As Apple explains in its documentation:

“Parallax is a subtle visual effect used throughout the system to convey depth and dynamism when an element is in focus. Through image layering, transparency, scaling, and motion, parallax produces a 3D effect with a sense of realism and vitality.”

It’s important that you make the selected element stand out, as shown in the illustration above, by not only having it elevated but also including a caption beneath, thereby adding extra bits of information that separates the active element from the inactive elements.

As we mentioned earlier, adapting elements to conform to the Focus Engine of Apple TV is generally fairly straightforward.

When you use the common UIKit elements, such as UIButton and grid-layout items like UICollectionViews, you inherit those behaviors for free. But if you add any custom UI elements, then you will need to make an extra effort to conform to Apple’s guidelines.

You are able to add additional custom animations and effects programmatically, but doing any unnecessary animation is discouraged, as a general UX principle. Instead, be sure to follow the points mentioned above and ensure the images are crisp and custom animations are subtle and fluid.

Navigation within your app should make logical sense and be intuitive. Most importantly, do not display a cursor. This is a big no no and goes against the Focus Engine philosophy of the Apple TV.

Icons and Images

As designers probably already know, when working with icons and images on iOS, you have to cater for @2x and @3x. But with the Apple TV, you will be working with @1x image assets.

You must ensure your icons are vivid and representative of your app and brand and follow the general icon design principles you would use for iOS (i.e don’t use words or make the design feel cluttered).

Working with parallax icons, you need to create a layering of your images in order to be able to create a sense of depth and vitality. Visually, this will enable users to see your icons pop out and tilt as they focus on them.

This is not only a suggestion but a requirement from Apple. So it’s important you consult Apple’s guidelines for tvOS icons and images.

You should also take a look at some of the resources that Apple provides, including tools and PSD templates that will make the design process easier.

The Top Shelf

The top shelf on the Apple TV home screen is a great place to showcase your app in a graphically and contextually rich manner.

This is the space that will encourage existing and new users to check out your app, whether it’s the next episode of a series (in the case of a video or audio app) or the last house they were inspecting (for a real estate app like Zillow).

Apple TV App: The Top Shelf

At a minimum, you are required to provide a static top shelf image when submitting your app. However, you are encouraged to go beyond what’s asked and provide dynamic content that is contextually-refined.

Contextual Content

The idea behind providing contextual and dynamic content in the top shelf is that it presents a great opportunity to remind users of what they saw the last time they were in your app.

For example, you could have existing users jump to the last saved position in a game they were playing or go directly to the next episode of a show they are watching. 

You can even use the Apple TV remote’s Play button to allow users to immediately start media playback where they last left off or to enter the game at that position. 

As Apple explains in its documentation:

“People are familiar with apps they put in the top row of the Apple TV Home screen and use them often. Consider personalizing the experience by showing targeted recommendations in the top shelf, allowing the user to resume media playback, or letting them jump right back into active gameplay.”

As referenced in the quote above, the top shelf can also be a way of displaying personalized content. For example, if you have a Netflix-style app, you could show the user’s wish list or top rated movies. 

Apple TV App: Top Shelf Contextual Content

Or, for a game app, you could display scores from the user and their friends —something captivating that will likely boost user retention.

From a designer’s perspective, be sure to provide enough content to fill up an entire row, rather than leaving gaps. (In the previously referenced documentation, Apple provides guidelines on the ratios you can work with to best ensure conformity.)

Navigating the UI

The tab bar takes up the top 140px of the content area and provides a great way for users to navigate to other parts of your app.

Synonymous with tab bars on iOS, you are able to add badges to indicate activity on specific tabs (e.g. number of unread videos).

Apple TV App: Navigating the UI 1

Apple recommends that designers use text instead of icons for tab items, not include too many tabs, and choose appropriate colors and names.

In addition, you can implement tables using iOS UITableViews in order to provide a scrolling experience that displays large sets of data.

As shown in the illustration below, you can also use split views, which makes greater use of large screen real estate.

Apple TV App: Navigating the UI 2

Collections is another form of visual structuring of elements, which is well-suited to elements that are not as sequentially-structured. Unlike table views, they can be composed of varying-sized elements.

Apple TV App: Navigating the UI 3

In their documentation, Apple goes into greater detail about working with other elements, such as page controls, buttons, alerts, text fields (and search text fields). These resources are definitely worth exploring and understanding.

Concluding Thoughts

Last fall, Apple introduced an exciting new Apple TV iteration that has finally made it a fully-fledged third-party developer’s platform. The tvOS framework builds upon the ecosystems established in other Apple platforms, such as iOS and Apple Watch.

In this article, we introduced you to the tvOS framework. We also explained the two approaches you can take in regard to development, including TVML and extending existing iOS apps via Swift or Objective-C, as well as the benefits and drawbacks of each approach.

In addition, we’ve explored many of the design principles involved in creating a tvOS app to ensure your apps are engaging and conform to Apple’s UX guidelines.

Being a completely new platform, we are confident that we’ll start to see new genres of Apple TV apps emerge, based on new use cases and design trends.

This is an exciting and pioneering area for designers and developers and a platform that Apple will no doubt double-down on, in terms of their efforts and resources.

Ready to start your project?

Learn how ThinkApps can get your product launched faster, better, and with more value than you knew was possible.

@ThinkApps on Twitter