Have you ever tried to type a movie title by using a TV remote control? Both Netflix and Amazon Video provide an interface (a virtual keyboard) that enables you to use the four arrow keys of a standard remote control to type letters. The letters are arranged in a regular grid and to navigate from one letter to another can require you to press the arrow keys many times. Fortunately, the software displays partial matches as you choose each letter, so you rarely need to type the entire title. Nevertheless, I was curious: Which interface requires the fewest number of key presses (on average) to type a movie title by using only the arrow keys?
The layout of the navigation screens
The following images show the layout of the navigation screen for Netflix and for Amazon Video.
The Netflix grid has 26 letters and 10 numbers arranged in a 7 x 6 grid. The letters are arranged in alphabetical order. The first row contains large keys for the Space character (shown in a light yellow color) and the Backspace key (dark yellow). Each of those keys occupies three columns of the grid, which means that you can get to the Space key by pressing Up Arrow from the A, B, or C key. When you first arrive at the Netflix navigation screen, the cursor is positioned on the A key (shown in pink).
The letters in the Amazon Video grid are arranged in a 3 x 11 grid according to the standard QWERTY keyboard layout. When you first arrive at the navigation screen, the cursor is positioned on the Q key. The Space character can be typed by using a large key in the last row (shown in a light yellow color) that spans columns 8, 9, and 10. The Space character can be accessed from the M key (press Right Arrow) or from the K, L, or '123' keys on the second row. The '123' key (shown in green) navigates to a second keyboard that contains numbers and punctuation. The numbers are arranged in a 1 x 11 grid. When you arrive at that keyboard, the cursor is on the 'ABC' key, which takes you back to the keyboard that contains letters. (Note: The real navigation screen places the '123' key under the 0 key. However, the configuration in the image is equivalent because in each case you must press one arrow key to get to the 0 (zero) key.) For simplicity, this article ignores punctuation in movie titles.
Which navigation interface is more efficient?
I recently wrote a mathematical discussion about navigating through grids by moving only Up, Down, Left, and Right. The article shows that nearly square grids are more efficient than short and wide grids, assuming that the letters that you type are chosen at random. A 7 x 6 grid requires an average of 4.23 key presses per character whereas a 4 x 11 grid requires an average of 4.89 key presses per character. Although the difference might not seem very big, the average length of a movie title is about 15 characters (including spaces). For a 15-character title, the mathematics suggests that using the Netflix interface requires about 10 fewer key presses (on average) than the Amazon Video interface.
If you wonder why I did not include the Hulu interface in this comparison, it is because the Hulu "keyboard" is a 1 x 28 grid that contains all letters and the space and backspace keys. Theory predicts an average of 9.32 key presses per character, which is almost twice as many key presses as for the Netflix interface.
Comparing two interfaces for selecting movie titles
You might wonder how well this theoretical model matches reality. Movie titles are not a jumble of random letters! How do the Netflix and Amazon Video interfaces compare when they are used to type actual movie titles?
To test this question, I downloaded the titles of 1,000 highly-rated movies. I wrote a SAS program that calculates the number of the arrow keys that are needed to type each movie title for each interface. This section summarizes the results.
The expression "press the arrow key," is a bit long, so I will abbreviate it as "keypress" (one word). The "number of times that you need to press the arrow keys to specify a movie title" is similarly condensed to "the number of keypresses."
For these 1,000 movie titles, the Netflix interface requires an average of 50.9 keypresses per title or 3.32 keypresses per character. the Amazon Video interface requires an average of 61.4 keypresses per title or 4.01 keypresses per character. Thus, on average, the Netflix interface requires 10.56 fewer keypresses per title, which closely agrees with the mathematical prediction that consider only the shape of the keyboard interface. A paired t test indicates that the difference between the means is statistically significant. The difference between medians is similar: 45 for the Netflix interface and 56 for the Amazon interface.
The following comparative histogram (click to enlarge) shows the distribution of the number of keypresses for each of the 1,000 movie titles for each interface. The upper histogram shows that most titles require between 30 and 80 keypresses in the Amazon interface, with a few requiring more than 140 keypresses. In contrast, the lower histogram indicates that most titles require between 20 and 60 keypresses in the Netflix interface; relatively fewer titles require more than 140 keypresses.
You can also use a scatter plot to compare the number of keypresses that are required for each interface. Each marker in the following scatter plot shows the number of keypresses for a title in the Amazon interface (horizontal axis) and the Netflix interface (vertical axis). Markers that are below and to the right of the diagonal (45-degree) line are titles for which the Netflix interface requires fewer keypresses. Markers that are above and to the left of the diagonal line are titles for which the Amazon interface is more efficient. You can see that most markers are below the diagonal line. In fact, 804 titles require fewer keypresses in the Netflix interface, only 177 favor the Amazon interface, and 19 require the same number of keypresses in both interfaces. Clearly, the Netflix layout of the virtual keyboard is more efficient for specifying movie titles.
Movie titles that require the most and the fewest key presses
The scatter plot and histograms reveal that there are a few movies whose titles require many keypresses. Here is a list of the 10 titles that require the most keypresses when using the Amazon interface:
Most of the titles are long. However, one (4 Months, 3 Weeks and 2 Days) is not overly long but instead requires shifting back and forth between the two keyboards in the Amazon interface. That results in a large number of keypresses in the Amazon interface (178) and a large difference between the keypresses required by each interface. In fact, the absolute difference for that title (75) is the largest difference among the 1,000 titles.
You can also look at the movie titles that require few keypresses. The following table shows titles that require fewer than 10 keypresses in either interface. The titles that require the fewest keypresses in the Netflix interface are M, Moon, PK, and Up. The titles that require the fewest keypresses in the Amazon interface are Saw, M, Creed, and Up. It is interesting that Saw, which has three letters, requires fewer keypresses than M, which has one letter. That is because the S, A, and W letters are all located in the upper left of the QWERTY keyboard whereas the letter M is in the lower left corner of the keyboard. (Recall that the cursor begins on the Q letter in the upper left corner.).
Summary
In summary, both Netflix and Amazon Video provide an interface that enables customers to select movie titles by using the four arrow keys on a TV remote control. The Netflix interface is a 7 x 6 grid of letters; the Amazon interface is a 3 x 11 QWERTY keyboard and a separate keyboard for numbers. In practice, both interfaces display partial matches and you only need to type a few characters. However, it is interesting to statistically compare the interfaces in terms of efficiency. For a set of 1,000 movie titles, the Netflix interface requires, on average, 10.6 fewer keypresses than the Amazon interface to completely type the titles. This article also lists the movie titles that require the most and the fewest number of key presses.
If you would like to duplicate or extend this analysis, you can download the SAS program that contains the data.
12 Comments
This is excellent, thanks. I love it when I read something like this and halfway through think "sure, for a random grid of different shapes that makes sense, but surely QWERTY is better for real words, you need data". And then I read the next sentence, where you pull in real data. In terms of neat data things that curious statisticians do, this post ranks high on my list, likely second only to your spiral of splatter post. https://blogs.sas.com/content/iml/2015/06/11/spiral-of-splatter.html
Great article! I think the Netflix interface allows you to wrap around the edges, saving a lot of keypresses when trying to jump from the end of a row to the beginning of it. I'm not sure about the Amazon interface. How would the results change with that modification?
Really? I'll have to try it. The result would be a "torus topology" in which the average number of keypresses would decrease. That is, if you can wrap around, the "torus distance" between two letters is always less than or equal to the "flat distance" that I computed.
At least on my Roku, the Netflix keyboard does not support wrap-around. When you click to the right from the right edge, it bumps you to the top-level menu (Search, Movies, TV, Account). And on the left edge, clicking left doesn't get you anywhere.
Thanks for checking, Chris. The interface on my Samsung Smart TV also does not support wrap-around.
Isn't there a better interface like some type of keyboard that you can hook to the TV so that you can type in the titles directly?
Yes, there are better interfaces. A colleague mentioned that his TV has voice recognition! No typing required!
I also think the QWERTY keyboard is slower in this scenario, at least for me. Letter location on a keyboard is seamless when going from my brain to my fingers on a keyboard, but it takes me longer to find the letters when it's a virtual keyboard like this and I'm clicking through. If I have a physical keyboard to use for the search the QWERTY search would speed up significantly, but that's not what we are examining here :-)
Great article! As you pointed out, your calculations are based on typing the entire title, which rarely happens. The "title prediction" capabilities are important and might also differ between the interfaces, but we probably don't have insight into that.
When I search for a title on either interface, I'm lazy and want to click the fewest letters possible. I perform a mental calculation to guess what I think will be the "most rare" word in the title I'm searching. (I'm probably the only person in my house who does this though...) For example "bomb" for Dr. Strangelove, as "strange" is more common in movie/show titles (I think). And "hallows" for the Harry Potter films, as I could use "potter" but then I'd have to click through at least 8 "Harry Potter" film matches. And the displayed results are based on more than just pure matches -- there is some prediction applied based on popularity and perhaps your known preferences.
The glorious Apple remote wins. You press 1 button and talk. Ta da! :)
A side effect of the design goal of the QWERTY keyboard is that it is non-optimal for a cursor interface. It comes from mechanical typewriters with a semi-circle of levers. Pressing two levers close to each other causes them to collide and jam. The distribution of letters in QWERTY is designed to make this less likely by placing letters which follow each other in words far apart. That means they are many cursor keypresses apart. By design this should be less efficient (for a cursor) than a random arrangement of letters. Care to test?
While preparing this post, I tested a "custom keyboard" that was very similar in spirit to your suggestion. I took a 7x6 grid and put the most commonly used letters (in English) in the middle: E, T, A, O, I, etc. Then spiraling out from the middle I put medium-use letters, The least-used letters (X, J, Q, Z) I put on the outside of the grid. I interspersed three space keys: one just to the right of center, one in the upper left, and one in the lower left. The idea was to reduce the distance that the cursor would have to travel to get from one letter to another. It worked! My keyboard reduced the average number of keypresses by 5, compared to the Netflix interface!
The problem, of course, is that such an "efficient" keyboard would be an aggravation to users who would constantly be asking, "Where's the R?" and " Where's the V?" Using a standard layout might be less efficient, but at least everyone knows where to look for the next letter they want to type.