Analyzing the Design of Websites: Apple, Microsoft & 37 Signals

Overview
Design is subjective, and design is an art, but can we automatically calculate metrics that indicate how 'good' a webpage's design is?
Author
Dan Zambonini
Date

Analyzing the design of websites

As a precursor to a design checklist we will be publishing next week, we thought it would be interesting to try some different experiments on the home pages of popular websites, to see if it was possible to automatically (programmatically) calculate whether a design is good or not. We chose Apple, Microsoft, 37 Signals, YouTube and Myspace. Analysis was carried out on the afternoon of Wednesday 10 June 2009.

Note that all images (except the typography hierarchies in the final section) are clickable for larger versions of each image.

Balance

Although not a hard-and-fast rule, a good design should typically (unless the idea is to cause tension through imbalance) take the balance of the composition into consideration, where smaller dark areas can be balanced out by larger areas of lighter areas.

We brainstormed some mathematical approaches to this; eventually Ben and Carey opted for a vector-based center of mass algorithm, using a weighted luminosity measure of each pixel to determine the weight.

In the results below, the green dot represents the absolute middle of each screen, with the red dot identifying our calculated center of mass.

 
Apple centerCentre of Microsoft home page

 

 
37 signals center
Youtube center
myspace center

 

 

As can be seen, most home pages are fairly well balanced, with the two largest discrepancies (relative to the size of the page) being YouTube and Myspace. Coincidentally (or perhaps not), these could be considered the least professional of the designs, so this does begin to support the assertion that some aspects of good design can be calculated automatically.

Colors

The colors used in a design should be harmonious, or at the very least, not clash (again, unless that is the specific intention). The pie charts below show the distribution of colors on each home page.

Note: The chart on the left includes the background color of each page, the chart on the right does not include the background color.

Apple

Apple colours with Apple colours with

Microsoft

microsoft colours withmicrosoft colours with

37 Signals

37 signals colours withoutthumb-8358ab0a7d2b25780298387006f8bdb3

YouTube

youtube colours withyoutube colours without 

Myspace

myspace colours withmyspace colours with

We have not yet used color theory to analyze this data (e.g. to test for complementary colors), but even the rudimentary charts above highlight the simplicity of the Apple design.

Typography

The text on a design should ideally follow a pre-defined typographical hierarchy, where every step in the hierarchy should be clearly differentiated: by size (at least one pixel), weight, style (e.g. italics) and/or font family.

We created a Javascript application that loops through all text elements on the page, and stores the typographical information about each. The variations in typography on each home page is shown below, with the percentages representing how much text (by character count) uses each style.

Apple

apple typography
apple typo

Microsoft

microsoft typography
microsoft typo

37 Signals

37 signals typography
37 signals typo

YouTube

Youtube typography
Youtube typo

Myspace

Myspace typography
Myspace typo

As can be seen, the Apple typography is as close to perfect as you could image; at least 1 pixel and/or weight difference between a very restricted number of styles. Microsoft almost gets it right, but with the largest two styles having less than a pixel between them, the difference becomes a little blurred (these differences can be caused by inaccurate CSS inheritance rather than purposefully designed, but the net result is the same).

37 Signals, although opting for a much large set of styles, is actually extremely systematic about how the family, weight, style and size cascade down the styles, with a clear hierarchy established.

YouTube also performs well in this test. Unfortunately, possibly as expected, Myspace fails badly, not only with a lack of clear hierarchy, but also by committing the typographical sin of mixing similar typefaces (Arial and Verdana).

Summary

These prototype experiments were designed to assess the possibility that a tool could partially critique a design, based on long-established rules of good design. The results are promising, showing at least some relationship between what many people would consider the ‘best’ designs and the output of these algorithms.

This is just the beginning, though. What other tests could be run? How else could this data be used and visualized? Would tests like these, if popular, cause a fear of creativity? Let us know your thoughts in the comments.

Comments

Add a comment

  • 11 Jun 2009 20:37

    Those center of mass dots should be relative to various screen sizes, since a viewer would never see the page the way our analysis see it.

    You would have to animate the dot location as the luminosity changes. It would not just be a matter of moving straight down.

    Your anlaysis hints towards many more. Great!

  • 12 Jun 2009 02:01

    Very interesting data. I especially love the data about the use of fonts and their density within the design. I love how 37 Signals has always been able to give plenty of information to make an informed decision as to the next course of action but never in a way that feels overwhelming. Great data and analysis.

  • 12 Jun 2009 08:55 Box UK Staff

    @David Yeah, we did consider also doing some dots relative to the various common 'viewports'. Luckily, most of the sites we analysed (except 37 Signals) were not too tall, so the 'viewport' typically contained the entire design. But that would be an interesting experiment.

    @Gregg Thanks for the comments!

  • 12 Jun 2009 09:13

    Epic post!

    Really good points here. There is something very intangible about a good design though, don't you think? All the balance and measures can be in place and it still doesn't look right.

    Gets me thinking. Thanks guys.

    Ramsay

  • 12 Jun 2009 09:14

    Why don't you guys release these tools so we can use them for personal use.

  • 12 Jun 2009 09:29 Box UK Staff

    @Ramsay Thanks! Yes, there's definitely an intangible thing about good design, which is why it's such an 'artform' I suppose (speaking as someone who has studied the 'rules' but couldn't create a good-looking design if my life depended on it!).

    I think I worded a question wrong in the post; rather than using rules/tools like these to assess 'good' design, I think they could instead be used to identify problems, i.e. they'll never be able to tell us what is 'great', but they may help us resolve common issues.

    @Killthedinosaur Yes, we might. This data was collected using some really rough prototype code, and we wanted to get the results out ASAP to see if there was an interest in it. If there is (which there does seem to be), we will definitely think about making them a bit more robust, more comprehensive, and then making them available to people.

  • 12 Jun 2009 09:33 Box UK Staff

    @David I found the search results test I ran particularly interesting in terms of viewport. On my 1200px vertical resolution monitor, the CoM for Google's search results was on the title of the fourth result. I found my eyes naturally drawn to the third result and scanning up, which I put down to the browser chrome, something else it would be good to allow/adjust for. Maybe the race for the #1 slot isn't all it's cracked up to be.

  • 12 Jun 2009 09:33

    @Dan Zambonini

    "I think I worded a question wrong in the post; rather than using rules/tools like these to assess 'good' design, I think they could instead be used to identify problems, i.e. they'll never be able to tell us what is 'great', but they may help us resolve common issues."

    Well said. And so true. You just put an intangible factor into words. Props.

    Ramsay

  • 12 Jun 2009 13:16

    excellent

  • 12 Jun 2009 15:04

    Cannot figure out how you calculated the center of mass. More details please?

  • 12 Jun 2009 15:06

    Very nice analysis!

    However, it's important to note that Apple (for example) keeps their palette consistent after you leave their home page. MySpace, on the other hand, has allowed their user pages to be customized, most especially in terms of color. While the Apple site remains a pleasing and consistent experience no matter where you go on their site, MySpace is a nightmare of arbitrary color choices based on the user.

    Related, I bet you'd find the color palette on the MySpace home page to be very similar to the Facebook home page. However, once you click through to user pages, the similarities end. I have to think this is one reason some people find Facebook preferable to MySpace.

  • 29 Jun 2009 20:03

    wow! nice analysis! thanks!

  • 01 Jul 2009 06:35

    After read this article,Just want to say that

    "its really informative,also some valuable comments"

    Read the replay of "David"(first from top),agree with him.

    Thanks for sharing…

  • 16 Jul 2009 22:04

    Dan, do you plan to run further tests? Would be very interesting to see this experiment for a larger list of websites with known good and bad designs and then have a user rating system to see what percentage of people agree with the test results.

  • 17 Jul 2009 08:15 Box UK Staff

    @James I'm not sure yet what our plans are. Ideally we'd release these tools as some kind of web app, so that others can run them and we could crowd-source the good/bad/rating.

    I'm being a bit tentative as I'm not sure there are enough tests to create a web app yet; if we (anyone?) could think of some more interesting 'design' tests to include, it might be worth the effort.

  • 21 Jul 2009 04:03

    ^^ Agree with you Helvetica is very beautiful font.

  • 21 Jul 2009 23:28

    Thanks for responding Dan. I subscribed to your RSS feed so that I can get future updates.

    Perhaps you could write a new blog post requesting test ideas and include a poll to see if people were interested or not.

  • 22 Jul 2009 06:34

    I think VAG Round is a beautiful font.

    Great blog post anyway.

  • 17 Aug 2009 03:17

    Apple is definitely the benchmark to what a website design should be. Great comparison!

  • 17 Aug 2009 16:55

    Microsoft in the coming years will give Apple a run for their money on the design front. Whether that transcends to the online world is a different story…knowing MS I doubt it.

  • 27 Aug 2009 08:06

    No two eyes are the same. So someone like lighter color and background and others might like differently. So this is not a much needed analysis.

  • 03 Sep 2009 22:09

    I am really hoping you do some further testing on other sites as well. This helped me a lot with the 3 sites I have created so far.

    I am new to the designing game by the way. I have enjoyed learning the new things though!

  • 04 Sep 2009 20:17

    The color analysis shows that the mild colors are the ones most liked by the people.

  • 06 Sep 2009 15:42

    yeah "Killthedinosaur" your´re right, they should release this tools for personal use!

    @Dan Zambonini: YES – there is an interest, i hope you will release it soon =) really nice analysis!

  • 18 Sep 2009 08:23

    Colors can be used as an expression as well as language. The background color of website, the color of header, the color of text, headlines and sub-headlines etc. can all have a psychological impact visitors.

  • 04 Apr 2010 23:40

    I particularly enjoyed the way the typography analysis is presented …

    I would love to see more of that …

  • 05 Apr 2010 01:02

    This was a great breakdown of the sites, I find it interesting that of all the sites, Myspace's center of mass was quite near a call to action…funny

Add your comment

If provided, we will link to this from your name