While writing the article for David Walsh’s blog I had posted a version of the article on Medium to have @cramforce, the lead on Google AMP, answer the questions brought up. Here is a link to the article where he posted those answers: bit.ly/1VCLBr0
As developers we know that site speed is extremely important. In recent years, the developer community has created and supported new browser specifications and techniques to enhance site speed. So when I heard that Google was launching the Accelerated Mobile Pages (AMP) project I got really excited and wanted to immediately begin using the standards Google set to make sure that the sites at Aristotle Interactive were performing the best they could.
The developers, designers, and project managers at Aristotle have worked very hard over the last year to create a site speed specification document and performance budget to ensure that every website we build is fast without sacrificing design or functionality. Naturally, we wanted to use AMP to make our clients' sites even faster, but before we started changing code we wanted to do some testing to explore the benefits of using AMP, aside from the SEO boost. The tests raised a question among our team and we wanted to post this question out to the community to start a discussion: Is AMP really needed to increase site speed?
First, let's look at the results of the tests. We used a tool called WebPageTest to check the site speed of every page below. We tested the pages using an iPhone 6 running iOS 9, simulating a 3G connection (1.6 Mbps/768 Kbps 300ms RTT). We tested each page 5 times and have shown the median result in this article. Our performance team at Aristotle uses 3 metrics that have proven to measure site speed and user experience: Time to First Byte, Speed Index (Perceived Performance), and Page Size. We tested pages that were displayed on the AMP demo link that Google provided (http://g.co/amp), AMPs "in the wild" that we found through Google searches, and then a few sites Aristotle has recently created.
Google AMP Demo Results
| Site | Time to First Byte | Speed Index | Page Size | WPT Test |
|---|---|---|---|---|
| NY Times | B | 6.44 secs | 649 kb | link |
| USA Today | B | 7.42 secs | 1,962 kb | link |
| New York Post | A | 6.72 secs | 443 kb | link |
| Mashable | A | 6.43 secs | 593 kb | link |
| The Verge | A | 5.69 secs | 465 kb | link |
"In the Wild" AMP Results
| Site | Time to First Byte | Speed Index | Page Size | WPT Test |
|---|---|---|---|---|
| Buzzfeed | D | 5.53 secs | 3,882 kb | link |
| Wordpress Blog | A | 5.78 secs | 324 kb | link |
| SE Round Table | A | 5.60 secs | 1,007 kb | link |
| BBC | A | 5.58 secs | 389 kb | link |
Aristotle Interactive Results
| Site | Time to First Byte | Speed Index | Page Size | WPT Test |
|---|---|---|---|---|
| Fort Smith * | A | 5.77 secs | 893 kb | link |
| Little Rock * | A | 8.30 secs | 754 kb | link |
| ASPSF | A | 7.24 secs | 411 kb | link |
| Beacon Legal Group * | A | 6.78 secs | 761 kb | link |
* means the redesigned site is nearly launched and a link is not available.
There is a difference between the content on the sites Aristotle has created and the Google AMPs. Our sites are tourism sites while the AMPs are news articles. We expected the tourism sites to be much larger in size and slower in speed considering the amount of images and Javascript used to create interactive experiences but found that wasn't always the case.
Questions
- The biggest question: why are pages such as Buzzfeed and USA Today boosted and preloaded on a user's device, but not pages like Fort Smith and ASPSF (unless they were to use AMP)? AMPs appear to load quickly because there are some assets that are preloaded from the page. Doesn't it make most sense to give an SEO boost pages that inherently meet a standard? We should reward the pages (and publishers) that build site speed into their site as a feature.
- Since there are assets that get preloaded from the Google search results page, is there a cap on how many kilobytes get downloaded from there? Sites like Buzzfeed that appear to abuse visitors' data usage could be a problem if you are preloading assets.
- It appears that some sources are saying that publishers should "have to maintain at least two versions of any article page: The original version of your article page that users will typically see, and the AMP version of that page." Since most SEO sources recommend a responsive site over a separate mobile site and most sites have spent a lot to make that happen, doesn't it seem like taking 2 steps back to have a separate AMP site? This could get complicated when you consider Facebook Instant Articles and other services that want to accomplish the same goal.
- There are a few nice features that AMP offers, such as how images and assets are loaded and displayed, but wouldn't it be better to standardize a way of doing these things rather than doing it a different way for each service?
Services like Google AMP and Facebook Instant Articles appear to be setting themselves up to award publishers that use "proprietary" markup (meaning not standardized markup), rather than awarding publishers who build their sites with performance as the foundation. Is that really the web we want to promote? What are your thoughts on Google AMP and other services like it?
Share your thoughts and get others involved by sharing this post.