SEO-Friendly Infinite Scroll with Progressive Enhancement

By Jim Robinson | SEO
Posted March 10, 2014






SEO-Friendly Infinite Scroll

Use progressive enhancement to ensure your infinite scroll experience works for both users and search engines.

Infinite scroll is a feature that’s gained popularity in recent years because of the slick, modern user experience it can provide. Implementing infinite scrolling does come with a number of challenges, and keeping it SEO-friendly is among the biggest.

This is a fairly technical post, so for the non-technical marketers who follow along, this is a great page to bookmark and share with your developers to get confirmation that your infinite scroll implementation is adhering to best practices for SEO.

What Exactly is Infinite Scroll?

Infinite scroll, as the name implies, allows a reader to scroll down a web page and continually view new content without ever reaching the end. It’s also sometimes referred to as “endless scrolling,” “continuous scrolling,” “lazy loading,” “autopager,” or “unpagination.” Well-executed, it’s an engaging web browsing experience because you never have to click on a link and wait for a new page to load to see the next set of headlines, images, or other content.

The challenge for web developers implementing infinite scroll is that you can’t deliver an endlessly long page of content and still maintain reasonable page load speed. To keep page loads snappy, you’ve got to serve it up in bite-sized pieces.

A typical infinite scroll implementation will render a specific number of items (usually headlines in my world) for the initial page load, but then use JavaScript to asynchronously load in new content once you scroll past a certain point in the page. Ideally the new content will begin loading in the background as you scroll before the asynchronous JavaScript calls are even noticed.

For an example of infinite scroll, check out Mashable.

Mashable's Infinite Scrolling

Infinite scroll implementation on Mashable, which uses a noscript tag

Chances are you’ve been on Facebook, so you’re probably familiar with infinite scroll from a user experience standpoint, even if you didn’t know what it was called. Most social networking sites use infinite scroll these days, along with Google Images, Mashable, and a variety of popular blogs and news sites. Anil Dash even suggested a couple years ago that it’s time for the Internet to stop publishing webpages and start publishing streams. Infinite scroll is a perfect solution for stream experiences.

Why Infinite Scroll is Bad for SEO

Simply put: Google doesn’t crawl JavaScript. Actually we know that Google has increasingly begun to execute JavaScript as they seek to scrape publisher’s valuable content organize the world’s information and make it accessible. For the purposes of this discussion (and your optimized website), we’ll assume that Google and other search engine crawlers don’t crawl JavaScript.

What this means is that the content delivered in the initial page load (and rendered in the HTML source code of your web page) is the only content visible to search engines. The second and subsequent page loads that appear as an endless stream are pulled in with asynchronous JavaScript calls, which means search engines won’t see any of it.

How to Have Your Cake and Eat It Too

Google’s recent guidance on infinite scroll tells us to ensure our content management systems produce a paginated series of pages (component pages) to be presented in tandem with the infinite scroll experience. The image below from Google shows how this looks, with the infinite scroll experience on the left, and the corresponding paginated series of component pages on the right.

Google's Infinite Scroll Guidance

Google’s infinite scroll guidance indicates the use of classic pagination in tandem with the infinite scroll experience.

This is exactly what I’ve been advising my clients to do for a while now. You can read the full post on the Official Google Webmaster Central Blog to get detailed guidance, but here’s a quick recap:

SEO-Friendly Recommendations for Infinite Scroll

  1. Create a series of pages that will serve as the component pages that get fed into the infinite scroll experience. Make sure each page is individually accessible and visible with JavaScript disabled. Include a reasonable number of items on each page so that they load quickly, and ensure there’s no overlap of items as you navigate from one page to the next.
  2. Follow best practices for SEO-friendly URLs and make sure each component page in the series can be accessed individually in a browser.
  3. Set up classic pagination that allows users and search engine crawlers to navigate directly to each component page in the series. Be sure to follow best practices for pagination using rel=next and rel=prev.
    Classic pagination example

    An example of classic pagination

  4. Use the HTML5 History API to implement replaceState/pushState on the infinite scroll page. This will allow you to update the URL in the address bar automatically as the user scrolls down the page. This mimics the URL change that that would occur if the user actually clicked on pagination links. If the user copies the URL out of the address bar after scrolling way down a page, the directly accessed link will go straight to the desired point in the series. It also allows the user to serially back up since your JavaScript code can store pages in the browsing history.

There are a number of resources out there for learning how to implement infinite scroll and leverage the HTML5 History API. This article, Manipulating History for Fun and Proft, is a great starting point. The Twitter Engineering Blog also published an insightful article on how they approach this technique, which they describe as “Hijax + server side rendering.” The idea is that they use AJAX to “hijack” the URL target from an onClick event and, instead of allowing the click to generate a call back to the server, they use it for the AJAX call. The end result just looks like a fast page load to the user.

What’s Missing from Google’s Infinite Scroll Guidance

If you decide to implement infinite scroll following Google’s guidance, you might wonder how, exactly, to provide the infinite scroll experience for users with modern browsers in tandem with classic pagination for users who don’t.  As is often the case, there’s more than one way to skin a cat.

What this boils down to are the concepts of “progressive enhancement” or “graceful degradation.” Progressive enhancement basically means you start with an experience that’s accessible to all browsers, and then progressively enhance that basic experience by adding new layers of functionality for modern browsers. Graceful degradation is very similar, but the nuanced difference is that you start with a full-featured experience, and then make sure stuff either doesn’t break or breaks gracefully in older browsers.

SEO-Friendly Infinite Scroll with Graceful Degradation

Mashable is one of the best known examples of a major publisher who has implemented infinite scroll using the principle of graceful degradation. If you have javascript enabled, you get the infinite scroll experience; if not, you’ll see the contents of a <noscript> tag that contains links to a big set of paginated archives.

Mashable's hidden pagination

A large number of links appears at the bottom of Mashable’s homepage in a noscript tag.

Although this is an example from a major tech-savvy publisher, it’s not the implementation I would recommend for my SEO clients. In Adam Sherk’s post, “SEO Tips for Infinite Scrolling,” he and I discuss in the comments area that a noscript tag would not be the most desirable method for exposing pagination links to non-JavaScript browsers. I’m not sure about Adam’s reasoning for that (maybe he’ll clarify in the comments here), but for me it’s because of the history of abuse by SEO spammers associated with this tag.

SEO-Friendly Infinite Scroll with Progressive Enhancement

My preferred method for implementing search-friendly infinite scroll is based on the progressive enhancement approach. Start with an experience that works without JavaScript enabled and progressively enhance that experience with infinite scroll. To accomplish that, it might be helpful to use a feature detection library, like Modernizr.

One of the most helpful and cleanly-coded examples I found for this approach is the “Infinite AJAX Scroll” jQuery plugin page. If you look at their example here, you’ll see classic pagination when you have JavaScript disabled, and infinite scroll with the URL in the address bar updating as you scroll down when you have JavaScript enabled.

I would love to wrap this up with a great SEO-friendly infinite scroll example from a major publisher, but I honestly couldn’t find one that really nails it. There’s Google Images, of course, but that’s a somewhat different use case. If you have a good example of SEO-friendly infinite scroll on a major news/media publisher site, by all means share it in the comments below. I’m going to have a great one to share in a couple months, so stay tuned.

Update 5/26/16: We’ve helped quite a few clients with these since this was written. Here’s a good example from Rolling Stone:

http://www.rollingstone.com/music/artists/jimi-hendrix

Try viewing that with and without JavaScript enabled and you’ll see the difference.

And here’s a slightly different approach on Us Weekly. This is more of a “finite scroll,” as three additional articles get loaded in via JavaScript if you keep scrolling:

http://www.usmagazine.com/entertainment/news/adele-forgets-the-lyrics-to-her-own-song-reacts-perfectly-watch-w207979

Notice the use of pushState to update the URL on scroll. If the user shares one of those URLs directly, it will generate a full server-side page load of the associated content. In this case, however, there’s no pagination component. We just wanted the “finite scroll” experience without impacting the the article that’s initially loaded.






Author: Jim Robinson

Jim is the Founder & CEO of ClickSeed, and a 16-year veteran in the digital media space. Stay connected with Jim by following any of the social media links below, or get in touch via our contact form any time.

More Posts »


Follow me on FacebookFollow me on Google+Follow me on TwitterFollow me on LinkedInFollow me on RSS

13 Responses to “SEO-Friendly Infinite Scroll with Progressive Enhancement”

  1. How would you implement static pagination on a news site (wordpress). Google’s solution says: “Be sure the URL shows conceptually the same content two weeks from now.” The content on page 1 is changing every day though. The way to do it would be to put the oldest articles on page 1. I haven’t seen anyone do that. Can’t wait to see your solution. Yours is one of the best articles on this topic.

    Reply
    • Thanks so much for the kind words, Anna! My apologies for taking so long to respond; I was making a lot of changes on the site and this got lost in the shuffle.

      The example Google shows when they address the issue you mention is time-based URLs (like “days-ago=3”). It’s an interesting point though, even with the more common types of pagination you see, because you have the moving target scenario where an entry appears on Page 1 one day and Page 2 the next, which leads to what looks like duplicate content. Mashable addresses this by using a month-based archive – everything is reverse chronological like you see on most publisher sites, but for archive pagination it gets locked into monthly archives so you don’t have this moving target scenario; every time you come back to the April 2014 page, for example, it contains the same list of results.

      That said, continually updating reverse chronological pagination has been around forever and is used by almost every major content management system. Google is able to deal with this pretty effectively. If you use standard pagination with properly implemented rel=prev/next markup and canonicalization, you should be fine. I don’t suggest putting your oldest articles on Page 1.

  2. Thank you for your helpful reply, Jim.
    Here’s the search friendly infinite scroll implementation for a publisher site
    http://news.artnet.com/
    Let me know what you think 🙂

    Reply
  3. Very nice, Anna. Well done! A little something I noticed: you have your homepage canonical set to: http://news.artnet.com/?infpage=1. I don’t think you want to include that parameter in the canonical URL of your homepage.

    Reply
  4. I’m using Drupal and there is an infinite scroll plugin which shows classic pagination. It works great in that regard. I just don’t like the visuals. I’m sure this is OK, but wanted to check to see if I’m missing something, could I just hide the pager with JS instead of CSS?

    Thank you.

    Reply
    • Jim Robinson

      Typically in an infinite scroll experience, the classic pagination appears in the source code, but the user never actually sees it because the next page loads automatically and covers it (for the same reason, the user also never sees the footer either). Are you saying the classic pagination still appears on the page when the infinite scroll functionality is enabled? I’m having a hard time picturing that.

  5. Hi Jim,

    Yes it does because it’s up to the developer to hide it. My question is it OK to hide with CSS? Google tends not to like things hidden with CSS. Or use JS? It’s a tiny snippet of JS to hide the page, which I imagine should be fine, and it degrades well if the user has JS disabled.

    Thank you.

    Reply
    • Jim Robinson

      I understand your concern and I think it’s fine to “hide” it with some combination of CSS/JS. My perspective is that you’re not actually hiding it; the same content can be accessed by clicking through the classic pagination in the case of non-JS browsers. You’re just progressively enhancing the experience by exposing subsequent pages automatically for modern browsers. That’s an entirely different thing than hiding content so that search engines can see it, but users can’t.

  6. DeRidder LA Real Estate

    I’ve been trying to implement Jim Mueller’s update all day long with no luck. I have a 45MB web page with real estate listings I that I need to bust up into infinite scroll with pagination. I can’t believe that it seems like the only thing that is easy to use is a wordpress plugin, but unfortunately I don’t use wordpress to display property listings.

    Reply
  7. Hi Jim,
    Just following up on your article to see if you have an example to share that nails the SEO friendly infinite scroll.

    Reply
  8. Hi Jim,
    I am confused by something here. Using IE-11, I see the loading ‘logo’ in the tab (For lack of a better term) keeps spinning all the time. I see this does something similar with Chrome, until the pop up opens and I close it, then it seems to behave. Would this give the end user a false sense that the page has not finished loading, as it did me? I saw this on a few of the links in this article too, so I am wondering about it. I have been looking at various infinite scrolls with how Google looks at them for a few days now.
    Thanks for your insightful article, good reading.

    Reply
  9. We found this example, which handles infinite scroll well. You’ll see the page numbers appended to the URL, as you scroll through the content: http://collider.com/

    Reply
  10. I’ve tried copying, and modifying, the Mueller test site code for my purposes.
    However, I’m having trouble with the initialization part (on page load).

    A number of values clearly need to be set appropriately, and somehow the test site is doing this, but I’m guessing it’s something server-side as can’t find it in the source code (html, js, css).

    Would love to have demo code that I can run on local test server.
    I’m sure they’d get lots more sites implementing this recommendation if clearer example of how to do it.

    Reply

Leave a Reply

  • (will not be published)