It’s a myth that if you use a client side MVC framework that your application’s content cannot be indexed by search engines. In fact, Discourse forums were indexable by Google the day we launched.
Search engine visibility does, however, require a little more work to implement. This is a real trade off you’ll have to consider before you decide to go with an MVC framework instead of an application that does its rendering on the server side.
Before you get scared off: I’d like to point out that our search engine code was done by Sam Saffron in a day! This extra work might take you less time than you thought.
Getting Started: Pretty URLs
Out of the box, most client side MVC frameworks will default to hash-based URLs that take advantage of
the fact that characters in a URL after an
application boots up it looks at hash data and figures out what it has to do.
Modern browsers have a better alternative to hash-based URLs: The HTML5 History API. The History API
http://yoursite.com/#/users/eviltrout you can support
There are two downsides to using the History API. The first is Internet Explorer only started supporting it IE10. If you have to support IE9, you’ll want to stick with hashes. (Note: Discourse actually works on IE9, but the URL does not update as the user goes around. We’ve accepted this trade off.)
and the basic markup for search engines in a
<noscript> tag. If you’re unfamiliar with a the
tag, it’s designed for rendering versions of a resource to a clients like search engines that don’t support
This is really easy to do in Ruby on Rails (and probably other frameworks that I’m less familiar with!).
application.html.erb can look like this:
With this approach, if any server side route renders a simple HTML document, it will end up in the
tag for indexing. I wouldn’t spend much time on what the HTML looks like. It’s meant to
be read by a robot! Just use very basic HTML. To preview what a search engine will see, you can turn
In Rails, you can reuse the same logic for finding your objects, and then
choose the JSON or HTML rendering path in the end. Here’s a simplified version of our
def show @user = fetch_user_from_params respond_to do |format| format.html do # doing nothing here renders show.html.erb with the basic user HTML in <noscript> end format.json do render_json_dump(UserSerializer.new(@user)) end end end
Note that you don’t have to implement HTML views for all your routes, just the ones that
you want to index. The others will just render nothing into
One More Thing
If you get an HTML request for a URL that also responds with JSON, there is a good chance your application is going to make a call to the same API endpoint after it loads to retrieve the data in JSON so it can be rendered.
You can avoid this unnecessary round trip by rendering the JSON result into a variable
if it exists in the document already. If it’s there, use it instead of making the extra
This approach is much faster for initial loads! If you’re interested in how it’s implemented in Discourse, check out:
preload_store.js - a simple interface to load data that’s been set in
application.html.erb how we set data in