Creative Google maps II: Loading markers efficiently

May 28 2009
By: Hannes Lilljequist
Category:  Tech

In a previous article we talked about how we collect and process location data in a recent project. We will follow up with discussing how we dealt with the performance issues involved with a map that contains a very large number of markers.

Gmap integrates with Views to provide an easy and flexible way of displaying nodes as markers on a map. In most cases this method provides all the functionality you need, including custom marker icons and clustering. There are a couple of things that prevented us from using it though.

Lazy loading of map markers

First of all, our map needed to scale to a very large number of markers. With Gmap and Views, all of the content you want to display for each node is transferred all at once to the browser. This may translate into megabytes of data depending on the number of markers (and also on how much info there is to display for each marker).

The solution was to transfer only the bare minimum of information needed from the beginning, and then transfer the node data only when the user actually clicks on a marker. We set up two JSON callback functions in a Drupal module: one to list all markers, including their location and node ID, and one to provide all information about a specific node. This way, the initial page load is very quick since it doesn't contain any map data yet.

<?php
function mymodule_list($num_res = false) {
 
$view = views_get_view('myview');
  if (
is_numeric($num_res) && $num_res > 0) {
   
$view->display['default']->display_options['items_per_page'] = $num_res;
  }
 
// Execute the view.
  // Note: preview() is not optimal, does unnecessary processing, but execute()
  // doesn't seem to be enough.
 
$view->preview();
 
$results = array();
  foreach (
$view->result as $row) {
   
$results[$row->nid] = new stdClass;
   
$results[$row->nid]->title = $row->node_title;
   
$results[$row->nid]->lat = $row->location_latitude;
   
$results[$row->nid]->lng = $row->location_longitude;
  }
  print
drupal_to_js($results);
}

function
mymodule_load($nids) {
 
$nids = explode(',', $nids);
 
$nodes = array();

  foreach (
$nids as $nid) {
    if (
is_numeric($nid) && $node = node_load($nid)) {
     
// Render node content.
      // An alternative approach is to just return node objects and do rendering on the client side.
     
$content = node_view($node);
     
$nodes[$nid] = $content;
    }
  }
  print
drupal_to_js($nodes);
}
?>

Now, in order to expose these functions to JavaScript through http, we also had to implement hook_menu().

<?php
function mymodule_menu() {
 
$items = array();
 
$items['mymodule/list'] = array(
   
'title' => '',
   
'page callback' => 'mymodule_list',
   
'access arguments' => array('access content'),
   
'type' => MENU_CALLBACK,
  );
 
$items['mymodule/load'] = array(
   
'title' => '',
   
'page callback' => 'mymodule_load',
   
'access arguments' => array('access content'),
   
'type' => MENU_CALLBACK,
  );
  return
$items;
}
?>

In order to actually make this useful we also need a page that contains a Google map. This can be done in several ways, like using Gmap macros or calling theme('gmap') in module code. See Gmap's README-file for more information.

On the client side

The final piece to make this work is, obviously, the JavaScript. We won't go into details here, but the following is a simple example of how the above functions can be accessed on the client side and used to put markers on a map. This script needs to be somehow added to the map page created above.

/**
* Extend a gmap.module map.
*/
Drupal.gmap.addHandler('gmap', function(elem) {
  var obj = this;

  /**
   * Init code for the map.
   * This seems to be the most reliable place for finding the actual map object
   * with gmap.module.
   */
  obj.bind('init',function() {
    var map = obj.map;
   
    // Load map markers. This could be moved to be triggered by some other event.
    $.getJSON("/mymodule/list", function(data) {
      $.each(data, function (key, item) {
        var marker;
        // Create this marker and add it to the map.
        marker = new GMarker(new GLatLng(item.lat, item.lng), {title:item.title});
        map.addOverlay(marker);
        // Add click event to the marker.
        GEvent.addListener(marker, 'click', function() {
          // Load this node and open in an info window.
          // Note: this should preferably be cached on the client side.
          $.getJSON("/mymodule/load/" + key, function(data) {
            $.each(data, function (key, item) {
              marker.openInfoWindowHtml(item);
            });
          });
        });
      });
    });
  });
});

Conclusion

We've shown how you can create your own method of loading markers and info windows on a Google map through JSON requests to a Drupal backend. The main advantage is that the browser doesn't have to load everything all at once, but rather gets the data in pieces as the user requests it. We will continue on to talk about marker clusters and custom info windows in the next article.

Kommentarer

First off thanks for this! It's just what I've been trying to do on my own but really needed someone to show me the way!

I've been able to get most of it to work except for the click listener.

When you add the listener to the marker you are using a var called 'nid' but I don't see where it is coming from?

I've tried the code but when I click on a node I get a javascript error that says 'nid not defined'.

Perhaps you forgot to show us the releveant code?


$.getJSON("/mymodule/load/" + nid, function(data) {
$.each(data, function (key, item) {
marker.openInfoWindowHtml(item);
});

Ok, I figured out that nid should be replaced by 'key'.

A second question though, why do you have a $.each loop in the listener handling code? The listener will ever only get one data (the marker you clicked), so no need for $.each, right? Is it just a simple way to extract 'item' from the the 'data' array?


$.getJSON("/mymodule/load/" + nid, function(data) {
$.each(data, function (key, item) {
marker.openInfoWindowHtml(item);
});

Ok, I was wrong, replacing nid with key does not help ... doing that causes the info bubbles to be randomly placed.

So how do you get the nid so you can pass it pack to your mymodule/load script?

Thanks!

Yes, I think nid should be key in the place you mention. Looks like I mistake that I probably made while generalizing our project code. I'll update the original post, thanks!

Not sure why your bubbles get randomly placed though. Please let me know if you work it out!

In answer to your second question, I'd have to say yes, each() was probably just a convenient way of extracting the one item from data.

Hi Hannes and thank you for taking the time to reply to my message!

First off, I know almost nothing about JQuery and JSON so I really appreciate what you've done.

As for my problem, actually the problem I see is that when I click on a marker, another marker's bubble pops up. It's almost as if I am assigning the wrong nid to the Listener? I'll keep looking ...

Another question I have is why does your mymodule_load() function take in a list if nids? The click listener only ever sends back one nid (the key), no?

The cause:

I had taken the 'var marker' declaration and put it outside the JSON call like this:


var marker = 0;
$.getJSON("/mymodule/list/" + params, function(data) {
$.each(data, function (key, item) {
// Create this marker and add it to the map.
marker = new GMarker(new GLatLng(item.lat, item.lng), {title:item.title});

For some reason that interferes with setting the marker in the Listener. The marker used by the listener is always the first (or last) marker created. Possibly 'var marker' not only declares but re-initializes at each iteration of the $.each loop?

My problem is fixed but if you can teach me why the placement of the 'var marker' declaration is important I would love to learn!

First, the loader function. It's designed like that to make it possible to load multiple markers at once, in case that would be desired. If I remember correctly, we didn't use that possibility in the end.

And I think I can explain why you would get the wrong marker reference if you put the var declaration in the init function. The thing you have to remember is that .each() is a function with its own variable scope. This is different from the classic each control structure you see in most programming languages, like PHP, where the variable scope is the same in each iteration.

What's also important is that the code in the click event function only runs once a marker is clicked – and at that point the .each() function has already finished iterating over the markers. If there is only one instance of marker, there can only be one marker value at that point.

By declaring the variable uniquely for each marker we make sure each marker gets it's own value saved. In other words, there isn't just one marker variable, which the code may give the impression of. The variable actually exists in multiple copies, one for each iteration.

Hope that makes things clearer. I had to think about this for a while myself. :)

Thank you for the extremely clear answer!

I'd like to implement the suggestion you have about caching the data:


// Note: this should preferably be cached on the client side.

Do you have some recommendations on how best to achieve this?

How about adding a variable to hold all loaded markers in the init function?


obj.bind('init',function() {
var map = obj.map;
var markers = new Array();
...

And then, just before loading a marker, look for an existing record:


// Load this node and open in an info window.
if (markers[key]) {
marker.openInfoWindowHtml(markers[key]);
}
else {
...

And finally, save the markers that do get loaded:


...
marker.openInfoWindowHtml(item);
markers[key] = item;

I just wrote this down without testing it, so I can't guarantee that it'll work right away.

Not a bad idea! I'll give that a try.

My database has about 10,000 markers and if the user plays enough with the map, eventually he will load them all. I wonder how much of a strain on the client's browser holding 10,000 makers is?

The actual info bubble text is 1k or less but how much overhead is there for the GMap Marker object itself?

I don't want to bring the client's browser to a crawl if 10,000 markers will eat up all it's resources :)

More Tech

The appification of the web

Recently it seems like we're at a tipping point, where a new generation of the web is being born. One aspect of this is the “appification” of the web. In this post I’ll try to give a bird’s eye view of this development.