Migrating Weather.com To Drupal: The Presentation Framework

A tale of spanning editorial and frontend needs

Last November, The Weather Channel launched their site on the Drupal framework, making it the highest-trafficked Drupal site in history. Mediacurrent, as the development firm that built the site, faced some unique technical challenges to ensure we not only met the requirements, but exceeded them wherever possible. There have been several presentations about the project, and a broad overview and case study were published around launch time, but little has been written about the details of how we approached their business requirements.

In an effort to change that, we are beginning a blog series in which each entry will explore a different aspect of the project in detail. So keep an eye out for more entries like this one in the coming months, and if you are curious about a specific aspect of the project, be sure to comment here and maybe we can address it in a later post.

For this first entry, we will explore part of the custom built solution we call the “Presentation Framework”. TWC needs to allow a (non-technical) content team to rapidly modify content and layout of pages throughout the site. Given the numerous types and rapidly changing nature of the content widgets required, we wanted to involve TWC’s own talented team of developers as much as possible in the implementation of the logic driving them.

Because their developers’ expertise is primarily in front-end javascript and java development, we wanted to provide them with a system that allows them to rapidly create widgets that the content team can use to enhance those layouts. Panels is a natural solution to this need, but requiring the development team or the content team to learn Drupal’s APIs or advanced panels configuration was not an attractive option, and thus the presentation framework was born.

The presentation framework manages the widgets created by the frontend development team and presents them to the content team. On the presentation side, not only are we able to deliver DOM templates, but also javascript and css payloads that are, wherever possible, rolled up into aggregates.

In architecting this framework, several requirements had to be considered and balanced. As you see in the above image, a typical page on the site is comprised of many sections or widgets, each of which has its own needs in terms of cacheability, configurability, and individualization. Our system needed to be be powerful enough to alter presentation based on the target device, or offload processing to the client side for delivering content where appropriate.

Our solution was to write an interpreter that simplifies the creation of new panels content types to creating a folder in a specific place, and writing a JSON file defining that type. From this JSON file, we generate the new panels content type programmatically, including any necessary configuration forms and settings. Here is an example:

{
  "readable_module_name": "Test context module",
  "description":          "Utility module to demonstrate functioning context dependent module",
  "category":             "Dev Test",
  "module_status":        "active",
  "version":        "1.0.2",
  "ttl":              "24000",
  "type":             "static",
  "presentation":     "inline",
  "admin_presentation": {},
  "template":         [
    {
      "label":              "Dynamically rendered",
      "machine_name":       "dynamic_rendered",
      "type":               "tpl.php",
      "file":               "templates/testmodule.tpl.php"
    }
  ],
  "context_required": [
    {
      "type": "twclocation"
    }
  ],
  "usable_in":        {
    "any": []
  },
  "add_js":           {
    "header": [
    ],
    "footer": [
    ]
  },
  "configuration":    {
    "fields": {
      "testfield": {
        "#title": "Test field",
        "#type": "textfield",
        "#required": false,
        "#default_value": "default text"
      }
    }
  }
}

There are several things worth noting in this JSON file. The “ttl” (aka “time-to-live”) allows us to specify a cache time, and the “presentation” allows us to specify how we will render this widget (some of other options besides “static” are “angular” and “esi”). We can set any required CTools contexts needed to generate this widget, which means it will only show up as a pane that can be added to panels that already have the required context associated with them. We can include any additional javascript needed in the appropriate scope. And finally, we can define configuration forms using what you may recognize as a JSON-ified version of Drupal’s Forms API.

Our module defines a CTools plugin in the typical fashion, and then uses the plugin’s ‘content type’ and ‘admin info’ callback attributes to populate our myriad widgets dynamically from the included sub-modules. Here’s our hook_ctools_plugin_directory() implementation that tells the CTools module where to look for plugins:

/**
 * Implements hook_ctools_plugin_directory().
 */
function angularmods_ctools_plugin_directory($owner, $plugin) {
  if ($owner == 'ctools') {
    return 'plugins/' . $plugin;
  }

  return NULL;
}

And here’s our simple plugin definition that registers our callbacks for building all of our widgets:

/**
 * Plugins are described by creating a $plugin array which will be used
 * by the system that includes this file.
 */
$plugin = array(
  'title'        => t('AngularJS Module'),
  'description'  => t('Add and configure an AngularJS module to the page'),
  'category'     => array(t('AngularJS Modules'), -9),
  'content type' => 'angularmods_ngpanes_content_type_content_type',
  'admin info'   => 'angularmods_ngpanes_content_type_admin_info',
  'defaults'     => array(),
);

Now, we begin to get into the meat of this system. This callback is the function that builds our pane metadata:

/**
 * Utility function to generate the pane configs.
 *
 * This function determines which panes are available for use on any
 * particular panel.
 *
 * @return array
 *   Collection of pane metadata
 */
function angularmods_ngpanes_content_type_content_type() {
  // No reason to generate repeatedly, so (static) caching results.
  $types = & drupal_static(__FUNCTION__, array());
  if (!empty($types)) {
    return $types;
  }

  // Get all of the module metadata from the AngularJS path.
  $modules = _angularmods_get_metadata();

  // Loop through the found modules and create panes for each.
  foreach ($modules as $module_key => $module) {
    // Create the pane config.
    $types[$module_key] = array(
      'category'    => !empty($module->category) ? $module->category : 'AngularJS Modules',
      'icon'        => 'icon_field.png',
      'title'       => $module->readable_module_name,
      'description' => $module->description,
      'defaults'    => array(),
      'cache'       => TRUE,
    );

    // Loop through required contexts, if any.
    if (isset($module->context_required)) {
      foreach ($module->context_required as $context_req) {
        $context = $context_req->type;
        if (isset($context)) {
          // Add the context requirement to the pane metadata.
          $types[$module_key]['required context'][] = new ctools_context_required(
            t('Required context'),
            $context
          );
        }
      }
    }
  }

  return $types;
}

If you’ve followed this far, though, you can probably see that even in the above function the majority of the work is being done in the function it calls, _angularmods_get_metadata(). It is separated out in this way because we use it several places to build our cacheable list of widgets, including during pane save and update. Before you have a look, it’s worth noting that what we present here is a simplified version of the function, for the sake of both readability and brevity. All the error handling and a few other things have been removed. Here it is:

/**
 * Utility function to get all AngularJS module metadata.
 *
 * @return array|stdClass
 *   Array of metadata objects found or object if specific module requested.
 */
function _angularmods_get_metadata() {
  $cached_metadata = & drupal_static(__FUNCTION__);

  if (!isset($cached_metadata)) {
    $angularmods_cache = variable_get('angularmods_cache', TRUE);
    if ($angularmods_cache && ($cache = cache_get('angularmods_ng_metadata'))) {
      $cached_metadata = $cache->data;
    }
    else {
      $app_info_file_path = drupal_get_path('module', 'angularmods') . '/app';

      // Collect all folders that contain plugins.
      $dirs_to_scan = module_invoke_all('angularmods_get_dir');

      // For each directory, add to the collection of plugins.
      $modules = array();
      foreach ($dirs_to_scan as $dir) {
        $modules += file_scan_directory(
          $dir,
          '/.*/',
          array(
            'nomask'  => '/.*\..*/',
            'recurse' => FALSE,
          )
        );
      }

      // Add the angular app to the modules array for later return.
      $twc            = new stdClass();
      $twc->uri       = $app_info_file_path;
      $twc->filename  = 'twc.app';
      $twc->name      = 'twc';
      $modules['twc'] = $twc;

      $cached_metadata = array();
      foreach ($modules as $module) {
        $info_file = $module->uri . '/' . $module->filename . '.info';
        if (file_exists($info_file)) {
          // fopen and json_decode our file.
          $res = _angularmods_load_and_validate_json($info_file);
          if ($res) {
            $res->uri         = $module->uri;
            $res->module_name = $module->filename;
            // Validate and extend the theme data.
            foreach ($res->template as &$template) {
              // If no label, try setting a default.
              if (empty($template->label)) {
                $template->label = $template->file;
              }
            }
            $cached_metadata[strtolower($module->filename)] = $res;
          }
        }
      }

      // Cache the data so that we don't have to build it again.
      // (if cache enabled).
      if ($angularmods_cache == TRUE) {
        cache_set('angularmods_ng_metadata', $cached_metadata, 'cache', CACHE_PERMANENT);
      }
    }
  }
  return $cached_metadata;
}

Our get_metadata function begins by collecting all directories that have been declared as having widget modules in them. It then goes through each subdirectory looking for .info files, which contain the JSON to be parsed. From that simple JSON file, we end up with a new panel pane content type that we can add to any page on the site, as seen below in both the Panels UI and on the page itself:

Why a Panels content type? This allows their developers to create new widgets, which their site managers can then add to their layouts using the highly customizable drag-and-drop panels interface into the various pages throughout the site. It means that widgets can be re-used as many times and in as many different places as needed, while their internal logic could remain independent, versioned and exportable. This also means that instead of having to manage different types of widgets from different places, the custom JSON / Angular widgets could be added in exactly the same way as a dynamic Drupal block, for instance, or even just a static piece of HTML content. As of this writing, TWC developers have written over 200 such widgets! Additionally there are even more shared libraries, directives and supporting code using the platform as modules.

That about wraps up our first peek behind the curtain of what has been a landmark project for Mediacurrent, and for the wider Drupal world. Many thanks to project lead Jason Smith (the presentation framework is his brainchild) and the rest of Mediacurrent’s weather.com team, and of course to you our readers.

Stay tunded for additional posts in our blog series about the Weather.com launch!

Additional Resources

You Stay Classy Panels Module | Mediacurrent Webinar
The Weather Channel's Journey to Drupal | Mediacurrent Blog Post
Weather Channel | Case Study