FFmpeg Converter's preset overview table.

Exportable configuration for your Drupal module with Ctools

I've worked a lot on making the configuration in FFmpeg Converter exportable during the last couple of days. I found it to be very rewarding, although the whole thing grew larger than I had expected, as always. In this post I'll give an overview of the steps required to create a fully featured exportable configuration interface in Drupal.

Note: since this article was written, Ctools has been extended with an export UI API, which makes this article more or less outdated. Please read this new post instead!

First a little background. There is a well known problem with configuration driven development, which is the way things work in Drupal most of the time. You can quickly set up large portions of a site just by clicking around in the admin interface. However, when it comes to doing things like deploying new features on running live sites, things get trickier. This is because there is no clean separation between the content and the configuration on the site. While you want to use the content on the live site, the most up to date configuration is usually found on a testing site, and there is no easy way to merge the two.

The solution to this problem, most of the time, is to capture the configuration in code somehow. The code can then be version controlled and distributed between different sites, or different versions of a site. This concept is usually referred to as exportability in Drupal development, and there are ongoing efforts to standardize the ways this is done. The most powerful choice is probably Ctools, which has the benefit of making its exportables available to the Features module automatically.

Getting started

I'll show how I used the Ctools export API to make FFmpeg Converter's configuration presets importable and exportable, and possible to define as default presets in module code. I'll also show how I developed the new interface for managing the presets. The result is a very smooth configuration interface with very powerful features under the hood.

All in all there's a lot of code in this post, but even so, this is not a complete working example. Please check the latest 6.x-2.x version of FFmpeg Converter for a complete, up-to-date implementation. On the other hand, it's possible to create exportable configuration without doing all of the UI stuff that's described in this post.

There are a couple of references you can check while you read this if you need more information:

Defining your data

First, we have to define the table structure for the presets. Ctools requires that the exportables live in their own database table and have at least a machine readable name field, and a database-only numeric primary key. This is all done in the ffmpeg_converter.install file:

<?php

/**
* Implementation of hook_schema().
*/
function ffmpeg_converter_schema() {
 
$schema['ffmpeg_converter_preset'] = array(
   
'description' => t('Table storing preset definitions.'),
   
'export' => array(
     
'key' => 'name',
     
'identifier' => 'preset', // Exports will be as $preset
     
'default hook' => 'default_ffmpeg_converter_preset'// Function hook name.
     
'api' => array(
       
'owner' => 'ffmpeg_converter',
       
'api' => 'default_ffmpeg_converter_presets'// Base name for api include files.
       
'minimum_version' => 1,
       
'current_version' => 1,
      ),
    ),
   
'fields' => array(
     
'name' => array(
       
'type' => 'varchar',
       
'length' => '255',
       
'description' => 'Unique ID for presets. Used to identify them programmatically.',
      ),
     
'pid' => array(
       
'type' => 'serial',
       
'unsigned' => TRUE,
       
'not null' => TRUE,
       
'description' => 'Primary ID field for the table. Not used for anything except internal lookups.',
       
'no export' => TRUE, // Do not export database-only keys.
     
),
     
'description' => array(
       
'type' => 'varchar',
       
'length' => '255',
       
'description' => 'A human readable name of a preset.',
      ),
     
'ffmpeg_wrapper' => array(
       
'type' => 'text',
       
'size' => 'big',
       
'description' => 'A serialized array of FFmpeg Wrapper options.',
      ),
    ),
   
'primary key' => array('pid'),
   
'unique keys' => array(
     
'name' => array('name'),
    ),
  );
  return
$schema;
}

/**
* Implementation of hook_install().
*/
function ffmpeg_converter_install() {
 
drupal_install_schema($module);
}

?>

That looks more or less like a standard Drupal install file, except for the export part of the schema array. This is specific to Ctool's export API, and it defines how this table's content will be imported and exported. This is the basic thing that is needed for the information to be exportable.

CRUD functions

The presets need a couple of standard CRUD (create, read, update, delete) functions. I ended up implementing only two of them: ffmpeg_converter_load() and ffmpeg_converter_delete(). It's certainly possible to supplement these with C and U too.

<?php
/**
* Load one preset or all presets.
*
* @param $name
*   This presets name value. Optional.
* @return
*   If $name is specified, an array of options for the specific preset.
*   If not, a nested array containing options for all presets.
*/
function ffmpeg_converter_preset_load($name = null) {

 
// Use Ctools export API to fetch all presets from the DB as well as code.
 
ctools_include('export');
  if (
$name) {
   
$presets = ctools_export_load_object('ffmpeg_converter_preset', 'names', array($name));
    return isset(
$presets[$name]) ? $presets[$name] : FALSE;
  }
  else {
    return
ctools_export_load_object('ffmpeg_converter_preset');
  }
 
}

/**
* Delete a preset.
*
* @param $preset
*   A preset object, or a preset's pid value.
*/
function ffmpeg_converter_preset_delete($preset) {
 
$pid = is_object($preset) ? $preset->pid : $preset;
 
db_query('DELETE FROM {ffmpeg_converter_preset} WHERE pid=%d', $pid);

 
// Clear the Ctools export API cache.
 
ctools_include('export');
 
ctools_export_load_object_reset('ffmpeg_converter_preset');
}
?>

As you can see, both functions make use of the Ctools API. Among other things, it provides us with an abstracted cache mechanism so that the database doesn't have to be queried every time we need to load a preset.

Defining default presets

In order to provide some default configuration, you can implement a couple of hooks that Ctools provides. These presets will show up in the admin interface without the user having to do anything other than activating the module. This is exactly how modules other than FFmpeg Converter would do to provide their own default, code defined, presets, and it's also how Features saves this data in code.

<?php

/**
* Implementation of hook_ctools_plugin_api().
*
* Tell Ctools that we support the default_ffmpeg_converter_presets API.
*/
function ffmpeg_converter_ctools_plugin_api($owner, $api) {
  if (
$owner == 'ffmpeg_converter' && $api == 'default_ffmpeg_converter_presets') {
    return array(
'version' => 1);
  }
}

/**
* Implementation of hook_default_ffmpeg_converter_preset().
*
* Provide a couple of default presets.
*/
function ffmpeg_converter_default_ffmpeg_converter_preset() {
 
$export = array();

 
$preset = new stdClass;
 
$preset->api_version = 1;
 
$preset->name = 'default_flv';
 
$preset->description = 'Default FLV preset';
 
$preset->ffmpeg_wrapper = ffmpeg_converter_default_options();
 
$export['default-flv'] = $preset;

  return
$export;
}

?>

The administration interface

In order to make use of the ground work we've done so far, we need to add some administration pages to expose the functionality. Let's start with hook_menu():

<?php

/**
* Implementation of hook_menu().
*/
function ffmpeg_converter_menu() {
 
$items = array();
 
$items['admin/settings/ffmpeg_converter'] = array(
   
'title' => 'FFmpeg Converter',
   
'description' => t('Setup automatic media file conversion.'),
   
'page callback' => 'ffmpeg_converter_admin_presets',
   
'access arguments' => array('administer ffmpeg wrapper'),
   
'file' => 'ffmpeg_converter.admin.inc',
  );
 
$items['admin/settings/ffmpeg_converter/presets'] = array(
   
'title' => 'Presets',
   
'type' => MENU_DEFAULT_LOCAL_TASK,
   
'weight' => -10,
  );
 
$items['admin/settings/ffmpeg_converter/presets/add'] = array(
   
'title' => 'Create new preset',
   
'file' => 'ffmpeg_converter.admin.inc',
   
'page callback' => 'drupal_get_form',
   
'page arguments' => array('ffmpeg_converter_admin_preset'),
   
'access arguments' => array('administer ffmpeg wrapper'),
   
'type' => MENU_CALLBACK,
  );
 
$items['admin/settings/ffmpeg_converter/presets/edit/%ffmpeg_converter_preset'] = array(
   
'title' => 'Preset settings',
   
'file' => 'ffmpeg_converter.admin.inc',
   
'page callback' => 'drupal_get_form',
   
'page arguments' => array('ffmpeg_converter_admin_preset', 5),
   
'access arguments' => array('administer ffmpeg wrapper'),
   
'type' => MENU_CALLBACK,
  );
 
$items['admin/settings/ffmpeg_converter/presets/delete/%ffmpeg_converter_preset'] = array(
   
'title' => 'Delete preset',
   
'file' => 'ffmpeg_converter.admin.inc',
   
'page callback' => 'drupal_get_form',
   
'page arguments' => array('ffmpeg_converter_admin_preset_delete', 5),
   
'access arguments' => array('administer ffmpeg wrapper'),
   
'type' => MENU_CALLBACK,
  );
 
$items['admin/settings/ffmpeg_converter/presets/export/%ffmpeg_converter_preset'] = array(
   
'title' => 'Export preset',
   
'file' => 'ffmpeg_converter.admin.inc',
   
'page callback' => 'drupal_get_form',
   
'page arguments' => array('ffmpeg_converter_admin_preset_export', 5),
   
'access arguments' => array('administer ffmpeg wrapper'),
   
'type' => MENU_CALLBACK,
  );
 
$items['admin/settings/ffmpeg_converter/presets/import'] = array(
   
'title' => 'Import preset',
   
'file' => 'ffmpeg_converter.admin.inc',
   
'page callback' => 'drupal_get_form',
   
'page arguments' => array('ffmpeg_converter_admin_preset_import'),
   
'access arguments' => array('administer ffmpeg wrapper'),
   
'type' => MENU_CALLBACK,
  );
 
//...//
 
 
return $items;
}

?>

This function defines all of the admin pages that the module uses. A quick note about %ffmpeg_converter_preset: This path argument makes it so that Drupal automatically loads the preset objects through ffmpeg_converter_preset_load() and feeds them to the corresponding page. Very powerful, once you learn how it works!

If that seemed like a lot of code, just take a look at the functions that implement these pages.

Overview page

This is the main configuration page, shown in the image at the top. Here we use our _load function to fetch all presets and construct an overview table of all of them. Since that function uses the Ctools export API, we automatically get all available presets, whether they are defined in the database or in module code. We can also check the storage status (export_type) for each preset and alter the presentation accordingly.

<?php

/**
* Page callback: presets overview.
*/
function ffmpeg_converter_admin_presets() {

 
// Load presets and present them in table form.
 
$presets = ffmpeg_converter_preset_load();
 
$header = array(t('Description'), t('Name'), t('Source'), '', '', '');
 
$rows = array();

  foreach (
$presets as $key => $preset) {
   
$row = array();
   
$row[0] = check_plain($preset->description);
   
$row[1] = check_plain($preset->name);

   
$row[3] = l(t('Edit'),
     
'admin/settings/ffmpeg_converter/presets/edit/' . $preset->name,
      array(
'attributes' => array('class' => 'link-edit')));
    switch (
$preset->export_type) {
      case
1:
       
// "Normal" type.
       
$row[2] = '<em>' . t('Normal') . '</em>';
       
$row[4] = l(t('Delete'),
         
'admin/settings/ffmpeg_converter/presets/delete/' . $preset->name,
          array(
'attributes' => array('class' => 'link-delete')));
        break;
      case
3:
       
// "Overridden" type.
       
$row[2] = '<em>' . t('Overridden') . '</em>';
       
$row[4] = l(t('Revert'),
         
'admin/settings/ffmpeg_converter/presets/delete/' . $preset->name,
          array(
'attributes' => array('class' => 'link-delete')));
        break;
      default:
       
$row[2] = '<em>' . t('Default') . '</em>';
       
$row[4] = '';
    }
   
$row[5] = l(t('Export'),
     
'admin/settings/ffmpeg_converter/presets/export/' . $preset->name,
      array(
'attributes' => array('class' => 'link-export')));
   
ksort($row);

   
$rows[] = $row;
  }

 
$table = theme('table', $header, $rows);

 
// Add links beneath the table.
 
$links = array(
   
'link-add' => array(
     
'title' => t('Add preset'),
     
'href' => 'admin/settings/ffmpeg_converter/presets/add',
    ),
   
'link-import' => array(
     
'title' => t('Import preset'),
     
'href' => 'admin/settings/ffmpeg_converter/presets/import',
    ),
  );
 
$links = theme('links', $links, array('class' => 'ffmpeg-converter-presets-links'));

  return
$table . $links;
}

?>

Create/edit form

The next fundamental configuration page is the preset add/edit form. It consists of a form builder, a validator, and a submit function, following the standard Form API workflow.

These functions have been simplified to save space, but they show the important pieces. The validation code was shamelessly stolen from ImageCache.

<?php

/**
* Preset settings form.
*
* @param $name
*   The preset's machine readable name.
*/
function ffmpeg_converter_admin_preset($form_state, $preset = NULL) {
 
// Check if we're supposed to edit a preset or add a new one.
 
if ($preset) {
   
drupal_set_title(t('#name settings', array('#name' => check_plain($preset->description))));
  }
  else {
   
$preset = new stdClass;
   
$preset->name = '';
   
$preset->description = '';
   
$preset_options = ffmpeg_converter_default_options();
   
$preset->ffmpeg_wrapper = $preset_options;
  }
 
 
$form = array();

 
$form['name'] = array(
   
'#type' => 'textfield',
   
'#title' => t('Preset name'),
   
'#description' => t('The unique, machine readable name of this preset.'),
   
'#default_value' => $preset->name,
   
'#size' => 20,
   
'#required' => true,
  );
 
$form['description'] = array(
   
'#type' => 'textfield',
   
'#title' => t('Preset description'),
   
'#description' => t('The human readable name or description of this preset.'),
   
'#default_value' => $preset->description,
   
'#size' => 20,
   
'#required' => true,
  );

 
// Add FFmpeg Wrapper's configuration form.
 
$form += ffmpeg_wrapper_configuration_form($preset->ffmpeg_wrapper, "edit-ffmpeg-converter-presets-$preset->name-ffmpeg-wrapper-");
 
$form['ffmpeg_wrapper']['#collapsible'] = true;
 
$form['ffmpeg_wrapper']['#tree'] = true;

 
// Check this preset's export type/storage.
 
if (isset($preset->export_type)) {
    switch (
$preset->export_type) {
      case
2:
       
// "Default" type.
       
$form['name']['#disabled'] = true;
       
$form['name']['#value'] = $form['name']['#default_value'];
        break;
      case
3:
       
// "Overridden" type.
       
$form['name']['#disabled'] = true;
       
$form['name']['#value'] = $form['name']['#default_value'];
        break;
    }
  }

 
// Save extra preset info for the submit handler.
 
if (!empty($preset->pid)) {
   
$form['pid'] = array(
     
'#type' => 'value',
     
'#value' => $preset->pid,
    );
  }

 
$form['submit'] = array(
   
'#type' => 'submit',
   
'#value' => t('Save')
  );

  return
$form;

}

/**
* Validate a preset form.
*/
function ffmpeg_converter_admin_preset_validate($form, &$form_state) {
 
// Check for illegal characters in preset names.
 
if (preg_match('/[^0-9a-zA-Z_-]/', $form_state['values']['name'])) {
   
form_set_error('name', t('Please only use alphanumeric characters, underscores (_), and hyphens (-) for preset names.'));
  }
}

/**
* Submit handler to save a preset.
*/
function ffmpeg_converter_admin_preset_submit($form, &$form_state) {
 
// Serialize ffmpeg_wrapper data.
 
$form_state['values']['ffmpeg_wrapper'] = serialize($form_state['values']['ffmpeg_wrapper']);

  if (empty(
$form_state['values']['pid'])) {
   
// Save new preset.
   
$res = drupal_write_record('ffmpeg_converter_preset', $form_state['values']);
  }
  else {
   
// Save existing preset.
   
$res = drupal_write_record('ffmpeg_converter_preset', $form_state['values'], 'pid');
  }
  if (
$res) {
   
drupal_set_message(t('Saved preset %preset.', array('%preset' => $form_state['values']['description'])));
  }
  else {
   
drupal_set_message(t('Failed to save preset %preset.', array('%preset' => $form_state['values'drupal_goto('admin/settings/ffmpeg_converter/presets');
  }
}

?>

Deleting and reverting presets

The delete page is defined as a form using confirm_form(). Once again, this is following standard Drupal development patterns, with some custom additions. One thing that's a little extra here is that the page doubles as a revert page. In technical terms, reverting an overridden preset and deleting a normal one both involves deleting a row in the database. We just make sure the interface texts reflect the status of the current preset.

<?php

/**
* Delete preset form.
*
* @param $name
*   The preset's machine readable name.
*/
function ffmpeg_converter_admin_preset_delete($form_state, $preset) {
 
$form = array();
 
$form['pid'] = array(
   
'#type' => 'value',
   
'#value' => $preset->pid,
  );
 
// Save a couple of values for the submit handler.
 
$form['preset_description'] = array( // 'description' is used by confirm_form().
   
'#type' => 'value',
   
'#value' => $preset->description,
  );
 
$form['export_type'] = array(
   
'#type' => 'value',
   
'#value' => $preset->export_type,
  );

 
// Adjust the UI texts depending on export type.
 
switch ($preset->export_type) {
    case
1:
     
// "Normal" type.
     
$message = t('Are you sure you want to delete %title?', array('%title' => $preset->description));
     
$button = t('Delete');
      break;
    case
3:
     
// "Overridden" type.
     
$message = t('Are you sure you want to revert %title?', array('%title' => $preset->description));
     
$button = t('Revert');
      break;
    default:
     
// There are no other export types that can be deleted.
     
drupal_goto('admin/settings/ffmpeg_converter');
      return array();
  }

  return
confirm_form($form,
   
$message,
    isset(
$_GET['destination']) ? $_GET['destination'] : 'admin/settings/ffmpeg_converter',
   
t('This action cannot be undone. If you are using this preset in other parts of your system, you should change those settings first.'),
   
$button,
   
t('Cancel')
  );

}

/**
* Submit handler to delete a preset.
*/
function ffmpeg_converter_admin_preset_delete_submit($form, &$form_state) {
  if (
$form_state['values']['confirm']) {
   
ffmpeg_converter_preset_delete($form_state['values']['pid']);
   
// Adjust message depending on export type.
   
$message = $form_state['values']['export_type'] == 1 ?
     
'Deleted preset %preset.' :
     
'Reverted preset %preset.';
   
drupal_set_message(t($message, array('%preset' => $form_state['values']['preset_description'])));
  }
 
$form_state['redirect'] = 'admin/settings/ffmpeg_converter/presets';
}

?>

Imports and exports

Time for some really fun stuff. The next function takes a preset and spits out a code snippet which contains all of its data. The export is super easy to do, thanks to the ctools_export_object() function.

The way drupal_eval() works – which you'll see in the import function – means we have to serialize the preset object in the exported code.

<?php

/**
* Page callback: export a preset.
*
* @param $preset
*   A preset object.
*/
function ffmpeg_converter_admin_preset_export($form_state, $preset) {
 
drupal_set_title(t('Export of preset %description', array('%description' => $preset->description)));
 
$form = array();

 
// Get export code with Ctools.
 
$export = ctools_export_object('ffmpeg_converter_preset', $preset);
 
$export = "<?php\n{$export}return serialize(\$preset);\n?>
";

  // Create the export code textarea.
  ctools_include('export');
  $form['export'] = array(
    '#type' => 'textarea',
    '#title' => t('Preset code'),
    '#rows' => 20,
    '#default_value' => $export,
  );

  return $form;
}

?>

In the version of this function that I'm working on in FFmpeg Converter, I also included some auto-generated module code for implementing a default preset based on the exported preset. This could be a great help when developing a module that uses FFmpeg Converter.

Finally, the last piece of the puzzle is the the import page, which is also fairly straightforward. Combined with the export page, it provides a very powerful way of backing up, cloning, and sharing presets.

<?php

/**
* Page callback: import presets.
*/
function ffmpeg_converter_admin_preset_import() {
 
$form = array();
 
$form['import'] = array(
   
'#type' => 'textarea',
   
'#rows' => 10,
  );
 
$form['submit'] = array(
   
'#type' => 'submit',
   
'#value' => t('Import')
  );
  return
$form;
}

/**
* Validate a preset import.
*/
function ffmpeg_converter_admin_preset_import_validate($form, &$form_state) {
 
// Run the import code, which should return a serialized $preset object.
 
$preset = unserialize(drupal_eval($form_state['values']['import']));
  if (empty(
$preset) || !is_object($preset) || empty($preset->name)) {
   
form_set_error('import', t('The submitted preset code could not be interperated.'));
  }
  elseif (
ffmpeg_converter_preset_load($preset->name)) {
   
form_set_error('import', t('A preset by that name already exists.'));
  }
  else {
   
// Pass the parsed object on to the submit handler.
   
$form_state['values']['import_parsed'] = $preset;
  }
}

/**
* Submit handler to import a preset.
*/
function ffmpeg_converter_admin_preset_import_submit($form, &$form_state) {
 
$preset = (array) $form_state['values']['import_parsed'];

  if (
drupal_write_record('ffmpeg_converter_preset', $preset)) {
   
drupal_set_message(t('Imported preset %preset.', array('%preset' => $preset['description'])));
  }
  else {
   
drupal_set_message(t('Failed to import the preset.'), 'warning');
  }
 
 
$form_state['redirect'] = 'admin/settings/ffmpeg_converter/presets';
}

?>

Conclusion

Hopefully this post can be a help to those who want to expose their modules' configuration to an import/export API. I believe that process is something that can take your module to the next level, and also benefit the Drupal community as a whole.

Good luck!

Kommentarer

I have yet to use CTools from a module of my own. When I do though, which will be shortly, I now know where to start (and who to bugger over GTalk!). Thanks for sharing!

Yes, the problem domain is very similar across different modules that implement Ctools for configuration. You could probably generate a lot of the ui from the 'export' definition in hook_schema(). Ideally one should just have to create the actual configuration form elements yourself I guess.

Export-UI is now part of the CTools release, so most of mentioned above is already done in a generic way (but overridable) by CTools.

Hey... great post.

I am basically attempting to do the same thing in the MediaFront module, but then realized that it would be huge to have us all reuse the same module.

I created the Preset module to do just that.

I would like to make you a maintainer so please shoot me an email and we can all work together on this.

Thanks again for your efforts!

Travis.

Very nice that there is now a generic reusable solution to this. Well, two in fact. It seems to me that Ctools' Export-UI might be the better option since Ctools is already a dependency for any module using exportables. I don't know if it provides everything that Preset does, but I'd prefer to avoid yet another dependency if I can. I'll definitely look into this anyway.

There is now a patch for rewriting FFmpeg Converter's preset administration to use Export UI: http://drupal.org/node/893992

Reviews and tests are welcome. When everything lands I'm planning to write a new blog post to describe the process.

Thanks for your input!

Mer Kunskap

Några av våra kunder