by finne on 14-06-2017

Drupal 8 development: caching on

Conventional wisdom has it that you should disable all caches when doing Drupal 8 development. Whilst this is the easiest way to make sure all development changes are incorporated when you reload the site, it has some drawbacks. The whole site becomes slow, sometimes much slower than with caches on. Drupal 8 has a great caching mechanism that all visitors benefit from, so why should you miss out as developer? Development is another way you can 'use' a Drupal 8 site. I spend most of my time developing Drupal 8, and do many page navigation and reloads on sites with the cache off. Besides that, for some things you need to rebuild the cache even with the caching mechanisms turned off, for example when you need a refresh of the plugin discovery. When this needs doing the usual tools are 

drush cache-rebuild (drush cr)
drupal cache:rebuild (drupal cr)

The disadvantage of this is that it also bootstraps drupal - again without caches - and then clears all caches, taking quite a while to complete.

The solution to all of this is to start developing with caching enabled. I now develop with all caching mechanisms on: opcache, D8 dynamic and static cache, JS and CSS cache, etc. I only refresh the part of the cache that needs updating. This makes my site faster during development, makes cache clear commands faster, and helps my understanding of which part of the site is covered by which cache bin / mechanism. I can recommend it to every Drupal 8 developer.

Cache bin reference table

Below is are some handy reference tables that shows the various ways you can selectively clear a cache bins / mechanisms.

When you work on Clear these cache bins / mechanisms Possible methods
Modules bootstrap / discovery bin Admin Toolbar > Flush All Caches > plugins
drush cache-clear module-list
drupal cache:rebuild bootstrap / discovery
sql truncate cache_bootstrap / cache_discovery
Plugins discovery bin Admin Toolbar > Flush All Caches > plugins
drupal cache:rebuild discovery
sql truncate cache_discovery
Routes data bin Admin Toolbar > Flush All Caches > routing and links
drush cache-clear router
drupal cache:rebuild data
drupal router:rebuild
sql truncate cache_data
Menus/Links/Actions/Tasks discovery / menu bin  Admin Toolbar > Flush All Caches > routing and links
drupal cache:rebuild discovery / menu
sql truncate cache_discovery / cache_data
Pages/Blocks render bin Admin Toolbar > Flush All Caches > render
drush cache-clear render
drupal cache:rebuild render
sql truncate cache_render
Content entity / render bin Admin Toolbar > Flush All Caches > render
drush cache-clear entity / render
drupal cache:rebuild entity / render
sql truncate cache_entity / cache_render
Views data / discovery bin drush cache-clear views
drupal cache:rebuild data / discovery
sql truncate cache_data / cache_discovery
Twig templates Twig cache, default bin drush twig-compile
drupal cache:rebuild default
sql truncate cache_default
CSS CSS cache Admin Toolbar > Flush All Caches > CSS and Javascript
drush cache-clear css-js
drush state-delete drupal_css_cache_files / system.css_js_query_string (?)
drupal state:delete drupal_css_cache_files / system.css_js_query_string (?)
JS JS cache Admin Toolbar > Flush All Caches > CSS and Javascript
drush cache-clear css-js
drush state-delete system.theme.files (?)
drupal state:delete system.theme.files (?)
Themes bootstrap / default / discovery bin drush cache-clear theme-registry / theme-list
drush state-delete system.theme.files / system.theme_engine.files (?)
drupal cache:rebuild bootstrap / default / discovery
drupal state:delete system.theme.files / system.theme_engine.files (?)
sql truncate cache_bootstrap / cache_default / cache_discovery

Find the relevant cache bin

To know which cache bin affects what is sometimes tricky - you will need to figure this out. Here is my current understanding. If you cannot get a refresh to happen using a specific cache you can always revert to using GUI/drush/drupal cache clear all.

The various cache bins are used for the following:

  • bootstrap
    • Data needed from the beginning to the end of most requests, that has a very strict limit on variations and is invalidated rarely.
    • base system, module system, and theme system information.
  • config
    • configuration (?)
    • config entities (?)
  • container
    • Drupal service container
  • data
    • Contains data that can vary by path or similar context.
    • library_info (CSS, JS)
    • routes
    • route_match
    • views config
  • default
    • arbitrary caches
    • module list
    • twig templates
    • theme registry
    • locale
    • views_data
  • discovery
    • Contains cached discovery data for things such as plugins, views_data, or YAML discovered data such as library info.
    • plugins
  • discovery_migration
  • dynamic_page_cache
  • entity
    • content entity data.
  • menu
    • menu trees
  • migrate
  • render
    • Contains cached HTML strings like cached pages and blocks, can grow to large size.
    • rendered pages
  • toolbar

Then there are some non-cache bin caching mechanisms:

  • JS
    • Contains compacted and combined JS files.
    • query string to refresh browser caches
    • collection_optimizer to combine and compact files
  • CSS
    • Contains compacted and combined CSS files.
    • query string to refresh browser caches
    • collection_optimizer to combine and compact files
  • Twig PHP Storage cache
    • Contains twig templates rendered to PHP files.
  • Drupal static repository
    • Contains static variables set by drupal_static().
  • key-value pairs
    • stored in SQL key_value table
    • Contains State items.


Background: functions called

As background to the above I include here a list of functions called by the various commands.

Configuration > Development > Performance > Clear all caches

  • drupal_flush_all_caches();

Configuration > Development > Performance > Save configuration

  • cssCollectionOptimizer->deleteAll();
  • jsCollectionOptimizer->deleteAll();
  • renderCache->deleteAll();


Admin Toolbar > Flush All Caches > 

  • CSS and Javascript (flush browser JS and CSS)
    • reset system.css_js_query_string
  • plugins (part of the discovery bin)
    • \Drupal::service('plugin.cache_clearer')->clearCachedDefinitions();
  • render (render bin)
    • $this->cacheRender->invalidateAll();
  • routing and links (menu and part of the discovery bin)
    • menu_cache_clear_all(); 
    • $this->menuLinkManager->rebuild();
      • $this->cacheTagsInvalidator->invalidateTags('*');
    • $this->contextualLinkManager->clearCachedDefinitions(); 
    • $this->localTaskLinkManager->clearCachedDefinitions(); 
    • $this->localActionLinkManager->clearCachedDefinitions();
  • static (resets drupal statics)
    • drupal_static_reset();


drush cache-clear

  • drush
    • drush_cache_clear_drush();
  • theme-registry (part of the bootstrap and part of the default bin)
    • \Drupal::service('theme.registry')->reset();
      • Cache::invalidateTags(['theme_registry']);
  • router (part of the data bin)
    • \Drupal::service('router.builder')->rebuild();
  • css-js
    • _drupal_flush_css_js();
      • reset system.css_js_query_string
    • drupal_clear_css_cache();
      • \Drupal::service('asset.css.collection_optimizer')->deleteAll();
    • drupal_clear_js_cache();
      • \Drupal::service('asset.js.collection_optimizer')->deleteAll();
  • module-list (part of the default bin)
    • \Drupal::cache()->delete('');
    • drupal_static_reset('system_get_info');
  • theme-list 
    • \Drupal::service('theme_handler')->rebuildThemeData();
      • $this->state->set('system.theme.files', $files_theme);
        • part of the key_value db table
  • render (render bin)
    • Cache::invalidateTags(['rendered']);
  • views (part of the data and part of the discovery bin)
    • views_invalidate_cache()
      • \Drupal::service('router.builder')->setRebuildNeeded();
      • \Drupal::getContainer()->get('views.route_subscriber')->reset();
      • \Drupal::service('plugin.manager.block')->clearCachedDefinitions();


drush cache-rebuild

  • drupal_rebuild();
  • drush_cache_clear_drush();


drupal cache:rebuild 

  • all
    • drupal_flush_all_caches();
  • bootstrap
  • config
  • container
  • data
  • default
  • discovery
  • discovery_migration
  • dynamic_page_cache
  • entity
  • menu
  • migrate
  • render
  • toolbar
  • static
    • A special cache bin that does not persist beyond the length of the request.

drupal console invokes $cache->deleteAll() on the specified bin.


SQL truncate table 

  • cache_bootstrap
  • cache_config
  • cache_container
  • cache_data
  • cache_default
  • cache_discovery
  • cache_discovery_migration
  • cache_dynamic_page_cache
  • cache_entity
  • cache_menu
  • cache_migrate
  • cache_render
  • cache_toolbar

The cache tables correspond to cache bins.


PHP (Drupal 8 cache API)

  • $cache = \Drupal::cache('bootstrap');
    • use any cache bin name, no parameter returns the 'default' cache.
  • $cache->deleteAll();
    • DatabaseBackend: Run SQL truncate on cache_bin_name table.
  • more granular get, set, invalidate and delete functions available.
    • NB: If you mark cache items as invalid, they may be returned in later calls to get(), if the $allow_invalid argument is TRUE.
  • DefaultPluginManager::clearCachedDefinitions();
    • Cache::invalidateTags($this->cacheTags); or
    • $this->cacheBackend->delete($this->cacheKey);
  • views_invalidate_cache()
    • \Drupal::service('router.builder')->setRebuildNeeded();
    • \Drupal::getContainer()->get('views.route_subscriber')->reset();
    • \Drupal::service('plugin.manager.block')->clearCachedDefinitions();
  • drupal_rebuild()
    • works even when drupal does not bootstrap
    • PhpStorageFactory::get('twig')->deleteAll();
    • $cache->deleteAll();
    • drupal_flush_all_caches();
  • drupal_flush_all_caches()
    • Clears all persistent caches:
      • $cache_backend->deleteAll();
    • Clears asset (JS/CSS) file caches.
      • _drupal_flush_css_js();
      • \Drupal::service('asset.css.collection_optimizer')->deleteAll();
      • \Drupal::service('asset.js.collection_optimizer')->deleteAll();
    • Resets all static variables that have been defined via drupal_static().
      • drupal_static_reset();
    •   Invalidate the container.
      • \Drupal::service('kernel')->invalidateContainer();
    • Wipe the Twig PHP Storage cache.
      • PhpStorageFactory::get('twig')->deleteAll();
    • Updates the system with latest information about extensions (modules and themes).
    • Updates the bootstrap flag for modules implementing bootstrap_hooks().
    • Rebuilds the full database schema information (invoking hook_schema()).
    • Rebuilds data structures of all modules (invoking hook_rebuild()). In core this means
      • blocks, node types, date formats and actions are synchronized with the database
      • The 'active' status of fields is refreshed.
    • Rebuilds the menu router.
      • \Drupal::service('router.builder')->rebuild();

about the author

Finne Fortuin is Drupal development and deployment configuration expert. He tries to understand quantum field theory and Drupal 8 caching.