Saturday, May 31, 2014

Feature Toggles

In Continuous Delivery, Feature Toggles provide a useful way to turn on or off behaviour within your application.  It provides tremendous value to QA by allowing progressive deployment of new features, enabling early customer testing while minimising impact of failure.

While most QA and test managers want to delay release until certain that there are no major defects in the system, this can significantly stymie delivery and inhibit innovation.  Feature Toggles allow new features to be deployed, but only exposed to limited users.  In some cases this may be named users, internal user groups, or a limited slice or percent of the customer base.  In this way, defects in new features have limited exposure and impact on the user base.  This enables the new features to be tested on production environments, and also allows feedback from alpha and beta users on the effectiveness of new features.

Simplistically, a Feature Toggle is a if statement surrounding a new feature that determines whether the feature is exposed to the user or not.  However, more powerful Feature Toggle libraries can use various aspects to determine whether the feature is exposed, such as:

  • user credentials and groups
  • employees, 
  • GEO location, 
  • Browser user-agent,
  • percentage of users
  • server load
  • etc

Enabling features earlier in a controlled approach speeds up feedback from customers.  For startups, and applications requiring innovation, often it is a greater risk that we are building the wrong product.  Often much more significant that we are building the product right.  In these situations we want to prioritise testing with real customers, getting feedback on the effectiveness of the features.  This may be higher priority on whether there are defects in the feature construction itself.  Getting user feedback early allows the user design of the feature to pivot, and we can return to other system tests once the feature user value has been fully realised.

Feature Toggles can be key to the A/B Testing process.  Toggles can partition features depending on the A/B or multivariate segments.  Performance measurements can then be compared between features exposed or hidden.

Adopting Feature Toggles has its gotchas that must be carefully managed.  At some stage, features should be mainlined and the toggle taken out of the code, or where the feature is unsuccessful and toggles are off, the feature is removed from the code base.

Testing feature toggles also requires care.  Integration tests will need to flick toggles between on and off to expose or hide the feature within tests.

Further reading:

2 comments:

  1. I'd like to hear more about your experience with the gotchas. Maintaining the coverage of integration and GUI driven automated manual tests as features are exposed and removed is tricky.
    I'm trying to utilise some of the workflows in distributed version control to help with this. Gitflow holds promise...

    ReplyDelete
  2. Hi Chris, have a read of some of the links at the bottom. There is some discussion of Feature Toggles versus Feature branches in GitFlow, etc. If you search "Feature Branches versus Feature Toggles" in google, you will find a lot of discussion on this issue.

    Yes, testing can get quite complex when you have different conditions enabling functionality. There was an article I read recently about different strategies (I just tried to find it but couldn't). Anyhow it talked about some organisations have a strategy of automated tests with all features enabled, some all features disabled, at different stages of the deployment process. You would have to develop a dev process policy on this to ensure your team understands the coverage.

    The other key element is that Feature Toggles should have a lifespan. The plan is to use them to build and test functionality, and then deploy to progressively wider audiences as you are confident. Then at some stage they need to be retired, either the feature is permanently enabled or it id disabled. If you don't have that cleanup in the lifespan the code complexity will progressively get out of control. It looks like some organisations have a suggested limit to the number of feature toggles the may concurrently support.

    I don't think feature branches really gets past this issue either. At the end of the day, Feature Toggle are primarily a way to get your code into production quicker, so that it can be testing in the production environment, or using samples of real customers. While feature branches allow you to work on different features concurrently, there is still the issue of what combinations of features is going to be deployed and to who, and how quickly you can do that. Feature branches also have to have different tests running for the different branches, and also we have to merge those at some stage.

    ReplyDelete