Continuous delivery at Yoast

At Yoast, we like to release often. We’re a fan of continuous delivery and currently release every two weeks. As we sometimes get questions about our release cycle, we’d like to tell the entire story on why and how we release and test our plugins before they are ready to be installed on millions of websites worldwide.

Yoast SEO: two week release cycles

At Yoast, we’re a fan of the software engineering approach called continuous delivery. Continuous delivery means that the software teams create and release software in short cycles. At any given time, the software should be stable enough to be released. Continuous deployment is one step further. Continuous deployment means that the software gets delivered to the user at any time using an automated system. For the yoast.com website, we use continuous delivery and continuous deployment. For the plugins, this is unfortunately not possible in the current WordPress environment. Our current software process is as close to continuous delivery for WordPress plugins can get. This is important, as this will result in fewer bugs, more focus, and more collaboration.

Fewer bugs

The Yoast SEO plugins are installed on over 9 million websites worldwide. We know that when our release has a bug, this has an impact on a lot of websites. Even if a bug is a, so called, edge case and only appears on 0,1% of all websites that run Yoast SEO, we’re talking about over 80 thousand websites.

Our developers, but also our board, support team, and everyone involved releasing the plugins, are well aware of the impact a release can have. This is why we stand behind our continuous delivery approach. If something goes wrong, we can fix and release the fix within a couple of hours to a couple of days. This is because the developers know exactly what they worked on the past two weeks. If a bug is introduced in the newest release, they only have to debug two week’s worth of code, instead of months.

More focus

This automatically means that our developers have more focus. They plan in advance what they’re going to work on. Our releases have certain themes. Think about the Schema release, or the word forms releases! This way the developers can dive into one big feature and solve one big puzzle at a time. It allows them to release this finished puzzle before starting on another puzzle, especially as the company is growing. When I first started working at Yoast, back in September 2014, we were with just a handful of developers. Right now, there are four development teams. Two of those teams work on our plugins. By focusing on certain subjects, the team leads can discuss where the focus will be on.

More collaboration between the various teams

This collaboration is not only happening between our developers. Collaboration is extremely important at Yoast. This means that with short release cycles, the developers can discuss any important features or issues with the support engineers, but also with our content team, our testers, our research team, our marketing, and social team and the board. So, when there is an important release coming up, every team lead knows quickly who to inform about what.

Our code gets tested excessively

Fortunately, we have dedicated testers working on testing the plugin. Our approach to verify the quality of the plugins can be divided into four separate testing solutions.

1. Automated testing

Every night the plugins with the latest development changes are tested against automated tests. The results of these tests are processed each morning. This means that the development teams receive early feedback if new code affected any existing functionality. All new features receive automated tests before these features are released.

2. Pre-production testing

Every release candidate gets tested on several testing environments. Among this, is a testing environment that resembles yoast.com. We believe that our code cannot be put live if it breaks yoast.com, so we test this excessively.

3. Acceptance testing

Another part of testing is the so-called acceptance phase. All release candidates are tested by acceptance testers. The testers test based on user experience, they test our software on various browsers and operating systems.

4. Feedback processing phase

If a user comes across issues after a new release, the issues get analyzed. First, the focus lies on fixing any breaking issue as fast and as good as possible. It gets tested thoroughly again. After that, we decide whether any new testing needs to be included in the development cycle. It could be that there will be new automated tests, a new step will be added for acceptance testing or that there is another approach needed.

Multiple releases within a few days: the real story

Sometimes it, unfortunately, happens that a bug is found immediately after or the day after a regular release. If the bug is severe, meaning that websites can break, it’s not an edge case, multiple websites suffer from this and it is introduced in the previous release, there might be a patch release upcoming. Fortunately, we have an excellent support team with team members all over the world. Please realize, we do not take our decisions about patch releases lightly. The reasons for releasing are carefully studied and processed within the various teams. We do not bring out a patch release if it is not necessary.

As explained before, our plugins are installed on millions of websites worldwide. Although they all run WordPress, not one website is the same. One website might run thirty plugins and an outdated theme, while another might only run Yoast SEO and keeps everything up to date. And this is exactly what makes it hard. With so many WordPress versions going around on the web, various hosting platforms, and PHP versions and with over 54,000 plugins and over 7,000 themes on WordPress.org alone, it is impossible for us to tackle each and every setup combination.

This is why our testers and developers work with the most common setups and PHP versions and check the current process after every patch release. This is also the reason why we partnered up with various plugin developers and discuss possible bugs to solve this as soon as possible.

In the future, we want to focus on an automated release process where you as a user do not have to manually hit the update button every time a Yoast plugin gets released. Our continuous delivery process is part of a culture of continuous improvement within Yoast. We aim to always deliver improvements to our users as fast as possible.