Implementing Automated Step Documentation for JavaScript BDD Framework

Whether you have a test automation framework that uses a behavior-driven development (BDD) approach, like JavaScript BDD, or prefer keyword driven testing, you should have at least some step documentation describing the setup, execution parameters, development guidelines, supported configurations and any other important information. However, not all projects have this type of documentation, particularly if there are already defined steps and keywords. Nevertheless, there are numerous reasons why such documentation is important. It helps maintain consistency throughout the project, allows team members to easily access information, and makes onboarding new project members much easier.

Now, you might be thinking about some of the drawbacks of step documentation. For instance, it can get outdated, it takes a lot of time to write, and is difficult to maintain. However, it doesn’t have to be all that bad. Thankfully, there is an easier way to prepare and manage documentation for test automation frameworks—automated step documentation.

The benefits of automated step documentation

Is automated step documentation worth the time and effort? Depending on the project, there can be many benefits of such an investment:

• No maintenance. When the generation, publishing and hosting of automated documentation is already implemented and processes are automated, there is no additional user effort required to add, remove or modify information in documentation.
• Up-to-date documentation. When manually writing documentation, there’s always the possibility to forget to update it. With automated documentation you can be sure that all documentation describing testing steps is up to date and on point.
• An overview of steps. Anyone can look at existing steps in documentation and see if a specific action is already automated. Depending on the processes within the team, automated documentation can be useful when requesting new steps to be implemented or avoid duplicates when implementing new ones.
• Faster and easier automated test creation. With a sufficient amount of steps defined and prepared, any team member—even those with less or no technical knowledge—can combine already developed steps into scenarios and help speed up test automation to reach required functional coverage much faster.

Generating step documentation automatically

There are various tools that can be used to generate step documentation automatically:

• JSDoc. This tool is designed to be used for JS code annotations but can also be used to describe steps. JSDoc generates HTML documentation with a plain and simple design which can be improved by using templates.
• Javadoc. Similar to JSDoc, Javadoc is also used for documenting code but for Java. The simple design with limited configuration possibilities can be adapted to specific needs by making adjustments to the code of the generated Javadoc HTML.

There are other tools and libraries that can be used to generate documentation for testing steps automatically, however, on most occasions documentation is generated only after test executions and they also include test execution results. While such documentation can also be useful, if you are looking for a fast way to generate documentation automatically—without waiting for test execution to be completed—then such tools might not be the best options.

Preparing to implement automated documentation

To automate the documentation process successfully, there are several requirements you should be mindful of:

• Use a CI platform like Jenkins to orchestrate this process.
• Keep in mind that comments that are added for each step in step definition files have strict formatting rules based on documentation generator requirements.
• Use a documentation generator like JSDoc or Javadoc.
• Have a place to publish the documentation—it can be in Jenkins, GitHub Pages or a custom server.

The flow of the conditions mentioned above can actually be quite simple. Jenkins is triggered manually, periodically or by changes in a specific repository branch’s step definition files. It gets the latest code from the repository, executes the script for documentation generation, and then publishes it. If it is required to have documentation in the repository, then generation can be triggered before commits (if step definition files are changed) and changes are automatically added to the commit.

With this approach, each new step or step update would be automatically included in the documentation as long as there is a control rule for the merge request approval process to create or update documentation for each implemented step.

Implementing automated step documentation for JavaScript BDD

Prerequisites: To better understand this section, it would be beneficial to be familiar with Continuous integration (CI) platforms such as Jenkins and GitHub Actions as well as solutions for local servers exposed to the public internet, like Ngrok (if Jenkins is hosted locally).

In this section, we will look at how to implement automated documentation for the JavaScript BDD test automation framework.

Generator installation

The first step is to install a documentation generator. You can find the installation of JSDoc here.

npm install —g jsdoc

Step description preparation

Below is an example of how to prepare descriptions for step definitions according to the generator’s formatting requirements. For more information on description possibilities, see this resource from JSDoc.

/**
* @name Then User checks that "<>" item is in the cart
* @param {string} itemName Name of the item.
* @description Asserts that item with provided name is in the cart.
* @example Then User checks that "Dress Trousers" item is in the cart
*/
Then(/^User checks that "(.*?)" item is in the cart$/, (itemName) => {
   return client.page.minicart()
       .assert.visible('@page')
       .assert.containsText('@itemTitle', itemName)
});

Template installation

To improve the look of the documentation, it’s necessary to install a template. In the example below, the FooDoc template is used.

npm install foodoc

Template configuration

Here is an example of a FooDoc template configuration .json file:

{
   "plugins": ["plugins/markdown"],
   "templates": {
       "includeDate": true,
       "dateFormat": "DD MMM YYYY",
       "systemName": "Test Automation",
       "systemSummary": "Setup and steps documentation.",
       "systemLogo": "",
       "systemColor": "",
       "copyright": "",
       "linenums": false,
       "collapseSymbols": true,
       "inverseNav": true,
       "inlineNav": false,
       "outputSourceFiles": false,
       "outputSourcePath": false,
       "disablePackagePath": true,
       "methodHeadingReturns": true,
       "showTableOfContents": true,
       "showAccessFilter": true,
       "sort": "linenum, longname, version, since",
       "search": true,
       "stylesheets": [],
       "scripts": []
     }
}

Pre-commit hook

Automating the process to keep step documentation up to date can be achieved by adding a pre-commit hook to verify if there are any changes in step definition or README files. If there are changes, add those files to the commit. It is still the user’s responsibility to add step descriptions that are required for documentation generation.

Install pre-commit and add it to the package.json file.

"pre-commit": [
   "stepscheck"
 ],

Step definition file check script

The “stepscheck” script should be defined in the package.json file to see if there are any changes to the step definition or README files. If there are, you will need to regenerate documentation.

"stepscheck": "status=$(git status cucumber/steps README.md --porcelain | grep .); if [[ $status ]]; then echo \"Generating docs...\" && npm run docsgen; fi;",

Documentation generation script

The “docsgen” script needs to be defined in the package.json file to generate documentation using the given template, README, and all step definition files. Also, the updated documentation is added to the commit.

 "docsgen": "jsdoc -c conf.json -t ./node_modules/foodoc/template -d ./docs -R README.md -r ./cucumber/steps && git add ./documentation"

Adding step documentation to your project repository might not be needed but it can be beneficial to have up-to-date information right after pulling the code. If that is not required, then documentation can be generated locally by the user launching a defined script and ignoring commits by adding  ‘./docs’ folder to the .gitignore file.

Publishing documentation

To publish automated step documentation, one of the easiest ways is to use existing CI infrastructure (if available) to host documentation. In the case of Jenkins, this can be done using the HTML Publisher plugin.

Once this plugin is installed, the Publish HTML reports action can be added to Post-build Actions.

Other than that, it is a matter of inspecting the code from the repository to be able to execute the post-build action.

Later, this task can be triggered remotely after merging any changes related to documentation in the main branch. This can be achieved using GitHub Actions which requires a specific .yaml configuration file in folder .github/workflows. More information about action creation can be found in GitHub Docs.

Example of GitHub Actions workflow

name: Jenkins docs update
 
on:
 push:
   branches:
     - master
   paths:
     - 'docs/**'
 
jobs:
 deploy:
   runs-on: ubuntu-latest
   steps:
     - name: Deploy Stage
       uses: fjogeleit/http-request-action@master
       with:
         url: 'https://986f40653662.ngrok.io/buildByToken/build?job=DOCS&token=GitTriggerToken'
         method: 'POST'

Ngrok is used to make Jenkins accessible. In the case shown above, it was hosted locally and could not be reached from GitHub.

GitHub Actions offer a free plan that includes 2,000 CI/CD minutes per month which is enough for smaller workflows like the one from our example. However, for longer workflows, an upgrade to one of their pricing plans will be necessary.

Remote trigger

There are several ways to configure remote triggers in Jenkins, however, in this example, the Build Authorization Token Root plugin was used.

Implementation result

The bottom line

To sum up, automated step documentation can be a good addition to your existing testing processes and can support test automation usage. It promises numerous benefits, however, it really depends on the project context, the test automation process, the test automation solution itself, and the available infrastructure to determine whether implementing automated step documentation would be useful for your project and whether it is even possible.

The good news is that there are no additional costs involved and implementation should not take too many of your resources, which makes it a great solution to keep your step documentation up to date and your testing teams on the same page.

Need some extra help testing your software solution? TestDevLab has the resources and experience to ensure your product is tested thoroughly and meets all your requirements. Get in touch and let’s discuss your project.