Postman to Newman - API Automation pipeline

Overview

Postman is an HTTP client and mostly used to test HTTP requests. Users can send requests, inspect responses, and can easily debug APIs. Postman offers a user friendly UI that allows any novice to learn it quickly and can easily run and test API requests. The test tab on the request page allows users to write and customize tests for any positive or negative scenarios on multiple parameters. Users can create collections of related API requests and can run the whole collection in one go and can view the test report in the UI itself. Daily runs can be scheduled using Postman monitors and users can observe daily test runs over monitors to view the APIs’ health. Whereas Newman is a command-line collection runner for Postman.

Why Newman?

Though users can create, and run the collections over the Postman UI and can view the test summary report over the same UI, we need a tool that can allow the user to run those collections over continuous integration servers. That’s where Newman comes into the picture. Newman is Postman’s open-source CLI tool that enables users to run and test a Postman collection directly from the command line. It is powered by the same runtime that powers the Postman native apps. In short, Newman is the Postman’s Collection Runner engine that sends API requests, receives the response, and then runs your tests against the response. Newman maintains feature parity with Postman and allows you to run collections the way they're executed inside the collection runner in Postman. It’s built with extensibility in mind so that users can integrate it with continuous integration servers like Jenkins, Travis CI, or any other code deployment pipeline tool and build systems. It also supports external reporters, provided that the reporter works with Newman's event sequence.

Let’s illustrate this through an example. Though there are other tools in the market to test and run the APIs, there was an urgent requirement in our project to start with the API testing and available automation talent pool was not too familiar with the Rest assured framework. We were looking for an easy-to-use tool that didn’t require learning a new language and could be a good choice for novice testers with minimal API side testing experience and at the same time available at a minimal cost. Another requirement was to use a tool that could be integrated with the CI-CD pipeline. Post thorough research we came up with the decision to start API testing using Postman. It’s an easy to use, easy-to-learn and free tool that has become a choice of a huge API testers community and supports many kind of API calls (REST, SOAP, or plain HTTP). Within a week of using it for all API testing, we started writing multiple assertions and found that we could easily integrate test suites into our preferred CI/CD service with Newman.

Now, we prepare test collections with multiple assertions in Postman and use Newman to run over Jenkins and generate html summary test reports. In the next section installation steps and steps that enable Newman for Postman collections, are explained in detail.

Run collections using Newman

As one needs to install Postman in order to run the API collections on UI, similarly to run those collections over command line, users need to install Newman using the following command through command line. One should have node.js installed before running npm commands.

install -g newman

Post-installation, the user can check the Newman version.

And the user is all set to run the first collection. But before that, one needs to export the Postman collection. It is suggested to run the collection over Postman prior to exporting, as this allows a clean run when using Newman.

Here are the steps: -

1. Run the collection over Postman
   
2. Export the collection
   
3. Run the collection over command line
   

   Newman run {collectionname}

   
   This also generates the test execution report at the end.
   
   
   

Though this report is good enough for the user to view test results but to share the reports , it’s always good to generate detailed and UI friendly report for the end users. Here is where Newman’s reporting capabilities shine through.

Newman’s HTML reporter

Newman has the capabilities to generate the test result summary report in HTML format and it offers different types of HTML reporting though it is always preferred to use a htmlextra report. It’s worth mentioning that having a plain HTML report is also quite readable and presentable.

One can install the Newman HTML reporter by using the following command:

$ npm install -g newman-reporter-html

$ newman run {collectionName} –r html

This is the HTML sample report that is generated.This report is in plain text and does not have multiple tabs or sections. Newman also offers more extensive reports with an additional reporting tool htmlextra that allows users to generate more user friendly test reports.

One can install the Newman htmlextra reporter by using the following command

$ npm install -g newman-reporter-htmlextra

$ newman run {collectionName} –r htmlextra

This is the HTML sample report that is generated. One can change the theme and can navigate to the different tabs for more details.

Newman Integration with Jenkins

By now we are quite familiarized with Newman. and it is time to integrate it with the build system Jenkins. This allows users to get quick feedback on the performance of APIs and any changes pushed will be captured and tested in the next scheduled build.

The very first step to start this integration is to create a job over Jenkins. By clicking the new item on the Jenkins home page, the user is prompted to enter the job name and the user also selects freestyle project as displayed here and clicks “ok”.

Users can enter the details for Build triggers to schedule the runs and should also provide commands similar to the one here in build-> Execute Windows batch command prompt.

Saving this will create a Jenkins job that will build daily as mentioned in the schedule and a daily report will be generated.

About Encora

Fast-growing tech companies partner with Encora to outsource product development and drive growth. Contact us to learn more about our software engineering capabilities.