As technology practices evolve and diversify, quality standards and best practices for software development remain a top priority for development teams.
Encora’s Technology Trends present the top 10 trends in 2022 from a software engineering perspective —trends that hold the promise for organizations to respond to emerging disruptions and succeed in a rapidly changing world.
We spoke with Encora’s Professional Services Unit Manager, Manuel Cubillo, and the DevOps Innovation Leader Isac Souza about 1 of the 10 trends selected: the evolution and widespread adoption of Software Quality Standards.
What are Software Quality Standards?
I: Software Quality Standards are a way to measure quality against a set of standards that the industry considers ideal. It’s a way to validate that your software is being built with quality in mind and it’s following the correct procedures. This set of standards can be specific to your enterprise or they can be industry standards.
Essentially, they are the standards that you must assure that your software development lifecycle is following.
To what do we owe the rise in this trend?
M: Quality will always be a trend. The market is demanding, users can be tough to satisfy, and competition is high. There’s always another application or another system that is going to do exactly what you are trying to do with your application. That element of competition is always setting the bar higher.
Software Quality Standards are directly related to other emerging trends. Look at a trend like Multicloud. There’s a quality concept behind that. Before you launch, you must make sure you’re not only creating software products that meet those Software Quality Standards but you can also certify that.
What are the main quality standards for software?
M: To answer that, we need to split the quality concept into multiple fronts.
On one side, there’s the concept of quality in its functional behavior. Any time you start looking for quality, you need to make sure that it behaves functionally, the way the user expects. When you are using a banking system and you need to transfer money to someone else, there are certain expectations that you have when making a transaction.
On the other side, there’s performance. Depending on the application, the performance is going to be measured differently. For example, if you are on a video streaming platform these days, you don’t expect to hit play and spend 10 or 15 seconds just loading, watching the spinning circle on the screen. You expect that after a couple of seconds, it’s going to play and you can watch your movie without any kind of delay or performance issues.
If you make a transaction on an operational banking system platform, you expect error-free processing and transactions. You are dealing with money, after all. Maybe you are closing a statement for an important management report or you are investing in something. There’s sensitive information involved. In this example, performance is about being quick, error-free, and secure.
You can see that quality standards depend on the app, the industry, etc. The standard may be functional, performance-based, or security-based. Security is, perhaps, one of the better-defined standards because specific markets or verticals need to comply with standards that have been set by third parties. There are ISO standards and software compliance standards for certain kinds of applications. If it’s health care, for example, we have HIPAA compliance standards.
What is resilience testing?
I: Testing for resilience means testing whether your software can tolerate certain failures. You do that by basically injecting problems into a running system (failures in third-party components, failures in infrastructure, failures in dependent services, and internal failures) and seeing how the software reacts and if your software is prepared for failure scenarios.
Resilient software will detect a failure and work around it either by showing the user a failure message or by using fallback services. The main thing is that the software is prepared and won’t completely fail and stop serving requests.
Tell us about shift-left testing and its impact on Software Quality Standards.
M: Shifting left is a well-known concept in the DevOps mindset. When you talk about shifting left in software development, you can think of the software development process as a timeline. The timelines always go from left to right. Everything starts as the first point on the left and as you start to move ahead in the process, like in the traditional model, you design a system. You start building that system and when you have something, you start validating it. When it’s ready, you deploy it, put it into production, and operate on that system. So, designing the system would be on the far-left side and when it’s actually running and operating, it’s going to be on the right side.
When we talk about shifting left in the context of testing, we are talking about moving our testing efforts as early in the cycle as possible. The main benefit of doing that would be that, the earlier you test, the less costly it’s going to be when fixing an issue.
If you are testing your design and an architect or a QA specialist identifies a possible issue, the fix is not going to be as costly because, most likely, it’s simply a matter of redesigning something, rewriting a model, or reorganizing some ideas. If instead, you identify an issue on the right side of the timeline, after you have already invested in designing, coding, deploying, and maybe even production, or in the worst-case scenario, it’s already in the hands of your users—that makes the issue much more resource-intensive to control. You may have sensitive data that has already been compromised. You may have compromised on performance or perhaps you will have to go offline for a while. For some organizations, going offline for only a few minutes can mean a rather large financial loss.
Shifting left in testing might seem like an extra cost in the beginning but it’s going to prevent any chaotic scenarios that organizations may face while testing on the far-right.
What kind of competitive edge will QAOps offer over regular DevOps?
I: By automating as much quality assurance testing as possible, you are gaining a competitive edge because you have a faster and more reliable development cycle. If it takes you five days to manually execute thousands of test cases, and you automate most or all of them, you can go from five days to executing them in one hour.
The development cycle and the feedback loop become shorter and you can produce more features and deliver more value to your users in a shorter period, with better quality.
But it’s not a competitive edge “over” regular DevOps, it’s more of an addition to DevOps because you are adding DevOps practices to QA, implementing the same mindset, and automating the QA part of the development cycle.
What is synthetic user monitoring and what are the advantages over real user monitoring?
M: Synthetic testing is, for example, when you have some transactions that you are going to inject into your system, and you need to be able to identify them as synthetic when they are something that you made up, but they are going to live in your system like any other transaction.
The whole idea of a synthetic transaction is to test your flows multiple times within a real environment so, you can test what the outcome is going to be. That is pretty useful, for example, in FinTech environments. If you have an internet banking application, you want to introduce some synthetic transactions to make sure that they successfully go through the whole process. You can synthetically test the process of investing or getting a credit statement, and you can put them into your production system. You can identify them so that they don’t affect any real data. You can put them aside for analysis, in production, or in stable environments. If you want to test, you can use it every time and it will produce results and insights for you to analyze if you are negatively affecting your end-users, which is something that we want to prevent by shifting left in our testing.The key idea is to replicate the interaction between a real user and the application using multiple workflows and paths, collecting & analyzing this interaction data, and using these analyses to better understand the application performance.
What are the principles of continuous testing?
I: Continuous testing is the practice of always executing your test cases in the development lifecycle. Instead of executing tests at the end of the development cycle, you have the tests running against every change and, by running tests more often, you catch problems earlier and are able to fix them with fewer resources than at the end of the development lifecycle.
So, continuous testing is similar to shift-left testing, where you execute the tasks as early as possible in the pipeline.
Tell us about codeless test automation and how it paves the way for QA testing.
Just as low-code/no-code platforms can be leveraged to execute specific tasks without having to write code, codeless (sometimes also called scriptless) test automation enables automated testing without having to write code. The benefits are significant and straightforward: the product development cycle can be accelerated, code coverage can be enhanced, test quality can be improved, and, importantly, coding skills cease to be a barrier. However, it is also important to understand the current limits of codeless test automation. As a result, in most cases, automated testing when combined with codeless test automation creates ideal outcomes, instead of either one of them being used exclusively.
Tell us about ISO5055:2021 and why it will continue to be a standard for software quality.
ISO5055:2021 is a software quality standard that was first published in early 2021 and computes quality metrics based on the count of best practices violations related to coding in the source code. These metrics are aligned with factors like security, reliability, and maintainability of the code. Prior to this standard, there was no way of understanding the structural weaknesses of any piece of code. Over time as the standard gains traction, we expect the standard to become an integral part of QA processes, and a way to benchmark code quality against best-in-class. When combined with “shift-left” QA & security strategies, ISO5055:2021 provides an excellent method of prioritizing what gets “shifted-left” first.
How can an organization implement more rigorous Software Quality Standards?
M: It’s not done overnight. Implementing Software Quality Assurance Standards starts with the basic steps. You need to make sure that you understand what your application is supposed to do, because based on that, you will apply specific techniques to define what your testing coverage is going to be. When you try to define test coverage, you need to make sure that you cover as much as you can but ideally, with the least effort. That can be done only if you apply certain techniques to make sure that your coverage is the right one. You don’t want to have 20 or 100 test cases that are very well implemented but are all the same. That’s going to delay your development process and you are going to have to wait longer for the product to be created and deployed. So, defining the right coverage is key.
For you to do that, you need to define the proper scenarios. Those need to be well-defined in terms of what they do and what the outcomes are so that they can be deterministic. Deterministic means you know that if you do a specific action, you will always get the expected output. Having this well-established enables you to automate those test cases because you will know exactly what you need to automate and the possible paths of execution, and you will know exactly what to expect on each path.
Any final thoughts?
I: Just start small. Start automating and collecting the low-hanging fruit and then, after you see the value of it, expand to cover a broader range of test cases.
A special thanks to Encora’s Professional Services Unit Manager Manuel Cubillo and DevOps Innovation Leader Isac Souza for taking the time to talk to us about the acceleration of the Software Quality Standards trend.
Software Quality Standards is one of ten trends featured in Encora’s 2022 Technology Trends. To read more, visit Encora’s 2022 Technology Trends.
“By automating as much quality assurance testing as possible, you are gaining a competitive edge because you have a faster and more reliable development cycle. If it takes you five days to manually execute thousands of test cases, and you automate most or all of them, you can go from five days to executing them in one hour.” -Isac Souza
About Encora
Encora is a digital engineering services company specializing in next-generation software and digital product development. Fast-Growing Tech organizations trust Encora to lead the full Product Development Lifecycle because of our expertise in translating our clients’ strategic innovation roadmap into differentiated capabilities and accelerated bottom-line impacts.
Please let us know if you would ever like to have a conversation with a client partner and/or one of our Innovation Leaders about accelerating next-generation product engineering within your organization.
