By
Kanhai
October 30, 2019
Quality of any software not only depends on good engineering but also on the quality of testing. More often than not engineers tend to overlook the bugs which are obvious to others. That’s exactly the problem we faced when we started building products for customers. Customers were reporting some really obvious bugs which we should have found before rolling out the product. We realized that we needed to find a fool-proof process that would help us get relevant feedback and avoid a future scenario as above. While brainstorming the different approaches we could take to eliminate maximum bugs in our product, we came across the idea of Bug Bash.
Bug Bash is a process in the software development lifecycle where features developed by the product team is tested by a whole organization. The basic idea behind bug bash is to run the product by all the stakeholders of the company and to make sure that the feature meets the expectation of all departments. Bug Bash provides a great platform for the product team to receive 360-degree feedback for the product they have developed. Also, it helps greatly in reducing the number of silly bugs that often get rolled out to customers.
At Almabase, we have a small product team which consists of 4 engineers and 1 product manager. We do not have any QA resource so developer/product manager is expected to test the features and make sure a stable product is being pushed to end-users.
So, here’s how it all started.
Three years ago, when we started building our product, we all thought we were ready to test the features which we were developing.
Well, to our disappointment we saw a lot of silly bugs being reported by customers.
Even after spending a considerable amount of time for testing, we were not able to meet our own expectations and we still kept receiving some pretty basic bugs being reported by customers. There were some scenarios where a few customers reported grammatical mistakes or at times, a few of the team members from other teams felt that the messages or the help texts could have been better.
That’s when we started brainstorming different ideas on how to improve our product quality.
We asked ourselves multiple questions:
Should we hire a resource whose only responsibility would be to test the product?
Should we introduce some stringent testing processes?
It was during these brainstorming sessions that we stumbled upon the idea of Bug Bash. Gradually, we incorporated bug bashes into our regime for product feature testing and as of today, it has become an integral part of our Software Development life cycle.
Having done nearly 18–20 Bug Bashes, we have figured out few absolutely necessary things to prepare for a Bug Bash.
Scope Document: We make sure that we have a document that explains in detail what this new product is all about. This document should ideally answer these questions:
Test Environment Setup: We make sure that the test environment has everything set up. Some things that are taken care of beforehand are: login credentials, data which is either replica of production or of similar nature. We also create credentials for different roles so the feature can be tested with different roles.
Team Division: We make sure that all Bug Bashers are properly divided into separate teams and are assigned different responsibilities. Assigning different responsibilities helps us ensure that maximum product features get tested while eliminating the problem of more than one individual spending time on one bug. Team division is a sure shot way to save time.
Time: We tend to keep our Bug Bash limited to 1.5 hours. We use the first 15-20 minutes to brief all the bug bashers about the product which is then followed by testing. For features which are huge and are hard to cover in a single Bug Bash, we divide it across multiple Bug Bashes.
In addition to the above points, it is advised to inform all the Bug Bashers well in advance so you would know how many people are going to attend the Bug Bash. That way, it becomes easier for you to plan it well.
Once the Bug Bash is done, the Product team sits together and figures out what bugs/suggestions need to be fixed before they release the new feature. If some aspects our delaying a release, those are assigned to future versions unless it’s critical and needs to be fixed right away. This way Bug Bash doesn’t delay our release significantly and our product features get rolled out on time with better quality.
.jpg)
All the Bug Bashes we have done till now, we were able to find at least 50 bugs each time. Whether it was a grammatical mistake or something major like a save button not working, bug bash has been extremely helpful in detecting bugs quickly. So if we look back, these Bug Bashes have helped us identify 900–1000 bugs proactively. They’ve also helped us get our team, across different departments, be on the same page regarding any feature or enhancement getting rolled out to customers.
We are still improving our Bug Bash processes and are continuously thinking about how to get the best out of them.
.svg)
