Bug Bash @ Almabase

Bug Bash @ Almabase




October 30, 2019

Quality of any software not only depends on good engineering but also on the quality of testing. More often than not engineers tend to overlook the bugs which are obvious to others. That’s exactly the problem we faced when we started building products for customers. Customers were reporting some really obvious bugs which we should have found  before rolling out the product. We realized that we needed to find a fool-proof process that would help us get relevant feedback and avoid a future scenario as above. While brainstorming the different approaches we could take to eliminate maximum bugs in our product, we came across the idea of Bug Bash.

What is Bug Bash?

Bug Bash is a process in the software development lifecycle where features developed by the product team is tested by a whole organization. The basic idea behind bug bash is to run the product by all the stakeholders of the company and to make sure that the feature meets the expectation of all departments. Bug Bash provides a great platform for the product team to receive 360-degree feedback for the product they have developed. Also, it helps greatly in reducing the number of silly bugs that often get rolled out to customers.

Our team busy finding bugs during a bug bash before the launch of our "Communities" feature

Why did we decide to do Bug Bashes at Almabase?

At Almabase, we have a small product team which consists of 4 engineers and 1 product manager. We do not have any QA resource so developer/product manager is expected to test the features and make sure a stable product is being pushed to end-users.

So, here’s how it all started.

Three years ago, when we started building our product, we all thought we were ready to test the features which we were developing.

Well, to our disappointment we saw a lot of silly bugs being reported by customers.

Even after spending a considerable amount of time for testing, we were not able to meet our own expectations and we still kept receiving some pretty basic bugs being reported by customers. There were some scenarios where a few customers reported grammatical mistakes or at times, a few of the team members from other teams felt that the messages or the help texts could have been better.

That’s when we started brainstorming different ideas on how to improve our product quality.

We asked ourselves multiple questions:

Should we hire a resource whose only responsibility would be to test the product?

Should we introduce some stringent testing processes?

It was during these brainstorming sessions that we stumbled upon the idea of Bug Bash. Gradually, we incorporated bug bashes into our regime for product feature testing and as of today, it has become an integral part of our Software Development life cycle.

How do we prepare for Bug Bash?

Having done nearly 18–20 Bug Bashes, we have figured out few absolutely necessary things to prepare for a Bug Bash.

Scope Document: We make sure that we have a document that explains in detail what this new product is all about. This document should ideally answer these questions:

  1. What the other features affected by this change?
  2. Which are the most critical workflows to test?

Test Environment Setup: We make sure that the test environment has everything set up. Some things that are taken care of beforehand are: login credentials, data which is either replica of production or of similar nature. We also create credentials for different roles so the feature can be tested with different roles.

Team Division: We make sure that all Bug Bashers are properly divided into separate teams and are assigned different responsibilities. Assigning different responsibilities helps us ensure that maximum product features get tested while eliminating the problem of more than one individual spending time on one bug. Team division is a sure shot way to save time.

Default template for our Bug Bash. We create a section for each bug bashers with few blank tasks. Every bug basher puts the bugs they find in their respective section.

Time: We tend to keep our Bug Bash limited to 1.5 hours. We use the first 15-20 minutes to brief all the bug bashers about the product which is then followed by testing. For features which are huge and are hard to cover in a single Bug Bash, we divide it across multiple Bug Bashes.

In addition to the above points, it is advised to inform all the Bug Bashers well in advance so you would know how many people are going to attend the Bug Bash. That way, it becomes easier for you to plan it well.

What do we do after a successful Bug Bash?

Once the Bug Bash is done, the Product team sits together and figures out what bugs/suggestions need to be fixed before they release the new feature. If some aspects our delaying a release, those are assigned to future versions unless it’s critical and needs to be fixed right away. This way Bug Bash doesn’t delay our release significantly and our product features get rolled out on time with better quality.

Product team after the bug bash trying to categorise the bugs based on priority & other factors

This is how we categorize task after bug bash between 4 different categories. 1) UX/UI issues, 2) Loss of feature function 3) Incorrect or inaccessible data 4) permanent loss of data

How has Bug Bash helped the Product Team?

All the Bug Bashes we have done till now, we were able to find at least 50 bugs each time. Whether it was a grammatical mistake or something major like a save button not working, bug bash has been extremely helpful in detecting bugs quickly.  So if we look back, these Bug Bashes have helped us identify 900–1000 bugs proactively. They’ve also helped us get our team, across different departments, be on the same page regarding any feature or enhancement getting rolled out to customers.

We are still improving our Bug Bash processes and are continuously thinking about how to get the best out of them.

PS: We are looking for a Rockstar Quality Assurance(QA) Engineer. If you have the knack to find ways to break an application & are motivated to lead the Quality Assurance for the product, that is the leader in Alumni Management Software in the US. Then apply here.