Friday, 4 November 2011

Zappers is a global Software Testing Community that holds testing events throughout the year internationally sponsored by TLC and uTest. I have attended their London events in the past and last night event was enjoyable as ever. This is not an event just for networking but doing actual testing on commercially released products. There is stiff competition to register bugs each time a testing window is opened. There are prizes to be won not only by submitting maximum number of bugs but also for writing quality bug reports. Before the testing can begin, the rules of the competition are explained by the organisers. Food and drinks are provided to make the evening satisfying.

During the evening we had two products to test each with test widow of 30 minutes. The testing window for the second application had to be extended by 15 minutes due to wifi problem and router rebooting. The bug statistic is displayed on a VDU screen and continuously updated with the results for everyone to see. The bugs reported from the first application are analysed and approved by the judges before moving on to the second phase. Judges decision is final but it can be contested by providing more information to support your claim but it is hardly challenged. We were asked to do exploratory testing on the applications below. You can try these applications for yourself, they are quite fun to play with. We were warned not to attempt hacking the sites or throw bottles at each others.

Brain Games and Brain Training - Lumosity

Angry Birds Chrome

We were divided into 5 teams and each had responsibility to report bugs on the bug tracking system. If you are new comer then it takes time to get used to the bug reporting system. We had couple of testers who were very familiar with the system so we did not waste time logging the bugs. As soon as a bug it is spotted it was logged straight away. I made few observations about the testing sessions that I find typical in a project.

Each team dived straight into testing with their heads down
We did spend a minute or two to discuss the strategy
It appeared that guys from my team knew what to expect and felt they have been there before. This ignored the fact that there are people who were there for the first time and came to learn
The push was to enter as many bugs as possible with as little details
Constantly watching the VDU display for progress of other teams

My assessment of the evening was that it was good fun testing new applications but there was little time to add useful details to the bug report for the developer to work on. Some of the bugs were supported with screenshots but not much information as to the navigation to the screen where it failed. Our team did not add hardware information with the bug report although OS details were entered. There was not much time for networking due to the seating arrangement.

My experience from BBST Bug Advocacy course is that all relevant information should accompany bug reports. Therefore, someone with little knowledge about the application should be able to reproduce the bug. Testers who report minor looking bugs with navigations and hardware details have their bugs fixed whereas those who report critical bugs with lack of details have their bugs rejected. Testers reporting minor bugs with little details lose credibility with the developers, PM and triage team and are called time waster. Their bugs are not looked at favourably due to the bias in bug fixing process. Testers who contest their bugs if they are not fixed jeopardise losing their credibility if they are rejected by the triage team. They have to provide credible information in the bug report if they are to be taken seriously. With projects constrained by time and resources, PM under pressure to keep the bug stats healthy and developers overworked at the end of the project, only the most important bugs are considered for fixing. It is, therefore, vital that all necessary details are logged with the bug so that overwhelmed developer does not dismiss them.

Pictures from the event