Sunday, 27 February 2011

My Lightning talk at Official STC Meetup in Nottingham!

Do Personas Play Part in Testing?

How I entered IT?

This was my personal story of 25 years in IT packed in 5 minutes. I started by telling where I was 25 years ago and how I entered the IT profession. My career as a space scientist while working on Space Shuttle Programme for the European Space Agency was cut short when I had to leave the profession after the Space Shuttle Challenger’s disaster in 28 January 1986 (Slide 2).  At the time, IT looked like a good option to choose as a profession but required considerable upfront preparation. I did not take the decision lightly and enrolled for MSc degree in Systems Engineering before entering the profession full time.

Started as a Business Analyst

My career took off and I became a Business Analyst although still called Analyst Programmer.  In my first job, I landed on a waterfall project. The first couple of projects (big and small) were like this, working through the whole development lifecycle using SSADM methodology. I spend a great deal of my time on V-model which was the basis of my talk.

While working through SDLC on a number of projects, I was sub-consciously building persona of the customer at the requirements stage that I used in acceptance tests. When I was working through the system design stage, I was linking the persona to the design through relationships to the attributes. This carried through system and integration testing providing a valuable link to the requirements.  This helped me in the latter phases of the project to link tests to requirements at subsystem level and an easy way of reporting system status during the course of the project.

In later years, I realised that waterfall is not very popular anymore and project after project adopting Agile approaches. I believed that the V-model collapsed to I-model turned flat and used for Agile SDLC approaches such as Scrum (Slides 7/8).

Creating a Persona

I picked the theme of creating persona by using example of CRM-R-US Requirement Document presented to us by Darren McMillan during Weeknight Testing 4 on 26 January. In that session, I collaborated with Lisa Crispin by sharing my desktop via Team Viewer. The approach to share screens was not a great success but we carried on irrespective. While Lisa was drawing up a mind map I was thinking more on the line of personas. Part of Darren’s requirements are listed in Slide 10 and the corresponding questions for interviewing him for creating a persona on Slide 12. I pointed out in my talk that slicing stories (Slide 11) from the requirements would look bare without the persona. It is industry practice to label each story with some kind of persona during the story mapping stage to provide better focus on development.
There was no time to actually create Darren’s persona or seek help from Andy Glover (@cartoontester) to draw up his sketch but the benefits of personas were highlighted. I am hoping in the coming weeks I would be able to have 1:1 talk with Darren and come up with a sketch.

Benefit of a Persona

Benefits of personas are all too clear from slide 15. It can add another dimension not only in the story mapping or acceptance testing stages but also in the system, integration and exploratory testing stages. If done properly, the tests from each stage can be linked via the persona tags. If these tests are stored in a single repository then the results can be extracted to produce status reports on demand.  If multitude of tools are deployed on your project then output from each phase can be linked using ALM tools.

People who influenced me

I did not realised that I have been using personas till I came across the work of Jeff Patton and Gojko Adzic and started applying vigorously to testing. I firmly believe that using personas in testing is a powerful approach that can help testers focus on test design more effectively.


My thanks to everyone attending the Nottingham Meetup and STC for sponsoring the event.
Slides from the talk are available here

Thursday, 17 February 2011

WeekNight Testing Take 5


I had a quick exchange of words with Sharath (facilitator) before the session suggesting I will be joining late and got the reply ‘don’t b too late...because we have a very interesting session’. It indeed was an interesting session where all of us were as enthusiastic as the facilitator to see everyone enjoy testing the applications. Unfortunately, due to personal difficulty, Sharath was not around for much of the debriefing period to see the climax of the session when Darren McMillan kindly stepped in and kept us well informed in his absence.


Sharath Byregowda (facilitator)Martin Jansson, vamshi, Lalitkumar S. Bhamare, Stephen Hill, David Liebreich, Oliver V., Mohinder Khosla, Chris Thomson Philip, Adam Yuret, Kavitha Deepak, Darren McMillan, Del Dewar,  Richard James Hill,  Sudhamshu Ailineni, Brian Osman

About Mission Charter

The session started in good humour, as ever, with everyone hoping to make the most from the session. The facilitator was sure that no one will be disappointed with the missions that followed and it would give everyone the opportunity to collaborate and discuss results openly.
Mission 1: An addictive fun cat game to Circle the Cat. This is a game that you can play by clicking green dots to stop the cat escaping the board.

Mission 2: Find bugs with Horse Dressage game


As testers, testing are one part of our job but we are also expected to investigate research and provide information to our stakeholders. Today the mission is to investigate, analyse and find a solution and yes find bugs on the way

If you meet the first mission within the hour move to the second mission

These are two contrasting missions, the former expecting an outcome of a solution to solve it whereas the latter was more like finding what’s wrong with the application.

Thinking about Testing Approaches

Whenever I approach testing I ask myself the obvious

1.       Do I understand the mission?
2.       Do I have enough information to start the mission?
3.       Do I need to speak to the customer, developer or BA to clarify requirements?
4.       Do I need any tools to record the test session, are these tools flexible enough to manipulate the test output meaning results for presentation?
5.       Does the application resemble anything like I have tested before?

There are two other things that I consider as part of test strategy to mitigate risks.
  • Create a Persona(s)

Whether I am doing manual scripted testing or exploratory I make a mental picture of the user who is going to be using the application that I am testing. I speak to the user to build his profile and establish

·         what kind of user he is
·         what role he plays
·         what level of experience he has with similar applications
·         what are his pain points

Based on this information I draw up a mental sketch of the user that I impose on the tests I plan to do. If I am doing scripted testing then in the title section of the test I record the name that I have given to the persona. If tests are written in an automated tool or Excel then each test is prefixed with the persona name. So when it comes to analysing the results I can run a query to bring the results in a report to show the status to the manager or the user for a particular persona. In the case of exploratory testing I do the same tag each test with the persona during the testing session. In the current mission I gave the name ‘gamer’.

Personas are normally created at the start of a project to tag each story with the user during the story mapping stage. Similar approach is applied when writing acceptance tests to help build live documentation that Gojko Adzic advocates in his coming up book on Specification by Example.  I have been using personas for many years without mentioning the word persona to anyone till now. I am pretty sure if you start using it you can see the power behind the simple concept.

  • Think about Heuristics
This is also very important aspect of exploratory testing to identify the heuristics that may be applicable in a particular situation. I tour to build a model of the product before deciding which heuristics would be appropriate that forms part of the tests.
When deciding a heuristic for mission 1, I was thinking of using map making heuristic but it turned out that I was actually applying trial by error heuristic instead.
In the case of mission 2, the choice was pretty much simple for me given the features of the game that follow a sequential order. I chose to use OFAT-one factor at a time ie changing the colour of the horse and clicking each menu option at a time.

Stakeholder Collaboration

One of the principles of Agile Manifesto is Customer collaboration over contract negotiation. I believe that success of any project depends on the active involvement of its stakeholders during each stage of the project. In our case we felt that such interactions with the customer were missing to some extent although there were plenty of co-operation from other testers on the mission which is a good thing. We had a number of pertinent questions that were not addressed as we progressed through the session. In order to meet the objectives of the mission to investigate, analyse and present information to the stakeholder, we felt that stakeholders co-operation is necessary if we are to be successful complete the mission.

In such situations, at least we should have been told that stakeholders would answer all queries related to the mission during the investigation stage if available. We felt that the information from the stakeholder was not forthcoming when it was needed that resulted in us making assumptions by looking back at our own experiences from past projects or agreeing among ourselves what should be the right answer in such situations. This, in my opinion, is not way the investigations in real projects are conducted. Full co-operation of the customers is vital in reaching right solutions through investigations.

Questions Asked Assumptions Made

In any projects there are bound to be questions from testers before, during and at the completion of testing phase. Questions such as

                Who is our stakeholder?
              What is the scope of testing?
            How the results will be represented and reported and to whom?
          How the results would be interpreted and used for?
        What information should be included from the investigation?
       Which of the missions would add high value and has high priority?
      Whether it is more important to just collect information or 
     timely information?

Would you assume that our mission facilitator is our stakeholder unless it is conveyed to you? I don’t think so. The mission was to ‘circle the cat’ and to investigate, analyse and present information about the product. We were desperate to find out whether we should complete mission 1 before moving on to the second mission or carry out deep investigative analysis for mission 1. We assumed that it was mission 1 objective to figure out patterns for the product and come up with a simpler solution. I didn’t think that was the right decision but we did not have much to go on.

Mission results

The key aim of the first mission was to encircle the cat but it was not clear what information is to be presented to non-existent or so called obtuse stakeholder. The mission statement expected information from the investigation and analysis of results with no idea how it should look like and what the content should be. This left us pondering when to finish with mission 1 and move onto the second. We felt whether we have done enough justification to mission 1 before continuing to next mission. The question came up whether spending too much time on one mission would jeopardise the second mission. In reality there were no bugs to be found from the first game so we were keen to jump straight into second testing challenge in order to find out whether it would be more fun and better use of our testing skills to report bugs.   

Circle the Cat

All of us approached circling the cat in different ways. To lighten up the atmosphere one of the participant even suggested photocopying the game board and drawing a circle around the cat with a pen. We were looking for a pattern to emerge but could not find one. There was consensus on the approach that we were all taking ie if the reset key is clicked multiple times to bring up at least 10 or more green dots on the board before attempting to circle the cat, we have a better chance of success. After couple of tries, using trial and error, I came to the conclusion that if you have the dots evenly scattered over the board except one corner have more dots than the rest you could corner the cat to the densely populated dots to trap it much easily. Others tried to draw a wider circle to achieve the same result. As one says there is more than one way to skin a cat and it turned out to be true as shown by the images below. You can watch the full sequence here. One of the sequences is shown here. If you want to follow up the discussions about the game then use the link.

3 Patterns of Circle a Cat
Horse Dressage

We needed clarity about this mission whether the goal is to find just bugs and not to report design or other information to the stakeholder. In spite of this confusion we carried on regardless. It was also not obvious how we should report bugs that we find. In other weekend testing sessions we are advised to use tools and techniques at our disposal and sometimes there is a mechanism to log bugs as well. I believe, for clarity or completeness, we need a similar system for week night testing to all tracked bugs and issues that are reported. This would help in providing feedback to a potential product owner if it is part of the brief.

Although, we discovered some bugs with the application but were not able to log them and present them in the debriefing for discussion. We concluded that the application is so buggy that it failed QA testing and should be returned to the developer. The captured video recording is shown here.

Domain Knowledge Matters

During the course of testing, we experienced that some domain knowledge would have helped.  This certainly would have helped construct better tests rather than firing random tests to explore the application. For example in the case of horse dressage(see the definition below), horse person would have spotted that there is no collected/extended canter offered, therefore, would have connected it to missing features, design fault or even inaccessible features. The other interesting thing that a horse person would have picked up about the movement of horse that it should be linear then walk, walk before trot and trot before canter. This shows domain knowledge has added value.

the dressage is occasionally referred as ‘Horse Ballet’ whose purpose is to develop a horse's natural athletic ability and willingness to perform.
It is a flash game that displays some of the dressage routines. If you are not used to testing such games then you might initially struggle before getting used to but it is not considered a handicap.

Message Got Lost in Translation

We were making all sorts of assumptions and draw conclusions of our own whether one should find out the best patterns to the game or a single solution that we all agree. Some of us thought the mission 1 is a puzzle while others said it a game. The message to complete mission 1 before moving on to mission 2 was not conveyed properly. There was nothing in the mission to identify patterns and understand how you circle the cat. Although mission clearly stated that it is to ‘circle the cat’ but still some were calling it trap the cat. These two contexts might sound similar but they are not. The dictionary meaning of circle is A plane curve everywhere equidistant from a given fixed point, the center whereas trap means to prevent from escaping. I wonder how many of us exactly circle the cat by definition because I saw no evidence of that. 


Both weekend and weeknight testing provide great opportunities for testers around the world to work together and share their experiences through the medium of social media. Testes working together pick up excellent tips from each other which they can apply to their own workplace. It not only gives testers exposure to solving problem skills that they would not have otherwise had. Some of the lessons learnt from the simple problem solving exercises above and discussing the observations openly with others give you more confidence that is crucial to your self-learning. You feel you are among friends who can guide you through issues that may help you in your future testing assignments. By working with guys whom I have not met boosted my moral to go out and explore new territories which I was reluctant to do before.  

My thanks to all participants who came up with innovative questions and made constructive contributions that made this report possible. Please feel free to comment for the benefit of others and add bits that I missed and you feel strongly should have been included in this report.

Tuesday, 8 February 2011

Welcome to Weekend Testing Session No WT52

Why Choose Virtual Keyboard? 

Hindi Version of Multi-lingual Virtual Keyboard
To Compare and Contrast Frontype virtual keyboard with standard Windows keyboard.

At the end of the session, fictitious DSC Organization would decide if it needs to buy Frontype license or stick with status quo. You are free to use any tool or technique or pair offline to achieve your mission. As usual, at the end of the investigative period of an hour, the group meets for the debriefing session to discuss how we approached the mission and what learning, mistakes or any other interesting experience we had.

Ajay Balamurugadas (facilitator), Jahira Banu, Vamshi, Sindhu (UAE), Balu and Mohinder
We started with 5 participants excluding Ajay but Vamshi had to drop out because he could not load the application due to antivirus program on his machine found Trojan horse and cancelled his installation.

Load issues
Participants were using a mixture of Windows XP and Windows 7. They all had some issues with loading but managed to overcome them to continue testing. For me, it took time to find the right unzip program but once the application was installed, I finished the tests very quickly and had some time to spare to go over the finer points.

About Frontype
Frontype is a real-time typing tutor with semitransparent on-screen keyboard. It reflects all your operations with physical keyboard and can be used for simulation of any national keyboard layout. It looks like a grey transparent film over the screen but can be customized by changing size, transparency, color, key sets and configured to appear in certain active programs such as Word (see illustration below)

Settings Menu
 System Requirements
- Windows XP/ Vista
- Pentium 4 / Athlon XP or higher
- At least 256MB of RAM
- 35 MB free space

It is available in two versions with / or without Microsoft .NET Framework. You can have a number of language layouts but you have to include them into windows. The settings can be changed using regions and languages options from the control panel.

 Approach to testing
Planning is probably the important part of any activity even for the smallest or non-trivial task. It helps you focus on the key objectives to achieve your goals in the allocated time. The application at first glance looked simple but you would not know the complexity till you actually tried it. I planned the steps below to learn about the application, apply heuristics as I feel appropriate and repeat the process till I am done.
  • Ensure that virtual key board matches the physical keyboard in all respect
  • Default presets can be changed using settings options
  • Check all settings are saved and permanent (restart the computer if necessary)
  • Check for any hidden features not so obvious
  • Usability Testing
 Features Working
 All keystrokes except those described in the next section worked as expected and the layout matches character by character with QWERTY keyboard. Setting options worked as expected without a glitch. Keyboard looks good from user perspective and can be customised to suit individual requirement. No problem adding programs. Keyboard pops up when these programs are active. I did notice word shortcut character appearing in the menu section underneath each menu option when pressing ALT key which I completely missed in the first instance. 

Features not working
The drawback of virtual keyboard is that selection of characters above non-alpha keys is not possible when shift is clicked.  If the same key is depressed on the QWERTY keyboard, these characters appear in the virtual image.
The above keys appear to be inactive for some reason. The windows key meant to open Start Menu whereas the key when depressed should display the selected item's shortcut window mimicking action of right-clicking the mouse.

Missing Features
The keys below are not available. As we found out later, virtual keyboard is designed as a teaching aid to practice typing without looking down on your keyboard. Finding hidden features was a bonus.

Functions keys (F1-F12), Home, End, Page (Up/Down), Insert, Delete, Esc, Arrows (Right, Left, Up and Down) and Prnt Scrn

Benefits of Frontype
1. Teaching Aid: Frontype is not promoted as a virtual keyboard as we led to believe at first thought. The application is designed to train your fingers by not looking down at your Windows keyboard when typing. If you are two finger typists then this would be quite a handy tool for you to improve your speed.

2. This can be used by people of all ages and with multilingual support built-in, it can help you type in other languages if you ever need to do that. The application would be invaluable to those who have migrated to other countries and want to encourage their children to learn their native language.

3. Virtual keyboard can also be used as a normal keyboard in spite of limited support for Control and Function keys. Since the weekend session ended, I learnt a bit more about MS Word shortcuts using this application that I never bothered to look up. In my opinion this is quite a handy feature that you can be invoked with one mouse click. The Illustration below shows shortcut characters appearing underneath each menu command for Word 2007 at the click of ALT key. When P for Page Layout shortcut is clicked you see another set of shortcut characters appearing below to other options. I think this is cool and confirms that the application can be an excellent teaching aid to help you build your confidence in typing.

4. If you are lucky enough to own touch screen monitor in your office or home then you may be able to use it as a keypad just like your iPhone or Blackberry (this function was not tested though).

Word Shortcuts
Awkward features
It is best to leave the keyboard in the float position in order to resize, drag or minimize so that you can happily continue with using the editor at hand. Virtual keyboard does not allow you to click the text or any pop-ups that appear beneath the keyboard therefore you have to drag it out of the way to continue typing. If left in this mode and you navigate away from the editor, it pops up in the other window although the program has not been added to the list of programs where it should be active. There is a work around to force the keyboard not to appear in other windows. For that to happen, it has to be set to a locked mode. That means when you return to the editor, you constantly have to unlock before you can use it effectively.

Lessons learnt

Be prepared
Before the mission starts do your homework. We were asked to download the application in advance which I did. As soon as the session started, I realised that I was going to need unzip program to uncompress the files before it can be installed. This resulted in me spending 20 min searching the web to find the free download program to unzip the files before finally installing it. Although, I finished the tests very quickly and had time to spare to go over the finer points but lessons have been learnt.

Other participants were using a mixture of Windows XP and Windows 7 and they also were not prepared before the session. They all had some issues with installation but managed to overcome them to continue with the testing.

Had every one was prepared then we would have more time to test and experience the application and write report properly.

Keep an eye on the ball
While I was busy finding a program to unzip RAR file, the facilitator decided to switch the participants from Skype to for rest of the discussions. I was expecting everyone to be using Skype but that was not to be the case. You could easily miss key points from the discussion if you are not alert and keep up with the updates from the facilitator.
This is a handy little communication tool but it has its limitations. Only one person was allowed to present his test report at a time to control the flow of discussion unlike Skype where everyone is jumping in and expressing their views constantly. I think by using you are limiting the scope of the discussion because while one person is presenting others are just spectators. You may be keen to jump in with your comments but you have to wait for a cue from the facilitator to put your case forward. This results in losing lot of valuable time from the debriefing slot.

On the same Page
If you expect everyone to be productive then make sure everyone is familiar and access to tools used for communication before the start of session. One of the participants was from UAE who did not have access to Skype because it is blocked in his country. On the hand I had no prior experience with and had trouble trying to catch up with the discussion and at times not sure where it is taking place in the live document. I was looking at the end of the document whereas the conversation was happening somewhere in the middle. Change of colour in the text made me realised where I should focus my attention to be part of the conversation.

At the end of first hour, I submitted my report via Skype but others did it via dropbox which I was not aware of. Therefore, I did not view reports from other participants report and not sure whether they had seen mine.

Rapid Reporter
In a rush to install the application, I completely forgot to use Rapid Reporter to record the session. This would have very helpful in capturing more information about the test session with screenshots to support various scenarios.

Heuristic Applied
I subconsciously applied comparison with Standards heuristic for testing but did not realise till Ajay pointed out to me during debrief. I guess it becomes a second nature as you get experienced in testing but it is a good idea to remind yourself from time to time as to the approach you are taking and why. The heuristic is explained very well here if you want to follow this up.

Objectives Met
Key objective to Compare and Contrast Frontype with standard windows keyboard was met by all with a degree of disappointment due to time constraint. Debriefing notes show that everyone had a reasonably good crack at the application and they all approached testing from different perspective depending on their experience. Those who were first timer had most to gain from more experienced participants. People had good experience from the session and showed enthusiasm to come back for more. Everyone learn from the experience and they were all happy bunny. The session was a success in the end.