Facilitating Weekend Testing Session WT55

Experience from Facilitation

It was an interesting experience in spite of my misgivings that something might go wrong. I set the testing challenge with a simple web-based application called Corkboard. The weekend testers were a great bunch of participants ready for weekend testing challenge. I had little trouble facilitating the session. In a word it was GOOD.

I had done some preparation such as gathering information about the application from the vendor, prepared notes for the introduction, mission and my expectations from the session. I made a list of questions that I was planning to ask them in the debrief session. The experience report for this session is available here.

Product Features

Corkboard is a web-based application that allows you to collaborate through the use of note pads and a chat viewer in real time. You share the same URL amongst your team to exchange ideas, whether it is to discuss routine project work or results or sprint stand up. We tested the application by logging 10 users simultaneously through various browsers with great success although the application is not limited to 10 users. Corkboard can be used to communicate with remote team members for brainstorming, storyboarding or simply exchanging results from your investigations. The real plus is that you can paste screen shots from your bug reports and mindmap on to the notes. Additionally you can use chat viewer to communicate other ideas that do not go on the notes. The Corkboard is limited to what it can do for you but consensus from the session was that it is a good collaborative tool to liaison with remote teams. See the recommendation below.

Corkboard with Chat Viewer and Note

The main features of the product are:

•   Ability to add unlimited notes and images to a single Corkboard        
•  Real time synchronisation between the notes
• Chat viewer for exchanging instant messages between the users
• The user names are colour coded so individuals be identified easily
• Built-in mini-map slider to navigate to notes that are  off the screen
• Clip and drag notes for organising
• Built-in Spell checker
• Corkboard  url is archived automatically for future use

Overall Mission

Mission was described to test the product and make recommendations to the customer whether it is a good tool for collaboration so that a decision can be made whether to buy the license. The testers were advised that it is not part of their mission to report bugs but it would be a bonus. Any bugs that are found will be reported to the vendor to fix. It was also stated that that the customer is not interested in knowing about ‘nice to have features’ but would welcome any suggestions which would be passed to the vendor as requirements for future releases.  The browser and OS compatibility is as important as the features.

Charter/Session

I expected participants to complete some of the session charters listed below. I observed them during the session by logging on to their URL as they were testing and guided them to test the features that I wanted them to go over. It was an interesting experience playing with notes and images that they pasted and moving them around the Corkboard while they were busy testing. I have added Mindmap of the charters that I created below .

Corkboard Charter/Sessions - A Mindmap

Charter 1

Browser compatibility-Firefox, Chrome, Safari, IE6 -8

Charter 2:

OS Compatibility – Linux, Windows XP and Windows 7

Charter 3:

1. Create a new note
2. Typing on the notes: single user, multiple user, typing and moving notes simultaneously, insert large text
3. Clip and drag notes
4. Paste image on a note
5. Resize notes

Charter 4

1. Open chat window and change name
2. Send messages: simple message, large messages
3. Checking status: Joining and leaving the chat

Charter 5

1. Find note with mini-map
2. Click to navigate to a note

Charter 6

1. Get updates: Submit Email and Not Now (Cancel request)

Multi-user Access

A question was raised in the debrief session about Corkboard limitation if too many users are sharing the same Corkboard. We were a small team and limited time to go over all the features but it would be interesting to explore this option within a large team.  Each team member can write and share his results in real time including screen shots.  As more and more notes are added to the Corkboard, they start to go off the screen and not easy to identify. Therefore, notes and images have to be organised so they are visible to everyone especially when editing same notes instantaneously. You can manually type numbers on notes for quick identification.

Moving the note out of way by one user while other is still typing on it results in losing the text which we found very confusing at times.

Main Limitations

•          No Printing of Corkboard
•          No Saving/printing individual notes
•          No Saving of chat
•          No VOIP
•          No Font changes allowed
•          Limited text message allowed in chat viewer
•          No colour coding of text identifying users typing the notes
•          No Indexing of notes for saving/printing purposes
•          No zoom in/out features available through within mini-map
•     Not able to move the chat viewer around the page

Issues

These are some of the issues identified during the testing:

• A note appears when Corkboard is accidently clicked while trying to clip a note to drag it around
• Registration to get updates is confusing. The users were expecting a confirmation message that the submission is successful
• Chat window distinguished users by colour coding their names but absence of similar facility for notes made it difficult to tell who is typing when sharing notes
• Same names are allowed to be used by different users that cause confusion. It would be nice that an error message is displayed showing the name is taken.
• It is hard to clip a note to bring it to the front unless you know where to hook it.  
• If two notes overlap exactly then it is hard to find and separate them. If notes can be indexed then this problem will go away

Bugs/Issues Reports

Apart from small issues with the browser, no real issues were reported with the current release of the application.The submitted reports by the weekend testers are available through links provided at the end of experience report.

Feedback/Recommendations

I received positive feedback about Corkboard from our testers, they liked what they saw but with some reservations due to its limitations. They recommended it to be a good collaborative tool that can be of great help to communicate with remote team members.  They liked the status on the chat viewer about who is joining or leaving the chat.

Conclusions

I enjoyed facilitating the session and weekend testers practiced their testing skills and submitted bugs/issues report that is available from weekend testing website. The verdict is that all-in-all Corkboard is a good starter for collaborating with distributed teams to discuss issues and project progress.

Since we tested the Corkboard more features have been added to the application. You can now give view-only access to your Corkboard. You can also embed the view-only link on your website along with twitter and linkedin.  The latest from the vendor is that support for mobile devices will be added soon.

Entaggle – An Unconventional Approach to Implementation

About me

I have worked over SDLC for 25 years in various roles predominantly as a BA, system designer, tester and application support. Coding has not been my strong points but kept strong passion for testing not just testing software but also requirements and design. This is the first time I noticed unconventional approach to system design and implementation so felt it necessary that I should express my views for the benefit of designers and alike. The reason for telling you about me is so you know where I am coming from.

How it came about?

As a routine, I was checking my emails this morning (Saturday 5, 2011) when I noticed that Weekend Testing Session was about to begin and I accepted the invitation from Ajay to join. Interesting enough, weekend testers have been thinking about testing Entaggle all this week and here I was about to test this shiny new application. All day, I have been going through my testing results and of others to extract key lessons learnt when I realised that there is something important that should be reported before I report my full findings. I found to my horror that common design and implementation mistakes been made. I strongly feel that these should be brought to the attention for future reference. They are described under 4 separate headings for clarity.

Naming Convention

The mission charter was declared and we were given the go ahead to start testing Entaggle on the staging environment. As soon as I clicked the link provided and signed up for an account, to my surprise, I was logged straight away without having to wait for the confirmation email. For 20 odd minutes I kept touring the site before creating a new tag and successfully used option ‘tag someone’ to tag Ajay. I soon realised that something is not right and reported the problem to Ajay. By that time he had already established that we were given the wrong URL so he gave us a new URL. URL he gave us later was not different from the previous one except prefix www was missing from it.  Comparing the two URL I concluded that test and prod staging URL differed by www, a prefix that a browser would automatically add at run time when you type in the URL window instead of clicking a link.   In our case test URL begins with xxxxxxx.entaggle.com whereas prod staging URL is www. xxxxxxx.entaggle.com. I never seen such naming convention applied in system development. In all our projects, we played it safe by added a prefix test or dev concatenated to the environment name so there is no confusion. I am very surprised that this point was not picked up at an early stage. It is very easy to guess staging server URL from the prod URL, therefore, due care should be taken when naming environments.

Database Synchronisation

The other thing that surprised me even more is the manner staging and production databases are synchronised. If you sign in via the staging server and make some changes then they are visible through the production server and vice versa. I have not been able to establish how the two environments are configured, whether both environments use the same virtual image of the database or both environments are constantly synchronised in real time. If the latter is true then I find this really odd.

Vulnerability

After observing Entaggle for the last couple of days, I am at pain to point out that Entaggle is vulnerable to spams because contrary to my past experiences with systems, the login page is built without any check for users. If there are no checks made at the sign up stage to distinguish between a genuine user and a hacker then there is likelihood that the website can be hijacked very easily. I believe you do need captcha code entering box to establish that a real user is signing up for an account. My recommendation would have been to integrate Entaggle with various internet accounts such as Google, twitter thereby getting rid of sign up functionality altogether thus eliminating potential spammers accessing the website. The latter has the advantage of avoiding the option to change password or requesting instructions for confirmation email and associated validations that goes with it.

Audit Trail

I find it at odds that I can’t see whether audit trail is kept for all the activities a users perform when using the website. It is just possible that activity details are not openly made available to an ordinary user but I find it surprising that it is not traceable considering it is an open system. I can see the same activity being updated rather than keeping a history of the changes that a user makes eg if a user is tagged it appears in the recent activity list and subsequently appears as updated when the tag is accepted. Due to the limited number of recent activities displayed on the home page, it is difficult to tell which of the activities failed to apply. I certainly did not see declined requests making to the list which may be deliberate.

Summary

Some of the points highlighted above are avoidable if careful consideration is given at the design stage. It is very easy to fake environments these days so due consideration should be given while designing systems such as Entaggle. It can potentially damage someone’s reputation which if happens would have serious repercussions. As always the advice is that all potential risks can be mitigated with good planning.

P.S. Although this post was written two weeks ago but I decided to publish it anyway.  Some of the points raised in the Naming Convention Section above have been resolved after exchanging messages between Elisabeth Hendrickson and I.

STC Meetup in Nottingham

How did I find the event?

The Nottingham Meetup was sponsored by STC and organised jointly by Andy Glover and Adam Brown.  This was one of the first events to be arranged through the auspices of STC to promote collaboration at grass root level.  More events such as these are planned in coming months in other parts of the country and internationally so if you are interested in attending such events then register here to get alerts. The event was very well planned and the atmosphere was electrifying with people mingling and chatting away. Due to the nature of the venue, it was rather a tight squeeze but not too intrusive. The evening passed away very quickly with welcome interruptions from lightning talk speakers and James. You can say Nottingham Meetup was a great success where everyone enjoyed and made new friends. You can find Adam’s write up for the event here.

Lightning talks

The lightning talks kicked off in high spirit with few technical glitches. I disagree with Adam assessment that the talks were not technical enough, I know they were short on content but they had some technical content worth noticing.

Selenium - Gemma Cameron

Gemma kicked off with her theme on Selenium IDE that she has been using for UI testing of web applications that she thinks complement her skills in manual testing. She did a short demo on how to use IDE to capture script which kind of worked but she made her point about the value of Selenium in building tests. She briefly covered Selenium RC and showed how a captured IDE script can be exported in other programming languages and changed into methods for running tests using Selenium RC. I think it was a well chosen topic and good introduction to those who are unfamiliar with Selenium. I noticed she writes about Selenium on her blog as well.

If you want to learn more about Selenium then visit the official website and get hold of two books published recently that covers the topic very well. They are:

Selenium 1.0 Testing Tools Beginner's Guide by David Burns

Selenium Simplified by Alan Richardson

Micro learning - Adam Brown

This is one of the topics that have been bandied about recently. According to Wikipedia
“Micro learning deals with relatively small learning units and short-term learning activities. Generally, the term "micro learning" refers to micro-perspectives in the context of learning, education and training. More frequently, the term is used in the domain of e-learning and related fields in the sense of a new paradigmatic perspective on learning processes in mediated environments on micro levels.”
One excellent use of micro-learning is to learn something daily - or at least very regular - bites of "learning" content. This kind of drip-feed learning is very useful for gradually building a body of knowledge.

 Adam spoke about Micro learning technique that helped him learn in small chunks for better results. He admitted that he has a very short attention span and this method is ideal is suited in his particular case. He believes if he is not learning then he feels trapped in a rut like everybody else in his situation. He is of the opinion that education should be something that is left to individuals and not imposed by the corporations. He listed a number of ways of one can learn but they don’t seem to work for him because of his short attention span. James picked this point about Adam’s admission in his presentation and explained that Adam is actually learning all the time because ideas constantly popping in his head. He may not be consciously thinking that way but his mind is constantly churning ideas which he can capitalise. Adam believes that if he picks a topic and spends even 15 minutes a day that adds up to a substantial amount time for learning a week. More here

Do Personas Play Part in Testing? – Mohinder Khosla

You can find YouTube video here and the write up here.

Software Testing Alone – Paul Berry

You can find YouTube video of his presentation here. He has also done a write up which is available here.

James Bach Presentation

You can watch full length video of James presentation below; make your own assessment and what message James is sending to testers about self learning and critical thinking. I am sorry to point out that last 3 minutes of James’s presentation is missing from the video due because I run out of space on the tape. I could not change the cassette quickly enough but at least there is enough for you to get the message. Here are some of my thoughts from his presentation that I find useful.

Self Learning

In his presentation he emphasised the importance of learning and made reference to his book on The Secrets of a Buccaneer Scholar: How Self-education and the Pursuit of Passion Can Lead to a Lifetime of Success. For him learning is a life style, he constantly thrives on learning and does not set a time slot when he should be learning new stuff. His advice to others is don’t limit yourself but constantly keep learning new stuff, carry a notebook all the time and jot down ideas that pop in your head and that’s what he does.

Practice memory

He spoke about how to practice memory and how to practice what he learnt. He took an example that hit him from the book he picked up from the Museum of Natural History called The Playful Brain. A sample chapter is available here if you like a preview. He believes he has to read scientific theories to defend himself from people who question the approach of exploratory testing saying there is no scientific proof it works. He stresses that you can learn from such research and apply that to testing.

Learning Through Play

He gave the example of a study on rats and their playful behaviour where rats trying out different things to find out what works. He advises that we should learn our art in a playful manner. We should try various things, challenge each other with silly games so we can work with the stress and learn to process it so we can deal with real situation more effectively.

He gave another example of Gerald International Airport parking calculator where someone spotted a bug in the system. So he threw a challenge to everyone to report the highest amount of money they will be charged and post the results on twitter. This allowed people to share their best tests; it was like playing a game and sharing information which is very important in our environment. The model of working as a team works better if you work as a team part of the time and rest alone.

“Message from these types of study is game play helps you deal with stress in real situations.”

Learning Through Challenge

He spoke about his method of teaching by putting people under stress and challenging them. He described his dice game as a good way of learning about test design. He says he observes what you do and what you don’t so that he can find out how best to coach you.

James gave a challenge to all of us to test a calculator which he claimed he dropped and gave a hint that perhaps it may have been damaged. James picked someone from the room and told him how would he test it. James questioned his ability to come up with a solution so he constantly poked him to think about possible scenarios and its context. He opened up the debate and we all spent good deal of time on the challenge. James pushed us to think through the context and get clarification before diving into testing. One of the things he pointed out about making assumptions, which to him is opposite to testing, is where to draw the line when an assumption is reasonable and when it is critical. We spent good 20 minutes learning about his teaching technique that he practices when training testers on his Rapid Software testing course.

Critical Thinking

He also spoke about critical thinking heuristic and asked us the question how do we approach critical thinking but got a muted response. He introduced three words that he uses when he teaches critical thinking and they are: Huh? Really? And So? Each one of these boost you in different directions and according to him indexing

Huh?  Part tells you do you understand what’s been asked.  I would call it an element of surprise. This is a starting point for you to question the assertion what you have been asked that means do you understand the situation. To me this is the What state.

Really? – This is where you start questioning the facts given to you and think about the reality of the situation. I call it Clarifying stage and to me this is the Why state.

So? - Here you try assessing the assertion whether may be it does not matter. This is where you start to think who matters and who you are solving the problem for. I call it How state.

James thinks this is a simplistic approach to teaching critical thinking that works for him. He suggests if you start following it then after a while it gets easier.

Wrapping Up

He spoke about TDD and advantages of pair programming from his own experiences with the techniues. After making in roads through assumptions, critical thinking and fighting rats, he concluded with a nice little story which is an extract from the book called Steps to Ecology of Mind: Collected Essays in Anthropology, Psychiatry, Evolution, and Epistemology by Gregory Bateson. James says he uses this all the time.

Video: James Bach Presentation at STC Nottingham Meetup

His message to testers is success will follow if you work hard, learn well and create your portfolio. If you find something you are passionate to do try very hard to do it and do it.