Skip to Main Content

Who Tested This Thing? Twitter Facebook Print E-mail
Written by Lorin Romrell   
Thursday, 15 May 2008

“Who tested this thing?” is a question we hope the customer never needs to ask. Software should just work, without explanations and without workarounds. In the movie My Big Fat Greek Wedding, the bride’s father carried around a bottle of Windex, which he claimed was the cure for all kinds of physical ailments in addition to being a great cleanser. When presented with a problem, he would say: “Just spray some Windex on it!” There are some in our industry who share this notion when it comes to testing software. It’s all too easy to just focus getting a project “completed” and then “spray on some quality” at the end. I’ve never seen a good outcome from this approach. The results are very painful for the customer and breach the very trust we are striving to build.

It’s been great to be a part of the evolution of Quality Assurance in ICS. In the past, our numbers were so small that we only “sprayed a little QA on” before projects went to the customer. We realized we needed to grow to be successful. This growth happened very rapidly and often miraculously, but the results have been excellent. We have been careful in how we scrutinized potential candidates for both the best skills and the best team fit.

As we grew, some teams welcomed the addition of QA resources and experienced enough benefit that they soon asked for more! Other teams were reluctant and asked: “Who are these people? What do they really think they can do to help?” Their reaction was not surprising since many teams had little or no exposure to QA. As we grow, QA is no longer an afterthought. ICS implemented basic changes that standardized defect tracking, identified user stories, introduced automated regression testing, started methodical approaches to product deployment, required continuous builds, and implemented many other fundamental building blocks for building quality software. QA was not solely responsible for all of these improvements, but as a larger ICS organization we have been driving to higher standards and greater quality. The organization now relies far more on our QA group. We now have a software development process where QA engages early in the planning stages and remains engaged throughout the entire project. With this early involvement, we are able to reduce project risk. It’s amazing just how fast a vague requirement is clarified when you think about how you will test it. Since many projects rely heavily on test automation, this early involvement helps us make critical design decisions up front, saving time later on. We have a strong working relationship with our peer groups and this continues to be critical. We work closely with interaction designers in reviewing prototypes. Many issues are resolved in the design, before coding even begins. Software developers have also engaged with QA to make applications more testable. We’ve found that working in close proximity with our peer groups helps all to identify issues quickly and solve them early—while it’s cheapest to solve them.

Now that we have some of the basics in place and are engaged with all of the project teams, there are several areas where we need to improve. We need to increase the rate at which knowledge flows through our QA organization. For example, if one team has a more efficient way for performance testing a product, we want to share that with others. We need to further standardize and centralize our metrics to provide meaningful data to our entire department. These metrics and reports need to monitor every critical step in our processes. The information should be accessible and usable from the top of the organization down to individual projects and teams. We need to expand our testing to help build in maintainability. We need to better understand scalability and release solutions that will grow with the Church. We need to increase our vision, our productivity, and our capabilities to fulfill the needs of the Church.

As we improve in these areas, we will be able to throw away the Windex bottle for good.

Lorin Romrell is a quality assurance group manager for the Church.




Learn how to become a full time or part time Missionary.