ross beazley

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

I unveiled a new exploratory testing mnemonic I use for testing mobile apps at Star West last week. I adapted it from James Bach's SFDPOT, but with a special focus on some unique challenges mobile apps can provide. Several people who saw the talk asked me to publish it so they could use it with their teams, and here it is: Test Mobile Apps with I SLICED UP FUN!.

I hope you find it useful.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

I wrote an article for EuroStar last year about creating value with testing. Some of you have asked for more specific ideas about determining whether your testing is creating value or not. In the article, I talk about getting feedback from stakeholders, but that isn't always easy or possible. One of the most important stakeholders on any project is you, so how do you go about satisfying yourself with your testing value?

The easiest way to get feedback is from other stakeholders. What does your manager think about your testing? How about the programmers, business analysts and customers (users) of your software?

The hard part with that answer is you may not be able to talk to all of those stakeholders. Or, they may not know what good testing looks like so they won't have answers that satisfy you. In some cases, the stakeholders around you may have such low expectations that their feedback might not help you at all. They may expect you to provide testing work that you might consider shoddy and negligent. In that case, you have to show them what great testing looks like. When that happens it's like graduating from a cheap box of wine to the good stuff. Once they've tasted the good stuff, it's hard for them to go back to expecting poor testing.

Even if you have good direction from other stakeholders, I recommend asking yourself some questions to help determine if you are creating value or not. This is hard to do, and will result in work for you over the long-term, much like personal growth endeavours. Don't expect quick fixes, but if you work in these areas over time, you will see changes in your testing. Here are some things to think about:

• Is my testing work defensible? (Cem Kaner talks a lot about this.) Think of a court case. What would a jury think if you testified and described what you did as a tester and why. How did you determine priority? Why did you test some things and not test others? (100% complete testing is impossible, so you have to make decisions to optimize your work. Are those decisions well thought out, or more subconscious? What sorts of things might you be missing that you haven't thought of?)
• James Bach talks about how important it is to have thought out and varied approaches to testing. What kind of approach do I have to testing? Do I consciously choose to have a varied approach using as many models of coverage as I can to discover important information about the product? Do I make the best use of tools, testing techniques and management approaches that I can? Or do I just do what the programmers or someone else tells me to do?
• In the absence of getting real feedback from real people on my project, what would happen if a well-known consultant came to visit me? Could I answer their questions about why I chose to test this way? What kinds of holes might they spot in my thinking? Would they see weak spots? More importantly, would I be proud to have Cem Kaner or someone else I look up to see what I actually do? Have I used ideas from testing thought leaders in my work and found out what works well for me and what might not work so well? Could I communicate my work to an expert outsider clearly and thoughtfully? If so, what might they think?
• Do I adapt my test plans and strategies from project to project based on the risks and rewards our project environment has at a particular point in time, or do I just copy and paste what I did last time, and repeat the same thing over and over?
• Do I track down and find repeatable cases for important intermittent bugs, or do I just file them and forget about them?
• Do I feel energetic, creative and proud of my work as a tester, or do I just feel like I am doing the same boring things over and over and filling in paper work and forms to please a manager?
• Can I look at a released product and identify ways in which my testing has improved the product experience for our end users?
• Do others on my team feel better with me around? Do they miss me and my creative input when I am away, or do they welcome the break from my negativity? Do they request that I work with them on other projects?
• Is my testing service in demand? Am I the person team members come to when they need help solving a particular problem that I am really good at helping solve?
• Am I aware of other approaches to testing that challenge my favorites? Do I understand approaches that I may not favor or I may even dislike, or do I just dismiss anything unfamiliar and threatening out of hand? Do I have an open-mind and look to challenge my ideas in testing to help improve?
• Am I learning about different ways I could improve my work? Am I aware of recent changes in testing techniques and tools? Do I know where to find information to learn from?
• Do I consistently try to do better than I did last time?

These are the kinds of questions I ask myself regularly. I don't always have the best answers to my own questions, but as time goes on, I feel much more confident about both my own answers to those questions, and more importantly, the value I know my testing work provides.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

My latest article on exploratory testing is in this month's issue of Better Software magazine. Check it out here: Demystifying Exploratory Testing. The references are available here: article references.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

I wrote an article for Modern Analyst that describes some of my process thinking over the past several years. They asked for a piece on post-Agilism, but I prefer talking about value now, so I wrote this piece for them instead.

I introduce my thoughts on a trend I have witnessed for a while now where people move from software ideas (let's build this killer app!), to process (let's go Agile!), to value (let's ensure our application blows our customers away, and everything we do feeds that effort.)

Three years ago, I didn't hear people talk about value that much at all. It was all process, process, process, and how following an Agile or other process would lead us to success. Now, I am seeing the "value" word pop up more and more, and more teams are using an overall vision to help focus their efforts (process and otherwise) towards creating value for their customers and themselves.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

Last month I wrote a piece for the STAR Tester newsletter on testing and value. You can check it out here.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

I wrote an article for Automated Software Testing magazine on test automation politics for the November issue. The magazine is here: magazine download page and the PDF is available for direct download here: PDF Download. My piece starts on page 20.

Article back story: I had a nice dinner with Dot Graham and Dion Johnson this spring. As we were sharing automation stories, Dion asked if I would be interested in writing an article on politics and test automation. Since most automation writing focuses on tools, results and ideals, there isn't a lot out there on the social aspects of automation. I thought I'd provide my take on some social challenges I learned about the hard way. I was inspired by Bach's Test Automation Snake Oil. Bach doesn't pull punches, and I tried not to either. I've often thought that it would have been nice to have been advised on potential controversies when I was starting out in test automation, so we took a handbook approach with the article.

I hope you enjoy it, and if you're starting out in test automation, you find the article informative. If you're currently facing any of the resistance I outlined, I hope you find the content encouraging.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

I posted about fuzzing a few days ago, and I think the tools are neat, and in the hands of good testers can be powerful. They are a nice way to augment existing security testing, to test data transfers or messaging, or to simply generate test data. However, some of my readers basically said: "Big deal. It's a neat toy for you, or some of your clients who have the time for that sort of thing, but why would I want to use one?"

The Register has posted an interesting article where fuzzers were used to discover potential security holes in XML libraries. Check it out here: XML flaws threaten 'enormous' array of apps. If my babblings about fuzzing don't get your attention, maybe this article will. The potential to use fuzzers (in conjunction with other security tools and techniques) to help people catch problems that bad guys will exploit is enormous.

Codemonicon, the company cited in the article, have a lot of interesting information and expertise in this area. They have a nice introduction to fuzzing on their website. This paper by Rauli Kaksonen has a lot of technical detail on fuzzing if you'd like to learn more.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

Back in June, I presented a keynote at the Better Software Conference called
"What's More Important: Being Agile or Creating Value?" SQE have posted the video.

The video is just over 50 minutes long.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

Here's a scenario: a programmer friend is creating a new web-based product for his company. He wanted someone to augment the existing testing that was already underway, and hired me to add more firepower. It is a distributed environment, which means that everyone in the office telecommutes. We decided to start with me doing an initial session, via instant messaging. We blocked out a time for the session, determined a focus ("mission") and I started testing.

As I tested, I typed what I was doing in the instant messaging (IM) client, along with questions, concerns, crashes and bug ideas. He followed along through the application as I tested, asked clarifying questions as I discovered problems.

After ten minutes, I had generated enough ideas for that mission. He logged 6 bugs in his fault-tracking system. I had discovered 5, he had discovered 1 himself while following me along and monitoring the server and database.

I've done session-based testing over IM quite a lot with different distributed projects, like open source projects such as Watir and Session Tester, and with some of my clients that use telecommuting, or are completely virtual. We've used and adapted some of the ideas from SBTM to help provide more structure around the actual test sessions.

Using session-based testing with IM seems to work best in pairs. Once a focus and rough timebox are determined, test setup is out of the way, both pairs initiate a session through an IM client. One drives by testing, and the other saves the conversation locally, and creates notes, logs bugs, investigates underlying behavior, etc.

This style is natural (we communicate a lot with IM clients anyway) and doesn't interrupt the natural testing flow if one person acts more as the primary test idea generator/executor and the other acts as the primary scribe. If you want to add some variation to your testing approach, give it a try.

jonathan@kohl.ca original post

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Warning: preg_replace() [function.preg-replace]: Compilation failed: unknown option bit(s) set at offset 0 in /var/www/wordpress/wp-includes/shortcodes.php on line 227

Original Post by jonathan@kohl.ca

I've developed some new ET tutorials. The newest is "Managing Exploratory Testing", addressing questions I hear the most from managers. Since I've done a lot of this sort of training already, it made sense to start offering this publicly. I'll be teaching it at EuroSTAR and STAR West this fall.

The other tutorial is an evolved version of my take on "Exploratory Testing Explained". Due to questions and interests of people who have taken the course, it has evolved into a hands-on, experiential workshop: "Exploratory Testing Interactive". I'm also teaching it at STAR West.

If you'd like to work with me to get a glimpse into my take on exploratory testing, and learn some new skills, drop in at one of the conferences. If you don't or can't attend these conferences, you can also consider bringing me in to work with you and your team one-on-one.

jonathan@kohl.ca original post