EuroSTAR 2011

While in China for Polteq, I wrote a submission for EuroSTAR 2011. The conference theme was "In the pursuit of quality" so I decided to go for a less explored subject "How To Test Mobile Applications". At the start of April I received an e-mail from the conference stating:

We are delighted to inform you have secured a speaking slot at the 2011 EuroSTAR Conference and we look forward to meeting you in Manchester later in the year.

So I got lucky enough to have my first submission to EuroSTAR accepted :-) The next months were needed to slowly build up the presentation (which also got accepted for Testnet). 

Tuesday the 22st of November we went to Manchester. The mist cancelled some flights, but our flight only had a delay of about 20 minutes. The first day we missed out on some presentations due to a slow restaurant, but I was able to hear Gojko Adzic speak about "Death To The Testing Phase". The main things I got from this presentation were that it is important to visualise quality and to apply post release testing. My colleague Kees Blokland was all day preparing for his presentation the next day.

In the evening Dorien and I went to visit the local Christmas market. We were surprised to see a lot of Dutch stands there, supplying Dutch cheese, Dutch mini pancakes and more. The touristic route (read: we got a bit lost) back to the hotel took a while, but we saw some nice parts of Manchester.

Wednesday I got to see more of the presentations. The day started with a keynote of James Whittaker: "Pursuing Quality? You Won't Get There By Testing". The story proved to be good, but only applicable to large companies. Smaller companies can't just use thousands of people to perform some beta testing like Google can...

Next I went to one of the short tracks where Martin Mussmann told us about "Mind Maps As Part Of The Agile Testing Process". In the high change rate that he has to work in, mind maps proved to be a nice way to show the backlog items and connect test ideas to them. By describing only what to test and not how to do it, these maps could be used as a help in talking to both the business as the developers.

I just had to attend Kees' presentation about "Chasing Quality In Cloud Computing - Testing Different Levels Of Quality Requirements". He showed us that using the cloud incorporates different types of risks that need to be mitigated. Of course the obvious ones like performance, security and functionality, but also new risks in the area of legislation and regulations and suppliers. After describing the risks he also described test measures to be taken. Ending with a description of the broader role of the test manager which needs involvement in selection, implementation and operation of cloud services.

Since automation is one of the topics I like, I also went to Julian Harty who introduces us to "Pushing The Boundaries Of User Experience Test Automation". He provided some nice examples of automatable user experience tests. Not everyone can (or will) use the mouse to navigate through websites, so tabbing and hotkeys should also be tested. We accept that the number of steps needed to perform a task by using the keyboard is more than with using the mouse, but there are limits. So also the number of tabs needed to return to the same point on a page should be limited, which can automatically be validated.

The closing keynote of wednesday was Daryl Elfield with "Power to the People - Achieve Fast, Sustainable Change in Quality by Harnessing the Crowd". He gave nice examples of how communities within the organisation not only provided ideas for change, but also created the footage to actually perform the change. So by giving power to the employees, the commitment to the company increased.

In the evening our friends from Hauwei invited us to eat with them in Chinatown. It was really nice to have some decent Chinese food again :-) We went for hot pot! I chose the spicy one, but was uncertain of my choice when two out of three Chinese took the normal while saying that spicy was too spicy for them. The first couple of bites were extremely spice, but I got used to the taste and had a great dinner.

Thursday started with Ben Walters' keynote "Create Customer Value with Agile Testing". I know I visited this keynote, but can't remember much of it. Probably I was too busy with preparing for my own presentation this day.

I attended the presentation previous to mine by John Montgomery "What The Top 10 Most Disruptive Technology Trends Mean For QA And Testing". He described a nice list of disruptive trends like agile, cloud and mobile, but in my eyes lacked to really show what it meant for testing.

Then finally my debut on the big stage with "How To Test Mobile Applications" ;-) A quick count showed more than 100 people in the audience, a new record since my 30 people audience at Testnet. I provided a road map for building mobile applications. Testing needs to be aware of store requirements, next to the normal business and user requirements. Another important part is that the impact of the app on the features provided by the phone and vice versa need to be tested. It would be a pity if the phone wasn't able to receive calls while using an app... After my presentation I had quite a lot chats with people that had attended and wanted to know more.

My state of mind didn't allow me to take in any information after my presentation, so I took the rest of the afternoon strolling around the Expo. At the end of the afternoon we flew back home. I have really enjoyed my first visit to EuroSTAR and saw good presentations. I hope to attend again next year in Amsterdam.

Testnet Fall Event

Yesterday I attended the Testnet Fall Event. It proved to be an interesting as well as a fun day! For the morning sessions I chose to attend an interactive session about “The future of testing”. The session started with a couple of PechaKuchas giving the presenters views on the future. After the last Pecha Kucha the audience could pick the presenter with whom they wanted to discuss a bit more. Since I liked the presentation about people the best, I ended up with HuibSchoots. We got into a discussion about training and skills needed by people. The thing I best remember is: you need to take control of what you want to learn and not wait until others suggest training. The next part of the morning session dealt with personas, defined by the presenters, which could exist in the future of testing. Again in little groups, the personas got more depth by adding why people chose a certain persona. The last part of the, in my eyes very successful, workshop existed of putting sticky notes with comments to other personas than you chosen one. Providing extra questions and room to motivate the personas better.

Next up was a keynote by a non-tester, Stefan Tilkov. He managed to give a clear description of the cloud and in the end provided some pointers to testing. The rest of the day existed of lots of presentations to choose from. I attended a couple and have learned new views on the world of testing, providing me with a lot of food for thought.

The last session of presentations contained my own presentation about “How to test an APP”. I expected only a few people, since my time slot was at the end of the day. That turned out differently when about 30 people joined my session. The audience was great! Interested and asking lots of questions.

Between all sessions there was enough time to talk to friends and meet new people. All the ingredients for a great day! Thanks to Testnet for facilitating this great event! 

Cloud in testperspectief, testen in cloudperspectief

Usually I blog in English, but since this is my article as published in the Dutch magazine Release (Jaargang 15, nummer 3), this post will be in dutch.

Cloud computing, vaak kortweg de cloud genoemd, is niet meer weg te denken. De aandacht die de cloud krijgt, heeft alle kenmerken van een hype, maar dat neemt niet weg dat het een wezenlijke ontwikkeling is van de manier waarop ICT-diensten worden verleend. Het afnemen van een cloud service is steeds vaker aantrekkelijker dan eigen software maken of kopen. In de cloud wordt er alleen betaald voor wat er wordt gebruikt (pay–per-use) en er kan flexibel capaciteit worden bij- of afgeschakeld (elasticiteit). Kortom, steeds meer ICT verdwijnt in de cloud en dat heeft invloed op testen en de rol van de testmanager. Cloud computing introduceert nieuwe eisen en risico’s waarvoor een speciale testaanpak nodig is. Hoe test je bijvoorbeeld elasticiteit?

Testen zelf verandert ook! Zo is er een significante rol weggelegd voor de testmanager in het selectieproces. Dat is een belangrijke fase om tijdig vast te stellen wat voor risico’s de gebruiker of de business loopt bij de keuze voor een bepaalde cloud service. Die risico’s kunnen liggen op allerlei vlakken. Dat is ook een verandering bij het testen: een accentverschuiving naar niet-functionele aspecten. Het zoeken naar functionele fouten in de software verschuift in de cloud naar de achtergrond.

Nieuwe eisen en risico’s

Cloud computing is volop in ontwikkeling. Het is dan ook een illusie om te verwachten dat voor alle nieuwe eisen en risico’s al pasklare testoplossingen beschikbaar zijn. Dit artikel beschrijft een aantal voorbeelden van cloud-specifieke risico’s en bijbehorende testaspecten en is een beknopt fundament voor verdere ontwikkeling.

Cloud computing omvat nogal wat. Kijk voor een goed bruikbare definitie op de website van het Amerikaanse NIST (www.nist.gov/itl/cloud/index.cfm).
Krachtige features van cloud computing zijn elasticiteit en pay-per-use. Is er meer capaciteit nodig, bijvoorbeeld omdat er meer gebruikers zijn, dan kan dat on-demand worden geleverd. De gebruiker ziet dit vanzelf terug in de afrekening. Maar hoe kan er getest worden of dit in de praktijk ook werkt? In de aanpak hiervoor is een combinatie nodig van load testen (wisselend gebruik nabootsen en zien dat er capaciteit bij komt en weer af gaat), grenswaardenanalyse (wat gebeurt er op de grens van mijn ’bundel’) en procescyclustest (het handmatige aanvraagproces en de financiële afrekening).

Risico’s

Met de transitie naar de cloud storten alle risico’s van het internet zich over de afnemer heen. Wat voorheen geen issue leek (de server stond in het eigen gebouw), wordt in één keer een punt van aandacht. Wie kan er allemaal bij mijn gegevens in de cloud? Hoe veilig is het transport eigenlijk? Dit leidt niet tot nieuwe soorten beveiligingstesten, maar security eist wel een nog belangrijkere plek op bij het testen.

In de public cloud kan de service trager worden als gevolg van de (piek)belasting van andere afnemers. Als de leverancier een abonneemodel hanteert waarbij de service wordt overboekt (dat levert meer winst op), dan is dat een goede reden om een real-time loadtest uit te voeren. Daarmee wordt bedoeld: het uitvoeren van de loadtest op het moment van de dag dat de piekbelasting zich naar verwachting zal voordoen.

Nieuwe acceptatiecriteria

Cloud services worden mondiaal aangeboden en daarmee komt een nieuw pakket van acceptatiecriteria om de hoek kijken: voldoet de cloud service aan allerlei wet- en regelgeving? Zo is onder meer het bewaren van privacygevoelige gegevens aan allerlei wetten en regels gebonden. De testmanager zorgt ervoor dat er, in samenspraak met bijvoorbeeld een jurist, een checklist van wettelijke controlepunten wordt samengesteld. Bij het afvinken daarvan kunnen er complicaties aan het licht komen, zoals conflicterende of onduidelijke wetgeving in bepaalde landen en de (on)betrouwbaarheid van de overheid waar de cloud provider is gevestigd. Dit kan tot verrassingen leiden, zoals in het voorbeeld van de Amerikaanse overheid die de Patriot Act in stelling bracht om toegang te krijgen tot de gegevens van WikiLeaks.

Beschikbaarheid

Wanneer een belangrijk business proces afhangt van een cloud service, is de beschikbaarheid van de service van groot belang. Als een cloud service leverancier (onaangekondigd) een wijziging aanbrengt in de service, kan dat verstrekkende gevolgen hebben, zoals een storing in de koppeling met andere systemen of handleidingen die plotseling niet meer kloppen. In een end-to-end testopstelling kan de impact van wijzigingen in een cloud service, maar ook in aangesloten systemen, worden getest. Er is een groeiende noodzaak voor een continu draaiende, end-to-end regressietest. Deze test moet niet alleen op functioneel gebied inzicht in de risico’s bieden, maar ook op non-functionele aspecten, zoals performance. De taak van de testmanager stopt dus niet na de implementatie, want ook in de operatie moet er continu worden getest.

Leveranciersaspecten

De afnemer van cloud services krijgt te maken met allerlei leveranciersaspecten. Wat gebeurt er bijvoorbeeld precies met de gegevens in de cloud als de leverancier failliet gaat? Is er een back-up scenario en, zo ja, is dat getest? Ook de migratie naar de cloud en van de ene cloud service (leverancier) naar een andere moet worden getest. Als er geen geteste migratiestrategie is, levert een afnemer zich over aan één leverancier (vendor lock in).

Fenomenen die in de cloud hype lijken mee te liften zijn Het Nieuwe Werken en Bring Your Own Device. Wordt de ondersteuning van een veelheid van vaste en mobiele platforms een belangrijk risico bij cloud computing of niet? De toekomst zal het leren.

Conclusie

Voor de testmanager is een belangrijke rol weggelegd bij de invoering van cloud computing. De testmanager moet al in actie komen bij de selectie van cloud services. Maar ook na de implementatie ervan loopt zijn rol door tot  in de operatie van cloud services. Centraal staan daarbij de risico’s en (test)gerelateerde maatregelen om die risico’s te beheersen.

Producing waste

Today a colleague pointed out that Agilistas produce a lot of waste. Since it's in the core of agile to reduce waste, this seemed like a strange statement. So we discussed a bit :-) I pointed out that we produce less overhead documentation than other people do, as can be found in almost all presentations about agile.


He then told me he didn’t mean in the projects themselves… Agilistas are responsible for a whole lot of blogposts, tweets, Google+-posts, etc. and he labelled it waste. Of course I responded that using ­social media is an act of being social. A trait of all true Agilistas, since we value communication. Using social media gives people the opportunity to read and reply in their own time and we win time by not using it for mindless discussions. This silenced him for a moment, but he’s still not convinced. Luckily I know the silence means I got him thinking.

Soft skills

Testing is a profession! As a profession it requires skills in testing techniques, domain knowledge for the specific case you are working on, etc. A often underappreciated skill is communication. From the test perspective you need to communicate with a lot of different types of people, where each type requires it's own approach. This seems simple, but is harder than it looks. It's not only the type you need to be aware of, but also the person itself. In some cases a direct approach is the way to go, where in other cases you need to dress up your comments. Be aware of what you say and to who you say it!

Agile and test planning

Agile is often used in combination with scrum, thus resulting in multidisciplinary teams and short release cycles. If we take these two points as a given, how do we adopt our test planning to be able to keep working? First I'll describe a way to work, then I'll give some pointers to take into account during the sprint planning.


Specification
When detailing the product backlog items, make sure the whole team has the same understanding of it. After the first detailing, developers can start creating and product owner, tester and analyst/designer can go over the specifics of the detailing. Then designer and tester can start making their corresponding designs. When done creating the specifications designer and tester review each other's work to see if they described the same system. Any open questions after this reviewing should be resolved with the product owner.


Execution
Start executing test scripts as soon as possible. Share testing knowledge with the developers, so that they benefit from this knowledge when creating the unit tests. Make sure you stay informed about what is delivered to the test environment and then prioritize your test cases according to what was in the last delivery. Usually when the pressure on test execution starts to build, the analysts/designers in your team will be less busy. They can help you execute the script you already designed. When developers have time left, ask them to automate the tests you want to run in continuous integration or regression.


Bugs
Bugs can be solved quicker and with less documentation than in non-agile projects. Discuss bugs you find within your team. In this way all members learn about the bugs found and help each other to find solutions. If the bugs get fixed directly after you find them, it might not be necessary to document them. This results in less time tracking bugs.


Sprint planning
With the sprint planning, keep in mind that you need time for specifying, executing, automating and retesting. To fit everything in the small amount of time, use specification techniques which result in a small number of test cases with high coverage. Make sure that developers also incorporate time for unit testing and for fixing bugs. In the case you will not do the automation yourself, make sure that the responsible person (sometimes the developers) will plan this time.
Experience learns that setting some milestones for a sprint, e.g. first delivery start or code freeze, help the team to finish the sprint successfully.

Locator problems

I'm still exploring using Selenium with page objects. For webelements that have an id or name tag it is easy to define the locators, but I'm having some problems creating a locator for some links. The xpath reference to the link, nicely supplied by FirePath, results in a "Unable to locate element".  The link I'm trying to locate is in the format:

<a href="xxx/yyy/" title="yyy">yyy</a>

I'll just keep trying to find how to locate it. Next to that I'm going to start working on FitNesse.

Selenium

Recently a collegue, Daniël Gollings, showed me Selenium. As an enthusiastic about test automation I was interested directly. So I started to follow the step-by-step tutorial he made. I downloaded all the needed compenents and started with Selenium IDE. Since I have a background in programming, I quicly switched to Selenium RC to be able to apply better control of the automation. The ease of setting up tests amazed me.


Recent conversations with Roy de Kleijn and Martijn de Vrieze, reignited the Selenium-fire. After following a webinar about page objects, I understand the concept. Now only experience can help me with applying the concept. Hopefully I can use my knowledge in a couple of assignments and broaden my knowledge.