Category Archives: Development and QA

BDD to the next level

Hello reader,

In this fine post I will write a story about the top level view on the BDD + Gerkin usage in a project.

Once upon a time in a land far far away the Qualitus Assuranceus, Developerus and Productus Ownerus were walking side by side. Little that they know about the Firespitter Comunicator-Braking Dragon. In their childhood all went fine. The Developerus was coming up with all those crazy ideas of gadgets. The Productus Ownerus was shooting requests all over the place and the Qualitus Assuranceus was listening to them. Because their yard was small and their parents were yet in power every evening they were sitting at the same table sharing thoughts. Oh such a beautiful world.

Soon the teenage period came. All of the grew more beautiful. They went to schools and specialized on this and that. The ideas were running even wilder. All of them teamed up with a pair of Bugs. Those Bugs brought their friends. The family was growing but the disaster was not so obvious yet. Mostly due to drinking when the thoughts came together again.

But one day, it happened. The clouds gathered, the sun was covered. The bugs caught wings. And they were rising higher and higher. Some became critically damaging to their family, even to their relationship. This was only the beginning!

Firespitter Comunicator-Braking Dragon joined the game. Spitting revenue loss and extra hours. Chaos itself landed on the project.

Gladly Cpt. BDD-erica came with a blue shield and mantle and restored peace.

The end.

This being the story, where do we go from now? Gerkin and BDD cover all the gaps of the story.

My thought is that the next step is to make it so that the testing framework creates a flow pattern of the application. This flow needs to become smart enough to ask questions about where it thinks that something will be decoupled. All will conclude in a graph of a project and testing a flow will become a query resulting in all the deviations of the possible ways to get from A to B.

The job of the QA should move upwards towards the client and be more business oriented. The goal would be to provide better requests to the developers team and the Product Owner should focus more on listening to what the users want.

 

This is tonight’s QA forecast.

Gabi

Behat Contexts – A stretch of imagination – Episode 3 – Our Poison

Hello,

This will be a short post. Out of all the poisons we have picked the http://www.php.net/traits one for our custom context “classes”. It keeps the OOP idea for later Behat versions and it allows us to have auto-complete in the FeatureContext.php. No auto-complete in the included contexts but since they are for helpers it is ok.

Table doubling + hash tables = love

Hello reader,

During my quest for learning I have encountered this thing, the “Table doubling”. Boy I was so happy about it that I wrote this article :).

The goal is to structure some data, any data, in such a way that we can delete, insert and retrieve really really fast while having a decent memory usage. A real life example would be the HTML code of a webpage. If we consider each HTML element (node) with its properties a set of data we want to get the one that we care about super fast, not in a second or so. So, in order to get a sense of why this is useful I will lay down some premises.

An array = a linear piece of memory of the length of the array. This will be always busy. The array must have the keys/indexes integers. We can’t have the key/index 2.3 or 0.1. Due to those two properties we can get to any element almost instantly because it exists and we know where it is exactly.

A linked list = A piece of data that besides its “internal” information that we care about (for our example let’s say the inner text), it has a link to location in the device’s memory for the next item after it. So 1 knows where 2 is, 2 knows where 3 is, so on and so forth. This is rather sweet but if goes only forward, with a bit of extra work we get to double linked list.

Double linked list = a linked list where the individual piece of information knows about the previous piece as well as the next. Although it has a bigger footprint it solves our previous request in a neat fast way.

Now, in order to nail down the difference, let’s look at what happens when we want to add an element close to the beginning of an array and of a list (double or linked, it doesn’t matter).

Operation Array List
Fetch an element by key
aka
Exact search
Go to key’s position Go to the first or last
Check if it is the one we need
If not ask where the next or previous is located
Repeat the previous two steps until found it*
Insert Increase the length of the array with a number of elements equal with how many we will insert
Move to the right all the other elements in a descending order
Add our data where we wanted to **
It will go at the end of the list and it will tell the last element to have it’s memory address stored.
Delete Remove the element
Move all the other elements to the left
Decrease the size of the array
Go to the desired element forwards or backwards
Tell the element to the left to point to the element to the right. If needed tell the element to the right
to point to the left one

* In case it is not obvious how slow it is. Take a piece of paper and write the steps for finding the 5th element and count the rows.
** It will not put it at the end of the array because the order matters. The premises was that we are close to the beginning of the array so don’t think of adding at the end.

In conclusion, as we can see, there are quite some advantages and disadvantages for each of the data types.

Now in order to find the holly ground of those two data types we will discover a new one, called Hash Table. The not so official definition would be an array that stores as values of its keys pointers to – in the worst case scenario – lists of data with all its properties.

The other concept of this post is Table doubling . This says in less words that instead of resizing an array 1 element at a time, when it gets filled we make it twice as long as it is now. This will allow us to hold twice as much data for future inserts after one operation, and eliminates the need of moving all the elements all the time. The second part is that on delete we shrink it by at least 1/3. This means that inserting and deleting a few elements will not trigger the resizing.

Let’s look at every operation and see the advantage.

Exact search:
Now due to being an array we get very fast navigation and finding.
Insert:
From time to time it triggers the array re-hashing but not so often. As long as we play with the same amount of data or fiddle a bit with the “1/3” value mentioned earlier, everything will work with the speed of the list.
Delete:
The deletion takes part only where there are a lot of free elements, and event after the deletion we have room for more than we had in the beginning. Almost as fast as the double linked list.

 

Conclusion:
I can see this saving seconds on every test case ran on a page with a few hundredth elements.

Behat Contexts – A stretch of imagination – Episode 2 – The almost solution

Hello reader,
in the previous post I have presented 4 viable alternatives to our problem.

“How to have multiple contexts in Behat and maintain the OOP structureparameterssupport an expanding number of context files for a scalable application and most of all, have code hinting available in all the contexts.”

None of them quite hit the sweet-spot. From the coding perspective, for PHP 5.3 we have no tools, we I have tried a hack. The idea was to have the smaller contexts extend each other in the IDE, this provides the code hinting, but at runtime replace the code so that it accommodates Behat’s useContext syntax and logic. This should provide a win-win.

in FeatureContext.php


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
public function __construct(array $parameters)
{
    //get the path to bootstrap in a variable
    $pathToBootstrap = join(DIRECTORY_SEPARATOR,array(getcwd(),"features","bootstrap"));

    //get an array with all the file names that have the Context.php in their name
    $filesFromBootstrap = glob(join(DIRECTORY_SEPARATOR,array($pathToBootstrap,"*Context.php")));

    //for every file modify the internal structure in order to behave as expected by Behat
    foreach($filesFromBootstrap as $filename){
        $handler = fopen($filename,"r");
        $finalClass = fread($handler,filesize($filename));
        $customContexts = array("HelperContext","FrontendContext","BackendContext");
        foreach($customContexts as $context){
            if(stristr($finalClass,"extends ".$context)){
                $finalClass = str_replace("extends ".$context,"extends BehatContext",$finalClass);
                $writeHandler = fopen($pathToBootstrap.DIRECTORY_SEPARATOR."temp.php","w");
                fwrite($writeHandler,$finalClass);
                fclose($writeHandler);
                break;
            }
        }
        fclose($handler);
        $className = str_replace(".php","",basename($filename));

        //include the temporary file
        include($pathToBootstrap.DIRECTORY_SEPARATOR.'temp.php');

        //create the context using Behat's implementation
        $this->useContext($className, new $className($parameters));
    }

    $this->login = $this->getSubcontext("login");
}

 

in the “lower” classes the code looks like:
class HelperContext extends MinkContext{ }
class Helper2Context extends HelperContext{ }
so on and so forth. This will provide the code hinting.

 

Note: The code is not complete, currently it crashes with “Fatal error: Cannot redeclare class BackendContext” ” at the temporary file inclussion. This happens because the contexts are already loaded by Behat before the FeatureContext constructor. In order to fix this, for PHP 5.3 we need to create a patch or drop the already initialized object or to modify the included class’ name. For the second I did not test the behavior or stability of the framework.

Feel free to use the code as desired and fix it.

The next post will include the PHP 5.4 “traits” functionality for the helpers contexts.

Have a nice day,

Gabi

Behat Contexts – A stretch of imagination – Episode 1

Hello reader,

I am writing this in order to document the many hours spend trying to find a solution for a very simple problem. The Php version used is 5.3 or 5.4 (depends on the project to be used on):

“How to have multiple contexts in Behat and maintain the OOP structure, parameters, support an expanding number of context files for a scalable application and most of all, have code hinting available in all the contexts.”

 

The client I am working with at the moment has multiple applications all extending one internal framework. The problem is that everyone wants automation for the main framework but nobody has time for it because the clients do not pay for that. A typical not out of the ordinary situation. A solution was to get what we have on the projects and create a repository with tests for the main framework based on an estimation of effort, complexity and deviation from the standard implementation. In theory it will worked brilliantly.

 

As in every good story we have set up a Git repo, we did establish the flow, added in the code reviews, coding standards and we are ready to go. After a week features, tests and helpers are flowing from all the directions. The joy!

 

The structure of our project would be a helperContext that together with MinkContext and any other context must be accessible into any other class which will be added later on. We must also be able to pass variables from one context to another at will.

 

Let’s set a baseline. Php can extend 1 class and only 1 and inherit any number of interfaces.
How Behat works:
There is a file called Application.php, inside it we get to ClassLoader.php which gets from composer a list of all the classes of the modules that have been installed in the project. From here on, each file is included (it is include not include_once).

 

1) Let’s to pull this out! In almost every Behat presentation there is this phrase stating that there is a function called setContext() which will be the holly grail. Youp, there is, here’s the documentation  http://docs.behat.org/guides/4.context.html . Works as intended BUT the MinkContext can’t be extended anymore since BehatContext has to be in order to accommodate subcontexts. The solution is to include MinkContext using the useContext() functionality. This will let us use it’s classes with the syntax

1
$this->getMainContext()->getSubcontext('subcontext_alias')->some_method()

. BUT we will have no code hinting anywhere.

2) How about creating a class that extends Mink and make FeatureContext extend that class. It should come packed with Mink and all our added functionalities. Youp it does. It even has auto-complete, self high-five BUT if we want to create another context called LoginContext, where we want to store Login related only functionality, we can’t extend FeatureContext. Because FeatureContext is the framework’s entry point and already exists at that point. We will get “Multiple definitions for FeatureContext exist” and it will crash after 100 self inclusions. Damn, so close! We can taste the victory!

3) Let’s have FeatureContext extend MinkContext and implement ContextInterface. This will make FeatureContext awesoommee, it will contain everything and also after we copy paste the implementation of ContextInterface from BehatContext we maintain the inheritance from the documentation. When we utilize useContext() on LoginContext we get all it’s insides at runtime. Here we have scalability. BUT no code-hinting anywhere else except in FeatureContext, also we have to manually add every new context file added to the project into the constructor. Oh god damn :( so close yet so far.

4) Maybe dropping the OOP and having the helperContext a simple file with methods. This should work BUT we are loosing everything and it must be included by hand all time if every other context. Will not go down this path on how many problems there are.

 

For now I am out of ideas. Probably I need something in the constructor of FeatureContext that enables the usage of useContext() but removes some code that allows the helperContext to have code hinting.

Assign the same tests to the Zephyr suites after they are moved to another project

Hello,

We have received the task of copying a very big suite of tests from a project to another and maintain their test cycle assignment. For the copy and moving part is easy, there are bulk actions. For the automatic assignment we couldn’t find a quick solution, probably there was.

 

Here’s our approach:

1) On the Test Cycles page we extend a test cycle that has tests which will be moved. This is necessary in order to figure out which to move.

In the browser’s console we run this code (search for domain in the ajax call and replace with your own domain):


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
(function($) {
    var cycleId = (function() {
            var el = $('li.expanded');
            return el.length > 0 ? el.attr('id').replace("cycle-","") : false;
        })(),
        projectId = (function() {
            var el = $('#zprojectId');
            return el.length > 0 ? el.attr('value') : false;
        })(),
        pageCount = (function() {
            var pageText = $('#getMoreSchedulesLbl').text(),
                count = pageText.match(/[0-9]{1,3}/g);
            return Math.ceil(count[count.length-1]/10);
        })(),
        i = 1,
        responses= [],
        offset,
        completed = 0;
    $('body').on('requestscompleted', function(){
        var len = responses.length,
            i = 0,
            summary = [],
            response;
        for(i; i < len; i++) { // iterate through all received responses
            response = responses[i];
            console.log(response);
            (function(item){
                var i = 0,
                    schedules, len;
                if(typeof item != 'undefined') {
                    schedules = item.schedules,
                        len = schedules.length;
                    for(i; i < len; i++) {
                        summary.push(schedules[i].summary);
                    }
                }
            })(response);
            console.log(summary);
        }
    });
    if(cycleId && projectId) {
        for(i; i <= pageCount; i++) {
            if(i == pageCount){
                lastPage = $('#getMoreSchedulesLbl').text().match(/[0-9]{1,3}/g);
                offset = (lastPage[lastPage.length-1] - ((i-1) * 10));
                console.log(offset);
            }else{
                offset = i * 10;
            }
            $.ajax('domain/rest/zephyr/latest/schedule?decorator=none&contentOnly=true&noTitle=true&cycleId=' + cycleId + '&action=expand&offset=' + offset + "&pid=" + projectId + "&vid=-1&sorter=ID", {
                method: "GET",
                async: false,
                complete: function(){
                    if(completed == pageCount -1) {
                        $('body').trigger('requestscompleted');
                    }
                    completed++;
                },
                success: function (response){
                    responses.push(response);
                }
            });
        }
    }
})(jQuery);

 

2) This returns an array of test names. Those names will be used as identifiers later on. Please copy them into another external document.

3) Copy or move the tests into the other project as desired

4) Open the tests search page on the new project and display as many as possible (the current functionality does not cycle through the pages)

5) Paste the following code and replace the first line with the array exported by the previous code:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
var incomingTitles = ["[Event Validation] Frontend - Checkout flow", "[Event Validation] Frontend - Mobile site", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 1", "test name 2", "test name 3", "test name 1", "test name 2"];

(function($,incomingTitles){
    var testSummary = (function(){
        var summaries = $('td.summary'),
            i=0,
            title="",
            currentItem = '',
            currentItemVal = '',
            keysHolder = new Array();
        summaries.each(function(){
            currentItem = $(this).find('a.issue-link').text();
            currentItemVal = $(this).find('a.issue-link').attr('data-issue-key');
            //console.log(currentItem);
            if(incomingTitles.indexOf(currentItem) != -1){
                keysHolder.push(currentItemVal);
            }
        });
        console.log(keysHolder);
        return keysHolder;
    })();
})(jQuery,incomingTitles)

This function will export an array of test IDs. Copy them in a document.

6) Open the “Plan Test Cycle” page and click the cogwheel to add tests

7) Paste the array of IDs returned by the second script

 

We have tried to use the filters functionality but we could not select only the tests assigned to a suite. It got even worse when the same tests were assigned to more widgets.

Thank you,

Gabi

Testing framework – EP 2 – What types of automation are there

Hello,

now that we have identified the fact that automation is required let’s have a look at the approaches already available.

  1. Code driven testing
  2. Graphical interface testing (GUI)

The code driven testing involves testing the classes, modules and/or libraries.
The graphical interface testing involves the emulation of keyboard and mouse actions. The output is visible on the screen.

How to chose between one and the other?

This question arises because the GUI projects are composed of code but not all the applications have a GUI.

If the project’s code exposure through visual feedback for both the configuration data (Admin) , manipulation feedback (Frontend) and it’s scope is to facilitate the actions of the visitors, the GUI only approach is enough.

If the configuration data is updated through a file (.tsv/.csv etc), some code driven testing is necessary for the input scenarios.

If all the project is an API, for instance, or its scope is to connect two systems, code driven testing is sufficient.

Based on this article we know for sure which way we should head with our automation process.

Testing framework – EP 1 – Why have automation

Hello,

the purpose of this series is to come up with a definition of what a “testing framework” is based on today’s needs and standards. The project in scope is a big custom Magento implementation.

The release methodology is Agile, the project is split in two teams. One team handles the issues and the client’s needs. The other the new functionalities. The release cycle lasts about 2 weeks. Rarely more, rarely less. If special events occur there are hotfixes in between. Generally they are avoided by both the team and the client.

The QA (Quality Assurance)’s role is to inspect the requirements and provide feedback, write acceptance criteria, write test cases, conduct UAT (User Acceptance Testing), performance testing, report issues and maintain a healthy build. All this, if possible, should be done by yesterday’s evening.

So far so good, pretty much a standard situation in most of the teams that feel the need for automation.

The challenge is to deliver the same amount of work as before in a shorter period of time in order to accommodate the new features if possible without cutting corners.

  • One solution would be to get more QA resources, but this does not fix or improve the process, this just enables a wider bandwidth at the expense of money.
  • Another solution would be to increase a bit the release time or lower the number of tickets. This has some monetary impact on the client and unless the quality is a problem it is very less likely to be accepted.
  • A third option is to implement an automation framework which absorbs some of the tasks allowing the tester and the team to focus on delivering.

 

Conclusion:

The automation framework is the extra kick needed by the team, as soon as possible, in order to deliver better quality in the same amount of time for a long period of time with marginal cost increase while providing adequate documentation.