Category Archives: Testing Framework

Behat Contexts – A stretch of imagination – Episode 2 – The almost solution

Hello reader,
in the previous post I have presented 4 viable alternatives to our problem.

“How to have multiple contexts in Behat and maintain the OOP structureparameterssupport an expanding number of context files for a scalable application and most of all, have code hinting available in all the contexts.”

None of them quite hit the sweet-spot. From the coding perspective, for PHP 5.3 we have no tools, we I have tried a hack. The idea was to have the smaller contexts extend each other in the IDE, this provides the code hinting, but at runtime replace the code so that it accommodates Behat’s useContext syntax and logic. This should provide a win-win.

in FeatureContext.php


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
public function __construct(array $parameters)
{
    //get the path to bootstrap in a variable
    $pathToBootstrap = join(DIRECTORY_SEPARATOR,array(getcwd(),"features","bootstrap"));

    //get an array with all the file names that have the Context.php in their name
    $filesFromBootstrap = glob(join(DIRECTORY_SEPARATOR,array($pathToBootstrap,"*Context.php")));

    //for every file modify the internal structure in order to behave as expected by Behat
    foreach($filesFromBootstrap as $filename){
        $handler = fopen($filename,"r");
        $finalClass = fread($handler,filesize($filename));
        $customContexts = array("HelperContext","FrontendContext","BackendContext");
        foreach($customContexts as $context){
            if(stristr($finalClass,"extends ".$context)){
                $finalClass = str_replace("extends ".$context,"extends BehatContext",$finalClass);
                $writeHandler = fopen($pathToBootstrap.DIRECTORY_SEPARATOR."temp.php","w");
                fwrite($writeHandler,$finalClass);
                fclose($writeHandler);
                break;
            }
        }
        fclose($handler);
        $className = str_replace(".php","",basename($filename));

        //include the temporary file
        include($pathToBootstrap.DIRECTORY_SEPARATOR.'temp.php');

        //create the context using Behat's implementation
        $this->useContext($className, new $className($parameters));
    }

    $this->login = $this->getSubcontext("login");
}

 

in the “lower” classes the code looks like:
class HelperContext extends MinkContext{ }
class Helper2Context extends HelperContext{ }
so on and so forth. This will provide the code hinting.

 

Note: The code is not complete, currently it crashes with “Fatal error: Cannot redeclare class BackendContext” ” at the temporary file inclussion. This happens because the contexts are already loaded by Behat before the FeatureContext constructor. In order to fix this, for PHP 5.3 we need to create a patch or drop the already initialized object or to modify the included class’ name. For the second I did not test the behavior or stability of the framework.

Feel free to use the code as desired and fix it.

The next post will include the PHP 5.4 “traits” functionality for the helpers contexts.

Have a nice day,

Gabi

Behat Contexts – A stretch of imagination – Episode 1

Hello reader,

I am writing this in order to document the many hours spend trying to find a solution for a very simple problem. The Php version used is 5.3 or 5.4 (depends on the project to be used on):

“How to have multiple contexts in Behat and maintain the OOP structure, parameters, support an expanding number of context files for a scalable application and most of all, have code hinting available in all the contexts.”

 

The client I am working with at the moment has multiple applications all extending one internal framework. The problem is that everyone wants automation for the main framework but nobody has time for it because the clients do not pay for that. A typical not out of the ordinary situation. A solution was to get what we have on the projects and create a repository with tests for the main framework based on an estimation of effort, complexity and deviation from the standard implementation. In theory it will worked brilliantly.

 

As in every good story we have set up a Git repo, we did establish the flow, added in the code reviews, coding standards and we are ready to go. After a week features, tests and helpers are flowing from all the directions. The joy!

 

The structure of our project would be a helperContext that together with MinkContext and any other context must be accessible into any other class which will be added later on. We must also be able to pass variables from one context to another at will.

 

Let’s set a baseline. Php can extend 1 class and only 1 and inherit any number of interfaces.
How Behat works:
There is a file called Application.php, inside it we get to ClassLoader.php which gets from composer a list of all the classes of the modules that have been installed in the project. From here on, each file is included (it is include not include_once).

 

1) Let’s to pull this out! In almost every Behat presentation there is this phrase stating that there is a function called setContext() which will be the holly grail. Youp, there is, here’s the documentation  http://docs.behat.org/guides/4.context.html . Works as intended BUT the MinkContext can’t be extended anymore since BehatContext has to be in order to accommodate subcontexts. The solution is to include MinkContext using the useContext() functionality. This will let us use it’s classes with the syntax

1
$this->getMainContext()->getSubcontext('subcontext_alias')->some_method()

. BUT we will have no code hinting anywhere.

2) How about creating a class that extends Mink and make FeatureContext extend that class. It should come packed with Mink and all our added functionalities. Youp it does. It even has auto-complete, self high-five BUT if we want to create another context called LoginContext, where we want to store Login related only functionality, we can’t extend FeatureContext. Because FeatureContext is the framework’s entry point and already exists at that point. We will get “Multiple definitions for FeatureContext exist” and it will crash after 100 self inclusions. Damn, so close! We can taste the victory!

3) Let’s have FeatureContext extend MinkContext and implement ContextInterface. This will make FeatureContext awesoommee, it will contain everything and also after we copy paste the implementation of ContextInterface from BehatContext we maintain the inheritance from the documentation. When we utilize useContext() on LoginContext we get all it’s insides at runtime. Here we have scalability. BUT no code-hinting anywhere else except in FeatureContext, also we have to manually add every new context file added to the project into the constructor. Oh god damn :( so close yet so far.

4) Maybe dropping the OOP and having the helperContext a simple file with methods. This should work BUT we are loosing everything and it must be included by hand all time if every other context. Will not go down this path on how many problems there are.

 

For now I am out of ideas. Probably I need something in the constructor of FeatureContext that enables the usage of useContext() but removes some code that allows the helperContext to have code hinting.

Testing framework – EP 2 – What types of automation are there

Hello,

now that we have identified the fact that automation is required let’s have a look at the approaches already available.

  1. Code driven testing
  2. Graphical interface testing (GUI)

The code driven testing involves testing the classes, modules and/or libraries.
The graphical interface testing involves the emulation of keyboard and mouse actions. The output is visible on the screen.

How to chose between one and the other?

This question arises because the GUI projects are composed of code but not all the applications have a GUI.

If the project’s code exposure through visual feedback for both the configuration data (Admin) , manipulation feedback (Frontend) and it’s scope is to facilitate the actions of the visitors, the GUI only approach is enough.

If the configuration data is updated through a file (.tsv/.csv etc), some code driven testing is necessary for the input scenarios.

If all the project is an API, for instance, or its scope is to connect two systems, code driven testing is sufficient.

Based on this article we know for sure which way we should head with our automation process.

Testing framework – EP 1 – Why have automation

Hello,

the purpose of this series is to come up with a definition of what a “testing framework” is based on today’s needs and standards. The project in scope is a big custom Magento implementation.

The release methodology is Agile, the project is split in two teams. One team handles the issues and the client’s needs. The other the new functionalities. The release cycle lasts about 2 weeks. Rarely more, rarely less. If special events occur there are hotfixes in between. Generally they are avoided by both the team and the client.

The QA (Quality Assurance)’s role is to inspect the requirements and provide feedback, write acceptance criteria, write test cases, conduct UAT (User Acceptance Testing), performance testing, report issues and maintain a healthy build. All this, if possible, should be done by yesterday’s evening.

So far so good, pretty much a standard situation in most of the teams that feel the need for automation.

The challenge is to deliver the same amount of work as before in a shorter period of time in order to accommodate the new features if possible without cutting corners.

  • One solution would be to get more QA resources, but this does not fix or improve the process, this just enables a wider bandwidth at the expense of money.
  • Another solution would be to increase a bit the release time or lower the number of tickets. This has some monetary impact on the client and unless the quality is a problem it is very less likely to be accepted.
  • A third option is to implement an automation framework which absorbs some of the tasks allowing the tester and the team to focus on delivering.

 

Conclusion:

The automation framework is the extra kick needed by the team, as soon as possible, in order to deliver better quality in the same amount of time for a long period of time with marginal cost increase while providing adequate documentation.