Is your Magento 2 IT agency still wasting time on manual tests?

Now you have the opportunity to test your application in Magento 2 faster and more accurately. Magento Testing Framework is a new tool that automates functional testing. Below, we present you how it works.

Nowadays, development without tests makes no sense, as we always have to be sure that the developed functionality works in the expected way. There are many types of tests that enable multi-level functional verification (e.g. unit, integration, static tests, etc.), but nothing can replace the actual behavior of the user while using the application. Functional tests are intended to simulate such user activity and verify that it’s met with the expected response of the system (i.e. functional tests are automated manual tests). Such tests used to be created in JAVA, which required knowledge of additional language and tools, so many companies didn’t conduct this type of testing (manual tests were more convenient, but they caused (and still do) certain problems). That’s why Magento has released its framework so that we are able to create functional tests in PHP.

MTF, or Magento Testing Framework is a Magento tool that allows you to test application functionalities without having to create test cases in languages other than PHP. All we need is:

• PHPUnit,
• Selenium Standalone Server,
• web browser,
• internet application :)

What are the benefits?

A developer who performs his/her job has the ability to write the code that corresponds to the activity, automate the tests and immediately deliver the finished product in the expected quality. This gives us a large operating profit (no need for a manual tester, preparing test environments for manual testing, etc.), so we are able to deliver products faster and at the highest possible quality. We also have the ability to quickly test the entire application after introducing even the smallest change. We can test using different browsers and different screen sizes, which gives us unlimited possibilities to trace each application path in different configurations.

As a result of MTF tests we get:

• the tested application,
• a result output returned by PHPUnit,

short information from PHPUnit,

and in case of an error:

• screenshots of the application at the moment of the error occurrence (very useful! In this case, the error was triggered deliberately by providing incorrect login data to the admin panel),

• logs (including the source code of the page) where the error occurred,

which in case of an error gives you a great opportunity to analyze what could’ve gone wrong.

Of course, you can run the tests individually or as packages. You also have the ability to prepare several variations for one test, so you don’t have to write a few tests to test one element in different ways. Thus, we also get rid of unwanted repetitions. Let’s take a look at how the variations look like and how they are configured.

Example use of variations

What we want to test: Validation of the required phone number box

Variation1: a normal sequence of digits corresponding to the phone number, e.g. 987555222

Variation2: a long sequence of digits, e.g. 123456789098787554443

Variation3: a sequence of igits and letters, e.g. 876Abc432

Variation4: empty box

Variation5: a sequence of digits and special characters, e.g. 123$%7888.

Example of MTF configuration variation (for a test that checks navigation in the admin panel):

These five variations allow you to test five different paths with one test.

It works!

Magento itself, not to discourage developers from learning new and unknown tools, and to show that it is possible, has prepared a large package of tests already written with MTF. The tests cover areas such as

• wishlist functionality,
• managing widgets (adding, deleting, etc.)
• managing saved credit cards,
• testing and configuration of delivery methods,
• testing and configuration of payment methods,
• filter, sorting and searching on the grid in the admin panel tests,
• calculation and configuration of taxes,
• managing multiple stores (store, storegroup, etc.)
• sitemap functionality,
• security testing (access restrictions, password reset requests, SSL, etc.)
• synonym functionality,
• sales rules,
• cart rules,
• reports
• newsletter,
• multishipping,
• layered navigation,
• checkout,
• managing products (all types)
• managing users and customer accounts,
• CMS functionality,
• currency,
• indexer functionality,

and many more!

With so much coverage of ready-made tests, developers and testers can work on maintaining and developing the tests instead of wasting time on tedious manual testing. In addition, the use of MTF in continuous integration environment greatly reduces the occurrence of errors and the time needed to implement functionalities data or manual tests.

Education

For educational purposes, we also created a small module (GitHub link), which adds a top bar in front that displays:

• the store phone number (downloaded from configuration in the admin panel).
• e-mail address to customer support (downloaded from configuration in the admin panel).
• link to login / registration

It’s a simple module that wasn’t aimed at impressing anyone with its functionality, but to present how you can write a module from scratch and test it functionally using MTF (whether in the admin panel or in front).

Four tests were created for the module, each has 1 variation:

1)

• Go to the admin panel,
• log in,
• enable bar functionality,
• go to the homepage,
• check if the bar is displayed correctly.

2)

• Go to the admin panel,
• log in,
• disable bar functionality,
• go to the homepage,
• check if the bar is hidden.

3)

• Go to the admin panel,
• log in,
• configure an email address for customer support,
• clear the application cache,
• go to the home page,
• check if the bar displays the data downloaded from configuration correctly.

4)

• Go to the admin panel,
• log in,
• configure a phone number for a store,
• clear the application cache,
• go to the home page,
• check if the bar displays the data downloaded from configuration correctly.

Contrary to appearances, writing tests using MTF is not difficult, it only requires some experience and knowledge of all the structures used in tests (Fixtures, Blocks, Pages, Variations, Repositores, etc.).

Link to Magento DevDocs, where you can find more information about the described solution.