Automated tests with Functional Testing Framework in Magento

Care to share?

Magento Functional Testing Framework a way to seriously write automated tests in your next project?

What is a Magento Functional Testing Framework? 

Magento Functional Testing Framework aka MFTF is:

  • Open source test framework project for the Magento ecosystem
  • MFTF is the evolution of MTF
  • The first step to learning tests automation
  • Enables automation of regression tests

What do you need to do and learn to be able to start implementing MFTF in your projects?

  • No requirements for knowledge of programming languages, you should know basic of XML tags  and Xpath to find specific element on site
  • Minimum version -Magento 2.2
  • Selenium Server Standalone 3.1 or later
  • Composer 1.3 or later
  • ChromeDriver 2.33 or alternatives
  • Java 1.8 or later
  • You can install instance of Magento on your own physical machine and start to learning and contributing MFTF
  • All tests run in live mode or headless mode

Why we should automate testing in MFTF from the beginning of a project

  • Easy start to learning automation because of low entry threshold
  • Improving the efficiency of the development team’s work
  • Implementation of automated tests, regardless of whether they are in the Client’s budget, because it improves the quality of work and code
  • You can learn new dedicated solution to Magento
  • Contributing tests – benefit for You and Divante
  • Dedicated Magento solution
  • Save time- you can reduce manual testing after every deploy
  • You can add a test cases of new and custom functionalities

MFTF components

  • Test – whole test scenario
    {Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/
  • Annotations- comments .descriptions, titles..
    {Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/
  • Before & After– steps before and after test
    {Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/
  • ActionGroup- group of actions (fill the form..){Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/ActionGroup/
  • Page – modified concept of PageObjects {Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/Page/
  • Section – part of Page,  UI elements on a page used in a test( footer, header..){Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/Section/
  • Entity- set that constains data(weight, password, name…){Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/Data/
  • Metadata- Api requests(CREATE, DELETE..){Magento_Root}/app/code/Magento/{Module}/Test/Mftf/Test/Metadata/

MFTF executing flow

MFTF

Integrations MFTF with other tools

  • Report integration with Allure
  • Integration with Browserstack

How does the test execution look like in the console?

allure

Sample Allure report

MFTF

Disadvantages and problems of MFTF

  • The very complicated file structure in the tests directory
  • Hard to install and configuration
  • Infinite migration from MTF tests
  • No specific guidelines for creating new test cases
  • No coverage of all functionalities
  • Only on >= Magento 2.2

How to start contributing to MFTF tests?

Learn more from the full presentation:


Presentation originally presented at Magento Lightning Talks meetup on October 3rd, 2019, in Divante HQ. Magento Lightning Talks is an event series, organized by Divante, to share and exchange knowledge on Magento development among Magento developers.

Learn more at Divante.

Check out more Magento Lightning Talks.

Powerful Product Information Management for Magento. Explore the benefits >

Published December 10, 2019