Show me that it works – Automated Functional MES Testing

In a previous article, we discussed how essential functional testing is for creating a reliable, high-quality MES solution. Depending on the complexity of business operations and processes in your MES, performing manual functional tests may prove to be tricky and require careful design and attention to detail. Automating these tests can add another layer of complexity and be a challenge – read on to find out why.

Wednesday, June 24, 2020
Show Me That It Works – Automated Functional MES Testing Banner

Finding a framework

Most of the commercially available MES platforms (ex. DELMIA Apriso) come with some out-of-the-box functionalities that can be customized to meet the company’s specific business processes and industry requirements.

The MES platform is usually available as a desktop or mobile application that generates HTML pages that users can view and interact with. The fundamental difference, however, is that unlike regular webpages, parts of those MES ‘pages’ are generated dynamically. For example, IDs of the scripts that control their behavior can change with each refresh of the page. If a framework were to latch on to those IDs, which is the standard setup, they would not be able to reliably assert, e.g., if an element was successfully displayed or not, resulting in a false-negative or a false-positive result.

Fortunately, you can adapt most frameworks so that they can locate other web elements – like the market-leading Selenium Web Driver, which is essentially an API designed for controlling browser actions. Based on that, a framework for handling MES pages can be developed, using, for instance, Behave or Robot Framework (both of them are Python frameworks).

Once the framework is built and configured, all it needs are periodic adjustments to the upgraded versions of the MES. Usually, this requires changing a dozen or so lines of code.

Emulating PLCs - the elephant in the room

Finding and customizing software to automate ERP and MES tests is relatively easy. The real head-scratcher is simulating industrial automation. PLC (Programmable Logic Controller) systems work in real-time, and no one in their right mind would want to start an actual industrial oven several times over to check, e.g., the heat curve parameters or product recipe calculations.

Theoretically, it should be possible to simulate PLC signals and behavior using web services – they are part of the OPC Unified Architecture standard (designed so that machines could communicate in a unified fashion). However, some industry-leading OPC server companies do not provide this as standard or offer solutions that require a workaround.

And why is it so crucial to find one? PLC systems are the lowest-level software that interacts directly with shop floor machines. With the ERP or MES systems malfunctioning or down, shop floor operations can continue for some hours, sometimes several days – without industrial automation, production can stop in a matter of minutes. Making sure that PLCs can handle normal operations (and perhaps some unusual ones, too) is essential for any sense of production security.

Despite the stakes being so high automated testing of industrial automation is still not an industry-wide standard, and surprisingly few companies include it in their testing processes.

PLC logic is implemented in one of the PLC languages to run a real automated end-to-end test. This logic (i.e., PLC signals) must then be simulated and translated into the programming language of the automated test framework (e.g., Python, Java, or C#). PLC behavior can be reproduced in several ways, including SCADA systems (which supervise the production process).

However, the problem with SCADA systems is that they only provide the infrastructure (which can be quite time-consuming to set up). You will still need someone to design, implement, and run tests on this infrastructure.

At Andea, we came up with a SCADA-less, multithreading solution that is straightforward to set up and more intuitive and transparent to use.

E2E testing – everything A to Z

  • The best way to test the functionality of business flows is end-to-end testing that not only checks the functioning of ERP, MES, and industrial automation systems separately. It also verifies how they all work together.

    The challenge is doing all these things with a single test automation framework while also simulating the behavior of PLCs. Because there is currently no such framework on the market, we have developed our own at Andea. We have tailored it to the requirements and architecture of MES systems and industrial automation. It covers all the necessary aspects:

  • Emulating the ERP messages
  • Checking the MES screens operation by operation
  • Simulating the PLC signals
  • Verifying the results saved in the database

Thorough and comprehensive functional testing of the MES software within its ERP and automation ecosystem is vital for fostering confidence and trust in your solution. At Andea, we have years of experience researching and customizing tools to deliver just that. So why reinvent the wheel?

Baner CTA Background

Contact us for details and see how we can put our automated testing experience to use and help you get more out of your MES implementation.

Contact Us

About the author

Łukasz Howorski is a Project Excellence Department Manager and a QA Architect with rich and diverse experience in test management, execution, and test automation. ISTQB and AGILE certificates help him create full test strategies that are tailored to client requirements. His technical and organizational skills allow him to implement frameworks not only in the test automation field but also in improving test processes using different approaches. He likes sharing his findings with the team, improving their skills one nugget of knowledge at a time.


About the co-author

Zofia Wąchocka is a Software Tester at Andea, who has worked for several US customers specializing in composite materials manufacturing. A graduate of the Jagiellonian University in Kraków, her analytical approach to life and a drive for communication and comprehension brought her over to the tech world, where she thrives asking questions, sharing knowledge in webinars and articles, and learning test automation.