MES performance should be tested by allowing the application to manage multiple transactions simultaneously right from moving material in and out, split & merge lots, to tracking material to and from the process and the overall process flow, all the while capturing engineering and equipment data for reporting and analysis. This would result in thousands of transactions with different hierarchy and importance.
An MES which is built to accommodate this complexity would be able to capture these transactions and its performance can be evaluated when this transaction loop is conducted across the production lines. Only if the application being evaluated can capture and report all transactions in the real-time, can it be deemed fit for other functionality based evaluations which may range from how good the SPC is to whether or not the LIMS tool provides in-depth insight of the operation.
Coming back to the main performance indicators, it should be clearly understood by the MES project team that the ability of an MES application to handle current complexity is not the testimony of its capability to handle future operations.
MES applications should be tested for scalability with addition of most cost effective and minimalistic hardware.
This is where the boys are separated from the men so as to speak.
An MES application which is truly world class, should be able to prove its performance and scalability in scenarios where the current complexity may be doubled or quadrupled.
In degree of importance, the ability of the application to capture and report shop-floor transactions with all inherent complexities in real-time is the most important. This is closely followed by the capability of the application to be able to scale up to requirements of increased volume, material, equipment and overall increase in transactions without affecting the speed of delivery in terms of data capture and report.
Then the application should be capable of working with standard hardware and should be capable of scaling up with addition of such hardware increasing cost effectiveness of the project overall. Only when an MES vendor can provide clear proof that the software they are marketing is capable of meeting these criteria it should be further evaluated, since the application which itself lacks in performance can’t possibly improve the performance of the plant’s operation.
Performance should be tested by analyzing data from the vendor and understanding how the application is working where it is currently deployed. Another way would be to run pilots of the few selected applications at the most demanding units to gauge whether the MES application can sustain the pressure or fails to deliver.
Complex processes tend to change quickly
At all times while testing the application, it should be kept in mind that complex processes tend to change quickly with the change in process related technology and today’s standards won’t necessarily be relevant tomorrow.
The hardware being used or required for the application would also reflect on the way the application will perform in near and distant future. Best applications are able to work with standard software as they are able to reduce the message footprint while retaining the important information of the message. This reduction in footprint and contention time speaks volumes about the performance and capability of an application.
Tests should be performed to analyze complex material flow scenarios, where higher numbers of materials pass through the process, in a time critical manner to establish how fast the MES application performs right from the creation of the material or sub-material type, to its flow in the process and replenishment in the fab.
Some MES applications may fail to accommodate and perform in such demanding conditions and this would be because of the way they are designed. The best applications can easily accommodate these needs as they are configured to work with complex flows, housing their functionality in a single layer, allowing them to scale up or down at the least cost.
So when choosing the MES application, remember the importance of scalability, time critical response, hardware requirement, material/process flow requirements and their combined effect of the overall performance of the MES. Only an MES capable of performing under current complexity and of allowing scalability for future high volume/complexity should be further analyzed functionally to be selected for you operation.
See also:
Test Report: Affordable MES. Performance and Scalability for Time-Critical Industry Environments by Critical Manufacturing