Every tester, technician and engineer we know would gladly let an automated test bench execute the painstaking and repetitive job of verifying the quality (See Verification & Validation) of the electronic systems they aim to deliver. Automation is thus an objective shared by mostly everyone in an electronic design business. But why so exactly?
Automation has much more to offer than to simply alleviate the boredom of the would-rather-not-be testers. We will here aim to list and explain all the advantages which can be garnered by diving head-first into the world of electronic hardware test automation.
Automation frees skilled resources
The first advantage of automation is to free the skilled resources which should be tasked (and let’s say it, would prefer to be tasked) on higher value endeavours.
R&D engineers and technicians
As they are the one who know best how these systems operate, in a no-automation case, the R&D engineers and technicians will spend a lot of time implicated in the execution or the supervision of tests. These resources should be designing the next product, not be tasked on the delivery of the previous one.
However, simply opting for automation will not necessarily free the R&D resources if they are the one automating and maintaining the production line. To really optimize the talents of the R&D personnel, the choice to dedicate an engineer (or a team) to transfer to production must be made. These resources will be allowed to focus on the quality of the systems, the efficiency of the production line and helping the testers operate and maintain the testbenches when needed. They will also be able to feed their expertise in new designs to improve testability. The remaining R&D’s involvement would be to guide the dedicated resources through the system’s specifications and operational objectives to allow them to design tests of the required coverage.
Production technicians and debuggers
When you are blessed with resourceful production technicians, you would rather use their skills on higher value tasks than manual test execution. When a testbench is automated, they can still operate it while using the test time for reparation and debugging of units failing the test. It allows for the production of compliant systems faster and allows your resources to enjoy more fulfilling tasks.
Automation improves standards of quality
Another advantage of automation is the improvement in the standards of quality of the delivered systems.
Automation allows high volume while increasing traceability and quality assurance.
Manual tests are prone to errors. They need complete and bullet proof test procedures to ensure proper execution. Even then, their results can be unreliable as there is no tangible proof the test has been executed according to the procedure.
Automation allows the direct enforcement of the test procedure and thus provides procedural certainty. Each test is executed exactly the same way every time, without any logging or manipulation error. It minimizes the occurrence of the worst possible outcome of a test -the false PASS- which leads to a faulty unit being delivered.
Another improvement to quality is the standardization of test outputs. Manual testing leaves you at the mercy of the goodwill of the tester to note every result as they are mandated. Supervising, verifying and enforcing the correct logging of results can be time consuming. Flaws in the logging of test outputs are generally found too late.
A well automated testbench will include standardized test results which will allow a quick verification of compliance and comparison between units. This will lead to gains in debugging time as the cause of failure can be found quicker in well defined results. It also increases ease of traceability when recalling past results in the case of an in-the-field event or return of merchandise.
Automation increases test efficiency
Manually testing an electronic system is not efficient. It takes time and is not cost-optimized, especially if you use skilled resources for the execution. One of the main advantages of automating a test is to improve its time and cost efficiency.
It is straightforward that automating the test will reduce its execution time. A well programmed computer will configure systems and test equipment, execute measures, compare criteria and output results faster than a human can. As discussed above, automation can also reduce human error and improves test reliability. This limits the occurrences of unnecessary retests and can save a lot of time.
So, even as total test time is down, the real efficiency gains come with the reduction of active test time. Tasking your skilled resources on other duties while tests execute drastically reduce test costs. Automation can therefore increase produced unit throughput while reducing per-unit cost.
An automated testbench can also improve debugging operations efficiency. For example, a test could execute supplementary, debug-targeted measures once a failure is detected, allowing for faster analysis of the problem and a smaller turn-around time for reparation.
Risks of automation
Unfortunately, automation can also create its load of problems if you are not careful.
Automated production lines need maintenance and improvements, or they do not keep up with changes to the product and the needs of the production department. Not executing maintenance on the line is essentially a ticking time bomb, as it is inevitable that the production will eventually cease at the worst possible moment due to a break in the process. If such an unexpected halt occurs, the turn-around time to solve the issue is usually large (and definitely too long for everyone involved).
It is therefore very important to plan testbench maintenance from the start in order to prevent halts or to drastically shorten halts if they do occur. For example, having redundancy in test equipment, or being able to quickly edit the test bench software and redeploy it are examples of planifiable pre-maintenance.
Production line readiness delays
As stated before, time-to-market is one of the most important objectives of a new product. Having delivery delays caused by an unfinished production line is a situation nobody wants to be in. Let’s just think of Tesla’s pain with the Model 3 delays. The good news is that, if planified ahead of time, humans can always bridge the gap of unfinished automation.
When automating, we often aim for the easy tests first and leave the risky automation heavy tests for the end. The rationale is that we will have 90% of the tests done in 50% of the projects, and thus automation will be mostly done. Based on our experience, it is important to go the other way around.
These simple tests, for example scanning an I2C bus for devices or reading a temperature sensor, are easily executable by humans, and in the case of an unfinished automation, will not completely halt deliveries. The development heavy tests which we usually keep for the end are the ones humans cannot execute. Front loading them allows the possibility to use them even in case of delay.