From Knock to Insight: TinyML Ripeness Detection with AutoML and Zephyr®
From Knock to Insight: TinyML Ripeness Detection with AutoML and Zephyr®
2026年09月04日
Question:
If people can judge ripeness from a knock, why does an edge device need more than a microphone and a model to do it reliably?
Answer:
Because the knock is only a physical input, not the complete intelligence. A reliable edge AI system must convert a variable acoustic response into a stable, deployable decision. Knocking force, striking location, background noise, and fruit-to-fruit variation can all affect the signal, which makes simple feature thresholds difficult to generalize. In this design, signal processing extracts time-frequency features from the raw audio, AutoML helps search for a compact CNN architecture suitable for quantized TinyML deployment, and Zephyr provides the embedded software framework to integrate audio acquisition, feature extraction, inference, and display. The result is not just a model listening to a knock, but a complete edge pipeline that turns a physical signal into a reliable real-time decision.
Introduction
Judging the ripeness of a durian (a tropical fruit) by listening to the sound of tapping is achieved by analyzing its vibration signal characteristics generated during the knock. This method indirectly evaluates the physical state of the pulp inside the durians, such as hardness, density, and moisture content. The core principle is that durians at different ripeness levels exhibit different vibrational responses due to variations in their internal structure.
By analyzing the time-frequency features of the vibrational responses, the system can determine ripeness without cutting open the durian. This process avoids product loss, provides instant analysis results, and is suitable for large-scale screening. This technology has been increasingly applied in agricultural sorting and fresh food retail scenarios, making it a typical example of acoustic nondestructive testing used for assessing the quality of agricultural products.
Conventional approaches extract the spectral centroid (the center of mass of signal energy in the frequency domain) as a key indicator for classification. While effective under controlled conditions, the centroid is highly sensitive to external noise, striking force and location, and specimen-level variability (for example, fruit size, rind thickness, and spine morphology), which often leads to degraded robustness across batches and environments. In contrast, AI-driven methods train lightweight convolutional or sequence models directly on time-frequency inputs (for example, short-time Fourier transform (STFT) or mel spectrograms) to learn discriminative, noise-tolerant representations. Combined with data augmentation/ domain adaptation, these models markedly improve cross-scenario generalization. Moreover, with model quantization and neural architecture search (NAS), compact networks can be deployed at the edge with low latency and low power, meeting throughput requirements for production sorting and retail quality inspection. Consequently, compared with single centroid thresholding, AI-driven durian ripeness classification offers superior robustness and scalability for real-world deployment; complemented by uncertainty estimation and online recalibration, it further ensures long-term stability and maintainability in operational settings.
System Architecture Overview
Figure 1 illustrates the overall system architecture, which consists of three parts: an audio codec (MAX9867), an AI microcontroller unit (MCU) (MAX78002), and a TFT-LCD display module (ST7789V). In a typical use case, a segment of vibration or acoustic signal is generated by tapping the durian with a metal knife. The microphone/vibration sensor mounted on the front end of the knife will then collect its echo signal. The analog signal is digitized by the MAX9867 and streamed to the MAX78002 in I²S format. Within the MCU, signal processing (SP) algorithms will be implemented for feature extraction and convolutional neural network (CNN) inference. Finally, the classification result of durian maturity will be displayed on the TFT-LCD screen.
This work represents a complete solution, combining hardware, AutoML tools, and system-level software support. The ai8x toolchain handles model search and quantization, while CodeFusion Studio™ offers a Zephyr-driven development environment that simplifies integration and accelerates deployment.
Dataset and Signal Pipeline
As discussed in the introduction, durian’s fruit maturity can be inferred from its corresponding vibration frequency response. Therefore, the STFT algorithm is applied to extract this feature. Figure 2 illustrates the signal pipeline.
Firstly, a segment of one second raw acoustic signal is sampled at the rate of 8kHz. Then normalization is applied to eliminate the differences in signal strength. Next, STFT is implemented to obtain the signal’s corresponding amplitude spectrum. Finally, since the MAX78002’s CNN accelerator only supports squareshaped input data, the STFT spectrum is zero-padded and folded before fed into the TinyDurianNet for inference.
In this solution, STFT spectrums are converted into RGB images to be used as datasets. The initial dataset has 105 samples (35 for approaching maturity, 35 for fully mature, and 35 for immature). The data was augmented by randomly sliding the STFT spectrum in the window to analog the behavior of real slide window sampling, as shown in Figure 3. In this case, the dataset is updated to 1155 samples: 20% of the samples are used as a test set and 80% of the samples as a train set.
From Supernet to TinyDurianNet—How AutoML Iterates a Quantized Edge Model Automatically
In edge intelligence deployment, the design of neural network models is crucial. Usually, customers select example models provided by ai8x tools or design custom network architectures and then perform training and inference on their own datasets. However, the training results are usually unsatisfactory. This is because the quantized models used by the MAX78002 rely on low-bit weights (INT8/INT4/INT2), where each weight carries very limited information. As a result, compared with training conventional FP32-based models, achieving stable convergence becomes significantly more challenging, which, in turn, places greater demands on network architecture design. Therefore, adopting AutoML to search for an optimal model architecture can be an effective approach.
The ai8x provides an AutoML training pipeline called once-for-all (OFA). The goal of OFA is to train a model (supernet) once and automatically obtain multiple small subnets with compatible performance. It consists of three steps:
Supernet Training
A supernet is a huge CNN network with deeper layers, larger convolutional kernel size, and more channels. It is FP32-based and can easily overfit the classification task but cannot be deployed on the MCU directly. In this task, a sequential CNN-based supernet was manually created, as shown in Figure 4. The main ideas of OFA supernet training are knowledge distillation (KD) and shared weights. Remember that the purpose of OFA is to train one supernet and obtain multiple subnets automatically. Its goal is not to iterate a subnet with the highest accuracy because searching for an optimal solution is time-consuming. It requires that any subnet found during the search should not have a large accuracy deviation. Therefore, the trained supernet should be as robust as possible. KD is a machine learning training technique that enables the output performance of a student model (usually a small model) to be aligned with that of a teacher model (usually a large model). In the original article describing OFA,1 the teacher model is the target supernet, and the student model is an architectural subnet sampled from the supernet. Subnet sampling is enabled by the supernet’s elastic design, where kernel size, depth, and width can be reduced when forming smaller subnets. During each KD iteration, the student model reuses the supernet’s weights, and its updated parameters are written back to the supernet directly. The combination of KD and shared weights ensures the accuracy and consistency among subnets.
Network Architecture Searching (NAS)
After training a robust supernet, NAS is applied to automatically search for a small subnet, referred to as TinyDurianNet. OFA adopts an evolutionary search strategy to iterate subnet architecture. Compared with conventional NAS searching approaches such as reinforcement learning (RL)-based NAS or DARTS,2 the evolutionary search used in OFA requires fewer GPUs and less time cost, making the search process very efficient. The NAS procedure outputs multiple candidate subnets that offer similar accuracy but differ in architecture. Depending on the target hardware’s performance characteristics and power-consumption constraints, a suitable subnet can be selected as the final TinyDurianNet. For example, if accuracy is the priority, a subnet with more parameters can be chosen, whereas a smaller subnet with fewer parameters may be preferable for resource-limited or low-power devices. Figure 5 demonstrates the FP32-based TinyDurianNet obtained from the NAS process.
Quantization Aware Training (QAT)
In the original OFA article, after NAS found the sub-model (FP32 based), sub-model can be deployed without additional training. However, the MAX78002 CNN accelerators only support INT1, INT2, INT4, and INT8 weights and, therefore, QAT is required to enable TinyDurianNet to be adapted to the MAX78002. The core idea of QAT is to simulate low bit quantization errors during training (fake quantization) so the model learns to tolerate quantization noise in both forward and backward passes, preserving accuracy when deployed on low-precision hardware. For ai8x’s QAT, the straight-through estimator (STE) rule is applied in back propagation. Figure 6 demonstrates the process of QAT. After QAT, INT8-based TinyDurianNet can be obtained and deployed on the MAX78002 directly.
Results: AutoML Training and Performance
One NVIDIA GeForce RTX 4090 GPU was used to complete the overall AutoML training. Table 1 summarizes the OFA training configuration and its time consumption.
| Task | Learning Rate | Epochs/Iteration Times | Time Cost |
| KD-Stage0 (Full model training) | 0.001 | 200 | 1 h |
| KD-Stage1 (Elastic kernel) | 0.001 | 3000 | |
| KD-Stage2 (Elastic depth) | 0.001 | 2500 | |
| KD-Stage3 (Elastic width) | 0.001 | 9300 | |
| NAS | 50 | 10 mins | |
| QAT-subnet1 | 0.001 | 300 | 8 mins |
| QAT-subnet2 | 0.001 | 300 | |
| QAT-subnet3 | 0.001 | 300 |
Three subnets are trained from the OFA procedure, and one of them is chosen as TinyDurianNet. Figure 7 shows the TinyDurianNet inference accuracy on the test set.
Results: Real System Tests
Figure 8 shows the test platform. Some durian tapping vibration signals were recorded in advance, converting them into wave files and playout on a PC. An audio signal is input into the MAX78002EVKIT in a random timing sequence via an audio line.
The test results are shown in Figure 9.
According to the test results, the system can classify the durian correctly, which demonstrates strong robustness. Table 2 summarizes the inference speed and power consumption metrics of the system.
| Metric | Performance |
| CNN Inference Time | ~3.06ms |
| Power Consumption | ~180mW |
How Zephyr OS Strengthens the AutoML-to-Deployment Pipeline
By leveraging the ai8x toolchain, a trained model can be exported as a bare metal project that runs on the MAX78002, along with reference example code for interacting with the CNN accelerator. This allows customers to bring up the inference pipeline quickly. However, neural network inference is just one component of a complete edge AI system. As shown in Figure 1, the audio stream is transferred in I²S + DMA (direct memory access) mode, and the classification result also needs to be displayed on the LCD with GUI supported. Therefore, for a practical application system, peripheral control, dataflow control, and LCD human interaction interface (light and versatile embedded UI ecosystem (LVGL) used in this solution) also require development. As these subsystems grow in number and concurrency, the complexity and maintenance cost of bare metal architecture increase significantly, requiring manual porting and modification of multiple drivers and middleware. Due to this reason, this solution runs on Zephyr OS 4.1, which provides a device-tree-based hardware abstraction, a unified driver model, multithreading and synchronization primitives, DMA-backed peripheral subsystems, and modular build and package architecture.
These capabilities enable rapid integration of I²S acquisition, STFT feature extraction, CNN inference, and LCD interaction on the MAX78002, greatly reducing integration effort and accelerating the transition from algorithm to deployable edge AI product.
Conclusion
This article highlights how the MAX78002, combined with AutoML and Zephyr OS, enables a complete and highly efficient edge AI workflow—from raw signal acquisition to fully deployed TinyML inference. With real-time performance, low-power consumption, and minimal model design effort, the solution demonstrates a practical path for customers to integrate machine learning into sensor-based applications.
The same architecture can be quickly adapted to other SP + machine learning pipeline-based tasks, showing the versatility of ADI’s edge AI platform in accelerating development and enabling robust, production-ready intelligence at the edge.
References
1Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, Song Han. “Once-for-All: Train One Network and Specialize it for Efficient Deployment.” April 2020.
2Hanxiao Liu, Karen Simonyan, Yiming Yang. “DARTS: Differentiable Architecture Search.” April 2019.
著者について
Zhiheng Li is a product applications engineer at Analog Devices CAC team in Shanghai, China. His work focuses on advanced audio and video signal chain applications, including DSP, ADI A2B™, GMSL™, and AI-enabled systems.
Tong Zhang is a senior software systems design engineer at Analog Devices in Nanjing, China. He focuses on embedded software and system-level design, supporting ADI A2B™, DSP, FPGA, and edge AI applications. He joined ADI
Chenhui Xu is a senior software engineering Manager at Analog Devices APGC system solutions group. He is responsible for a software engineering team dedicated to software solution development and field support for SAP and