A convolutional neural network with image and numerical data to improve farming of edible crickets as a source of food—A decision support system

Abstract

Crickets (Gryllus bimaculatus) produce sounds as a natural means tocommunicate and convey various behaviors and activities, including mating,feeding, aggression, distress, and more. These vocalizations are intricately linkedto prevailing environmental conditions such as temperature andhumidity. Byaccurately monitoring, identifying, and appropriately addressing these behaviorsand activities, the farming and production of crickets can be enhanced. Thisresearch implemented a decision support system that leveragesmachinelearning (ML) algorithms to decode and classify cricket songs, along with theirassociated key weather variables (temperature and humidity).Videos capturingcricket behavior and weather variables were recorded. From thesevideos,sound signals were extracted and classified such as calling, aggression, andcourtship. Numerical and image features were extracted from the sound signalsand combined with the weather variables. The extracted numerical features,i.e., Mel-Frequency Cepstral Coe cients (MFCC), Linear Frequency CepstralCoe cients, and chroma, were used to train shallow (support vector machine,k-nearest neighbors, and random forest (RF)) ML algorithms.While imagefeatures, i.e., spectrograms, were used to train different state-of-the-art deepML models, i,e., convolutional neural network architectures (ResNet152V2,VGG16, and E cientNetB4). In the deep ML category, ResNet152V2had thebest accuracy of 99.42%. The RF algorithm had the best accuracy of 95.63% inthe shallow ML category when trained with a combination of MFCC+chromaand after feature selection. In descending order of importance, the top 6ranked features in the RF algorithm were, namely humidity, temperature,C#, mfcc11, mfcc10, and D. From the selected features, it is notablethattemperature and humidity are necessary for growth and metabolicactivitiesin insects. Moreover, the songs produced by certain cricket speciesnaturallyalign to musical tones such as C# and D as ranked by the algorithm.Usingthis knowledge, a decision support system was built to guide farmers aboutthe optimal temperature and humidity ranges and interpret the songs (calling,aggression, and courtship) in relation to weather variables. With this information,farmers can put in place suitable measures such as temperature regulation