Climate Change, a highly debated high school topic, but do we really care about it? We see politicians, scientists, researchers, philanthropists, conservationists and almost everyone blaming each other for every climate catastrophe occurring today. The real question is what climate change or change are we talking about?
CliSensio – Climate Sensing and Insect Infestation Control is a new adaptive device that runs SensiML AutoML audio recognition on the edge to recognize disease spreading vectors, hungry pests and threatful species. CliSensio device firmware can be flashed for mosquito species identification, pest, rodents identifications and elimination responses.
Change is never easy for any species or life form but it happens in order to survive and evolve and this is the change we need in order to challenge climate change – the change could be in form of laws to reduce carbon emissions, conserve biodiversity, advanced technologies which can outperform existing techs in terms of greener energy or total innovative revamping of traditional ways of all human activities.
According to soil scientists, at current rates of soil destruction (i.e. decarbonization, erosion, desertification, chemical pollution), within 50 years we will not only suffer serious damage to public health due to a qualitatively degraded food supply characterized by diminished nutrition and loss of important trace minerals, but we will literally no longer have enough arable topsoil to feed ourselves.
Without protecting and regenerating the soil on our 4 billion acres of cultivated farmland, 8 billion acres of pastureland, and 10 billion acres of forest land, it will be impossible to feed the world, keep global warming below 2 degrees Celsius, or halt the loss of biodiversity. Important Note: This project was the result of a discussion with my parents on environment conservation especially saving food crops, beneficial insects and supporting our farmers.
We need to understand a very critical thing- “We all know the changes happening, we all can see climate catastrophes and loss of living organisms and their habitats but we haven’t made much effort in saving those because over the past decades we have only made detection and warning systems when the world needs self-activation systems to control the situation.” Those words were like North Star for my project development. Also, this project was initiated on May 20, World Bee Day – aiming to protect the unsung heroes of nature. My innovation and solution here are to ‘adapt and mitigate’ and not to ‘detect and forget’.
Supplies
- QuickLogic Corp. QuickFeather Dev Kit with UART Cable + SensiMLhe QuickFeather is powered by QuickLogic’s EOS™ S3, the first FPGA-enabled Arm Cortex®-M4F MCU to be fully supported with Zephyr RTOS. AutoML tool enabled. All the steps mentioned can be utilized for Arduino Nano BLE 33 Sense, since SensiML provides support for Arduino as well, the only thing you need t do in last step is to get Arduino nano BLE 33 firware after ML from SensiML
- Seeed Grove – Water AtomizationAtomize liquids, perfume or solution in vapour form to take your project to an advanced level. The liquid is fed through the centre of a nozzle with a relatively large orifice, without pressure, and is atomized due to ultrasonic vibrations in the nozzle.
- Solar Panel, 2.5 W
- SparkFun Sunny Buddy – MPPT Solar Charger
- SensiML studio
Project source codes:
https://github.com/vilaksh01/CliSens.io
Step 1: Background Research
1. Infestations of Insects turning Farmland into Deserts:
A peer-reviewed study stated that the threat of rising temperatures and unpredictable weather patterns has multiplied tremendously. By 2050, droughts and insect infestations could destroy 26.2 per cent of the World’s largest tea exporter, Kenya’s tea-growing areas. A research study concluded that climate change is more dangerous than we can imagine, it could empty ‘World’s Grain Bowls’ – Rising temperatures will stimulate insects’ appetites — and make some prone to reproducing more quickly — spelling danger for key staples like wheat, corn and rice which feed billions of people.
While the world is still reeling from the coronavirus pandemic, several countries Brazil, Argentina, India, Pakistan and other East African nations are facing locust attacks that threaten to trigger food crises in them.
“When it gets warmer, pest metabolism increases, ” said Scott Merrill, a researcher at the University of Vermont and co-author of the study in the journal Science. And when pest metabolism increases, insect pests eat more food, which is not good for crops. Overall, “global yield losses of these grains are projected to increase by 10 to 25 per cent per degree of global mean surface warming, ” said the report. – http://13.235.153.162/world/global-warming-will-m…
- France stands to lose about 9.4 per cent of its maize to pests in a world that is 2 degrees C warmer, compared to about 6.6 per cent of yield losses today due to pests.
- In Europe, currently, the most productive wheat producing region in the world, annual pest-induced yield losses could reach 16 million tons.
- Eleven European countries are predicted to see 75 per cent or higher losses in wheat from pests, compared to current pest damage.
- In the United States, the world’s largest maize producer, insect-induced maize losses could rise 40 per cent under current climate warming trajectories, meaning 20 fewer tons of maize per year.
- China, home to one-third of the world’s rice production, could see losses of 27 million tons annually.
Pandemic, war, climate change fuel fears of global hunger, food scarcity. The first challenge is to accept the problem. The second one is to accept responsibility. Climate change is unlike any challenge humanity has faced because, unlike pandemics and wars, here we have a warning. We have time to prepare for the impending disaster, but we are not exactly making good use of it. We’re arguing and grandstanding, instead of acting. When the oceans rise, there is no fix. When the air becomes unbreathable, masks will not work. – The world needs to come together and find solutions to the climate disaster problems.
http://13.235.153.162/world/climate-change-the-act…
2. Billions at risk of mosquito-borne diseases:
Climate Change Will Expose Half of World’s Population to disease-spreading Mosquitoes By 2050. According to a study from PLOS Neglected Tropical Diseases by the end of this century, almost all of the world’s population would be exposed to mosquito-borne diseases. Aedes aegypti and Aedes albopictus are two species of mosquitoes known to carry and transmit potentially deadly viruses like dengue, chikungunya, Zika and yellow fever. These bugs can thrive in urban environments, often travelling alongside humans as we ship goods and travel around the world. And, according to the study, these tropical mosquitoes are poised to expand their ranges, exposing a billion additional people to the diseases they carry in the next 50 years. It’s important to note that this study is on a global scale, and mosquito management efforts will need nation-by-nation or even city-by-city information while trying to combat the viruses. Detecting mosquito breeding grounds is more difficult which is one of the reasons for such a widespread population of mosquitoes.
With a changing climate and extreme weather like floods expected to become more frequent and intense, this could mean more outbreaks of mosquito-borne diseases such as dengue in the coming years. New mosquito species would pose more burdens on the public health system.
https://news.mongabay.com/2021/03/as-climate-chang…
Western Germany is cleaning up the worst natural disaster to hit the country in almost 60 years. The flood has been devastating for all Europeans.
3.Losing Grassland Birds is another Widespread Ecological Crisis:
Growing unnecessary and competitive use of pesticides and insecticides that sweep out all insects affect the entire food chain of birds. Moreover, the insects are becoming resistant to the chemicals day by day but the birds which eat those insects barely survive for reproduction. Using lots of chemicals to boost farm yields but we tend to forget that due to excessive use of those pesticides, insecticides and inorganic manures, cross-pollinating agents, the arthropods( 90% of living organisms found on earth are insects) don’t get attracted to those plants or otherwise die due to excessive chemical exposure and pollution. More than 2, 000 pollinating insects are extinct by now and only 500 are left to become extinct if not protected – “Even if there is no insect disease in farms, farmers uselessly spray chemicals and kill useful insects like honeybees and other birds too.”
4.Eliminating threat species to our pollinators(our best friend, bees):
Insects are pollinators and waste management experts, controlling other insects, keeping the soil healthy while also serving the most important function – being food for those higher up the food chain. In the absence of pollinators, many plant species will disappear.
Also, scientists are now on a hunt for the hornets, hoping to eradicate the species before they wipe out US bees. According to the scientists of the Washington State University (WSU), “it’s a health hazard, and more importantly, the shockingly large hornets are a significant predator of honeybees.” They will begin trapping queen murder hornets this spring, aiming to detect and eradicate the species.
Step 2: Current Solutions VS CliSensio Solutions:
1. As far I have researched the only potential solution in long term for insect infestation control is to develop a model to predict breeding areas that may have been missed by ground monitoring- ”The focus will be on stopping hoppers from becoming adults, as that leads to another cycle of infestation. We want to avoid that. We want to advise governments early before an upsurge happens.” So far it has attracted an investment of £35 million by the United Kingdom as part of its Weather and Climate Information Services for Africa program($60m but, if an upsurge occurs, the cost will soar to $500m) to forecast the movement of locusts using data such as wind speed and direction, temperature, and humidity.
Resource-deprived countries have already started deploying aircraft and drones to spray insecticide on the swarms to deal with the situation. But figuring out the exact time and location of the swarms has remained a challenge for them. We need something small, feasible and as the first order of defence to prevent future locust or other insect outbreaks: – Our CliSensio device is here to solve this challenge.
2-3. For saving the grassland birds and beneficial organisms, the only solution is to avoid the unnecessary use of insecticides and pesticides. Our CliSensio device will stop the competition among the farmers for using excess fertilizers, pesticides, insecticides to boost production. Using the correct farm audio analysis the device would auto-assist and spray insects/pests repellents with precise localization in the farm without chemical overdosage or wastages and thus save farmers money, efforts and stop the degradation of the environment.
4. Most mosquito-breeding spots are found in homes and not at construction sites. This makes the job of detecting breeding spots harder because, unlike entering construction sites, the authorities cannot go into homes easily or frequently to check on them. We need such a solution that works on the ground level autonomously for everywhere including home. Our device CliSensio has a special feature to combat this problem at community and household levels. Source:
https://www.straitstimes.com/singapore/why-detecti…
The heart of the device is QuickFeather EOS3 https://www.quicklogic.com/products/soc/eos-s3-mi… All SensiML AI algorithms are going to be performed on this device, however, the application can be modified for Arduino Nano BLE 33 Sense boards as well since SensiML provides Arduino board firmware SDKs too. See all the images, for device setup.
How is liquid-chemical atomizer utilized in Clisensio?
The liquid atomizer has two components, a piezoelectric atomising head and a driver circuit. A wick is used for supplying the chemical liquid to a piezoelectric atomising head arranged in the spray apparatus. In CliSensio, the devices are used for chemical liquid spraying for targeted species,i.e. insect removal, insect growth retardation, insect repellent, disinfection based upon SensiML audio recognition and feature extraction models trained to detect harmful pests and insects.
Furthermore, the present innovation relates to a method for controlling harmful organisms by effectively and economically releasing active ingredients having the functions of insect removal, insect growth retardation and insect repelling in the air. Stopping harmful insect and pest breeding in their growth period, saving farmers and communities from any perils. The method would massively reduce the overdosage of chemicals used in traditional agriculture or insect control.
Step 4: Device Firmware Setup: Part 1 – Mosquito Breed Classification
The device firmware is based on QORC-SDK,
https://github.com/QuickLogic-Corp/qorc-sdk all instructions are well documented here.
Clone this QORC SDK repository using the shown command, and check if make and GCC compiler is set correctly or not (if not follow the steps in GitHub link README.md from above link).
$ git clone --recursive https://github.com/QuickLogic-Corp/qorc-sdk $ gcc -v & make -v
All the data extraction and training have been done using SensiML Analytics Studio and SensiML data capture lab: https://sensiml.com/products/ Currently data capture lab is only for windows platform, that’s one downside for Linux and Mac users, but I managed it since I have both systems at my hand(windows and Linux separately not WSL). Our CliSensio device runs two different firmware:
- Mosquito Species Detection and Control
- Pest Detection and Control
1. Mosquito Species Detection and Control
- Data Collection: Since we are using the microphone as the only data source for our CliSensio (I had planned to bring in audio as well as sensor data into the project but currently either sensors or audio ML models are supported at a time, not both together.) The audio dataset was already collected, this project is inspired by the other projects I made in past to solve pest and insects problems, links here: 1. https://www.hackster.io/vilaksh01/tensorcrop-crop… 2. https://www.hackster.io/vilaksh01/tensorcrop-crop… All audio data used is uploaded on this Github page: https://github.com/vilaksh01/CliSens.io/tree/main… However, I recommend you to record it via your own device through the SensiML data capture lab to achieve more accuracy, as those audio files are not recorded on the same microphone architecture and bit rate.
- Data Segmentation: This was performed through the Data capture lab after the data collection step. I recommend using Audacity to check spectrograms for the audio data before proceeding to segmentation, it gives clear analysis about noise-to-valuable data in audio files.
- Training and Saving model: Log into your SensiML cloud or Data Analytics Studio to check our SensiML data and proceed with training steps. We trained on two pipelines to make the model more efficient and accurate. Pipeline 1 Check the below images carefully for all steps.
- Pipelines for our model: I tried testing two pipelines for mosquito breed detection with different Optimization metrics and Feature Extractors. Pipeline 1, is default parameters with f1-score as optimization metric whereas Pipeline 2 has Sensitivity as Optimization Metric and unknown classifier parameter is allowed with custom feature generator set.
- Advantages of using Sensitivity as Optimization Metric (Ratio of true positives to total or actual positives in the data.) Important when: Identifying the positives is crucial.Used when: The occurrence of false negatives is unacceptable/intolerable. You’d rather have some extra false positives (false alarms) over saving some false negatives. For example, when predicting financial default or a deadly disease.
- Visualizing Model Summary: This is one of the most important and highly rewarding features for ML training engineers(they get to know their models better).
Our generated model JSON file for Pipeline 2, check the class maps- 1. Aedes Aegypti- is known to transmit the dengue virus, yellow fever virus, chikungunya virus, and Zika virus. It is suggested to be a potential vector of Venezuelan Equine Encephalitis virus and is capable of transmitting the West Nile virus., 2. AedesAlbopictus- This mosquito species is a known vector of chikungunya virus, dengue virus and dirofilariasis., 3. AnophelesMinimus- association with malaria, filariasis and arbovirus infections., 4. Background- Noises like wind, white noise, etc., 5. Culex tarsalis- spread the viruses that cause West Nile fever, St. Louis encephalitis, and Japanese encephalitis, as well as viral diseases of birds and horses., 0. Unknown Classifier;
{"NumModels": 1,"ModelIndexes": {"0": "Pipeline_2_rank_4"},"ModelDescriptions": [{"Name": "Pipeline_2_rank_4","ClassMaps": {"1": "AedesAegypti","2": "AedesAlbopictus","3": "AnophelesMinimus","4": "Background","5": "CulexTarsalis","0": "Unknown"},"ModelType": "PME","FeatureNames": ["channel_0PctTimeOverZero","channel_0PctTimeOverThreshold","channel_0Mean","channel_0NegativeZeroCrossings","channel_075Percentile","channel_0Sum","channel_0ZeroCrossingRate","AvgSigMag","channel_0TotArea","channel_0TotAreaDc"] "AIF": [93,33,23,131,52,54,47,86,51,76,120,61,113,83,85,80,64,87,23,52,36,56,75,25,24,71,107,23,20,22,23,137,76,43,20,28,53,44,23,133,24,22,48,120,95,117,26,21,41,27,123,25,30,112,132,67,46,20,46,20,52,61,78,58,30,25,107,45,78,134,160,160,160,160,160],"Category": [4,2,3,4,5,4,5,2,5,3,4,3,5,5,5,3,3,4,2,3,5,2,3,2,2,1,2,1,1,1,2,2,5,2,2,1,3,5,2,3,1,1,2,4,3,4,1,2,2,2,2,2,1,3,2,5,1,2,3,1,2,4,2,1,2,1,5,2,1,1,2,2,1,1,3],"Vector": [[208,208,210,52,89,210,58,210,210,15],[59,59,180,90,19,180,84,181,180,21],[196,196,195,57,35,195,65,195,195,3],[30,30,196,0,22,196,47,196,196,1],[152,152,197,24,35,197,22,197,197,0],[169,169,195,2,24,195,32,196,195,2],[195,195,197,20,27,197,41,197,197,0],[123,123,181,62,26,181,79,181,181,21],[197,197,197,61,29,197,91,197,197,1],[154,154,194,13,25,194,71,195,194,3],[100,100,195,0,23,195,31,195,195,3],[198,198,194,110,51,194,110,194,194,3],[132,132,197,16,69,197,14,197,197,1],[201,201,197,104,33,197,126,197,197,0],[212,212,197,25,85,197,25,197,197,2],[175,175,195,79,34,195,89,195,195,3],[204,204,193,34,118,193,33,193,193,6],[152,152,195,0,24,195,16,195,195,3],[65,65,181,59,20,181,57,181,181,21],[182,182,195,38,29,195,40,195,195,3],[199,199,197,39,29,197,59,197,197,0],[121,121,181,178,25,181,180,181,181,21],[147,147,194,45,30,194,37,194,194,4],[50,50,180,74,19,180,70,180,180,22],[49,49,181,59,19,181,67,181,181,21],[110,110,180,134,24,180,128,180,180,21],[201,201,181,41,45,181,47,181,181,20],[61,61,180,45,19,180,56,180,180,22],[36,36,180,49,18,180,46,181,180,21],[52,52,181,53,19,181,55,181,181,21],[30,30,180,54,18,180,45,180,180,22],[164,164,180,87,40,180,86,180,180,21],[229,229,194,16,60,194,16,195,194,3],[114,114,181,37,25,181,33,181,181,20],[78,78,180,78,21,180,67,180,180,22],[64,64,180,52,20,180,45,180,180,22],[214,214,195,67,40,195,66,195,195,3],[194,194,197,54,37,197,62,197,197,0],[74,74,180,48,20,180,41,180,180,22],[182,182,194,121,44,194,123,195,194,3],[95,95,181,37,22,181,50,181,181,21],[78,78,181,52,21,181,56,181,181,21],[94,94,180,39,22,180,35,180,180,22],[208,208,202,36,111,202,44,202,202,6],[193,193,194,67,42,194,70,194,194,4],[189,189,204,50,67,204,56,204,204,10],[80,80,181,52,20,181,40,181,181,20],[51,51,180,66,19,180,52,180,180,22],[59,59,181,24,18,181,26,181,181,21],[44,44,180,32,19,180,25,180,180,22],[148,148,181,39,45,181,38,181,181,21],[15,15,180,7,17,180,7,181,180,21],[80,80,180,131,21,180,124,180,180,22],[230,230,194,52,54,194,52,194,194,4],[83,83,180,141,21,180,137,180,180,21],[183,183,197,24,37,197,22,197,197,1],[78,78,181,76,21,181,80,181,181,21],[14,14,180,21,17,180,17,180,180,21],[196,196,193,33,55,193,32,193,193,4],[53,53,181,48,19,181,40,181,181,21],[59,59,180,62,19,180,53,180,180,22],[193,193,196,4,25,196,9,196,196,1],[80,80,180,65,20,180,47,180,180,22],[37,37,181,64,18,181,60,181,181,21],[35,35,180,37,18,180,28,180,180,22],[31,31,181,23,18,181,25,181,181,21],[197,197,195,17,55,195,15,196,195,2],[30,30,181,41,18,181,39,181,181,21],[23,23,180,29,17,180,26,180,180,22],[53,53,180,40,19,180,33,180,180,22],[6,6,181,10,17,181,6,181,181,21],[105,105,181,53,24,181,40,181,181,21],[68,68,181,88,20,181,93,181,181,21],[124,124,180,201,26,180,202,180,180,21],[168,168,195,34,43,195,32,195,195,4]],"Identifiers": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74],"DistanceMode": 0}]}
- Integrating Knowledge Pack with QORC-SDK and our firmware: The app structure is similar to qf-apps/qf_ssi_ai_app in QORC-SDK. The first step is to set up our GPIO pin for the liquid-atomizer module in the src/pincfg_table.c file.
{ // setup Water Atomizer Digital Output.ucPin = PAD_23,
.ucFunc = PAD23_FUNC_SEL_GPIO_7,
.ucCtrl = PAD_CTRL_SRC_A0,
.ucMode = PAD_MODE_OUTPUT_EN,
.ucPull = PAD_NOPULL,
.ucDrv = PAD_DRV_STRENGTH_4MA,
.ucSpeed = PAD_SLEW_RATE_SLOW,
.ucSmtTrg = PAD_SMT_TRIG_DIS,},
After setting up GPIO let’s modify the src/sml_output.c function to direct the action response based upon audio classification or inference performed. Also, from here we can see the serial communication prints out classifications which can be used to send to the dashboard for Spatio-Temporal Vector Ecology map over LoRa or cellular, helping the government to be prepared for unforeseen events(epidemic or pandemic).
static void sml_output_serial(uint16_t model, uint16_t classification)
{
int count;
int wbytes = 0;
int buflen = sizeof(sensor_ssss_ai_result_buf)-1;
int ret;
kb_get_feature_vector(model, sensor_ssss_ai_fv_arr, &sensor_ssss_ai_fv_len);// you can send other sensors or GPS data in below line for dashboard
count = snprintf(sensor_ssss_ai_result_buf, buflen,"{\"ModelNumber\":%d,\"Classification\":%d", (int)model, (int)classification);
wbytes += count;
buflen -= count;
switch((int)classification)
{
// case if unknown audio found
case 0:
HAL_GPIO_Write(GPIO_4, 1); //blue LED for unknown
HAL_GPIO_Write(GPIO_5, 0); //green LED off
HAL_GPIO_Write(GPIO_6, 0); //red LED off
break;
// case if mosquito species 'Aedes' 'Anopheles' 'Culex' detected
case (1 || 2 || 3 || 5):
HAL_GPIO_Write(GPIO_7, 1); // chemical atomizer GPIO 7 on
HAL_DelayUSec(15*1000*1000); // wait 15 seconds for atomizer
HAL_GPIO_Write(GPIO_7, 0); // chemical atomizer GPIO 7 off
HAL_DelayUSec(15*1000*1000); // wait 15 seconds before resuming
break;
case 4:
HAL_GPIO_Write(GPIO_5, 1); //green LED for background
HAL_GPIO_Write(GPIO_4, 0); //blue LED off
HAL_GPIO_Write(GPIO_6, 0); //red LED offbreak;
break;
}
After everything is in place, let’s compile and flash the firmware to the device using the below command in the project directory.
$ make clean $ make $ qfprog --port /dev/ttyACM0 --m4app ./qf_ssi_ai_clisensio_app.bin --reset --mode m4
Step 5: Device Firmware Setup: Part 2 – Pests Classification
Data Collection: We are going to collect audio datasets for different pests, currently, for this POC, I am more focused on locust sounds here (as India’s agriculture ministry is hoping to control the invasion before monsoon season hits north India at the end of June when locusts mature and breed. If the infestation is not controlled, it may threaten summer crops such as rice, maize and sorghum.) However, gathering locust swarm audio data was difficult so I tested with cricket sound files available for free here(Locusts belong to the same order of insects like grasshoppers, katydids and crickets – the Orthoptera). Here, are all the audio data for cricket sound https://mixkit.co/free-sound-effects/crickets/
Data Segmentation: We have segmented the audio data for two categories, viz. Cricket and Background.
Training and Saving model: In SensiML Analytics Studio, we are going to create the model and train the data for classification. Our query name is Q_PESTS_SPECIES. Training for Pipeline_1 is shown in one of the images.
Integrating Knowledge Pack with QORC-SDK and our firmware: Here’s our Pest detection and control JSON file generated:
{"NumModels": 1,"ModelIndexes": {"0": "Pipeline_1_rank_3"},"ModelDescriptions": [{"Name": "Pipeline_1_rank_3","ClassMaps": {"1": "background","2": "cricket","0": "Unknown"},"ModelType": "PME","FeatureNames": ["channel_025Percentile", "channel_0ZeroCrossingRate", "channel_0Sum", "channel_0100Percentile", "channel_0SigmaCrossingRate", "channel_0Skew", "channel_02ndSigmaCrossingRate", "channel_0MeanCrossingRate", "channel_0IQR", "channel_075Percentile", "channel_0Kurtosis", "channel_0Mean" ], "AIF": [ 150, 133, 150, 128, 109, 150, 133, 126, 105, 144, 150, 116, 104, 150, 121, 85, 87, 93, 84, 89, 86, 140, 140, 100, 90, 150, 89, 90, 150 ], "Category": [ 2, 2, 1, 2, 2, 1, 1, 2, 2, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 2, 2 ], "Vector": [ [ 231, 221, 138, 22, 209, 180, 107, 220, 25, 25, 22, 138 ], [ 224, 79, 131, 22, 95, 179, 97, 79, 31, 30, 18, 131 ], [ 202, 15, 227, 50, 20, 169, 31, 17, 59, 64, 28, 227 ], [ 250, 127, 138, 6, 146, 180, 168, 126, 5, 5, 27, 138 ], [ 253, 43, 137, 8, 66, 206, 56, 45, 2, 2, 91, 137 ], [ 153, 13, 115, 29, 41, 180, 0, 13, 102, 100, 1, 115 ], [ 212, 11, 130, 36, 24, 157, 10, 11, 45, 47, 23, 130 ], [ 251, 55, 133, 23, 59, 182, 178, 56, 4, 3, 59, 133 ], [ 233, 17, 129, 32, 40, 181, 31, 17, 21, 19, 69, 129 ], [ 248, 13, 107, 28, 6, 195, 15, 13, 6, 5, 98, 107 ], [ 239, 3, 168, 37, 7, 181, 5, 3, 20, 24, 84, 168 ], [ 236, 15, 118, 17, 22, 148, 36, 14, 19, 20, 40, 118 ], [ 253, 104, 137, 2, 148, 176, 110, 106, 2, 2, 21, 137 ], [ 219, 15, 136, 115, 7, 175, 25, 16, 36, 35, 127, 136 ], [ 238, 10, 124, 30, 9, 188, 20, 9, 18, 19, 48, 124 ], [ 217, 15, 143, 14, 37, 176, 10, 16, 37, 35, 4, 143 ], [ 233, 12, 151, 20, 19, 198, 46, 12, 22, 21, 21, 151 ], [ 254, 17, 146, 1, 35, 170, 25, 17, 1, 2, 13, 146 ], [ 238, 7, 162, 8, 17, 174, 10, 8, 17, 16, 10, 162 ], [ 232, 9, 147, 11, 24, 189, 13, 10, 23, 24, 8, 147 ], [ 241, 12, 135, 13, 27, 190, 36, 13, 14, 14, 20, 135 ], [ 232, 23, 136, 15, 39, 182, 31, 24, 23, 23, 12, 136 ], [ 251, 38, 134, 3, 63, 183, 36, 39, 3, 3, 17, 134 ], [ 249, 67, 138, 12, 88, 179, 158, 67, 6, 6, 36, 138 ], [ 251, 69, 136, 7, 63, 183, 107, 69, 4, 4, 35, 136 ], [ 253, 67, 140, 15, 45, 183, 138, 67, 2, 2, 178, 140 ], [ 247, 65, 143, 7, 88, 176, 64, 66, 9, 9, 19, 143 ], [ 252, 85, 137, 2, 111, 180, 92, 86, 3, 3, 18, 137 ], [ 253, 69, 138, 2, 99, 193, 117, 67, 2, 2, 18, 138 ] ], "Identifiers": [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28 ], "DistanceMode": 0 } ]}{ "NumModels": 1, "ModelIndexes": { "0": "Pipeline_1_rank_3" }, "ModelDescriptions": [ { "Name": "Pipeline_1_rank_3", "ClassMaps": { "1": "background", "2": "cricket", "0": "Unknown" }, "ModelType": "PME", "FeatureNames": [ "channel_025Percentile", "channel_0ZeroCrossingRate", "channel_0Sum", "channel_0100Percentile", "channel_0SigmaCrossingRate", "channel_0Skew", "channel_02ndSigmaCrossingRate", "channel_0MeanCrossingRate", "channel_0IQR", "channel_075Percentile", "channel_0Kurtosis", "channel_0Mean" ], "AIF": [ 150, 133, 150, 128, 109, 150, 133, 126, 105, 144, 150, 116, 104, 150, 121, 85, 87, 93, 84, 89, 86, 140, 140, 100, 90, 150, 89, 90, 150 ], "Category": [ 2, 2, 1, 2, 2, 1, 1, 2, 2, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 2, 2 ], "Vector": [ [ 231, 221, 138, 22, 209, 180, 107, 220, 25, 25, 22, 138 ], [ 224, 79, 131, 22, 95, 179, 97, 79, 31, 30, 18, 131 ], [ 202, 15, 227, 50, 20, 169, 31, 17, 59, 64, 28, 227 ], [ 250, 127, 138, 6, 146, 180, 168, 126, 5, 5, 27, 138 ], [ 253, 43, 137, 8, 66, 206, 56, 45, 2, 2, 91, 137 ], [ 153, 13, 115, 29, 41, 180, 0, 13, 102, 100, 1, 115 ], [ 212, 11, 130, 36, 24, 157, 10, 11, 45, 47, 23, 130 ], [ 251, 55, 133, 23, 59, 182, 178, 56, 4, 3, 59, 133 ], [ 233, 17, 129, 32, 40, 181, 31, 17, 21, 19, 69, 129 ], [ 248, 13, 107, 28, 6, 195, 15, 13, 6, 5, 98, 107 ], [ 239, 3, 168, 37, 7, 181, 5, 3, 20, 24, 84, 168 ], [ 236, 15, 118, 17, 22, 148, 36, 14, 19, 20, 40, 118 ], [ 253, 104, 137, 2, 148, 176, 110, 106, 2, 2, 21, 137 ], [ 219, 15, 136, 115, 7, 175, 25, 16, 36, 35, 127, 136 ], [ 238, 10, 124, 30, 9, 188, 20, 9, 18, 19, 48, 124 ], [ 217, 15, 143, 14, 37, 176, 10, 16, 37, 35, 4, 143 ], [ 233, 12, 151, 20, 19, 198, 46, 12, 22, 21, 21, 151 ], [ 254, 17, 146, 1, 35, 170, 25, 17, 1, 2, 13, 146 ], [ 238, 7, 162, 8, 17, 174, 10, 8, 17, 16, 10, 162 ], [ 232, 9, 147, 11, 24, 189, 13, 10, 23, 24, 8, 147 ], [ 241, 12, 135, 13, 27, 190, 36, 13, 14, 14, 20, 135 ], [ 232, 23, 136, 15, 39, 182, 31, 24, 23, 23, 12, 136 ], [ 251, 38, 134, 3, 63, 183, 36, 39, 3, 3, 17, 134 ], [ 249, 67, 138, 12, 88, 179, 158, 67, 6, 6, 36, 138 ], [ 251, 69, 136, 7, 63, 183, 107, 69, 4, 4, 35, 136 ], [ 253, 67, 140, 15, 45, 183, 138, 67, 2, 2, 178, 140 ], [ 247, 65, 143, 7, 88, 176, 64, 66, 9, 9, 19, 143 ], [ 252, 85, 137, 2, 111, 180, 92, 86, 3, 3, 18, 137 ], [ 253, 69, 138, 2, 99, 193, 117, 67, 2, 2, 18, 138 ] ], "Identifiers": [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28 ], "DistanceMode": 0 } ]}
Similar to the previous firmware, the first step is to set up our GPIO pin for the liquid-atomizer module in the src/pincfg_table.c file.
{ // setup Water Atomizer Digital Output
.ucPin = PAD_23,
.ucFunc = PAD23_FUNC_SEL_GPIO_7,
.ucCtrl = PAD_CTRL_SRC_A0,
.ucMode = PAD_MODE_OUTPUT_EN,
.ucPull = PAD_NOPULL,
.ucDrv = PAD_DRV_STRENGTH_4MA,
.ucSpeed = PAD_SLEW_RATE_SLOW,
.ucSmtTrg = PAD_SMT_TRIG_DIS,
},
After setting up GPIO let’s modify the src/sml_output.c function to direct the action response based upon audio classification or inference performed.
static void sml_output_serial(uint16_t model, uint16_t classification){int count;int wbytes = 0;int buflen = sizeof(sensor_ssss_ai_result_buf)-1;int ret;kb_get_feature_vector(model, sensor_ssss_ai_fv_arr, &sensor_ssss_ai_fv_len);// you can send other sensors or GPS data in below line for dashboardcount = snprintf(sensor_ssss_ai_result_buf, buflen,"{\"ModelNumber\":%d,\"Classification\":%d", (int)model, (int)classification);wbytes += count;buflen -= count;switch((int)classification){// case if unknown detectedcase 0:HAL_GPIO_Write(GPIO_4, 1); //blue LED for unknownHAL_GPIO_Write(GPIO_5, 0); //green LED offHAL_GPIO_Write(GPIO_6, 0); //red LED offbreak;// case if 'cricket' detectedcase 2:HAL_GPIO_Write(GPIO_7, 1); // chemical atomizer GPIO 7 onHAL_DelayUSec(15*1000*1000); // wait 15 seconds for atomizerHAL_GPIO_Write(GPIO_7, 0); // chemical atomizer GPIO 7 offHAL_DelayUSec(15*1000*1000); // wait 15 before resumingbreak;// case if 'background' detectedcase 1:HAL_GPIO_Write(GPIO_5, 1); //green LED for backgroundHAL_GPIO_Write(GPIO_4, 0); //blue LED offHAL_GPIO_Write(GPIO_6, 0); //red LED offbreak;}
After everything is in place, let’s compile and flash the firmware to the device using the below command in the project directory.
$ make clean $ make $ qfprog --port /dev/ttyACM0 --m4app ./qf_ssi_ai_clisensio_pest_app.bin --reset --mode m4
This pest detection and control use case can be improved for detecting killer Hornets and kill those in their breeding seasons before they reproduce in surplus and pose threat to our friend, bees. If pests are not coming near your device, you can put some luring compounds to attract the pests, the attraction will lead them to make wingbeat sounds near the device and the device, in turn, would activate the chemical-atomizer killing those mature pests before they make breeding grounds everywhere.
Step 6: Device Power Requirements:
The device runs on solar power and SensiML compute consumes very low power along with a microphone, so it can last several months provided, chemical repellent is changed frequently or better in the newer version of this Clisensio, I will be adding sensors to alert when to refill using LoRa or cellular.
Also, I am working to add LoRa and Cellular connectivity for the device to extend the notifications and demographic status features (Just the same IoT stuff, sending the data from RX/TX and visualizing on the dashboard) but for a POC this is well-understood……More works ahead 🙂
News Updates & Future Implementations:
We were already troubled and suffering from Covid now “Australia is overrun with its worst plague in decades: millions and millions of mice. The eastern region of New South Wales has been swarmed by an immeasurable number of mice, reports The Associated Press. The mice will cost over half a billion dollars in crop damage, according to the AP.” Our CliSensio device can prevent such future outbreaks by detecting mice squeaks and classify their sex, populations in all homes and grain storage houses and eliminate those before the area is swarmed by them using a liquid-chemical atomizer module.
The state government has ordered 5,000 litres (1,320 gallons) of the banned poison Bromadiolone from India. The federal government regulator has yet to approve emergency applications to use the poison on the perimeters of crops. Critics fear the poison will kill not only mice but also animals that feed on them. including wedge-tail eagles and family pets.
Latest Update: Added Arduino nano ble 33 sense to make image classification and identification for pest, vectors.
Step 7: References:
- https://www.theguardian.com/global-development/202…
- https://patents.google.com/patent/KR100511485B1/en
- https://news.mongabay.com/2021/03/as-climate-chang…
- https://www.straitstimes.com/singapore/why-detecti…
- https://e360.yale.edu/digest/climate-change-will-e…
- https://www.wionews.com/science/earth-is-losing-2-…
- https://www.wionews.com/india-news/watch-india-suf…
- https://apnews.com/article/plague-ravenous-destruc…
Source: CliSensio – Climate Sensing and Insect Infestation Control
