Try my AI Rotifer Classifier

Do you have any microscopy questions, which you are afraid to ask? This is your place.
Post Reply
Message
Author
User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Try my AI Rotifer Classifier

#1 Post by josmann » Tue May 30, 2023 12:05 am

Hey folks - wanted to share a little project I've been working on. I've been learning deep learning techniques and trained a model which is supposed to differentiate between rotifers belonging to bdelloidea or monogononta. All you have to do is upload an image and it will make its best guess!

https://huggingface.co/spaces/DietToms/ ... lassifiers

Please try it out with some images and, if you find any misclassifications that you think should have been easy, post them here or on my discord in #microbe-ai (https://discord.gg/H4wKRkkYJj). One genus I know confuses it is Asplanchna - probably because I only had two images of those in my training set and they're so unusual compared to other monogononts. I don't expect it to be 100% accurate, but I want to understand why it fails when it fails. Hopefully this leads to bigger and more general classification tools in the future :)
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

User avatar
actinophrys
Posts: 194
Joined: Tue Oct 21, 2014 6:45 am
Contact:

Re: Try my AI Rotifer Classifier

#2 Post by actinophrys » Tue May 30, 2023 2:12 am

Neat experiment. I tried it out with some from my page. It did a good job recognizing the wider and loricate ploimids as Mongononta, but missed more worm-like types like Taphrocampa, Notommata, and Cephalodella. It also mislabeled all the gnesiotroch rotifers as Bdelloidea, even Testudinella, I would guess being misled by the lobed corona. So I imagine you need more of those more bdelloid-like types in the training set for it to sort them out.

User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Re: Try my AI Rotifer Classifier

#3 Post by josmann » Tue May 30, 2023 3:04 am

actinophrys wrote:
Tue May 30, 2023 2:12 am
Neat experiment. I tried it out with some from my page. It did a good job recognizing the wider and loricate ploimids as Mongononta, but missed more worm-like types like Taphrocampa, Notommata, and Cephalodella. It also mislabeled all the gnesiotroch rotifers as Bdelloidea, even Testudinella, I would guess being misled by the lobed corona. So I imagine you need more of those more bdelloid-like types in the training set for it to sort them out.
Excellent feedback - thank you! It usually does get testudinella, so it will be interesting to see why that particular image is problematic. Would you mind if I use your images in my validation set for future models?

There's definitely going to be a major challenge in getting this thing to differentiate between the wormier looking monogononts and bdelloids. And then there's just the general challenge of how different specimens can look depending on illumination technique and focus depth. I don't think it's insurmountable, though - just need more data and better fine tuning of the model.
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

Sure Squintsalot
Posts: 393
Joined: Mon May 16, 2022 3:44 pm

Re: Try my AI Rotifer Classifier

#4 Post by Sure Squintsalot » Tue May 30, 2023 3:57 pm

That's pretty damn slick!

Unfortunately, I don't encounter enough rotifers to help your project.

Maybe on your day off you could do one for plankton?

LOL.

....sorta.

No really, could you?

User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Re: Try my AI Rotifer Classifier

#5 Post by josmann » Tue May 30, 2023 9:40 pm

Sure Squintsalot wrote:
Tue May 30, 2023 3:57 pm
Maybe on your day off you could do one for plankton?
Well I'm definitely planning on building more useful classifiers in the future - this is my first go. I chose monogononts and bdelloids because it seemed like a good balance of challenging and doable. Putting this tool out here isn't so much about trying to make a perfect rotifer classifier as it is about trying to understand where the pitfalls are for a microscopy deep learning model when it's given to the public. The model may be 90% accurate according to internal testing, but if one species causes it to fail consistently, or all photos from a certain user fail due to their particular lighting or formatting, that's not so good.

I'm assuming you mean marine plankton - one issue there is that there's a pretty steep drop-off from freshwater images to saltwater images on the internet. For a deep learning tool to work well in general, you need a lot of photos of every group in your classifier under a lot of different conditions (or, you make a tool that only works under very specific conditions). I don't know of anywhere to get a ton of marine plankton images, but please let me know if you do! My current image set is all images under the "microscopy" project on iNaturalist - about 32000 of them.
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

Javier
Posts: 806
Joined: Tue May 09, 2017 11:19 am
Location: Buenos Aires, Argentina.

Re: Try my AI Rotifer Classifier

#6 Post by Javier » Wed May 31, 2023 2:10 pm

What a fun project, I will give it a go for sure.

Sure Squintsalot
Posts: 393
Joined: Mon May 16, 2022 3:44 pm

Re: Try my AI Rotifer Classifier

#7 Post by Sure Squintsalot » Wed May 31, 2023 6:29 pm

josmann wrote:
Tue May 30, 2023 9:40 pm
Sure Squintsalot wrote:
Tue May 30, 2023 3:57 pm
Maybe on your day off you could do one for plankton?
Well I'm definitely planning on building more useful classifiers in the future - this is my first go. I chose monogononts and bdelloids because it seemed like a good balance of challenging and doable. Putting this tool out here isn't so much about trying to make a perfect rotifer classifier as it is about trying to understand where the pitfalls are for a microscopy deep learning model when it's given to the public. The model may be 90% accurate according to internal testing, but if one species causes it to fail consistently, or all photos from a certain user fail due to their particular lighting or formatting, that's not so good.

I'm assuming you mean marine plankton - one issue there is that there's a pretty steep drop-off from freshwater images to saltwater images on the internet. For a deep learning tool to work well in general, you need a lot of photos of every group in your classifier under a lot of different conditions (or, you make a tool that only works under very specific conditions). I don't know of anywhere to get a ton of marine plankton images, but please let me know if you do! My current image set is all images under the "microscopy" project on iNaturalist - about 32000 of them.
As far as I know, marine biologists are desperate for a fast, real-time plankton ID system that they can marry to an underwater sampler for doing population census. Keeping optics clean and pumps and samplers crud-free is a real challenge for the hardware guys. Then there's the next best thing: doing a ship-board census whereby samples are drawn from a Nansen Bottle and analyzed belowdecks by a tech. This is all extremely time and labor intensive.

Imagine a system in which your water sample is poured into an AI-driven plankton counter, you flick a switch, and come back 20 minutes later to a complete spreadsheet of all plankton species present (with 98% accuracy!).

User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Re: Try my AI Rotifer Classifier

#8 Post by josmann » Thu Jun 01, 2023 12:40 am

Sure Squintsalot wrote:
Wed May 31, 2023 6:29 pm
As far as I know, marine biologists are desperate for a fast, real-time plankton ID system that they can marry to an underwater sampler for doing population census. Keeping optics clean and pumps and samplers crud-free is a real challenge for the hardware guys. Then there's the next best thing: doing a ship-board census whereby samples are drawn from a Nansen Bottle and analyzed belowdecks by a tech. This is all extremely time and labor intensive.

Imagine a system in which your water sample is poured into an AI-driven plankton counter, you flick a switch, and come back 20 minutes later to a complete spreadsheet of all plankton species present (with 98% accuracy!).
I did an internship for this company a while ago - they have a submersible holographic camera that can get great images of plankton in the optical path: https://www.sequoiasci.com/product/lisst-holo/

If you go to the gallery, the image with the copepod and trichodesmium were captured by 19 year old me :) Would be very cool to get that paired with a deep learning classifier. Again, though, the issue is you must have lots of preclassified images to train a model. And then you always at least need to spot-check the results for a while to make sure nothing weird is happening.
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

Sure Squintsalot
Posts: 393
Joined: Mon May 16, 2022 3:44 pm

Re: Try my AI Rotifer Classifier

#9 Post by Sure Squintsalot » Thu Jun 01, 2023 4:46 am

I tried your rotifer identifier with a dozen stills taken from my rock pools video here: https://www.microbehunter.com/microscop ... 30&t=18416

The identifier clearly knew all of the images to be Bdelloidea save for one; a rotifer squashed between cover glass and slide that was definitely fatter looking than the others:
Incorrectly identified as Mongononta
Incorrectly identified as Mongononta
Screenshot 2023-05-31 223643.jpg (65.54 KiB) Viewed 1815 times
100% Success
100% Success
Screenshot 2023-05-31 223232.jpg (35.61 KiB) Viewed 1815 times
When you used iNaturalist's image set, did you have to ask their permission or do they provide a creative commons license for that sort of thing?

User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Re: Try my AI Rotifer Classifier

#10 Post by josmann » Thu Jun 01, 2023 5:23 am

Sure Squintsalot wrote:
Thu Jun 01, 2023 4:46 am
I tried your rotifer identifier with a dozen stills taken from my rock pools video here: https://www.microbehunter.com/microscop ... 30&t=18416

The identifier clearly knew all of the images to be Bdelloidea save for one; a rotifer squashed between cover glass and slide that was definitely fatter looking than the others:

When you used iNaturalist's image set, did you have to ask their permission or do they provide a creative commons license for that sort of thing?
Cool, thanks for trying it out! I'm going to take another swing at the model training soon - I've been wrapped up in other stuff.

Regarding licensing - everything in the iNaturalist dataset is either marked as some sort of CC license or the license field is blank. There's a lot of debate right now about what constitutes copyright infringement in the field of AI. That said, I think you'd be hard-pressed to argue an image classifier infringes on copyright even if it were trained solely on copyrighted material. Certainly nobody could find out if a specific image was used to train it unless you told them.
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

Chas
Posts: 421
Joined: Tue Jan 19, 2021 3:11 pm

Re: Try my AI Rotifer Classifier

#11 Post by Chas » Sat Jun 03, 2023 11:50 am

It is interesting that such things are getting possible :-)
How many examples of each does the AI seem to need, do you feel?
MichelG, I think, posted a link to this site a while ago (but diatoms) and it made we wonder if it might be AI fodder:
https://naturalhistory.museumwales.ac.u ... kip=0&#top

User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Re: Try my AI Rotifer Classifier

#12 Post by josmann » Sat Jun 03, 2023 10:37 pm

Chas wrote:
Sat Jun 03, 2023 11:50 am
It is interesting that such things are getting possible :-)
How many examples of each does the AI seem to need, do you feel?
MichelG, I think, posted a link to this site a while ago (but diatoms) and it made we wonder if it might be AI fodder:
https://naturalhistory.museumwales.ac.u ... kip=0&#top
Hey Chas - so there's no exact answer to "how many images" it depends a lot on the level of classification that you're trying to drill down to, how similar the images are, and what the goal of the tool is.

A classifier that sorts images of cats vs dogs might be very accurate with just 100 photos. A classifier that identifies the breeds of cats and dogs may require substantially more.

I think about diatoms a lot - these sites might be helpful or they might not be. I know a diatom researcher who live streams SEM imagery of diatom strews. Since strews show diatoms in all sorts of orientations and SEM imagery is very regular, I think it's a great candidate for an image classifier if I can get enough images. I imagine it would be possible to write an automated program that identifies individual diatoms at low mag and zooms in via SEM on each one to make an ID. So the user preps the strew, sputters, sticks it in the SEM, pushes a button, and goes for a coffee. When they come back, they get images of every diatom sorted taxonomically. So you could do some very effective diatom censuses and probably accelerate research by a lot. Big project, but I don't see any reason why it can't be done.
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

sreynolds
Posts: 113
Joined: Sat Oct 10, 2020 1:17 am
Location: Vermont, USA

Re: Try my AI Rotifer Classifier

#13 Post by sreynolds » Mon Jun 12, 2023 2:15 am

The devil made me do it - I gave it a caddis fly larva Hydropsychidae just to see if it could tell a non-rotifer - nope. Sorry.
common netspinner caddis fly.jpg
common netspinner caddis fly.jpg (84.47 KiB) Viewed 1573 times
Steve

User avatar
josmann
Posts: 340
Joined: Fri Sep 17, 2021 1:23 am
Location: San Jose, CA

Re: Try my AI Rotifer Classifier

#14 Post by josmann » Mon Jun 12, 2023 3:41 am

sreynolds wrote:
Mon Jun 12, 2023 2:15 am
The devil made me do it - I gave it a caddis fly larva Hydropsychidae just to see if it could tell a non-rotifer - nope. Sorry.
Haha, this is what's referred to as "out of domain". The whole universe from the perspective of this AI model is just a bunch of number sequences which all can be exhaustively categorized as "Bdelloid" or "Monogonont". It doesn't know what rotifers are, even!
The highest quality live-streamed microscopy in the world.
Sundays around 8PST: https://www.youtube.com/@diettoms/streams
Occasionally (for now): https://www.twitch.tv/diettoms

Join the Discord: https://discord.gg/FgpUUnJaSE

Post Reply