Arkitus has some gesture recognition sans neural net here.

You can also do colour detection on images by just scanning all of the pixels (skipping every other pixel will help with speed).

And there's also the computer vision libraries that could help you come up with some kind of compromise.


However - if you're avoiding neural nets simply because they're confusing or hard then I wouldn't hold out any hope for your final piece being worth looking at. No one likes a slacker (unless of course they are particularly cheeky).