In this workshop we use ML5 as an alternative input method for p5.js, turning pose detection (hands, body, or face) into real-time controls for our sketches. We also train Teachable Machine models and run them in the browser so our own gestures and examples can reliably trigger actions, animations, and interactive responses.

David Stein

[email protected]

Instagram: @colonelPanix

ML5 Library

https://ml5js.org/

ml5.js is a friendly, beginner-oriented machine learning library built on top of TensorFlow.js, designed to work seamlessly alongside p5.js.

The core idea is to make ML accessible to artists, designers, and creative coders — without needing a deep background in data science or Python. It wraps complex TensorFlow.js APIs into simple, high-level calls that feel at home in a p5.js sketch.

What it can do:

• Pose estimation — detect body keypoints in real time (via MoveNet/PoseNet)
• Hand pose — track hand landmarks from webcam
• Facemesh — 468-point 3D face landmark detection
• Image classification — classify what's in a photo (MobileNet)
• Object detection — identify and locate multiple objects (COCO-SSD)
• Sentiment analysis — judge the tone of text
• Sound classification — recognize spoken words or environmental sounds
• Neural network — train your own custom classifier or regressor right in the browser

Typical ML5 Sketch:

Most of the ML5 sketches we will work on will have the following format. (Note that this will only work if you add the ml5 library to the index.html (See getting started in ml5)