top of page

PoseNet Generative Type

Updated: Feb 24




  • Explore ways of changing the form of typography with generative coding

  • Illustrate characters using basic shapes

  • Experiment with changing the parameters with organic inputs like PoseNet (hand/body/face)

  • Changing width/height/thickness/color/roundness/shape/etc of characters with PoseNet

  • Generate text & random phrases

  • possibly from a database (movie quotes/weather/fortune reading)

  • To create a fun and explorative way to interact with typography

  • The output should be visually intriguing and vary depending on the input



Experimenting with pixel manipulation with typography and text. Implementing from Processing-Tutorial: Kinetic Typography 1, using createCanvas();, but very heavy on computing power.

p5.js code



Using textToPoints(); to get the outline and points in text. Following Using mouse location to control the size of the ellipses that are mapped to the points retrieved from textToPoints();


Instead of using mouse location, the size of the ellipse has an initial animation mapped to the sine wave. The size is then mapped to the location of the tip of the index finger by using ml5.

Problem: People don’t usually realize there’s an element that requires the hand to be captured by the camera.

Instead of having a set text, I also added an input and pushed the newest input into an array, and updated it on the canvas.

Problem: The longer the input text is, the more computing power it requires. The animation and interaction slow down a lot when the input is long.


To fix the issues that came up in the previous version. I changed the interaction of using the hand to the tip of the nose so users get an initial response on the screen. To solve the other issue I mapped the “resolution” (number of points that are retrieved from the outline of text) of the points to the length of the input, so that the longer the input is the lower the resolution.

I also added two more forms of interaction so that when users turn their faces to the side, the font changes. When uses turn their heads up or down it toggles between two different modes.

p5.js code

bottom of page