Thumb Tracking (and Possibly Hand Turkeys)
After our class discussion, I was thinking a lot about the various ways that one could use the finger tracking code to distinguish between the thumb and other fingers. I tried a lot of different ways to do this, and I still think that the most accurate way would be to measure the angles between fingers and assume the largest angle to be the one between the thumb and forefinger. Unfortunately, that would probably mean making some changes to the underlying finger tracking library, since it joins the contours of all the fingers into one hand span. Oh well, that will be a project for another day.
I ultimately decided that the best way to do this was to assume that users would place hands in front of the Kinect oriented right-side up instead of upside down, and that each hand would likely be contained in either half of the screen. All of this hand outlining reminds me of doing drawings of hand turkeys for Thanksgiving in kindergarten, so my final tweak to this project will be adding left and right facing turkey heads to the appropriate thumbs. For the moment, they are identified with a blue circle and a red circle.
Here is the code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | // import the fingertracker library // and the SimpleOpenNI library for Kinect access import fingertracker.*; import SimpleOpenNI.*; // declare FignerTracker and SimpleOpenNI objects FingerTracker fingers; SimpleOpenNI kinect; // set a default threshold distance: // 625 corresponds to about 2-3 feet from the Kinect int threshold = 605; void setup() { size(640, 480); // allFingers = new ArrayList(); // initialize your SimpleOpenNI object // and set it up to access the depth image kinect = new SimpleOpenNI(this); kinect.enableDepth(); // mirror the depth image so that it is more natural kinect.setMirror(true); // initialize the FingerTracker object // with the height and width of the Kinect // depth image fingers = new FingerTracker(this, 640, 480); // the "melt factor" smooths out the contour // making the finger tracking more robust // especially at short distances // farther away you may want a lower number fingers.setMeltFactor(100); } void draw() { // get new depth data from the kinect kinect.update(); // get a depth image and display it PImage depthImage = kinect.depthImage(); image(depthImage, 0, 0); // update the depth threshold beyond which // we'll look for fingers fingers.setThreshold(threshold); // access the "depth map" from the Kinect // this is an array of ints with the full-resolution // depth data (i.e. 500-2047 instead of 0-255) // pass that data to our FingerTracker int[] depthMap = kinect.depthMap(); fingers.update(depthMap); //PVectors for storing thumb positions PVector leftThumb = new PVector(); PVector rightThumb = new PVector(); // iterate over all the contours found // and display each of them with a green line stroke(0,255,0); for (int k = 0; k < fingers.getNumContours(); k++) { fingers.drawContour(k); } // iterate over all the fingers found // and draw them as a red circle noStroke(); fill(255,0,0); for (int i = 0; i < fingers.getNumFingers(); i++) { PVector position = fingers.getFinger(i); if(position.x > width/2) { if(position.y > rightThumb.y) { rightThumb.x = position.x; rightThumb.y = position.y; } } else { if(position.y > leftThumb.y) { leftThumb.x = position.x; leftThumb.y = position.y; } } } noStroke(); fill(255,0,0); ellipse(rightThumb.x - 5, rightThumb.y - 5, 10, 10); fill(0, 0, 255); ellipse(leftThumb.x - 5, leftThumb.y - 5, 10, 10); // show the threshold on the screen fill(255,0,0); text(threshold, 10, 20); } // keyPressed event: // pressing the '-' key lowers the threshold by 10 // pressing the '+/=' key increases it by 10 void keyPressed(){ if(key == '-'){ threshold -= 10; } if(key == '='){ threshold += 10; } } |