Wes Bos

Designer, Developer & Entrepreneur making the web an awesome place.

facedetection

This morning I saw this link on youtube which was a little mashup of some HTML5 technologies. I thought it would be funny if I could do the same, but with goofy pair of glasses. I’ve also been itching to put the CCV JavaScript Face Detection library to use. This library shows a few examples on static images, but after a quick look at the code, it shows that the underlying element to the script is a canvas element. So instead of running it on a single image, I am running it on a feed of frames coming from an HTML5 video element.

I’ll go into the technical details further on in the post, but here is a demo as well as a youtube video showing the effect as it can be a little sluggish on older machines. Currently tested and working in Google Chrome 14 and Firefox 6.0. 

Try the Demo | Download the Source

Setting up our document

To get started, we dont really need that much. Only two of the files from the CCV Library are required are CCV.js which does the acutal detection of the the face, and face.js which holds the data for what faces look like. We will also need an empty canvas element, a HTML5 video element with .MP4 and .OGG encoded files  (I used Miro to convert mine), and a blank scripts.js file. Things will look like this when we are setup:

<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>HTML5 Face Detection</title>
<link rel="stylesheet" href="style.css"/>
</head>
<body>
<div class="wrapper">
	<h1>HTML5 GLASSES</h1>
	<p>Created by <a href="http://twitter.com/wesbos" target="_blank">Wes Bos</a>. See full details <a href="">here.</a></p>
	<!-- Our Main Video Element -->
	<video height="426" width="640" controls="false">
		<source src="videos/wes4.ogg" />
		<source src="videos/wes4.mp4" />
	</video>

	<!-- Out Canvas Element for output -->
	<canvas id="output"  height="426" width="640" ></canvas>

	<!-- div to track progress -->
	<div id="elapsed_time">Press play for HTML5 Glasses!</div>
</div>
<script type="text/javascript" src="scripts/ccv.js"></script>
<script type="text/javascript" src="scripts/face.js"></script>
<script type="text/javascript" src="scripts/scripts.js"></script>
</body>
</html>

 

Let’s write some JavaScript!

The core of this application is just a single function called html5glasses() that runs every 200 miliseconds. The function grabs the current frame from the window, spits it onto the canvas and then lets the CCV JS library detect the face. When it returns the data we loop through each of the found faces and and apply the silly glasses.

I should note that this isn’t the fastest thing in the world, and it is blocking. In the CCV examples, they provide a web worker example so we could do this asynchronously, but in my tests it was significantly slower. This is new technology and will only get better. On my computer I see a new frame about every 300 miliseconds, or 3 times a second.

Setup the JavaScript Variables

I’ve commented these inline.

var		
		// Store the first HTML5 video element in the document
		video = document.querySelector('video'),
		// We use this to time how long things are taking. Not that important..
		time_dump = document.getElementById("elapsed_time"),
		// Create a new image that will be our goofy glasses
		glasses = new Image(),
		// Store the canvas so we can write to it
		canvas = document.getElementById("output"),
		// Get the canvas 2d Context
		ctx = canvas.getContext("2d");
		// Finally set the source of our new glasses img element
		glasses.src = "i/glasses.png";

Create our main html5glasses() function

This is where all the magic happens. Read through each line and realize what is happening. Pay specific attenton to the ccv.detect_objects() function.

function html5glasses() {
	// Start the clock 
	var elapsed_time = (new Date()).getTime();

	// Draw the video to canvas
	ctx.drawImage(video, 0, 0, video.width, video.height, 0, 0, canvas.width, canvas.height);

	// use the face detection library to find the face
	var comp = ccv.detect_objects({ "canvas" : (ccv.pre(canvas)),
									"cascade" : cascade,
									"interval" : 5,
									"min_neighbors" : 1 });

	// Stop the clock
	time_dump.innerHTML = "Process time : " + ((new Date()).getTime() - elapsed_time).toString() + "ms";

	// Draw glasses on everyone!
	for (var i = 0; i < comp.length; i++) {
		ctx.drawImage(glasses, comp[i].x, comp[i].y,comp[i].width, comp[i].height);
	}
}

Finally, trigger it!

We trigger the detection when the video element is played and stop it when it reaches the end.

/* Events */ 

video.addEventListener('play', function() {
	vidInterval = setInterval(html5glasses,200); 
});

video.addEventListener('ended', function() {
	clearInterval(vidInterval);
	time_dump.innerHTML = "finished";
});

Pretty cool, eh?

I'm just getting into CCV, but the library can be extended to recognize much more than just faces. If you take at look at the github repo, they have examples for detecting all kinds of things.

What else?

Next time I'm in the mood for a little hack, I'll try and hook this up to a flash webcam app or Mozilla's Rainbow to stream the data right from the device itself, giving us realtime funny glasses!

Let me know if you have any ideas or questions. I'm @wesbos on twitter and have hosted the source on Git Hub

This entry was posted in browsers, Code Snippets, HTML5, JavaScript. Bookmark the permalink.

43 Responses to JavaScript Face Detection + Canvas + Video = HTML5 Glasses!

  1. Darcy Clarke says:

    Great post and a very neat example. Can’t wait to see what’s next in your bag of tricks ;)

  2. Mike Taylor says:

    AMAZING. It would be trivial to get this working with the getUserMedia API, which is available in an experimental build of Opera Mobile for Android (and one day on the Desktop): http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview. Nice work!

  3. Pingback: HTML5 Video Face Detection

  4. Nick Lombardi says:

    Wow the possibilities with this script are endless can do a lot with this! Thanks for the tutorial!

  5. Leonard says:

    Really cool trick, very neatly done!

  6. spidermanit says:

    Great post!!! Thanks you.

  7. baz kika says:

    Really good explaination and idea. I was going to use flash for a similar effect for my 3rd year uni project. Really confused which way to go now!!! Thanks for sharing. Baz

  8. Pingback: Ultimate Collection and a Must-Read HTML5 Tutorials and Resources | Web Design Habits

  9. Pingback: HTML5 Video Face Tracking with Canvas and JavaScript - Web Design Blog – DesignM.ag

  10. Pingback: Weekly Design News – Resources, Tutorials and Freebies (N.106)

  11. Pingback: Weekly Design News – Resources, Tutorials and Freebies | VapvaruN | Wp Experts

  12. Pingback: Web Development articles, tutorials, help » Blog Archive » Weekly Design News – Resources, Tutorials and Freebies (N.106)

  13. Paddy says:

    Finally Opera have released a desktop build that allows the browser access the webcam.
    http://labs.opera.com/news/2011/10/19/

    Rich Tibbett’s original face recognition demo works perfectly http://people.opera.com/richt/release/demos/device/facerecognition/

  14. Vladimir says:

    hey man you are crazy) this perfect, I am shocked by such an effect

    bigUp

  15. Pingback: SiteDesignHQ Blog

  16. Pingback: Back to my desk (down under)

  17. Derek says:

    It is amazing that HTML5 can do all this stuff! :D
    However, it is quite slow, and you may want to use some Workers.

  18. seutje says:

    where’s my .getUserMedia?

    slacker! :P

    I kiiid <3

  19. Pingback: Working with HTML5 canvas: JavaScript library | Craft Idea

  20. Kevin G says:

    Where can I find some documentation for ccv?

  21. Pingback: Face Detection with HTML5 Video/Canvas and JavaScript | Michael Deol ⚓

  22. Paul Neave says:

    Thanks loads for this! I took your example and optimised the code a bit, and hooked it up to the webcam using getUserMedia. Have a look at my face detection demo here: http://neave.com/webcam/html5/face/

    All the source code is available on that page :)

  23. Gad says:

    Hi … wesbos .. Really it’s a great application ..
    i have a question .. on local host i can make it
    works ..
    Because i downloaded it to my pc .. and i see just
    an image of the See …

    Thank you my email is gberenstein@gmail.com

  24. Pingback: O’REILLY Fluentconf 2012: Dag 3 | VERTICA BLOG

  25. xuanchien247 says:

    Hi, this tutorial experiment.
    I want to improve performance this application with webworker on html5, can you help me this problem.
    thanks you very much.

  26. Daniël Voogsgerd says:

    You could speed up and increase performance by using rAF (requestAnimationFrame) instead of the interval.

  27. Pingback: JavaScript Face Detection |

  28. Pingback: Webcam and Annotation With HTML5 | Document Imaging

  29. Pingback: 4UEyes: Face Recognition Experience – Globant labs blog Blog

  30. Ramsundar Kandasamy says:

    Does anyone know how to enable parallelism inside ccv?

  31. Pingback: Augmented Reality demos and projects « IDentifEYE

  32. Pingback: Augmented Reality demos and projects – an overview « IDentifEYE

  33. Pingback: Object Detection with HTML5 getUserMedia | TechSlides

  34. Joshua says:

    hey Im not sure what to do. I have the .js script saved in 3 files: face.js, ccv.js, and an empty file scripts.js. i then have my .mp4 file ready to go as well. the rest of the codes, i dont know how to start putting it all together to make it work. Im trying to make a virtual try-on for an eyeglass company for the iphone, using css and javascript. I was thinking about a video one like this too but im new to this stuff as i am an seo guy. any help would be greatly appreciated! Thanks and great post btw :)

  35. Eranga says:

    sir what is the name of the algorithm behind the ccv library???

  36. Aadil says:

    Nice work buddy!!

    I am going to use for my web app.

    Thanks a lot!!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>