I just launched a new course on ES6! Use the code WESBOS for $10 off.

Strengthen your core JavaScript skills and master all that ES6 has to offer. Start Now →

JavaScript Face Detection + Canvas + Video = HTML5 Glasses!

This morning I saw this link on youtube which was a little mashup of some HTML5 technologies. I thought it would be funny if I could do the same, but with goofy pair of glasses. I’ve also been itching to put the CCV JavaScript Face Detection library to use. This library shows a few examples on static images, but after a quick look at the code, it shows that the underlying element to the script is a canvas element. So instead of running it on a single image, I am running it on a feed of frames coming from an HTML5 video element.

I’ll go into the technical details further on in the post, but here is a demo as well as a youtube video showing the effect as it can be a little sluggish on older machines. Currently tested and working in Google Chrome 14 and Firefox 6.0. 

Try the Demo | Download the Source

Setting up our document

To get started, we dont really need that much. Only two of the files from the CCV Library are required are CCV.js which does the acutal detection of the the face, and face.js which holds the data for what faces look like. We will also need an empty canvas element, a HTML5 video element with .MP4 and .OGG encoded files  (I used Miro to convert mine), and a blank scripts.js file. Things will look like this when we are setup:

<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>HTML5 Face Detection</title>
<link rel="stylesheet" href="style.css"/>
<div class="wrapper">
	<h1>HTML5 GLASSES</h1>
	<p>Created by <a href="http://twitter.com/wesbos" target="_blank">Wes Bos</a>. See full details <a href="">here.</a></p>
	<!-- Our Main Video Element -->
	<video height="426" width="640" controls="false">
		<source src="videos/wes4.ogg" />
		<source src="videos/wes4.mp4" />

	<!-- Out Canvas Element for output -->
	<canvas id="output"  height="426" width="640" ></canvas>

	<!-- div to track progress -->
	<div id="elapsed_time">Press play for HTML5 Glasses!</div>
<script type="text/javascript" src="scripts/ccv.js"></script>
<script type="text/javascript" src="scripts/face.js"></script>
<script type="text/javascript" src="scripts/scripts.js"></script>


Let’s write some JavaScript!

The core of this application is just a single function called html5glasses() that runs every 200 miliseconds. The function grabs the current frame from the window, spits it onto the canvas and then lets the CCV JS library detect the face. When it returns the data we loop through each of the found faces and and apply the silly glasses.

I should note that this isn’t the fastest thing in the world, and it is blocking. In the CCV examples, they provide a web worker example so we could do this asynchronously, but in my tests it was significantly slower. This is new technology and will only get better. On my computer I see a new frame about every 300 miliseconds, or 3 times a second.

Setup the JavaScript Variables

I’ve commented these inline.

		// Store the first HTML5 video element in the document
		video = document.querySelector('video'),
		// We use this to time how long things are taking. Not that important..
		time_dump = document.getElementById("elapsed_time"),
		// Create a new image that will be our goofy glasses
		glasses = new Image(),
		// Store the canvas so we can write to it
		canvas = document.getElementById("output"),
		// Get the canvas 2d Context
		ctx = canvas.getContext("2d");
		// Finally set the source of our new glasses img element
		glasses.src = "i/glasses.png";

Create our main html5glasses() function

This is where all the magic happens. Read through each line and realize what is happening. Pay specific attenton to the ccv.detect_objects() function.

function html5glasses() {
	// Start the clock 
	var elapsed_time = (new Date()).getTime();

	// Draw the video to canvas
	ctx.drawImage(video, 0, 0, video.width, video.height, 0, 0, canvas.width, canvas.height);

	// use the face detection library to find the face
	var comp = ccv.detect_objects({ "canvas" : (ccv.pre(canvas)),
									"cascade" : cascade,
									"interval" : 5,
									"min_neighbors" : 1 });

	// Stop the clock
	time_dump.innerHTML = "Process time : " + ((new Date()).getTime() - elapsed_time).toString() + "ms";

	// Draw glasses on everyone!
	for (var i = 0; i < comp.length; i++) {
		ctx.drawImage(glasses, comp[i].x, comp[i].y,comp[i].width, comp[i].height);

Finally, trigger it!

We trigger the detection when the video element is played and stop it when it reaches the end.

/* Events */ 

video.addEventListener('play', function() {
	vidInterval = setInterval(html5glasses,200); 

video.addEventListener('ended', function() {
	time_dump.innerHTML = "finished";

Pretty cool, eh?

I'm just getting into CCV, but the library can be extended to recognize much more than just faces. If you take at look at the github repo, they have examples for detecting all kinds of things.

What else?

Next time I'm in the mood for a little hack, I'll try and hook this up to a flash webcam app or Mozilla's Rainbow to stream the data right from the device itself, giving us realtime funny glasses!

Let me know if you have any ideas or questions. I'm @wesbos on twitter and have hosted the source on Git Hub

This entry was posted in HTML5, JavaScript. Bookmark the permalink.

56 Responses to JavaScript Face Detection + Canvas + Video = HTML5 Glasses!

  1. Darcy Clarke says:

    Great post and a very neat example. Can’t wait to see what’s next in your bag of tricks 😉

  2. Mike Taylor says:

    AMAZING. It would be trivial to get this working with the getUserMedia API, which is available in an experimental build of Opera Mobile for Android (and one day on the Desktop): http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview. Nice work!

  3. Addy Osmani says:

    Very slick! Nice work 🙂

  4. Pingback: HTML5 Video Face Detection

  5. Nick Lombardi says:

    Wow the possibilities with this script are endless can do a lot with this! Thanks for the tutorial!

  6. Leonard says:

    Really cool trick, very neatly done!

  7. spidermanit says:

    Great post!!! Thanks you.

  8. baz kika says:

    Really good explaination and idea. I was going to use flash for a similar effect for my 3rd year uni project. Really confused which way to go now!!! Thanks for sharing. Baz

  9. Pingback: Ultimate Collection and a Must-Read HTML5 Tutorials and Resources | Web Design Habits

  10. Pingback: HTML5 Video Face Tracking with Canvas and JavaScript - Web Design Blog – DesignM.ag

  11. Pingback: Weekly Design News – Resources, Tutorials and Freebies (N.106)

  12. Pingback: Weekly Design News – Resources, Tutorials and Freebies | VapvaruN | Wp Experts

  13. Pingback: Web Development articles, tutorials, help » Blog Archive » Weekly Design News – Resources, Tutorials and Freebies (N.106)

  14. Paddy says:

    Finally Opera have released a desktop build that allows the browser access the webcam.

    Rich Tibbett’s original face recognition demo works perfectly http://people.opera.com/richt/release/demos/device/facerecognition/

  15. Vladimir says:

    hey man you are crazy) this perfect, I am shocked by such an effect


  16. Pingback: SiteDesignHQ Blog

  17. Pingback: Back to my desk (down under)

  18. Derek says:

    It is amazing that HTML5 can do all this stuff! 😀
    However, it is quite slow, and you may want to use some Workers.

  19. seutje says:

    where’s my .getUserMedia?

    slacker! 😛

    I kiiid <3

  20. Pingback: Working with HTML5 canvas: JavaScript library | Craft Idea

  21. Kevin G says:

    Where can I find some documentation for ccv?

  22. Pingback: Face Detection with HTML5 Video/Canvas and JavaScript | Michael Deol ⚓

  23. Paul Neave says:

    Thanks loads for this! I took your example and optimised the code a bit, and hooked it up to the webcam using getUserMedia. Have a look at my face detection demo here: http://neave.com/webcam/html5/face/

    All the source code is available on that page 🙂

  24. Gad says:

    Hi … wesbos .. Really it’s a great application ..
    i have a question .. on local host i can make it
    works ..
    Because i downloaded it to my pc .. and i see just
    an image of the See …

    Thank you my email is [email protected]

  25. Pingback: O’REILLY Fluentconf 2012: Dag 3 | VERTICA BLOG

  26. xuanchien247 says:

    Hi, this tutorial experiment.
    I want to improve performance this application with webworker on html5, can you help me this problem.
    thanks you very much.

  27. Daniël Voogsgerd says:

    You could speed up and increase performance by using rAF (requestAnimationFrame) instead of the interval.

  28. Pingback: JavaScript Face Detection |

  29. Pingback: Webcam and Annotation With HTML5 | Document Imaging

  30. Pingback: 4UEyes: Face Recognition Experience – Globant labs blog Blog

  31. Ramsundar Kandasamy says:

    Does anyone know how to enable parallelism inside ccv?

  32. Pingback: Augmented Reality demos and projects « IDentifEYE

  33. Pingback: Augmented Reality demos and projects – an overview « IDentifEYE

  34. Pingback: Object Detection with HTML5 getUserMedia | TechSlides

  35. Joshua says:

    hey Im not sure what to do. I have the .js script saved in 3 files: face.js, ccv.js, and an empty file scripts.js. i then have my .mp4 file ready to go as well. the rest of the codes, i dont know how to start putting it all together to make it work. Im trying to make a virtual try-on for an eyeglass company for the iphone, using css and javascript. I was thinking about a video one like this too but im new to this stuff as i am an seo guy. any help would be greatly appreciated! Thanks and great post btw 🙂

  36. Eranga says:

    sir what is the name of the algorithm behind the ccv library???

  37. Ravi Teja says:

    Hi wesbos,

    Nice work on face detection using javascript. I have an issue ,ccv.detect_objects() is not gettting called when i was tracing using alert();
    control is not going uptil that method.
    can u suggest me what is the issue ..i m facing
    thank You.

  38. s1wand says:

    Is there any simple way to put 3d textures on webcam image?
    something like on this site:

  39. kartheeswaran says:

    I downloaded Facedetection from Github. Then what can i do? Is i run the index file only? help me

  40. kartheeswaran says:

    I created new file called detect.html and used HTML code for start the app. When i press play the video is playing. But the face is not detecting and glass is not rendering.

    I created new script called script.js then post the html5glasses() code. Can you help me?

  41. Steve says:

    nice demo..Is it possible to fetch the detected face and replace it with the mannequin’s face stored as an image in a canvas element. Is it possible to get the ctx.arc() parameters and copy that face and replace with the mannequin’s face. Also does luilui provides algorithm for shoulder or body detection also?

  42. Aadil says:

    Nice work buddy!!

    I am going to use for my web app.

    Thanks a lot!!

  43. attorney says:

    Hello there, just became alert to your blog through Google, and found that it’s really informative.
    I am going to watch out for brussels. I’ll appreciate if you continue this in future.
    Many people will be benefited from your writing.

  44. oh,
    it is so late to find this.
    very nice.

  45. law says:

    Useful information. Fortunate me I found your web
    site by chance, and I’m shocked why this coincidence did not took place earlier!
    I bookmarked it.

  46. Vinicius says:

    Hey, how are you?
    So, I am a Unity/ActionScript 3 programmer, but I am giving a try to Construct 2, and then I would like to know if your framework works with Construct 2. Is that possible? Or do you suggest me other options?

  47. Ritu agrawal says:

    it is not working on mobile phones, canvas is not able to play video

  48. Jhon says:

    hi, can add another image for glassess ? example . . . a image 598 X 476 . . . what need modify ?

  49. very helpful, finally i get my problem resolved.

Leave a Reply

Your email address will not be published. Required fields are marked *