George McKinney Adventures in Software Development

My friend @Yosun on Twitter of AReality3D asked me to create a mobile AR app using Adobe PhoneGap, http://www.phonegap.com, and the JS Aruco library, https://code.google.com/p/js-aruco. JS Aruco utilizes a port of OpenCV, http://opencv.org, for JavaScript, http://www.uco.es/investiga/grupos/ava/node/26. It takes advantage of HTML5 browser capabilities getUserMedia (http://caniuse.com/stream) and WebGL (checkout Three.js, https://github.com/mrdoob/three.js) to enable desktop browsers to create augmented reality using JavaScript.

I used Google’s Chrome Browser to verify the web application worked on the desktop. This got me thinking about the recently released BlackBerry 10 phones and their heavily touted “better than desktop Chrome” mobile browser, and I thought I would give it a try. Heading over to one of the JS Aruco demos, http://inmensia.com/files/aruco/debug-posit/debug-posit.html, in the Z10’s mobile browser, it worked. I decided to take things a bit further and actually build a mobile app rather than use the website application. I used WebWorks which is the BlackBerry-engineered Cordova PhoneGap distribution with deeper integration with device capabilities than Cordova on BlackBerry mobiles. The result of the test was this video, http://www.youtube.com/watch?v=eb10iEwDvvs. Performance is a little slow and the video a bit hazy but I know it will perform better with some code optimization…

A little background about augmented reality. I started experimenting with augmented reality back in 2009 using using the FLAR Toolkit, http://www.libspark.org/wiki/saqoosha/FLARToolKit/en, with Papervision3D, https://code.google.com/p/papervision3d, for demoing virtual fashion shows in LA. FLAR Toolkit enabled Flash developers to create browser-based experiences using 3D and the video stream from a computer’s web camera. This technology uses black-and-white markers for tracking. A pretty-cool way for developers playing around with computer vision.

Total Immersion, http://www.t-immersion.com, took this experience to a higher level when the company released its first Flash-based AR SDK for their industry-leading D’Fusion studio. This brought their hallmark markerless tracking ability using live video streams with excellent performance to corporate brands. Take a look at this project I worked on for Kraft using it, http://www.georgemckinney.com/2010/08/augmented-reality-august-2010. Unfortunately, the licensing cost of their solution kept it out of reach for most developers.

Fast-forward a couple years and now Flash is a less emphasized technology and being rapidly displaced by HTML5 on destkop and by native coding in mobile devices. Qualcomm released its augmented reality SDK for Android and iOS devices, Vuforia, for free which brings native and Unity 3D mobile developers the ability to create AR. Many brands have adopted this solution including American Apparel, http://www.qualcomm.com/media/releases/2012/06/27/vuforia-extends-image-recognition-cloud, for use with smartphone apps.

Mobile Augmented Reality with JavaScript can be considered another piece of the puzzle being filled in to replace Flash. With this success, I see many more applications that are leveraging the dynamic nature of JavaScript and the ease of use and device capabilities a modern phone provides. Congratulations to all the projects and teams involved here in breaking new ground.