As you may know, you can easily develop native applications for Windows 8 using JavaScript and WinJS.

Among a lot of other things, WinJS allows you to create custom control in order to factorize your UI.

Today, I would like to show you how to create this kind of control and to do so, please let me introduce you with the final version of the control:

Read More

After creating a GIF file using my previous article, I propose you to use the power of WinRT to create a component in order to produce video file from your drawings (using for instance a canvas with Javascript).

To do so, we will use Media Foundation COM components. Media Foundation is the next generation multimedia platform for Windows that enables developers, consumers, and content providers to embrace the new wave of premium content with enhanced robustness, unparalleled quality, and seamless interoperability.

And as I said before, Media Foundation is based on COM components. But thanks to C++ projects for Windows Store, we can create a WinRT component on top of Media Foundation in order to use it from our Javascript or .NET projects.

Read More

Why using GIF files?

For one of my project (Flipflop: https://apps.microsoft.com/windows/app/flipflop/99c01512-fe4f-4d1a-872e-eb9fd6638ff4), I needed to create an animated GIF based on a group of HTML5 canvas.

A GIF produced by Flipflop

The main goal of Flipflop is to allow users to create an animated flipbook. And to allow them to share it, I finally found that GIF files were a great deal.

Creating a GIF using WIC components

So I propose you to share the small class that I used to achieve this goal: GifMaker.

To work this class requires you to specify frames size and obviously all frames:

using System;
using System.Collections.Generic;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.Foundation;
using Windows.Graphics.Imaging;
using Windows.Storage;

namespace Utilities
{
    public sealed class GifMaker
    {
        readonly List<byte[]> frames = new List<byte[]>();
        private readonly uint frameWidth;
        private readonly uint frameHeight;

        public GifMaker(uint width, uint height)
        {
            frameWidth = width;
            frameHeight = height;
        }

        public void AppendNewFrame([ReadOnlyArray]byte[] frame)
        {
            frames.Add(frame);
        }

 

This class mainly uses WIC or actually Windows.Graphics namespace. This namespace contains a class called BitmapEncoder that will allow us to generate files using specific encoders such as GifEncoder.

The point is that GIF files are composed of a list of frames. You just have to browse through them and send each of them to the encoder using GoToNextFrameAsync to register a new frame:

public IAsyncInfo GenerateAsync(StorageFile file)
{
    return AsyncInfo.Run(async ctx =>
        {
            var outStream = await file.OpenAsync(FileAccessMode.ReadWrite);

            var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.GifEncoderId, outStream);

            for (int i = 0; i < frames.Count; i++)
            {
                var pixels = frames[i];
                encoder.SetPixelData(BitmapPixelFormat.Rgba8, BitmapAlphaMode.Ignore,
                                     frameWidth, frameHeight,
                                     92.0, 92.0,
                                     pixels);

                if (i < frames.Count - 1)
                    await encoder.GoToNextFrameAsync();
            }

            await encoder.FlushAsync();
            outStream.Dispose();
        });
}

Easy, isn’t it ?

Adding delay between frames

To be complete, you also have to handle the delay between frames and for this point, I must admit it took me a long time to figure out how to do that.

The solution is handled by frame properties (encoder.BitmapProperties) and especially via the property named “/grctlext/Delay” (!!).

To the final code is like the following:

public IAsyncInfo GenerateAsync(StorageFile file, int delay)
{
    return AsyncInfo.Run(async ctx =>
        {
            var outStream = await file.OpenAsync(FileAccessMode.ReadWrite);

            var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.GifEncoderId, outStream);

            for (int i = 0; i < frames.Count; i++)
            {
                var pixels = frames[i];
                encoder.SetPixelData(BitmapPixelFormat.Rgba8, BitmapAlphaMode.Ignore,
                                     frameWidth, frameHeight,
                                     92.0, 92.0,
                                     pixels);

                if (i == 0)
                {
                    var properties = new BitmapPropertySet
                        {
                            {
                                "/grctlext/Delay",
                                new BitmapTypedValue(delay / 10, PropertyType.UInt16)
                            }
                        };

                    await encoder.BitmapProperties.SetPropertiesAsync(properties);
                }

                if (i < frames.Count - 1)
                    await encoder.GoToNextFrameAsync();
            }

            await encoder.FlushAsync();
            outStream.Dispose();
        });
}

 

Using it from other languages

For instance if you want to use it from JavaScript, you just have to do something like that:

var picker = new Windows.Storage.Pickers.FileSavePicker();
picker.fileTypeChoices.insert("Gif files", [".gif"]);

picker.pickSaveFileAsync().then(function(file) {
    if (!file) {
        return;
    }
    var gifMaker = new Utilities.GifMaker(800, 600);

    for (var commandsIndex = 0; commandsIndex < currentFlipflop.pages.length; commandsIndex++) {
        // This code is used to retrieve canvases bytes
        var bytes = Flipflop.Tools.GetBytesfromFlipflop(currentFlipflop.pages[commandsIndex],
            800, 600);

        gifMaker.appendNewFrame(bytes);
    }

    gifMaker.generateAsync(file, 200).done();
});

Obviously it works the same way from .NET!

A good-looking application must provide user with visual feedback. User must always know that an order (a click, a tap or whatever) is well received and understood by the application and animations are a great tool to do so.

The new HTML 5 specification (to be honest, I should say “the new CSS 3 specification”) introduces a great tool to handle simple animations: the transitions.

According to “CSS Transitions Module Level 3” specification on W3C site, CSS3 Transitions allows property changes in CSS values to occur smoothly over a specified duration.

The aim of this article will be to first describe the concept of transitions and then to see how CSS3 Transitions works and how we can handle browsers that don’t support the feature:

1. CSS3 Transitions
2. Putting it all together
3. Transitions without CSS3 Transitions
4. Conclusion
5. Going further

In addition, I suggest you to read the “Introduction to CSS3 Animations” (by David Rousset) which is an excellent companion for this article.

To see how CSS3 Transitions can be used, I developed is a sample of a game which uses CSS3 Transitions to animate cells of a puzzle (and which will fallback to JavaScript if your browser doesn’t support CSS3 Transitions): https://www.catuhe.com/msdn/transitions/index.htm

To launch a game, just click on “Mix it!” and try to resolve the puzzle by clicking the cells!

The code of this game is available here.

CSS3 Transitions

Introduction

At the beginning, the W3C CSS workgroup resisted adding transitions to CSS arguing that transitions are not really style properties. But eventually designers and developers managed to convince them that transitions is about dynamic styles and can take place in a CSS file.

Declarations

To declare a transition in a CSS file, you just have to write the following code:

This declaration defines that any update on any property will be done in 0.5s (and not immediately so).

You can also define your translations on a per property basis:

And finally you can use the shorthand property “transition” to define all you need in a single line:

In this shorthand version you can precise as many properties as you want separated by a comma:

The transitions will be triggered when a property of the target object is updated. The update can be done with JavaScript or using CSS3 by assign new class to a tag.

For example, using IE10 if you have the following CSS3 declaration:

When you update the opacity of your tag, the current value will be animated to the new value over 0.5s with a ease timing function (which give a smooth animation).

Non linear transitions

The “transition-timing-function” line defines that the transition will not be linear but will use a timing function to produce a non linear animation.

Basically, CSS3 transitions will use cubic bezier curve to smooth the transition by computing different speed over its duration.

The following functions are supported:

• linear: Constant speed
• cubic-bezier: Speed will be computed according to a cubic bezier curve define by two control points: P0 et P1 (so you will have to define 4 values here: P0x,P0y and P1x, P1y.
• ease: Speed will be computed with cubic-bezier(0.25, 0.1, 0.25, 1)
• ease-in: Speed will be computed with cubic-bezier(0.42, 0, 1, 1)
• ease-inout: Speed will be computed with cubic-bezier(0.42, 0, 0.58, 1)
• ease-out: Speed will be computed with cubic-bezier(0, 0, 0.58, 1)

Here is a simulation tool (using SVG of course) to show the impact of each timing function: https://www.catuhe.com/msdn/transitions/easingfunctions.htm

This simulator is written with pure JavaScript code to facilitate the understanding of the function:

This code is the implementation of the cubic bezier based on this definition and you can find the source of the simulator here.

Delay

The “transition-delay” line defines the delay between an update of a property and the start of the transition

Events

An event is raised at the end of a transition: “TransitionEnd”. According to your browser the correct name will be:

• Chrome & Safari: webkitTransitionEnd
• Firefox: mozTransitionEnd
• Opera: oTransitionEnd
• Internet Explorer: MSTransitionEnd

The event will give you the following information:

• propertyName: Name of the animated property
• elapsedTime: The amount of time the transition has been running, in seconds

Here is an usage sample for IE10:

More about CSS3 transitions

I can mainly propose two reasons why CSS3 transitions are really useful:

##

• Hardware acceleration: CSS3 Transitions are directly handled on the GPU (where available) and produce smoother results. And it is really important on mobile devices where computing power is really limited
• Better separation between code and design: For me, the developer must not be aware of animations or anything related to design. In the same way the designer/artist must not be aware of JavaScript. That’s why CSS3 Transitions are really interesting as designers can describe all the transitions in the CSS without needing developers

Support and fallback

Since PP3, IE10 (which you can download with Windows “8” Developer Preview here) supports CSS3 Transitions:

This report was produced by https://caniuse.com/#search=CSS3 transitions.

Of course, as the specification is not finished (working draft), you must use vendor’s prefixes such as –ms-, –moz-, –webkit-, –o-.

We can obviously see that we need to provide a transparent solution in order to address all kind of browsers. The best way will be to develop an API that can detect the support of CSS3 transitions. If the browser doesn’t support the feature, we will fallback to some JavaScript code.

It is important to support a fallback method if you rely on transitions for websites functionalities. If you don’t want to do that, you should consider using transitions only for design enhancements. In this case, the site will still work but only supported browsers will deliver the full experience. We speak here of “progressive enhancements” as the more powerfull the browser is, the more features he gets.

Transitions without CSS3 Transitions

So to be able to support a fallback to CSS3 Transitions, we will develop a small toolkit to provide transitions by code.

First of all, we will create a container object for our namespace:

To support the same level of easing functions, we must declare an “enum” with all required fields.

The toolkit is based on a function which is called every 17ms (to achieve animations at 60 fps). The function will enumerate through a collection of active transitions. For each transition the code will evaluate the next value given the current value and the target value.

We will need some handy functions to extract value of properties and units used:

The main function will process active transitions and will call the cubic bezier function to evaluate current values:

The current version of the toolkit only supports numeric values but if you want to animate complex values (such as color) you just have to decompose them to simple values.

Registering a transition in the system will be done using the following code:

The “tick” function is launched when the first transition is activated.

Finally you just have to use modernizr to define if CSS3 Transitions is supported by the current browser. If not, you can fallback to our toolkit.

The code for the TransitionsHelper can be downloaded here: https://www.catuhe.com/msdn/transitions/transitionshelper.js

For example, in my puzzle game, the following code is used to animate the cells:

We can note that I could use another way to animate my cells when CSS3 transitions are supported: I could have defined a collection of CSS3 classes with predefined left and top values (one for each cell) to affect them to right cells.

Some frameworks and toolkits already exist to support software transitions:

• jQuery.transition.js: https://louisremi.github.com/jquery.transition.js/test/index.html
• jQUery-Animate-Enhanced: https://github.com/benbarnett/jQuery-Animate-Enhanced

By the way, you can also use the old good animate() method of jQuery.

Conclusion

As we saw, CSS3 Transitions is a really easy way to add animations to your project. You can produce a more reactive application just by using some transitions when you want to change values.

By the way, there are two solutions if you want to implement a JavaScript fallback:

• You can do all in the JavaScript side and if you detect the support of CSS3 transitions, you will inject CSS3 declarations in the page.
• Or you can use standard way (using true CSS3 declarations in the CSS files) and just detect the need of fallback in JavaScript. For me, it is the better option as the fallback must be an option and not the main subject. In a near future, all browsers will support CSS3 Transitions and in this case you will just have to remove your fallback code. Furthermore, it is a better way to let all the CSS under the control of the creative team and not in the code part.

Going further

• Blog about CSS3 Animations by David Rousset: https://blogs.msdn.com/b/davrous/archive/2011/12/06/introduction-to-css3-animations.aspx
• CSS3 Transitions specifications: https://www.w3.org/TR/css3-transitions/
• IE Test Drive on CSS3 transitions: https://ie.microsoft.com/testdrive/Graphics/hands-on-css3/hands-on_transitions.htm
• Others useful posts:
  • https://addyosmani.com/blog/css3transitions-jquery/
  • https://css3.bradshawenterprises.com/
  • https://webdesignerwall.com/trends/47-amazing-css3-animation-demos

During a French event called TechDays 2013, I presented a session called “Coding4Fun”. And during this session, I demoed a game called Kinect Dawanoid.

The goal of Kinect Dawanoid is to control a paddle (the racket of the Arkanoid game) with ambient noise only.

The big deal is that there is not one player but hundreds . In this case, the paddle will go to the left if there is more noise to the left and obviously the paddle will go to the right if there is more noise to the right:

Infographic courtesy of Michel Rousseau

 

To well understand how it looks like, I suggest you to have a look at this replay: 

The complete session (in french sorry):   
https://www.youtube.com/watch?v=wTtnhal35Rc&feature=youtu.be / https://www.microsoft.com/france/mstechdays/

The game itself

Before looking at the Kinect part, let me introduce you with the concept (!!) of the game:

 

The principle is simple: You control the paddle (until now with cursors keys) and the game uses a really simple physic system to simulate the movement of a ball. You lose the game when the ball falls down.

For the sake of clarity, I did not include the bricks.

Kinect Dawanoid is a simple WPF application. To control the ball, you need the following variables:

Point ballPosition = new Point(0, 0);
Vector ballDirection = new Vector(2, 2);

The game itself is based on two functions:

void StartGame()
{
    CompositionTarget.Rendering += CompositionTarget_Rendering;
    KeyDown += MainWindow_KeyDown;
}

The paddle is controlled using the _MainWindow_KeyDown_ method:

void MainWindow_KeyDown(object sender, KeyEventArgs e)
{
    switch (e.Key)
    {
        case Key.Left:
            padDirection -= padSpeed;
            break;
        case Key.Right:
            padDirection += padSpeed;
            break;
    }
}

The rendering part is based on the _CompositionTarget_Rendering_ method:

void CompositionTarget_Rendering(object sender, EventArgs e)
{
    // Pad
    padPosition += padDirection;
    if (padPosition < 0)
    {
        padDirection = 0;
        padPosition *= -1;
    }
    else if (padPosition > playground.RenderSize.Width - pad.Width - 1)
    {
        padPosition = playground.RenderSize.Width - pad.Width - 1;
        padDirection *= -1;
    }

    padDirection *= padInertia;


    // Ball
    ballPosition += ballDirection;

    // Walls
    if (ballPosition.X < 0)
    {
        ballPosition.X = 0;
        ballDirection.X *= -1;
    }
    else if (ballPosition.X >= playground.RenderSize.Width - ball.Width)
    {
        ballPosition.X = playground.RenderSize.Width - ball.Width - 1;
        ballDirection.X *= -1;
    }

    if (ballPosition.Y < 0)
    {
        ballPosition.Y = 0; 
        ballDirection.Y *= -1;
    }
    else if (ballPosition.Y >= playground.RenderSize.Height - ball.Height)
    {
        CompositionTarget.Rendering -= CompositionTarget_Rendering;
        ScoreText.Text = "Final score: " + (score / 10).ToString();
        return;
    }

    // Collisions
    var padRect = new Rect(padPosition, playground.RenderSize.Height - 50, pad.Width, pad.Height);
    var ballRect = new Rect(ballPosition, new Size(ball.Width, ball.Height));

    if (padRect.IntersectsWith(ballRect))
    {
        ballPosition.Y = playground.RenderSize.Height - 50 - ball.Height;
        ballDirection.Y *= -1;
    }

    // Moving
    Canvas.SetLeft(ball, ballPosition.X);
    Canvas.SetTop(ball, ballPosition.Y);

    Canvas.SetTop(pad, playground.RenderSize.Height - 50);
    Canvas.SetLeft(pad, padPosition);

    // Score
    ScoreText.Text = (score / 10).ToString();
    score++;
}

Nothing special here. The system evaluates the new ball position, check the collisions with the pad and the walls and finally update the position of the ball (which is represented by a XAML ellipse Inside a canvas) and the paddle (a rectangle).

 

Controlling the paddle with ambient noise

Thanks to Kinect, we have a wonderful way to detect ambient noise (via beam angle).

Indeed, to capture audio, the Kinect sensor receives sounds from every direction. However, like the cone of light from a lighthouse where the light is the brightest, the audio capture hardware has an imaginary cone that is able to capture audio signals the best. Audio waves that propagate through the length of the cone can be separated from audio waves that travel across the cone. If you point the cone in the direction of the audio that your application is most interested in capturing, you can improve the ability to capture and separate that audio source from other competing audio sources.

Similar to the sound source angle, the beam angle is also defined in the x-z plane of the sensor perpendicular to the z-axis. The beam angle and the sound source angle are both updated continuously once the sensor has started streaming audio data (when the Start method is called).

Furthermore, Kinect SDK provides you with an event raised when beam angle changes. So, using the following code, you can easily track the position of the strongest sound source:

 

using Microsoft.Kinect;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Dawanoïd
{
    public class BeamManager : INotifyPropertyChanged
    {
        readonly KinectAudioSource audioSource;

        public BeamManager(KinectAudioSource source)
        {
            audioSource = source;
            audioSource.BeamAngleChanged += audioSource_BeamAngleChanged;
        }

        void audioSource_BeamAngleChanged(object sender, BeamAngleChangedEventArgs e)
        {
            if (PropertyChanged != null)
            {
                PropertyChanged(this, new PropertyChangedEventArgs("BeamAngle"));
            }
        }

        public double BeamAngle
        {
            get
            {
                return Math.Max(0, Math.Min(1, (audioSource.BeamAngle - KinectAudioSource.MinBeamAngle)
                            / (KinectAudioSource.MaxBeamAngle - KinectAudioSource.MinBeamAngle)));
            }
        }

        public void Dispose()
        {
            audioSource.BeamAngleChanged -= audioSource_BeamAngleChanged;
        }

        public event PropertyChangedEventHandler PropertyChanged;
    }
}

 

So instead of controlling the paddle with the keyboard, we can just add this line in the _CompositionTarger_Rendering_ method:

padDirection += beamManager.BeamAngle * 2.0 - 1.0;

And voila!!

The best way to understand Kinect Dawanoïd is to try it: https://www.catuhe.com/msdn/dawanoïd.zip

How about being able to write a single code base for handling mouse, pen, touch in your web site that will work across all modern browsers?
Here is a polyfill that will help you use Pointer Events and offer users a great experience on your site independently of the modern browser they are using.

Back in September 2012, Microsoft proposed a specification to W3C for unifying touch, pen and mouse events called Pointer Events and based on the APIs available today in IE10 on Windows 8. The W3C has since created a new Working Group which already published a last call draft.The Microsoft Open Technologies, Inc. team has also released a Pointer Events initial prototype for Webkit on HTML5Labs to further contribute to the technical discussions in the W3C Pointer Events Working Group.    

My smart colleague David Rousset wrote an excellent article on this subject: https://blogs.msdn.com/b/davrous/archive/2013/02/20/handling-touch-in-your-html5-apps-thanks-to-the-pointer-events-of-ie10-and-windows-8.aspx.

As this specification is not final, modern browsers are not implementing it yet (only Internet Explorer 10 on Windows 8 implements it with vendor-prefixes MSPointerXXX). To help you ready your code for this upcoming standard, this article will show you how to write a polyfill that will allow you to have a single markup code that will work across all modern browsers even if they don’t support the Pointer Events specification yet.    

To illustrate how simple Hand.js is to use, I took the first sample in David Rousset’s article, I changed every MSPointerXXX to PointerXXX and I added a reference to “Hand.js” and voila! (just move your finger or your mouse on top of this iframe):

  

The test code

To test Hand.js I developed a simple HTML page (available here: https://www.catuhe.com/msdn/handjs/index.html)

<!DOCTYPE html>
<html xmlns="https://www.w3.org/1999/xhtml">
<head>
    <title></title>
    <link href="index.css" rel="stylesheet" />
    <script src="hand.js"></script>
    <script src="index.js"></script>
</head>
<body>
    <div id="title">Hand.js: a framework for pointer events</div>
    <canvas id="plainCanvas"></canvas>
</body>
</html>

The following JavaScript is used to draw inside the canvas using normalized pointer events:

var context;
var plainCanvas;
var pointerDown = {};
var lastPositions = {};
var colors = ["rgb(100, 255, 100)", "rgb(255, 0, 0)", "rgb(0, 255, 0)", 
              "rgb(0, 0, 255)", "rgb(0, 255, 100)", "rgb(10, 255, 255)", "rgb(255, 0, 100)"];

var onPointerMove = function(evt) {
    evt.preventDefault();

    if (pointerDown[evt.pointerId]) {

        var color = colors[evt.pointerId % colors.length];

        context.strokeStyle = color;

        context.beginPath();
        context.lineWidth = 2;
        context.moveTo(lastPositions[evt.pointerId].x, lastPositions[evt.pointerId].y);
        context.lineTo(evt.clientX, evt.clientY);
        context.closePath();
        context.stroke();

        lastPositions[evt.pointerId] = { x: evt.clientX, y: evt.clientY};
    }
};

var onPointerOut = function (evt) {
    evt.preventDefault();
    pointerDown[evt.pointerId] = false;
};

var onPointerUp = function (evt) {
    evt.preventDefault();
    pointerDown[evt.pointerId] = false;
};

var onPointerDown = function (evt) {
    evt.preventDefault();
    pointerDown[evt.pointerId] = true;

    lastPositions[evt.pointerId] = { x: evt.clientX, y: evt.clientY};
};

var onload = function() {
    plainCanvas = document.getElementById("plainCanvas");

    plainCanvas.width = plainCanvas.clientWidth;
    plainCanvas.height = plainCanvas.clientHeight;

    context = plainCanvas.getContext("2d");

    context.fillStyle = "rgba(50, 50, 50, 1)";
    context.fillRect(0, 0, plainCanvas.width, plainCanvas.height);

    plainCanvas.addEventListener("PointerDown", onPointerDown, false);
    plainCanvas.addEventListener("PointerMove", onPointerMove, false);
    plainCanvas.addEventListener("PointerUp", onPointerUp, false);
    plainCanvas.addEventListener("PointerOut", onPointerUp, false);
};

document.addEventListener("DOMContentLoaded", onload, false);

You will note that I’m using “PointerXXX” which is not supported right now on modern browsers, except for Internet Explorer 10 on Windows 8 that supports the prefixed version “_MSPointerXXX”.

_

The goal of Hand.js is to allow this code to work on every browser whether it supports touch or not, allowing for a seamless and unified experience independent of the input method utilized (touch, mouse or pen).

Rerouting addEventListener

The usage of Hand.js must be transparent for the developer. He just has to reference the library and let the magic works:

<script src="hand.js"></script>

The way it works in Hand.js is by intercepting calls to addEventListener to seamlessly inject our code:

 

var supportedEventsNames = ["PointerDown", "PointerUp", "PointerMove", "PointerOver", "PointerOut", 
                            "PointerCancel", "PointerEnter", "PointerLeave",
                            "pointerdown", "pointerup", "pointermove", "pointerover", "pointerout", 
                            "pointercancel", "pointerenter", "pointerleave"];

// Intercept addEventListener calls by changing the prototype
var interceptAddEventListener = function (root) {
    var current = root.prototype.addEventListener;

    var customAddEventListener = function (name, func, capture) {
       // Branch when a PointerXXX is used
       if (supportedEventsNames.indexOf(name) != -1) {
           makeTouchAware(this, name);
       }

       current.call(this, name, func, capture);
    };

    root.prototype.addEventListener = customAddEventListener;

    return (root.prototype.addEventListener != customAddEventListener);
};

interceptAddEventListener(HTMLBodyElement);
interceptAddEventListener(HTMLCanvasElement);
interceptAddEventListener(HTMLDivElement);
interceptAddEventListener(HTMLImageElement);
interceptAddEventListener(HTMLSpanElement);       

The interceptAddEventListener function calls a custom function (makeTouchAware) to generate pointer events on specific DOM objects when standardized pointer events are registered (PointerXXX).

We can do that by replacing the addEventListener function on base prototypes.

For almost all browsers, we just have to work on HTMLElement root object. But some of them do not support this way and forces us to work on specific DOM root element (body, canvas, div, image, span, etc…)

Supporting MSPointerXXX

The makeTouchAware function starts like this:

var makeTouchAware = function (item, eventName) {
    // If item is already touch aware, do nothing
    if (item.onpointerdown !== undefined) {
        return;
    }

    // IE 10
    if (item.onmspointerdown !== undefined) {
        var msEventName;

        if (eventName == eventName.toLowerCase()) {
            var indexOfUpperCase = supportedEventsNames.indexOf(eventName) - 
                      (supportedEventsNames.length / 2);
            msEventName = "MS" + supportedEventsNames[indexOfUpperCase];
        }
        else {
            msEventName = "MS" + eventName;
        }

        item.addEventListener(msEventName, 
           function (evt) { generateTouchClonedEvent(evt, eventName); }, false);

        // We can return because MSPointerXXX integrate mouse support
        return;
    }

If onmspointerdown is not undefined, then we can just register on prefixed event (MSPointerXXX) and call a function to raise a standardized event (PointerXXX) when the source event is fired (We will come back to the generateTouchClonedEvent later).

Supporting touchstart / touchmove / touchend / touchcancel

For browsers that not supported Pointer events but implement the Touch events specification, we simply have to map the events names by adding the following code:

// Chrome, Firefox
if (item.ontouchstart !== undefined) {
    switch (eventName.toLowerCase()) {
        case "pointerdown":
            item.addEventListener("touchstart", function (evt) { 
                            handleOtherEvent(evt, eventName); }, false);
            break;
        case "pointermove":
            item.addEventListener("touchmove", function (evt) { 
                            handleOtherEvent(evt, eventName); }, false);
            break;
        case "pointerup":
            item.addEventListener("touchend", function (evt) { 
                            handleOtherEvent(evt, eventName); }, false);
            break;
        case "pointercancel":
            item.addEventListener("touchcancel", function (evt) { 
                            handleOtherEvent(evt, eventName); }, false);
            break;
    }
}

Based on the existence of the ontouchstart function, we can use the following function to generate pointer events:

var handleOtherEvent = function (eventObject, name) {
    if (eventObject.preventManipulation)
        eventObject.preventManipulation();

    for (var i = 0; i < eventObject.changedTouches.length; ++i) {
        var touchPoint = eventObject.changedTouches[i];
        var touchPointId = touchPoint.identifier + 2; // Just to not override mouse id

        touchPoint.pointerId = touchPointId;
        touchPoint.pointerType = POINTER_TYPE_TOUCH; 
        touchPoint.currentTarget = eventObject.currentTarget;

        if (eventObject.preventDefault !== undefined) {
            touchPoint.preventDefault = function () {
                eventObject.preventDefault();
            };
        }

        generateTouchClonedEvent(touchPoint, name);
    }
};

handleOtherEvent mainly dispatches every changedTouch to a pointer event. Please note the added function named preventDefault in order to route call from pointer event to root touch event.

Integrating mouse events

The Pointer Event specification indicates that mouse events are also gathered inside the same model. Hand.js handles mouse events and generates corresponding Pointer events:

// Fallback to mouse
switch (eventName.toLowerCase()) {
    case "pointerdown":
     item.addEventListener("mousedown", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
    case "pointermove":
     item.addEventListener("mousemove", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
    case "pointerup":
     item.addEventListener("mouseup", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
    case "pointerover":
     item.addEventListener("mouseover", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
    case "pointerout":
     item.addEventListener("mouseout", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
    case "pointerenter":
     item.addEventListener("mouseenter", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
    case "pointerleave":
     item.addEventListener("mouseleave", function (evt) { generateMouseProxy(evt, eventName); }, false);
     break;
}

The generateMouseProxy produces a pointer event based on the mouse event:

var generateMouseProxy = function (evt, eventName) {
    evt.pointerId = 1;
    evt.pointerType = POINTER_TYPE_MOUSE; //event.POINTER_TYPE_MOUSE
    generateTouchClonedEvent(evt, eventName);
};

Generating custom events

The last missing piece of code is the generateTouchClonedEvent function. This function is in charge of mimicking the pointer event structure:

var POINTER_TYPE_TOUCH = "touch";
var POINTER_TYPE_PEN = "pen";
var POINTER_TYPE_MOUSE = "mouse";

// Touch events
var generateTouchClonedEvent = function (sourceEvent, newName) {
    // Considering touch events are almost like super mouse events
    var evObj = document.createEvent('MouseEvents');
    evObj.initMouseEvent(newName, true, true, window, 1, sourceEvent.screenX, sourceEvent.screenY,
        sourceEvent.clientX, sourceEvent.clientY, sourceEvent.ctrlKey, sourceEvent.altKey,
        sourceEvent.shiftKey, sourceEvent.metaKey, sourceEvent.button, null);

    // offsets
    if (evObj.offsetX === undefined) {
        if (sourceEvent.offsetX !== undefined) {

            // For Opera which creates readonly properties
            if (Object && Object.defineProperty !== undefined) {
                Object.defineProperty(evObj, "offsetX", {
                    writable: true
                });
                Object.defineProperty(evObj, "offsetY", {
                    writable: true
                });
            }

            evObj.offsetX = sourceEvent.offsetX;
            evObj.offsetY = sourceEvent.offsetY;
        }
        else if (sourceEvent.layerX !== undefined) {
            evObj.offsetX = sourceEvent.layerX - sourceEvent.currentTarget.offsetLeft;
            evObj.offsetY = sourceEvent.layerY - sourceEvent.currentTarget.offsetTop;
        }
    }

    // adding missing properties
    evObj.pointerId = sourceEvent.pointerId;
    evObj.pointerType = sourceEvent.pointerType;

    if (sourceEvent.isPrimary !== undefined)
        evObj.isPrimary = sourceEvent.isPrimary;
    else
        evObj.isPrimary = true;

    if (sourceEvent.pressure)
        evObj.pressure = sourceEvent.pressure;
    else {
        var button = 0;

        if (sourceEvent.which !== undefined)
            button = sourceEvent.which;
        else if (sourceEvent.button !== undefined) {
            button = sourceEvent.button;
        }
        evObj.pressure = (button == 0) ? 0 : 0.5;
    }


    if (sourceEvent.rotation)
        evObj.rotation = sourceEvent.rotation;
    else
        evObj.rotation = 0;

    // Timestamp
    if (sourceEvent.hwTimestamp)
        evObj.hwTimestamp = sourceEvent.hwTimestamp;
    else
        evObj.hwTimestamp = 0;

    // Tilts
    if (sourceEvent.tiltX)
        evObj.tiltX = sourceEvent.tiltX;
    else
        evObj.tiltX = 0;

    if (sourceEvent.tiltY)
        evObj.tiltY = sourceEvent.tiltY;
    else
        evObj.tiltY = 0;

    // Width and Height
    if (sourceEvent.height)
        evObj.height = sourceEvent.height;
    else
        evObj.height = 0;

    if (sourceEvent.width)
        evObj.width = sourceEvent.width;
    else
        evObj.width = 0;

    // PreventDefault
    evObj.preventDefault = function () {
        if (sourceEvent.preventDefault !== undefined)
            sourceEvent.preventDefault();
    };

    // Constants
    evObj.POINTER_TYPE_TOUCH = POINTER_TYPE_TOUCH;
    evObj.POINTER_TYPE_PEN = POINTER_TYPE_PEN;
    evObj.POINTER_TYPE_MOUSE = POINTER_TYPE_MOUSE;

    // If force preventDefault
    if (sourceEvent.currentTarget.handjs_forcePreventDefault === true)
        evObj.preventDefault();

    // Fire event
    sourceEvent.target.dispatchEvent(evObj);
};

Using JavaScript custom events mechanism, we are able to dispatch the new event with dispatchEvent function. This code is fairly simple. It copies event properties from source to destination and creates default values if required.

Please note that for Opera we have to update offsetX and offsetY properties to make them writable and for Firefox, we have to compute offsetX and offsetY.

Cleaning things

To be complete we must also override the removeEventListener to remove the event handlers injected previously.

First of all, we have to hook the removeEventLister with our own function:

// Intercept removeEventListener calls by changing the prototype
var interceptRemoveEventListener = function (root) {
    var current = root.prototype.removeEventListener;

    var customRemoveEventListener = function (name, func, capture) {
        // Branch when a PointerXXX is used
        if (supportedEventsNames.indexOf(name) != –1){
            removeTouchAware(this, name);
        }

        current.call(this, name, func, capture);
    };

    root.prototype.removeEventListener = customRemoveEventListener;
};

// Hooks
interceptAddEventListener(HTMLBodyElement);
interceptAddEventListener(HTMLCanvasElement);
interceptAddEventListener(HTMLDivElement);
interceptAddEventListener(HTMLImageElement);
interceptAddEventListener(HTMLSpanElement);

interceptRemoveEventListener(HTMLBodyElement);
interceptRemoveEventListener(HTMLCanvasElement);
interceptRemoveEventListener(HTMLDivElement);
interceptRemoveEventListener(HTMLImageElement);
interceptRemoveEventListener(HTMLSpanElement);

The removeTouchAware function is similar to the makeTouchAware except that it removes the event handlers:

var removeTouchAware = function (item, eventName) {
    // If item is already touch aware, do nothing
    if (item.onpointerdown !== undefined) {
        return;
    }

    // IE 10
    if (item.onmspointerdown !== undefined) {
        var msEventName;

        if (eventName == eventName.toLowerCase()) {
            var indexOfUpperCase = supportedEventsNames.indexOf(eventName) - 
                (supportedEventsNames.length / 2);
            msEventName = "MS" + supportedEventsNames[indexOfUpperCase];
        }
        else {
            msEventName = "MS" + eventName;
        }
        item.removeEventListener(msEventName, function (evt) { 
                generateTouchClonedEvent(evt, eventName); });
        return;
    }

    // Chrome, Firefox
    if (item.ontouchstart !== undefined) {
        switch (eventName.toLowerCase()) {
            case "pointerdown":
         item.removeEventListener("touchstart", function (evt) { handleOtherEvent(evt, eventName); });
         break;
            case "pointermove":
         item.removeEventListener("touchmove", function (evt) { handleOtherEvent(evt, eventName); });
         break;
            case "pointerup":
         item.removeEventListener("touchend", function (evt) { handleOtherEvent(evt, eventName); });
         break;
            case "pointercancel":
         item.removeEventListener("touchcancel", function (evt) { handleOtherEvent(evt, eventName); });
         break;
        }
    }

    // Fallback to mouse
    switch (eventName.toLowerCase()) {
        case "pointerdown":
         item.removeEventListener("mousedown", function (evt) { generateMouseProxy(evt, eventName); });
         break;
        case "pointermove":
         item.removeEventListener("mousemove", function (evt) { generateMouseProxy(evt, eventName); });
         break;
        case "pointerup":
         item.removeEventListener("mouseup", function (evt) { generateMouseProxy(evt, eventName); });
         break;
        case "pointerover":
         item.removeEventListener("mouseover", function (evt) { generateMouseProxy(evt, eventName); });
         break;
        case "pointerout":
         item.removeEventListener("mouseout", function (evt) { generateMouseProxy(evt, eventName); });
         break;
        case "pointerenter":
         item.removeEventListener("mouseenter", function (evt) { generateMouseProxy(evt, eventName); });
         break;
        case "pointerleave":
         item.removeEventListener("mouseleave", function (evt) { generateMouseProxy(evt, eventName); });
         break;
    }
};

Extending Navigator object

According to the specification, the navigator object must provide two properties:

• pointerEnabled to indicate if the browser wiil fire pointer events for point input. For Hand.js it will return true.
• maxTouchPoints indicate the maximum number of simultaneous touch contacts supported by the device. Alas only Internet Explorer 10 supports this feature. For others browsers I was not able to find an API for this value.

 

For the sake of the specification, I added the following code to hand.js:

// Extension to navigator
if (navigator.pointerEnabled === undefined) {

    // Indicates if the browser will fire pointer events for pointing input
    navigator.pointerEnabled = true;

    // IE
    if (navigator.msPointerEnabled) {
        navigator.maxTouchPoints = navigator.msMaxTouchPoints;
    }
}

 

Integrating “touch-action” css rule

The touch-action CSS rule determines whether touch input may trigger default behavior supplied by user agent. This includes, but is not limited to, behaviors such as panning or zooming.

If you want to remove default behavior, you can use this rule like that (or obviously the vendor-specific version):

#plainCanvas {
    position:absolute;
    left:10%;
    top:10%;
    width:80%;
    height:80%;
    touch-action:none;
}

Hand.js will have to determine if the user defined this rule. To do so, we have to handle both inline and referenced styles. And we must take in account that browsers will remove unknown css rules (for instance the touch-action rule). So we cannot use th stylesheets property of the document (because it contains only valid rules).

To get unfiltered styles we must use this code which download the original css file or get the inner HTML for inline styles:

// Handling touch-action css rule
if (document.styleSheets) {
    document.addEventListener("DOMContentLoaded", function () {

        var trim = function (string) {
            return string.replace(/^s+|s+$/, '');
        }


        // Looking for touch-action in referenced stylesheets
        for (var index = 0; index < document.styleSheets.length; index++) {
            var sheet = document.styleSheets[index];

            if (sheet.href == undefined) { // it is an inline style
                continue;
            }

            // Loading the original stylesheet
            var xhr = new XMLHttpRequest();
            xhr.open("get", sheet.href, false);
            xhr.send();

            var unfilteredSheet = xhr.responseText.replace(/(n|r)/g, "");

            processStylesheet(unfilteredSheet);
        }

        // Looking for touch-action in inline styles
        var styles = document.getElementsByTagName("style");
        for (var index = 0; index < styles.length; index++) {
            var sheet = styles[index];

            var unfilteredSheet = trim(sheet.innerHTML.replace(/(n|r)/g, ""));

            processStylesheet(unfilteredSheet);
        }
    }, false);
}

The processStylesheet function uses regular expressions to determine if the “touch-action:none” string is present in the style:

var processStylesheet = function(unfilteredSheet) {
    var globalRegex = new RegExp(".+?{.*?}", "m");
    var selectorRegex = new RegExp(".+?{", "m");

    while (unfilteredSheet != "") {
        var block = globalRegex.exec(unfilteredSheet)[0];
        unfilteredSheet = trim(unfilteredSheet.replace(block, ""));
        var selectorText = trim(selectorRegex.exec(block)[0].replace("{", ""));

        // Checking if the user wanted to deactivate the default behavior
        if (block.replace(/s/g, "").indexOf("touch-action:none") != -1) {
            var elements = document.querySelectorAll(selectorText);

            for (var elementIndex = 0; elementIndex < elements.length; elementIndex++) {
                var element = elements[elementIndex];

                if (element.style.msTouchAction !== undefined) {
                    element.style.msTouchAction = "none";
                }
                else {
                    element.handjs_forcePreventDefault = true;
                }
            }
        }
    }
}

The handjs_forcePreventDefault is a boolean used by the generateTouchClonedEvent to define if the preventDefault must be forced:

// If force preventDefault
if (sourceEvent.currentTarget.handjs_forcePreventDefault === true)
    evObj.preventDefault();

Final code

Finally the complete code of Hand.js can be downloaded here: https://handjs.codeplex.com

Feel free to use this code for your own web pages in order to be able to handle touch and mouse in a unified way leveraging the Pointer Events model.

In order to further help the discussions in the W3C Pointer Events Working Group, we would love to hear from you on the specification itself. Do not hesitate to comment on this post and I will forward!

I love JSON format. It is human readable and really fast to serialize to and deserialize from. However the major drawback of JSON is its size on disk.

The following code will show you how to leverage System.IO.Compression (from .NET) to reduce the disk footprint of your JSON files.

To do so, just create a .NET WinRT component with this code:

public static class Compression
{
    public static IAsyncOperation<IList<byte>> Compress(string data)
    {
        return AsyncInfo.Run(async (ct) =>
            {
                UTF8Encoding encoding = new UTF8Encoding();
                using (MemoryStream zipMemoryStream = new MemoryStream())
                {
                    using (ZipArchive zipArchive = new ZipArchive(zipMemoryStream, ZipArchiveMode.Create))
                    {
                        var buffer = encoding.GetBytes(data);
                        ZipArchiveEntry entry = zipArchive.CreateEntry("flipflop");
                        using (Stream entryStream = entry.Open())
                        {
                            await entryStream.WriteAsync(buffer, 0, buffer.Length);
                        }
                    }

                    return (IList<byte>)zipMemoryStream.ToArray();
                }
            });
    }

    public static IAsyncOperation<string> Decompress([ReadOnlyArray] byte[] data)
    {
        return AsyncInfo.Run(async (ct) =>
            {
                UTF8Encoding encoding = new UTF8Encoding();
                using (MemoryStream zipMemoryStream = new MemoryStream(data))
                {
                    using (ZipArchive zipArchive = new ZipArchive(zipMemoryStream, ZipArchiveMode.Read))
                    {
                        ZipArchiveEntry entry = zipArchive.GetEntry("flipflop");
                        using (Stream entryStream = entry.Open())
                        {
                            byte[] result = new byte[entry.Length];
                            var offset = 0;
                            var globalOffset = 0;
                            do
                            {
                                offset = await entryStream.ReadAsync(result, globalOffset, result.Length - globalOffset);
                                globalOffset += offset;
                            }
                            while (offset > 0);
                            return encoding.GetString(result, 0, result.Length);
                        }
                    }
                }
            });
    }
}

Two thing worth noting here:

• The usage of IAsyncOperation to make your component compliant with WinRT asynchronous calls (it will generate a promise for JavaScript)
• The usage of ZipArchive to create in memory compressed streams

The beauty appears when it comes to use this code from JavaScript:

=> Reading compressed data:

 Windows.Storage.FileIO.readBufferAsync(file).then(function (buffer) {
     if (buffer) {
         var reader = new Windows.Storage.Streams.DataReader.fromBuffer(buffer);
         var data = new Array(buffer.length);
         reader.readBytes(data);
         Utilities.Compression.decompress(data).then(function (decompressedData) {
        var mvar myData = JSON.parse(decompressedData);

         });
     }
 });

=> Writing compressed data:

var picker = new Windows.Storage.Pickers.FileSavePicker();
picker.fileTypeChoices.insert("Toto files", [".toto"]);
picker.pickSaveFileAsync().then(function (file) {
    if (file != null) {
        var data = JSON.stringify(myData);

        Utilities.Compression.compress(data).then(function (compressedData) {
            var uint8 = new Uint8Array(compressedData);
            Windows.Storage.FileIO.writeBytesAsync(file, uint8).then(function () {
                // Notify success
             });
        });
    }
});

Isn’t it beautiful when .NET and JavaScript work together?

One issue I have with UrzaGatherer is related to the search. As you may know, I have a huge list of cards saved locally and my application allows the user to search specific cards using a part of their name. But browsing more than 20 000 cards using a text search can be really expensive mainly on low-end hardware.

Currently the code looks like something like that:

cards = UrzaGatherer.CardsList.createFiltered(queryFunction)

 

I use the WinJS.Binding.List object to create a filtered view using my search pattern.

The filter function uses a simple indexOf function:

var queryFunction = function (card) {
    if (card.name.indexOf(textSearch) !== -1) {
        return true;
    }

    return false;
};

 

But obviously, it can take ages to perform a search using this solution. The complexity is almost O(n * m) which can be simplified to O(n²) (where n is the number of cards and m the average length of cards name).

So the question is: How can I optimize my search?

Building a full text search tree

One solution can be found with a search tree. This kind of structure allows you to perform a search with a complexity of O(n) where n is the average length of a card’s name.

You have to build the tree by feeding it with the strings you want to search. For each string, the tree will generate a branch with all the characters then a branch with n-1 character (starting after the first one) and so on.

For example, if we use the “urza” string the tree will look like that:

 

The leafs contain the id of the associated card.

If I add a new string like “Pizza” to my previous tree the resulting tree is:

 

Please note that some leafs can contain many cards (like for “ZA” and “A”)

The related code is pretty simple:

var root;

var processString = function(string, offset, node, object) {
    if (!string || offset == string.length) {
        return;
    }

    var currentNode = node;

    for (var index = offset; index < string.length; index++) {
        var c = string[index];

        if (currentNode === undefined) {
            currentNode = {};
        }

        currentNode = currentNode;
    }

    if (currentNode.ref == undefined)
        currentNode.ref = [];

    if (currentNode.ref.indexOf(object.id) == -1) {
        currentNode.ref.push(object.id);
    }

    processString(string, offset + 1, root, object);
};

var addString = function (string, object) {
    if (!string)
        return;

    if (!root)
        root = { };

    processString(string.toLowerCase(), 0, root, object);
};

Searching using the full text search tree

The search function is then a simple tree traversal:

var concatArray = function (source, data) {
    for (var index = 0; index < data.length; index++) {
        source[data[index]] = {};
    }
};

var gatherResults = function (node, results) {
    for (var prop in node) {
        if (prop == "ref")
            concatArray(results, node[prop]);
        else
            gatherResults(node[prop], results);
    }
};

var searchString = function (string) {
    var currentNode = root;

    for (var index = 0; index < string.length; index++) {
        var c = string[index];

        if (currentNode === undefined)
            return {};

        currentNode = currentNode;
    }

    var results = {};
    gatherResults(currentNode, results);

    return results;
};

When the searched string is completely found, the algorithm gathers all the children leafs to produce the final result.

You can also persist your tree easily with JSON:

var persistIndex = function(filename) {
    var data = JSON.stringify(root);
    return Windows.Storage.ApplicationData.current.localFolder.createFileAsync(filename, 
Windows.Storage.CreationCollisionOption.replaceExisting)
        .then(function (file) {
        return Windows.Storage.FileIO.writeTextAsync(file, data);
    });
};

var resetIndex = function() {
    root = undefined;
};

var loadIndex = function(filename) {
    return Windows.ApplicationModel.Package.current.installedLocation.getFolderAsync("data")
        .then(function(localFolder) {
        return localFolder.getFileAsync(filename).then(function(file) {
            return Windows.Storage.FileIO.readTextAsync(file).then(function(data) {
                if (data) {
                    root = JSON.parse(data);
                }
            });
        });
    });
};

For instance, the complete index for my 20 000 cards weights 6 Mb.

 

Results

Blazing fast !! Nothing more . Using my search tree I can search through the entire collection in less than 300ms where before it can take up to 20s for the same result.

But beware: this optimization is costly in terms of memory consumption (due to the index).

At the end of the day, it is a good tool for you if you want to search your data without using a backend server (or when you are offline).

Parallax background is a really simple but extremely cool feature you can add to your application. You just need a div containing your background picture and a listview on top of it:

<div id="backgroundHolder">
</div>
<div class="blocksList" id="blocksList" data-win-control="WinJS.UI.ListView">
</div>

Then you have to use this CSS to display background picture larger than the current window:

#backgroundHolder {
    background-size: 120% 100%;
    height: 100%;
    width: 100%;

}

Next you have to add a cool picture as a background picture of your div.

Then with the little following function you can handle the scroll event of your list and move the position of your background:

 var attachParallax = function (listQuery) {
     var listViewDOM = document.querySelector(listQuery);
     var listView = listViewDOM.winControl;
     var backGroundHolder = document.querySelector("#backgroundHolder");
     document.querySelector(listQuery + " .win-viewport").addEventListener("scroll", function (e) {
         setImmediate(function() {
             backGroundHolder.style["background-position-x"] = 
              -((listViewDOM.scrollLeft / listViewDOM.scrollWidth) * 
                 listViewDOM.clientWidth * 0.2) + "px";
         });
     });
 };

Considering that the background image is 20% larger than the main window, the idea of this function is to get a scaling ratio based on the scroll bar position. The result is used to move the left position of the background (from 0 to 20% of the window’s width).

But we have an issue with the smoothness of the translation. To resolve this problem, you can use the wonderful power of CSS3 transitions

#backgroundHolder {
    background-size: 120% 100%;
    height: 100%;
    width: 100%;
    transition-property: background-position-x;
    transition-timing-function: ease;
    transition-duration: 0.8s;
}

So thanks to these CSS rules, the position of the background will change smoothly instead of jumping from a position to the next one!

Obviously the next version of UrzaGatherer will include this feature!

At the begining:

Then after you scroll a bit, you can see that the background (1) only moved a few pixels where the foreground (2) moved almost the entire screen:

Performance is always an important point in any project. And you must be well equipped to find potential issues in your code in order to provide the best experience for your user regardless of the device he uses.

In my case, I had an issue with UrzaGatherer on low-end hardware: it took ages to open and display the first screen which is mainly a listview presenting the list of expansions:

On my computer (which is what we can call a high-end computer ), I had no issue because the raw power of my system compensates the performance issue.

This is the first lesson of this article: ALWAYS TEST YOU APPLICATION ON LOW-END HARDWARE! Windows 8 (and Windows RT) is able to run on so many hardware that you cannot afford to not test on low-end devices.

When I found this issue, I decided to use a great tool of Visual Studio 2012: the integrated performance analysis tool which is available on the Debug (or Analyze) menu:

 

Using the Visual Studio performance analysis tool

To use this tool, you just have to launch it by using [Start Performance Analysis] if you want to integrate the start phase of your application in your analysis or by using the [Start Performance Analysis Paused] if you want to analyze a specific part of your application.

In my case, I needed to analyze why it took ages to start so I used the first menu.

The following screen is then displayed and your application is launched:

At any moment you can stop or pause the analysis.

And when the analysis is over, you can access the profiling report:

 

Please note the following to understand this report:

• Inclusive time defines the time spent inside the function including children functions
• Exclusive time defines the time spent inside the function WITHOUT children functions

 

Finding the problem

Using this report, you should be able to find some bottlenecks.

One obvious way to use this report is to analyze the hot path which shows you the main consuming path code.

In my case, we can see that something related to the UI (processElement —> marginLeft) is displayed. I you click on a line, you can see the associated code:

For this example, there is no code because the marginLeft property is inside the rendering engine (IE 10) and not inside the WinJS library. But if you click on calling functions, you will be able to see calling code:

Analyzing this part of code, I can see that it is related to WinJS UI code so for now, I will leave it unchanged

Back to the main report screen, we can also see that JSON.parse if the function with the most individual work associated (13,47% of the total time was spent inside this function). This is normal as I load a 20Mb json file which contains more than 20 000 cards information. I want to have all my cards in memory to be able to search through them quickly so I decided to also leave it unchanged.

In my quest to reduce the loading time of my application I then went to the functions page:

I found that a specific function inside ui.js was called more than 379 000 times! By clicking on the line, I discovered this code:

I added a breakpoint on it to understand that the listview used these function to determine the occupancyMap which is used to populate and draw the listview.

But 379 000 calls is a bit too much for only 40 entries in my screen:

And then I realized my error!! In order to be able to display the random card AND the expansions tiles, I decided to use a really small cell size for the grid (The items must have a size proportional with the cell size so with a really small size I was able to draw any virtual size):

ui.setOptions(listView, {
    oniteminvoked: this.itemInvoked,
    itemTemplate: this.itemRenderer,
    groupHeaderTemplate: this.headerRenderer,
    layout: new ui.GridLayout({
        groupHeaderPosition: "top",
        groupInfo: function () {
            return {
                enableCellSpanning: true,
                cellWidth: 28,
                cellHeight: 13
            };
        }
    })
});

But by doing that, I’m forcing the listview to go through a too important count of cells to build its layout!!

 

Fixing the problem

Fixing the problem is obvious. I just need to use bigger cells (so there will be less cells to go through):

ui.setOptions(listView, {
    oniteminvoked: this.itemInvoked,
    itemTemplate: this.itemRenderer,
    groupHeaderTemplate: this.headerRenderer,
    layout: new ui.GridLayout({
        groupHeaderPosition: "top",
        groupInfo: function () {
            return {
                enableCellSpanning: true,
                cellWidth: 280,
                cellHeight: 130
            };
        }
    })
});

I also need to update my CSS to accommodate the new layout and voila!

The new performance report is now the following:

The UI code is now called only 6 867 times which is clearly more logical.

With this optimization, UrzaGatherer launches twice as fast!

Another small thing…

Another thing really simple to fix is related to Math.floor which is called more than 130 000 times. I used it with the code for cropping cards to generate live tiles:

var pixels = imageData.data;
for (var index = 0; index < pixels.length; index += 4) {
    var r = pixels[index];
    var g = pixels[index + 1];
    var b = pixels[index + 2];

    var luminance = Math.floor(r * 0.3 + g * 0.59 + b * 0.11);

    luminance += (255 - luminance) * amplification;

    if (luminance > 255)
        luminance = 255;

    pixels[index] = luminance;
    pixels[index + 1] = luminance;
    pixels[index + 2] = luminance;
}

To fix this issue, just use a bitwise operator with 0 (old developer trick ^^):

var pixels = imageData.data;
for (var index = 0; index < pixels.length; index += 4) {
    var r = pixels[index];
    var g = pixels[index + 1];
    var b = pixels[index + 2];

    var luminance = (r * 0.3 + g * 0.59 + b * 0.11) >> 0;

    luminance += (255 - luminance) * amplification;

    if (luminance > 255)
        luminance = 255;

    pixels[index] = luminance;
    pixels[index + 1] = luminance;
    pixels[index + 2] = luminance;
}

And the result:

Far better no?

Obviously many other things can be optimized but I just wanted to write an article and not a book