Dev►Collecto

Migrating Collecto to PWA

Collecto is one of my two “pet project” apps (With UrzaGatherer).

I recently decided to move it from UWP to PWA because I wanted to use it on mobile devices as well (not just desktop as it is right now). My goal was also to develop once but use it everywhere,

You can find the current version here: https://www.collecto.app

And here is a quick demo reel of the app: https://youtu.be/BUhDcvvfts4

The web can definitely do as good as native apps

I was a bit worried at the beginning because I wanted to get as close as possible to the look and feel of my native app. It turned out that it was not a real problem as the Web is clearly mature now.

If you like gory tech details, here is a list of web features / tools I used and why:

  • React: The main reason I picked React among the gazillions other framework was clearly because I wanted to learn it 🙂
  • TypeScript: Creating a complete app without a strong typed language could prove to be really challenging and I was not really up to that challenge
  • Service workers: They are the key of the PWA ecosystem. They let you hook the network calls and thus they provide a way to work online / offline by letting you cache your assets locally
  • IndexedDB / Local storage: IDB and local storage are used to store user preferences as well as the user’s collection in case there is a network issue
  • OneDrive SDK: Collecto does not store any user data on a server but instead uses the user OneDrive to store his collection
  • History API: I used the JavasScript history API to control the navigation inside the app
  • CSS Grid layout and CSS Flex: The UI is made of a composition of both grids and flex boxes
  • CSS animations and transitions: In order to bring the same feeling of smoothness that I had on my native app, I tried to use transitions and animations as much as I can.
  • CSS filter: Well because it is just fun (I used to to blur the app when a wait ring is displayed for instance)
  • Webpack: I use webpack to bundle all my JavaScript (compiled from TypeScript), CSS, fonts and all other resources into a single .js file usable on any modern browser.
  • VSCode: What else could it be?

Deploying your PWA

Once ready, your PWA can be directly used as is by just sharing its URL. It will work on all mordern desktop and mobiles browsers (Do not forget to test on all browsers if you see what I mean). On Android devices you can even add them to your home screen where they will be indistinguishable from a native app.

If you also want to enlist your app in various app stores (to get more visibility for instance), I would recommend to use www.pwabuilder.com which will prepare for you all the packages you need to submit an app on Windows Store, iOS App Store and Google Play Store.

Dev►Babylon.js

My fat objects are slower than my thin ones

It’s been a long time since I blogged here but we recently stumbled upon a so interesting performance issue that it motivated me enough to share it with the entire world (well the entire world of people aware of this blog).

The virtue of always testing (even the obvious)

We are about to ship Babylon.js v3.3 and while running our performance validation tests we found that a specific demo was running slower than in v3.2 whereas it should be faster (at least in our minds because we specifically optimized some code used by the demo).

So the scene we tested is a simple scene made of 8000 identical spheres: https://www.babylonjs-playground.com/debug.html#QQGCL6#5

This scene is mostly designed to make sure that the time spent traversing scene graph is same (or smaller) than with previous versions.

Funnily (or sadly if you ask my wife who did not see me for a couple of days), this scene runs HALF the speed we expected even though the code used was supposed to be faster (in the latest Chrome release).

You can try it by yourself:

The art of optimization

When something like that happens, the first thing to do is to run both versions side by side, using your browser profiler to see what is going on.

We quickly found that the culprit was the _evaluateActiveMesh function which is here to determine which meshes are in the camera frustum (in other words, which meshes can be seen by the camera).

The code of the function did not evolve a lot and the changes should have increased optimization.

Should have.

When you face this kind of problem, the best way is to work step by step. So we decided to restore the function to its original version (the v3.2 version). And guess what? Nothing changed. The function was STILL slower.

Understanding what is going on under the hood

So same code is now behaving differently. Cool.

In this case, the only explanation is that the browser cannot optimize the new version like it was able to previously. And this is not related to the code itself because, well, it is now the same as it was.

So, the only other reason is that the supporting class (the Mesh class) is less “optimizable”.

And as a proud member of the “I Run Only In World of Warcraft” association, I know that if you want to run fast, it is better to be thin than fat. So we slowly removed properties from the class itself until we reached a point where, suddenly, performance was the same.

The fix is not obvious as we will need to be able to add properties to our Mesh class. But beyond a certain point, going through an array of fat objects is just slower.

We are now in a position where adding this line…

public heyWantToSlowDown = true;

…will reduce the test scene performance from 60 fps to 40 fps. Just removing the line will restore performance.

So now what?

I contacted Chrome’s team because this is something we will need their help with.

I’ll keep this post updated as soon as I get more info.

[Edit – 9/7/2018 at 4:00]: The Chrome dev team replied to my cries on Twitter and kindly offer their help through a Chromium ticket. You can track it here: https://bugs.chromium.org/p/chromium/issues/detail?id=881977

Dev►Babylon.js

Dev log #1: Morph targets

This is the first post of a series about what I’m working on for Babylon.js.

Morph targets were on my todo list since ages (since Babylon.js 1.4 if I remember correctly). I decided recently to do it because I wanted to be sure we can cover all aspects of gltf 2.0

Morph targets are used to deform meshes. A morph target can be built from a mesh with the EXACT same amount of vertices as the original mesh.

Morph targets are used by the GPU to create the final geometry by applying the following formula:

final mesh = original mesh + sum((morph targets – original mesh) * morph targets influences)

__ 

You can then move from this:

Morph Target Before

to this:

Morph Target After

You can find a live example right here: https://www.babylonjs-playground.com/#HPV2TZ#4

 

Thanks to our smart shader system, the implementation was not really hard. I first introduced two shader includes:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/morphTargetsVertexDeclaration.fx

https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/ShadersInclude/morphTargetsVertex.fx

The integration in the Standard and PRB materials was then straightforward:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Shaders/default.vertex.fx#L107

Basically based on number of target, the vertex shader will apply the initial formula:

position = initialPosition + sum((targetPositionX – initialPosition) * influenceX)

The same operation is applied to normal if they are present.

 

On JavaScript (well TypeScript actually) side, I added a new function to add vertex attributes when a mesh is used with a morph target:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Materials/babylon.materialHelper.ts#L245

Ultimately, when you bind morph targets to a mesh, the geometry of the mesh is updated to add a vertex buffer per target:

https://github.com/BabylonJS/Babylon.js/blob/master/src/Mesh/babylon.mesh.ts#L2003

By updating the geometry we ensure that the vertex buffers will be captured within VAO (Vertex array objects) which were introduced recently in Babylon.js.

 

So we have here:

  • A hardware accelerated morphing of geometries
  • Support of VAO
  • Works on WebGL 1 and 2
  • Can be used on mobile devices
  • Can used from 1 to x morph targets (where x is limited by maxVertexAttribs which is usually 16 on recent browsers. So with position, normal and uv, this leaves 13 free attributes so either 13 targets with position only or 6 targets with position and normal)
Dev►Babylon.js

Babylon.js v2.5 and what's next?

Last week we released Babylon.js v2.5. I do not want to catalog here all features we stuffed this version with but instead I would like to underline some really cool ones.

I will also present here non core features that are worth a try.

But before going to the specifics of this article I would like to celebrate the use of Babylon.js as rendering engine in www.remix3d.com.

Babylon.js was also use by Neo Pangea and National Geographics in two multimedia experiences:

And now let’s talk about great stuff that came with v2.5.

Highlight your world

Sebastien Vandenberghe (one of our core developer) worked on a system to allow developers to highlight specific meshes in your scene: The HighlightLayer.

This tool can be used to produced smooth and nice glow effects around your objects:

You can try it live here: https://www.babylonjs.com/Demos/Highlights/

As usual with Babylon.js, the required code is kept to the minimum amount of code:

More information in our documentation: https://doc.babylonjs.com/tutorials/Highlight_Layer

Rendering special effects with the StandardRenderingPipeline

Julien Moreau-Mathis (another core contributor) created the StandardRenderingPipeline which is a set of postprocess effects that can be used to simulate real world effects like lens flare, depth of field or advanced illumination effects:

You can try it live here: https://www.babylonjs.com/Demos/StandardRenderingPipeline/

The StandardRenderingPipeline is a monster with tons of advanced shaders that work together. But for a developer, you only have to instantiate it and play with properties:

More information in our documentation: https://doc.babylonjs.com/tutorials/Using_Standard_Rendering_Pipeline

Build your own Babylon.js

Simplicity is the foundation of Babylon.js and we struggle everyday to keep it our main objective. This is why the framework tries to provide all core services in one single file.

But to avoid bloating it with tons of optional features we also provide additional services available through various .js files.

This is the case for instance for additional materials or post processes.

In order to make them more discoverable, we created the Babylon.js Version Builder page: https://www.babylonjs.com/versionBuilder/

On this page you can find all optional module and get a merged file that you can then use in your own code.

Introducing the Babylon.js editor

Julien Moreau-Mathis also worked on a complete WYSIWYG editor for Babylon.js: https://editor.babylonjs.com/

To illustrate the power you can harness with this editor he created a really impressive demo: https://www.babylonjs.com/demos/ruins/ (please note that Julien is also a talented musician)

To learn more about the editor, you can read the article written by Julien on Medium.

The DebugLayer is dead. Long live the Inspector

Julian Chenard (Once again a core contributor) spent a lot of time revamping the good old DebugLayer.

The DebugLayer is an internal HTML layer that you can activate on any scene using scene.debugLayer.show().

The display UI can help you debug a scene but giving you control over engine features. You can then turn on/off specific channels, display bounding boxes and much more.

The only problem was that the UI was not really responsive nor really modern (ok let’s admit it was awful).

The new version is much cleaner and provide far more features. You can now:

  • get and control the active camera
  • get the list of shaders and materials
  • get the list of lights
  • Textures can be displayed
  • Properties of type text or number can be edited
  • The debugLayer can be moved to a popup window

Please note that this feature is still under heavy development so expect some bugs.

Using Visual Studio Code to contribute to Babylon.js

I used to use Visual Studio to develop Babylon.js. But after the release of Visual Studio Code, I was really interested to move our pipeline to this smaller (and faster) IDE.

This is now done thanks to Sebastien who wrote a really convenient gulp scripts to build all the library in a VS Code compatible way.

In a nutshell, after forking and cloning Babylon.js repo, you are 3 minutes await from being able to test and debug.

You can learn more here: https://doc.babylonjs.com/generals/how_to_start

Call to action

As a conclusion, I would like to give you some pointers about Babylon.js:

Dev►UWP

How to improve your XAML controls to make them more accessible

Since UWP Community Toolkit v1.1, we started encouraging developers to follow some simple guidances to make controls more accessible.

As a developer, I did not consider accessibility like I should have. And most of the time it was more because I did not realize how simple it could be to at least provide some basic support.

This is why we added the following chapter to our contribution guidelines for UWP Community Toolkit:

  • Your control must be usable and efficient with keyboard only
  • Do not use custom colors
  • Add AutomationProperties.Name on all controls to define the controls purpose.
  • Use Narrator Dev mode to test the screen reader experience.

Let’s have a deeper look on each of these topics.

Your control must be usable and efficient with keyboard only

This is an easy one to test: Unplug your mouse and try to use your control. If you can use it in an efficient way (i.e. without becoming mad) then you are good to go. 

If not, then you must think about the following options:

  • Use IsTabStop=false to avoid moving the focus to unwanted area.
  • Define correct Tab order using TabIndex
  • Ensure that all focused controls are visible

For instance take a look at the tab order for the HamburgerMenu:

clip_image003

So basically, you should be able to use Tab key to navigate between control’s area (in a logical way like top to down).

Selecting a button or an action with Enter key must also be enabled by default. If you are using stock controls, you should be fine.

But for some really custom controls, do not forget to add it by handling key up/down events.

This is also important for controls with moving parts. For instance, I had to manually add support for Left/Right manipulation on the RangeSelector to allow the user to manipulate the thumbs with keyboard only:

clip_image004

Do not use custom colors

Custom colors can be a problem if the user wants to use high contrast theme. As they are statically defined, they will not update accordingly to the user needs.

So just avoid them and instead rely on Theme colors like this example for the HeaderedTextBlock:

clip_image005

 

You can find the list of theme resources here: https://msdn.microsoft.com/en-us/library/windows/apps/dn518235.aspx

Add AutomationProperties.Name

Another easy one. Just add an AutomationProperties.Name attribute to control which can have the focus. This will be used by the narrator to “read” your UI.

Please use self-explanatory text like “Button to create an order” instead of “Button1” and don’t use the same Name on two different elements unless they have different control types

Example here for one of the ListView used by the HamburgerMenu:

 

clip_image006

Please note that Name is minimum, but there are some other things too that can really help the screen reader: https://msdn.microsoft.com/en-us/library/ff400332(VS.95).aspx

Use Narrator Dev mode

You can start the narrator by pressing [WinKey+Enter].

Then you should ask yourself if the information is sufficient, meaningful and helps the user navigate and understand your control. Close your eyes, only use the keyboard and try to use your control.

If it does not work, then you may have failed one of the previous steps .

Call to action

If you want to help the community creating more accessible controls, we created the following issue where we track the current state of UWP Community Controls:

https://github.com/Microsoft/UWPCommunityToolkit/issues/426

 

Feel free to take one control from the list and apply the recipe I presented here.

Dev►Babylon.js

What's new in Babylon.js 2.4

Woot! What a version again! It seems to me that this is my state of mind for every version but, whaou! we put so much love again in this version.

You can find the complete release notes here but I still would like to highlight a few features (Even if they all deserve some attention).

Canvas2D

Canvas2D (by Loic Baumann aka Nockawa) is a HUGE addition to babylon.js. It is basically an entire 2D accelerated engine available out of the box.

Indeed babylon.js is a full featured 3D engine, but the need to handle 2D content is often encountered when working on games or complex applications.

It is to provide an answer to this demand that we’ve developed the Canvas2D feature.

Canvas2D in a non HTML feature, 100% WebGL based, developed to serve the purpose of drawing, animating and interaction with 2D Content.

Here are some samples demonstrating the raw power of this new feature:

Dev►UWP

Adding smooth transitions when navigating in your UWP app

Early this week, I published a new version of UrzaGatherer which added support to navigation transitions.

You can see how it looks in this video:



 

I like these transitions because they really smooth out the user experience. You feel more comfortable using the app just thanks to these subtle animations.

So let me explain you how you can do the same.

General idea

The overall idea is pretty simple. We will use a split view and its ability to embed a frame (which is the host of the navigation).

Just before navigating, we will render the client area of the split view to a picture.

This picture will be added on top of the frame while the frame is navigating.

Then based on the effect you want to achieve, you can play with the picture.

In my example, I slowly animate the picture opacity from 1 to 0 to make it disappear.

To make it just a little bit more dynamic I also translate the frame itself to give user the impression that the frame is coming from the bottom.

Pretty simple, right? Let’s now see how to implement it.

Implementation

First of all, let’s start with a simple split view:

Capture

This split view contains a picture control and a content control that you need to bind to a frame:

frame = new Frame { Language = ApplicationLanguages.Languages[0] };

SplitContent.Content = frame;

Once connected, you can ask the frame to navigate to a specific page:

frame.Navigate(typeof(Page1));

So the idea is to save the frame content to a picture when navigating. To do so we first need a method to capture frame’s content and save it to the picture control:

RenderTargetBitmap rtb = new RenderTargetBitmap();
async Task StartDissolvingAsync()
{
    try
    {
        await rtb.RenderAsync(SplitContent);
        DissolveImage.Source = rtb;

        DissolveImage.Visibility = Visibility.Visible;
        DissolveImage.Opacity = 1.0;
        AnimateDouble(DissolveImage, "Opacity", 0.0, 200, () =>
        {
            DissolveImage.Visibility = Visibility.Collapsed;
        });

        SplitContentTransform.Y = 25;
        AnimateDouble(SplitContentTransform, "Y", 0, 150);
    }
    catch
    {
        // Ignore error
        DissolveImage.Visibility = Visibility.Collapsed;
    }
}

We are using a RenderTargetBitmap to save the current frame texture. This image is then displayed by the picture control on top of the frame.

Animations are done using storyboards thanks to this method:

public static void AnimateDouble(DependencyObject target, string path, double to, double duration, Action onCompleted = null)
{
    var animation = new DoubleAnimation
    {
        EnableDependentAnimation = true,
        To = to,
        Duration = new Duration(TimeSpan.FromMilliseconds(duration))
    };
    Storyboard.SetTarget(animation, target);
    Storyboard.SetTargetProperty(animation, path);

    var sb = new Storyboard();
    sb.Children.Add(animation);

    if (onCompleted != null)
    {
        sb.Completed += (s, e) =>
        {
            onCompleted();
        };
    }

    sb.Begin();
}

Last step is to actually launch the dissolve effect before navigating:

private async void Page1_Click(object sender, RoutedEventArgs e)
{
    await StartDissolvingAsync();
    frame.Navigate(typeof(Page1));
}

private async void Page2_Click(object sender, RoutedEventArgs e)
{
    await StartDissolvingAsync();
    frame.Navigate(typeof(Page2));
}

That’s it! Now you are ready to create better looking applications! (Please find the Visual studio project here…).

Dev►Babylon.js

Going further with Babylon.js 2.3 advanced features


We’ve just released Babylon.js 2.3 with the biggest set of features we ever shipped


David (the other one) presented on his blog the demo we built with Michel Rousseau: The Sponza demo. This demo can really showcase what can be done on the web today regarding visual quality rendering (Even on low end devices).


Among all the new features we added, the one I’m the most proud about is dynamic point light shadows (DPLS).


To better understand what DPLS are, here is a small video of what they look like:


You can even see it live on babylon.js website: https://babylonjs.com/demos/SponzaDynamicShadows/


During this article I’m going to first explain what points lights are and then how we can achieve having realtime shadows in 3d.


Point lights


Lights in 3d are used to produce the color received by each pixel. This color is then used by materials to determine the final color of every pixel.


They are various kind of lights and you can think about point light like a light defined by a single point. This point emits his energy in every direction (Like a really tiny sun).


Here is a super simple 3d scene with just a plane and a point light:


https://www.babylonjs-playground.com/#J6ZHG#0




We can feel where the light is by looking at its impact on the ground.
Without a light the same scene would look like this:




Lights are a key point in 3D because it is really hard to feel any volume without a light.


Let’s take a simple red box for instance:




Now the same box with a simple light:




As you can see lights help your brain feel the 3D.


Shadows


To get even more realism, you can add shadows to your scene. However shadows in 3D are hard to reproduce because they involve the concept of occlusion.


Indeed, a shadow is just a place where the light is occluded.


This implies the need to be able to check if the light can actually be reached or not for every pixel you want to render.


If you look at this new scene, you will see a sphere and a box with a point light but no shadow:


https://www.babylonjs-playground.com/#J6ZHG#1




By default, shadows are off because they required a lot of GPU power.


Here is the same scene but now with shadows enabled:


https://www.babylonjs-playground.com/#J6ZHG#2




Babylon.js makes this incredibly easy to do by allowing you to create a ShadowGenerator, define who casts shadows and who receives them, and rendering it.


Boom you’re done.


Points lights + shadows


Under the hood, things get a little bit more complicated.


To determine if a pixel is inside a shadow, we need a first pass where we will render the scene from the point of view of the light. This pass will generate the shadow map.


This map will contain the distance between every pixel visible from the light and the light itself.


During the main pass, we will use this shadow map to check if the distance of the current pixel and the light is greater than the one store inside the shadow map. This is done for every pixel. If this is the case, then the pixel is in the shadow.


The problem with point lights is that they emit light in all directions. You cannot simply generate a texture from the point of view of the light because there is no single point of view (i.e. a position and a direction).


This is where the point light shadows enter the game.


To fix this issue, we need to generate 6 textures: one for each direction (up, down, left, right, front and back). This way, the engine can always find the best texture to use depending on where the pixel to render is regarding the light.

For instance, here are the textures generated on every frame for the Sponza demo in my video (we call them cube textures, because, well, they form a cube):


screenshot_2016-1-8_15-10 (2)screenshot_2016-1-8_15-10 (1)screenshot_2016-1-8_15-10screenshot_2016-0-27_8-24 (5)screenshot_2016-0-27_8-24 (4)screenshot_2016-0-27_8-24




Going back to the demo…


Now let’s get back to the demo to see how Michel built it.


First of all, you can go to the url and use the bottom control panel to enable the debug layer. it will allow you to play with engine’s parameters:




Let’s start from scratch by turning lights off:




Because the scene is based on purely dynamic lighting, everything turns black(like in the real world!).


Let’s then turn the lights on again and enable clickable labels to display all the entities living in the scene:




Lights are displayed in yellow and you can click on the labels to turn them on and off (Omin001 and Omni002).


So basically, the scene is built like this:















First light


Second light


Merge


Final rendering with shadows


Based on what we saw before, there are 2 cubemaps generated for this scene.


But because only Omni001 moves, we can generate cubemap for Omni002 at startup and save a lot of bandwidth this way.


Looking at the future with WebGL 2.0


The current version of Babylon.js uses WebGL 1.0 so it requires 6 passes to generate a complete cubemap.


To increase rendering output, we plan to use an extension (which is part of WebGL 2.0) named “WEBGL_draw_buffers”. This extension will allow us to render all faces of a cubemap in one call improving a lot the overall performance.


In the meantime, we are really happy with the current technique which works on all modern browsers and devices which support WebGL 1.0.


During this article we saw why lights and shadows are important in realtime 3D. We also discovered how to add realtime shadows to point lights using cubemaps.


If you liked it you may find these links interesting:

Dev►Vorlon.js

Improve your website accessibility with Vorlon.js and aXe

To celebrate the recent publication of Vorlon.js 0.2.1. I’d like to talk about a feature we recently introduced to the Best Practices plugin: Accessibility Analysis.

If you’re not familiar with Vorlon.js, I suggest you visit these pages first:

· https://vorlonjs.io/#getting-started

· https://blogs.msdn.microsoft.com/eternalcoding/2015/04/29/why-we-made-vorlon-js-and-how-to-use-it-to-debug-your-javascript-remotely/

A quick glance at the Best Practices plugin

Before jumping into the main topic of this article, let me present what the Best Practices plugin does.

You can find it in your Vorlon.js dashboard, here:

Vorlon.js dashboard with Best Practices Tab open

 

This plugin is designed to provide guidance regarding web standards, accessibility, performance and the mobile web.

Internally, it uses a set of rules that you can find here: https://github.com/MicrosoftDX/Vorlonjs/tree/master/Plugins/Vorlon/plugins/webstandards/rules

We’re looking to provide more rules in order to help web developers create the best possible experience.

Once an analysis is done, clicking on a rule with a red cross provides more context about the issue detected:

Vorlon.js Best Practices Audit results

Whenever possible, we’ll suggest a solution (like this example, where we’ve highlighted missing CSS rules).

Analyzing accessibility of a webpage

But the really big news here is we’ve completely rewritten the accessibility analysis.

Providing a decent analysis for accessibility is a gigantic task and no one on the Vorlon.js team felt really confident about developing it. That’s why we decided to team up with Deque Systems in order to embed their accessibility engine (aXe) into our analysis.

In their own words:

axe The Accessibility Engine

“Deque System’s aXe (The Accessibility Engine) open source library is a lightweight (~100 KB), fast, portable JavaScript library that executes automated accessibility testing inside your testing framework or browser of choice”

aXe runs client-side checks against webpages for common accessibility problems, mapping them to accessibility standards such as WCAG 2.0 Level AA or Section 508. A portion of these checks include:

· Missing form labels

· Color contrast issues

· Incorrect use of ARIA

· Disabled user scaling in viewport meta tags

· Missing language attributes

· Focusable elements missing text

The result is pretty impressive. Once the Best Practices plugin has run the complete analysis, you can go to the Accessibility tab and get really precise information about what you may need to change:

aXe Accessibility Audit results with failures in Vorlon.js

aXe provides you with a convenient explanation of the issue. It also pinpoints where the issue is located, and even references the Web Content Accessibility Guidelines 2.0 (WCAG).

So what’s your ultimate goal, then? Get a perfect score like this one (I ran the Best Practices plugin on Deque’s website):

Successful aXe Accessibility Check

Want to learn more about Vorlon.js?

If you want to go deeper and learn more about Vorlon.js, here’s a collection of links that you might find useful:

· Main website

· Why we made vorlon.js and how to use it to debug your JavaScript remotely

· Focus on DOM Explorer

· Debugging your HTML5 on Xbox One / MS Edge with the Xbox Windows Store app and Vorlon.js

· How Vorlon.js helps you improve your web code

· How to create a Vorlon.js plugin

· Vorlon.js 0.1.0 is out

Misc.

Hints for successfully managing an open-source project

Today I want to change things a bit: instead of covering a technical subject, I want to share with you what it means to run an open source project

For more than two years, my friend David Rousset and I have led Babylon.js. We started the project after hearing that IE11 would support WebGL and we wanted to make it easier for people to build 3D scenes and games. For the following two years I spent all of my spare time making Babylon.js a simple and powerful 3D engine for web developers.

The beginning of a fantastic journey

During the first 2 months, it was like a dream: you develop and give life to all the ideas you have in mind. It is pure bliss.

This phase is what I call the preliminaries. Like in a love affair, that is the best period, just before taking it into more serious territory, where complications can happen.

At some point in time, you are ready – or you think you are.

Shipping is important. No seriously. I still see a lot of projects that stay in development for ages. I know this is hard but this is necessary for you to ship your project.

David and I ran into the first challenge quickly: coding the product is not enough; you also need to work on your communication. We wrote and agreed on the articles we wanted to publish about Babylon.js, the forums we wanted to post on, the events we wanted to attend in order to present it, etc.

_This is the first important tip: Developing a framework is just half the job. Letting people know that the framework exists and why they should care about it is the other – much more important – half.___

The feedback from the web development community was huge. One of the reasons was that we worked with a gifted designer (Michel Rousseau). He helped us to not just ship a framework, but also to develop tons of 3D scenes that showcase what people can achieve with our tool.

That’s the second important tip: having a great framework is not enough. You have to provide a lot of examples and concentrate on having some really shiny ones to impress developers enough to take a closer look.

The puppy syndrome

I’ve first heard about this concept at dotjs conference in 2012 from @fat

While you are the happiest developer in the world, something strange is bound to happen. It will be subtle at the beginning but will soon become more tedious. User feedback will become overwhelming and demanding:

  • Some users will start acting strangely, asking for more and more features and you need to be firm or bring up reasons why you aren’t supporting or don’t intend to support these features. Always remember that more features means more work and also more choice for implementers. What is a seen as a great idea for one user can be seen as overhead and annoying to others.
  • People will ask you to fix their code and thereby monopolizing your time (and prohibiting you from being able to get necessary work done on the project)
  • Others will demand you to completely change the framework to address their needs regardless of your vision and what you want to achieve with it.

 

This is the PUPPY SYNDROME! This is when your lovely puppy that you pet every day with love and dedication mutates into a monster you barely can control.

From:

clip_image001[4]

to:

clip_image002[4]

This is the most complicated part of your job. You have to stick with your vision but you also have to bend it a little bit to accommodate the needs of your users. Use your common sense here!

This phase can take quite a while and you need to be good in your communication to get allies in your user base. From these allies you can form a community that brings you to next level.

Leveraging the community

You can have the best product in the universe, if you don’t have users, you have nothing. This is why I keep forcing myself to respect the following 12 rules:

  1. Be prepared to spend a lot of time on NOT developing your product
  2. Be prepared to spend a lot of time on EVANGELIZING your product
  3. Never leave an issue open or without response more than one business day
  4. Never leave a post on your forum without response more than one business day
  5. Go to your forum every day to show users that the forum is alive (and yes at the very beginning this is a tough and tiring job to do. It is important though – nobody writes on an empty wall)
  6. Tweet about what you are doing and working on
  7. Have a public roadmap
  8. Ask users for feedback on your roadmap
  9. Have a uservoice (https://babylonjs.uservoice.com)
    1. Be nice with people. Don’t be arrogant. You need them more than they need you
    2. Invest in a decent documentation system: I know that you hate writing documentation but this is a mandatory piece of your project. We spent a lot of time on https://doc.babylonjs.com to make it nice, easy to use and community oriented (The documentation itself is a Github project where the community can contribute)
    3. Provide an easy way to learn by doing. This point was really key for us. We created www.babylonjs-playground.com to enable web developers to test babylon.js without having to setup anything. Reducing all the friction is important. We are all lazy and if you can remove all the roadblocks then you will get more people trying your framework. And just like jsfiddle, it is also a way to share your code with others and even easily provide a bug repo. We tried to make it really convenient by adding intellisense which is a fantastic tool to learn while coding.

_An aside about having a forum: we started with issues on Github to communicate with users but it turned out to be less convenient than a regular good old forum. And I want to thank Richard Davey (photonstorm) for hosting our forum on HTML5GameDevs.com___

It is all about being attentive to your user’s needs and questions. Users need to be able to trust your product and having good communication channels that are maintained gives them confidence in your product.. The faster you handle requests the more confident your users will become. This is the main building block of how you build the reputation of your framework. You can start with less features than other tools as long as you care about your users.

A community forms

If you manage to support your community, they will offer you a wonderful gift. Someone from the community will start answer question from others in a nice and eloquent form – in some cases even better than you could have done it. I think that when I saw this happening for the very first time, I cried.

If you’re lucky, you will get three kinds of users:

  1. Regular users who use your product and ASK questions
  2. Power users who use your product, ask questions and ANSWER questions
  3. Super users who use your product, ask questions, answer questions and CONTRIBUTE code

 

This is the final stage of your journey. Now users can commit code and do wonderful things without you. For instance, on the latest release of Babylon.js more than 40% of the new features are produced by the community.

You are now the leader of the project who gives the direction, the goals and works on keeping everything coherent and aligned.

I still feel the same pleasure when I receive pull requests from smart people all over the globe, working together to make a better 3D engine.

Conclusion

To summarize, here is what it looks like from the point of view of the project leader (graph generated with my hand©)