Home       About Me       Contact       Downloads       Part 81    Part 83   

Part 82: Platforms

May 21, 2013

At the end of the last part, I listed some changes that needed make everything work under WebGL/Emscripten without application code changes. I've now done most of that.

Images

I got the LibPNG library integrated into my code and used PNG format for all my transparent images, replacing the JPG+GIF pair of images I had been using. This was more of a nuisance than it should have been. For the block definitions in Crafty and McrView, I was relying on the presence of a second image to indicate whether the texture was transparent. I had to change some of that code to do this right.

Unfortunately, I still have some problem loading RGBA images. They are fine in some cases, like the glass and water patterns in the demos. For the planet texture in SeaOfMemes, I'm still out of luck.

I traced through a lot of the Emscripten support code, and it's kind of grotesque what's being done to load a texture. We have a large byte array containing the virtual file system. The PNG file is extracted from that and used to create a Javascript Image object.

We'd like to just pull the RGBA from the Image object, but there's no method for that. Instead, it gets painted on a Canvas, then the bytes are extracted from the canvas. That's where I'm losing my independent alpha channel (perhaps it's being premultiplied?)

But it turns out that the situation isn't even that simple. Emscripten saves the canvas created from rendering the image, but then renders that to a second canvas before extracting the bytes. To preserve C semantics, it has to then copy these extracted bytes into their HEAP object so that C code can read the data. Then when the C code calls glTexImage2D to create a texture, the bytes are extracted from the HEAP object and sent off to WebGL.

This is all pretty slow, and apparently damages the image alpha channel, but the only way around it is to change their framework. I could also completely avoid their OpenGL code by building my own Javascript OpenGL framework and declaring it as C code. Then the translated Emscripten code would call my framework directly and I could handle images as I like.

Overlay Text

In the last part, I had taken out my GUI code, since I couldn't draw any overlay text or graphics in the demos. That code relies on FreeType to render text and my own simple graphics primitives for things like lines and filled rectangles.

The UI doesn't do much of anything in the demos, but I wanted a long-term solution. To keep the code consistent across platforms, I had to render text and graphics, and have some version of my GUI. So I tried compiling FreeType and all the rest of the code with Emscripten. To my surprise, it works.

The big problem with it is speed and size. Rendering each character in Emscripten generated code is very slow, and with all the FreeType and GUI code in the build, the size has increased considerably. To top it off, rendering my own text with FreeType requires including some font files in the demos, and they are large. The bottom line is that the minimum size demo (TestCube) is over five megabytes to download.

After I tried the GUI code on Android, I realized I have another problem. I can't open the on-screen keyboard from Javascript. The browser doesn't realize my GUI input fields are text fields, since they are just graphics drawn with WebGL.

Googling, I see there's some way to call into the Android library from Javascript, so perhaps I can solve this problem. A similar search on iOS says there's no way to do it, but since iOS doesn't support WebGL, it doesn't matter.

Threads

I stubbed out all the threading, lock and event code. A simple #ifdef EMSCRIPTEN sets the threadCount option to zero. I've been using that path to debug my multithreaded code. It just executes the thread procedure between frames of rendering, without creating any threads.

Audio

Audio is currently stubbed out. There is 3D audio support under Chrome, but apparently the whole spec is still under development, so I haven't bothered with it.

Different Browsers

I've been testing the code under Chrome, since that's available on all platforms I'm interested in. I've already run into differences between Chrome and Firefox (capturing the pointer and audio), but it's been tolerable.

A reader asked if the demos could run under Opera. I started with the handwritten Javascript demos in Part 77, since those are easier to debug. There were two missing functions -- performance.now() and requestPointerLock. Once I coded around these, Opera ran the demo, but not well:

Opera running TestCube

Opera is doing something horrible to the textures. My first thought was that RGB were reversed, but that's not it. The "X-" face is supposed to be yellow. In any case, that doesn't explain all the blue holes in the moon. That texture has been damaged somehow.

I would debug this, but note the frame rate at top left. It's getting only 32fps on a simple rotating cube! That's just useless and not worth bothering with.

With these Javascript fixes, the Part 77 demos run under Safari. Unfortunately, the Emscripten-generated versions don't. The code is loading its virtual file system with all my textures and shaders. There's a note on their site that synchronous file loading is not allowed in some browsers. So they do it asynchronously and poll until they see the data arrive. But under Safari (and Opera), the data never does arrive and so the demos hang.

To debug this, I'm going to have to get under the covers of their generated code, which I don't want to do. This problem has already been reported as a bug on the Emscripten site, so perhaps it will get fixed soon.

Android

Two of the Emscripten demos ran under Android Chrome, but two did not. After installing the debugging tools, you can use the debugger of a Chrome instance running on the PC to debug a copy running on Android. That let me see the console messages from WebGL, which told me one of the shaders was failing to link.

The problem turned out to be the precision declaration. In my first example WebGL shader, I had seen the line "precision mediump float;" in the fragment shader. Either it wasn't in the vertex shader, or I missed it. In my demo code, I did not add this to the vertex shaders.

This becomes a problem when you pass a float variable from vertex to fragment shader, as in

varying float fogSlope;

In this case, in the vertex shader, it gets whatever default precision the platform supports, and in the fragment shader, it gets mediump, and the link fails. Oddly, passing a complete vec3, which is after all made of floats, does not trigger this error. And of course it did not fail under WebGL on the PC. The solution was to add the precision statement to all the vertex shaders as well. At that point, all the demos ran under Android.

But of course, that's not the end of the story. The Landscape demo looks like this:

Android Chrome running Landscape

There are cracks in the landscape, which might be where I do multiple passes to avoid using up all the z buffer resolution. The fog is also far too heavy, completely obscuring the water. This pretty much has to be a bug in the shaders or setup of the OpenGL draw calls, but I can't find it and there are no error messages. This works just fine on the PC.

I can also run the SeaOfMemes demo, with the planet and moon, but it's very odd. The moon constantly flickers in and out of the scene. I thought perhaps the frame rate had dropped to one frame per second or something, but the code is reporting 46 fps and the planet rotation seems smooth.

Debugging these Emscripten demos is a huge pain. I had to just put in print statements, which means several iterations just to find the area where the problem is. When the code fails, Emscripten just burps out some terse "Exception at 234234234" kind of message with no details. It's frustrating and slow work.

Building My House On Sand

I started this project as a Windows/DirectX program. Readers encouraged me to switch to OpenGL, since it runs "everywhere". I was reluctant since my last experience (in 2000) with OpenGL was a disaster. It has been much better this time around, but still a bit quirky.

On some of my machines, the drivers are so terrible that the performance is 1/3 as fast as similar Windows code. Plus I keep running into odd bugs. For example, on one machine, the DontHitMe demo gives this result:

Missing graphics, but only sometimes

As you can see, there's one strip of triangles missing on each of the three hull shapes of the ship. There's nothing wrong with this code. The same program runs perfectly on most of my machines. The same logic translated into Javascript never does this. I have no idea what's going on.

I also have a problem with my skybox. Sometimes there are cracks at the edges of the box textures:

Excuse me while I crack the sky!

This happens with the Javascript demo (in both Chrome and Firefox) and the C++ code. Oddly, it only happens under Linux. Running the Chrome browser under Windows on the same machine has no cracks, and neither does the C++ code. Even more oddly, running the Windows version of the demo on Linux under Wine has no cracks. And neither does the Emscripten-converted version of the code running in the browser!

As mentioned above, WebGL doesn't run exactly the same under Android Chrome as it does on the desktop versions of Chrome. And code that runs in Chrome won't run in Firefox or Opera or Safari.

The performance of the demos is really poor under Android Chrome. It's not the lack of graphics performance. The SeaOfMemes demo, with its complicated planet shader, still runs at 45 fps. The problem is the Javascript interpreter. The Emscripten code runs much slower than the hand-coded Javascript. I assume that's just due to a less tuned Javascript on that platform.

What To Do?

I started playing with WebGL in Part 77 to avoid these platform problems, so I'm not very happy with this situation. I don't really know what to do going forwards.

You could argue that no program is ever going to run on 100% of machines. The problem is that the program doesn't just fail. A nice error message saying "I can't run that" would be fine. Instead, it does all these weird things. I'm going to get error reports from users and not know if its the code, the browser or the drivers. I'm not going to be able to reproduce any of the bugs unless they also appear on one of my machines.

I find that kind of infuriating. I spend a lot of time on the details in my code and really don't like it when the end result is some brain-damaged mess that people can't run, or something like that Opera screenshot that looks like hell.

I'm playing with Android this week and considering my options. I'll let you know what I decide.

Home       About Me       Contact       Downloads       Part 81    Part 83   

blog comments powered by Disqus