March 2015

I often find some piece of nice software and I like to read about "behind the scenes" about it, when a blog is available. Then it makes me slightly sad, when I find out the latest post is from 2 years ago. And yesterday I was like: "Hey, that's MY blog as well!" :)

Then I thought about it. It's quite hard to do a job and then to blog about it. Even I'd like to, but I can't promise I'll do better in the future. Self made publicity steals time from the object of publicity, more time blogging means less time coding.

On the other hand - who says I have to write a War and Piece every time? I have about 20 unfinished posts in stock, so now it might be a right time to slice them into smaller finished ones.

Anyway. What I've been up to with QetriX? Finishing the concept. I did another iteration, quite finished Java version after PHP has been usable. From that point I understood the PHP version is wrong yet again, so now I'm reeingineering and refactoring.

The finished will be even more consistent among platforms. I discarded "index" script as the entry point and introduced "app" and "file". For PHP it's handled by .htaccess, for Java it's handled by web.xml.

The WebApp will contain a HTML generator (basically Components with Converters to HTML) for internal WebView, which fits perfectly into the QetriX idea. I love this!

Finally I settled to separate data from users and data from geodata. Coordinates and user info in data (= mostly in Particles) will be still supported, but not recommended.

I finished my tracker app for iOS, which works well and replaced Moves for good. I was really crossed that Moves is not working offline and unfortunately the biggest test for the tracker so far, my short trip to Dubai, failed, because I screwed up a final update and the app crashed on startup. But never mind.

Last month I exported my first worksheed from my time tracker as well, works like a charm. I'm thinking of merging it together with the tracker, so it will know when I arrived to work and when I left.

I would like to write "and that's about it" now, but it isn't. But I'll get into it in some of the next posts.

All about performance

My key request for the new system was good performance, no matter how achieved. I ended up with a single data table and a lot of cache around it. In Particle Database I have a search cache and coordinate cache in MySQL and each entity in JSON file. Entity contains all own attributes and relations, plus top 5 foreign relations. Each foreign relation type to each entity type has own JSON file cache.

MySQL

Indexing of MySQL tables is crucial, too little is bad and too much is bad as well. Best practice is to work tightly with DESCRIBE command and index columns used in JOIN and WHERE clauses.

There are several situations, where indexes can't be used. If you perform LIKE wildcard search, only col LIKE 'abc%' or col LIKE 'a%c%' use indexes, but col LIKE '%abc%' and col LIKE %abc' and col LIKE col2 don't - because of the way B-Tree index is designed. Or if you have to do a string operation before comparsion, e.g. UPPER(col) LIKE 'ABC', all rows has to be loaded. In such case consider adding a new column, with cached upper-case values.

PHP

I always checked what approach in PHP would be the fastest, especially for functionality in iteration (loop). I used simple script with a timer:

function test1()
{
 $t = microtime();
 while ($i < 1000) {
  // HERE TESTED CODE
  ++$i;
 }
 return microtime() - $t;
}

This way I tried the best approach for zero padding or type casting from string to number.

Server

Activate GZIP compression.

Encourage browser caching for static resources, using Expires header, Last-Modified date or ETags in the HTTP header.

Use different URL root for your resources, because browsers allows 4 to 8 simultaneous requests per domain, so using different domain will increase. And it will be cookie free (cookies are sent even for CSS or images). There are public CDN resources for widely used stuff, like jQuery or Bootstrap.

Client side

I also deeply enhanced client side for speed. First thing was to minimize external files - CSS and JS. Minimized code strips out all unnecessary stuff, like whitespaces or comments. Deep minification also rewrites variable and method names to contain as few characters as possible. Every byte counts.

Minimized file can be even smaller using GZip compression, widely supported by browsers. This way you can 58 kB file compress to 12 kB. But always keep the original file and never edit the minified one. The only downside is debug on production server, as you can`t get a proper line number, where problem occured, becuse everything may be on a single line.

I know JS libraries/frameworks, like jQuery or Dojo, are really handy and I like to use them, but only if I can't do it with pure ("Vanilla") JS. And it's valid for CSS boilerplate frameworks as well (Bootstrap, Foundation). Because it's universal, it contains a lot of code you never use, but browser parses it on every page load. If you want to use it, remove all unnecessary stuff, if you can (there are tools for that). jQuery is 85 kB of code just to make another code works. Using JS lib only for element addressing is pure blasphemy and I don't even want to comment using more than one JS library on single page.

Next step is reducing HTTP request count. I merged all JS into single file, the same with CSS. Then I created so calloed "CSS sprite" - all graphis (pictures), where possible, migrate into single PNG file ("spritesheet") and instead whole CSS background, specify just backgroud-position. You have to do it cleverly, because of mixing repeating (backgrounds) and non-repeating images (icons). I can't really recommend data URIs, because it can't be cached and therefore you will download it over and over again. The only time it was extremely hand was when I got the picture already as base64 from REST web service.

One of the most difficult steps was to minimize HTML code. Some browsers (with blue "E" in logo ;-) in some previous (but still used) versions can't handle innerHTML manipulations without another DIV, so every time I want to use element.innerHTML = someText, I have to specifiy extra inner DIV. There are certain situations, when the HTML code is really bloated and I can't do anything about it. If you write the code in HTML5, you can at least get rid of the ending slash on non-pair HTML tags, like INPUT or IMG.

It's also good to check for invalid stuff over time. You may not use the widget or page layout any more, but linked scripts and CSS may still be out there. Also check for unused JS functions/methods and unused CSS classes.

Webdesign

Responsive webdesign is neat, but you have to transfer all the code thru slow mobile connections (LTE or even 3G is still a dream in a lot of countries). Much better is to create lite mobile version. Don't immitate specific GUI (like iOS), it will look ridiculous on other platforms or after a platform redesign!

Always use proper image format - JPEG (JPG) for photos and PNG for everything else. Because JPEG is lossy format, you can tweak quality (from 0 being crappiest but smallest to 100 being best, but large) to achieve best ratio between quality and file size. You may need just to re-save JPEG photos, even it slightly impacts quality - but often greatly lowers file size. For both formats there are online and downloadable optimization tools, that help lower their file size.

Many effects, achieved by images in the past, are nowadays possible just by CSS effects. They are significantly smaller, of course, but increases CPU time required for their rendering. A lot of shadows, gradients and transitions may kill performance on (low end = mostly used) mobile devices.

Consider resizing large photos, especially for mobile websites. On the other hand, because of different dpi, you may need double the size of graphics elements based on image to look good or hi-res displays. The best option is to use SVG, but like CSS, it may also affect performance if not used wisely.

You may not load the content (photo, video...), if it's not yet visible. This technique is called "lazy loading". You can detect it's vertical position and compare it to the scrollbar position. When they meet, you can trigger download.

Try not to use webfonts on mobile version. The only sensible use there is for icon fonts, which are often better choice, than image icons - if you want different icon color on :active or even :hover, you just change CSS color property, not the whole image.

Social network buttons can be trouble too, especially when they are loaded directly from that site (with Like counters etc). If you want them, make it yourself and add them to your spritesheet / icon font.

Conclusion

Optimize your web for optimal performance is not easy task. It may feel it's better to start from scratch, but who says you have to adopt all of it? Try this and that now, but next time you build a new site, try to use as much as possible. Not only it will be faster for your readers or visitors, but you will decrease server load as a bonus. If you host your site in the cloud, it means you can reduce required hardware and lower the cost.

Why Apmin?

Some of you may wonder, why QetriX Apps are managed by "Apmin". What a weird word!

Well, not really. I like jamming words together, so what will be result of application administration, or app admin for short? Appmin. And "apmin" is just one "p" away, not mentioning that it looks like "admin", only with fallen "d".

I like a word "admin" for administration, but /admin/ may be too obvious for unwanted visitors. I don't expect QetriX to be hugely popular, so HTTP 404 on /admin/ may be a good thing :)

Mravenci as QetriX WebApp

6 years ago I made a remake of quite popular Czech card game Mravenci (inspired by Arcomage) for Pocket PC, written in C# for .NET Compact Framework. Because CF is a subset of regular .NET Framework, it worked on PC as well.

I was recently contacted, if there's an Android version of the game. I replied there isn't, because I tried and failed - as my experience with Android is shallow. But it got me thinking and then it hit me.

As I mentioned earlier, I have some experience with IBM Worklight MobileFirst Platform Foundation. Mobile side in the platform is based on Apache Cordova, which basically is a fullscreen chrome-less WebView (mobile browser), displaying the single page app, written in HTML, CSS and JavaScript.

I really like the idea, so I tried to make the game this way. All platforms have support for such apps, so I created basic WebView app in Swift and Objective-C (iOS), Java (Android) and C# (WinPhone). It worked really well and the final app is so tiny!

Everything is based on QetriX structure, with DataStores, "content" dir and such. I took the liberty of developing the game in Firefox on my PC. The development was nice and smooth.

Of course I had to reengineer some major stuff. Because of different aspect ratios, I had to make the whole game fluid (all sizes are percentage based). It may get little bit distorted on 3:2 displays though, so I added media queries.

Mravenci. Build your castle to 100 or destroy opponent's castle.

Also the original graphics was really low-res, which looks OK, but the blurring isn't that good so I decided to recreate everything possible in HTML (like cards). Besides I didn't want to make gray version of each card (when player doesn't have resources to play it) and grayscale filter doesn't work on Android 4.3 and older.

The sound is made by HTML5 Api, but causes some troubles, which I consider acceptable: it sometimes plays up to 1 sec later than it should and sometimes doesn't play at all.

You can download the game from Google Play and iTunes Store.