Tuesday, August 18, 2015

Side effect of struggling

I had a struggle with QList.add method - I extended the method so much I actually duplicated some ways of handling passed data. I was wondering how to describe it in the best way, so I created a yet another document for this very purpose.

During documentation I noticed I was wondering about how it will actually handle same scenarios and my solution was to create a small script, which will answer this.

It didn't take me long before realizing this was actually inventing a wheel, because this is probably what unit testing (and ultimately TDD, Test-Driven Development) is all about.

Unit testing is for testing many small pieces of software, like methods, from different sides, in different scenarios. You call the tested code with many different input values (in different states, when present), even - or better especially - wrong ones. You may have dozens tests for a single method. Then you compare code output with expected result and if it matches, the test passed. For wrong input you set try-catch block and in the catch part you check the exception for correct one.

Because of the vast number of tests, the process is often automatized and launched after build, before commit, after server/platform software update or something like that.

My example of an unit test (in pseudo-language):

// Tested method, defined somewhere else in the app:
bool isNumericString(string value) { ... }






// Test suite for the method isNumericString:

Array testResults = [];

testResults.push("Test 1: " + !isNumericString("test") ? "Passed" : "Failed"); // Unit Test 1testResults.push("Test 2: " + isNumericString("123") ? "Passed" : "Failed"); // Unit Test 2
try { isNumericString(null); } catch (ex) { // Unit Test 3
    testResults.push("Test 3: " + ex.Type == NullException ? "Passed" : "Failed");
}

// and do something with testResults, like check what test failed and do nothing if all tests passed.


In TDD you start not by coding the method, but with writing such unit tests. Of course they all initially fail, because the method is blank. You define expected results for all different inputs and then you start making insides of the method to pass all tests you initially created.

So I created a test suite, defined some test cases and made it compatible with QList, so it would be possible to visualize the output in a QetriX way. I also introduced com.qetrix.tests namespace.

Testing QList

I may finally get what it takes to do nice unit tests, which was probably the last big aspect of development I was still quite missing.

Wednesday, July 29, 2015

Cross Platform Satisfaction

I switched from PHP to Java after few months to add new features to Jetrix (Java QetriX :-) Those switches always hurt: different environment, different editor, different coding habit requirements, different debugging. But this time it was... different.

Even after quite a long time with PHP I didn't have much struggle, because core parts of QetriX has been already made and my current task was mostly to update converters, datastores and modules. And the only difference between QetriX in Java and PHP were in variable types - hints and "arrays".

This was my main goal with QetriX, to unify programming. I know QetriX isn't optimal and it even may take worse parts of each language, but ultimately it saves me time otherwise spend on Stack Overflow and it works quite well actually.

I know I mostly make rather simple CRUD stuff nowadays, but even than it's wonderful to see, that QetriX is capable to make it work smoothly only with general QView, QList and QForm components.

I've always tried to do the most optimal solution, but sometimes it's obviously pointless. On the other hand, QetriX is for me a nice way how to teach myself new languages in the futures. Perl? Python? Ruby? Well, probably all of them, one at a time.

Monday, May 25, 2015

Qules, Unite!

I've always wanted to make a computer game. I got my two cents by Ants (Mravenci), but I had many other ideas.

In my early programmer years I had a bunch of games in my head and after some time some of them started to be quite thinked out. Unfortunately I didn't have the skills to bring it to life, I tried and failed many times in many programming languages.

When a decade passed by with no signs of completing any of them, I decided to take good parts from all of them and put them into a single game, which I will finish. I thought about a perfect main character and decided to go with a simple sphere. It will be easy to make, with lower polygon count than person or creature (although a nice sphere is often quite hi-poly) and it can roll really fast, which can alter as a car ride.

In 2013 I attended MS Fest in Prague and saw Tomáš Slavíček's lecture about MonoGame (follow-up of dead XNA), so I gave it a try. It was OK, but still... making 3D game was beyond my skills. I kept it in 2D, but wanted 3D inside. MonoGame was exactly what I wanted and I started to see how complicated making of a 3D game is. Keeping the code small is absolutely impossible. The same problem I had with WebGL, which was a hot topic at that time.

I didn't want to use some pre-made 3D engine or game makers, because I saw some of them and they felt overcomplicated, with many features I probably won't use or need and will run slowly on some machines.

During that time I thought about a name for the game. I somehow decided all stuff I do will start with "Q" and I many, many years ago I felt in love with a name of an old Czech game "Koules" (for UNIX, OS/2 and Linux). It's plural of "koule", which means "sphere" in Czech. I like the fusion of a Czech word with English grammar. After many variations I ended up with "Quly" (and "Qule" for the spheres).

About two years after my tries with XNA my younger brother showed Unity. It was quite funny, when I described my idea of the game to him and he put the RollerBall Character into the scene, which was pretty much it :-) My first tries were OK, but I was quite lost and didn't really know how to do stuff. I was otherwise engaged at that time, so I decided to continue when Unity 5 is out.

The release happened few months later. I really appreciated YouTube tutorials, especially those from Unity team itself. Unfortunately, some of them still used legacy features, which has been often replaced by newer approaches in Unity 5. Later I often followed the tutorial and spend few hours of fiddling and tweaking, but ultimately (and often accidentally) found out it's legacy and therefore not recommended for a new game. Damn!

My brother gave me a nice jump-start and I started to build a learning version of Quly. I always do it this way, so I'm not affraid to mess it up, because for the real thing I'll start again from scratch.

Thanks to great capabilities of Unity I was able to concentrate on gameplay and features only. Many things are already made, like collisions, physics, vector maths and tons and tons of other stuff, so I didn't need to bother with them (for example in XNA I had to deal with all of them).

I realized how foolish I was for rejecting pre-made 3D engines. I'll never be good enough to make an optimized game engine by myself! There's such vast amount of things it needs to be done and of course done properly.

In Unity I simply make a short script, attach it to a GameObject and test it. It's really fast, straight forward and I love it! Also, C# is my favorite programming language and the system of scripts and stuff in Unity really suits me. And cooperation with my favorite 3D software of all times, Trimble SketchUp Make, puts me on cloud nine! :-)

Tuesday, April 21, 2015

Converters

I suspect converters to be my second greatest idea for QetriX after Particles. In the past, QetriX used Renderers for output in desired format (HTML, JSON...). You simply handed over an object, like QList, to a Renderer of chosen kind and you got the output. It was simple as that.

But I faced the opposite situation - I have a string in some format and I need to convert it to an object. Imagine FileSystem DataStore, the data could be XML, JSON, flat file etc. I don't want to have FileSystemXml DataStore, not to mention Xml is for Soap DataStore as well. And I don't want to duplicate the code either.

I tried to create "coders", with encode and decode method. Therefore I was able to call a DataStore, the output process thru a coder.decode, then create an object and render it. Maybe it sounds complicated but in fact it was quite simple. Just few steps to make.

Well... no. It didn't work as well as I expected and I didn't like the name - coders. So back to a drawing board. I was working on a HTML coder, when the obvious came. I was dealing with conversion of a HTML code to an object and back, which in many aspects duplicates what Renderers are for. But Renderers are one-way only and I need two ways.

Even Renderers were in QetriX from the beginning, the new idea was beyond that. I renamed coders to converters (convs for short) and created few basic ones, like QForm to HTML (qform-html), Wiki to HTML (wiki-html). I was particularly happy about the second one, because before that it was as an add-on library. This way it makes much more sense.

I'm even able to make converters like PNG to JPEG, XML to QList, Image to Thumbnail or Integer to String.

To be honest, I can see the concept (or architecture, if you will) of Converters may be fundamentally wrong, because most functions are in fact converters, like "Math.add" converts two numbers into one number (sum total), and input/output types are not consistent, which is difficult to manage and impossible to unite into e.g. Interface. I may think of some boundaries in the future though.

Tuesday, April 14, 2015

Qarate

I like watching TV series, my list of favorites is quite long. I also like to re-watch them, from time to time, but sometimes only good episodes. I had a list of good ones on my personal wiki, but it was boring to note them each time (and when I watch multiple episodes in a row, I simply forget to note it and then I forget which one was the good one). So I discovered Trakt and was happy about it. Rating was quick and easy enough for me to actually do it.

One day Trakt creators upgraded the website and rating system was still in "to-do", so now I had nowhere to rate. I tried to google some replacement, but without any success. So, out of frustration, I added a new table "rating" to my personal wiki. It allowed me to rate not only episodes, but anything else - movies, songs, cars, vacations, etc. I was so pleased I thought this might be a nice service for others to use as well.

The biggest drawback was, that I had to enter the thing I was rating, mostly episode identification. I don't want my users to add anything. First of all, it's not quick and easy. Second of all, it will get wrong and duplicate (sorry, users...). It would require something, where all such data already is. Oh, wait!

I'm building qb for several years now and it will be the perfect marriage: a big database with "everything", plus a website, that allows you to rate anything. Sounds great.

I looked for a fitting name, which starts with "Q" and contains "rate" and in the shower (which is a common place for ideas for many people, according to one article I read) I thought of "Qarate". You can pronounce it as "Karate" (martial art) or "Carat" (unit of mass for gems and pearls). Therefore users may form dojos and level as gem rarity and belt colors (like blue diamond is more, than black sapphire).

Rating system will consist of 5 grades with matching score for that entity: +2 (excellent), +1 (good), 0 (meh), -1 (bad) and -2 (terrible). User may have limited +3 grades (the best), which can be used to mark notable things of it's kind, like one episode in a TV series season.

And I can imagine that rating of episodes may be relative to a rating of the whole series - +2 episode in +2 series is better, than +2 episode in +1 series. But user won't need to know that, it's a technicality for internal statistics and leaderboards.

Saturday, April 11, 2015

box-sizing: border-box;

One of the biggest pains in a web development, apart from vertical centering (except for inside a table cell), is a W3C box model, which doesn't include padding and borders into the width. You can't use "padding: 10px; width: 100%;", because the element will be 20px wider, than the parent element.

Fortunately, in CSS3 there's box-sizing property. Default value is "content-box", which behaves as the W3C model, but if you set the property to "border-box", it solves the problem with padding/border and the width. I noticed almost all major CSS frameworks adds this property to every element, like this:

* {
    box-sizing: border-box;
}


In the past (in terms of internet it was more like an ancient history), comparable pain was an old Internet Explorer's own box model concept, which included border and padding into the width (so  basically like "box-sizing: border-box"), whilst Netscape Navigator (ancestor of Firefox) followed W3C standards.

Side note: I don't like Microsoft doing stuff his own way, but this was actually better, than standards.

Anyway, when designing a web page you had to hack this differences, usually by using two divs instead of one. Outer div had the 100% width, inner div had the padding and border. Width of the inner div has been calculated automatically. And this wasn't the worst hack at the time, webdesign was often close to a witchcraft :-)

Microsoft added "standard mode" (now we call it "almost standard") into IE6, but kept so called "quirks mode" default and switching of those modes has been guided by DOCTYPE. This changed with IE7, where W3C mode became default (according to my test in IE11 dev tools).

Now, if you use HTML5 doctype <!DOCTYPE html>, you will get the best results - so use it always!

And about the vertical centering, you can use display property with CSS2.1 value "table-cell", so even div can behave like a table cell, with working vertical-align: middle, like this:

.verticenter {
    display: table-cell;
    vertical-align: middle;
}


But don't forget you should to make it a proper table! So parent div will have "display: table-row;" and it's parent div will have "display: table;". Happy styling!

Tuesday, April 7, 2015

WPF styles

I knew WPF styles are powerful, but the XML code is sooo long that I've never felt the need to expand my app with tons of new lines. Until now.

Recently I moved to dark skins everywhere. In PhpStorm I switched to Dracula theme with modified Zenburn schema, Geany has Zenburn schema as well, for Thunderbird and Firefox I found magnificent DeepDark theme and earlier this year I updated Quiky editor to Monokai-like colors.

I finally decided to darker even the viewer part of Quiky. Panels were easy, it's possible to set colors as variables (e.g. <SolidColorBrush x:Key="GuiBackgroundBrushColor" Color="#666666" /> and then <Setter Property="Foreground" Value="{DynamicResource GuiForegroundBrushColor}"/>). The troubles came with a TreeView.

I spent few minutes googling how-to and I found a StackOverflow a solved question about styling TreeView items. I also wanted the whole line to be selected, not only the text background.

It took me a while to get the solution working, because honestly I didn't want to understand all that, felt like I won't need it. But I had to, so ultimately I even tweaked it a little. And because I learned what's going on (restyling is often recreating controls, all in XML), I decided to style the whole app.

My next concern were scrollbars. I wanted Apple-like and I found a solution as well. It also needed some tweaking, but it looks quite good. I added hover effect on the thumb (slider), so it fades in and out.

Finally I styled buttons, textboxes and comboboxes, but it would need a little more time (some finishing touches), so I did only some basic stuff. I want some subtle effects (animations), but I didn't like anything I made so far.

I'm satisfied, for now:
Before redesign
After redesign

Wednesday, March 25, 2015

March 2015

I often find some piece of nice software and I like to read about "behind the scenes" about it, when a blog is available. Then it makes me slightly sad, when I find out the latest post is from 2 years ago. And yesterday I was like: "Hey, that's MY blog as well!" :)

Then I thought about it. It's quite hard to do a job and then to blog about it. Even I'd like to, but I can't promise I'll do better in the future. Self made publicity steals time from the object of publicity, more time blogging means less time coding.

On the other hand - who says I have to write a War and Piece every time? I have about 20 unfinished posts in stock, so now it might be a right time to slice them into smaller finished ones.

Anyway. What I've been up to with QetriX? Finishing the concept. I did another iteration, quite finished Java version after PHP has been usable. From that point I understood the PHP version is wrong yet again, so now I'm reeingineering and refactoring.

The finished will be even more consistent among platforms. I discarded "index" script as the entry point and introduced "app" and "file". For PHP it's handled by .htaccess, for Java it's handled by web.xml.

The WebApp will contain a HTML generator (basically Components with Converters to HTML) for internal WebView, which fits perfectly into the QetriX idea. I love this!

Finally I settled to separate data from users and data from geodata. Coordinates and user info in data (= mostly in Particles) will be still supported, but not recommended.

I finished my tracker app for iOS, which works well and replaced Moves for good. I was really crossed that Moves is not working offline and unfortunately the biggest test for the tracker so far, my short trip to Dubai, failed, because I screwed up a final update and the app crashed on startup. But never mind.

Last month I exported my first worksheed from my time tracker as well, works like a charm. I'm thinking of merging it together with the tracker, so it will know when I arrived to work and when I left.

I would like to write "and that's about it" now, but it isn't. But I'll get into it in some of the next posts.

Monday, February 2, 2015

All about performance

My key request for the new system was good performance, no matter how achieved. I ended up with a single data table and a lot of cache around it. In Particle Database I have a search cache and coordinate cache in MySQL and each entity in JSON file. Entity contains all own attributes and relations, plus top 5 foreign relations. Each foreign relation type to each entity type has own JSON file cache.

MySQL

Indexing of MySQL tables is crucial, too little is bad and too much is bad as well. Best practice is to work tightly with DESCRIBE command and index columns used in JOIN and WHERE clauses.

There are several situations, where indexes can't be used. If you perform LIKE wildcard search, only col LIKE 'abc%' or col LIKE 'a%c%' use indexes, but col LIKE '%abc%' and col LIKE %abc' and col LIKE col2 don't - because of the way B-Tree index is designed. Or if you have to do a string operation before comparsion, e.g. UPPER(col) LIKE 'ABC', all rows has to be loaded. In such case consider adding a new column, with cached upper-case values.

PHP

I always checked what approach in PHP would be the fastest, especially for functionality in iteration (loop). I used simple script with a timer:

function test1()
{
 $t = microtime();
 while ($i < 1000) {
  // HERE TESTED CODE
  ++$i;
 }
 return microtime() - $t;
}

This way I tried the best approach for zero padding or type casting from string to number.

Server

Activate GZIP compression.

Encourage browser caching for static resources, using Expires header, Last-Modified date or ETags in the HTTP header.

Use different URL root for your resources, because browsers allows 4 to 8 simultaneous requests per domain, so using different domain will increase. And it will be cookie free (cookies are sent even for CSS or images). There are public CDN resources for widely used stuff, like jQuery or Bootstrap.

Client side

I also deeply enhanced client side for speed. First thing was to minimize external files - CSS and JS. Minimized code strips out all unnecessary stuff, like whitespaces or comments. Deep minification also rewrites variable and method names to contain as few characters as possible. Every byte counts.

Minimized file can be even smaller using GZip compression, widely supported by browsers. This way you can 58 kB file compress to 12 kB. But always keep the original file and never edit the minified one. The only downside is debug on production server, as you can`t get a proper line number, where problem occured, becuse everything may be on a single line.

I know JS libraries/frameworks, like jQuery or Dojo, are really handy and I like to use them, but only if I can't do it with pure ("Vanilla") JS. And it's valid for CSS boilerplate frameworks as well (Bootstrap, Foundation). Because it's universal, it contains a lot of code you never use, but browser parses it on every page load. If you want to use it, remove all unnecessary stuff, if you can (there are tools for that). jQuery is 85 kB of code just to make another code works. Using JS lib only for element addressing is pure blasphemy and I don't even want to comment using more than one JS library on single page.

Next step is reducing HTTP request count. I merged all JS into single file, the same with CSS. Then I created so calloed "CSS sprite" - all graphis (pictures), where possible, migrate into single PNG file ("spritesheet") and instead whole CSS background, specify just backgroud-position. You have to do it cleverly, because of mixing repeating (backgrounds) and non-repeating images (icons). I can't really recommend data URIs, because it can't be cached and therefore you will download it over and over again. The only time it was extremely hand was when I got the picture already as base64 from REST web service.

One of the most difficult steps was to minimize HTML code. Some browsers (with blue "E" in logo ;-) in some previous (but still used) versions can't handle innerHTML manipulations without another DIV, so every time I want to use element.innerHTML = someText, I have to specifiy extra inner DIV. There are certain situations, when the HTML code is really bloated and I can't do anything about it. If you write the code in HTML5, you can at least get rid of the ending slash on non-pair HTML tags, like INPUT or IMG.

It's also good to check for invalid stuff over time. You may not use the widget or page layout any more, but linked scripts and CSS may still be out there. Also check for unused JS functions/methods and unused CSS classes.

Webdesign

Responsive webdesign is neat, but you have to transfer all the code thru slow mobile connections (LTE or even 3G is still a dream in a lot of countries). Much better is to create lite mobile version. Don't immitate specific GUI (like iOS), it will look ridiculous on other platforms or after a platform redesign!

Always use proper image format - JPEG (JPG) for photos and PNG for everything else. Because JPEG is lossy format, you can tweak quality (from 0 being crappiest but smallest to 100 being best, but large) to achieve best ratio between quality and file size. You may need just to re-save JPEG photos, even it slightly impacts quality - but often greatly lowers file size. For both formats there are online and downloadable optimization tools, that help lower their file size.

Many effects, achieved by images in the past, are nowadays possible just by CSS effects. They are significantly smaller, of course, but increases CPU time required for their rendering. A lot of shadows, gradients and transitions may kill performance on (low end = mostly used) mobile devices.

Consider resizing large photos, especially for mobile websites. On the other hand, because of different dpi, you may need double the size of graphics elements based on image to look good or hi-res displays. The best option is to use SVG, but like CSS, it may also affect performance if not used wisely.

You may not load the content (photo, video...), if it's not yet visible. This technique is called "lazy loading". You can detect it's vertical position and compare it to the scrollbar position. When they meet, you can trigger download.

Try not to use webfonts on mobile version. The only sensible use there is for icon fonts, which are often better choice, than image icons - if you want different icon color on :active or even :hover, you just change CSS color property, not the whole image.

Social network buttons can be trouble too, especially when they are loaded directly from that site (with Like counters etc). If you want them, make it yourself and add them to your spritesheet / icon font.

Conclusion

Optimize your web for optimal performance is not easy task. It may feel it's better to start from scratch, but who says you have to adopt all of it? Try this and that now, but next time you build a new site, try to use as much as possible. Not only it will be faster for your readers or visitors, but you will decrease server load as a bonus. If you host your site in the cloud, it means you can reduce required hardware and lower the cost.

Monday, January 26, 2015

Why Apmin?

Some of you may wonder, why QetriX Apps are managed by "Apmin". What a weird word!

Well, not really. I like jamming words together, so what will be result of application administration, or app admin for short? Appmin. And "apmin" is just one "p" away, not mentioning that it looks like "admin", only with fallen "d".

I like a word "admin" for administration, but /admin/ may be too obvious for unwanted visitors. I don't expect QetriX to be hugely popular, so HTTP 404 on /admin/ may be a good thing :)

Sunday, January 18, 2015

Mravenci as QetriX WebApp

6 years ago I made a remake of quite popular Czech card game Mravenci (inspired by Arcomage) for Pocket PC, written in C# for .NET Compact Framework. Because CF is a subset of regular .NET Framework, it worked on PC as well.

I was recently contacted, if there's an Android version of the game. I replied there isn't, because I tried and failed - as my experience with Android is shallow. But it got me thinking and then it hit me.

As I mentioned earlier, I have some experience with IBM Worklight MobileFirst Platform Foundation. Mobile side in the platform is based on Apache Cordova, which basically is a fullscreen chrome-less WebView (mobile browser), displaying the single page app, written in HTML, CSS and JavaScript.

I really like the idea, so I tried to make the game this way. All platforms have support for such apps, so I created basic WebView app in Swift and Objective-C (iOS), Java (Android) and C# (WinPhone). It worked really well and the final app is so tiny!

Everything is based on QetriX structure, with DataStores, "content" dir and such. I took the liberty of developing the game in Firefox on my PC. The development was nice and smooth.

Of course I had to reengineer some major stuff. Because of different aspect ratios, I had to make the whole game fluid (all sizes are percentage based). It may get little bit distorted on 3:2 displays though, so I added media queries.

Mravenci. Build your castle to 100 or destroy opponent's castle.
Also the original graphics was really low-res, which looks OK, but the blurring isn't that good so I decided to recreate everything possible in HTML (like cards). Besides I didn't want to make gray version of each card (when player doesn't have resources to play it) and grayscale filter doesn't work on Android 4.3 and older.

The sound is made by HTML5 Api, but causes some troubles, which I consider acceptable: it sometimes plays up to 1 sec later than it should and sometimes doesn't play at all.

You can download the game from Google Play and iTunes Store.