Pitching the idea

Like many other developers, especially from my region, my crucial problem is to pitch my idea of the particle database. I crashed and burned many times before I realized I need to focus on just a few main features and don’t try to explain the whole concept.

But in this case, the main thing isn’t that much exciting and everyone immediately sees I’m doing just an n-th copy of Freebase or something like that. Maybe I do, but not realized it yet...

However, I tried to sign up for a startup event at local dev conference, but I didn’t get their attention. At least the thought I may get picked made me thinking more about what I do and how I can explain that to the crowd.

Then I tried to write few paragraphs about QetriX to local tech website’s forum and got some responses. Fortunately all criticism was constructive and pushed me a leap forward. They mostly pointed out that my UI is too complex and hard to understand, plus the database is quite empty.

They got me thinking and ultimately I wrote a nice web crawler for semantic data extraction. I also redesigned the UI and dramatically reduced number of offered features, like comments or rating.

I slowly learned how to start simple and suppress the urge to make everything perfect at once. Roll out is much more important than perfection. You can always add more features later. On the other hand, if you don’t get user’s attention in the beginning, you may not get a second chance from him.

You have to determine your target group – geeks will demand features, but everyone else will demand simplicity. You can combine both, if you cleverly hide all the features and reveal them after the user is more experienced.

Q-Logo

Sun Microsystems

I like clever usage of name or at least a key letter in logo of product or company. My dream target was something like Sun Microsystems.

My requirements were quite clear:

• It should be stylized “Q”,
• looks good (or at least is recognizible) as 16x16 px favicon,
• circle or square (not rectangular),
• bold lines, carvable for stickers,
• not too complex in vectors.

My minor requirements were to utilize golden ratio or be symmetric.

QeX (animated)

I felt little depleted and without invention, so I asked my younger brother, who does graphics and has good ideas, to help me. He offered me few designs and the first one gave me an idea to try invert and merge some colors. It had everything I was expecting.

I did some improvements, grooming and polishing, like determine widths and thicknesses. The width would be 11 units, 1+3+3+3+1. The blue circle in the middle would have the same radius, as border of the outer blue rectangle; 1.5 units.

The biggest surprise was when I tried to recreate it in SVG. I used Inkscape for edit, but the exported SVG was too big. So I tried to rewrite it by hand in a text editor instead. I found out the final layout will consist only from a single square, a single circle and a single line. Wow!

I played with the blue as well. I really like the blue Google uses on focused input elements (#4D90FE), but it was too random for my taste. After some tries I ended up with nicer and web safe #6699CC (6-9-12; 102,153,204).

I really like it, even I keep hearing it looks like a Pac-Man ;-) I see a hard disk, which is great icon for a data platform.

I tried to modify it over the time, for example it still bugs me that I didn’t use golden ratio (which is hard to achieve in a concentric design) or that I didn’t use guide circles better. But so far all changes always ended by a rollback.

The Name

In 2001 we started our Internet forum, called “QeX”. Letters Q and X has been chosen by my friend Pavel, because those two are the only letters in Czech language, which you can’t pronounce as a single letter.

Domain “qx.cz” was already taken, but I remember we saw it available when we looked for the first time couple weeks before. So we decided to put “e” in between (for better pronunciation, to make a word out of it) and, as a remembrance of the original idea, we kept “Q” and “X” uppercase.

In 2006 I discovered a leetspeak and at that time I was looking for available short “.net” domain, which serves better for Internet stuff. I tried to convert “qex” into leet and I got “q3x”. I tried different pronunciations and then it hit me: “Q-tri-X” – sounds like “Matrix”. So I registered q3x.net. I saved this name for my emerging project.

One day I tried various speech plugins in different languages, like Google Translate or Text-to-Speech in Windows, and I found out, that transcription as “qetrix” sounded quite the same in all major languages, which was my wish for project name. Since then I used “qetrix” much more often.

I always wanted a “.com” domain, because it’s the most common TLD and it’s somehow the best choice for a new startup. I already pronounced “q3x” as “qetrix”, and to my delight (but not a surprise) “qetrix” was TLD-wide available. It took me just a few secs to decide and I claimed generous amount of TLDs :)

I kept q3x.net a primary and all qetrix.* redirected to it. Almost a year later I decided to use “qetrix.com” as primary and everything else as redirect.

And this way the word “QetriX” was born.

Everything makes sense now

I was quite happy about the total reengineering. I started in a brand new directory and created brand new scripts. It was totally new approach, but the puzzle finally clicked together.

My first neglected todo was renaming the key table. There was no reason for calling it “a”, it was historical reason when everything was put into “attribute” table. Because I came up with the name “particle” for the data structure and I still like single-letter name for tables, I renamed “a” to “p”. As a result I was unable to use anything I created so far right away, I had to dip into everything. And it was a good thing.

I knew everything will be either a list, detail or edit form. In my opinion this realization was the second most important invention for QetriX, after particle. I was able to stick with these three components and create a compact code, as I always wanted. It would require much more discipline from me to keep everything in order.

Even QetriX is more about methodology, the idea without implementation is quite worthless. So I decided I’ll rebuild the system in PHP again, maybe later in C# .NET or it will be my case-study for Java, Android, iOS, HTML5 or whatever... :)

I designed a new object model. I don’t like bloated object apps with giant code overhead, mostly because I didn’t use advantages of OOP, like inheritation. All my code uses KISS approach (Keep It Simple, Stupid), so I didn’t feel the urge to organize my functions into static methods. But I knew if I want to make the code open source one day, I need to abandon my 1990s coding habits :)

I was going to create something like framework, so for a few weeks I studied some of the most used – Nette, Zend, CodeIgniter, CakePHP, Symfony, FatFree (lightweight counterpart), plus MediaWiki and Wordpress. Closest “to my heart” was CodeIgniter.

I created URL parser (routing) first. It will combine class-method approach with URL rewriter (for particular entity, loaded from database). I established a basic set of rules for certain URL patterns for determining what part of URL will be detected as class, method, entity ID and parameters. If no class is found, system asks database for entity URL and if no correspoding URL is found, HTTP 404 is fired. And if URL contains /data/, system searches for corresponding file to offer for download.

After quite long and sometimes quite frustrating debug it works like a charm. I’m still thinking about reducing some lines in code though, I’ll be happy to omit file_exists checks for certain cases as this code is executed all the time. On the other hand without this part working I was unable to continue in development, so eventually I sacrificed few milliseconds for sturdy and reliable code.

New Destinations

I was going to try the engine on fleet management and dispatching system. It was pretty much the target usage, so I was particularly happy about it.

First showcase was a small fiasco, because future users didn't like it. It was too different from their current program, which was slowly dying for several unimportant reasons. The platform was (and still is) very flexible, so I promised to bend it as far as I could to meet their requirements.

It took me about 6 months of non-full-time development and the progress was tremendous. I started as “Win95” and ended as “Win7”. It took so long because I stumbled upon some major problems of initial design and I had to reengineer – twice!. But much worse was that end users needed, but didn't want the new system. They were happy with status quo, so they basically requested to recreate their current system (Win32 app) exactly as it was, with no changes or improvements whatsoever.

I don't want to go into much detail, as it's irrelevant for this story, but I want to write about changes this experience led to.

The most notable change was a new approach to generating a form. Since then only controls for value has been generated. Now controls are generated for relation, order and flag as well.

For datetime control I created extremely rich/smart, but quite buggy (I must admit) control. It was single input (textbox), but thanks to a lot of JavaScript around the control it checked, what part of datetime are you currently on and offered appropriate functionality.

Like when you wrote “8” on “day” part, it knew there are up to 31 days and “8” is the final day, so it skipped to “month“ part automatically (in Czechia we use “dd.mm.yyyy” date format). On the other hand, if you wrote “2” on the day part, you stayd on the day part, because the control didn't know, if you wanted to write just “2”, or e.g. “24”. And there are more such things.

For number control I allowed user to change value using up-down arrows and because on Czech keyboard you have to press Shift to write number, I checked for the other without-Shift characters and convert them into appropriate numbers.

I also had to think more about more functionality, like “suggest” for values, drawing tables, forms, pages etc. I noticed some repeating patterns, but unfortunately I didn't have time for deeper analysis. But I kept it in mind.

The project finally didn't happened, even after I finished the last request and killed the last reported bug. It wasn't so big for the hassle, I already built an intranet for our company on it, which saves a lot of my coworkers' time. After I dropped this load off my shoulders, I was able to dust off the knowledge for the final reengineering. But this time absolutely from scratch.

Birth of QetriX

So I was going to create an ultimate database model and try the second option, mentioned in one of my previous blog posts. Now I was skilled enough to see much further ahead and avoid any performance issues in advance.

Thanks to the decision I reduced the model from 18 tables to 6. But I'm getting ahead of myself.

I was fairly familiar with parent-child approach, utilizing hierarchical database model, so I decided to use it. Each row has “parent_fk” column; attributes and relations will point to entity's primary key there and if the parent_fk is the same, asi primary key, it's an entity.

For value I defined “Twitter's” VARCHAR(140) column / domain. Longer values would go to a different table, when needed.

Then it was time to deal with relations. My first idea was to put foreign key into the varchar column for attribute value, but it was no-go even for my anarchistic side, not mentioning it would give many invalid relations (with numeric values) and sooo many invalid JOINs (esp. with text values). Adding new numeric column to the table was inevitable, but later I realized it was a pivotal moment.

From some experiments emerged it would be useful to add order and flag/strenght/significance. I tried to combine it into single numeric column, but it wasn't safe and might collide in some cases. To keep the model compact I didn't want to add two columns, so I made a Solomon's decision and created decimal column, where integer part would be significance and fractional part order.

In MySQL I used FLOAT for this column, but later I found out, when I use a datetime in numeric format (yyyymmddhhmmss) as order, the precision isn't sufficient and the number gets crooked. After some testing I redesigned the datatype as DOUBLE(18,14). I figured I wouldn't need more than 4 digits for significance.

After I finished the “data” table, I moved to “type” table. Design of its structure was just about what I expected from the system. I had parent type, type and format od value (text, number, url, isbn, date, zip...), order in entity, max allowed occurencies in entity, units (for converting), max length of value etc.

A saw a major flaw – entities can't share the same attribute type, each entity type allows only own attributes. This led to a new table, but this one was fairly useful, so I didn't mind. After all, it was the third one and I made i optional.

I used this table for multiple purposes. Except for reason mentioned above, after I added a column for a relation, I was able to define a list of entity types, to only which the particular relation can point.

My employer gave me a golden opportunity to use my data platform on a real project, which was a fleet management system, and I was ready to take the challenge.