Nerd wannabe

lazy weblog

SQL Dependency Injection

with 2 comments

“All problems in computer science can be solved by another level of indirection, but that usually will create another problem.” — David Wheeler, inventor of the subroutine

This is the story of a “framework” that doesn’t exist: SqlDi

Most database frameworks’ authors consider that it is wrong to write SQL in your source code and provide many kinds of wrappers you need to deal with, instead of pure query strings.

Their reasons are along the lines of:

Portability: SQL is not database independent

Sadly, this is true. Although there are ANSI standards – ANSI SQL 99, 2003 etc. – no RDBMS supports all the features from ANSI and most vendors force you to use some of their custom extensions to accomplish things from the standard.

(For example, SQL 2003 introduces ‘window functions’ that could be used instead of the vendor specific ROWNUM, LIMIT or TOP keywords – and the window functions allow you to do even more. Few database systems support SQL 2003 though..)

Abstraction: SQL queries contain details about the storage mechanism

The relational model is a good way of assuring the integrity and non-redundancy of data. JOINs allow you to reverse the normalization process, while keeping the “Single Point of Truth” – storage model. INDEXes allow you to lookup faster.

Having SQL (in your source) that displays all these storage details is bad in practice. Even if your attributes (columns) are not subject to change, the way of retrieving them may be.

For example, if you want to display a (paginated) list of books on a web page, with their authors, a minimal query would be:


SELECT books.title as title, books.date_published as date_published, authors.name as author
FROM
  books
INNER JOIN
  authors
ON
  books.author_id = autors.id
WHERE
  ROW_NUMBER() OVER (ORDER BY books.title) BETWEEN 1 AND 10

This is both an example of SQL that reveals the storage internals and that is not portable (because ROW_NUMBER() is supported by very few ‘chosen’ ones)

Correctness: Query strings are validated only when run against the database

Even if you’re using an editor that knows how to highlight syntax from the SQL strings, you are still prone to mistakes, syntactical or not (e.g. wrong column name).

The usual solution is to write objects that construct your query – here’s an example for Hibernate’s Criteria API


Criteria crit = session.createCriteria(Cat.class);
crit.add( Expression.eq( "color", eg.Color.BLACK ) );
crit.setMaxResults(10);
List cats = crit.list();

This still fails if the “color” string is mistyped, for example.

SqlDi

The proposed SqlDi solution emphasizes some of the positive things about SQL, while working around the above-mentioned issues.

First let’s talk about the “Domain Specific Language” well known meme – SQL is one of the best examples of DSL around, but suddenly everybody wants to write in lolcode like this:

include StarbucksDSL

order = latte venti, half_caf, non_fat, no_foam, no_whip
print order.prepare

As opposed to this ruby Starbucks language I found on the web, SQL has proper syntax rules, and everybody knows at least its basics.

So SqlDI will allow you to write a subset of SQL called SqlDi.

But how about the aforementioned Abstraction ?

In order to decouple the storage model from the application query, SqlDi uses a kind of remote-process inversion of control (RP/IoC) pattern where the actual storage-specific part of the query gets injected in your code by the RDBMS process (instead of your code depending on the storage layout).

Here’s an example of the previous query, re-written in SqlDi:

SELECT title, date_published, author
FROM books_with_authors
LIMIT 1,10

You can already guess the syntax of SqlDi, but I will paste it here anyway:

SELECT attribute_expr [, attribute_expr]* FROM thing [WHERE boolean_expr] [LIMIT n [,m]]

Now how does the ‘thing’ called ‘books_with_authors’ got injected?

Simple, it’s part of the database “configuration file” (the script that creates the schema):

CREATE TABLE authors
( id INTEGER PRIMARY KEY, name VARCHAR(80), birthday DATE);

CREATE TABLE books
( id INTEGER PRIMARY KEY, date_published DATE, author_id INTEGER FOREIGN KEY
  REFERENCES authors(id) );

CREATE VIEW books_with_authors AS
SELECT books.title as title, books.date_published as date_published, authors.name as author
FROM books INNER JOIN authors ON books.author_id = autors.id;

Now, if some other time you discover that one book can have multiple authors, all you have to do is to re-write the storage-dependent part of the query.

But how does SqlDi provide Portability ?

SqlDi is a parser for a subset of SQL – you have already seen the SELECT statement syntax. The various functions dealing with numbers, dates and strings, even the operators (such as string concatenation) are defined one way or another in SqlDi and translated into the native dialect.

For example, the LIMIT keyword is translated into ROWNUM, LIMIT, TOP or ROW_NUMBER() depending on the used database.

This translation is accomplished in two steps:

  • First, the query is tokenized and parsed, and an object which represents the root of the Abstract Syntax Tree is constructed.
  • Second, the AST spills out the native dialect – based on some form of run-time pluggable dialect writers (plus ‘drivers’)

In this way you’ll never need to change the “SELECT x FROM y WHERE …” from your source, even if you change the used database. (You do, however, need another schema creation script).

Yeah, but how about the compile-time Correctness ?

This is the most interesting part.

SqlDi is implemented in D (because I don’t know LISP), and the first step (parsing) is accomplished at compile-time, through the use of mixin() statement. The output of parsing is code, like this:

SqlDi.Select ( "books_with_authors", null /*the where clause*/, 1, 10 );

For example, if you have WHERE name LIKE ‘Richard %’, instead of a null you will have SqlDi.Like ( SqlDi.Column(“name”), “Richard %” ) – where “SqlDi” is the package name.

Of course you could write this yourself if you are a hardcore SqlDi user;)

Conclusion

To summarize what we achieved:

  • you can write SQL in your source code,
  • it will be database independent and
  • you will get compile-time syntax checking
  • bonus points: the DBA will become your best friend

What we didn’t do:

  • no object mapping was even considered
  • we didn’t enter into details of INSERTs and UPDATEs
  • SqlDi does not exist
Advertisements

Written by vlad

April 24, 2008 at 8:28 pm

Posted in ideas, Uncategorized

what gtkaml is not

with 2 comments

Updated nov. 2010

I noticed that the first reaction when people hear about gtkaml is “But GtkBuilder/glade already does this!” or “How is gtkaml better than glade?” or something along these lines.

Let me put this straight: there is no resemblance between gtkaml and glade (other than using markup). None. They are not mutually exclusive, and do not do the same thing.

You can tell this by looking at them!

what they are

First, libglade is a library (and GtkBuilder comes bundled with Gtk+). You link against it and it parses your UI markup at runtime.. Glade is also a visual editor for that markup.

On the other side, gtkaml is a preprocessor. Your markup becomes code, Vala code, which and eventually becomes static C code.

language bindings

glade and GtkBuilder can be used from many different programming languages.

gtkaml is only available for Vala. However, libraries created with Vala can be used from any language.

run-time vs compile-time

GtkBuilder and glade make it possible to change the UI markup without compiling. This is because they call Gtk+ functions from libgtk, functions determined at runtime.

gtkaml only knows what you meant when compiling – it uses Vala’s AST to do this. You have to recompile to change the UI, therefore.

syntax

GtkBuilder and glade have their own markup which is verbose and usually must be written by means of an interactive tool (glade).
Every new Gtk property or class must have Glade/GtkBuilder support for it to work.

gtkaml simply maps tag names to classes and tag attributes to properties/fields/signals.

signals

Because of the run-time capabilities, GtkBuilder and glade require you to export your signal handlers from your own executable, like an .so/.dll does, or to write GtkBuilderConnectFunc’s for them.

gtkaml simply uses Vala’s signal .connect() which translates into g_signal_connect () function calls.

widget referencing

GtkBuilder requires calling gtk_builder_get_object (“by name”) to get a reference to a widget. Similarly, glade has lookup_widget (“by name”).

gtkaml optionally lets you declare widgets as public or private fields, so you can use them directly.

using custom widgets

glade requires specific code to instantiate custom widgets (set_custom_handler ()). GtkBuilder requires the widgets to implement GtkBuildable (which is a Good Thing).

gtkaml only needs them widgets to be compatible with the parent add method (wether is Container.add or something else).

Update: They don’t even have to be widgets, they can be Clutter actors or some business model objects.

creating custom widgets

GtkBuilder or glade do not have support for creating custom widgets.

gtkaml does only this. It creates custom (composite) widgets.

other than Gtk

glade and GtkBuilder are Gtk+ specific.

gtkaml can now accomodate any library that has a Vala vapi file (such as Clutter, Hildon, MX really, any) – you only have to point out in an .implicits file the methods used to add childs to a container.

Conclusion: use GtkBuilder if you need to change the GUI without recompiling. Use gtkaml to write custom composite widgets (with Vala code within) or use a gtkaml with the MVC pattern to separate view from behavior.

Written by vlad

March 25, 2008 at 1:29 pm

Posted in gtkaml

C++ and C++ TR1 (Technical Report 1)

with 5 comments

In this post I will assume you’re familiar with other languages like Java and C#, which are OO languages, and you start to learn the ‘multi-paradigm’ C++. Yeah, you couldn’t be that guy, could you.

So you already know the syntax.. so quick, let’s write classes!

We’ll start with a ‘bitmap’ that supposedly encapsulates a bitmap image, allocating memory at construction and freeing it at destruction:

/* bitmap.h */
#include <string>
class bitmap
{
    unsigned char *data;
public:
    bitmap(std::string path);
    ~bitmap(void);
};
/* bitmap.cpp */
#include "bitmap.h"
bitmap::bitmap(std::string path)
{
    //allocate memory
    data = new unsigned char [1024];
    //read data from path
    //parse it as an image
}
bitmap::~bitmap(void)
{
    //free the memory
    delete [] data;
}

Now that’s pretty simple and obvious, given that details about reading/parsing the bitmap are left out.Let’s see how we can use this class; we can allocate a bitmap object on heap like this:

/* main.cpp */
#include "bitmap.h"
int main (int argc, char * argv[])
{
    bitmap *icon = new bitmap ("./icon");
}

Oops.. not good, our destructor doesn’t get called. We have to manualy call delete icon so that it calls its delete [] data.

Hmm.. ugly.

But we do know that desctructors are automatically called when an auto variable gets out of scope and so we do this:

 bitmap icon = bitmap ("./icon");

Bummer! this time the destructor is called twice! Obviously, it fails the second time:(

The reason is that we weren’t aware that the default assignment operator is called in what we thought was an initializer (that ‘=’ that stands between the object and the constructor call).

This is actually one of the reasons most C++ code reads like:

bitmap icon("./icon");

The other is performance.
(yeah, event a simple int f = 0; gets obfuscated to int f(0); which looks like a goddamn function call)

Introducing auto_ptr<T>

We can further get rid of: the stars ‘*’ and the destructor by declaring the data to be an auto_ptr:

#include <string>
#include <memory>
using namespace std;
class bitmap
{
    auto_ptr<unsigned char> data;
public:
    bitmap(string path);
};
#include "bitmap.h"
bitmap::bitmap(string path)
{
    data = auto_ptr<unsigned char> (new unsigned char [1024]);
}

auto_ptr is a class that wraps a pointer. Its instance being used without dynamic allocation, the instances are freed (after being destructed) when they are out of scope.

This simply allows the ~auto_ptr () destructor to call ‘delete’ on the wrapped heap object just like with an auto object: when it gets out of scope.

It allows us to write code like this – once again:

 bitmap icon = bitmap ("./icon");

and like this:

 data = auto_ptr<unsigned char> (new unsigned char [1024]);

That is because the assignment operator of the auto_ptr transfers ownership of the wrapped object, NULL-ing the old reference, and because the right-hand expressions in the above two examples are not used further in the code.

You’re still fucked up if you write:

bitmap largeicon = icon;

since the right-hand object is left with a NULL reference!

Introducing shared_ptr<T>

shared_ptr is a class that boost provided long time ago, and now will be part of the upcoming C++ standard (C++09 probably, but currently named C++0x).

This is, one way or another, already supported in the upcoming Visual C++ 9 SP1 (today as a beta feature pack) and also in GNU g++ 4.x

The simple fact that this feature, now in the standard, can be provided by a library tells something about the power of C++. Unfortunately, the fact that the boost implementation is unreadable tells another thing – about C++ compilers. (to think about it, what other languages have as many compilers – and compilers’ problems – as C and C++?)

Basically, shared_ptr is counting the references to a pointer, and, when copied, increments the references, and when destructed, decrements their number. Finally, when no more references exists, it calls ‘delete’.

So our pointer-free source (sic!) becomes:

#include <string>
#include <memory>
using namespace std;
using namespace tr1;
class bitmap
{
    shared_ptr<unsigned char> data;
public:
    bitmap(string path);
};
#include "bitmap.h"
bitmap::bitmap(string path)
{
    data = shared_ptr<unsigned char> (new unsigned char [1024]);
}
#include "bitmap.h"
int main (int argc, char * argv[])
{
    bitmap icon = bitmap ("./icon");
    bitmap largeicon = icon;
}

If read up until here, you have just seen the way to aquire the memory resource on instantiation, so that you don’t need to manage it’s release.

TR1 has other new things too.. next time we’ll check on the <functional> classes function, bind and result_of.

(to compile the above in g++ 4.x, use #include <tr1/memory> and using namespace std::tr1;).

Written by vlad

March 12, 2008 at 3:32 pm

Posted in trivia

attribute values in gtkaml

leave a comment »

Rule of the thumb: when you don’t enclose your attribute values in {}’s, something magic is going to happen.

That is because gtkaml tries to make your life easier. Let’s take the first example,

Properties

<Window title="my window" type="{WindowType.TOPLEVEL}" ...&#91;/sourcecode&#93;
The difference between using and not using curly braces is obvious: if you use {}, you get to write a valid <b>code</b> expression. Otherwise, gtkaml tries to guess a literal for you: in this case, title being a <b>string </b>property, the value &lt;&lt;my window&gt;&gt; gets translated to &lt;&lt;"my window""&gt;&gt; in Vala.

This would be the equivalent of <b>title='{"my window"}'</b> (you have to use the alternate single-quotes of xml for this).If the  property is <b>bool</b>, you get exactly the "false" or "true" literal, and if the property is an <b>integer</b>, the same.

Actually, whatever the property is that is not a string, you get what you typed - the "magic" happens only for string properties.

Alternate 'spellings' for the code above are possible, along the lines of:

<Window>
   <title>my window</title>
   <type>{WindowType.TOPLEVEL}</type>
</Window>

or if your value contains invalid xml characters:

   <title><!&#91;CDATA&#91;my window >= your window&#93;&#93;></title>

Signals

<Window destroy="{Gtk.main_quit}" />
<Window delete_event="Gtk.main_quit()" />
<Window delete_event="{ widget => { Gtk.main_quit (); }" />

The difference is that the first is an expression of type “function” (an expression that returns a function), while the second is executable code. The generated Vala source is like this:

//with curly braces:
this.destroy += Gtk.main_quit;

//without curly braces
this.delete_event += (target, event) => { Gtk.main_quit(); };

//third way - just like the first
this.delete_event += widget => { Gtk.main_quit (); };

Second, the value without curly braces does the “magic” of creating an anonymous function and to automatically fill the signal parameters for you. This can be helpful if you want to write short executable code in a simple attribute; otherwise you can write a ‘full-blow’ function body in a CDATA section:

<Window>
   <delete_event><!&#91;CDATA&#91;{
widget => {
     Gtk.main_quit ();
}
}&#93;&#93;> </delete_event>
</Window>

Notice, however, that you have to use curly braces two times: one for the method body, the other for gtkaml to know not to do any ‘magic’.As you can see, no matter how you specify the signal, gtkaml adds the last “;”.

Construct

gtkaml 0.2 (svn version as we speak) features two new properties, named gtkaml:construct and preconstruct. This is because gtkaml creates the class construct for you (where it sets up the attributes, adds children to containers etc), but you might need to add yourself code there. You can use gtkaml:preconstruct and gtkaml:preconstruct on all tags:

  • preconstruct is executed just after the object is instantiated, before setting the other properties
  • construct is executed just before the object is added to it’s parent container

As a side effect, the root tag has ‘preconstruct’ executed before all others preconstructs and the ‘construct’ after all other constructs.

These attributes are to be used exactly like a signal: you write code, and you have the “target” parameter (the widget)

In the case of using {}, you can directly assign an existing function that receives a widget or void as parameter, or write a lambda expression in-place.

Example:

<Window 
   class:preconstruct="{view_init}" 
   class:construct="post_init(target)" />

As with the signals above, the first attribute simply sets “view_init” as the method to be called, and the second method actually wraps the call into a lambda function defined like target => { post_init (target); };

There isn’t much to do with the target parameter in this example: you can always use this. instead. But it may be very useful if you want to modify an unnamed widget.

Written by vlad

February 26, 2008 at 12:42 am

Posted in gtkaml

Layout: Windows, Java and “Others”

leave a comment »

99% of visual application developers use some tool to draw/drag/resize or even dock/anchor their child widgets visually.

One of the first tools I used myself is Resource Workshop from Borland:

Borland Resource Workshop

Yes, the Win16 version:)

Anyway, it’s a visual WYSIWYG editor. Just like the current Microsoft tools from .NET (now they are integrated in the development environment). One difference is that instead of editing a simple .res/.rc file, now they edit a partial C# class)

Why partial? because initially it was editing your class. And since the code generated was awful, you had to scroll down (or collapse) entire pages of code before getting at your stuff.

Here’s how some of the ‘partial’ code looks:

            this.label1 = new System.Windows.Forms.Label();
            this.textBox1 = new System.Windows.Forms.TextBox();
            this.button1 = new System.Windows.Forms.Button();
            this.SuspendLayout();
            //
            // label1
            //
            this.label1.AutoSize = true;
            this.label1.Location = new System.Drawing.Point(46, 14);
            this.label1.Name = "label1";
            this.label1.Size = new System.Drawing.Size(46, 13);
            this.label1.TabIndex = 0;
            this.label1.Text = "User ID:";
            //
            // textBox1
            //
            this.textBox1.Location = new System.Drawing.Point(98, 12);
            this.textBox1.Name = "textBox1";
            this.textBox1.Size = new System.Drawing.Size(51, 20);
            this.textBox1.TabIndex = 1;
            //
            // button1
            //
            this.button1.Location = new System.Drawing.Point(66, 65);
            this.button1.Name = "button1";
            this.button1.Size = new System.Drawing.Size(75, 23);
            this.button1.TabIndex = 2;
            this.button1.Text = "&OK";
            this.button1.UseVisualStyleBackColor = true;
            //
            // Form1
            //
            this.AcceptButton = this.button1;
            this.AutoScaleDimensions = new System.Drawing.SizeF(6F, 13F);
            this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
            this.ClientSize = new System.Drawing.Size(206, 104);
            this.Controls.Add(this.button1);
            this.Controls.Add(this.textBox1);
            this.Controls.Add(this.label1);
            this.Name = "Form1";
            this.Text = "Login Please";
            this.ResumeLayout(false);
            this.PerformLayout();

Notice the number of integer literals used to represent screen coordintates: it can get better, if you use anchors, but still the default layout is ‘absolute’. And this is not resize-friendly (actually these kinds of dialogs always have resize disabled;))To make up for the DPI change in today’s monitors/resolutions, .NET provides a property named “AutoScale” for dialogs, with values “Font” or “DPI”. That is, keep your design absolute at this font/dpi and I will compute for you another size at the client’s font/dpi settings…

Java got further and reckon’d the need of relative layouts: the choices are many, BoxLayout, GridLayout, SpringLayout, FlowLayout and last but not least, GridBagLayout. None of them was usable for any serious stuff (oh, I forgot the ‘null’ layout) but GridBagLayout.

This layout is so complex that maybe one may think that nested GridBagLayouts approach the Turing-completeness of PostScript/PDF (they don’t). Here’s an example of parameters one must specify to add a component to a container that has the GridBagLayout:


public GridBagConstraints(int gridx,
                          int gridy,
                          int gridwidth,
                          int gridheight,
                          double weightx,
                          double weighty,
                          int anchor,
                          int fill,
                          Insets insets,
                          int ipadx,
                          int ipady)

Oh well. You have to call that constructor for each child add..Actually it does not resemble PS/PDF in any way, on second thought: it is HTML table layout or CSS layout (whichever, they are equally stupid).Ok, what did Others?

Well, others are (at least):

  • Qt
  • gtk+
  • wxWidgets (sortof)
  • Mozilla XUL
  • Flex

They proved that any layout can be expressed with only three simple constructs: VBox, HBox and Grid!

VBox is for laying childs vertically, HBox is for laying them horizontally, and, if you ever need something like “two HBox’s in a VBox” and you need each child of a HBox to be aligned with the same child in the other HBox, then you actually need a two-row Grid;)

Don’t believe it? just look at the source of the XUL Mozilla Amazon Browser (does it contain any integer literals?)

Written by vlad

February 15, 2008 at 11:10 pm

Posted in morons

how gtkaml works

with 5 comments

If you are writing something based on GTK+ / GObject then you should be interested by this:

  • Vala is a very productivity-boosting language based on GObject OOP-ness, which translates to C and therefore can export GObject classes and interfaces from shared libraries.
  • gtkaml is a thin SAX Parser on top of Vala that allows you to write XML user interface descriptions

This is how a minimal Vala class described with gtkaml looks like:

<VBox class:name="MyVBox"
      xmlns="Gtk" xmlns:class="http://gtkaml.org/0.1">
        <Label label="Hi_story"
               expand="false" fill="false" padding="0" />
</VBox>

This code creates a class named MyVBox that extends VBox and contains a Label. The label has the value “History”, it does not expand or fill its box and is not padded.Easy, no?

But GtkLabel doesn’t have expand, fill, or padding properties! I hear you say. Yes, but the pack_start () method has them as parameters.

Which leads us to the first subject:

Container Add Methods

gtkaml uses a file named implicits.ini to determine which containers have which add methods. This is for “implicit” decisions it makes based on your UI layout.

For the example above, it simply finds this:

[Gtk.Container]
adds=add_with_properties
[Gtk.Box]
adds = pack_start;pack_end;pack_start_defaults;pack_end_defaults

This simply lists all the functions that one can use to add widgets to a Box and to a Container. Since VBox is a Box and Box is a Container, gtkaml finds these two entries ‘significant’ and merges them in this order (most specialized first):

pack_start;pack_end;pack_start_defaults;pack_end_defaults;add_with_properties

Then it loops over these methods – in this order – finds the first matching one and ‘picks’ the parameters from the child widget tag: that is, it removes “expand”, “fill” and “padding” from the Label and makes a note to use their value when adding the label in the VBox.

What if I want to use another add method?

Well, in that case simply specify the method’s name as an attribute of the child tag, with the value “true”:

<Label pack_end="true" label="Hi_story"
               expand="false" fill="false" padding="0" />

Creation methods

In this same example we notice that the label contains an underscore: this is intended to show up as a “mnemonic” (an keyboard shortcut) or simply as “History”.

Looking again at the GtkLabel documentation we notice that the only ways to get a mnemonic is to use either gtk_label_set_text_with_mnemonic () or gtk_label_new_with_mnemonic (). But since gtkaml can only help you with public fields, properties and signals, and not with function calls, how does one specify a mnemonic?

The answer is that you can select a “creation method”, and gtk_label_new_with_mnemonic () is not an ordinary function but a creation method. That is, a method that does not require the object as the first parameter but returns a new object.

So simply specify the creation method name as an attribute with a value of “true”: with_mnemonic=”true”.

Wait, but how about the creation method’s parameters? with_mnemonic has one parameter named str..

Well, usually Gtk widgets’ creation methods have their parameters named after the field/property they are initializing. gtkaml recognizes them from the specified attributes.

In this case, though, str is not a field or parameter of GtkLabel. But we know that it actually refers to the label property.

So what gtkaml does is to specify, again, in implicits.ini, the attributes that will go as parameters to the creation method:

[Gtk.Label]
new = label
new.with_mnemonic = label

So the code becomes:

<Label pack_end="true" with_mnemonic="true" label="Hi_story"
               expand="false" fill="false" padding="0" />

That’s it! – more on what gtkaml 0.2 will do for you in a future article.

p.s. as I am writing this, gtkaml was accepted on freshmeat.net! yay!

Written by vlad

February 7, 2008 at 3:06 pm

Posted in gtkaml, trivia

spam, rant, or code? (all verbs)

leave a comment »

(for those who need help deciding)

spam, rant or code?

  1. Put a link as IM ‘status message’ to my first drawing.
  2. Then write a rant about OOP.
  3. Then announce my first software release.
  4. ??
  5. Profit!?

Written by vlad

February 5, 2008 at 4:05 pm

Posted in laugh