Nerd wannabe

lazy weblog

Archive for the ‘ideas’ Category

Auto-validating JSON with optional documentation

leave a comment »

The idea is to have one of the nodes documenting the structure of the rest of them and to have the ability to write textual documentation for the values.

The node "schema?" is used to introduce this structure as a top-level node. The contents of the schema have keys in the form "key? The documentation is here", where the ? sign plays the role of a separator but also says something about the requirement for a key or value’s presence.

  • key! => key must be present
  • key? => key is optional
  • key+ => value must be non-null, and in the case of arrays it must have a length>0
  • key* => value may be null

"key", "key!" and "key+" are shorthand for "key!+",
"key?" and "key*" are shorthand for "key?*",

Any value is accepted and documents the type of the value: "", 0, 0.0, true, [], {} etc.

  • "" => any string
  • integer value = 0, value > 0 or value < 0 => integer number, positive only, negative only
  • double value = 0.0, value > 0.0 or value < 0.0 => double number, positive only, negative only
  • boolean value = true or false => boolean
  • array value = [], array value = [value1, value2..] => any array, or array containing elements in the form of either value1 or value2 (nested specification)
  • object value = {}, object value = { "key" : value, ... } => any object, or object with given schema (nested specification)

No constraints can be set on the order of elements.

Example:

{
  "schema?": {
    "title! This is shown in the window's titlebar" : "", 
    "count! The number of things" : 1, 
    "price! The price in USD" : 1.0,
    "head? Optional tag with metadata" : {
      "script? One or more script elements" : [{
        "source! Each one with a source string" : ""
      }], 
      "meta!  Array of metadata strings with at least one element" : [""], 
      "style!* Any array, even empty one"  : [], 
      "what?  Optional, any array"  : []  
    },
    "body! Main content" : {
      "div? Any number of divs": [],
      "span?": []
    }
  },

  "count" : 5,
  "title" : "JSON",
  "price" : 4.5,
  "head" : {
    "meta" : ["data"],
    "style" : [],
    "script" : [{ 
        "source" : "http://...js"
      }, { 
        "source" : "spdy://...dart", 
        "type" : "text/dart" 
      }]
  },
  "body": {
    "div" : []
  }
}

 

This also has the advantage of being valid JSON, after all. It can be a separate document used to validate objects passed over the wire, or included inline in e.g. a configuration file. And even if a JSON parser is not validating against the "schema?" the rest of the data is still usable.

Plus, you get comments in configuration files (because JSON does not allow comments..).

Advertisements

Written by vlad

July 4, 2014 at 8:03 am

Posted in ideas

Tagged with , ,

Static analysis screenshot

leave a comment »

Apple seems to make good GUIs and recently it makes compilers too, and IDEs of course.

Actually I don’t have a mac, never saw XCode, and never used CLang (yet).

But this makes me want them all:

static-analyzer

Remember, this is primary a C compiler! I saw few static analysis on C.

And then… just look, it draws a diagram. Mmm… arrows!

Good work!

Written by vlad

September 1, 2009 at 2:21 pm

Posted in ideas

Pointless update

with 5 comments

So.. I needed to re-cap a never-learned science, the computer one.

You see, when I joined a computer science university, I had no “official” CS background (math high-school, yadda-yadda).
I did had a computer at home in highschool – which allowed me to poke around with some ‘exotic’ VGA poking.. but never got into backtracking or other stuff my friends from CS highschool got.

Then, just when the 3rd year in university got to those subjects, I was _so_ employed I haven’t had the time to.. you know.. think before the exams.

So I bought some books now, that I am old enough to afford myself shopping from amazon (heh):
– the algorithm design manual – which I favoured over ‘CLR’ because everybody said CLR was ‘sooo basic’. Now I have to struggle with my english skills because the ADM is explained in plain english rather than scientific way
– the dragon book – yeah..
– learn to tango with D – the only D book available today (I love reading paper, you know..)
– erlang – the only book to – influenced by my reddit fellows

Soo.. I am planning re-writing gtkaml, the markup for UI that I ripped from my XAML/MXML/XUL experience.

But is it worth it?

I want to:
– stop using libxml2, switch to GMarkup, to minimize dependencies. But GMarkup doesn’t even discern CDATA from regular text (I need that for code in a gtkaml class)
– cleanly separate parsing from semantic analisys (lexing is XML).
– generate native Vala AST instead of text (I currently feed a generated source to the Vala parser)

Again.. why would I do it? It’s my first open-source attempt, I tried to announce it on vala list, I had as little as.. one user, then a couple of _month_ later tried to announce it on gnome-announce.

It seems like nobody’s getting where I’m going or doesn’t want to. Since I have this (mainly MXML) background, I concieved gtkaml as a shortcut for _me_ to learn the _Gtk_ UI widgets and their properties. I used it a couple of times to prototype UI for something that would have been to boring to write in C or in Vala.

I got at maximum 100 downloads per release, starring on gnomefiles and freshmeat. It lowers – the latest version, posted on gnome-announce, had less downloads.
Is it because, not having a well organized parser/code generator, every hacker scares away (from the error messages for example)??

I also get compared with glade/GtkBuilder that often that I explained in a previous post the differences. But still nobody cared.

Can somebody point me their open-source experience? is this normal? (i.e. to have an unique itch and scratch it?)

Written by vlad

August 21, 2008 at 8:00 pm

Posted in gtkaml, ideas, morons

SQL Dependency Injection

with 2 comments

“All problems in computer science can be solved by another level of indirection, but that usually will create another problem.” — David Wheeler, inventor of the subroutine

This is the story of a “framework” that doesn’t exist: SqlDi

Most database frameworks’ authors consider that it is wrong to write SQL in your source code and provide many kinds of wrappers you need to deal with, instead of pure query strings.

Their reasons are along the lines of:

Portability: SQL is not database independent

Sadly, this is true. Although there are ANSI standards – ANSI SQL 99, 2003 etc. – no RDBMS supports all the features from ANSI and most vendors force you to use some of their custom extensions to accomplish things from the standard.

(For example, SQL 2003 introduces ‘window functions’ that could be used instead of the vendor specific ROWNUM, LIMIT or TOP keywords – and the window functions allow you to do even more. Few database systems support SQL 2003 though..)

Abstraction: SQL queries contain details about the storage mechanism

The relational model is a good way of assuring the integrity and non-redundancy of data. JOINs allow you to reverse the normalization process, while keeping the “Single Point of Truth” – storage model. INDEXes allow you to lookup faster.

Having SQL (in your source) that displays all these storage details is bad in practice. Even if your attributes (columns) are not subject to change, the way of retrieving them may be.

For example, if you want to display a (paginated) list of books on a web page, with their authors, a minimal query would be:


SELECT books.title as title, books.date_published as date_published, authors.name as author
FROM
  books
INNER JOIN
  authors
ON
  books.author_id = autors.id
WHERE
  ROW_NUMBER() OVER (ORDER BY books.title) BETWEEN 1 AND 10

This is both an example of SQL that reveals the storage internals and that is not portable (because ROW_NUMBER() is supported by very few ‘chosen’ ones)

Correctness: Query strings are validated only when run against the database

Even if you’re using an editor that knows how to highlight syntax from the SQL strings, you are still prone to mistakes, syntactical or not (e.g. wrong column name).

The usual solution is to write objects that construct your query – here’s an example for Hibernate’s Criteria API


Criteria crit = session.createCriteria(Cat.class);
crit.add( Expression.eq( "color", eg.Color.BLACK ) );
crit.setMaxResults(10);
List cats = crit.list();

This still fails if the “color” string is mistyped, for example.

SqlDi

The proposed SqlDi solution emphasizes some of the positive things about SQL, while working around the above-mentioned issues.

First let’s talk about the “Domain Specific Language” well known meme – SQL is one of the best examples of DSL around, but suddenly everybody wants to write in lolcode like this:

include StarbucksDSL

order = latte venti, half_caf, non_fat, no_foam, no_whip
print order.prepare

As opposed to this ruby Starbucks language I found on the web, SQL has proper syntax rules, and everybody knows at least its basics.

So SqlDI will allow you to write a subset of SQL called SqlDi.

But how about the aforementioned Abstraction ?

In order to decouple the storage model from the application query, SqlDi uses a kind of remote-process inversion of control (RP/IoC) pattern where the actual storage-specific part of the query gets injected in your code by the RDBMS process (instead of your code depending on the storage layout).

Here’s an example of the previous query, re-written in SqlDi:

SELECT title, date_published, author
FROM books_with_authors
LIMIT 1,10

You can already guess the syntax of SqlDi, but I will paste it here anyway:

SELECT attribute_expr [, attribute_expr]* FROM thing [WHERE boolean_expr] [LIMIT n [,m]]

Now how does the ‘thing’ called ‘books_with_authors’ got injected?

Simple, it’s part of the database “configuration file” (the script that creates the schema):

CREATE TABLE authors
( id INTEGER PRIMARY KEY, name VARCHAR(80), birthday DATE);

CREATE TABLE books
( id INTEGER PRIMARY KEY, date_published DATE, author_id INTEGER FOREIGN KEY
  REFERENCES authors(id) );

CREATE VIEW books_with_authors AS
SELECT books.title as title, books.date_published as date_published, authors.name as author
FROM books INNER JOIN authors ON books.author_id = autors.id;

Now, if some other time you discover that one book can have multiple authors, all you have to do is to re-write the storage-dependent part of the query.

But how does SqlDi provide Portability ?

SqlDi is a parser for a subset of SQL – you have already seen the SELECT statement syntax. The various functions dealing with numbers, dates and strings, even the operators (such as string concatenation) are defined one way or another in SqlDi and translated into the native dialect.

For example, the LIMIT keyword is translated into ROWNUM, LIMIT, TOP or ROW_NUMBER() depending on the used database.

This translation is accomplished in two steps:

  • First, the query is tokenized and parsed, and an object which represents the root of the Abstract Syntax Tree is constructed.
  • Second, the AST spills out the native dialect – based on some form of run-time pluggable dialect writers (plus ‘drivers’)

In this way you’ll never need to change the “SELECT x FROM y WHERE …” from your source, even if you change the used database. (You do, however, need another schema creation script).

Yeah, but how about the compile-time Correctness ?

This is the most interesting part.

SqlDi is implemented in D (because I don’t know LISP), and the first step (parsing) is accomplished at compile-time, through the use of mixin() statement. The output of parsing is code, like this:

SqlDi.Select ( "books_with_authors", null /*the where clause*/, 1, 10 );

For example, if you have WHERE name LIKE ‘Richard %’, instead of a null you will have SqlDi.Like ( SqlDi.Column(“name”), “Richard %” ) – where “SqlDi” is the package name.

Of course you could write this yourself if you are a hardcore SqlDi user;)

Conclusion

To summarize what we achieved:

  • you can write SQL in your source code,
  • it will be database independent and
  • you will get compile-time syntax checking
  • bonus points: the DBA will become your best friend

What we didn’t do:

  • no object mapping was even considered
  • we didn’t enter into details of INSERTs and UPDATEs
  • SqlDi does not exist

Written by vlad

April 24, 2008 at 8:28 pm

Posted in ideas, Uncategorized

gtkaml 0.1 released!

with 3 comments

Well, actually gtkaml 0.1.0.1…

I had problems with autotools.. my examples installed in the /usr/bin directory – which was uncool.

And the googlecode decided that no release should be deleted – so I bumped the version up and released once again..

Anyway, in case you don’t know, gtkaml is an XML preprocessor that transforms all your container/widget descriptions into Vala. I proposed myself to build this project some time ago.

And now it’s done! Next thing I know, I’ll be that proficient in GTK that I will event start my own Norton-Commander clone in gtkaml + vala!

So here it is, the announce: giocommander will be a NortonCommander clone written in GTK with the new GIO api that’s bundled in GLIB.

The Linux version will be featureful, if you have GVFS installed, and the Windows version will be “O.K”

You know why? Because it’s simpler to develop a software that’s featureful in Linux and crippled in Windows:-P

And if you add enough features, you might end up installing Linux just to get them! (What, you never installed Windows just to get Photoshop or Autocad? Why not the other way?!?)

Ok, jokes aside, let’s get back to the drawing board..

Written by vlad

February 3, 2008 at 8:55 pm

Posted in gtkaml, ideas

An Xml User-interface Language for GTK+ (dubbed ‘gtkaml’)

with 5 comments

(update 2008-02-03: gtkaml 0.1 has been released and the syntax is changed too)

Every major UI toolkit today has some XML counterpart, don’t they?

Mozilla has XUL (pronounced ‘zool’) which is used along with JavaScript;
.NET has XAML (pronounced ‘zammel’) which is used with/instead of C# or other .NET language;
Flex has MXML which is used with ActionScript 3;
PHP-Gtk has Gul which is used with PHP, of course.

All these are markup languages have in common is that they are using the structure/hierarchy of the XML source file to build the hierarchy of the UI widgets/containers.

Plus, they separate UI from controllers/algorithms etc.

Great idea, I’d say.

What I didn‘t find, was a such XUL/XAML for GTK+ in C. Why search that?

Well, GTK is by far the most popular Linux widget toolkit. Starting from GIMP, Gaim (now pidgin), Inkscape, AbiWord and continuing with the GNOME desktop environment, all these use GTK+. Plus, wxWidgets and SWT use this on Linux (and native controls on Windows).

But GTK+ has one big disadvantage (which is its advantage too): it is written in C!

Why is this an advantage? It has language bindings for ruby, perl, python, C#, C++, Java and any language you can find in the world (Ada?). If you don’t know already, operating systems are written in C and the only generally compatible/available calling convention is the C one!

The disadvantage? It’s Object-Oriented. It mimics inheritance through appends to structs (GObject). It has a considerable steep learning curve, because of all the boilerplate code written for a Class.

Now, gtkaml aims to do two things:

  1. write the inheritance boilerplate for you
  2. write the UI for you

So, given that you provide the following gtkaml code (this is just an example):

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<GtkVBox xmlns="http://b100dian.com/gtkaml/0.01/gtk+-2.0">
	<GtkLabel translatable="true" use_markup="true"
		label="&lt;b&gt;Dialog Box Title Here&lt;/b&gt;" expand="false">
	</GtkLabel>
	<GtkNotebook can_focus="true" tab_vborder="1">
		<tabs>
			<GtkLabel label="label for a tab" fill="FALSE"/>
			<GtkCheckButton label="check button" active="TRUE" fill="FALSE" />
		</tabs>
		<GtkLabel>
			<label>label
multiline</label>
		</GtkLabel>
	</GtkNotebook>
	<GtkHButtonBox expand="false">
		<GtkButton has_default="true" label="_abort"/>
		<GtkButton label="gtk-redo" stock="true"/>
		<GtkButton label="fail"/>
	</GtkHButtonBox>
</GtkVBox>

It will generate the .h and .c file for you that will compile into this widget:gtkaml1.png

Now, all seems perfect but:

  • how one defines signals? How does one consume signals? in C??
  • how does one declare properties? with tags like <gboolean id=”my_boolean” /> and <gchar id=”my_character” /> ?

The answer could come from Vala!

Vala is an OO language especially made around the GObject mechanism (others don’t fit exactly). The .vala file is compiled in.. exactly an .h and .c fil:

So I am taking this a step ahead, and say that gtkaml would be valaml and would compile the .valaml into .vala!

//example Vala source
using Gtk;

public class MyWidget : VBox {

private Label label5;
private Notebook notebook1;
private Label label4;
private Label label2;
private CheckButton check_button;
private VBox empty_notebook_page;
private HButtonBox hbuttonbox1;
private Button abort;
private Button button2;
private Button fail;

construct {
label5 = new Label(“Dialog Box Title Here“);
label5.use_markup = true;
this.pack_start(label5, false, false, 0 );

notebook1 = new Notebook();
this.pack_start(notebook1, true, true, 0 );

label4 = new Label(“label\nmultiline”);
notebook1.add( label4 );

label2 = new Label(“Page 1”);
notebook1.set_tab_label( notebook1.get_nth_page(0), label2);

empty_notebook_page = new VBox( false, 0 );
notebook1.add( empty_notebook_page );

check_button = CheckButton.with_label(“check button”);
check_button.active = true;
notebook1.set_tab_label( notebook1.get_nth_page(1), check_button );

hbuttonbox1 = new HButtonBox();
this.pack_start( hbuttonbox1, false, true, 0 );

abort = Button.with_mnemonic(“_Abort”);
abort.can_default = true;
hbuttonbox1.add( abort );

button2 = Button.from_stock(“gtk-redo”);
button2.can_default = true;
hbuttonbox1.add( button2 );

fail = Button.with_mnemonic(“fail”);
fail.can_default = true;
hbuttonbox1.add( fail );
}
}
Long live C!

Written by vlad

November 26, 2007 at 8:51 pm

Posted in ideas