Discussion:
[Carbonado-interest] Query limits
Donald Schneider
2007-06-03 16:08:40 UTC
Permalink
I'm not keen on introducing new syntax to the filter string syntax.
It's obscure, hard to discover/document. Given that this is
reasonably advanced, wouldn't it be more natural to introduce a new
Filter class for this purpose?

-----Original Message-----
Brian S O'Neill
Sent: Sunday, May 27, 2007 10:28 AM
Subject: [Carbonado-interest] Query limits
I'm trying to come up with a way to specify how Carbonado queries can
limit their results. For example, "the first ten rows" or "rows 20 to
30". This can be emulated with a Cursor using the skip and close
operations, but many potential optimizations are not available with
this
technique.
I started experimenting with a Query.view(start, end) method, which
returns a new Query limited over a range of rows. This doesn't feel
right to me, since the values to the view aren't parameterized. Also,
I've been struggling with the implementation. The current query engine
assumes that a query is only ever a filter and sort specification.
Tacking on the view means that I've introduced a complex query
composition feature, which is not something I'd like to deal with yet.
For example, the results of a view could be filtered and sorted some
more, and then this be used to construct another view.
Looking at the standard way of limiting results in SQL gave me a new
idea. SQL 2003 supports a ROW_NUMBER window function, which allows you
to derive a row number. With this, you can filter out rows using the
usual WHERE expression. Following this approach, limiting results
is the
same as a filter, and so the query engine still sees everything as a
filter and sort. The query filter expression syntax I'm thinking of
# Returns only rows 20 to 30 (zero based rows)
storage.query("# >= ? & # < ?").with(20).with(30);
The current filter syntax only allows property names on the left
side of
the operator, and the '#' represents a special row number property,
which is only valid for the filter. I cannot think of a good way of
exposing this number in the cursor results, since they're just
Storables. You can certainly reproduce the number if needed, by
counting
results yourself as you iterate.
Does this idea sound crazy? Is it confusing? Suggestions?
----------------------------------------------------------------------
--
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Carbonado-interest mailing list
https://lists.sourceforge.net/lists/listinfo/carbonado-interest
----
“Necessity is the plea for every infringement of human freedom. It is
the argument of tyrants, it is the creed of slaves.”
—William Pitt, House of Commons, November 18, 1783
Brian S O'Neill
2007-06-03 17:00:09 UTC
Permalink
After thinking about this strategy some more, I realized it won't work
as a filter anyhow. All the operators currently available in the filter
are commutative. The order in which an 'and' or 'or' evaluates is not
relevant. This is very important for supporting filter transformations,
which the query engine needs to do.

A filter like "# < ? & a = ?" is not the same as "a = ? & # < ?" because
the row numbers are not fixed. They are affected by the outcome of the
"a = ?" filter.

I was trying to avoid adding support for query composition in order to
support query limits. Proper query composition is the only way to go,
and it is consistent with the SQL way of limiting results as well.
Post by Donald Schneider
I'm not keen on introducing new syntax to the filter string syntax.
It's obscure, hard to discover/document. Given that this is
reasonably advanced, wouldn't it be more natural to introduce a new
Filter class for this purpose?
-----Original Message-----
Brian S O'Neill
Sent: Sunday, May 27, 2007 10:28 AM
Subject: [Carbonado-interest] Query limits
I'm trying to come up with a way to specify how Carbonado queries can
limit their results. For example, "the first ten rows" or "rows 20 to
30". This can be emulated with a Cursor using the skip and close
operations, but many potential optimizations are not available with
this
technique.
I started experimenting with a Query.view(start, end) method, which
returns a new Query limited over a range of rows. This doesn't feel
right to me, since the values to the view aren't parameterized. Also,
I've been struggling with the implementation. The current query engine
assumes that a query is only ever a filter and sort specification.
Tacking on the view means that I've introduced a complex query
composition feature, which is not something I'd like to deal with yet.
For example, the results of a view could be filtered and sorted some
more, and then this be used to construct another view.
Looking at the standard way of limiting results in SQL gave me a new
idea. SQL 2003 supports a ROW_NUMBER window function, which allows you
to derive a row number. With this, you can filter out rows using the
usual WHERE expression. Following this approach, limiting results
is the
same as a filter, and so the query engine still sees everything as a
filter and sort. The query filter expression syntax I'm thinking of
# Returns only rows 20 to 30 (zero based rows)
storage.query("# >= ? & # < ?").with(20).with(30);
The current filter syntax only allows property names on the left
side of
the operator, and the '#' represents a special row number property,
which is only valid for the filter. I cannot think of a good way of
exposing this number in the cursor results, since they're just
Storables. You can certainly reproduce the number if needed, by
counting
results yourself as you iterate.
Does this idea sound crazy? Is it confusing? Suggestions?
----------------------------------------------------------------------
--
-
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Carbonado-interest mailing list
https://lists.sourceforge.net/lists/listinfo/carbonado-interest
----
“Necessity is the plea for every infringement of human freedom. It is
the argument of tyrants, it is the creed of slaves.”
—William Pitt, House of Commons, November 18, 1783
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Carbonado-interest mailing list
https://lists.sourceforge.net/lists/listinfo/carbonado-interest
Loading...