RE: [c-jdbc] Re: Horizontal scalability

[LAUGIER Frédéric <FLAUGIER@xxxxxxxxxxxxxxxxxxxxx>]

Title: RE: [c-jdbc] Re: Horizontal scalability

Hello,

I've just finished to implement the DispathRead tag, I'll be committing tonight.
I'm starting to understand (well ...I think so ...) how the request gets distributed in C-JDBC.
I've added a DispatchRead tag just like you said, in the load balacing tag for RAIDb1, RAIDb1ec, RAIDb2 and RAIDb2ec , yet I think the 'dispatching read request' concept might conflict with the error checking or at least be redundant, since i beleive the error checking will give the ability to dispatch read request to ensure data consistency (am I wrong ?)

I've added the tag like another policy for the loadbalancer (copied from CreateTable policy).
I've added two test scenario too (one for RAIDb1, one for RAIDb2), for now the only thing that is tested is the configuration file read process. (Well, as a matter of fact it's the only thing to test right now ...).

Well tell me if i've done anything wrong with the code or how things should be implemented, where classes should be ... well anything.

I'll do my best to continue working on this and finish as soon as possible.

By the way, how do you do to send the commit list to this mailing list ? is it automated ? (well I guess if it is I'll have my answer quite soon ...)

Hopes this helps,
Fred.

-----Message d'origine-----
De : Nicolas Modrzyk [mailto:Nicolas.Modrzyk@xxxxxxxxxxxx]
Envoyé : mercredi 26 mai 2004 09:30
À : LAUGIER Frédéric; c-jdbc-commits@xxxxxxxxxxxxx
Objet : Re: [c-jdbc] Re: Horizontal scalability

Hi Fred,

Sorry for the delay ...
Usually we use eclipse for editing code. If you checkout the project,
all the settings should be directely imported, and then you can format
the code like the other sources files.

You can commit things in the main development tree anytime when it's
compiling with the whole rest of the code.
We also need a test like the ones in
$CJDBC}/test/org/objectweb/c-jdbc/scenario/.
Almost all tests are working and we would like to use those ones for
regression testing really soon.
If the commit is not working we just roll it back so there's no problem !

The main JDK to target with is 1.3. We have been using the Regexp
libraries from Apache so you can already use them in 1.3 although :) the
code won't compile with a 1.3 compiler, but it will work with a JRE 1.3

I don't know whether the DistributedRule you want to do should be just
like the rewriting rule.
In the C-JDBC controller, the loadbalancer element is responsible for
dispatching requests to backends.
We may just want to add a new loadbalancer attribute for Raidb1 and
Raidb1 that dispatch read request with Regexp.
In Raidb1 and Raidb2 there are:
- RoundRobin
- LeastPendingRequestFirst
- WeightedRoundRobin
What we need is a test on the executeReadRequest on these loadbalancers
that, if matched, would send the query to all backends ( or a subset of
them if we define nested configuration tag )

So maybe in:
<!ELEMENT RAIDb-1 (WaitForCompletion?, (RAIDb-1-RoundRobin |
RAIDb-1-WeightedRoundRobin | RAIDb-1-LeastPendingRequestsFirst))>
add a DispatchReadRule tag like
<!ELEMENT RAIDb-1 (WaitForCompletion?, ,
DispatchReadRule?,(RAIDb-1-RoundRobin | RAIDb-1-WeightedRoundRobin |
RAIDb-1-LeastPendingRequestsFirst))>

and
<!ELEMENT DispatchReadRule (BackendName*)>
<ATTLIST DispatchReadRule
    readRegexp="select \0 from \1 where id=\3"
 >

backendName element being already defined as:
<!ELEMENT BackendName EMPTY>
<!ATTLIST BackendName
    name   CDATA #REQUIRED
 >

and maybe we should dispatch on all backends if none are selected ...
and do the same for Raidb2.

What do you think ?
Any question is welcome, it's even better if you ask them because it
makes the project going forward !

Nicolas,

LAUGIER Frédéric wrote:

> Hello, Nicolas
>
> I've started looking in the code to see how the 'distribution query'
> could be added.
> I was thinking of a tag named 'DistributeRule'. The first question i
> have is what is the JDK version to use ?
> I'd like to use the java.util.regex.Pattern to implement a regular
> _expression_ rule. Since this class exists only in the 1.4 JDK, is that ok ?
>
> Another thing, i beleive this rule could be used wether or not the
> controllers are in a distributed environment. But only with RAIDb-1
> databases, or with RAIDb-2, etc ... databases that have sequences
> spread over several backends. The question is shall we let the user
> decide where the query is to be dispatched (using a nested
> configuration tag where he can list the backends) or shall the
> application decide for itself, using the schema definition that is
> read when the backends are enabled. I didn't look at this part of the
> code at this point, but maybe that would be a possible risk, espcially
> if the controller can't read the database metadata for any reason.
>
> At this point i've coded the DistributedRule tag, added a
> distriutedRule ArrayList to the
> org.objectweb.cjdbc.controller.backend.DatabaseBackend, added a
> package org.objectweb.cjdbc.controller.backend.distribute (well i've
> done just like for the rewriting tag)
>
> I didn't have the time yet to test the whole thing so i didn't commit
> anything. But maybe you would like to see if everything is coded the
> way you want before i commit anything ? or maybe i could use another
> branch so that i will not commit things in your development tree. In
> fact it's the first time where i can participate in an open source
> project so excuse me if some of my questions are ... err ... stupid ...
>
> Fred.
>
> -----Message d'origine-----
> De : Nicolas Modrzyk [mailto:Nicolas.Modrzyk@xxxxxxxxxxxx]
> Envoyé : mardi 25 mai 2004 09:34
> À : flaugier@xxxxxxxxxxxxxxxxxxxxx
> Cc : c-jdbc@xxxxxxxxxxxxx
> Objet : Re: [c-jdbc] Re: Horizontal scalability
>
>
> Hi Fred,
>
> I've added you to the CVS list ... so you can checkout and commit
> changes.
> If you don't have time to do the changes it's fine,we don't blame people
> for that !
>
> In this case, just fill the details for a new task and assign it to the
> both of us.
>
> We'd be really happy if you'd help the project !
> If you need any help to get started, let us know.
>
> Thanks,
>
> Nicolas,
>
> flaugier@xxxxxxxxxxxxxxxxxxxxx wrote:
>
> >Hi, i'm already registered as 'hint' (id 2502, same email address).
> >
> >The driver throws a SQLException with the message 'setReadOnly cannot
> be called
> >while in the middle of a transaction.' But it seems like it's coded
> this way,
> >maybe because of the way C-JDBC handles transaction, it wouldn't be a
> good
> >thing if a connection could change it's readonly behavior in the
> middle of a
> >transaction ?
> >
> >I'll see what i can do with the tag to enable a select to be
> dispatched on all
> >backends with a regexp rule maybe, I can't tell you exactly when I'll
> be able
> >to do this because we have a deadline by the end of the week and
> currently not
> >everything is going so well over here ...
> >But since C-JDBC is part of the same project I think I'll be able
> take a close
> >look.
> >
> >I would be really glad if I can help you in any way !
> >Fred.
> >
> >
> >
> >------------------------------------------------------------------------
> >
> >
> >
> >
>
>