Tugger the SLUGger!SLUG Mailing List Archives

Re: [SLUG] today's scary thought


On 16 July 2010 17:15, Daniel Pittman <daniel@xxxxxxxxxxxx> wrote:

> Jake Anderson <yahoo@xxxxxxxxxxxxxxx> writes:
> > On 15/07/10 16:14, Daniel Pittman wrote:
>
> [...]
>
> > We cant be the first people to come across this "branch office" scenario.
>
> Nope.  Lots of people have, and wished there was a good solution, but it is
> a
> *really* hard problem.  The difficulty curve in fixing it looks like a
> backward L, basically: the most trivial bit is trivial, then the problem
> more
> or less instantly gets insanely hard.
>

I think it's important to mention why it's hard: the speed of light is
constant, so as distance increases, latency increases.  The longer it takes
to round trip, the more time you spend waiting for confirmation that your
"atomic" operation (eg writing new data) has committed at both ends, and the
more chance you have of conflicts arriving when both sides try to write (as
your window of opportunity is now wider). Add to that the difficulties in
sequencing events in a distributed system (i.e. how do both ends know the
other end's clock is accurate enough (but then see also vector clocks)) and
you see that suddenly there's a whole bunch of expensive problems to solve.

So people work around it by sacrificing some of the things they need, like
atomicity, or consistency, or currency; this is what they talk about in
these newfangled "NoSQL" cloud storage systems.   The less you need to
write, or check (like, say, a foreign key to mainttain referential
integrity) the faster you can write and replicate this data.  If you don't
cae that the immediate data is inconsistent at one end, but will be "soon",
then you don't have to wait for the write to sync.  But sometimes you can't
make those sacrifices, such as in the case of trying to maintain POSIX
filesystem semantics.  The cloudy things like Google's GFS aren't POSIXy at
all for just this reason.  They like appends, but not overwrites, for
example.

Hope this helps. (Understanding, that is -- I know it doesn't help solve
anything.)