SLUG Mailing List Archives
Re: [SLUG] tailing, following and filtering
- To: Penedo <penedo@xxxxxxxxx>
- Subject: Re: [SLUG] tailing, following and filtering
- From: Matthew Hannigan <mlh@xxxxxxxxxx>
- Date: Thu, 23 Nov 2006 06:25:32 +1100
- Cc: SLUG <slug@xxxxxxxxxxx>
- User-agent: Mutt/126.96.36.199i
On Thu, Nov 23, 2006 at 05:10:14PM +1100, Penedo wrote:
> On 23/11/06, Matthew Hannigan <mlh@xxxxxxxxxx> wrote:
> >You don't get the output in as timely a fashion;
> >it comes with spurts as the buffer is filled up
> >then written out.
> What buffer? tail's manual doesn't mention anything about buffers but
grep's buffer, the normal stdio buffer I guess
> my experience is that you get individual lines even then its output is
> piped (i.e. even when the output is about 80 bytes long, i.e. much
> less than any reasonable buffer size except a line buffer).
> With grep you can force line-buffer with --line-buffering, if you
> think this is the problem.
> Maybe what you see is tail's sleep between retries. The default is 1
> but see below.
> >Try this out:
> > while true ; do s=$RANDOM; echo $s; usleep $s; done >crap &
> > tail -f crap |grep 3
> > tail -f crap | grep 3 | grep -v 8
> I get output every more-or-less 1 second, which makes me suspect that
> my "tail's sleep(1)" explanation is indeed the reason. Adding "-s 0"
> gives me continuous output. And that's even without forcing
Doesn't me; I get it spurty; like 50 seconds or so between output.
> --line-buffering on grep. So far I thing this confirms my theory.
With --line-buffered on the first grep I get it continuous.
If only the second grep is --line-buffered, then it goes spurty again.
That points to the first grep buffering by default, to me.
> Another option to test this might be to "strace -tt -f ...", but so
> far I wasn't convinced by your explanation so I'll leave it up to you
> to experiment :^).
> Another experiment - slow down the data writing into "crap" and see
> whether you get individual lines of 5-6 bytes each - this again should
> prove the "buffer theory" wrong, IMHO.
It's possible it's a difference in kernels/shells/something else,
but that's certainly unexpected.
I didn't even know about tail's -s option. That's what I get
for spending too much time in a non-GNU wilderness.
I still think there's a case for a 'less' like tool, or
less itself to do some filtering, modifiable on the fly.
I'm sure there's some perl module/program to do it.