Is KQueue still broken on Darwin?

SM Ryan <wyrmwif RemoveThis @tango-sierra-oscar-foxtrot-tango.fake.org> writes:

> I have a program that gets in an uninterruptible wait for
> hours and days. I'm using sockets and kqueue. I have never
> had problems with sockets before, but kqueue used to crash
> the kernel. Does anyone know if kqueue still borks the machine?

Never heard about it crashing the kernel before, but kqueue received a
substantial overhaul in Tiger. Spotlight uses it to index updated files,
Finder to update its display immediately, etc.

sherm--

--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net

In article <m27iscyqon.fsf DeleteThis @local.wv-www.com>,
Sherm Pendley <spamtrap DeleteThis @dot-app.org> wrote:

> SM Ryan <wyrmwif DeleteThis @tango-sierra-oscar-foxtrot-tango.fake.org> writes:
>
> > I have a program that gets in an uninterruptible wait for
> > hours and days. I'm using sockets and kqueue. I have never
> > had problems with sockets before, but kqueue used to crash
> > the kernel. Does anyone know if kqueue still borks the machine?
>
> Never heard about it crashing the kernel before, but kqueue received a
> substantial overhaul in Tiger. Spotlight uses it to index updated files,
> Finder to update its display immediately, etc.

I know about Finder, but do you have any reference for Spotlight using
kqueue? That seems really wrong and/or fragile.

G

Gregory Weston <uce.TakeThisOut@splook.com> wrote:
> In article <m27iscyqon.fsf.TakeThisOut@local.wv-www.com>,
> Sherm Pendley <spamtrap.TakeThisOut@dot-app.org> wrote:
>
>> SM Ryan <wyrmwif.TakeThisOut@tango-sierra-oscar-foxtrot-tango.fake.org> writes:
>>
>> > I have a program that gets in an uninterruptible wait for
>> > hours and days. I'm using sockets and kqueue. I have never
>> > had problems with sockets before, but kqueue used to crash
>> > the kernel. Does anyone know if kqueue still borks the machine?
>>
>> Never heard about it crashing the kernel before, but kqueue received a
>> substantial overhaul in Tiger. Spotlight uses it to index updated files,
>> Finder to update its display immediately, etc.
>
> I know about Finder, but do you have any reference for Spotlight using
> kqueue? That seems really wrong and/or fragile.

It is wrong. Spotlight uses fsevent, a private API which is presumably
related to kqueue in some fashion. Kqueue simply wouldn't work for
spotlight since it doesn't allow monitoring entire trees of a filesystem
without opening a file descriptor on every directory within, which is
impractical when doing the entire hard drive.

For more details on what Spotlight is doing, see this page:

http://www.kernelthread.com/software/fslogger/

--
Michael Ash
Rogue Amoeba Software

Gregory Weston <uce.RemoveThis@splook.com> writes:

> In article <m27iscyqon.fsf.RemoveThis@local.wv-www.com>,
> Sherm Pendley <spamtrap.RemoveThis@dot-app.org> wrote:
>
>> SM Ryan <wyrmwif.RemoveThis@tango-sierra-oscar-foxtrot-tango.fake.org> writes:
>>
>> > I have a program that gets in an uninterruptible wait for
>> > hours and days. I'm using sockets and kqueue. I have never
>> > had problems with sockets before, but kqueue used to crash
>> > the kernel. Does anyone know if kqueue still borks the machine?
>>
>> Never heard about it crashing the kernel before, but kqueue received a
>> substantial overhaul in Tiger. Spotlight uses it to index updated files,
>> Finder to update its display immediately, etc.
>
> I know about Finder, but do you have any reference for Spotlight using
> kqueue? That seems really wrong and/or fragile.

I can't recall where I first read that, but at the time I didn't know too
much about kqueue and simply accepted it as read. Googling for "spotlight
kqueue" wasn't very enlightening - lots of people saying it uses it, and
lots of people saying it doesn't.

Now that you mention it though, it does seem wrong; to use kqueue, Spotlight
would have had to register interest in every last file and directory, which
I can't imagine that it does.

sherm--

--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net

Gregory Weston <uce DeleteThis @splook.com> wrote:
# In article <13277khfed5tv24 DeleteThis @corp.supernews.com>,
# SM Ryan <wyrmwif DeleteThis @tango-sierra-oscar-foxtrot-tango.fake.org> wrote:
#
# > I have a program that gets in an uninterruptible wait for
# > hours and days. I'm using sockets and kqueue. I have never
# > had problems with sockets before, but kqueue used to crash
# > the kernel. Does anyone know if kqueue still borks the machine?
#
# It never has for me. What's the nature and circumstance of the
# misbehavior you're seeing?

It hangs on customer machines not mine where I cannot examine it
too closely (no gdb, etc). It monitors the file system, running
kqueue on up to 2000 disk inodes. It has a unix domain socket
to another process. When I look at with ps -x the status is Us.
If I kill the companion process, one of the threads will abort
the entire process on a broken pipe; then the status is Es and
it's still hung.

This can persist for days, until someone notices it. The only
recovery I know of is a reboot. I do know kqueue panicked 10.3.
I wonder if it's still broken or 10.4.9 broke it.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Why are we here?
whrp

In article <1327i8uinfdkk99 DeleteThis @corp.supernews.com>,
SM Ryan <wyrmwif DeleteThis @tango-sierra-oscar-foxtrot-tango.fake.org> wrote:

> Gregory Weston <uce DeleteThis @splook.com> wrote:
> # In article <13277khfed5tv24 DeleteThis @corp.supernews.com>,
> # SM Ryan <wyrmwif DeleteThis @tango-sierra-oscar-foxtrot-tango.fake.org> wrote:
> #
> # > I have a program that gets in an uninterruptible wait for
> # > hours and days. I'm using sockets and kqueue. I have never
> # > had problems with sockets before, but kqueue used to crash
> # > the kernel. Does anyone know if kqueue still borks the machine?
> #
> # It never has for me. What's the nature and circumstance of the
> # misbehavior you're seeing?
>
> It hangs on customer machines not mine where I cannot examine it
> too closely (no gdb, etc). It monitors the file system, running
> kqueue on up to 2000 disk inodes. It has a unix domain socket
> to another process. When I look at with ps -x the status is Us.
> If I kill the companion process, one of the threads will abort
> the entire process on a broken pipe; then the status is Es and
> it's still hung.
>
> This can persist for days, until someone notices it. The only
> recovery I know of is a reboot. I do know kqueue panicked 10.3.
> I wonder if it's still broken or 10.4.9 broke it.

I've got a few thousand users running a program that's been reliant on
kqueue for ... about 2.5 years now. Seems like there's got to me more
detail than "kqueue panicked 10.3."

What I'm kind of getting at is that maybe you're not looking at the
right culprit.

Gregory Weston <uce RemoveThis @splook.com> wrote:
# In article <1327i8uinfdkk99 RemoveThis @corp.supernews.com>,
# SM Ryan <wyrmwif RemoveThis @tango-sierra-oscar-foxtrot-tango.fake.org> wrote:
#
# > Gregory Weston <uce RemoveThis @splook.com> wrote:
# > # In article <13277khfed5tv24 RemoveThis @corp.supernews.com>,
# > # SM Ryan <wyrmwif RemoveThis @tango-sierra-oscar-foxtrot-tango.fake.org> wrote:
# > #
# > # > I have a program that gets in an uninterruptible wait for
# > # > hours and days. I'm using sockets and kqueue. I have never
# > # > had problems with sockets before, but kqueue used to crash
# > # > the kernel. Does anyone know if kqueue still borks the machine?
# > #
# > # It never has for me. What's the nature and circumstance of the
# > # misbehavior you're seeing?
# >
# > It hangs on customer machines not mine where I cannot examine it
# > too closely (no gdb, etc). It monitors the file system, running
# > kqueue on up to 2000 disk inodes. It has a unix domain socket
# > to another process. When I look at with ps -x the status is Us.
# > If I kill the companion process, one of the threads will abort
# > the entire process on a broken pipe; then the status is Es and
# > it's still hung.
# >
# > This can persist for days, until someone notices it. The only
# > recovery I know of is a reboot. I do know kqueue panicked 10.3.
# > I wonder if it's still broken or 10.4.9 broke it.
#
# I've got a few thousand users running a program that's been reliant on
# kqueue for ... about 2.5 years now. Seems like there's got to me more
# detail than "kqueue panicked 10.3."

http://www.google.com/search?hl=en&safe=off&client=safari&rls=en&q=kqu...+kernel

It's well known kqueue was broken on 10.3. I still start whenever
a window fills with grey top to bottom.

# What I'm kind of getting at is that maybe you're not looking at the
# right culprit.

Problem for me is I really don't know what the culprit is
or how to track it down. Something is putting the process
into uninterruptible wait. Usually that means broken I/O,
but attached disks look fine.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
She broke your heart and inadvertently drove men to deviant lifestyles.

In article <13283qfb2h7a4de.DeleteThis@corp.supernews.com>,
SM Ryan <wyrmwif.DeleteThis@tango-sierra-oscar-foxtrot-tango.fake.org> wrote:

> Gregory Weston <uce.DeleteThis@splook.com> wrote:
> # In article <1327i8uinfdkk99.DeleteThis@corp.supernews.com>,
> # SM Ryan <wyrmwif.DeleteThis@tango-sierra-oscar-foxtrot-tango.fake.org> wrote:
> #
> # > Gregory Weston <uce.DeleteThis@splook.com> wrote:
> # > # In article <13277khfed5tv24.DeleteThis@corp.supernews.com>,
> # > # SM Ryan <wyrmwif.DeleteThis@tango-sierra-oscar-foxtrot-tango.fake.org> wrote:
> # > #
> # > # > I have a program that gets in an uninterruptible wait for
> # > # > hours and days. I'm using sockets and kqueue. I have never
> # > # > had problems with sockets before, but kqueue used to crash
> # > # > the kernel. Does anyone know if kqueue still borks the machine?
> # > #
> # > # It never has for me. What's the nature and circumstance of the
> # > # misbehavior you're seeing?
> # >
> # > It hangs on customer machines not mine where I cannot examine it
> # > too closely (no gdb, etc). It monitors the file system, running
> # > kqueue on up to 2000 disk inodes. It has a unix domain socket
> # > to another process. When I look at with ps -x the status is Us.
> # > If I kill the companion process, one of the threads will abort
> # > the entire process on a broken pipe; then the status is Es and
> # > it's still hung.
> # >
> # > This can persist for days, until someone notices it. The only
> # > recovery I know of is a reboot. I do know kqueue panicked 10.3.
> # > I wonder if it's still broken or 10.4.9 broke it.
> #
> # I've got a few thousand users running a program that's been reliant on
> # kqueue for ... about 2.5 years now. Seems like there's got to me more
> # detail than "kqueue panicked 10.3."
>
> http://www.google.com/search?hl=en&safe=off&client=safari&rls=en&q=kqueue+kern
> el+panic&btnG=Search

About 98% of those results don't seem to involve OS X.
A random sample of those that remain suggest that few of them actually
implicate kqueue in kernel panics.
Those that do implicate kqueue in kernel panics describe a fairly
specific scenario that's not particularly common.

That last would be the "more detail" I was interested in. And it doesn't
particularly look like it would be either intermittent or delayed when
it does happen, so it doesn't sound like a good candidate for the source
of your issue.

> It's well known kqueue was broken on 10.3.

To whom? Honestly, I have a hard time accepting that a problem is fairly
described as "well known" if I can support a deployed product that
relies fundamentally on the implicated technology for years without
hearing of it.

G