[Supervisor-users] Startup delay parameter?

[Lennart Regebro]

Hiya!

When starting my zeo-clients I don't want them to all connect at the
same time, but stagger it. I found a buildout with a sleep script that
does this, but it fails when killing the processes during shutdown.
(http://rpatterson.net/blog/stagger-supervisord)

Is there another way of doing this? :-)

I took a quick look at the process.py code and it does not seem *very*
hairy to implement a startupdelay parameter. Is there any reason not
to implement this? If not do you want me to try?

[Chris McDonough]

Post by Lennart Regebro
Hiya!
When starting my zeo-clients I don't want them to all connect at the
same time, but stagger it. I found a buildout with a sleep script that
does this, but it fails when killing the processes during shutdown.
(http://rpatterson.net/blog/stagger-supervisord)
Is there another way of doing this? :-)
I took a quick look at the process.py code and it does not seem *very*
hairy to implement a startupdelay parameter. Is there any reason not
to implement this? If not do you want me to try?

If you've got it in, you sure...

- C

[Roger Hoover]

In that blog post, it says there was a way to avoid the need for
staggering. Is there a real use case for this?

If a startdelay parameter is introduced, should there also be a delay
applied between starting multiple processes in the same process group
(numprocs > 1)?

What about a global setting telling supervisord to start everything serially
(don't fork the next process until the previous one is up or failed (as
determined by it's startsecs command)?

Post by Chris McDonough

Post by Lennart Regebro
Hiya!
When starting my zeo-clients I don't want them to all connect at the
same time, but stagger it. I found a buildout with a sleep script that
does this, but it fails when killing the processes during shutdown.
(http://rpatterson.net/blog/stagger-supervisord)
Is there another way of doing this? :-)
I took a quick look at the process.py code and it does not seem *very*
hairy to implement a startupdelay parameter. Is there any reason not
to implement this? If not do you want me to try?

If you've got it in, you sure...
- C
_______________________________________________
Supervisor-users mailing list
http://lists.supervisord.org/mailman/listinfo/supervisor-users

[Chris McDonough]

Post by Roger Hoover
In that blog post, it says there was a way to avoid the need for
staggering. Is there a real use case for this?
If a startdelay parameter is introduced, should there also be a delay
applied between starting multiple processes in the same process group
(numprocs > 1)?
What about a global setting telling supervisord to start everything
serially (don't fork the next process until the previous one is up or
failed (as determined by it's startsecs command)?

Something like that would be pretty useful. We've always known that
there's some dependency map lurking here obviously. e.g.

[program:foo]
depends = bar

... we've just never got around to doing it.

- C

Post by Roger Hoover

Post by Lennart Regebro
Hiya!
When starting my zeo-clients I don't want them to all connect at the
same time, but stagger it. I found a buildout with a sleep script

that

Post by Lennart Regebro
does this, but it fails when killing the processes during shutdown.
(http://rpatterson.net/blog/stagger-supervisord)
Is there another way of doing this? :-)
I took a quick look at the process.py code and it does not seem

*very*

Post by Lennart Regebro
hairy to implement a startupdelay parameter. Is there any reason not
to implement this? If not do you want me to try?

If you've got it in, you sure...
- C
_______________________________________________
Supervisor-users mailing list
http://lists.supervisord.org/mailman/listinfo/supervisor-users

[Roger Hoover]

Should the dependency map come from the "priority" parameters? So no order
would be guaranteed for processes with the same priority number but would be
guaranteed for different priorities.

Post by Chris McDonough

Post by Roger Hoover
In that blog post, it says there was a way to avoid the need for
staggering. Is there a real use case for this?
If a startdelay parameter is introduced, should there also be a delay
applied between starting multiple processes in the same process group
(numprocs > 1)?
What about a global setting telling supervisord to start everything
serially (don't fork the next process until the previous one is up or failed
(as determined by it's startsecs command)?

Something like that would be pretty useful. We've always known that
there's some dependency map lurking here obviously. e.g.
[program:foo]
depends = bar
... we've just never got around to doing it.
- C

Post by Roger Hoover

Post by Lennart Regebro
Hiya!
When starting my zeo-clients I don't want them to all connect at the
same time, but stagger it. I found a buildout with a sleep script

that

Post by Lennart Regebro
does this, but it fails when killing the processes during shutdown.
(http://rpatterson.net/blog/stagger-supervisord)
Is there another way of doing this? :-)
I took a quick look at the process.py code and it does not seem

*very*

Post by Lennart Regebro
hairy to implement a startupdelay parameter. Is there any reason not
to implement this? If not do you want me to try?

If you've got it in, you sure...
- C
_______________________________________________
Supervisor-users mailing list
http://lists.supervisord.org/mailman/listinfo/supervisor-users

[Lennart Regebro]

Post by Roger Hoover
In that blog post, it says there was a way to avoid the need for
staggering.  Is there a real use case for this?

Yes.

Post by Roger Hoover
If a startdelay parameter is introduced, should there also be a delay
applied between starting multiple processes in the same process group
(numprocs > 1)?

Good question. I have no opinion on this.

Post by Roger Hoover
What about a global setting telling supervisord to start everything serially
(don't fork the next process until the previous one is up or failed (as
determined by it's startsecs command)?

That could work. How does supervisord tell if a process is "up"?

[Lennart Regebro]

Post by Roger Hoover
In that blog post, it says there was a way to avoid the need for
staggering.  Is there a real use case for this?

Yes.

I forgot to add this: Although I can't come up with something that
wouldn't be better served by the "depends" parameter mentioned by
Chris.

[Roger Hoover]

The existing priority parameter would accomplish the same thing as a new
depends parameter, yes? It seems to me like all cases for depends can be
mapped to use priority but not vice-versa. With priorities, you can create
a config where everything that has the same priority gets started
concurrently but each priority level is started after the previous one
completes. That would allow you specify which parts of your config need
start order dependencies and which don't.

Post by Lennart Regebro

Post by Roger Hoover
In that blog post, it says there was a way to avoid the need for
staggering. Is there a real use case for this?

Yes.

I forgot to add this: Although I can't come up with something that
wouldn't be better served by the "depends" parameter mentioned by
Chris.
--
Lennart Regebro: Python, Zope, Plone, Grok
http://regebro.wordpress.com/
[phone]

[Mike Naberezny]

Post by Lennart Regebro
That could work. How does supervisord tell if a process is "up"?

Specifically with regards to starting the process, it is when the process
transitions from the STARTING state to the RUNNING state. You'll see this in
the log at the INFO level.

Regards,
Mike

[Chris McDonough]

Post by Mike Naberezny

Post by Lennart Regebro
That could work. How does supervisord tell if a process is "up"?

Specifically with regards to starting the process, it is when the process
transitions from the STARTING state to the RUNNING state. You'll see this in
the log at the INFO level.

More descriptively, it transitions from STARTING to RUNNING if the process is
still "up" after startsecs have gone by.

- C

[Lennart Regebro]

Post by Chris McDonough
More descriptively, it transitions from STARTING to RUNNING if the process is
still "up" after startsecs have gone by.

Aha. Well, in that case, depends+startsecs and startdelay would be
pretty much the same thing, actually. Although startdelay would be
easier to implement.

Post by Chris McDonough
The existing priority parameter would accomplish the same thing as a new
depends parameter, yes?

You mean that it could be changed to wait for lower priority processes
to get into the running state before starting the higher ones? That
could work too, but doesn't that change the behavior from now?

[Stephan Zeissler]

AFAIK the priority parameter currently has no purpose, so I guess no one
uses it ;)

- Stephan

Post by Lennart Regebro

Post by Chris McDonough
More descriptively, it transitions from STARTING to RUNNING if the process is
still "up" after startsecs have gone by.

Aha. Well, in that case, depends+startsecs and startdelay would be
pretty much the same thing, actually. Although startdelay would be
easier to implement.

Post by Chris McDonough
The existing priority parameter would accomplish the same thing as a new
depends parameter, yes?

You mean that it could be changed to wait for lower priority processes
to get into the running state before starting the higher ones? That
could work too, but doesn't that change the behavior from now?

[Lennart Regebro]

I did some testing today, and starting one client at a time means the
first client is up in about 20-25 seconds instead of in 50 seconds
(when all four clients come online at once in that case). I would like
to start one client first, then haxproxy will notice it's up after 25
seconds and start redirecting requests there. Then the others can take
another 50 seconds if they want. :)

So that's the concrete usecase I have. As you notice, this is no
deal-breaker, I it would just be nice. :-)

[Chris McDonough]

Post by Lennart Regebro
I did some testing today, and starting one client at a time means the
first client is up in about 20-25 seconds instead of in 50 seconds
(when all four clients come online at once in that case). I would like
to start one client first, then haxproxy will notice it's up after 25
seconds and start redirecting requests there. Then the others can take
another 50 seconds if they want. :)
So that's the concrete usecase I have. As you notice, this is no
deal-breaker, I it would just be nice. :-)

The easiest way to get there today is to use Mike's suggestion of a
listener script that uses childutils. The script would be set
autostart=true, and all other processes (or at least the ones you want
to stagger-start) would be started via XML-RPC by the childutils script.

- C

[Mike Naberezny]

Post by Lennart Regebro
When starting my zeo-clients I don't want them to all connect at the
same time, but stagger it. I found a buildout with a sleep script that
does this, but it fails when killing the processes during shutdown.
(http://rpatterson.net/blog/stagger-supervisord)
Is there another way of doing this? :-)

I think it would be nice for us to come up with a little dependency system
like what's being discussed. I would like to suggest an alternative that may
help you in the meantime.

You can set all of your subprocesses to autostart=False, and then create your
bootstrap subprocess with a small Python script of your own. You can set
autostart=True on that one.

There is a module included with Supervisor called supervisor.childutils that
provides a very convenient way for processes running under Supervisor to get
an xmlrpclib.ServerProxy to Supervisor.

In your bootstrap script, you can use our XML-RPC interface to start your
subprocesses. The methods for starting and stopping subprocesses have a
"wait" parameter that controls whether Supervisor will block the XML-RPC
client until the operation has finished or not. You can control the startup
as you like in not many lines of Python.

Regards,
Mike