[Supervisor-users] Dynamic numprocs ? Possible ?

[Simone Fumagalli]

Hello.

My scenario:

- I've a web app that send messages to a supervisord tube and a
worker that consume those messages. Nothing special. Things are
working well.
- The worker takes around 50 seconds to process a message
- Messages are sent to the queue mostly all at the same time (this is
because the web app is used for short periods of time by a lot of
users)

My doubt is about the value of numprocs to put in the config file. If
I put it too high my system waste resources for 99% of the day if I
put it too low the queue takes long time to be processed.

How would you handle this situation ? I'm just wondering if is there
a way to have a dynamic value for numprocs. Something like "If there
are more than 10 messages in the queue than start another process"

Thank

--
Simone

[Calvin Hendryx-Parker]

Hi Simone,

One thing you could do would be to have multiple process groups sitting with autostart false and have an event that can measure some metric and turn them on and off as needed. I have an example of that from a talk I did a few years back here:

https://presentations.sixfeetup.com/2010PyOhioSupervisorTalk/

Cheers,
Calvin
--
Six Feet Up, Inc. | Where sophisticated web projects thrive
Direct Line: [phone] x602
Email: ***@sixfeetup.com
Try Plone 4 Today at: http://plone4demo.com

Post by Simone Fumagalli
Hello.
- I've a web app that send messages to a supervisord tube and a
worker that consume those messages. Nothing special. Things are
working well.
- The worker takes around 50 seconds to process a message
- Messages are sent to the queue mostly all at the same time (this is
because the web app is used for short periods of time by a lot of
users)
My doubt is about the value of numprocs to put in the config file. If
I put it too high my system waste resources for 99% of the day if I
put it too low the queue takes long time to be processed.
How would you handle this situation ? I'm just wondering if is there
a way to have a dynamic value for numprocs. Something like "If there
are more than 10 messages in the queue than start another process"
Thank
--
Simone
_______________________________________________
Supervisor-users mailing list
https://lists.supervisord.org/mailman/listinfo/supervisor-users

[Simone Fumagalli]

Hello Calvin.
This setup can solve a bit my problems.

Thanks for sharing

--
Simone


On Fri, Jun 27, 2014 at 3:56 PM, Calvin Hendryx-Parker

Post by Calvin Hendryx-Parker
https://presentations.sixfeetup.com/2010PyOhioSupervisorTalk/