mugli
4/19/2016 - 1:10 PM
nodejs clustering, zero downtime deployment solutions
nodejs clustering, zero downtime deployment solutions
Clustering: The basics
The trick? pass the file descriptor from a parent process and have the server.listen reuse that descriptor. So multiprocess in their own memory space (but with ENV shared usually)
It does not balance, it leaves it to the kernel.
In the last nodejs > 0.8 there is a cluster module (functional although marked experimental)
- http://nodejs.org/api/cluster.html
- Simple cluster example: https://gist.github.com/dsibilly/2992412
- Simple cluster example + domains: http://shapeshed.com/uncaught-exceptions-in-node/
- Isaacs gist that was used as inspiration for the cluster doc - https://gist.github.com/isaacs/5264418
Note: not yet found a 100% reason to favor multi process/cluster nodejs over nginx/haproxy stuff:
- http://blog.argteam.com/coding/hardening-node-js-for-production-part-3-zero-downtime-deployments-with-nginx/
- Also see this actionhero related blogpost on elegant downtime in relation to sockets , websockets etc..
- The rest off this post is to do clustering etc... yourself, otherwise you might want to check actionhero.js
Domains: keep it up , just a little bit longer
- doing a try/catch with async code doesn't work well
- therefore it's very common in the nodejs world to just exist on the exception
- Other approaches like catching the process.on('exception'), kinda work , at least you'll get some logging
- Therefore many examples use a forever or systemd to restart their service
- But imagine , 1 url gives an eror, it will actually exit the complete server! Lots of CPU power waisted for a single error
- Therefore the concept of domains were introduced
- http://nodejs.org/api/domain.html
- with it you can create per 'domain' exceptions that will emit an on('error') to handle them
- Domains & express: https://github.com/mathrawka/express-domain-errors
- But it seems connect grabs express exception with a try catch in it's code
- Stack overflow explanation: http://stackoverflow.com/questions/16174664/nodejs-error-handling-with-domains-and-socket-io
- Domains & Connect: https://github.com/baryshev/connect-domain
- Example on how to use connect-domain
Current Cluster/Domain Enhancements:
The core included module is basic, it tells you to take care of all the stuff you need in zerodowntime environments. Most of the tools below allow you to:
- wait for workers to correctly close or sigKill if past timeout
- sigUSR2 to reload workers one by one
- some provide a cli to do the work using a socket/network connection
- kill a worker that has become unresponsive by waiting for a heartbeat
- put itself offline and not accepting any new requests
Recluster: https://github.com/doxout/recluster/
Code at: https://github.com/doxout/recluster/blob/master/index.js
Good: Simple and in use
Bad: No CLI , No domains, Cannot pass args
Cluster2: https://github.com/ql-io/cluster2
Framework in use at ebay:
Good: complete with monitoring, control URL
Bad: Seems to be massive ...
Naught: https://github.com/superjoe30/naught
Note: naught2 was a temporary fork, but it got merged into master again
talks about Zero Downtime Crashed by intelligently handling express errors with domains
Good: simple, uses domains
Bad: seems to do it's own logging
cluster-master : https://github.com/isaacs/cluster-master
- Build by nodejs god @isaacs
- To be investigated
Up: https://github.com/LearnBoost/up
- Seems to be the new learnbooost way for zero-downtime
- http://thechangelog.com/up-node-powered-zero-downtime-reloads-and-load-balancing/
Others to check:
- All Cluster npm modules: https://npmjs.org/browse/keyword/cluster
- Bowl: https://github.com/waka/node-bowl
- Herd: https://github.com/segmentio/herd
- Jumpstarter: https://npmjs.org/package/jumpstarter
- Multi Cluster: https://npmjs.org/package/multi-cluster
- Pluribus: https://github.com/twistdigital/pluribus
- Simple node cluster: https://github.com/audreyt/node-cluster-server
Fleet: Extending the easy Git -> deploy
Initial blogpost on fleet: http://blog.nodejs.org/2012/05/02/multi-server-continuous-deployment-with-fleet/
Fleet - uses drones & propagit: https://github.com/substack/fleet
Propagit: A cascading git deployment: https://github.com/substack/propagit
blogpost on fleet usage: http://opsite.wordpress.com/2013/05/04/automated-drone-management-system-for-node-js-fleet/
Flotilla: https://npmjs.org/package/flotilla
All nodejs fleet modules: https://nodejsmodules.org/new/tags/fleet
Not related, but also cool: EC2-fleet https://github.com/ashtuchkin/ec2-fleet
Older solutions:
(+2 years no updated & probably not nodejs > 0.8 compliant) So you can safely ignore these, but they can give inspiration
- InfoQ blogpost on multi-core nodejs (is from 2010) - http://www.infoq.com/articles/multi-core-node-js
- The inspiration: has some cool options but alas < 0.8 compliant http://learnboost.github.io/cluster/
- https://github.com/pgte/fugue/wiki/How-Fugue-Works
- https://github.com/kriszyp/multi-node + good writeup
- https://github.com/dvv/stereo