Managing real-time cloud resources using the EVRYTHNG distributed resource manager

At EVRYTHNG we’ve recently released an open-source library called the ‘distributed resource manager’ which is available to download through npm for Node.js.

The library provides a way of managing a set of real-time resources across multiple nodes. Resources could be websocket connections to other services, urls to be called when an event occurs or any set of things that must be handled in a distributed manner.

The manager abstracts away the problem of the allocation and distribution of these resources so users can focus on the core business logic of their application. They don’t need to worry about maintaining the resource in the cloud when new nodes are added to the system or if one of them goes down.

So when would you use it?

The resource manager is useful when the following are true:

  1. You need to manage a set of real-time resources that connect to a third-party service in some way e.g. websocket connection, http polling, redis pub/sub
  2. These resources need to be handled in a distributed system
  3. You need to ensure that if a node handling a resource goes down that it is then reallocated somewhere else

At EVRYTHNG we use a version of this library for the following:

  • Maintaining permanent websocket connections to third party clouds. The users of the customers of the EVRYTHNG platform can have their data synchronised with other clouds. When this happens we set up a websocket connection to the other cloud and the distributed resource manager library takes care of keeping that websocket connection alive in case of node failure or some other error.
  • Allocating webhook resources to a particular node. In the EVRYTHNG platform internal components can subscribe to events in the platform by specifying a url. This url is then called when the event occurs. The distributed resource manager handles the allocation of these webhooks to specific nodes which ensures a url is only ever called once for a single event.

I also gave a talk about this project at the London Node.js User Group that you can view here:

https://www.youtube.com/watch?v=lwmYq8KtvJA

If this library seems useful to you I would encourage you to try it out and give me any feedback you may have. Any contributions to the project would also be most welcome!