Muskepeer

Distributed computing using browsers and WebRTC

Muskepeer-Framework – "One for all, all for one"

The Muskepeer-Framework is a JavaScript based prototype for decentralized Distributed computing using web browsers. A project can be defined in a JSON file and then passed to the Muskepeer-Framework. The browser downloads the needed resources and connects to other browsers to create a WebRTC-based Peer-to-Peer-Network (P2P).

Each client executes a Worker-Script to generate results. These results get stored to a local database and synchronized between all clients. A project can have an additional Factory-Script which creates Jobs (tasks) for the Worker-Scripts, or even read and write files to the local filesystem. It's build to give you a bunch of options inside the Worker- or Factory-Scripts. To make a long story short:

  • JavaScript framework for Distributed computing
  • Decentralized Clients through WebRTC
  • WebRTCs DataChannels for a direct communication between the browsers
  • Custom Signaling Channel using Web Sockets
  • WebWorker isolated code-execution
  • Results & Jobs get stored via the IndexedDB
  • Muskepeer-Dashboard to visualize the progress
  • Use REST-Services (like parse.com) for external storage
  • Files can be used as ressources though the FileSystem API
  • Network API usage for offline computation
  • Released under a MIT-License, so feel free to use and contribute

This prototype was created for my (Matthieu Holzer) master's thesis in the end of 2013.

Architecture - Client, Node, Service
Architecture of the Muskepeer-System [SVG]

Muskepeer-Client

The Muskepeer-Client is the web application that does all the magic. It's written in JavaScript and allows you to execute whatever JavaScript code you want in a WebWorker environment. Just include it into your website and create a project-file. To start Muskepeer, the following block should suffice:


    // Initialization tests the browser support
    window.Muskepeer = Muskepeer.init();

    // Start the Muskepeer Framework
    Muskepeer.start({
        project: https://muskepeer.net/examples/knn-digits/settings.json,
        nodes: [
            {
                host: 'node01-muskepeer.rhcloud.com',
                isSecure: true,
                port: 8443
            },
            {
                host: 'node02-muskepeer.rhcloud.com',
                isSecure: true,
                port: 8443
            }
        ]
    });


Muskepeer-Node

As the WebRTC connection between two browsers needs a signaling channel (see http://www.html5rocks.com/en/tutorials/webrtc/infrastructure/). The Muskepeer system uses a WebSocket implemented using Node.js to close this gap between to peers.

Muskepeer-Dashboard

Currently the framework has no GUI or visualization at all. A webbased dashboard to control the system is currently under development.

Demos

To test the framework there are two examples:

Make sure to open each demo with more than one device or browser to see how it actually works.

Thesis

The thesis itself and a corresponding presentation, both in german.