Writing decentralized applications in JavaScript — libp2p basics

Libp2p is one of the most known p2p libraries. It is the base of the Interplanetary Filesystem and it will be the base of Ethereum 2.0, so…

Writing decentralized applications in JavaScript — libp2p basics

Source: https://discuss.libp2p.io/

Libp2p is one of the most known p2p libraries. It is the base of the Interplanetary Filesystem and it will be the base of Ethereum 2.0, so it is a very good choice if you want to develop decentralized applications. In this article, I will show the basic conceptions of libp2p and we will develop a minimalistic browser-based (no own server needed) chat application using the JavaScript implementation of libp2p and vue.js.

PeerId

PeerId is one of the main elements of libp2p. It is something like a BitCoin or Ethereum address that is generated from the public key part of a keypair. Like a BitCoin address, it is unique and can be used to sign a message by the private key that can be validated by the public key that is associated with the peerId. You can generate a peerId by the peer-id utility.

You can simply install peer-id by npm:

sudo npm i peer-id -g

After the installation, you can simply generate your very own peer-id by the peer-id CLI tool:

peer-id
{
“id”: “QmPMKmmRyepsh4Wq24msEdeGJQWSszy7eWNSjL53bxz3sS”,
“privKey”: “CAASqAkwggSkAgEAAoIBAQC6/4DQXtW9HYbx5wPNbcsNbwMITH8Mg70O+zhQ9t9w/9Z7BuOzwQqu/ABaRclkl1FIuGfIayKSzqkczYkdY3I6bMUtn8dyHuJi1JzJj5PtaGfK81ra6hNvkVRqttYKJdgpPdGWCa6ouRbFPf3l0w2sK0ejf1Im/h4bAL1ltgZK7G+fXZTlUdAoEvQJja3bl9b47zqWiq+oVaez7uBhemrJQ6Ao09Tlr/RpVfbsc48NAczuDqh9nZzUq27LQEu9OXOf6xk/hPaDcof0ubQs9REXDCcDq01Cg6dQ5odKKHcT9hTeytZlvzfZZgleU8pME9r1EVpG5aEMn5qC0zfoe3PTAgMBAAECggEAWBoa+ZFEyHYJ5xy9WOMaoLilyBoqXZ4Py+gmj1bQzS9sQMhtLXqM6waFsAJjMUZtoIJpOy7muh4t5QkdScBZyBcJC0bVM/pDFOcw+3Hu8xKWnDLtomhYQd9J04FS9LMB1eRvQ25KYOnbRZDAd7BpJ624cdqBvSdKzdQaZ7pL2q4d/P1DUVLVfNDHzHlHyiLoxL1Ilm5YQtGGA3PwTLmSMqWGPvrzK4n0qL2rXUMbio0wf7w3lWDnBNQpncwdcPqUSTPDItwa2GNImH5yZRIF/vP2Qdky7LvNoxIzr+UhyQ6s2YSfRLwgnSPNL4IRuRaUWCtFprdaMd/x65xkBDVP2QKBgQDqXV922hbpMWgGPUco0uv7C8UApWiA/TP95pDzJuoD/SL0pn15QQrPMYTbOf32DrwC/sVMuuIbStqJd8RSApTsQO48KiPgmJM6WgD35umlYpmPQh8djvhPTe3nCadoxy+40i7NuuDde9AtvDlqwti4t1E/6lnYNZecbrXnT3ELZwKBgQDMQryboWcrUS+pfrJADtNiBOBtAAIQz7aHZRmk9ZFY7L8ekSB1Cuee4/FXswkrie+jE1rGbV97OQ1f6qW10SkoN64OIs8AVx408MxLdEejLg0vLwjEqEVXE2Wc1/TkOuEOpsrniKgisIZThrdC2RhmOqtaKRnAWz761ypCluP8tQKBgQCzqzV+Zh9eUpQPBHdDIr/qS9GRdz0wdeyf31yMK+8Hc86Sg/h5NpXU1X+mmUTKl+0m1q3m7vZcOfxjmr+Up4oHvJdm5F9w1Uc5WrqXUh0Yvwg+PVChVnOiSHnzvwDqYJmDNQ7QhU3SPhMQnNXftNR0d4UAXObXy+4Y7P7i/5IITQKBgQCtIxh6Frq7lep/kjwHXknA+P8+hVY658YBSCoPkHOuW6a4gy1u6FpibTZCLyjjtdzhbuNv9H+NlFOI7P2fevaW93Na2hh6Yl3hZAbXIm4inENirkyRXUzBPVjRNKCI4HuqDqlIzqYuGVES9crbJ+etp6ddGh+Q1AczWjShEwOXTQKBgF2cuW8w7ghAXapb2BKQU9bZTMAevohw6cxEmajNngVGSGMQJIJdhEoaGUXdtBOpOxAfYdycegf9VkPraVERXAbCMDk5mAh4Aor1SsHUDnQ7VKMLQfGzH3Jfk2TpXqy/11rJlLAl05LCyAeHGLVM4CQrvpLOIeM5n6j/3QWO3Jdh”,
“pubKey”: “CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC6/4DQXtW9HYbx5wPNbcsNbwMITH8Mg70O+zhQ9t9w/9Z7BuOzwQqu/ABaRclkl1FIuGfIayKSzqkczYkdY3I6bMUtn8dyHuJi1JzJj5PtaGfK81ra6hNvkVRqttYKJdgpPdGWCa6ouRbFPf3l0w2sK0ejf1Im/h4bAL1ltgZK7G+fXZTlUdAoEvQJja3bl9b47zqWiq+oVaez7uBhemrJQ6Ao09Tlr/RpVfbsc48NAczuDqh9nZzUq27LQEu9OXOf6xk/hPaDcof0ubQs9REXDCcDq01Cg6dQ5odKKHcT9hTeytZlvzfZZgleU8pME9r1EVpG5aEMn5qC0zfoe3PTAgMBAAE=”
}

Calling peer-id will generate a JSON with the private and the public key, and the id which is a multihashed public key. This id is your unique peer address.

Multihash

Multihash is a future-proof hashing format that is used by libp2p and IPFS. It is built from 2 parts. The first 2 characters are the hash header that defines the hashing algorithm, and the second part is the hash itself. In the above example (in the id part of the peerId JSON) Qm means it is an SHA-256 hash and the second part is the base58 encoded 256 bits hash.

Multiaddress

Multiaddress is something like an HTTP URL, but it’s more general. A multiaddr is a list of protocol/value pairs that can point to a host, a service, a peer, etc. Let’s see some examples:

/ip4/1.2.3.4 — this multiaddr points to a host

/ip4/1.2.3.4/tcp/80 — this points to TCP port 80 of the host

/dns4/example.com/udp/123 — this points to the UDP port 123 of example.com

/p2p/QmbEKwwsuzArDenmWJfcFgyQ8uATQrmkjAsaT8VaGocV1x — this points to a peerId

/ip4/7.7.7.7/tcp/4242/p2p/QmYyQSo1c1Ym7orWxLYvCrM2EmxFTANf8wXmmE7DWjhx5N — this multiaddr exactly points to a host and a TCP port where we can communicate to the given peer

Now we know the basics, let’s go forward to the implementation. Libp2p is a modular network stack with many implementations. In this article, I will use the JavaScript implementation that can run on the server-side in node.js or in your browser. Let’s see some code to understand how js-libp2p works:

‘use strict’
const Libp2p = require(‘libp2p’)
const TCP = require(‘libp2p-tcp’)
const { NOISE } = require(‘@chainsafe/libp2p-noise’)
const createNode = async () => {
  const node = await Libp2p.create({
    addresses: {
      listen: [‘/ip4/0.0.0.0/tcp/4321’]
    },
    modules: {
      transport: [ TCP ],
      connEncryption: [ NOISE ]
    }
  })
  await node.start()
  return node
}

The above example shows how can we start a libp2p node. The Libp2p.create method has two options here. The first is the address where the node will listen. In this case, the node will listen on every IP address and on 4321 port. The second option is the definitions of the modules. Libp2p is a strongly modular stack. Every module is defined by an interface, and you can freely choose which one would you use, or you can create your own modules. Every part of the stack is modular. You can define the transport layer, the encryptions, you can define your own peer discovery mechanism, etc. In this case, we will use TCP transport and NOISE encryption.

The above code works fine in node.js, but in a web browser, it’s not possible to open server ports. To solve this problem, libp2p can use relay servers that can open the required server ports and forward the traffic to the browser.

this.libp2p = await Libp2p.create({
  addresses: {
    listen: [
     "/dns4/wrtc-star1.par.dwebops.pub/tcp/443/wss/p2p-webrtc-star",
     "/dns4/wrtc-star2.sjc.dwebops.pub/tcp/443/wss/p2p-webrtc-star",
    ],
  },
  modules: {
    transport: [Websockets, WebRTCStar],
    connEncryption: [NOISE],
    streamMuxer: [Mplex],
    peerDiscovery: [Bootstrap],
    dht: KadDHT,
  },
  config: {
    peerDiscovery: {
      [Bootstrap.tag]: {
      enabled: true,
      list: [
"/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmZa1sAxajnQjVM8WjWXoMbmPd7NsWhfKsPkErzpm9wGkp",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
"/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
      ],
      },
    },
    dht: {
      enabled: true,
    },
  },
});

This more complex example uses the wrtc-star1.par.dwebops.pub and wrtc-star2.sjc.dwebops.pub servers as relays through the webrtc protocol. These servers are provided by the libp2p community, only for testing. If you building a real-world application, you should install your own webrtc star servers.

Another new thing is the bootstrap nodes. These nodes are the starting point of peer discovery. When your node starts, it will connect to these nodes firstly. After the connection, our node will get other nodes addresses that are also connected to the bootstrap nodes, and it will connect some of them, so after the bootstrap, the bootstrap nodes are not needed anymore, the p2p network builds automatically. The bootstrap nodes in the example are also provided by the libp2p community. By defining different bootstrap nodes, you can define your own, separated p2p networks.

The last thing that I want to talk about is the DHT. Distributed Hash Table is a distributed key/value store that is stored on the nodes. This key/value store is used to store the current internet address of the peerId. It is something like the DNS system, where the DNS servers store the IP addresses for the domain names. DNS is also distributed like the DHT, but in this case, we use the peerId instead of the domain and the endpoint’s multiaddress instead of the IP.

let peerId = PeerId.parse(this.otherPeerId);
let result = await this.libp2p.peerRouting.findPeer(peerId);
this.otherPeerMultiaddrs = result.multiaddrs;

The above example shows, how can you find a peer by its peerId. The peerRouting module’s findPeer method tries to discover the peer by the peerId and if it finds it the multiaddress will be available in the result.

Now we can start our own nodes in the browser, and find other peers. The next step is communicating with them.

First of all, we have to define a protocol and handle the incoming connections. It is something like open a server port in a traditional client-server application.

this.libp2p.handle('/chat/1.0.0', 
  ({ connection, stream, protocol }) => { 
    this.remotePeerId = connection.remoteAddr.getPeerId(); 
    pipe( 
      stream, 
      (source) => { 
        return (async function* () { 
         for await (const buf of source) 
           yield array2str(buf.slice()); 
        })(); 
      }, 
      async (source) => { 
       for await (const msg of source) { 
        this.messages.push(“> “ + msg); 
       } 
      } 
    ); 
});

The above code defines the ‘chat/1.0.0’ (this is the standard format of a protocol) protocol and handles the incoming connections to it. The handler function has 3 parameters where the 2. is the most important, the incoming stream. JavaScript streams are very tricky things, but hopefully, there are some nice tools to handle them. In this example, we are using the pipe function. The first parameter of the pipe is a stream and the last is a consumer. The middle parameters can be stream transformers. In our case, we transform the stream elements by the array2str function because the stream is binary and we need simple UTF8 strings. The last function (the consumer) simply read the messages from the stream and write them out.

Now we have nodes in the browser that can find each other, and we have a chat protocol handler. The last step is connecting from one peer to another, and send some messages on the chat protocol.

const { stream, protocol } = await this.libp2p.dialProtocol( 
  peerId, chatProtocol 
); 
this.chatQueue = pushable(); 
pipe( 
  this.chatQueue, 
  (source) => { 
    return (async function* () { 
      for await (const msg of source) yield str2array(msg); 
    })(); 
  }, 
  stream 
);

Libp2p has a dialProtocol method for connecting other peers. The method has 2 parameters. The target peerId and the protocol identifier. The result of the method is a stream, so we use the pipe function again. The first parameter of the pipe is a pushable. Pushable is a queue that can be read as a stream. We will push the chat messages to this queue. The second stage of the pipe is a transformer that calls str2array to convert the string messages to binary form. The last stage of the pipe is the stream itself.

Let’s see the full code. I’ve used Vue, cause it is compact and easy to read.

You can test it online on https://thebojda.github.io/js-libp2p-browser-chat/index.html. Open this URL on 2 browser tabs. Copy the one tab’s peerId to the other tab and push the ‘Find’ button. If everything goes well, the browser node will find the other node in the other tab. Then push ‘Dial protocol’ button and you can start chatting between the two tabs.

Libp2p is the basis of many p2p applications. I hope, this short introduction will help you to start developing your own p2p applications (maybe a new sharing economy application, or your very own new blockchain). So, let’s coding…