Pure JS data channel implementation #1
Comments
|
Tagging people who I've discussed this with: @mafintosh, @substack, @fippo, @jbenet, @maxogden, @modeswitch, @DamonOehlman, @jhiesey. |
|
I think we should setup a hangouts session for people interested in implementing this and discuss different approaches asap. |
|
Sounds like a plan to me, TZ here is +11 so I'll try and participate if I can. |
|
Thanks for settings this up. The approach I took with Mozilla's WebRTC implementation is tightly coupled to Firefox, so it would likely be as bad as It would be nice to have a solution that can support media streams as well. I get feature requests for that all the time. If it's done in a modular way then it doesn't need to impact consumers who don't want it. Another (crazy?) option is to work with emscripten to generate JS for some of the native library components. I have no idea how nasty that might be, but it's worth mentioning anyway. Sorry I've been silent on these discussions. I have more time for this now and I intend to be involved. Count me in for a hangout session; I'm in GMT-5. |
|
Hello everyone! excited this is shaping up -- thanks for including. As @feross mentioned, this discussion has some other people: jbenet/random-ideas#13 and links thoughts. +1 for hangout.
Agreed, let's get the UX (for both users and devs) to be great!
My vote is for this. In terms of time investment, i think it it's both easier, and more far reaching.. Ripping out the sources (not compiling from chromium source but actually extracting the relevant code, removing the chaff) may be a lot simpler than an implementation from scratch (particularly if we can talk / ask questions to the people who implemented it -- the chromium lists are pretty active). Plus, making a {libdc, libdatachannel, libpeerdc, ...} would be extremely helpful to all the other languages in the world. We can do Node prebuilt binaries really well (e.g. @mafintosh's leveldown-prebuilt). (Of course, given plenty of time, i'm all up for helping with a pure js impl, but i think our mileage may go much further if we consider a broader community). |
+1 |
|
GMT+1 (Spain), but prefer to use email/github issues. A pure JS is almost impossible, or impractical. There's a lot of libraries involved and to have some performance we would need to use compiled modules at some degree. As a teaching project it's ok, but not production ready. I think a better alternative is to have a dinamyc library that can be used by several languajes. |
|
selfishly, can it (the C/whatever lib) please be something that also works on iOS? |
|
I'm also interested in seeing something that allows browser < - > server communication using only data channels. I would love to see something like this be released a module where we can build applications upon. I think that going with an c library where Node can just bind in to makes more sense here as it would allow other languages leverage the work. |
|
+1 to what @jbenet said - a small C library that builds fast and that any language could bind to would be awesome. I'm interested in experimenting with this in a side project I have and helping/learning as much as I can. |
|
I'd say for anyone interested in looking at what's involved in creating a more modular implementation utilizing C/C++ libraries, having a look at the prereqs for janus is a great place to start. I've started thinking about this previously, and if I had managed to find the time I probably would have tackled the problem in two parts:
|
There's also clib by which has a growing ecosystem of micro-libs and has a similar feel to npm. |
|
I still think a pure JS version is worth trying, as well as a small c version |
|
if you care only about browser<->server then you can probably get away with using ice-tcp (only available in chrome today) in pure js and bindings to usrsctp for now. The latter is easy enough to install, especially compared to the webrtc.org lib. |
|
sctp-refimpl is an excellent library written by Michael Tuexen, one of the authors of the Data Channels specification in the IETF and SCTP expert. Besides, it's being used in both Chrome and Firefox, so we simply cannot go wrong doing the same :-) As explained in https://github.com/meetecho/janus-gateway/blob/master/sctp.c (the code where I handle most of the Data Channels stuff) I based the implementation on a sample application that the library provided: This example shows how you can transfer SCTP messages on top of a different "transport", that is, not using SCTP directly but on top of, let's say UDP or something else. In my case, I obviously wanted to have something that I could have relayed up to the code handling DTLS connectivity, so that I could have it sent that way. Same thing for incoming messages, i.e., data retrieved through DTLS and to be passed to the SCTP stack somehow. My integration in Janus currently only supports strings, but supporting octets should, at least in principle, just be a matter of using the binary PPID (53) instead of strings (51), as specified in https://tools.ietf.org/html/draft-ietf-rtcweb-data-channel-11#section-8 libnice+openssl+sctp-refimpl should be enough to cover the requirements. |
|
Looks like a C library now exists: http://opentools.homeip.net/webrtc |
|
I'm going to try ripping out all the |
|
Quick update: I've made some good progress on stripping down libjingle. Data channels are working and all the media streams code is unreferenced. I'm now removing source files and updating build scripts to see how quick/easy this can get. There are very few system deps required now to build. |
|
@modeswitch That's awesome news mate - very cool :) |
|
I've released an update for A few notes:
Any other feedback, please let me know! |
|
just wanted to say this makes my heart smile! :) |
|
I tried to install with Mac OS 10.9, but ran into issues because I'm using command line tools, (can't install xcode due to company policy). Not sure I want to log this as an issue, but might want to note that Xcode is required for Mac users? (for build obviously) |
|
You can also install the command line tools without xcode. See https://github.com/kennethreitz/osx-gcc-installer for more details. |
|
I've just tried it on OS X 10.10 and got the following error: It doesn't really tell much what went wrong... |
|
@szimek I'll have it dump messages directly to stdout. |
|
@szimek Pushed a new version. Build output should stream to your console. I've also re-added depot_tools, since most people won't have |
Looks like some issue with boringssl. |
|
@szimek I removed boringssl because it's not used in the linux build, but I'll add it back. |
|
@szimek Fixed. Also, I think we're detracting from the thread. Let's continue the discussion over here: js-platform/libwebrtc#1 |
|
Latest version of One possibility for developing a pure JS implementation would be to start pulling parts of the native library into JS, iteratively. It sounds like a lot of work, though. |
|
Ah, I am late to the story but I am really sad to see that community has decided to drop all the media stuff in favor of shorter compilation times. I just got an idea how great it would be that both the server and client would communicate over WebRTC and can have the same standard API to do that, both for data and for media. I wanted to create a WebRTC/mesh based community radio where servers would be doing recoding and clients would send their stream to them. But without the MediaStream implementation it means that one has to reimplement the whole audio processing in node.js and on top of data channels. Is it really so problematic that the compilation takes a lot of time? Maybe I just remember Qt times where 4 hours of compiling was completely normal. :-) |
|
So probably what I wanted to say with my rant was that there is a case for interoperability. If majority of nodes in your P2P network speak MediaStream (browsers) it is really sad that servers (much more powerful devices) would not as well. |
|
One more comment and then I will stop. The compilation issues also became simpler in the last year with Docker containerization. Now you get it to compile once inside a Docker image and then you have it. You can even cache it in a Docker layer. |
|
@mitar why isn't this just a layering problem? i'm failing to see the problem. AFAIU, you should be able to do mediastreams just fine, and have a native library compiled in your code for it? |
|
There are some speculations I am not sure about, but:
To me it just looks like duplication of the code in JS for things which have to be low-latency and are tricky to setup correctly. Also, having exactly the same audio codecs on the client and server would help a lot. So I would anyway have to compile the audio codecs to use in node.js. So why not have them compiled together with WebRTC? |
If you want a complete WebRTC stack on the server, there are already multiple you can pick from. See http://webrtc.org/, http://www.openwebrtc.org/, and https://www.npmjs.com/package/wrtc (for use in node.js). All we're saying is that there's lots of value to be gained by having a library that's lightweight and only does data channel. :) |
|
There is nothing for node.js:
|
|
Electron works for server side webrtc + node just fine using xvfb-run + electron-spawn |
|
@maxogden that's really interesting! Do you know of any write ups or examples of doing that? |
|
@xdumaine I figured it out when writing https://github.com/moose-team/peerbot#running-on-headless-ubuntu |
|
He he, I was also thinking about that. That why is node.js just V8 on the server side, why not simply the whole Chrome. :-) |
|
@mitar try running chrome on a server... there is a reason a certain browser with 300 million users doesn't support the latest and greatest web features ;-) |
|
I was thinking about using it for a Pi drone project. |
A bit late to the party, I've written a standalone C++ library with a C API implementing WebRTC data channels: https://github.com/paullouisageneau/libdatachannel |
|
@paullouisageneau Are there Node.js bindings? We're open to exploring using an alternative to |
|
@feross Yes, @murat-dogan has been writing node.js bindings here: https://github.com/murat-dogan/node-datachannel |
WebRTC let's us do browser<–>browser p2p communication.
But what about browser<–>server and server<–>server?
WebRTC everywhere
There are significant advantages to using the same transport (i.e. WebRTC) for all peer connections in a decentralized system, including:
Why a pure JS data channel implementation?
The data channel can be used in node.js via the wrtc module by @modeswitch. It's impressive work, but not without it's drawbacks.
It takes 25+ minutes to install on my system, and pre-built binaries are not available :(. It has to download and compile Google's libwebrtc from scratch. libwebrtc is huge! It contains libjingle, chromium's depot_tools, lots of video/audio codec stuff, and more! And there are install issues for users without a suitable compiler installed.
A pure JS implementation (relying only on node.js built-ins like
dgramandnet) would greatly simplify installation for end users.Open questions
Related discussion: jbenet/random-ideas#13
The text was updated successfully, but these errors were encountered: