Bangle.js - an Open Source Smartwatch

[00:06] - Hi, so I've been working on a project called Esprino for several years now. It's a JavaScript interpreter for microcontrollers. So these are really constrained devices. [00:16] And the kind of things we're looking at are like double digits of kilobytes of RAM. [00:21] So Espino itself will run with the execution stack in under 8 kilobytes of RAM. [00:27] which is very different from something like V8 that will be needing almost in the region of a megabyte. [00:33] And you might ask why on earth you would want to run something like a JavaScript interpreter on such a constrained device? [00:40] And really the reason is that [00:43] They are literal black boxes. [00:45] If you don't have a actual hardware debugger attached to one of these, [00:50] And if something happens, [00:52] you have no idea what happens, it just stops working. [00:54] And it's not just software problems you have to worry about. [00:57] These things have a bunch of inbuilt peripherals that could be configured wrong and could cause problems [01:02] and they're part of a circuit with a bunch of other components. [01:05] which could be configured wrong, could also be wired up wrong. [01:08] There could be incorrect values on components [01:12] And what you really need is a way to... [01:14] get inside the circuit and [01:17] sort of inspect it. [01:18] And having a REPL there where you can actually... [01:21] execute commands, [01:22] view the stated variables even change code on the fly makes a massive difference [01:28] And not to mention that [01:29] You know, a lot of devices you might be having a microcontroller in might be wearables. [01:34] or something like that.

[01:36] People might be wearing this and a software error happens. [01:40] actually having a stack trace that you can save [01:43] is hugely valuable for development and just [01:46] even when your device is in production, making sure it stays reliable. [01:50] And so it's a bit like having a... [01:54] an operating system running on your device, but without all of the complexity of having to worry about that and the extra RAM and stuff that you might need. [02:02] It also happens to be quite good for producing low power code. [02:06] because JavaScript developers are used to writing code [02:09] that [02:11] is [02:12] well, that is event-based. [02:13] So the interpreter can put itself to sleep when it's [02:17] It's not needed. [02:19] So this software is entirely open source, it's MPLv2. [02:25] and [02:26] It will run on a variety of platforms. [02:28] but [02:29] in order to support the development, I have to make money somehow. So we design and sell... [02:35] a bunch of open hardware products, [02:37] All of these actually using Arm Cortex-M chips [02:40] mainly the NF52 which are Bluetooth chips [02:44] And the STM32. [02:46] But over time, what I found out is [02:49] the things that are the most popular are the... [02:52] devices that are nearer to a finished product. You know, people use these things, they create a prototype, and then they just like, well, why don't we ship it? [03:01] So if I took all the things that I thought were good about some of these products,

[03:07] the case [03:08] rechargeable battery, a display, then fitted it in a sort of small form factor. [03:13] then what you get looks a lot like a watch. [03:16] So that's what this is. This is bangle.js, which is what I'm talking about today. [03:21] and [03:22] It's... [03:22] basically the same JavaScript interpreter, but with a always on color screen. [03:27] bunch of sensors and up to around a four week battery life. [03:32] Thank you. [03:32] but it is entirely open source so the interpreter is written in C [03:39] and all the apps and kind of all the stuff that you interact with on the watch is written in JavaScript. [03:45] and all of that's MIT licensed. [03:48] and it's all available sort of in a GitHub repository where you can access every bit of it. [03:53] But having those 600 apps is no use if you can't get them onto the device in the first place. [03:58] so [03:59] As a relatively small developer, [04:02] And... [04:03] trying to write and then maintain a whole bunch of apps to access the underlying hardware is a complete nightmare. [04:11] and web Bluetooth is a complete savior here. [04:14] So for those of you who haven't come across it before, it's a API that's built into Chromium-based browsers on pretty much all the platforms apart from iOS. [04:24] and it allows you to [04:26] safely and very securely [04:29] Um, [04:31] and just access web Bluetooth devices on quite a low level. [04:35] So we can make all our tools.

[04:37] based on this. [04:40] and it saves a bunch of time, there's nothing to install, it makes it really easy for users and developers. [04:46] But this opens up a really interesting possibility now, because we've got a device that can execute JavaScript code that you send it over Bluetooth. [04:54] and we've got a really easy way to send stuff to it from a web page. So this code up here is [05:02] I mean there's a bit of like standard website stuff but you've got four lines that are actually doing anything useful. [05:08] you've got the uot.write which is sending JavaScript code [05:12] to the bangle.js itself to execute. [05:15] turn the heart rate monitor on, [05:16] when you receive a raw value from the sensor [05:20] send it over bleeteeth and then [05:22] online [05:24] process it and put it up and you now have in that tiny bit of code on any device [05:29] that sort of Chromium based [05:31] a way to view live data. [05:34] And at this point, you might be wondering, why am I talking at a WebAI talk? [05:39] Well, [05:40] this ability to get [05:42] Um... [05:43] to get live data in from the various sensors on something you're wearing, is it's a really fun way to start playing with WebAI. [05:52] And Jason saw this and he thought, along with Google, that it would be really good if you guys had a chance to use this too. [06:00] So in your swag bag, you should be getting one of these that you can have a play with.

[06:08] So a huge thanks to Google for this. [06:11] because it just wouldn't be possible without that funding. [06:13] and [06:15] yeah the rest of this talk really is just me showing you how to get started with this [06:19] So... [06:21] You'll get something that looks like this. [06:22] The straps aren't attached. [06:24] It's very easy to stick them in. There's a little metal lever, and you just... [06:29] poke the peg in, pull the lever down, [06:31] put it in and let go of the leaf and it'll just clip into place. [06:36] There's only one button. So you just press and hold that button. There'll be a little equals... [06:41] like a progress bar will go across the screen and you release it at the end. If you release it too early, it'll go to a bootloader and if you release it too late, it'll go to a recovery menu. [06:49] but you just press it and hold it. [06:50] and it'll go all over again. It's the problem with only having one button, you're very limited. [06:55] You'll get something like this. [06:57] This is like a little intro, you can just... [07:00] swipe the screen or press a button to get through it. [07:03] And at the end of it, you will get a QR code. [07:06] Now this QR code with a little bangle.js under it [07:09] Um... [07:11] This will [07:13] Thank you. [07:13] allow you to connect... [07:15] to your device in particular. [07:17] If you press a button and you get back to the clock, you can... [07:22] go back to that QR code again by you press the button once to unlock if it's locked [07:28] another time to go to the launcher and then you just tap on the connect button [07:32] and this is because there are going to be 500 of these. [07:35] There's a little device chooser window that will appear in the browser and you don't want to be going through 500 different

[07:43] different devices trying to find yours [07:45] So we've made these not advertised on Bluetooth by default. They'll only advertise when they're displaying that QR code. And if you scan that QR code, it will take you to a special WebAI landing page [07:57] which will be set up to [07:59] only talk to your particular device. [08:02] If you need to just find your device again, it advertises itself with bangle.js and four digits, which is basically the last four digits of the MAC address. And they're shown in the top right of the watch screen as well. [08:17] You'll get to a page like this. You can click connect... [08:20] And [08:22] and you get this, and basically it's running WebAI in the browser. It's some really simple code. [08:27] but it'll detect when you're clapping and, you know, [08:30] have nice [08:31] Nice hands arriving on the screen. [08:33] There's a little [08:34] text box in there, which is for a... [08:38] social URL. [08:40] So... [08:41] If you fill in your LinkedIn X, [08:44] web page, whatever, and hit the set watch QR code button, you will get a [08:51] QR code when you double tap the watch. [08:53] So like if I get to mine, [08:55] your [08:56] you'll get a code like this, and then you just swipe or press a button to get rid of it. So hopefully it's a nice, easy way to share your social details with people [09:03] at the conference. [09:06] Now... [09:07] I mentioned those 600 apps. [09:10] These are actually... [09:12] in a website that comes off github pages basically um you can check them out there's a link in that landing page

[09:20] but there's also the URL up there. [09:23] It's very easy to just connect [09:26] click to upload a whole bunch of different apps. [09:30] Now, the really cool thing about this is that there's a little GitHub icon [09:33] underneath every app. It's a bit hard to see in there. [09:36] But if you tap that, it will bring you [09:38] to the folder on GitHub where that particular app is, and you can look at all the code [09:44] you can fork the repo, enable GitHub pages, [09:47] And then you have your own personal uploader, which will upload whatever apps you've made. [09:51] And from there you can do PRs back [09:53] and add whatever things you need. [09:56] If you're going to make changes... [09:57] you'll probably want to use the web IDE, which is another link in the uploader. [10:03] Click up the top left, click Web Bluetooth and choose your device. [10:07] You'll have like a REPL on the left, which you can just... [10:10] type in and interact directly with the device. [10:13] On the right, there's an editor with some code completion [10:16] And everyone's got their favourite editor, so... [10:20] like plenty of you won't be happy with this [10:22] All you have to do is click on the [10:26] the little down arrow below the open [10:28] icon. [10:29] It'll show you a window where you can say [10:32] watching up late. [10:34] and then you watch a file, every time you save that file in your editor it will re-upload it. [10:39] no software to install, use your favorite editor, it doesn't matter. [10:43] But we're really here for web AI. So, [10:47] In the landing page, there is a... [10:50] There's a link to this website or you can

[10:54] You can just take a picture of it. [10:55] Yeah, just-- [10:57] Take a look at it, fork it if you want to. [11:00] and start hacking and see what you can do. [11:03] And... [11:04] All you really have to do, it's pretty basic, there's a library called UART.js which takes a little bit of the pain off of [11:12] using WebButees. [11:13] The first time you call the ORT.Write, it will try to connect to the device. [11:17] Um... [11:18] you need to-- [11:19] ideally call reset first which will wipe everything out of RAM [11:23] and take it back to a kind of completely plain state [11:26] Because if you don't do that and you kind of keep connecting, [11:29] you're going to keep like you're going to add multiple handlers for various events and you're going to get duplicate data [11:35] You wait half a second, you do another one with the code that you want to execute, which you should probably test out in the IDE first to make sure it's producing data the way you want and you're not getting any errors. [11:47] and then you just handle the data. [11:49] So, [11:51] When you start playing around with this, you almost certainly get yourself into a state where... [11:55] the watch isn't displaying the clock anymore and you want to get back to it. All you have to do is press the button for about two seconds and it will load you back to the clock. [12:04] If everything has gone horribly wrong, which probably someone will manage to do and delete all the JavaScript, [12:10] you can just [12:11] press and hold it for another 10 seconds, get to that recovery menu I mentioned. [12:15] and then do a factory reset. [12:17] You don't need anything else. It'll just get back to exactly the state that you got it today. [12:22] Um... [12:23] The bangle has a bunch of fun features. It's got TensorFlow Lite for mic controllers inside it as well.

[12:29] Um... [12:30] but because you're running on a 64 megahertz processor, [12:34] you are kind of limited, you know, you're not going to do computer vision on this thing. [12:38] say, [12:39] definitely web ai is the way to go for like [12:42] for pen number crunching. [12:44] But if you did want to do... [12:47] like some proper work on our mic controllers. [12:50] you know there's been a talk earlier today about this there's a arm Clyde AI library which is going to provide you with some really highly optimized [13:00] sort of [13:01] functions to use with all the popular AI libraries like TensorFlow. [13:08] Um... [13:09] Yeah, so I'm really excited to see what you guys come up with. [13:13] Um... [13:14] There are lots of options for games, fitness, [13:18] quantified self like you know trying to see how many times you raise your hand to drink water or something during the day. [13:24] and [13:25] I'm sure a lot of you have seen... [13:28] talked to a bunch of you already. [13:30] There's a demo table. [13:31] and [13:33] Just outside... [13:34] If you have any questions, any problems whatsoever, which I'm sure there will be, [13:38] Come and talk to me. I'd love to hear from you. [13:41] Um... [13:42] Just to say that [13:43] I'm here with the [13:45] Army Ambassador Programme [13:47] uh, [13:49] it's a long way to come from England, it's quite expensive [13:53] They have very graciously helped to cover the flights and my stay here. So if you're doing work with ARM processors,

[14:02] Um... [14:04] get in touch because they would love you to spread the love and um [14:09] Yeah, they have a table outside as well, I think. [14:13] Have a chat with him. [14:15] And yeah, that's it. Thank you. And a huge thank you to Google and Jason [14:19] for sorting this out and [14:21] Hopefully you guys have some fun toys to take home. [14:24] Thank you.