Categories
Bluetooth Windows

Bluetooth Virtual COM Ports

I was thinking about a comment on a Gist I wrote some time ago and thought I’d written a blog post on the topic, but when I couldn’t find any sign of it I decided to start from scratch as I’d have probably needed to update it anyway.
The Bluetooth stack in Windows has supported virtual COM ports since the beginning (Windows XP SP2 if you can remember that far back!). It’s important to understand the difference between these and any Serial Port Bluetooth device. The Virtual COM port functionality exists purely for interop with older software – you can make a Bluetooth device appear to the system as a wired Serial device and open a COM port and talk to the device as if it was plugged into your PC.
If you’re writing modern software which talks to devices then you would use a Bluetooth API – either 32feet.NET or the platform specific Bluetooth API directly. If not you can create a virtual COM port. Throughout Windows 10’s lifetime the Bluetooth options have been slowly moving to the modern Settings experience but the old Control Panel still exists and is used for some additional functionality – Virtual Serial ports are still in there.
The port can be either incoming – you have a listening service which other devices can connect to; or outgoing where you connect to a specific remote device. From Settings > Devices select “More Bluetooth options” from the righthand menu.

2020-11-22 (1)

From the resulting “classic” control panel applet select the “COM ports” tab to see configured ports or to set one up.

2020-11-22

When I reworked the 32feet.NET library I decided not to include the COM port functionality as it’s only relevant to Windows and is for legacy code only. However I put together the required code to enumerate all the Bluetooth virtual COM ports on the system in a Gist which you can find here:-

https://gist.github.com/peterfoot/b4f61c81023a1e181b9f3940bca344ba

I’ve just updated it as I noticed an issue with the null termination of the port names so it is now correctly trimmed. The class is very simple to use – the following will write out all the configured ports to the debug console:-

foreach (var p in BluetoothDiagnostics.BluetoothComPort.FindAll()) 
{
   System.Diagnostics.Debug.WriteLine($"{p.PortName} {p.DeviceId} {p.BluetoothAddress:X6}"); 
} 

This will match the contents of the Control Panel shown above.

Categories
Bluetooth Xamarin

Asyncify a method and event response

It’s common to come across a pattern where you call a synchronous method to kick off an activity then await an event which gives you the result (or an error). In the process of building 32feet‘s Bluetooth LE library this became a familiar sight both for Android and iOS APIs.

The goal was to create a single async method which starts the process and completes when the response (good or bad) is received. The API model for the resulting cross-platform API is based on Web Bluetooth but with some extensions. The Web Bluetooth API is JavaScript based and each operation returns a promise – the .NET approach is to use a Task which you can await or ContinueWith.

In an early implementation I had used a WaitHandle and had a thread blocking on this but this is not a clean async approach. I needed a solution which would handle the method and event handler and use a standard Task. Luckily there is the TaskCompletionSource type which can facilitate this. It wraps a Task and allows you to set a result (or exception) which we’ll be setting inside the event handler. Therefore each resulting method follows basically the pattern below:-

TaskCompletionSource<IReadOnlyList<GattDescriptor>> tcs = new TaskCompletionSource<IReadOnlyList<GattDescriptor>>();
CBPeripheral peripheral = Service.Device;

void handler(object sender, CBCharacteristicEventArgs args)
{
   peripheral.DiscoveredDescriptor -= handler;

   if (args.Error != null)
   {
      tcs.SetException(new Exception(args.Error.ToString()));
      return;
   }

   List<GattDescriptor> descriptors = new List<GattDescriptor>();

   foreach (CBDescriptor cbdescriptor in _characteristic.Descriptors)
   {
      descriptors.Add(new GattDescriptor(this, cbdescriptor));
   }

   tcs.SetResult(descriptors.AsReadOnly());
}

peripheral.DiscoveredDescriptor += handler;
peripheral.DiscoverDescriptors(_characteristic);

return tcs.Task;
  • Create a TaskCompletionSource with the required return type
  • Get the native object we’ll be starting the operation on
  • Create a handler for the event. Inside the handler we
    • Remove the event handler
    • Check the response – set either positive or negative result to TaskCompletionSource.
  • Add the event handler
  • Call the method to start the process
  • Return the TaskCompletionSource Task

I hope this provides a handy pattern to follow if you’re trying to use similar APIs, if not I’m sure it will at least jog my memory. I’ll post again soon about the full Bluetooth library in more depth.

Categories
Bluetooth Xamarin

Changes coming in Xamarin Android BluetoothSocket

While debugging an app I came across this new message in the Output window:-

"Not wrapping exception of type Java.IO.IOException from method `Read`. This will change in a future release."

When working with an RFComm service on Android you use a BluetoothSocket which in turn owns two Streams – one for Input and one for Output. Currently when there is a failure such as a broken connection the Read or Write operation will throw a Java.IO.IOException even though these streams are exposed as regular .NET Streams. It looks like Microsoft plan to change this behaviour and I presume this will be to wrap the exception in the equivalent System.IO.IOException. This makes a lot of sense and will improve consistency with other .NET flavours but it is something you need to be aware of as it will require code changes in your exception handling.

Categories
Bluetooth Windows Xamarin

New Year, New 32feet.NET Library

I’ve been working away on the Bluetooth code for some time. I’d been meaning to modernise the code and build with .NET Standard and NuGet in mind and ideally support more platforms. However the project has stopped and started a few times and gone down a few dead ends.

I had planned to rework the library into a full implementation of the UWP API set for Windows.Devices.Bluetooth covering Bluetooth Classic and Bluetooth LE but while I got quite far and had a set of working code for Windows, Android and iOS I decided that that wasn’t the right approach. Some aspects are clearly tied to underlying Windows concepts – the whole device finding/picking process uses magic strings and is not intuitive from a purely Bluetooth perspective. Also there was no real need to integrate both Bluetooth Classic and Bluetooth LE into a single library because there was very little shareable code and these could be maintained separately.

Instead I decided to go with a much simpler option. Build a new library around fundamentally the same API which the library has used for 16 years with a few necessary tweaks, cutting out a few of the more complex features and making it easy to do the most common tasks with Bluetooth Classic. The Bluetooth LE functionality will be revisited in 2020 with a more logical cross-platform API and packaged in a separate NuGet package (but still part of the 32feet.NET project).

What has changed

The library has been renamed InTheHand.Net.Bluetooth to make clear it is a different package and also to make clear the focus as I’ll no longer be including the IrDA functionality. It includes .NET Standard, Classic .NET Desktop, Android and iOS versions. As you can imagine there will be no new version for .NET Compact Framework. The iOS version is not yet complete but will have some limitations as imposed by the platform. I’ll discuss the details further in a separate blog post because it does some fun things over Apple’s quirky API and should make cross-platform Bluetooth Classic a lot easier for iOS.

In due course it will be joined by UWP, Linux and macOS implementations. Linux will require reworking for BlueZ 5 and macOS will be built on top of the IOBluetooth and IOBluetoothUI framework wrappers which are slowly taking shape. Potentially there may be other platforms too, such as Tizen, and hopefully we can grow the community around the project to target other platforms.

There is no BluetoothListener support in this release but it will be included in future builds (except on iOS). 95% of 32feet.NET users are using BluetoothClient to connect out to other devices so this wasn’t a top priority for this release.

The SelectBluetoothDeviceDialog which was heavily tied to WinForms has been replaced with the BluetoothDevicePicker. This offers a similar level of functionality but in a way which feels comfortable from any UI stack and supports async/await.

DiscoverDevices again is vastly simplified to remove concepts which only make sense in the Win32 implementation. Instead this method is used to discover “live” devices in the vicinity. It is accompanied by the new property PairedDevices which offers a more efficient way of retrieving paired devices without doing a live discovery.

When I started working on a modified version of the library for Unity development there was an odd problem. The Sockets stack in the Mono runtime doesn’t correctly support Bluetooth sockets and so the workaround was to P/Invoke the native Sockets APIs directly. In order to simplify the code the Win32 version uses a single implementation which calls the relevant Win32 APIs rather than using System.Net.Sockets. The library defines its own version of NetworkStream to be able to provide a familiar API which also not being tied to .NET Sockets. This makes it easier to support other platforms such as Android where the underlying implementation is completely different but can be wrapped inside a familiar .NET Stream. All of this means that the desktop .NET version of the library will work for the Windows target of a Unity project.

Where is it going?

The aim with this new version is to have a modern code base able to run on the current spectrum of .NET platforms – From IoT devices, to mobile phones to desktop PCs. Alongside the current Bluetooth Classic library and the Bluetooth LE library I’m also planning for other technologies to join the 32feet.NET project all aimed at handling short range device to device communications.

View the project on GitHub

See the latest InTheHand.Net.Bluetooth NuGet package

Categories
Bluetooth Xamarin

Xamarin macOS Binding Libraries

In creating an IOBluetooth binding for Xamarin Mac I learned about Objective Sharpie and binding libraries. There is little documentation on this but it is fairly similar to Xamarin iOS and for that there is a lot more source material. The output from Objective Sharpie gives you binding definitions which you can use in a dll project to produce a binding library you can then call from any other Xamarin dll or app. This was fine to a point but there are issues with some of the complex types used and these cannot always be marshalled automatically. This left me with an API with a few missing bits as I tried and failed to manually adjust the binding via trial and error.

The project sat around untouched for some time but recently I’ve begun to revive it and hope to sort out these bits so it can be released as a complete functioning API for Xamarin Mac. At first I thought I was going to have to create two libraries – one with the raw API calls and another with a clean API over the top but I had missed something buried in the docs and it turns out there is an easier solution.

When you have a binding library it will, by default, have two files – ApiDefinition.cs which contains all the API calls and has a Build action of BindingApiDefinition and a StructsAndEnums.cs which contains (well I’ll let you guess from the name) and this has a Build action of ObjcBindingCoreSource. When the classes are generated from the interface definitions in ApiDefinition.cs they are actually partial classes. This means you can extend them and have additional functionality built cleanly into the library. If you have a particularly messy API call you can mark it as internal and then surface it in a more friendly way from a partial class. To do this add another source file to the library project (I’ve called it Extra.cs because I saw that in a sample but the name isn’t important) and set the Build action to Compile. Here you’ll need to create a partial class with the same name and namespace as the “interface” you want to extend from ApiDefinition.cs, and then add methods, properties etc.

The first time I added this my build failed. I subsequently found out that there is one additional step to tell the binding compiler to ignore this file. Open the project properties, under Build select the Objective-C Binding Build page. Here in Additional btouch arguments box add -x:Extra.cs (or replace with your own filenames). This stops the initial binding compilation from using the partial class, which then gets built normally in the subsequent managed code build. The project should now build and expose the combined functionality. I did find that intellisense often gets confused when editing the partial class because there isn’t another definition of a partial class at this time (remember in ApiDefinition.cs it’s actually an interface). However it seems you can safely ignore this!

This in theory allows you to completely change the API surface which you expose to Xamarin from whatever you started with. I don’t want to go too crazy with IOBluetooth – my feeling is that it should match the native API with a few tweaks for C# naming conventions, using namespaces rather than huge class names, and .NET friendly types where appropriate. Objective Sharpie struggled with some of the enum/constant definitions and so these still require a bit of massaging. It should be obvious how it maps to the native API.

If you have feedback on the API or would like to get involved in getting the library up to release standard please let me know. All the current code is on GitHub in the IOBluetooth and IOBluetoothUI folders.

Categories
Bluetooth

32feet.NET and Audio

There are a few different Bluetooth profiles which handle audio, but they all work in a very similar way. There are two connections open between the client (usually a phone) and the server (some kind of audio device such as a speaker or car entertainment system).

The first of these is an Rfcomm channel which handles commands between the devices. Rfcomm is essentially a serial connection emulated over Bluetooth and these commands are often a mixture of AT commands from the world of modems and other commands for associated functionality (think phone book contacts, track names etc).

The second channel is a low level SCO (Synchronous Connection-Oriented) connection which is better suited to real-time audio data. Depending on the profile this may be used for one-way (audio) or two-way (hands-free etc) audio.

32feet.NET has only support for Rfcomm out of the box. This means it is possible to establish a connection to a headset device and even do things like capture button presses and send rings but it does not support opening an audio channel. Also if you connect to a headset device or similar rather than use the platform’s built in support you’ll block the device from using its native functionality. Mobile devices have support and drivers for headset/hands-free etc and this will go through the normal audio APIs on the platform so there is rarely a need to try and interfere with this.

If you want your app to play audio over Bluetooth then pair the device with the OS and just play audio and the system will handle it for you.

Categories
Bluetooth

Bluetooth with Xamarin Mac

I’ve been working on adding macOS support to 32feet.NET and there are two frameworks in macOS for Classic Bluetooth – IOBluetooth and IOBluetoothUI. I soon discovered that neither of these had bindings in the standard Xamarin.Mac package which is referenced by all Xamarin Mac applications. I decided to build binding libraries for both APIs and publish the code as part of 32feet.NET. This means you can either use the IOBluetooth lower-level APIs yourself or later use the platform-agnostic 32feet API.

Today I’ve published the first release of the InTheHand.IOBluetoothUI package. There are fewer APIs than IOBluetooth and I’ve already begun the manual process of simplifying and making it more “.NET friendly”. There is also documentation to add, though even Apple’s documentation on IOBluetooth is rather thin at best…

These are by no means final and there will be changes to the APIs as names are cleaned up and more is tested and fixed. If you’d like to try them in your projects please let me know your feedback via GitHub. Those NuGet packages are:-

https://www.nuget.org/packages/InTheHand.IOBluetooth/

https://www.nuget.org/packages/InTheHand.IOBluetoothUI/

 

Categories
Bluetooth

Bluetooth from Unity

An ongoing issue with 32feet.NET is that it wouldn’t work inside Unity. The reason is that the System.Net.Sockets classes behave slightly differently in the Mono runtime to the desktop .NET framework and you can’t create a Socket using the Bluetooth specific address family.

In order to work around the issue it was necessary to P/Invoke into the native winsock functions, essentially rebuilding a subset of the Socket class. In parallel to this work I’ve been rebuilding 32feet with a more modern API which is less tied to Sockets (primarily just used on desktop Windows) and able to map onto a range of platforms. Another big change for this version is support for Bluetooth LE alongside classic Rfcomm on supported platform. Currently this library supports Xamarin Android and iOS along with UWP, Windows desktop .NET 4.6 and Mono .NET 2.0 for Unity. I’m working on a macOS implementation too. The API is essentially designed to be a more friendly version of the UWP API. In order to support such an old version of .NET, the Unity version is entirely synchronous whereas most of the API is normally async.

In order to test this I wrote a very simple script for Unity which picks a specific paired device, connects to a serial port service over Rfcomm and sends a string. Yes that’s right I have a 3d game that I can print from!

This is currently in preview (but available on NuGet now). There is a lot still to finish including generating the documentation. I’m hoping for some useful feedback, particularly on the Unity work but also any of the other current platforms. Feel free to join in the discussions on GitHub.

Categories
Bluetooth Windows

Read iBeacons from UWP

I recently got some estimate beacons and have been trying out various things with them. By default they are configured to support Apple’s iBeacon format and could be used in an iOS app to provide location awareness in a close environment. You can read the same data from UWP and can add some location/context awareness in this way. In this post I’ll just discuss the iBeacon approach.

In UWP development there is a BluetoothLEAdvertisementWatcher which is used to read advertisement data from nearby Bluetooth Low Energy devices. The watcher fires the Received event for each advertisement found and you can read the data as required. The key to using iBeacon is to understand how the data is encoded. Advertisement sizes are limited so they need to be designed to be as compact as possible while providing enough information to uniquely identify each device. The iBeacon format consists of a UUID (Guid) and two unsigned short integers. These should be thought of as a hierarchical format:-

UUID > Major > Minor

A location aware app would use a unique UUID for its own use, for example a chain of stores. The Major id would then represent an individual store and the Minor id a location within that store. In iOS the raw iBeacon advertisements are “hidden” from the CoreBluetooth API and instead exposed by CoreLocation. In UWP we use the BluetoothLEAdvertisementWatcher and reconstruct the elements of the beacon. These are stored in a ManufacturerData section with Apple’s manufacturer id (0x4C) used. Within this we access the raw data as an iBuffer (WinRT/UWP equivalent of a byte[] array). The DataReader class is used to sequentially read through the data. The data is:-

Byte 0 – type – 2 for iBeacon

Byte 1 – data length: 21 bytes for iBeacon

Bytes 2-17: UUID

Bytes 18-19: Major ID

Bytes 20-21: Minor ID

We must be careful to respect the byte ordering of the Guid element, the following Gist wraps up the operation:-

public App()
{
this.InitializeComponent();
this.Suspending += OnSuspending;
_watcher = new BluetoothLEAdvertisementWatcher();
_watcher.Received += _watcher_Received;
_watcher.Start();
}
private void _watcher_Received(BluetoothLEAdvertisementWatcher sender, BluetoothLEAdvertisementReceivedEventArgs args)
{
foreach(BluetoothLEManufacturerData md in args.Advertisement.ManufacturerData)
{
if(md.CompanyId == 0x4C) // Apple
{
DataReader reader = DataReader.FromBuffer(md.Data);
byte advertismentType = reader.ReadByte(); // 0x02 – iBeacon
byte len = reader.ReadByte(); // 0x15 (21) – iBeacon
int a = reader.ReadInt32();
short b = reader.ReadInt16();
short c = reader.ReadInt16();
byte[] d = new byte[8];
reader.ReadBytes(d);
Guid uuid = new Guid(a, b, c, d);
ushort major = reader.ReadUInt16();
ushort minor = reader.ReadUInt16();
Debug.WriteLine(uuid + " " + major + " " + minor + " " + args.RawSignalStrengthInDBm);
}
}
}

view raw
App.xaml.cs
hosted with ❤ by GitHub

The event also gives you the Rssi, you can use this to make a general assumption about the relative distance of multiple beacons but should not assume a direct measurement of distance from it.

Categories
Bluetooth GitHub

Migrating Wiki content from CodePlex to GitHub

I’ve seen a few articles on migrating your projects from CodePlex but they kind of ignore the Wiki and suggest just copying and pasting the text across. There is a way which will copy the entire contents and only require a little manual work. The instructions below are the steps I took to migrate the 32feet documentation across. There were 76 files in total so much easier that copying and pasting!

First from the Home page of your CodePlex project you’ll see a button “Download Wiki” in the page toolbar. Click this to download a Zip file containing all the documents and supporting attachments for your Wiki.

migrate-wiki-1

This contains two folders, one in raw CodePlex format and the other (called “docs”) in Markdown. You’ll want the Markdown version. There are two standard files in this folder – Home.md is the homepage – the equivalent to your Readme on GitHub. This one you’ll probably want to copy and paste into a Readme.md file in your new repository. The Documentation.md is the entry point into your documentation Wiki. I removed the Home.md after copying the contents and renamed Documentation.md to Home.md as this is now the entry point into the Wiki documentation. If there are hard links back from child pages these may require fixing.

migrate-wiki-2.png

In your GitHub project go to the Wiki tab and you’ll see an option on the right “Clone this wiki locally”. You can then use Visual Studio or any Git tool of your choice to work on a local clone of the Wiki repository.

migrate-wiki-3.png

This should be empty for a new GitHub project. Once you’ve done this you can copy all the Markdown and attachment files you downloaded and unzipped from CodePlex into this folder. Commit and Push this git repository and you’ve uploaded a (mostly) working Wiki to your new site.

After doing this I noticed a couple of issues which required further changes. Firstly the inline code examples were broken:-

migrate-wiki-4

A bit of digging revealed a slightly different “tag” for code in GitHub markdown which should be:-

migrate-wiki-5

Another quick fix was that GitHub requires tables to have a preceding blank line otherwise it just renders as raw text full of pipes.

The last big formatting issue was the main Wiki page uses a number of anchors to various points in the table of contents and this was broken in conversion. It appears that there are automatically generated anchors for top level headers only. It is possible to workaround (although it feels dirty) by using inline HTML A tags e.g. replace

{anchor:General Bluetooth Data Connections}

with

and link to it using

[General Bluetooth Data Connections](#user-content-general-bluetooth-data-connections)

I hope this helps you as you migrate your own projects from CodePlex.