[.NET 6][Uno Platform][Ubuntu 21.10] Using Leap Motion

Intro
In this time, I will try using Leap Motion Controller from the Uno Platform application on Ubuntu.
I use this project to add LeapSDK.
Environments
  • Ubuntu 21.10
  • .NET 6.0.101
  • LeapDeveloperKit ver.2.3.1
  • Using Leap Motion Controller on Ubuntu
    To use Leap Motion Controller on Ubuntu I have to install tracking software.
    After unpacking the tgz file, install a deb file.
    sudo dpkg --install Leap-2.3.1+31549-x64.deb
    And I also install Mesa OpenGL.
    sudo apt-get install libgl1-mesa-dri libgl1-mesa-glx
    After installing, I start the daemon of Leap Motion.
    sudo leapd
    I can open a control panel.
    LeapControlPanel &
    To open the control panel, I have to execute the command twice.
    I don't know why :(
    Controller was not working
    Though I didn't get any errors, my machine hasn't recognized the controller yet.
    According to this post, I could know I had to add a config file(leapd.service).
    Now I can use the Leap Motion Controller on the control panel.
    Add a LeapSDK library file in .NET 6 projects
    In the tgz file, I can also get library files to control the Leap Motion Controller.
    So I add a dll file for .NET Framework 4.0.
    UnoSample.Skia.Gtk.csproj
    <Project Sdk="Microsoft.NET.Sdk">
      <PropertyGroup>
        <OutputType Condition="'$(Configuration)'=='Release'">WinExe</OutputType>
        <OutputType Condition="'$(Configuration)'=='Debug'">Exe</OutputType>
        <TargetFramework>net6.0</TargetFramework>
        <LangVersion>latest</LangVersion>
        <Nullable>enable</Nullable>
        <ImplicitUsings>enable</ImplicitUsings>
      </PropertyGroup>
      <ItemGroup Condition="exists('..\UnoSample.UWP')">
        <EmbeddedResource Include="..\UnoSample.UWP\Package.appxmanifest" LogicalName="Package.appxmanifest" />
        <Content Include="..\UnoSample.UWP\Assets\StoreLogo.png" Link="Assets\StoreLogo.png" />
        <Content Include="Assets\Fonts\uno-fluentui-assets.ttf" />
      </ItemGroup>
      <ItemGroup>
        <UpToDateCheckInput Include="..\UnoSample.Shared\**\*.xaml" />
      </ItemGroup>
      <ItemGroup>
        <Reference Include="Leap">
          <HintPath>References/LeapSdk/LeapCSharp.NET4.0.dll</HintPath>
        </Reference>
      </ItemGroup>
      <ItemGroup>
        <PackageReference Include="Microsoft.Extensions.Logging" Version="5.0.0" />
        <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="5.0.0" />
        <PackageReference Include="Uno.UI.Skia.Gtk" Version="4.0.11" />
        <PackageReference Include="Uno.UI.RemoteControl" Version="4.0.11" Condition="'$(Configuration)'=='Debug'" />
        <PackageReference Include="Uno.UI.Adapter.Microsoft.Extensions.Logging" Version="4.0.11" />
      </ItemGroup>
      <Import Project="..\UnoSample.Shared\UnoSample.Shared.projitems" Label="Shared" />
    </Project>
    libLeapCSharp can't be found
    Though I don't get any compile errors and I can get static value like "Gesture.GestureType.TYPE_CIRCLE",
    when I add classes for controlling the Leap Motion Controller, I get a runtime exception.
    LeapCtrlListener.cs
    using Leap;
    
    namespace UnoSample.Skia.Gtk.MotionControllers
    {
        public class LeapCtrlListener: Listener
        {
            public LeapCtrlListener()
            {
                Console.WriteLine("LeapCtrlListener");
            }
        }
    }
    Program.cs
    using GLib;
    using Uno.UI.Runtime.Skia;
    using Leap;
    using UnoSample.Skia.Gtk.MotionControllers;
    
    namespace UnoSample.Skia.Gtk
    {
        class Program
        {
            static void Main(string[] args)
            {
                // OK. "Gesture: TYPE_CIRCLE"
                Console.WriteLine("Gesture: " + Gesture.GestureType.TYPE_CIRCLE);
    
                // Error
                var leapCtrlListener = new LeapCtrlListener();
    
                ExceptionManager.UnhandledException += delegate (UnhandledExceptionArgs expArgs)
                {
                    Console.WriteLine("GLIB UNHANDLED EXCEPTION" + expArgs.ExceptionObject.ToString());
                    expArgs.ExitApplication = true;
                };
    
                var host = new GtkHost(() => new App(), args);
    
                host.Run();
            }
        }
    }
    Error
    Unhandled exception. System.TypeInitializationException: The type initializer for 'Leap.LeapPINVOKE' threw an exception.
     ---> System.TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception.
     ---> System.DllNotFoundException: Unable to load shared library 'LeapCSharp' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libLeapCSharp: cannot open shared object file: No such file or directory
       at Leap.LeapPINVOKE.SWIGExceptionHelper.SWIGRegisterExceptionCallbacks_Leap(ExceptionDelegate applicationDelegate, ExceptionDelegate arithmeticDelegate, ExceptionDelegate divideByZeroDelegate, ExceptionDelegate indexOutOfRangeDelegate, ExceptionDelegate invalidCastDelegate, ExceptionDelegate invalidOperationDelegate, ExceptionDelegate ioDelegate, ExceptionDelegate nullReferenceDelegate, ExceptionDelegate outOfMemoryDelegate, ExceptionDelegate overflowDelegate, ExceptionDelegate systemExceptionDelegate)
       at Leap.LeapPINVOKE.SWIGExceptionHelper..cctor()
       --- End of inner exception stack trace ---
       at Leap.LeapPINVOKE.SWIGExceptionHelper..ctor()
       at Leap.LeapPINVOKE..cctor()
       --- End of inner exception stack trace ---
       at Leap.LeapPINVOKE.new_Listener()
       at Leap.Listener..ctor()
       at UnoSample.Skia.Gtk.LeapMotions.LeapController..ctor() in /home/example/Documents/workspace/UnoSample/UnoSample.Skia.Gtk/LeapMotions/LeapController.cs:line 7
       at UnoSample.Skia.Gtk.Program.Main(String[] args) in /home/example/Documents/workspace/UnoSample/UnoSample.Skia.Gtk/Program.cs:line 13
    Add link to shared object file
    To use the library file, I have to add link to "libLeapCSharp.so" on my machine.
    I can execute this command before run the project.
    export LD_LIBRARY_PATH="$Home/home/example/Documents/leap_developer_kit"
    (In "leap_developer_kit" folder, there are "libLeapCSharp.so", "libLeap.so", and so on)
    But because I don't want to do this everytime.
    So I add "leap_developer_kit.conf" into "/etc/ld.so.conf.d".
    leap_developer_kit.conf
    /home/example/Documents/leap_developer_kit
    After I execute "sudo ldconfig", I can watch the link of the files by "ldconfig -p".
    ...
      libLeapCSharp.so (libc6,x86-64) => /home/example/Documents/leap_developer_kit/libLeapCSharp.so
        libLeap.so (libc6,x86-64) => /home/example/Documents/leap_developer_kit/libLeap.so
    ...
    (Sample) Getting hand count
    According to the sample code in the tgz file, I add classes and get tracking hand count.
    Program.cs
    using GLib;
    using Uno.UI.Runtime.Skia;
    using UnoSample.Skia.Gtk.MotionControllers;
    
    namespace UnoSample.Skia.Gtk
    {
        class Program
        {
            static void Main(string[] args)
            {
                using(var motionController = new MotionController())
                {
                    ExceptionManager.UnhandledException += delegate (UnhandledExceptionArgs expArgs)
                    {
                        Console.WriteLine("GLIB UNHANDLED EXCEPTION" + expArgs.ExceptionObject.ToString());
                        expArgs.ExitApplication = true;
                    };
                    var host = new GtkHost(() => new App(), args);
                    host.Run();
                }
            }
        }
    }
    MotionController.cs
    using Leap;
    
    namespace UnoSample.Skia.Gtk.MotionControllers;
    public class MotionController: IDisposable
    {
        private LeapCtrlListener leapCtrlListener;
        private Controller controller;
        public MotionController()
        {
            this.leapCtrlListener = new LeapCtrlListener();
            this.controller = new Controller ();
            this.controller.AddListener(this.leapCtrlListener);
        }
        public void Dispose()
        {
            this.controller.RemoveListener(this.leapCtrlListener);
            this.controller.Dispose();
        }
    }
    LeapCtrlListener.cs
    using Leap;
    
    namespace UnoSample.Skia.Gtk.MotionControllers;
    public class LeapCtrlListener : Listener
    {
        public LeapCtrlListener()
        {
            Console.WriteLine("LeapCtrlListener");
        }
        public override void OnInit(Controller controller)
        {
            Console.WriteLine("Initialized");
        }
        public override void OnConnect(Controller controller)
        {
            Console.WriteLine("Connected");
            controller.EnableGesture(Gesture.GestureType.TYPE_CIRCLE);
            controller.EnableGesture(Gesture.GestureType.TYPE_KEY_TAP);
            controller.EnableGesture(Gesture.GestureType.TYPE_SCREEN_TAP);
            controller.EnableGesture(Gesture.GestureType.TYPE_SWIPE);
        }
        public override void OnDisconnect(Controller controller)
        {
            //Note: not dispatched when running in a debugger.
            Console.WriteLine("Disconnected");
        }
        public override void OnExit(Controller controller)
        {
            Console.WriteLine("Exited");
        }
        public override void OnFrame(Controller controller)
        {
            var leapFrame = controller.Frame();
            var handList = leapFrame.Hands;
            Console.WriteLine($"OnFrame Hand: {handList.Count}");
        }
    }

    35

    This website collects cookies to deliver better user experience

    [.NET 6][Uno Platform][Ubuntu 21.10] Using Leap Motion