When working with the APIs in Visionary Render (VisRen) to integrate a third party service, it is quite common for that service to involve communicating over the internet. This type of communication can take between tens and thousands of milliseconds, and so for real-time graphical applications like VisRen it is important that this type of operation is performed asynchronously - that is, it will not stop the rest of the application from running while it is waiting for the result.
The plugin API in VisRen provides a number of entry points for plugins to use - and most of these are executed by the main thread - the same thread that does the rendering. It does this because it needs to have the latest information available before drawing to the screen, and plugins need to be able to update that information too.
Asynchronous results
So how do we, as plugin authors, make sure we use this data safely? Usually it depends what we need to do with it. Let’s use an example like the Speech to Text plugin - this plugin takes results of speech to text, which are provided asynchronously, and puts them into the Lua state in VisRen, which needs to be updated on the main thread.
How do we achieve this?
There are three distinct “parts” we need for this.
- The code that receives the data from the service (asynchronously)
- A message queue to queue up the speech result from its callback (which is probably not the main thread)
- A function that does run on the main thread, which reads the queue and controls the Lua state
The first two are straightforward in this case - and C# makes this easy with its <inline-code>ConcurrentQueue<inline-code> class.
Queue up the speech results
We define a queue to receive spoken text, somewhere it is available to both the callbacks from the external service, and our main-thread function
Then our speech recogniser callback can push into this queue safely
Put the queue contents into Lua
The VisRen Lua environment has a generic callback system which is ideal for this. There is a function for registering an arbitrary function as a named callback, <inline-code>__registerCallback("some-arbirary-name", function() … end)<inline-code>, and a function for calling named callbacks, <inline-code>__callback("some-arbirary-name")<inline-code>.
We can make use of this and define our own callback for “onSpeech”, which we can call with the text from the queue.
We need to do this on the main thread, and the plugin API includes an “Update” callback, which is called once every frame before rendering takes place. This is a good place to gather results of asynchronous tasks and process them safely.
In the implementation of our update function we can do something like this to process the queue and trigger the callbacks:
The FFI (foreign function interface) is part of the native API and it enables direct control over the Lua environment in VisRen. Here we are calling the callback with the spoken text. So if we said “Hello”, this executes <inline-code> __callback("onSpeech", "Hello")<inline-code>in the Lua environment.
And there we have a thread-safe way of getting data, asynchronously, and passing it through to Lua where scripts in a scene can make use of it, by calling <inline-code>__registerCallback("onSpeech", function(text) print(text) end)<inline-code>.