A Visual Studio Debugger Extension for the Raspberry Pi Camera


While developing a sample application for a Raspberry Pi with a camera using the RaspiCam library, it occurred to me that it would be convenient and fun to be able to see the current camera input while debugging the application.  The Visual Studio debugger supports type-specific custom visualizers and end-user extensions that implement UI for these visualizers. I decided to make one for the RaspiCam camera types that would display the current image from the camera. The image below is the end result, showing Visual Studio debugging a program running on the Raspberry Pi and displaying the content of a Raspberry Pi camera object in a pop-up debugger visualizer.

Picture of rasperry pi displaying camera

Build a RaspiCam Application


"configurePresets": [ { "name": "raspberrypi-Debug", "generator": "Ninja", "binaryDir": "${sourceDir}/out/build/${presetName}", "installDir": "${sourceDir}/out/install/${presetName}", "cacheVariables": { "CMAKE_BUILD_TYPE": "Debug", "CMAKE_TOOLCHAIN_FILE": "C:/temp/raspberry/toolchain-rpi.cmake", "OpenCV_DIR": "D:\opencv-4.5.5\opencv-4.5.5\out\install\RaspberryPi-Debug\lib\cmake\opencv4", "raspicam_DIR": "D:\raspicam\out\install\raspberrypi-Debug\lib\cmake" }, "environment": { "RASPBIAN_ROOTFS": "c:/temp/raspberry/arm-linux-gnueabihf/sysroot", "PATH": "c:/temp/raspberry/bin;$env{Path}" } } ], "buildPresets": [ { "name": "cross-build", "environment": { "PATH": "c:/temp/raspberry/bin;$penv{PATH}" }, "configurePreset": "raspberrypi-Debug" } ]

My Raspberry Pi application uses the OpenCV library and the RaspiCam_Still_Cv camera type for capturing and manipulating the camera image.

   raspicam::RaspiCam_Still_Cv Camera;     cv::Mat image;    Camera.open();    Camera.grab();    Camera.retrieve(image);     cv::imwrite("raspicam_cv_image.jpg", image);

I wanted to debug this application on the Pi so I added the launch configuration below. This instructs the debugger to first deploy the application to my RaspberryPi device in directory ~/camera. I copied all the shared OpenCV and RaspiCam libraries I needed to this directory too.

   {      "type": "cppgdb",      "name": "DebugOnPi",      "project": "CMakeLists.txt",      "projectTarget": "simpletest_raspicam",      "debuggerConfiguration": "gdb",      "MIMode": "gdb",      "args": [],      "env": {},      "deployDirectory": "~/camera",      "remoteMachineName": "",      "preDebugCommand": "export LD_LIBRARY_PATH=~/camera"     }

I could now debug on the Raspberry Pi.

Add Visualizers to the Linux Debugger

Although the Visual Studio Windows debugger supports UI visualizers, it turned out that the Visual Studio Linux debugger did not have this feature implemented.  Since the Raspberry Pi runs Raspian, a Debian Linux variant, the first thing I had to do was fix that. The Linux debugger is an open source project called MIEngine. The MIEngine runs as a Visual Studio debugger “engine” that controls a gdb process running remotely on a Linux host. The MIEngine already supported custom variable visualization using natvis files (example below). However, it did not support the UIVizualizer tag in natvis files which is necessary to open a new visualization window containing, for example, a camera image. The work described here is now part of the MIEngine and in Visual Studio. The completed PR with the changes is here. Remember it’s open source, so you too can contribute in this way!


There were two changes I needed to make to the MIEngine: First was to recognize and process the UIVisualizer element in natvis files and second was to automatically load natvis files for registered Visual Studio extensions.

  • Recognizing the element was simple. The MIEngine already parsed natvis files, it was just ignoring the UIVisualizer element. I simply changed the natvis lookup code to return not just the visualized string value for an expression, but also a list of any UIVisualizers that were present for the expression type. The MIEngine returns an expression value to the Visual Studio debugger as an AD7Property. I updated the AD7Property class to also return the list of UIVisualizers found.
  • The MIEngine makes requests of Visual Studio via its DebugEngineHost. In order to support visualization, I needed to call into Visual Studio from the DebugEngineHost and ask it for an IVsExtensionManagerPrivate service reference. I then called into the resulting service asking for a list of all resources tagged as “NativeCrossPlatformVisualizer”. It returned a list of file names for resources that have this tag. These are given to the MIEngine natvis processor for parsing.

Finally, there was an additional feature that was missing from the MIEngine. It didn’t have a convenient way for a caller holding an AD7Property object to determine the execution context (that is, stack frame) where the expression that generated that property was evaluated. So, I extended the AD7Property class to implement a new interface IDebugMIEngineProperty that contains a method for returning the properties IDebugExpressionContext2. Now all the parts are in place for developing a UIVisualizer for Linux applications.

Create a Visualizer VSIX Project

The UIVisualizer for a data type is identified in natvis by a GUID ServiceId and an integer Id. These are the values the debugger will use to find and open the associated visualizer. The ServiceId identifies a type that implements VsCppDebugUIVisualizer. The Id identified which of its visualizers to open – in the case that more than one visualizer is supported by the service. I needed to author a service for my camera picture viewer as a visual studio extension.

To start, I created a VSIX (Visual Studio extension project) called PictureViewer (you need to have the “Visual Studio extension development” workload installed). Then I defined an interface for my visualizer tagged with my viewer’s ServiceId and attributed the VSIX Package object with this interface. I now had to define the relationship between the UIVisualizer element in the natvis file with this VSIX package.

[Guid("0A73397B-D550-4BFE-94C9-C0E5122DC06F")] public interface IPictureViewerService { } … [PackageRegistration(UseManagedResourcesOnly = true, AllowsBackgroundLoading = true)] [ProvideService(typeof(IPictureViewerService), ServiceName = "PictureViewerService", IsAsyncQueryable = true)] [Guid(PictureViewerPackage.PackageGuidString)] public sealed class PictureViewerPackage : AsyncPackage { … }

I then added a new PictureViewerService that implements both IPictureViewerService and IVsCppDebugUIVisualizer. The only method it implements is IVsCppDebugUIVisualizer.DisplayValue.

int DisplayValue(uint ownerHwnd, uint visualizerId, IDebugProperty3 debugProperty)

This is the API called by the debugger to display a value using a custom UIVisualizer. The already evaluated expression value is passed to the visualizer in the debugProperty object.

I implemented my UI in class PictureViewerViewModel. The UI in this example is very simple, just an Image and a Button in a pop-up window. When the service activates the UI the view model object fires off a task to load the image from the application being debugged and then invokes ShowDialog. The XAML snippet below defines the content of the dialog.


Get paid to share your bandwidth! Peer2Profit is the BEST side income for you! → https://www.peer2profit.io