Getting image watch like debugging in Qt Creator

So a big sticking point for me with working on Linux was the lack of Image Watch which is a great extension for viewing OpenCV images in memory.  Without this functionality debugging can be a huge pain.

Luckily there is a reasonable solution that works nearly as well.

By using GDB's python scripting capabilities, we can download the image from the program that we're debugging and display it with OpenCV's python bindings in an OpenCV imshow window.

I really liked the ability to zoom in on a pixel and view the actual numeric value.  Luckily if you compile OpenCV with Qt and OpenGL, that functionality is available.  I'm not sure which one actually gives this functionality, because I just use both anyways.


Anyways on to the process:


Compile OpenCV with Qt, OpenGL and Python2.7 from source.

This creates a cv2.pyd file that contains the correct python bindings that we need.

Check if GDB is built against python 2 or python 3.  If it's built against python 3 we'll have some issues.  Primarily because OpenCV doesn't have great python 3 support yet.  You can do this in gdb by typing:

python print(sys.version_info)

If this says it is Python 3, we'll have to either build a version of gdb that is compiled against Python 2 or use a different gdb.  I ended up downloading gdb and building it from source against python 2 (all I did was set the python option to on with:

./configure --with-python

Alternatively, if you're using cuda-gdb it already is built against Python 2.

Next make sure gdb can load OpenCV's python module, inside gdb type:

python import cv2

If that works correctly, then hopefully you're accessing the correct build of opencv.  If it doesn't work, or it's the wrong version of OpenCV, take the cv2.pyd and cv2.so file that was compiled from source and put it in your gdb data-drectory. (This can be found from within gdb with "show data-directory")

Now to the fun part, dumping a cv::Mat in python and displaying it with cv2.imshow.

A nice little script was started by Asmwarrior which works for previous versions of OpenCV.

I've since updated it to work appropriately.  It isn't finished but I'll update this as I improve it.

***************************** Script Start **********************************

import gdb
import cv2
import sys
import numpy


class PlotterCommand(gdb.Command):
    def __init__(self):
        super(PlotterCommand, self).__init__("plot",
                                             gdb.COMMAND_DATA,
                                             gdb.COMPLETE_SYMBOL)
    def invoke(self, arg, from_tty):
        args = gdb.string_to_argv(arg)
     
     
        # generally, we type "plot someimage" in the GDB commandline
        # where "someimage" is an instance of cv::Mat
        v = gdb.parse_and_eval(args[0])
     
        # the value v is a gdb.Value object of C++
        # code's cv::Mat, we need to translate to
        # a python object under cv2.cv
        image_size =  (v['cols'],v['rows'])
        # print v
        # these two below lines do not work. I don't know why
        # channel = gdb.execute("call "+ args[0] + ".channels()", False, True)
        # channel = v.channels();
        CV_8U =0
        CV_8S =1
        CV_16U=2
        CV_16S=3
        CV_32S=4
        CV_32F=5
        CV_64F=6
        CV_USRTYPE1=7
        CV_CN_MAX = 512
        CV_CN_SHIFT = 3
        CV_MAT_CN_MASK = (CV_CN_MAX - 1) << CV_CN_SHIFT
        flags = v['flags']
        channel = (((flags) & CV_MAT_CN_MASK) >> CV_CN_SHIFT) + 1
        CV_DEPTH_MAX = (1 << CV_CN_SHIFT)
        CV_MAT_DEPTH_MASK = CV_DEPTH_MAX - 1
        depth = (flags) & CV_MAT_DEPTH_MASK
        IPL_DEPTH_SIGN = 0x80000000
        cv_elem_size = (((4<<28)|0x8442211) >> depth*4) & 15
        if (depth == CV_8S or depth == CV_16S or depth == CV_32S):
                mask = IPL_DEPTH_SIGN
        else:
                mask = 0
        ipl_depth = cv_elem_size*8 | mask  

     
        # conver the v['data'] type to "char*" type
        char_type = gdb.lookup_type("char")
        char_pointer_type =char_type.pointer()
        buffer = v['data'].cast(char_pointer_type)
     
        # read bytes from inferior's memory, because
        # we run the opencv-python module in GDB's own process
        # otherwise, we use memory corss processes      
        buf = v['step']['buf']
        bytes = buf[0] * v['rows'] # buf[0] is the step? Not quite sure.
        print('Reading image')
        inferior = gdb.selected_inferior()
        mem = inferior.read_memory(buffer, bytes)
        print('Read successful')
        if(depth == CV_8U):
            print('8bit image')
            img = numpy.frombuffer(mem, count=bytes, dtype=numpy.uint8)
        if(depth == CV_16U):
            print('16bit image')
            img = numpy.frombuffer(mem, count=bytes, dtype=numpy.uint16)
        img = img.reshape((v['rows'], v['cols'], channel))
        print(img.shape)
        # create a window, and show the image

        cv2.imshow("debugger", img)
     
        # the below statement is necessory, otherwise, the Window
        # will hang
        cv2.waitKey(0)
PlotterCommand()

***************************** Script End **********************************

Now we just edit .gdbinit to auto load the python script by adding the following to .gdbinit:

source {path to script}

And now we can view an opencv image in gdb with:

plot {variable name}


Now this isn't quite as nice as what it could be, QtCreator has built in image viewing functionality for QImage's.  Unfortunately I've already spent a good bit of time trying to get that to work, but with no luck.  So for now, I just keep the gdb console open in QtCreator and use this method.

Bugs:

Running into "Gtk-CRITICAL **: IA__gtk_widget_style_get: assertion `GTK_IS_WIDGET (widget)' failed" when trying to display an image.
Execute the following before opening QtCreator:
export LIBOVERLAY_SCROLLBAR=0