gst-plugins-cl - OpenCL plugins for gstreamer, written in Vala.

Some of you may know OpenCL - the open computing language. This language allows us to write kernels in a C99 dialect, which can be run on data - in parallel.
A common use case for such a language - and processing concept - is the classical convolution - which was my motivation. Sure, this can also be realized using the frequency domain, but - hey - I feel fine in the spatial one.

Anyhow, there are bindings for OpenCL - which is vendor (NVidia, AMD, Intel, F/LOSS, ...) and hardware (GPU vs. CPU - AMD, Intel, IBM Cell, ...) independent for many languages, including MatLab, python, C, C++  and so on ...
In my case I needed a way to apply some filter function (or [OpenCL] kernel) to a video stream from some FireWire camera.
The experienced reader might directly associate FireWire with gstreamer our multimedia framework of choice under linux.
I had the option to Gather the data from a gstreamer pipeline reading the FireWire camera or create a gstreamer element providing, as of wrapping, the OpenCL functionality. The mentioned plugin does the latter, it provides a couple of elements to apply some OpenCL kernel to the data of a gstreamer pipeline.

You need vala and some gstreamer sources to build this project:

$ sudo yum install gstreamer-devel vala

.. might be a good start.
You also need some OpenCL implementation, as of today pocl can be used, but it's just intensively tested with the Nvidia implementation of it's GPU SDK and with Intel OCL SDK.
Building the plugin is quite simple

$ ./autogen.sh
$ ./configure
$ make

Testing should also be quite easy:

$ export GST_PLUGIN_PATH=src/.libs/
$ gst-launch audiotestsrc ! clkernel ! autoaudiosink

This doesn't do much, as the default kernel just passes the data - but at least you can test if the OpenCL is available. There are currently three plugins:

• clkernel - An element to manipulate any buffer
• clkernel2d - An element to manipulate a 2D buffer, like an gray image/video
• clvideofilter - An element to manipulate an RGBA image/video

The OpenCL kernel can reside in an external file. You can pass the filename of that file and the kernel to use to the element:

$ gst-launch audiotestsrc \
  ! clkernel kernel-file=mykernellib.cl kernel-name=somekernel \
  ! autoaudiosink

Have a look at the sources or kernel.cl and kernel2.cl files (they are plaintext files, the kernel are build on the fly - heard of JIT?) in the src directory to see what kind of signature the kernel has to provide. (This should be documented more thoroughly).
If you've got more than one platform which is OpenCL capable you can use the platform-idx property to specify a specific platform.

This plugins should help to offload some functionality to one or more devices (like GPU, CPU and accelerator, like the CELL) and you don't need to care about the details.
This can now be used to do things like color conversion, all sorts of filtering, mixing and what else can be done in parallel.
Thos plugins are just a start, it would be nice to be able to provide more sinks which then can be used as an input to the kernels.

~ https://gitorious.org/valastuff/gst-plugins-cl