Opus now/soon in gstreamer-plugins-bad-free

Opus is a new audio codec covering a wide range of use cases. You can read more about the codec (at least) here and here.

If you want to get started with it, just try a freshly rebuild gstreamer-plugins-bad-free package which packages the opus plugin.
You will be able to use the en- and decoders in your normal gst pipeline.

It should soon land as an update on Fedora near you.

[Update]
The update has landed in updates testing:

su -c 'yum update --enablerepo=updates-testing gstreamer-plugins-bad-free-0.10.23-11.fc18'

gstreamer as a multimedia backend in Firefox

Wow. After such a long time it happened. Gstreamer can be used as a multimedia backend in Firefox!

So what does this mean? I believe not much yet for the enduser, at least this is a very nice infrastructure change to enable stuff like hardware acceleration e.g. on mobile devices. And from my point of view also a good separation: Do one thing and do it well. Let gstream do the multimedia stuff and Mozilla all that compositing.

Maybe this can be enabled in Fedora - at compile time - to get an maybe accelerated experience. or better codec support e.g. h264 for those who need it (e.g. by using Fluendos codec pack) or even my always favored Dirac!

gst-plugins-cl - OpenCL plugins for gstreamer, written in Vala.

Some of you may know OpenCL - the open computing language. This language allows us to write kernels in a C99 dialect, which can be run on data - in parallel.
A common use case for such a language - and processing concept - is the classical convolution - which was my motivation. Sure, this can also be realized using the frequency domain, but - hey - I feel fine in the spatial one.

Anyhow, there are bindings for OpenCL - which is vendor (NVidia, AMD, Intel, F/LOSS, ...) and hardware (GPU vs. CPU - AMD, Intel, IBM Cell, ...) independent for many languages, including MatLab, python, C, C++  and so on ...
In my case I needed a way to apply some filter function (or [OpenCL] kernel) to a video stream from some FireWire camera.
The experienced reader might directly associate FireWire with gstreamer our multimedia framework of choice under linux.
I had the option to Gather the data from a gstreamer pipeline reading the FireWire camera or create a gstreamer element providing, as of wrapping, the OpenCL functionality. The mentioned plugin does the latter, it provides a couple of elements to apply some OpenCL kernel to the data of a gstreamer pipeline.

You need vala and some gstreamer sources to build this project:

$ sudo yum install gstreamer-devel vala

.. might be a good start.
You also need some OpenCL implementation, as of today pocl can be used, but it's just intensively tested with the Nvidia implementation of it's GPU SDK and with Intel OCL SDK.
Building the plugin is quite simple

$ ./autogen.sh
$ ./configure
$ make

Testing should also be quite easy:

$ export GST_PLUGIN_PATH=src/.libs/
$ gst-launch audiotestsrc ! clkernel ! autoaudiosink

This doesn't do much, as the default kernel just passes the data - but at least you can test if the OpenCL is available. There are currently three plugins:

• clkernel - An element to manipulate any buffer
• clkernel2d - An element to manipulate a 2D buffer, like an gray image/video
• clvideofilter - An element to manipulate an RGBA image/video

The OpenCL kernel can reside in an external file. You can pass the filename of that file and the kernel to use to the element:

$ gst-launch audiotestsrc \
  ! clkernel kernel-file=mykernellib.cl kernel-name=somekernel \
  ! autoaudiosink

Have a look at the sources or kernel.cl and kernel2.cl files (they are plaintext files, the kernel are build on the fly - heard of JIT?) in the src directory to see what kind of signature the kernel has to provide. (This should be documented more thoroughly).
If you've got more than one platform which is OpenCL capable you can use the platform-idx property to specify a specific platform.

This plugins should help to offload some functionality to one or more devices (like GPU, CPU and accelerator, like the CELL) and you don't need to care about the details.
This can now be used to do things like color conversion, all sorts of filtering, mixing and what else can be done in parallel.
Thos plugins are just a start, it would be nice to be able to provide more sinks which then can be used as an input to the kernels.

~ https://gitorious.org/valastuff/gst-plugins-cl

Notes about writing a gstreamer plugin in vala.

Median/max/sobel filters/OpenCL kernels applied to 

Sintel using Intels OCL SDK and a gstreamer plugin.

Vala, as some of you may know, is a "compiler for the GObject type system". This means, vala is providing a high-level language with e.g. literals for (nearly) all features of gobject and glib. Because vala get's translated to C, it easily integrates with existing C code. Vala code can also be used from C (and other languages, most easily using gobject intropsection) too.

Gstreamer is a nice multimedia framework - and component of the GNOME ecosystem, now more related to freedesktop.org - and is also based on gobject/glib. So it is quite obvious to use vala to build gstreamer plugins and elements.

A gstreamer plugin is organized as a plugin, which wraps one or more elements, which provide the actual functionality. Detailed informations on how to write plugins can be found in the plugin writers guide.

But what is needed to write a plugin and elements in vala? And if it's good to do so.

gst_plugin_desc and plugin_init ()
There are two special structures that need to appear in a Gstreamer plugin: A plugin description (namely the Gst.PluginDesc [vala class]) and the plugin_init () function.
The identifiers of these structures (struct and function) need to match those expected by gstreamer. The plugin description identifier needs to be called gst_plugin_desc and the plugin_init() function also needs to have this name.
In vala it is important to place these methods on the toplevel of your file, and not within a namespace, otherwise vala will prefix the resulting C functions with prefix dervied from the namespace. An alternative is to use a CCode attribute for those structures, but why do this if it can be done easier (as described before).
An working example with correctly named plugin description and init function can be found here.

Namespace
Because of the naming you should also put your elements of the plugin within a Gst namespace e.g. namespace Gst.Myplugin. Because this translates to C structures (generated by vala) prefixed by gst_myplugin_… and the gst (or Gst or GST) prefix is needed, because the buildsystem is looking for those symbols and exports them.
Look in the example above or below to see where to put the element and what kind of namespace to use.

Elements: Gst.BaseTransform
It's quite easy to implement a simple filter. This can be done by subclassing Gst.BaseTransform and overriding (so providing an implementation for) a couple of functions. Have a look here to find out what you need to implement. Or here to see an actual working implementation.

gst-template and buildsystem
You can use the plugin template provided by gstreamer but you obviously need to modify the buildsystem to generate c code from the vala files.
Have a look at this Makefile.am.

Packages & Testing
The basic requirements can be met with the following packages:

$ sudo yum install gstreamer-devel \
gstreamer-plugins-base-devel gstreamer-plugins-base-devel-docs \
vala devhelp

Have you finally compiled your sources try the plugin with gst-launch, remember to add the hidden .libs dir to your path:

$GST_PLUGIN_PATH=src/.libs/ gst-launch  videotestsrc ! myelem ! autovideosink

So yes - you can write gstreamer plugins in vala - so what element are you going to write today?

presence is up for review.

To get presence into Fedora a so called "review" is needed - as you might know. If not it's a must to do one.
So if you needed something entertaining tonight, just grab a nice bottle of Beck's beer (or Bionade Holunder) and a box of Pringles (or Rosinen) and enjoy the few spec file lines.

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=754554

Presence and adjustable video quality.

Presence - a small vala, gstreamer, clutter and dirac/schroedinger - based bi-directional video tool, is configurable now.

You can actually use it to establish uni- or bi-directional audio/video streams in trusted/local networks. Sometimes this can be handy, as those streams can be set-up to have a much better quality than generic VoIP solutions ...
You can even receive more than one stream.

Currently you can adjust the video size, the compression quality and the framerate to get the optimal balance between performance and capabilities of the underlying hardware.
A next thing is to tune those parameters automagically - but I'll need to find out how to detect dropped frames or an increased latency. Additional there should be a way to adjust the latency/buffer on the receiving side, to match the amount of data coming in.

Find it on gitorious.

Presence now has »picture in picture (PiP)« mode.

The next branch of presence gained PiP support, this allows the parallel view of "secondary" streams besdes the primary one. Some refactoring needs to be done, before it lands in master.
Another thing on the todo list are more convenient dialogs. Maybe someone has some styling ideas?
If you wonder how I got Big Buck Bunny into presence, just continue reading.