Thursday, December 22, 2011

Added augeas.vapi and augeas.dep.

If you have ever dreamed of using augeas from vala, your dreams have been heard - just before christmas.
The binding is nearly complete, just aug_span is missing. So dive into xml using augeas, even in relative paths.

Wednesday, December 21, 2011

Da - Presence lands in fedora

Finally, presence - that small video tool - has landed in the fedora-updates repo.

You can install it using:
$ sudo yum install presence

That's all there is to do.

Thursday, December 15, 2011

De-Duping files on BTRFS.

Brave souls can test BTRFS for a couple of Fedora releases.

Removing duplicate/redundant files on filesystems is a common thing, e.g. when creating regular backups or so. On ext4 this can be realized using traditional hardlinks.
Hardlinks all point to the same blocks on the logical drive below. So if a write happens to one of the hardlinks, this also "appears" in all other hardlinks (which point to he same - modified - block).
This is no problem in a backup scenario, as you normally don't modify backuped files.

In my case I wanted to remove redundant files that might get modified and the changes shouldn't be reflected in all other copies. So what I want to achieve is to let several links (files) point to the same block for reading, but if a write happens to one block this should be just happen to the one file (link). So, copy the file on write. Wait, don't we know that as CoW? Yep.

Luckily BTRFS allows cow files using the cp --reflink command.
The following snippet replaces all copies of a file with "light weight" aka cow copies.

#!/bin/bash
# Usage: dedup.sh PATH_TO_HIER_WITH_MANY_EXPECTED_DUPES
mkdir sums
find $@ -type f -print0 | while read -d $'\0' -r F
do
  echo -n "$F : "
  FHASH=$(sha256sum "$F" | cut -d" " -f1);
  # If hashed, it's probably a dupe, compare bytewise 
  # and create a reflink (so cow)
  if [[ -f "sums/$FHASH" ]] && cmp -s "sums/$FHASH" "$F";
  then
    echo "Dup." ;
    rm "$F" ;
    cp --reflink "sums/$FHASH" "$F" ;

  # It's a new file, create a hash entry.
  else
    echo "New." ;
    cp --reflink "$F" "sums/$FHASH" ;
  fi
done
rm sums/*
rmdir sums

And in general, btrfs didn't yet eat my data, it even survived two power losses ...
Update: Updated to handle files with special characters. This script also makes some assumptions, e.g. the files should not be modified while running this script.

Friday, December 9, 2011

OpenCL on Fedora: A quick look at building pocl.

Finally a 3.0 release candidate of llvm is packaged for Fedora and available via rawhide. Many of the current OpenCL implementations like pocl and clover depend on llvm-3.0 and with this release it is finally an easy thing to build pocl on Fedora.
pocl has some other pre-requirements besides clang/llvm, after installing them you can just build the software as usual:

$ sudo yum install --enablerepo=rawhide llvm-devel llvm clang \
libtool-ltdl

$ git clone 
$ bzr branch lp:pocl
$ bash autogen.sh
$ ./configure
$ make
$ make check