ok i found and i'm a little confused, my error is too stupid.
I had not found the option about the amount of memory reserved for graphics card in the bios when i have setup my box and i guess this was not tunable. Now i found it and set it to 512M as recommended here (http://www.mythtv.org/wiki/VDPAU
), problem is solved!
Thanks to all for your help!