That part went fine, but whenever I tried to build my package that was supposed to use qt4-vlc, it would use the system libs in %{_libdir}/qt4...I tried to use rpath as a solution but couldn't get the syntax right. I looked at the gnu documentation but that didn't work because gcc kept complaining that it didn't know what the -rpath option meant.
The solution was to escape all the -rpath options that are for the linker (ld) and not gcc. Using -Wl, passes arguments to the linker and ignores them in the compiler. The final line I arrived at is:
LDFLAGS='-Wl,-rpath -Wl,%{_libdir}/qt4-vlc' $LDFLAGS
export LDFLAGS
I put this in the %build and the %install stages. In %build I put it before %configure and in %install it's before %makeinstall.
How to read this is that the -Wl, just says, pass the next argument to ld, so
-Wl,-rpath -Wl,%{_libdir}/qt4-vlc
is sent to the linker as
-rpath %{_libdir}/qt4-vlc
Once I figured that out, it was all good. But it seems a bit silly to have two of the -Wl, clauses, since you can just put in another comma. In the end I just used this:
-Wl,-rpath,%{_libdir}/qt4-vlc
It's not only more compact, it's easier to read..