The core algorithms of OpenCV will compile pretty much without any problems for Windows CE. Naturally, GUI related stuff like the HighGUI library, should be removed, since WindowsCE uses different GUI APIs.
Most of the platforms (i.e. CPU architecture) that run Windows CE (and other embedded OSs, for that matter) do not support floating-point operations. When you compile OpenCV for these platforms, the compiler uses a software library to emulate floating-point operations, making these operations very slow.
If you must use these operations and performance is critical rewrite the offending functions using fixed point math.