Few comments to my previous post. I was thinking about how is it possible that cartoonizer if working over layer with previously reduced colors, is sometimes not able to pick right colors. (and all of them)
It seems that my algorithm is good for "normal" images where RGB values are distibuted continuously. Then method of small iterations works well. But if input colors are discrete my method may fail.
So in regard to my workaround I adviced to increase color weight, but it do not guarantee success. But the effect of smoothing sharp angless can be achieved anyway.