You are here

pthreads (POSIX multithreading) on windows?

Hi, today I spend some time learning pthreads ( ~ multithreading) programming.

I would rewrite my plugins for this capability, but basic question is if such plugin can be compiled for windows. So this is question for peoples like samj, who tend to compile plugins for windows.

Specifically, It needs to include following into .c file:

#include <pthread.h>

#include <semaphore.h>

I read that these libraries are not common/supported/whatever on windows.

So, may I go on with this job? Thanks

Bonjour,

http://sourceware.org/pthreads-win32/

I can not compile plugins for a while.
I'll wait for the release of Gimp 2.8 for build new environments to compile on Windows.

I failed to find out when gimp 2.8 is due to be released.

In theory, it would be too much work to maintain two versions - threaded and non-threaded, so I will not do this, but will go on with threaded version. Anyway, I have no new features in mind to implement (talking about aumask)....

There are threads in glib, and your plug-iuns already depends on that library:

http://developer.gnome.org/glib/2.29/glib-Threads.html

You can also consider to implement GEGL ops and make your plug-in use them:

http://www.gegl.org/

And finally, the GIMP 2.8 release date:

http://tasktaste.com/projects/Enselic/gimp-2-8

This a list of tasks that have to be finished before GIMP 2.8 can be released. Obviously, instead of a fixed date, we get a moving target (i.e. if you publish the currently displayed date anywhere, if will already invalid tomorrow), but previous experience has shown that fixed dates aren't a useful approach.

fine, glib threading would do as well, I just looked at it and see there are no semaphores - this is a little complication, but...

as for gegl - with my programming knowledge about plugins and how plugins get and return data from and to drawable, I dont understand where would the gegl fit in this scheme. Simple tutorial would be nice - to understand basic concepts. But threading is my priority now...

tibor95: have you made any more progress? If you have any references or examples, I would appreciate it.

I have been trying to thread a plugin. It seems to work fine using POSIX pthreads. Then I switched to glib GThread. Now it crashes sometimes. I could be mistaken, they both might be flawed and only one manifests itself. (Once you start using threads, there is randomness i.e. chance i.e. luck involved.)

The documentation on gtheads seems sparse. What concerns me most is needing to call g_thread_init(NULL) early in the program. This link also points out the need to wrap calls to gdk in gdk_threads_enter();" and "gdk_threads_leave()".

http://forums.opensuse.org/english/other-forums/development/programming-...

I understand that glib itself is threadsafe once you call g_thread_init. I suppose you must tell gdk and gtk functions that threads are calling them, if the called functions keep internal state in global variables.

The plugin I am threading does mostly computation in the threads, with few calls to glib, gdk, or gtk. It does use many glib types. but I think those are mostly macros. My best guess at the moment is that the progress functions like gimp_progress_update(), which are called from the threads, are at fault.

Hi bootchk,
indeed documentation is sparse and also it is hard to find somebody who is competent enough and willing to share his knowledge.
I have currently two multithreading plugins (saturation equalizer and advanced unsharp mask), if you want to look into code, look at first one, this one is simplier. Though both sources might be ugly.

I went with glib threading and I believe it is rock solid. In my code I use only these related calls:

g_thread_init
g_thread_create (for each thread)
g_thread_join (to wait untill all thread exits)

So Im not able to advice in relation to "gdk_threads_enter();" and "gdk_threads_leave()".

I also run into problem (my fault) when threads kept overwriting the same variables, it is up to programmer to prevent it.

My technique is to split image (let say it has 1000 rows) into equal parts according to number of threads - 3 in my case.
So first thread processes rows 0-333, second: 334-667 and the third thread remaining part of image.

As for progres indicator, only the first one thread is allowed to upgrade it.

Feel free to ask more questions :)

Thanks,

An update on my plugin: I found that the problem indeed *SEEMS* to be the call to gimp_progress_update. My solution was to put a mutex on the call to it. So each thread can update progress. As you say, another solution is to have the threads communicate with the main thread and let it be the only thread that updates progress.

How exactly did you update progress? Do you just update progress as each thread completes, in your case, in thirds? Or did the threads update a mutexed global variable which the main thread spun on?

Your original post said you needed semaphores, but you latest response says you didn't? In my case, I used mutexes, because threads are reading pixels another thread is writing.

How did you decide how many threads to start? There doesn't seem to be any support in glib or elsewhere to determine how many threads can actually execute concurrently on the hardware. I am going to use 8. I don't think that hurts performance to have more threads than actual hardware cores or hyperthreads. My computer only has two.

Are you releasing your plugin in only the threaded version? I suppose if as you say glib is rock solid and it compiles on other platforms, there is no reason not to have just a single version. Could there be users out there wanting plugins running on older distros where glib possibly was NOT rock solid for threading?

What was your perfomance improvement? I found that with two cores, the plugin was at best 3/4 the time of unthreaded performance.

As for my original question, you must understand that I was just learning multithreding and did not have exact idea what I want. At the end I ended with very simple dizajn, where I dont need mutexes nor semaphores.

As I said I run only three threads and run them once. In some early version I run new thread for every single pixel (it was with posix threads) and overhead was huge.

By my opinion at least linux/unix kernels are able to cope with many more threads than cores and are able to distribute CPU time evenly. I have two computers – single core and dual core. I decided for three threads because it seemed good compromise. But I would not afraid of 8 threads neither but I belive there are few peoples who has more than 4 cores. Anyway, in source you can easily increase number of threads. (search for #define THREADS 3)

In theory you can set „1“ there, but of course it would not avoid multithreading but just run one thread only instead. I believe multithreading in glib is old thing so no need to care for peoples would have old version of glib...

As for updating progress, this is the code:

if (data->id ==0 && x % 10 == 0 && !data->preview)
gimp_progress_update ((gdouble) x / (gdouble) data->endline);

That say, if the id of current thread is 0 and number of current row(x) can be divided by 10 and it is not preview, progress bar is updated by:

curent line / number of lines in first section of image

first section is section to be processed by first thread.

I dont have exact number of CPU utilization on dualcore, but It would be very close to 2x100%.

Feel free to ask more, I am not sure I answered all your questions.

Intel quad cores with hyperthreading can have eight threads, and newer cpu's have six cores, and the trend is more cores.

Remember, main is a thread too. So if you create three threads, you actually have four threads. But probably the main thread is just waiting on the others, and on a quad core, main wouldn't be using its core fully.

The way you implemented progress, if the first thread completes first, progress bar says 100% but two thirds of the work is yet to be done. You can't guarantee that the threads will get equal CPU time. But usually they do, so probably it doesn't matter.

Many image algorithms are moving to the GPU, so threading could be a dead end. I think that is why one poster suggested GEGL. But that is a whole other topic. I think you would need to learn GL (OpenGL), not GEGL, to make a plugin using the GPU. I think GEGL uses the GPU via GL, but doesn't expose the GPU to plugins. I could be wrong.

Thanks again. You have given me an idea I need to use threads differently to get more speed up.

On my single core PC I did not noticed any discrepancies between progress bar (based on first thread) and actual (visual) completion of work. But mainly - this is just image processing application, so it just doesnt matter much. Even one second inaccuracy would not be very noticeable. Remember also that after threads completion plugin does also other work which might not be reflected in progress bar altogether..

I was looking also at GEGL but found it not usable (for me), first there is absolute lack of tutorials on beginners level and second I found that I have to hand-code my algorithms anyway...

As for GL/openGL - Im comletely unamiliar with this topic, but I can agree that it has some potential in future....

And that you for your interest, you are first person ever I could discuss this topic :)

Subscribe to Comments for "pthreads (POSIX multithreading) on windows?"