You are here

programming question non-rectangular shapes in gimp_pixel_rgn_get_rect background layer with alpha

This is a question about programming. I want to process each pixel of a layer. The layer contains a non-rectangular image (for example, something I lassoed, copied, and pasted into a new layer.) At least to the user, it looks non-rectangular. But it seems like when programming Gimp, every region is rectangular? When iterating over the pixels of a region, how do you tell a pixel is inside the non-rectangular shape?

It looks complex. For a background layer with an alpha channel, it seems as though you could check for pixels of (0,0,0,0) (black with a transparent alpha, which appears as a checkerboard to the user.) But for a background layer without an alpha channel, it seems as though you would need to check for a pixel of (255, 255, 255) (appears white, without an alpha channel) but that would exclude white pixels within the non-rectangular shape?

The context is that I am looking at the resynthesizer plugin. Suppose you rotate an image, leaving wedges of transparency at the corners, and select one wedge to resynthesize. The context is the invert of the selection and includes transparent wedges that you haven't resynthesized yet. I think the resynthesizer should exclude those other wedges from the context.

How can I get a downloaded plugin installed in Gimp for XP

Put it in your user directory if it's only for you.

For scripts (.scm) put them into the following directory
c:\Documents and Settings\'username'\.gimp-2.6\scripts

and for plugins (.py) the following directory
c:\Documents and Settings\username\.gimp-2.6\plug-ins

Restart Gimp. The plugins/scripts are now available to you.

Plug-ins indeed work with rectangular-shaped regions (as covered in Dave Neary's Plug-in Writing Tutorial) and the plug-in's code typically processes every pixel in that region. More precisely the plug-in will work on a copy of the original rectangular region and it is only after the region has been processed that the selection mask is taken into account -- the copy is composited back with the original region using the selection as a mask (using 'gimp_drawable_merge_shadow').

Conceptually this is identical to duplicating a layer, adding a layermask to the duplicate (initialized to the selection), filtering the entire duplicate layer, and then merging the duplicate layer down onto the original.

If I understand your description correctly, you wish the Resynthesizer plug-in to ignore a specified region (or regions) for context while processing a selected region. Given the way that Resynthesizer algorithm works, I would suggest selecting those "ignore" regions and actually resynthesizing them; and in this way exclude them from the context. When finished with the reconstruction, these resynthesized "ignore" regions should not be merged back to the original (while the resynthesized selected region would be).

There is still the problem of how to specify the "ignore regions", and unless they can be determined algorithmically then you will need to have the user specify an additional mask/channel (besides the selection). But this approach would probably be better than modifying the core resynthesizer algorithm to incorporate an "ignore mask" on a pixel-by-pixel basis (maybe not, though).

Thanks.

Just to clarify, suppose I lasso a round shape, copy and paste it into a new layer. Gimp creates an alpha channel then. Now Select>None. If the layer is passed to a plugin, and gimp_drawable_mask_intersect() is called on the drawable it returns False, since here 'mask intersect' refers to the selection mask (which there is only one for the whole image) and is distinct from what a user calls 'layer masks', which are separate drawables. There is no other mask that defines the round shape (as the user sees it), and a call to gimp_pixel_rgn_get_rect () returns data for every pixel in the rectangular bounds of the layer. The pixels outside the round shape will be (0,0,0,0) meaning black color and transparent alpha. If you delete the alpha, then the pixels outside the round shape will be (255,255,255) meaning white and no alpha. If you filter the layer and post it back to Gimp with gimp_drawable_merge_shadow, then a rectangle of pixels is posted, since there is no selection.

Yes the problem is how to make the Resynthesizer more user friendly. This problem with black pixels and alpha channels might be biting users and getting reported as a bug. For example, suppose you rotate an image and then select one of the transparent wedges with a crudely drawn rectangular selection. Whether you use Filter>Heal Selection or Map>Resynthesizer (with the option 'Use border'), the filter looks at a band just outside the selection as context (called corpus in the code) to make a match between the newly synthesized pixel and a source pixel somewhere else in the image. (That source somewhere else can also be in the band, e.g. if you are using Heal Selection.) In this case, since all the wedges touch each other, there is a thin sliver of black, transparent in the context for some target pixels. So it chooses a new source pixel that also has black in its context (neighborhood.) And the result is not what most would expect. They see a checkerboard, which means transparent, but the Resynthesizer sees it as black!

Even if you make a separate, rectangular layer of source texture, this problem exists, unless you select all the wedges at once, or as you say, create a separate layer mask (which means you must use Map>Resynthesize and can't use Heal Selection, which doesn't consider separate layer masks. Aren't most filters ignorant of separate layer masks? )

So yes, I am considering whether ignoring black, transparent in the context algorithmically is better than requiring the user to think about creating a separate layer mask.

The Resynthesizer is already creating its own mask (bitmap) of the the non-selected context pixels. And it only does this once at the start, it is not a bottleneck in the algorithm. So ignoring black, transparent would not hurt the speed.

I'm just confirming that there is no other API or algorithm for determining which pixels in a layer were actually pasted into the layer and which are black, transparent alpha created by Gimp to fill the rectangle.

(As a separate matter, the resynthesizer is calling gimp_drawable_mask_bounds to determine if the selection intersects the layer, which I think is a bug. That returns True if ANY selection exists, even if the selection does not intersect the layer.)

So yes, I am considering whether ignoring black, transparent in the context algorithmically is better than requiring the user to think about creating a separate layer mask.

If the algorithm doesn't already take into account the alpha of the context regions then I'd think doing so would be quite reasonable from an intuitiveness standpoint.

The Resynthesizer is already creating its own mask (bitmap) of the the non-selected context pixels. And it only does this once at the start, it is not a bottleneck in the algorithm. So ignoring black, transparent would not hurt the speed.

That is good. It would seem that merely intersecting the layer's alpha channel with Resynthesizer's "context mask" should be sufficient to accomplish what you wish (unless I am misunderstanding things).

(As a separate matter, the resynthesizer is calling gimp_drawable_mask_bounds to determine if the selection intersects the layer, which I think is a bug. That returns True if ANY selection exists, even if the selection does not intersect the layer.)

Quite true. 'gimp_drawable_mask_intersect' shoud be substituted for 'gimp_drawable_mask_bounds' per the instructions provided in the following Bugzilla comment:

https://bugzilla.gnome.org/show_bug.cgi?id=155733#c16

Subscribe to Comments for "programming question non-rectangular shapes in gimp_pixel_rgn_get_rect background layer with alpha"