You are here

Detecting whether selection is rectangular?

Are there any GIMP functions to detect whether a selection is purely rectangular (as opposed to round, rounded, feathered, non-contiguous, etc.)?

I mean something relatively simple, not involving iterating through every pixel in the selection bounds.

You could use the histogram function to get the pixel count of the selection and compare that to the width x height. (cadddr (gimp-histogram HISTOGRAM-VALUE 0 256)) will return the number of pixels in the current selection (or the whole layer, if no selection) and (gimp-selection-bounds ) returns a list if (non-empty x1 y1 x2 y2) so x2 - x1 times y2 - y1 would provide the rectangular area. I tested this from the scheme console and it appears to work: (let* ( (bounds (gimp-selection-bounds 1)) (area (cadddr (gimp-histogram 2 HISTOGRAM-VALUE 0 256))) ) (= (* (- (list-ref bounds 3) (list-ref bounds 1)) (- (list-ref bounds 4) (list-ref bounds 2))) area) ) -Rob A>

This one works on some conditions and not others, although I think it would be suitable for my purposes since (1) I don't expect to try this on a transparent layer (2) having a mask where none is needed wastes some bytes, but doesn't impact the visible results.
  • No selection:
    • Background: no mask (correct)
    • Non-transparent layer: no mask (correct)
    • Transparent layer: no mask (correct)
  • Rectangular selection:
    • Background: no mask (correct)
    • Non-transparent layer: no mask (correct)
    • Transparent layer: mask (incorrect but harmless)
  • Rectangular selection inverted:
    • Background: mask (correct)
    • Non-transparent layer: mask (correct)
    • Transparent layer: mask (correct)
  • Rectangular selection feathered:
    • Background: mask if feathered 1 pixel or more (correct enough)
    • Non-transparent layer: mask if feathered 1 pixel or more (correct enough)
    • Transparent layer: mask if feathered at all (correct)
  • Rectangular selection with pixel punched out:
    • Background: mask (correct)
    • Non-transparent layer: mask (correct)
    • Transparent layer: mask (correct)
  • Rectangular selection with internal pixel faded:
    • Background: mask (correct down to even 1% opacity for the faded pixel; sweet!)
    • Non-transparent layer: mask (correct down to even 1% opacity for the faded pixel; sweet!)
    • Transparent layer: mask (correct down to even 1% opacity for the faded pixel; sweet!)
  • Oval selection:
    • Background: mask (correct)
    • Non-transparent layer: mask (correct)
    • Transparent layer: mask (correct)
so I'll probably use this because the code is simpler, although it is helpful to know the other method as well.

The following sets a boolean variable 'rect?' if the selection is rectangular. It assumes a selection exists -- test the value of '(car (gimp-selection-bounds...' at the beginning if you need to ensure this. I left out the variable declarations. (set! channel (car (gimp-selection-save image))) (set! bounds (cdr (gimp-selection-bounds image))) (gimp-rect-select image (car bounds) (cadr bounds) (- (caddr bounds) (car bounds)) (- (cadddr bounds) (cadr bounds)) CHANNEL-OP-REPLACE FALSE 0) (gimp-channel-combine-masks (car (gimp-image-get-selection image)) channel CHANNEL-OP-SUBTRACT 0 0) (set! rect? (= (car (gimp-selection-bounds image)) 0)) (gimp-selection-load channel) (gimp-image-remove-channel image channel)

It is nice to see alternate ways to accomplish the same thing! (You sure love channels!) -Rob A>
Subscribe to Comments for "Detecting whether selection is rectangular?"