PImage.mask() broken

[hkiel]

Most appropriate sub-area of Processing 4?

Image, Core/Environment/Rendering

Processing version

4.4.3

Operating system

macOS

Steps to reproduce this

Run the following snippet.

Using mask() to mask an image leads to mask() can only be used with an image that's the same size. though both images are created with same size. Worked until 4.4.2.

snippet

PImage img;

void setup() {
  size(150, 150);
  setupHexagon(50);
  hexagon(75, 75, 50);
}

void draw() {
}

private void setupHexagon(int radius) {
  noiseSeed(1004);
  img = createImage(radius*2, radius*2, RGB);
  for (int px = 0; px < img.width; px++) {
    for (int py = 0; py < img.height; py++) {
      float n = noise(px/(float)img.width, py/(float)img.height);
      img.pixels[px+py*img.width] = color(n*255);
    }
  }
  img.updatePixels();

  PShape s = createShape();
  s.beginShape();
  s.fill(255);
  for (int i=0; i<6; i++)
    s.vertex(cos(PI/3*i) * radius, sin(PI/3*i) * radius);
  s.endShape(CLOSE);
  PGraphics maskImage;
  maskImage = createGraphics(img.width, img.height);
  maskImage.beginDraw();
  maskImage.shape(s, img.width/2., img.height/2.);
  maskImage.endDraw();
  img.mask(maskImage);
}

private void hexagon(float x, float y, float radius) {
  imageMode(CENTER);
  image(img, x, y, 2*radius, 2*radius);
}

Additional context

Snippet draws a hexagon with noisy pattern.

Would you like to work on the issue?

assign to someone else, please

[Stefterv]

Hey @hkiel Thank you for reporting this, this is a result of us switching to the pixelDensity = displayDensity by default.
You can revert back to previous behaviour by calling pixelDensity(1) in your setup.

As for what happened here, turns out that createImage does not create an image based on the current pixel density and createGraphics does, leading two different buffers with different sizes. I'll look into it later to see if either would need to be adjusted.

[Stefterv]

I think a great start is to add an error here if the pixelDensities of the images do not match

processing4/core/src/processing/core/PImage.java

Lines 841 to 844 in c83f44c

	public void mask(PImage img) {
	img.loadPixels();
	mask(img.pixels);
	}

Would you have been able to resolve the error if you would have been warned that the pixeldensity between img and maskImage did not match @hkiel ?

[hkiel]

I don't really get why pixel densities matter here. I'm working on two images of exactly the same size in pixels. Why would the density matter?
It would help if there was a createGraphics(img) method that clones the image properties (width, height, density, etc.?)

[SableRaf]

Hi @hkiel,

One thing to know is that pixelDensity controls how many physical pixels are used for each logical pixel in your sketch.
For example, at a pixel density of 2, what looks like a single pixel in your sketch is actually made up of 4 smaller pixels (2x2) behind the scenes. This helps sketches look sharper on high-resolution displays.

The issue comes from the fact that createGraphics() automatically adjusts for the pixel density of the sketch, but createImage() does not. That’s why even if the width and height match in your code, the underlying pixel arrays can end up being different sizes. Since mask() directly compares the pixel data, this mismatch becomes a problem.

This discrepancy wasn't as apparent when pixelDensity(1) was the default, but now that we use the screen’s density by default (starting with Processing 4.4.3), this issue shows up for high-DPI screens.

Here's a sketch that demonstrates this:

void setup() {
  size(100, 100);
  pixelDensity(1); // comment this line out (or set to 2) to see the difference

  PImage img = createImage(50, 50, RGB);

  PGraphics pg = createGraphics(50, 50);
  pg.beginDraw();
  pg.endDraw();

  println("img pixels:", img.pixels.length);
  println("pg pixels:", pg.pixels.length);
}

Output for pixelDensity(1)

img pixels: 2500
pg pixels: 2500

Output for pixelDensity(2)

img pixels: 2500
pg pixels: 10000

[hkiel]

What is the 'correct' way of creating an apropriate mask in my example? I would let pixelDensity be at default value.

[SableRaf]

To create an appropriate mask while avoiding the pixelDensity mismatch, you can use createGraphics() for both your drawing and mask buffers, then call .get() to extract a PImage since that is what the mask() function expects.

Here’s a code example:

PImage drawImage;
PImage maskImage;

PGraphics drawBuffer; 
PGraphics maskBuffer;

void setup() {
  size(200, 200);
  
  // Create two PGraphics of the same dimensions
  drawBuffer = createGraphics(100, 100);
  maskBuffer = createGraphics(100, 100);
}

void draw() {
  background(200);
  
  drawBuffer.beginDraw();
  drawBuffer.background(255, 0, 0); // Red
  drawBuffer.circle(mouseX, mouseY, 20);
  drawBuffer.endDraw();

  // Get the draw image
  drawImage = drawBuffer.get();

  // Note: if you don't need to animate the mask 
  // it will be more efficient to do this in setup() (including the .get call)
  maskBuffer.beginDraw();
  maskBuffer.background(0);
  maskBuffer.fill(255);
  maskBuffer.circle(50, 50, map(sin(frameCount*0.01),-1,1,40,80));
  maskBuffer.endDraw();

  // Get the mask image
  maskImage = maskBuffer.get();

  drawImage.mask(maskImage);

  image(drawImage, 0, 0);
}

[hkiel]

So, finally only the error message of mask() is misleading, since .width and .height attributes of image and mask are the same, but pixelDensity is different for the two objects, resulting it buffers of different size, which is the problem. Maybe you can add an assertion for the densities in mask().

@Stefterv I would not have had a clue how to fix the problem, if I had been warned about different densities, but at least I could have seen that the problem is not the size.

[SableRaf]

@Stefterv @hkiel: What would you think of replacing the existing error with this?

mask() error: image and mask must have the same dimensions and pixel density.

[Stefterv]

@SableRaf We should not replace the existing error in mask(pixelArray), as in that spot it only has the array length to check, we should add a new error in the mask(PImage) function.

[SableRaf]

@Stefterv Even better! What do you think would be a good error message?

[Stefterv]

@SableRaf What you wrote seems correct, maybe it can be a bit more specific

image and mask must have the same dimensions (320px by 240px) vs (300 by 200px), mask should be 320px by 240px

and in case of different pixel densities

image and mask must have the same pixel density, image is pixelDensity(1) and mask is pixelDensity(2). mask should also be pixelDensity(1)

Dynamically based on the current situation

[hkiel]

I would suggest to not repeat the values. You never know which one is really wrong.
image and mask must have the same dimensions, image is 320x240, mask is 300x200.
image and mask must have the same pixel density, image is pixelDensity(1) and mask is pixelDensity(2).