[gl2ps] Error when drawing triangle perpendicular to viewport and GL2PS_POLYGON_OFFSET_FILL enable.

[Jean-Baptiste Silvy]

Hello,

I'm using GL2PS version 1.3.2 and I used to experienced blank outputs
when enabling GL2PS_POLYGON_OFFSET_FILL.
Actually, the issue was occurring when displaying triangles
perpendicular to viewport.

Here are the faulty lines in gl2ps.c:
-----------------
area =
        (prim->verts[1].xyz[0] - prim->verts[0].xyz[0]) *
        (prim->verts[2].xyz[1] - prim->verts[1].xyz[1]) -
        (prim->verts[2].xyz[0] - prim->verts[1].xyz[0]) *
        (prim->verts[1].xyz[1] - prim->verts[0].xyz[1]);
      dZdX =
        ((prim->verts[2].xyz[1] - prim->verts[1].xyz[1]) *
         (prim->verts[1].xyz[2] - prim->verts[0].xyz[2]) -
         (prim->verts[1].xyz[1] - prim->verts[0].xyz[1]) *
         (prim->verts[2].xyz[2] - prim->verts[1].xyz[2])) / area;
      dZdY =
        ((prim->verts[1].xyz[0] - prim->verts[0].xyz[0]) *
         (prim->verts[2].xyz[2] - prim->verts[1].xyz[2]) -
         (prim->verts[2].xyz[0] - prim->verts[1].xyz[0]) *
         (prim->verts[1].xyz[2] - prim->verts[0].xyz[2])) / area;
-----------------
When a triangle is perpendicular to the viewport, area equals 0.0 and
divisions don't works.

I patched these lines using:
-----------------
/* test added here by scilab team to avoid division by 0 */
if (area > GL2PS_EPSILON)
      {
          dZdX =
            ((prim->verts[2].xyz[1] - prim->verts[1].xyz[1]) *
             (prim->verts[1].xyz[2] - prim->verts[0].xyz[2]) -
             (prim->verts[1].xyz[1] - prim->verts[0].xyz[1]) *
             (prim->verts[2].xyz[2] - prim->verts[1].xyz[2])) / area;
          dZdY =
            ((prim->verts[1].xyz[0] - prim->verts[0].xyz[0]) *
             (prim->verts[2].xyz[2] - prim->verts[1].xyz[2]) -
             (prim->verts[2].xyz[0] - prim->verts[1].xyz[0]) *
             (prim->verts[1].xyz[2] - prim->verts[0].xyz[2])) / area;
          maxdZ = (float)sqrt(dZdX * dZdX + dZdY * dZdY);
          dZ = factor * maxdZ + units;
          prim->verts[0].xyz[2] += dZ;
          prim->verts[1].xyz[2] += dZ;
          prim->verts[2].xyz[2] += dZ;
      }
-----------------

It works well in my case, but it might be a bit rough. Maybe we could
set dZ to units if area is null.

I also tried to use to use GL2PS_SIMPLE_LINE_OFFSET. However,
GL2PS_ZOFFSET_LARGE was a bit too large for my application (Scilab), it
was better when setting its value to 1.0 instead of 20.0. Does anyone
had similar problems?

Jean-Baptiste Silvy