mask is a pointer to a $32 ~times~ 32$ stipple pattern that is stored in memory just like the pixel data supplied to a glDrawPixels call with height and width both equal to 32, a pixel of GL_COLOR_INDEX, and data type of GL_BITMAP. That is, the stipple pattern is represented as a $32 ~times~ 32$ array of 1-bit color indices packed in unsigned bytes. glPixelStore parameters like GL_UNPACK_SWAP_BYTES and GL_UNPACK_LSB_FIRST affect the assembling of the bits into a stipple pattern. Pixel transfer operations (shift, offset, pixel map) are not applied to the stipple image, however.
To enable and disable polygon stippling, call glEnable and glDisable with argument GL_POLYGON_STIPPLE. Polygon stippling is initially disabled. If it's enabled, a rasterized polygon fragment with window coordinates $x sub w$ and $y sub w$ is sent to the next stage of the GL if and only if the ($x sub w~roman mod~32$)th bit in the ($y sub w~roman mod~32$)th row of the stipple pattern is 1 (one). When polygon stippling is disabled, it is as if the stipple pattern consists of all 1's.