curious problem with glDrawArrays and glVertexPointer
 

We're having an odd problem with some of our older GL code and driver
275.28. I have on the heap an array of vertices that define a convex
polygon and am using that array like below:

glEnableClientState (GL_VERTEX_ARRAY);
glVertexPointer (3, GL_DOUBLE, 0, vertices);
glDisableClientState (GL_NORMAL_ARRAY);
glDisableClientState (GL_TEXTURE_COORD_ARRAY);
glDrawArrays (GL_LINE_STRIP, 0, num_vertices);
glDisableClientState (GL_VERTEX_ARRAY);

Similar snippets are called for various convex polygons, and when we
switched from driver 256 and 195 to 275.28 we started getting odd
segfaults in glDrawArrays, which I found through core dumps.

When I attached to the process with gdb, the process would segfault
immediately, and the segfaults made no sense. The place the segfault
occurred looked like a perfectly valid chunk o'memory. We found out
the the process didn't have permissions, though it was one our C++
objects. It seems that the debugger doesn't understand
copy-on-write. We could continue through the segfault, but the
process would soon segfault on glDrawArrays.

Next, we noticed that if the array of vertices is located on the stack
rather than the heap, neither the copy-on-write segfault nor the
segfault on glDrawArrays occurred. My next experiment showed that if
we mmaped an entire page of memory and put the vertices there, then
neither segfault happened. If the array of vertices isn't modified,
then it makes sense that no copy-on-write segfault would occur, but
that doesn't explain why the segfault on glDrawArrays doesn't happen.
I've tried to reproduce them with and without the debugger. It isn't
conclusive, but it could be that there is some correlation between the
two segfaults.

I haven't been able to reproduce this glDrawArrays segfault in a small
example, but I'm having trouble understanding why an array allocated
with 'new GLdouble [n*3]' would be different from an array allocated
with mmap. The size of our vertex arrays means that they would be
allocated with sbrk/brk rather than mmap. In the end, it seems that I
have a way of avoiding the segfault on glDrawArrays, but no real
explanation why. Anyone have any ideas?

A few extra points. Most of the machines this testing was performed
on were running ubuntu 10.04 and using quadro 5800 fx graphics cards.
Also, I did check that the vertices are valid.