This will rotate about a rational structure direction [uvw], regardless of the current Model mode or orientation of the structure. If you are drawing a molecule and have used Cartesian reference axes, the rotation direction is simply a vector with the coefficients u, v and w in the original molecule axis directions a, b and c. The indices or coefficients refer to a coordinate system fixed with respect to the structure (always rotating with it). The indices must be integers, but this is not a bar in practice to rotation about irrational vectors, since the indices may take very large values.
Each time you click the Rotate button, a rotation is added to a master rotation matrix; the number of such rotations is listed at the bottom of the dialog box. When you have finished rotating, click the Done button and the master rotation matrix is applied to the structure. If you click Cancel, no rotations are applied.