When XviD is modulating quantizers, is it using any sort of method for deciding which quantizer to use? or is it just going to H.263 if the quantizer is above 3?

Exactly. No fancy stuff.

Do you think it's possible that there's a way to decide which quantizer to use depending on the complexity of the scene which is determined by examining the luminance? meaning a scene with high complexity would get H.263 and a scene with low complexity would get MPEG or vice-versa (I would prefer H.263 on complex scenes cause it gets rid of edge compression artifacts, and in low complexity scenes it the MPEG quantizer is able to be sharp without introducing edge artifacts)

2nd_pass_frame_quant = 1st_pass_frame_quant * (1st_pass_frame_bitrate / 2nd_pass_frame_needed_bitrate) I think to have red something about this in :rolleyes: Thought the quantizer effect (size cut) over bitrate isn't totally predictable nor really linear ...