Last year, I found a curious bug in Windows regarding the handlingof certain invalid opcode sequences. At the time, I simply documentedit and then forgot about it. Recently, however, I was reminded of thebug, so I thought that other people might be interested in readingabout it.
Because of the way in which the Intel x86 architecture works, whenan invalid opcode exception occurs, there is no easy way to tell why itoccurred. By this, I mean that without actually looking at the faultingopcode sequence, it's not possible to tell the difference between anunsupported opcode and an invalid use of the LOCK prefix. For thisreason, Windows runs this code:
mov ecx, 4 ;maximum prefix count
look_op:
mov al, byte ptr es:[esi] ;points to faulting opcode sequence
cmp al, f0h ;looks like LOCK?
je op_lock ;yes
add esi, 1 ;no, continue with next byte
loop look_op...