The Instruction Set Virtualization Toolkit (formerly CAVA) is an opensource toolchain for designing instruction sets. It includes tools that build a gcc-based compiler and assembler and an instruction emulator based on a textual instruction set definition.
Go start a discussion on this project's forums