This is a change in behaviour compared to jruby-engine: public void testErrorBehaviourForUnknownSymbol() throws Exception { ScriptEngineManager manager = new ScriptEngineManager(); ScriptEngine engine = manager.getEngineByExtension("rb"); try { engine.eval("no_method_with_this_name"); fail("Expected ScriptException"); } catch (ScriptException e) { RaiseException cause = (RaiseException) e.getCause(); cause.toString(); StringWriter writer = new StringWriter(); cause.printStackTrace(new PrintWriter(writer)); assertTrue("Not seeing the expected string at the start", writer.toString().startsWith("undefined local variable or method")); } } This test now fails because there is an additional RuntimeException between the ScriptException and RaiseException. As RaiseException is already a RuntimeException it's hard to see why the extra level might have been added.