After looking this over a bit, I'm not sure it will be easy or valuable enough to make the ir package removable.
I was able to make a simple change to let JRuby boot without the IR package present, by lazily allocating IRManager. Having done that, I removed everything under org.jruby.compiler.ir, annd the jruby.jar file only shrank by about 360k, from 10174515 to 9813649. That's nontrivial, but it's not a huge reduction either.
Unfortunately, making it so that the dx tool would not want to compile IR stuff is more complicated. IRStaticScope is referenced in several places, and it references topmost classes in the IR package. We could go through the effort of creating some utilities that could either produce normal runtime structures or IR runtime structures, but it's a bit of work.
Also, once the dx tool is finished compiling JRuby, it's likely to be less than 360k size reduction. If we use pack200 as an example, the normal jruby.jar compresses to 2268272, and the version without IR compresses to 2196262, a whopping 72k reduction.
I think a better direction would be for us to strip out libraries like unused YAML backend, JNR packages we don't need, ASM, and any Ruby extensions we don't need.