Piyush, I am the sole developer of my program. I chose to code it in Ruby because Ruby offers me the ability to extend and manipulate the language for my purposes. It was a good fit for porting subsets of programs written in other more obscure programming languages. I chose JRuby because it offers me the ability to incorporate existing Java libraries into my Ruby program. I am not going to be on-boarding new developers anytime soon. These reasons I have outlined here are enough to nullify your arguments.
I am well aware that the Ruby specifications are not written in stone. But AFAIK, there has yet been no change to any specifications indicating that Fixnum (exclusively and specifically!?) operator overloading is to be eliminated for any reason.
Both the 1.8.7 and the 1.9.2 versions of the existing reference implementation of Ruby clearly provide this ability for developers, "stupid idea" or not.
(Not to mention that every version of JRuby prior to 1.6 supports Fixnum operator overloading, and this speed increase that is of "utmost" importance to you has been around for all of 30 days.)
For myself, I will take the power and flexibility of the Ruby language over minute performance tweaks.