Could establish an interface for those extensions to participate in the JIT and ICs. JSC does that for the DOM.
Other than that just try to make calls into JITed code as fast as possible. But that needs to be the backup plan, with the main plan being that perf-sensitive extensions play with the JIT.
But we have no control over these C extensions. And there's a huge corpus of existing C extensions that need to keep working. 500 million lines of C and C++, in fact, even for just the publicly available code.
And by the way - there isn't really any proper interface at all! There's just basically the whole internals of the C Ruby implementation exposed to C extension authors. There's no abstraction with handles and things like that.
There was an effort at one point to get C extension authors to use the FFI instead, and a JNI-style API to permit a moving garbage collector was proposed, but these approaches weren't successful in gaining any momentum.
All in all... that's why we don't just do it the JavaScriptCore way. We have different constraints to yours.
You don’t need to make all native extensions fast. You need to make some of the most common things fast and keep the rest of them working. It’s fine if not all extensions adopt a JIT interface.
The thing about moving GC is a red herring. Ruby wouldn’t benefit from it. JSC doesn’t use moving GC.
Other than that just try to make calls into JITed code as fast as possible. But that needs to be the backup plan, with the main plan being that perf-sensitive extensions play with the JIT.