First, making it fast is extremely difficult. JavaScript compilers are under extreme pressure to compile code quickly, unlike any other type-inferring compiler, because JS compilation keeps the user from seeing the page.
Second, in order to be useful, type inference for JavaScript has to be speculative. JS's semantics require that ints overflow into doubles, for example, meaning that a conservative type inference engine would have to assume every number can potentially be a double. This is too imprecise to be useful, so the inference engine speculates. If the engine guesses wrong, the compiler must not only recompile the function under the new assumptions but also (since this is a global analysis) potentially update every other function that the function called.
Finally, the presence of eval() and the like mean that almost no inference can be totally sound, requiring even more dynamic checks.
There's a reason that (despite what some others are saying in this thread) V8 and Chakra haven't done this yet: it takes a long time to get right.
> JavaScript compilers are under extreme pressure to compile code quickly, unlike any other type-inferring compiler, because JS compilation keeps the user from seeing the page
You only run type inference after you detect the hot loops. Same with other expensive optimizations. This is something that all JITs out there do -- you start with simple interpretation (or extremely cheap compilation, in the case of V8), and then when you detect that you're spending a whole lot of time in one section of code, you replace it with a highly optimized version.
Type inference is not that expensive compared to several more interesting optimizations done on several JITs that are in production systems. In some long running JITs, the compilation of some hot loops takes far longer than static compilations, running in a low priority thread in the background (while the less optimized code keeps going in the foreground).
Finally, if anything, JITs allow more time to run heavy optimizations because they can leave those running in the background without affecting the use of the program or hurting developer productivity.
> JS's semantics require that ints overflow into doubles, for example
On recent x86 processors, you would just use doubles in most cases. It turns out that they're on par with integers for speed, although I believe there were some differences on latency. I'd have to look back at the Intel opt manual to confirm, though. However, if you really want to solve the issue (say, on ARM where integer/float performance actually matters -- especially since lots of architectures are still on softfloat), then you'd use a combination of guards, versioning, and VRP to solve the issue, not type inference.
> Finally, the presence of eval() and the like mean that almost no inference can be totally sound, requiring even more dynamic checks.
Yep, eval is a terrible language feature if you care about making things fast. Regardless of whether you have type inference or not.
Actually, empirical data gathered in Tracemonkey shows that specializing to integers where possible is a performance win even on x86.
This is especially the case in the common situation of code computing array indices. Since those end up being treated as integers in the end, if you have them as doubles for the arithmetic step you have to keep paying double-to-integer conversion costs. The same considerations apply for some DOM methods that take an integer, and similar considerations apply to cases when numbers need to be converted to strings (e.g. when setting .style.top on an element): this is a lot cheaper to do for integers than for doubles.
> You only run type inference after you detect the hot loops.
That is the traditional approach, yes. But it will only get you so far. If you just analyze the current piece of code, you will miss out on a lot of potential optimizations.
For example, the types of the variables in the current function may be always of the same type, but without analyzing all the callers you wouldn't notice that - which means you would need to check the type each time you enter. With a more global analysis you could detect that once and be much faster.
There are a lot of other optimizations of that sort, like global variables - you can prove statically the type of many of them, just by looking in the whole program.
What is cool about this new Type Inference engine is that it is a hybrid approach. It does both local and global, and both static and dynamic analyses. You are very correct that local/dynamic type inference is a known thing, and JITs like TraceMonkey have done a form of it for a long time. But this new engine is something much better.
> If you just analyze the current piece of code, you will miss out on a lot of potential optimizations.
I didn't say that you only analyze the current piece of code. Type inference is, by nature, a global optimization. However, you don't do it until you've identified the hot spots, and you've established that it's worth investing time into it.
First, however, you get something quick running so that there are no delays, and the cost doesn't matter much to the user.
In any case, this is far from low-hanging fruit. It's very difficult to to properly, so it isn't surprising that this is the first mainstream JS engine to do it. It began as a research project - it wasn't clear it would ever end up succeeding at all.
On recent x86 processors, you would just use doubles in most cases. It turns out that they're on par with integers for speed, although I believe there were some differences on latency.
Integer operations have 2-3x the throughput and 4-5x better latency than floating-point operations on recent Intel CPUs.
According to the Intel optimization manual, there's a 5 cycle latency for doubles (4 for floats), but both fp multiply/add and alu ops have a one operation per cycle throughput.
I guess it comes down to how effectively you can schedule stuff.
On the other hand, most of the dynamic checks (type assertion barriers) will be there anyway as the JIT-generated code needs them (for type-specialized traces).
Static type inference may even allow for doing away with some, if it's possible to statically prove types are fully known at compile time for a code section.
Yeah the biggest problem is that there are millions of pre-existing codebases all over the web that mozilla needs to make sure don't break. Since it is a speculative engine, the possibility of breaking even a small percentage of those codebases becomes very likely, so it's really not an easy thing to do.
I'll be curious if it's good enough to enable by default, or if the developer needs to flag the engine somehow to enable the feature.
Since this type inference is speculative and dynamic, if the speculative type optimizations fail you can fall back to less optimized code, or interpreted JS, that handles the badly-typed code. Nothing will break.
First, making it fast is extremely difficult. JavaScript compilers are under extreme pressure to compile code quickly, unlike any other type-inferring compiler, because JS compilation keeps the user from seeing the page.
Second, in order to be useful, type inference for JavaScript has to be speculative. JS's semantics require that ints overflow into doubles, for example, meaning that a conservative type inference engine would have to assume every number can potentially be a double. This is too imprecise to be useful, so the inference engine speculates. If the engine guesses wrong, the compiler must not only recompile the function under the new assumptions but also (since this is a global analysis) potentially update every other function that the function called.
Finally, the presence of eval() and the like mean that almost no inference can be totally sound, requiring even more dynamic checks.
There's a reason that (despite what some others are saying in this thread) V8 and Chakra haven't done this yet: it takes a long time to get right.