Architecture
The Scala compiler has more than twenty phases. When compiling a program with the Scala compiler, each Scala source goes sequentially through these many phases until a classfile (or Javascript file if you are using Scala.js) is emitted.
Independently of the hardware architecture, the Scala compiler always uses only one core to compile. Considering the architectural shift that happened in the past ten years, when every laptop and desktop machine is equipped with at least two if not four or more physical cores, it’s a pity not to use the full potential of the hardware. This is exactly why we created Hydra, the world’s only parallel compiler for the Scala language. When compiling a Scala program with Hydra sources are intelligently partitioned into different batches and compiled in parallel. Hydra parallelizes all compiler phases.
It just works!
Hydra seamlessly integrates with your development environment. It is easy to install and it doesn’t require any change in your coding habits. Hydra meets you where you are, and more importantly it adds no additional complexity to your development environment or build chain. Today, the following integration points are available:
IntelliJ
sbt
Maven
Gradle
Scala.js
Eclipse (coming soon)
Command line
Achieve impressive speedups
The speedup Hydra can deliver depends on project and hardware specifics. Server-grade hardware is designed for parallel loads and speedups can go well beyond 6x. Modern laptops can achieve up to 3x speedups using 4 Hydra workers. But the good news is that with a powerful enough machine obtaining a 10x speedup is no longer a dream!| Project | Speedup |
|---|---|
| shapeless coreJVM/main | 1.78x 1.96x |
| shapeless coreJVM/test | 2.86x 3.38x |
| specs2 commonJS/main | 1.95x 1.98x |
| specs2 coreJS/main | 1.77x 1.82x |
| specs2 coreJS/test | 3.04x 2.41x |
| specs2 tests/test | 3.43x 2.82x |
How is Hydra different from the Scala compiler?
Hydra is based on the open-source Scala compiler with a focus on parallelizing the work inside the compiler and maintaining strong compatibility guarantees. Certain components are replaced or improved in order to make parallelization possible.
Being based on the open-source compiler means that Hydra supports the full Scala language, including macros and compiler plugins. Hydra is more than binary compatible with open-source Scala: it generates the same bytecode.
We keep Hydra in-sync with Scala and we release on the same schedule. New features and improvements in Scala are also available in Hydra in the corresponding release. From a language feature perspective, Hydra and the Scala compiler are 100% identical.
Compile time monitoring!
As a project grows it is normal for compilation times to grow with it. But how can one know if the longer compile time is due to the amount of code that needs to be compiled or if it’s a symptom of a deeper issue? This is a challenging question to answer because a compiler is usually a black box that emits class files or error messages. Investigating slow compile times is extremely difficult. That’s why we built-in metrics in Hydra that help you, the developer, detect and address such inefficiencies.
Hydra reports the time that each source file takes to compile and more metrics are going to be available over the next few months. Just by knowing how much time a file takes to compile it is now possible to detect pathological cases and go after the highest impact source first.
If the current tools are not enough to help you bring compilation times down, the good news is that we’re working on better ones! In the meantime don’t hesitate to contact us if you need help optimizing your projects compile times.
