One year ago I posted on moving from Java to Scala [1]. At that time my experience with Scala was limited to reading some technical materials and to implementing a few simple mathematical algorithms such as weighed average. Even though I hadn't developed any project in Scala yet, I had a good feeling that moving from Java to Scala could be similarly right decision as moving from C/C++ to Java 10 years ago. Having only a good feeling usually is not enough to adopt a new programming language in a commercial project but it is certainly sufficient to use it for a research open source project. Because I was just about to start a new research project to develop artificial intelligence aided trading simulator [2], I decided to choose Scala programming language instead of a well known and commercially proven Java. Now, one year later, after writing 8622 lines of code in Scala [3], reading Programming in Scala book [4] and attending Martin Odersky's Scala course [5], I believe I'm in a good position to review my original statement on moving from Java to Scala. Was it a good decision or I made a mistake? I will try to answer this question in a context of research open source project as well as I will make an attempt to apply my observations to enterprise and commercial environment.
Evaluation methodology
Reading this Scala evaluation please put a bit of attention to what is it about and what it is not. For sure, I'm not trying to examine Scala just by looking only at its design, syntax and other language capabilities. Instead of that, I'm aiming to answer the questions raised in the introduction above. Was it a good decision or I made a mistake adopting Scala? Is Scala ready for Enterprise project or not yet? To answer those questions, in addition to evaluating Scala itself, a few other factors have to be considered:
- Programming language - Here, I evaluate features of Scala programming language, its paradigm, design and so on. In other words, all parts and aspects of Scala that were important for me developing trader simulator.
- Testing - How do we test Scala code?
- Performance - How fast Scala programs are?
- Tools - Doing any serious software development and not using some useful tools is hardly possible. The most worth to look at are: Integrated Development Environment, build tool, continuous integration, code coverage, static code analysis, memory and cpu profiler.
- Language extensions - Powerful and robust programming language doesn't guarantee any success yet. Third party tools, libraries and frameworks, that come from both open source community and commercial vendors, are very important as well.
- Interoperability - Interoperability with other programming languages by no means is a must requirement in developing some kinds of computer systems, especially with the most popular ones such as Java and C/C++.
- Monitoring and maintenance - In enterprise, assuring high availability at the level 99.99 and higher is a fundamental thing and requires wide range of tools for production monitoring and analysis.
- Support - Adapting new technology, particularly programming language, always involves some risk. Active world wide community, availability of trainings, forum, mailing list is very important. Additionally, well established global commercial company that stays behind programming language makes us to believe that we won't stay alone in the future.
- Scala skills - Even the most powerful weapon won't win the battle if you don't have a well skilled army that is able to use it.
- Time to Market - Eventually, whether or not it is worth to use a particular technology is a business decision that is based on some short and long term financial indicators. Though, we do software for money and having a fun may be just a nice side effect of that.
One more aspect that has to be taken into account for completeness of this evaluation is a business domain to which programming language is applied to. In this case it is risk management, data collection, market simulation and trading on betting exchange [6] markets.
There is not much point evaluating any programming language and not taking business domain into account. There are many programming languages in the world and some may fit one business domain whereas others master somewhere else. For instance Erlang fit well telecom, C is great for developing embedded systems, dynamic languages such as Ruby or Groovy are useful for developing web applications and Java fit well everywhere as a general purpose language. Scala has some better and worse appliances as well.
To evaluate Scala programming language in a context of all categories and business domain mentioned above, I will examine all of them one by one applying the following point system:
1 - poor, 2 - fair, 3 - good, 4 - very good, 5 - excellent
Finally I will make an attempt to interpret those scores and to come up with some relevant conclusions.
Evaluation
Over last year, developing trading simulator I was taking some notes on many aspects of adopting Scala programming language in a light of all evaluation categories presented above. I share with you all my thoughts and observation I made on Scala and I hope you will find them useful when taking your own decision on adopting Scala for some of your projects.
Programming language
The trading simulator, that I decided to use Scala for, can be characterized as a complex algorithmic system that each line of code matters rather than a CRUD (create, read, update, delete) application [7]. Writing most of the code requires a lot of thinking up front and looking at a written code should make it obvious what it does and how does it work. For many reasons Java doesn't fit here well, mostly due to following an imperative programming [8] paradigm and its verbose nature of a programming style. What I'm looking here for is a functional programming [9] and Scala is a place where I found it being simple and natural to use.
Before I carry on, please forgive me for presenting hardly any Scala code in this post. I do it on purpose, because my goal is not to show you how Scala syntax looks like and how to use it. For that, there are much better resources to go for, for instance great Programming in Scala book. In reverse, I prefer to focus on sharing with you my experience with Scala and then you can decide yourself whether or not it's worth to pursue it further.
Now, probably some people would question, why Scala and not other functional programming language such as Lisp [10] or Erlang [11]? Well, the main reason is that Scala code compiles to a byte code and runs on Java virtual machine, thus it is compatible with other Java applications. It is especially important if a majority of your code and libraries that you already use are coming from a Java world. The two libraries I was using myself for Trading Simulator project were Esper Complex Event Processing [12] and Jackson JSON parser [13]. I would probably stay with Java if I couldn't use them in Scala.
At this moment, others might ask why Scala and not let say Clojure [14] that is a functional programming language running on Java virtual machine as well? To answer this question we need to look at two features that distinguish both Scala and Clojure. First, Scala syntax is similar to Java, whereas Clojure is a Lisp dialect, that for me, a person who has been working mostly with Java for last 10 years, makes it far easier to use Scala than Clojure. Second, Scala is a statically typed language [15] and Clojure is a dynamic programming language [16]. Static typing is a core feature for me when developing critical and complex applications, First of all, debugging and maintaining statically typed code is easier and faster, secondly code written in a statically typed language is less error prone due to picking up some errors on a compile time and that is true regardless how good unit tests are.
There is one more important aspect of Scala that I'd like to mention to you. Scala is a multi-paradigm programming language, it means that it allows for mixing multiple programming styles such as object oriented, imperative and functional programming. Whether it's good or not, it is not a question that could be answered with a simple yes or no. Supporters would say that programmers can pick up the best of all styles depends on theirs needs. Others would argue that putting together some features found in many other programming languages can't work well, mostly by increasing complexity and making a programming language obscure.
If you ask myself on this topic, to some degree, I would agree with both positive and negative arguments above on multi-paradigm in Scala. On one hand, if Scala would be a pure functional language I wouldn't go for it. I like functional programming and I'm finding it very useful in many aspects of software development but I would lie if I said that imperative programming is a devil that we should all avoid. It's just not true and it's fair enough to look how popular Java nowadays is. One example of a computer system that imperative programming fits very well is a stateful high performance betting exchange engine that is a part of trading simulator I was developing over last year. Moreover, it fits there even better when mixed up with functional programming.
Regarding to a negative argument on a high complexity of Scala I have to admit that to a some level I agree with that. For instance please take a look at the one of the most commonly used method in a Scala version of a List class.
//Builds a new collection by applying a function to all elements of this list.
def map [B, That] (f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]) : That
Is it obvious to you what it is for? Would a novice Scala programmer understand it? Well, probably not easily, but definitely such syntax could discourage many of developers. To be fair, I have to mention that in Scala 2.8 this method in a Scala doc is presented following much simpler syntax, but in overall, it may not change a quite bad initial impression.
def map [B] (f: (A) ⇒ B) : List[B]
Additionally, some complexity in Scala is dictated by assuring compatibility with Java. For example in Scala there is an Option class that can take two values, Some and None. It's a great replacement for null value in Java and all would be fine if null value wasn't available in Scala too. When I asked Martin Odersky why is that, he explained me it's mainly to be able to stay compatible with Java. I think it's a relatively low cost of that.
There are many other examples that I could bring up to demonstrate how potentially evil Scala could be but it doesn't make too much sense. The Scala code that for someone is obscure, for others could be a nice and neat form of solving some specific problems. Personally, in most cases I found Scala syntax simple and natural to use and my opinion has not been changed after spending one year developing quite advanced application in Scala.
Next positive aspect of Scala is its conciseness. We all know how Java verbose is and how many boiler code developers write everyday including constructors, getters, variable initialisation (type and generics), semicolons and many others. To illustrate that I present how a Person class is defined in both Java and Scala:
Java version:
public class Person {
private final String firstName;
private final String lastName;
public Person(String firstName, String lastName) {
this.firstName=firstName;
this.secondName=secondName;
}
public String getFirstName() { return this.firstName; }
public String getLastName() { return this.lastName; }
//hashcode, equals and toString methods
}
Scala version:
case class Person(firstName:String, lastName: String)
If you think that I'm showing to you some marketing gadget similar to presentations on 'How to build a web based application in 5 minutes', I do assure you it is not the case. Over last year, I was discovering every month, more and more examples that were proving to me how useful concise syntax in Scala is.
There are a few other features in Scala that are worth some attention, I won't describe them in details but I encourage you to read about them yourself in a Programming in Scala book. Those features include, creating DSL (domain specific language) in Scala [17], pattern matching [18], tail recursion [19], type inference [20], implicit conversions, xml and json parsing and concurrent programming with Actors [21].
Until now, I wasn't too critical on Scala, does it mean that Scala is a perfect programming language? Unfortunately, every rose has its thorn and Scala has one as well. It's a poor backward compatibility in Scala. For example, I started developing trading simulator with Scala 2.7 and then I migrated to version 2.8. Even though I wrote no more than 3K lines of code I faced a few compilation errors and one failure of a functional unit test. One of areas widely affected by those changes in Scala is Collections API (see Scala 2.8 Release Notes [22]). Scala team and Martin Odersky himself are justifying all those significant changes by improving overall quality and design of Scala. That of course makes sense but sometimes I feel that this improvement process is happening to fast and some formal body similar to JCP (Java Community Process) [23] would help. That is especially important now, at the day when Scala is entering Enterprise world.
To summarise, how I'm describing Scala so far, it's a statically typed and multi-paradigm programming language that runs on Java virtual machine and provides Java like syntax with a few improvements. It sounds a bit familiar, doesn't it? Isn't it the old good Java with support of closures and some syntax improvements. If so, why I don't wait for some time until closures are supported in Java? Though I could live without closures for 10 years, so why I can't wait a bit more time? Actually this was one of the main questions I asked myself one year ago, before I started trading simulator. Then I asked the same question one year later to Martin Odersky and now I will try to answer this question to myself as well as to you dear readers in the end of this post.
My overall score to Scala programming language is very good (4).
Testing
Developers are provided with all they need to test Scala applications efficiently. To follow a Test Driven Development methodology [24], they can use popular in Java world Junit tool [25]. If someone is more keen on Behaviour Driven Development [26] then ScalaTest is a way to go [27]. I use both of them and they work very well. Especially, the second tool deserves some attention here, as it's a good example to illustrate one more powerful feature of Scala that I haven't presented to you yet. Please look at the following test written in a ScalaTest tool.
scenario("pop is invoked on a non-empty stack") {
given("a non-empty stack")
when("when pop is invoked on the stack")
then("the most recently pushed element should be returned")
and("the stack should have one less item than before")
}
What is interesting here that it's an absolutely correct Scala code. Its syntax is provided by a domain specific language, which is built on top of Scala. Can Java make it? probably not. To make it working in Java we would need to create a DSL on top of Groovy and then call it from Java, for instance Easyb testing tool [28] represents this approach. Though, it's a bit problematic as calling Java code from Groovy carries on some level of overhead and complexity due to integrating two different languages.
Until now, there are not too many development tools created specifically to serve Scala, but fortunately, thanks to compatibility with Java it doesn't look so bad.
For a purpose of a memory and CPU profiling I use Jprofiler [39] and Eclipse MAT (Memory Analyser Tool) [40]. Despite of being dedicated Java tools, they can be used for analysing Scala applications as well. Personally, I found some difficulties while using those tools with Scala, for example they won't let you to analyse parts of your application that use some Scala features that are not available in Java, for instance companion objects and closures. I wouldn't say that it will stop you from using those tools, just the user experience may not be so good as when analysing Java applications.
One family of tools that in my opinion at the moment are hardly usable in Scala are those, which do some sort of code analysis, for instance code coverage [41] and static code analysis [42]. Especially the code coverage tools are those, that I always use quite extensively. I tried to use Eclipse Emma plugin [43] to check ad-hoc a code coverage for a written Scala code and it just didn't work, blowing up with some errors. I have no evidence on other code coverage tools but even if they work somehow, the reports they generate may not be well tuned against Scala code.
Static code analysis tools are even less usable. I evaluated two popular Java tools, pmd [44] that operates on a source code level, and find bugs tool [45], which inspects Java byte code. Both of them report a big number of warnings and errors that are not relevant to a Scala code. That is not a surprise as those tools are optimized to run against Java code.
To summarise Scala evaluation, this is a scores matrix for all categories (1 - poor, 2 - fair, 3 - good, 4 - very good, 5 - excellent).
Scala already has beaten Java for developing my research projects and I have no temptation to move back. How about Scala in Enterprise? Here, I'm not so optimistic. Java is a really strong player and being just a better programming language may not be enough. What if Java finally provides functional programming and some better and more concise syntax? What if then Scala disappears from a market and we are in a half way through migrating our applications from Java to Scala? Will we move back to Java? Who will pay for that? Those are the questions that everyone must answer himself.
One advice I can give you myself is to not adapt Scala, just because someone says it's cool, so I want to be cool too. Something that works well in someone's business domain may not work for you. Be reasonable and try to get some experience with Scala first, for instance by getting involved in some open source projects.
Thank you for your attention and I hope you enjoyed it. I invite you to comment on this post and to share your thoughts with others. Daniel.
Over last year, developing trading simulator I was taking some notes on many aspects of adopting Scala programming language in a light of all evaluation categories presented above. I share with you all my thoughts and observation I made on Scala and I hope you will find them useful when taking your own decision on adopting Scala for some of your projects.
Programming language
The trading simulator, that I decided to use Scala for, can be characterized as a complex algorithmic system that each line of code matters rather than a CRUD (create, read, update, delete) application [7]. Writing most of the code requires a lot of thinking up front and looking at a written code should make it obvious what it does and how does it work. For many reasons Java doesn't fit here well, mostly due to following an imperative programming [8] paradigm and its verbose nature of a programming style. What I'm looking here for is a functional programming [9] and Scala is a place where I found it being simple and natural to use.
Before I carry on, please forgive me for presenting hardly any Scala code in this post. I do it on purpose, because my goal is not to show you how Scala syntax looks like and how to use it. For that, there are much better resources to go for, for instance great Programming in Scala book. In reverse, I prefer to focus on sharing with you my experience with Scala and then you can decide yourself whether or not it's worth to pursue it further.
Now, probably some people would question, why Scala and not other functional programming language such as Lisp [10] or Erlang [11]? Well, the main reason is that Scala code compiles to a byte code and runs on Java virtual machine, thus it is compatible with other Java applications. It is especially important if a majority of your code and libraries that you already use are coming from a Java world. The two libraries I was using myself for Trading Simulator project were Esper Complex Event Processing [12] and Jackson JSON parser [13]. I would probably stay with Java if I couldn't use them in Scala.
At this moment, others might ask why Scala and not let say Clojure [14] that is a functional programming language running on Java virtual machine as well? To answer this question we need to look at two features that distinguish both Scala and Clojure. First, Scala syntax is similar to Java, whereas Clojure is a Lisp dialect, that for me, a person who has been working mostly with Java for last 10 years, makes it far easier to use Scala than Clojure. Second, Scala is a statically typed language [15] and Clojure is a dynamic programming language [16]. Static typing is a core feature for me when developing critical and complex applications, First of all, debugging and maintaining statically typed code is easier and faster, secondly code written in a statically typed language is less error prone due to picking up some errors on a compile time and that is true regardless how good unit tests are.
There is one more important aspect of Scala that I'd like to mention to you. Scala is a multi-paradigm programming language, it means that it allows for mixing multiple programming styles such as object oriented, imperative and functional programming. Whether it's good or not, it is not a question that could be answered with a simple yes or no. Supporters would say that programmers can pick up the best of all styles depends on theirs needs. Others would argue that putting together some features found in many other programming languages can't work well, mostly by increasing complexity and making a programming language obscure.
If you ask myself on this topic, to some degree, I would agree with both positive and negative arguments above on multi-paradigm in Scala. On one hand, if Scala would be a pure functional language I wouldn't go for it. I like functional programming and I'm finding it very useful in many aspects of software development but I would lie if I said that imperative programming is a devil that we should all avoid. It's just not true and it's fair enough to look how popular Java nowadays is. One example of a computer system that imperative programming fits very well is a stateful high performance betting exchange engine that is a part of trading simulator I was developing over last year. Moreover, it fits there even better when mixed up with functional programming.
Regarding to a negative argument on a high complexity of Scala I have to admit that to a some level I agree with that. For instance please take a look at the one of the most commonly used method in a Scala version of a List class.
//Builds a new collection by applying a function to all elements of this list.
def map [B, That] (f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]) : That
Is it obvious to you what it is for? Would a novice Scala programmer understand it? Well, probably not easily, but definitely such syntax could discourage many of developers. To be fair, I have to mention that in Scala 2.8 this method in a Scala doc is presented following much simpler syntax, but in overall, it may not change a quite bad initial impression.
def map [B] (f: (A) ⇒ B) : List[B]
Additionally, some complexity in Scala is dictated by assuring compatibility with Java. For example in Scala there is an Option class that can take two values, Some and None. It's a great replacement for null value in Java and all would be fine if null value wasn't available in Scala too. When I asked Martin Odersky why is that, he explained me it's mainly to be able to stay compatible with Java. I think it's a relatively low cost of that.
There are many other examples that I could bring up to demonstrate how potentially evil Scala could be but it doesn't make too much sense. The Scala code that for someone is obscure, for others could be a nice and neat form of solving some specific problems. Personally, in most cases I found Scala syntax simple and natural to use and my opinion has not been changed after spending one year developing quite advanced application in Scala.
Next positive aspect of Scala is its conciseness. We all know how Java verbose is and how many boiler code developers write everyday including constructors, getters, variable initialisation (type and generics), semicolons and many others. To illustrate that I present how a Person class is defined in both Java and Scala:
Java version:
public class Person {
private final String firstName;
private final String lastName;
public Person(String firstName, String lastName) {
this.firstName=firstName;
this.secondName=secondName;
}
public String getFirstName() { return this.firstName; }
public String getLastName() { return this.lastName; }
//hashcode, equals and toString methods
}
Scala version:
case class Person(firstName:String, lastName: String)
If you think that I'm showing to you some marketing gadget similar to presentations on 'How to build a web based application in 5 minutes', I do assure you it is not the case. Over last year, I was discovering every month, more and more examples that were proving to me how useful concise syntax in Scala is.
There are a few other features in Scala that are worth some attention, I won't describe them in details but I encourage you to read about them yourself in a Programming in Scala book. Those features include, creating DSL (domain specific language) in Scala [17], pattern matching [18], tail recursion [19], type inference [20], implicit conversions, xml and json parsing and concurrent programming with Actors [21].
Until now, I wasn't too critical on Scala, does it mean that Scala is a perfect programming language? Unfortunately, every rose has its thorn and Scala has one as well. It's a poor backward compatibility in Scala. For example, I started developing trading simulator with Scala 2.7 and then I migrated to version 2.8. Even though I wrote no more than 3K lines of code I faced a few compilation errors and one failure of a functional unit test. One of areas widely affected by those changes in Scala is Collections API (see Scala 2.8 Release Notes [22]). Scala team and Martin Odersky himself are justifying all those significant changes by improving overall quality and design of Scala. That of course makes sense but sometimes I feel that this improvement process is happening to fast and some formal body similar to JCP (Java Community Process) [23] would help. That is especially important now, at the day when Scala is entering Enterprise world.
To summarise, how I'm describing Scala so far, it's a statically typed and multi-paradigm programming language that runs on Java virtual machine and provides Java like syntax with a few improvements. It sounds a bit familiar, doesn't it? Isn't it the old good Java with support of closures and some syntax improvements. If so, why I don't wait for some time until closures are supported in Java? Though I could live without closures for 10 years, so why I can't wait a bit more time? Actually this was one of the main questions I asked myself one year ago, before I started trading simulator. Then I asked the same question one year later to Martin Odersky and now I will try to answer this question to myself as well as to you dear readers in the end of this post.
My overall score to Scala programming language is very good (4).
Testing
Developers are provided with all they need to test Scala applications efficiently. To follow a Test Driven Development methodology [24], they can use popular in Java world Junit tool [25]. If someone is more keen on Behaviour Driven Development [26] then ScalaTest is a way to go [27]. I use both of them and they work very well. Especially, the second tool deserves some attention here, as it's a good example to illustrate one more powerful feature of Scala that I haven't presented to you yet. Please look at the following test written in a ScalaTest tool.
scenario("pop is invoked on a non-empty stack") {
given("a non-empty stack")
when("when pop is invoked on the stack")
then("the most recently pushed element should be returned")
and("the stack should have one less item than before")
}
What is interesting here that it's an absolutely correct Scala code. Its syntax is provided by a domain specific language, which is built on top of Scala. Can Java make it? probably not. To make it working in Java we would need to create a DSL on top of Groovy and then call it from Java, for instance Easyb testing tool [28] represents this approach. Though, it's a bit problematic as calling Java code from Groovy carries on some level of overhead and complexity due to integrating two different languages.
In my opinion, there are some business domains where ability to create domain specific languages in Scala is an extremely useful feature. The one example is a domain specific language for trading on financial markets.
My overall score to Scala testing is very good (4).
Performance
At first sight the performance in Scala is very good, comparable to Java, thus Scala code compiles to the same byte code as Java does and runs on the same Java virtual machine. Nevertheless, how fast Scala code is, may vary from case by case and usually it's up to the way how the code is written. This awareness how to write a high performance Scala code is especially important for Java developers who are not very experienced in both Scala and functional programming.
For instance poor knowledge of Collections API in Scala may lead developers to call often a size() method on a List class. Because this method is not an indexed sequence as it is in Java, therefore calling it is a very expensive operation as it has to traverse through all elements in a List to count all of them. The other example is to implement a function that multiplies all numbers in a List by 2 and then filter out all that are greater than 10. In Java, we would probably create a single loop that does this work but in Scala, following functional programming we could write:
For instance poor knowledge of Collections API in Scala may lead developers to call often a size() method on a List class. Because this method is not an indexed sequence as it is in Java, therefore calling it is a very expensive operation as it has to traverse through all elements in a List to count all of them. The other example is to implement a function that multiplies all numbers in a List by 2 and then filter out all that are greater than 10. In Java, we would probably create a single loop that does this work but in Scala, following functional programming we could write:
list.map(e => e*2).filter(e => e>10)
Looks simple and in many cases that's the solution we would use but under the hood it's much more expensive algorithm that the iterative version in Java (more details). On the other hand, having a choice between imperative and functional programming, availability of tail-recursion and so on, we can achieve better performance in Scala than in Java by carefully choosing proper algorithms for particular tasks.
There is one important thing for me that I noticed myself after one year I spent with Scala. In Java, usually development can be characterized as 'write first then think' approach, in Scala it's in an opposite way 'think first then write'. In Java, most of code is constructed by combining some loops and variables together that in many cases is a routine work. Whereas in Scala, developer is much more encouraged to think first and then write. From my own experience, when applying 'think first then write' approach, the final result of software development work can be impressive.
Next thing that I learnt myself by adopting the 'think first then write' approach and gaining some good skills in functional programming is that I became a better developer, whichever programming language I use, either Scala, Java or something else.
Going back to a performance aspect in Scala, I would like to mention one improvement that was added to Scala version 2.8. It's a specialisation for auto boxing [29], which regarding to Iulian Dragos [30], creator of this feature, provides a performance improvement of a factor of 4 to 30. Iulian gave an interesting talk on this topic on December 2010 at Skills Matter in London. The pod cast is available here:
My overall score to Scala performance is very good (4).
Tools
Until now, there are not too many development tools created specifically to serve Scala, but fortunately, thanks to compatibility with Java it doesn't look so bad.
For a build tool, I adopted maven [31], which with some help of maven-scala-plugin [32] works very well. Having that we use maven, there shouldn't be any problem to adopt some of popular Continuous Integration [33] tools such as Bamboo [34] or Ant Hill Pro [35]. Alternatively you could use SBT [36], simple build tool for Scala. You will find a pod cast on a Simple Build Tool here:
As Integrated Development Environment I utilised Eclipse Scala IDE plugin [37]. When I started using this plug-in more than one year ago it was very unstable and it was crashing my Eclipse a couple times a day. It was so frustrating to a level that I nearly abandoned Scala. By good luck I've been keeping up and sometime later, Scala IDE plug-in was getting better and better. At the moment there are still some features barely supported such as re-factoring or debugging, nonetheless at least it is stable enough to not complain too much. I heard some good opinions on IntelliJ Scala plugin [38] but because I haven't used it myself I'm not able to give you more information.
For a purpose of a memory and CPU profiling I use Jprofiler [39] and Eclipse MAT (Memory Analyser Tool) [40]. Despite of being dedicated Java tools, they can be used for analysing Scala applications as well. Personally, I found some difficulties while using those tools with Scala, for example they won't let you to analyse parts of your application that use some Scala features that are not available in Java, for instance companion objects and closures. I wouldn't say that it will stop you from using those tools, just the user experience may not be so good as when analysing Java applications.
One family of tools that in my opinion at the moment are hardly usable in Scala are those, which do some sort of code analysis, for instance code coverage [41] and static code analysis [42]. Especially the code coverage tools are those, that I always use quite extensively. I tried to use Eclipse Emma plugin [43] to check ad-hoc a code coverage for a written Scala code and it just didn't work, blowing up with some errors. I have no evidence on other code coverage tools but even if they work somehow, the reports they generate may not be well tuned against Scala code.
Static code analysis tools are even less usable. I evaluated two popular Java tools, pmd [44] that operates on a source code level, and find bugs tool [45], which inspects Java byte code. Both of them report a big number of warnings and errors that are not relevant to a Scala code. That is not a surprise as those tools are optimized to run against Java code.
My overall score to Scala tools is fair(2).
Language extensions
Most of libraries and frameworks are available to Scala due to compatibility with Java, otherwise we wouldn't have too much choice. The two worth to mention native Scala extensions are Lift web framework [46] and Akka actors platform [47]. I have no much experience with Lift web framework so far, so I will leave you alone with it. In terms of Akka, it's a very interesting project lead by Jonas Bonér [48]. In a few words, it's a platform for developing event driven, distributed, scalable and fault tolerant systems adopting Actors concurrency model.
The majority of libraries and frameworks from a Java world, should be possible to use quite smoothly, which among other things, is thanks to implicit conversions [49] in Scala. For example, in Java to convert between two different implementations of a List we would need to call explicitly some custom code that populates an output list by iterating through all elements of an input list. In Scala we also need to write a such code, but the difference is that it is called behind the scene. Please consider the Scala code example below.
//Initialises java.util.ArrayList object with three integers and converts it implicitly to a Scala List.
val newList:List = Arrays.toList(1,2,3)
It is like extending generic language rules for type conversions such as those between numeric types and it really makes it far easier to use external libraries in Scala.
My overall score to Scala language extensions is good (3).
Language extensions
Most of libraries and frameworks are available to Scala due to compatibility with Java, otherwise we wouldn't have too much choice. The two worth to mention native Scala extensions are Lift web framework [46] and Akka actors platform [47]. I have no much experience with Lift web framework so far, so I will leave you alone with it. In terms of Akka, it's a very interesting project lead by Jonas Bonér [48]. In a few words, it's a platform for developing event driven, distributed, scalable and fault tolerant systems adopting Actors concurrency model.
The majority of libraries and frameworks from a Java world, should be possible to use quite smoothly, which among other things, is thanks to implicit conversions [49] in Scala. For example, in Java to convert between two different implementations of a List we would need to call explicitly some custom code that populates an output list by iterating through all elements of an input list. In Scala we also need to write a such code, but the difference is that it is called behind the scene. Please consider the Scala code example below.
//Initialises java.util.ArrayList object with three integers and converts it implicitly to a Scala List.
val newList:List = Arrays.toList(1,2,3)
It is like extending generic language rules for type conversions such as those between numeric types and it really makes it far easier to use external libraries in Scala.
My overall score to Scala language extensions is good (3).
Interoperability
Interoperability between Scala and other programming languages, including Java/C/C++, is very good, mostly because of running on Java virtual machine. What Java may talk to, Scala can do as well. Taking into account support for implicit conversions described above, I would state that Scala is one of the leaders of interoperability among all programming languages.
My overall score to Scala interoperability is excellent (5).
Interoperability between Scala and other programming languages, including Java/C/C++, is very good, mostly because of running on Java virtual machine. What Java may talk to, Scala can do as well. Taking into account support for implicit conversions described above, I would state that Scala is one of the leaders of interoperability among all programming languages.
My overall score to Scala interoperability is excellent (5).
Monitoring and maintenance
For me, the main monitoring tool to analyse production Scala applications is JMX (Java Management Extensions) [50], and again, this is with no surprise thanks to compatibility with JAVA. This tool does its job well when we want to analyse some predefined statistics, but sometimes we need to investigate some aspects of a production application while it's running and JMX can't provide all required data.
To deal with such scenarios, Java provides Java Virtual Machine Tool Interface (JVM TI) [51] that allows for inspecting running Java applications but it can be used for Scala too. Imagine, there is a method inside running Scala application that calculates and returns some data and we want to be informed every time when this method is invoked and what is the value it returns. We can do such things with a btrace [52], command line dynamic monitoring tool from open source world. Eventually if we prefer more sophisticated and powerful platform for monitoring production applications we could choose Dynatrace [53] commercial product. The first one works well with Scala, though not as good as with Java, because it's not a dedicated Scala tool. I haven't tried the second one with Scala but I expect it to work as well, thus it uses the same JVM TI interface as btrace does.
My overall score to Scala monitoring and maintenance is good (3).
For me, the main monitoring tool to analyse production Scala applications is JMX (Java Management Extensions) [50], and again, this is with no surprise thanks to compatibility with JAVA. This tool does its job well when we want to analyse some predefined statistics, but sometimes we need to investigate some aspects of a production application while it's running and JMX can't provide all required data.
To deal with such scenarios, Java provides Java Virtual Machine Tool Interface (JVM TI) [51] that allows for inspecting running Java applications but it can be used for Scala too. Imagine, there is a method inside running Scala application that calculates and returns some data and we want to be informed every time when this method is invoked and what is the value it returns. We can do such things with a btrace [52], command line dynamic monitoring tool from open source world. Eventually if we prefer more sophisticated and powerful platform for monitoring production applications we could choose Dynatrace [53] commercial product. The first one works well with Scala, though not as good as with Java, because it's not a dedicated Scala tool. I haven't tried the second one with Scala but I expect it to work as well, thus it uses the same JVM TI interface as btrace does.
My overall score to Scala monitoring and maintenance is good (3).
Support
Scala community, including forum, mailing list and blogs, is not the biggest in the world, but it's very energetic. My feeling is that most people who are part of a Scala community are very passionate developers, which are always happy to help others to solve theirs problems. It was especially important for me on the beginning of my adventure with Scala.
Next important part of a Scala support are books and here I also found a great position. It's a Programming in Scala book by Marting Oderski, Lex Spoon and Bill Venners. There is big chance that I wouldn't pursue Scala if this book hadn't existed. I'm a such sort of person who usually start learning some new technology by reading some good book. If one doesn't exist, then probably this technology is not worth a penny (at least not yet).
One thing that would definitely help Scala to spread around the world is a global commercial vendor, such as Oracle or IBM. But even if no one ever exist it doesn't mean that Scala can't became a globally used programming language. Nobody should under estimate the power of an open source world wide community.
My overall score to Scala support is good (3).
Scala community, including forum, mailing list and blogs, is not the biggest in the world, but it's very energetic. My feeling is that most people who are part of a Scala community are very passionate developers, which are always happy to help others to solve theirs problems. It was especially important for me on the beginning of my adventure with Scala.
Next important part of a Scala support are books and here I also found a great position. It's a Programming in Scala book by Marting Oderski, Lex Spoon and Bill Venners. There is big chance that I wouldn't pursue Scala if this book hadn't existed. I'm a such sort of person who usually start learning some new technology by reading some good book. If one doesn't exist, then probably this technology is not worth a penny (at least not yet).
One thing that would definitely help Scala to spread around the world is a global commercial vendor, such as Oracle or IBM. But even if no one ever exist it doesn't mean that Scala can't became a globally used programming language. Nobody should under estimate the power of an open source world wide community.
My overall score to Scala support is good (3).
Scala skills
Even though I don't posses any professional data on a number of Scala developers in the world, I think no one will argue that it is far behind other programming languages such as Java or C/C++. I performed two simple experiments to get some data for you. First I checked for Scala job trends [54] and the number of job offers for Scala developers is barely noticeable but is growing dynamically over last two years. Second I searched for Scala developers on the linkedin.com social site and I got 1709 results. Just to mentioned that querying for Java developers returned 249,589 entries. It doesn't look good, does it?
One positive aspect I found is that looking for Scala developers we could search for Java developers with Scala interests as well. The learning curve is not massive as Scala follows Java syntax. In such case, at least on experienced Scala developer would help the team adopting new programming language.
My overall score to Scala skills is poor (1).
Summary
To summarise Scala evaluation, this is a scores matrix for all categories (1 - poor, 2 - fair, 3 - good, 4 - very good, 5 - excellent).
- Programming language - very good (4)
- Testing - very good (4)
- Performance - very good (4)
- Tools - fair (2)
- Language extensions - good (3)
- Interoperability - excellent (5)
- Monitoring and maintenance - good (3)
- Support - good (3)
- Scala skills - poor (1)
Scala already has beaten Java for developing my research projects and I have no temptation to move back. How about Scala in Enterprise? Here, I'm not so optimistic. Java is a really strong player and being just a better programming language may not be enough. What if Java finally provides functional programming and some better and more concise syntax? What if then Scala disappears from a market and we are in a half way through migrating our applications from Java to Scala? Will we move back to Java? Who will pay for that? Those are the questions that everyone must answer himself.
One advice I can give you myself is to not adapt Scala, just because someone says it's cool, so I want to be cool too. Something that works well in someone's business domain may not work for you. Be reasonable and try to get some experience with Scala first, for instance by getting involved in some open source projects.
Thank you for your attention and I hope you enjoyed it. I invite you to comment on this post and to share your thoughts with others. Daniel.
References
- Post on moving from Java to Scala - http://blog.danmachine.com/2009/12/time-to-change-moving-from-java-to.html
- Initial post on applying artificial intelligence to betting exchange research - http://blog.danmachine.com/2010/04/betting-ai-artificial-intelligence.html
- Betting AI project sources - http://code.google.com/p/betting-ai/source/browse/#svn%2Ftrunk
- Programming in Scala book by Marting Oderski, Lex Spoon and Bill Venners - http://www.goodreads.com/book/show/5680904-programming-in-scala
- Scala course by Marting Odersky and Julain Dragos - http://skillsmatter.com/course/scala/object-oriented-meets-functional-an-exploration-of-scala
- Betting exchange - http://en.wikipedia.org/wiki/Betting_exchange
- CRUD - http://en.wikipedia.org/wiki/Create,_read,_update_and_delete
- Imperative programming - http://en.wikipedia.org/wiki/Imperative_programming
- Functional programming - http://en.wikipedia.org/wiki/Functional_programming
- Lisp programming language - http://en.wikipedia.org/wiki/Lisp_(programming_language)
- Erlang programming language- http://en.wikipedia.org/wiki/Erlang_(programming_language)
- Esper Complex Event Processing tool - http://http://esper.codehaus.org/
- Jackson high-performance JSON parser - http://jackson.codehaus.org/
- Clojure programming language - http://clojure.org/
- Static typing - http://en.wikipedia.org/wiki/Static_typing#Static_typing
- Dynamic typing - http://en.wikipedia.org/wiki/Static_typing#Dynamic_typing
- DSL (Domain Specific Language) - http://en.wikipedia.org/wiki/Domain-specific_language
- Pattern matching - http://en.wikipedia.org/wiki/Pattern_matching
- Tail recursion - http://en.wikipedia.org/wiki/Tail_recursion
- Type inference - http://en.wikipedia.org/wiki/Type_inference
- Actor model - http://en.wikipedia.org/wiki/Actor_model
- Scala 2.8 Release Notes - http://www.scala-lang.org/node/7009
- JCP (Java Community Process) - http://jcp.org
- Test Driven Development - http://en.wikipedia.org/wiki/Test_driven_development
- Junit Test Driven Development tool - http://junit.org/
- BDD (Behavior Driven Development) - http://en.wikipedia.org/wiki/Behavior_driven_development
- ScalaTest BDD tool - http://www.scalatest.org/getting_started_with_feature_spec
- Easyb BDD tool - http://code.google.com/p/easyb/
- Auto boxing - http://en.wikipedia.org/wiki/Object_type_(object-oriented_programming)#Autoboxing
- Iulian Dragos - http://lamp.epfl.ch/~dragos/
- Maven build tool - http://en.wikipedia.org/wiki/Apache_Maven
- Maven-scala-plugin - http://scala-tools.org/mvnsites/maven-scala-plugin/
- Continuous Integration - http://en.wikipedia.org/wiki/Continuous_integration
- Bambo CI tool - http://www.atlassian.com/software/bamboo/
- Ant Hill Pro CI tool - http://www.anthillpro.com/html/default.html
- Simple Build Tool - http://code.google.com/p/simple-build-tool/
- Eclipse Scala IDE - http://www.scala-ide.org/
- IntelliJ Scala IDE - http://confluence.jetbrains.net/display/SCA/Getting+Started+with+IntelliJ+IDEA+Scala+Plugin
- Jprofiler tool - http://www.ej-technologies.com/products/jprofiler/overview.html
- Eclipse Memory Analyser Tool - http://www.eclipse.org/mat/
- Code coverage - http://en.wikipedia.org/wiki/Code_coverage
- Static code analysis - http://en.wikipedia.org/wiki/Static_code_analysis
- Eclipse Emma code coverage plugin - http://www.eclemma.org/
- PMD static code analysis tool - http://pmd.sourceforge.net/
- Find bugs static code analysis tool - http://findbugs.sourceforge.net/factSheet.html
- Lift web framework - http://liftweb.net/
- Akka actors platform - http://akka.io/
- Jonas Bonér home page - http://jonasboner.com/
- Type conversion - http://en.wikipedia.org/wiki/Type_conversion
- Java Management Extensions (JMX) - http://en.wikipedia.org/wiki/Java_Management_Extensions
- Java Virtual Machine Tool Interface (JVM TI) - http://en.wikipedia.org/wiki/Java_Virtual_Machine_Tools_Interface
- Btrace dynamic tracing tool - http://kenai.com/projects/btrace/pages/Home
- Dynatrace monitoring production applications tool - http://www.dynatrace.com/en/
- Job trends report on Scala jobs - http://www.indeed.com/jobtrends?q=scala&l=
- Time to market - http://en.wikipedia.org/wiki/Time_to_market
thank you for this detailed article
ReplyDeleteI chose Scala language as the main language to develop my systems. Very useful information :)
ReplyDeleteThanks Daniel.
I believe that Scala is not ready for business and in the Future Java will add best feature of Scala to itself.
ReplyDeleteJust use Haskell retard.
ReplyDeletePlease use hyperlinks for the bracketed references ([1] etc.) in your text.
ReplyDeleteHaving a list of citations is nice, especially for older people used to this "paper" style of referencing, but you really shouldn't force the reader to manually scroll to the bottom of the page instead of providing a link.
Thanks!
Thanks for spotting it. I just added hyperlinks, I hope it will help.
ReplyDeleteWhat about time to market?
ReplyDeleteExcellent review!
ReplyDeleteAlthough I'm a Clojure user mostly, it is always good to know what is happening in Scala world :)
I must agree with you: Scala (the same could be applied on Clojure) are probably not ready for the enterprise yet, as they are solving java problems with uncommon approach (functional way), making a quite challenge from developer to get used to that way of thinking.
As you said, at the end, java will probably pick the best features from both languages...
Hi Daniel,
ReplyDeleteAwesome article, thanks for collecting so valuable resources under one roof, hoping myself would go on the same way..
There was a time where there were no Java skills around...
ReplyDeleteHave no illusions, people - Java will *not* pick the best features of Scala. Vast amount of cruft accumulated in Java over the years makes that simply impossible without breaking most of software written in it. Martin Odersky had realised it once, and that was one of reasons to make Scala's syntax incompatibile.
ReplyDeleteJava will add new syntax, indeed, but with even more subtle rules, that you have to know in order to write bug-free code. Java's as a language is a dead end.
Instead of continuing to wade through this swamp, you're better of trying out Scala, sharing your experience with it and helping spreading the knowledge. While in some cases Java is still going to remain an optimal choice (for some time), Scala is constantly getting more attractive in new domains.
Hi Daniel,
ReplyDeletethanks for very good article. One of very few 'I moved' articles with neutral judgement. I moved about 10 months ago and could not agree more.
Also give shot to Idea, it works great with Scala
Very useful, thanks for sharing.
ReplyDeleteOne tiny comment : interoperability goes both ways. And while it's easy to see scala interoperability to other languages is very good, the reverse is (imho) much less.
ReplyDeleteCalling Scala from C, or even from Java is not at all easy.
Very good read. And thanks for introducing me to Scala over 1 year ago. :)
ReplyDeleteAlso, Scala is simply more fun to use than Java. Scala is like Ariel Atom, while Java is like Bugatti Veyron. ;)
I also switched to scala about a year ago. I simply cannot turn back anymore to Java. Consider although that I am working in a research company where quick prototyping, the ability to do both extremely flexible code and quick-and-dirty tools is a must. I find myself often programming in Scala to test an idea, just to reimplement it in C++ and assembly as we have real-time constraints.
ReplyDeleteI still use Java of course, but I don't think enterprise is such a big obstacle to jump. What Scala needs now are tools. Quality tools.
Hey,
ReplyDeleteA quick correction, it's Iulian Dragos[1], not Julian. There are two instances of the typo.
Best,
Ismael
[1] http://lamp.epfl.ch/~dragos/
Hi Ismael,
ReplyDeleteThank you for your spot. It is fixed now.
Regards.
Daniel
Excellently written article Daniel, with an unbiased evaluation on several criteria.
ReplyDeletehaving been a Java developer myself for the last 6 years, I decided to try Python(3) 4 months ago, last week I decided to try Scala simply cause I'm more a Java guy ... thanks for all this info, very useful
ReplyDeleteI am currently using Groovy/Grails. Despite the productiveness boost it delivers compared to Java, I just hate that it is not statically typed and that many errors are not detected by the IDE/compiler. I've looked at Scala and liked it very much. However, for it to be useful to me, I would also need a robust library to interact with the database. Lift seems to be very html-centric (my front ends are in Flex), with a steep learning curve. Querulous from Twitter looks good but it is for an older version of Scala. Hibernate seems to be difficult to use with Scala .... Any suggestions?
ReplyDeleteNice article. Agree with most of the points.
ReplyDelete@Eduardo, Lift can be great for doing REST-backed frontends as well. People on the ML have built e.g. apps with a Cappuccino frontend and Lift backend.
@Eduardo: Google "Squeryl" and "Circumflex ORM" -- both work.
ReplyDeleteThanks for the information
ReplyDeletehttp://extreme-java.blogspot.com
@Jeppe and @sorhed: thanks for the tip, I will check them out! And, of course, thanks to Daniel for posting this and hosting this discussion.
ReplyDeleteMany thanks for the follow up, very useful.
ReplyDeleteI would only say that if there was no Scala-plugin for IntelliJ, I would not be using Scala language. It is a must!
Also, JRebel has proven to be VERY useful, basically if you develop a web application in scala with maven then you can
export JREBELHOME=$HOME/java/lib/ZeroTurnaround/jrebel-3.5
then start project with
mvn jetty:run
and in separate window mvn scala:cc
and then development experience is close to RubyOnRails, or PHP - change code and immedialy test it - very very cool
To eduardo: Scala driven Wicket is awesome. Also look at my previous post for tips on making development very productive and dynamic
ReplyDelete@Eduardo And there's scalaquery too.
ReplyDelete@Daniel: thanks. I get the impression there are quite a few options. But it also seems that none (except Lift) is mainstream with a lot of followers and good docs. I think I will try to learn Lift. I've started learning Scala by reading David Pollak's book, which I found excellent. I am also very impressed at how he (who has rock-star status in the Scala world) actively promotes the language and the framework in web forums, even answering beginner's questions! I heard the rest of the community is also very helpful. I just hope there would be slightly better docs for the database persistence part of Lift. I've read there is Mapper and that the future is Record, but could not find any docs on the latter. The examples I found for Mapper seem to have too much boilerplate compared to Grails/Gorm.
ReplyDelete@daniel
ReplyDeleteI tried pollack´s book too, same conclusion: it´s really excellent!
I just finished my bachelor thesis about scala (german) last week. Again, almost the same conclusion as Daniel´s post.
Hi Daniel,
ReplyDeleteExcellent review, thanks for sharing your experience !
Note on code coverage tools : the young "scct" already does marvels (it *just worked* out of the box in a few seconds on my two big projects without any issue, I was just amazed).
http://mtkopone.github.com/scct/
Cheers
@Eduardo Datanucleus works reasonably well with scala.
ReplyDeleteExcellent review, just saw the link on scala-lang
ReplyDeleteI do recommend IntelliJ's scala plugin, its working really nice with maven and scalatest support
If you are planning to start a new open source project in Scala I would like to work on one. I have read the entire book Programming in Scala second edition cover to cover. I have started writing small Scala programs.
ReplyDeleteHere are my details :
<>
gmail id : prathamesh.mone@gmail.com
Hi Prathamesh,
ReplyDeleteAll enthusiastic developers are welcome to join my betting exchange research project. Please contact me by email: daniel [dot] korzekwa [at] gmail [dot] com to discuss it further.
Regards.
Daniel
Excellent review!
ReplyDeleteI am evaluating akka, this is very useful.
Hi Daniel!
ReplyDeleteBeing a java & ruby developer, I've been hearing a lot about scala in the recent past and have been contemplating to use it on more full time basis (Just learning the basics right now). I must say that due to your 2 posts here, my affinity towards scala has been reaffirmed.
I'll be going through the books/tools you mentioned over a period of time now and maybe post something about my experiences on my blog too.
First, someone up in the comments said that calling Scala from Java is not easy. Well I disagree. I do it in a small part of a Java project, where we have both Java > Scala and Scala > Java communication, and after configuring the maven scala plugin, it works like a charm.
ReplyDeleteNext, I mostly agree with you in terms of tools; this is probably the place where Scala still needs to improve the most. That being said, NetBeans (yes, you ignored it =p) has a very nice plugin - usually, refactoring works nicely, for example.
Finally, regarding enterprise adoption, the scene is not so bad. The scala-lang page has a nice list of Scala users in the enterprise: http://www.scala-lang.org/node/1658 - companies like Twitter, LinkedIn and Foursquare are included. This last one even seems to use a pure Scala solution, with the Lift framework.
I thought clojure was statically typed, too...?
ReplyDeleteInteresting to note that since publication of this article LinkedIn ratio of Scala/Java has gone from .6% to 2%, Java climbing 3x and Scala 10x.
ReplyDeleteStraw analysis
Java: Demand flat, supply increasing
Mature/Stagnant Market => Hold/Sell
Scala: Demand and supply rapidly increasing
Growth Market => Buy
your article's anniversary is coming up soon;
ReplyDeleteupdate is due? :-)
I think there is not much to update on, which would make me desired to write next 'one year later' post on Scala. I wish I could write, 'yes, one year later, Scala programming language finally entered Enterprise market'. I think next time I post on Scala is when either it vanishes from the market or it achieves amazing success...in three years, maybe more.
ReplyDeleteGreat post man, before I move to Scala I was looking for experience someone has in Scala and you highlighted quite a few things. Thanks
ReplyDeleteWith Scala, you'll think like a mathematician or a player planning a strategy. Even if Java 8 (seen Java 8 lambda) or 9 are out, I find Java are still largely limited on imperative language, it still doesn't remove the boilerplate code altogether and will likely take sometime for Java 8 to adopt in the enterprise.
ReplyDeleteAs far as I can see JDK7 + Scala 2.10 is the best contender.