Saturday, 22 January 2011

Moving from Java to Scala - One year later...

One year ago I posted on moving  from Java to Scala [1]. At that time my experience with Scala was limited to reading some technical materials and to implementing a few simple mathematical algorithms such as weighed average. Even though I hadn't developed any project in Scala yet, I had a good feeling that moving from Java to Scala could be similarly right decision as moving from C/C++ to Java 10 years ago. Having only a good feeling usually is not enough to adopt a new programming language in a commercial project but it is certainly sufficient to use it for a research open source project. Because I was just about to start a new research project to develop artificial intelligence aided trading simulator [2], I decided to choose Scala programming language instead of a well known and commercially proven Java. Now, one year later, after writing 8622 lines of code in Scala [3], reading Programming in Scala book [4] and attending Martin Odersky's Scala course [5], I believe I'm in a good position to review my original statement on moving from Java to Scala. Was it a good decision or I made a mistake? I will try to answer this question in a context of research open source project as well as I will make an attempt to apply my observations to enterprise and commercial environment.

Evaluation methodology

Reading this Scala evaluation please put a bit of attention to what is it about and what it is not. For sure, I'm not trying to examine Scala just by looking only at its design, syntax and other language capabilities. Instead of that, I'm aiming to answer the questions raised in the introduction above. Was it a good decision or I made a mistake adopting Scala? Is Scala ready for Enterprise project or not yet? To answer those questions, in addition to evaluating Scala itself, a few other factors have to be considered: 
  • Programming language - Here, I evaluate features of Scala programming language, its paradigm, design and so on. In other words, all parts and aspects of Scala that were important for me developing trader simulator.
  • Testing - How do we test Scala code?
  • Performance - How fast Scala programs are?
  • Tools - Doing any serious software development and not using some useful tools is hardly possible. The most worth to look at are: Integrated Development Environment, build tool, continuous integration, code coverage, static code analysis, memory and cpu profiler.
  • Language extensions - Powerful and robust programming language doesn't guarantee any success yet. Third party tools, libraries and frameworks, that come from both open source community and commercial vendors, are very important as well. 
  • Interoperability - Interoperability with other programming languages by no means is a must requirement in developing some kinds of computer systems, especially with the most popular ones such as Java and C/C++.
  • Monitoring and maintenance - In enterprise, assuring high availability at the level 99.99 and higher is a fundamental thing and requires wide range of tools for production monitoring and analysis.  
  • Support - Adapting new technology, particularly programming language, always involves some risk. Active world wide community, availability of trainings, forum, mailing list is very important. Additionally, well established global commercial company that stays behind programming language makes us to believe that we won't stay alone in the future. 
  • Scala skills - Even the most powerful weapon won't win the battle if you don't have a  well skilled army that is able to use it.
  • Time to Market - Eventually, whether or not it is worth to use a particular technology is a business decision that is based on some short and long term financial indicators. Though, we do software for money and having a fun may be just a nice side effect of that.
One more aspect that has to be taken into account for completeness of this evaluation is a business domain to which programming language is applied to. In this case it is risk management, data collection, market simulation and trading on betting exchange [6] markets. 

There is not much point evaluating any programming language and not taking business domain into account. There are many programming languages in the world and some may fit one business domain whereas others master somewhere else. For instance Erlang fit well telecom, C is great for developing embedded systems, dynamic languages such as Ruby or Groovy are useful for developing web applications and Java fit well everywhere as a general purpose language. Scala has some better and worse appliances as well.

To evaluate Scala programming language in a context of all categories and business domain mentioned above, I will examine all of them one by one applying the following point system:

1 - poor, 2 - fair, 3 - good, 4 - very good, 5 - excellent

Finally I will make an attempt to interpret those scores and to come up with some relevant conclusions.

Evaluation

Over last year, developing trading simulator I was taking some notes on many aspects of adopting Scala programming language in a light of all evaluation categories presented above. I share with you all my thoughts and observation I made on Scala and I hope you will find them useful when taking your own decision on adopting Scala for some of your projects.

Programming language

The trading simulator, that I decided to use Scala for, can be characterized as a complex algorithmic system that each line of code matters rather than a CRUD (create, read, update, delete) application [7]. Writing most of the code requires a lot of thinking up front and looking at a written code should make it obvious what it does and how does it work. For many reasons Java doesn't fit here well, mostly due to following an imperative programming [8] paradigm and its verbose nature of a programming style. What I'm looking here for is a functional programming [9] and Scala is a place where I found it being simple and natural to use.

Before I carry on, please forgive me for presenting hardly any Scala code in this post. I do it on purpose, because my goal is not to show you how Scala syntax looks like and how to use it. For that, there are much better resources to go for, for instance great Programming in Scala book. In reverse, I prefer to focus on sharing with you my experience with Scala and then you can decide yourself whether or not it's worth to pursue it further.

Now, probably some people would question, why Scala and not other functional programming language such as Lisp [10] or Erlang [11]? Well, the main reason is that Scala code compiles to a byte code and runs on Java virtual machine, thus it is compatible with other Java applications. It is especially important if a majority of your code and libraries that you already use are coming from a Java world. The two libraries I was using myself for Trading Simulator project were Esper Complex Event Processing [12] and Jackson JSON parser [13]. I would probably stay with Java if I couldn't use them in Scala.

At this moment, others might ask why Scala and not let say Clojure [14] that is a functional programming language running on Java virtual machine as well? To answer this question we need to look at two features that distinguish both Scala and Clojure. First, Scala syntax is similar to Java, whereas Clojure is a Lisp dialect, that for me, a person who has been working mostly with Java for last 10 years, makes it far easier to use Scala than Clojure. Second, Scala is a statically typed language [15] and Clojure is a dynamic programming language [16]. Static typing is a core feature for me when developing critical and complex applications, First of all, debugging and maintaining statically typed code is easier and faster, secondly code written in a statically typed language is less error prone due to picking up some errors on a compile time and that is true regardless how good unit tests are.

There is one more important aspect of Scala that I'd like to mention to you. Scala is a multi-paradigm programming language, it means that it allows for mixing multiple programming styles such as object oriented, imperative and functional programming.  Whether it's good or not, it is not a question that could be answered with a simple yes or no. Supporters would say that programmers can pick up the best of all styles depends on theirs needs. Others would argue that putting together some features found in many other programming languages can't work well, mostly by increasing complexity and making a programming language obscure.

If you ask myself on this topic, to some degree, I would agree with both positive and negative arguments above on multi-paradigm in Scala. On one hand, if Scala would be a pure functional language I wouldn't go for it. I like functional programming and I'm finding it very useful in many aspects of software development but I would lie if I said that imperative programming is a devil that we should all avoid. It's just not true and it's fair enough to look how popular Java nowadays is. One example of a computer system that imperative programming fits very well is a stateful  high performance betting exchange engine that is a part of trading simulator I was developing over last year. Moreover, it fits there even better when mixed up with functional programming.

Regarding to a negative argument on a high complexity of Scala I have to admit that to a some level I agree with that. For instance please take a look at the one of the most commonly used method in a Scala version of a List class.

//Builds a new collection by applying a function to all elements of this list.
def map [B, That] (f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]) : That

Is it obvious to you what it is for? Would a novice Scala programmer understand it? Well, probably not easily, but definitely such syntax could discourage many of developers. To be fair, I have to mention that in Scala 2.8 this method in a Scala doc is presented following much simpler syntax, but in overall, it may not change a quite bad initial impression.

def map [B] (f: (A) ⇒ B) : List[B]

Additionally, some complexity in Scala is dictated by assuring compatibility with Java. For example in Scala there is an Option class that can take two values, Some and None. It's a great replacement for null value in Java and all would be fine if null value wasn't available in Scala too. When I asked Martin Odersky why is that, he explained me it's mainly to be able to stay compatible with Java. I think it's a relatively low cost of that.

There are many other examples that I could bring up to demonstrate how potentially evil Scala could be but it doesn't make too much sense. The Scala code that for someone is obscure, for others could be a nice and neat form of solving some specific problems. Personally, in most cases I found Scala syntax simple and natural to use and my opinion has not been changed after spending one year developing quite advanced application in Scala.

Next positive aspect of Scala is its conciseness. We all know how Java verbose is and how many boiler code developers write everyday including constructors, getters, variable initialisation (type and generics), semicolons and many others. To illustrate that I present how a Person class is defined in both Java and Scala:

Java version:

public class Person {
  private final String firstName;
  private final String lastName;

  public Person(String firstName, String lastName) {
    this.firstName=firstName;
    this.secondName=secondName;
  }

  public String getFirstName() { return this.firstName; }
  public String getLastName() { return this.lastName; }

  //hashcode, equals and toString methods
}

Scala version:

case class Person(firstName:String, lastName: String)

If you think that I'm showing to you some marketing gadget similar to presentations on 'How to build a web based application in 5 minutes', I do assure you it is not the case. Over last year, I was discovering every month, more and more examples that were proving to me how useful concise syntax in Scala is.

There are a few other features in Scala that are worth some attention, I won't describe them in details but I encourage you to read about them yourself in a Programming in Scala book. Those features include, creating DSL (domain specific language)  in Scala [17], pattern matching [18], tail recursion [19], type inference [20], implicit conversions, xml and json parsing and concurrent programming with Actors [21].

Until now, I wasn't too critical on Scala, does it mean that Scala is a perfect programming language? Unfortunately, every rose has its thorn and Scala has one as well. It's a poor backward compatibility in Scala. For example, I started developing trading simulator with Scala 2.7 and then I migrated to version 2.8. Even though I wrote no more than 3K lines of code I faced a few compilation errors and one failure of a functional unit test. One of areas widely affected by those changes in Scala is Collections API (see Scala 2.8 Release Notes [22]). Scala team and Martin Odersky himself are justifying all those significant changes by improving overall quality and design of Scala. That of course makes sense but sometimes I feel that this improvement process is happening to fast and some formal body similar to JCP (Java Community Process) [23] would help. That is especially important now, at the day when Scala is entering Enterprise world.

To summarise, how I'm describing Scala so far, it's a statically typed and multi-paradigm programming language that runs on Java virtual machine and provides Java like syntax with a few improvements. It sounds a bit familiar, doesn't it? Isn't it the old good Java with support of closures and some syntax improvements. If so, why I don't wait for some time until closures are supported in Java? Though I could live without closures for 10 years, so why I can't wait a bit more time? Actually this was one of the main questions I asked myself one year ago, before I started trading simulator. Then I asked the same question one year later to Martin Odersky and now I will try to answer this question to myself as well as to you dear readers in the end of this post.

My overall score to Scala programming language is very good (4).

Testing

Developers are provided with all they need to test Scala applications efficiently. To follow a Test Driven Development methodology [24], they can use popular in Java world Junit tool [25]. If someone is more keen on Behaviour Driven Development [26] then ScalaTest is a way to go [27]. I use both of them and they work very well. Especially, the second tool deserves some attention here, as it's a good example to illustrate one more powerful feature of Scala that I haven't presented to you yet. Please look at the following test written in a ScalaTest tool.

scenario("pop is invoked on a non-empty stack") {
  given("a non-empty stack")
  when("when pop is invoked on the stack")
  then("the most recently pushed element should be returned")
  and("the stack should have one less item than before")
}

What is interesting here that it's an absolutely correct Scala code. Its syntax is provided by a domain specific language, which is built on top of Scala. Can Java make it? probably not. To make it working in Java we would need to create a DSL on top of Groovy and then call it from Java, for instance Easyb testing tool [28] represents this approach. Though, it's a bit problematic as calling Java code from Groovy carries on some level of overhead and complexity due to integrating two different languages.

In my opinion, there are some business domains where ability to create domain specific languages in Scala is an extremely useful feature. The one example is a domain specific language for trading on financial markets. 

My overall score to Scala testing is very good (4). 

Performance

At first sight the performance in Scala is very good, comparable to Java, thus Scala code compiles to the same byte code as Java does and runs on the same Java virtual machine. Nevertheless, how fast Scala code is, may vary from case by case and usually it's up to the way how the code is written. This awareness how to write a high performance Scala code is especially important for Java developers who are not very experienced in both Scala and functional programming.

For instance poor knowledge of Collections API in Scala may lead developers to call often a size() method on a List class. Because this method is not an indexed sequence as it is in Java, therefore calling it is a very expensive operation as it has to traverse through all elements in a List to count all of them. The other example is to implement a function that multiplies all numbers in a List by 2 and then filter out all that are greater than 10. In Java, we would probably create a single loop that does this work but in Scala, following functional programming we could write:

list.map(e => e*2).filter(e => e>10)

Looks simple and in many cases that's the solution we would use but under the hood it's much more expensive algorithm that the iterative version in Java (more details). On the other hand, having a choice between imperative and functional programming, availability of tail-recursion and so on, we can achieve better performance in Scala than in Java by carefully choosing proper algorithms for particular tasks. 

There is one important thing for me that I noticed myself after one year I spent with Scala. In Java, usually development can be characterized as 'write first then think' approach, in Scala it's in an opposite way 'think first then write'. In Java, most of code is constructed by combining some loops and variables together that in many cases is a routine work. Whereas in Scala, developer is much more encouraged to think first and then write. From my own experience, when applying 'think first then write' approach, the final result of software development work can be impressive.

Next thing that I learnt myself by adopting the 'think first then write' approach and gaining some good skills in functional programming is that I became a better developer, whichever programming language I use, either Scala, Java or something else.

Going back to a performance aspect in Scala, I would like to mention one improvement that was added to Scala version 2.8. It's a specialisation for auto boxing [29], which regarding to Iulian Dragos [30], creator of this feature, provides a performance improvement of a factor of 4 to 30. Iulian gave an interesting talk on this topic on December 2010 at Skills Matter in London. The pod cast is available here:  

My overall score to Scala performance is very good (4).

Tools

Until now, there are not too many development tools created specifically to serve Scala, but fortunately, thanks to compatibility with Java it doesn't look so bad.

For a build tool, I adopted maven [31], which with some help of maven-scala-plugin [32] works very well. Having that we use maven, there shouldn't be any problem to adopt some of popular Continuous Integration [33]  tools such as Bamboo [34] or Ant Hill Pro [35]. Alternatively you could use SBT [36], simple build tool for Scala. You will find a pod cast on a Simple Build Tool here:

As Integrated Development Environment I utilised Eclipse Scala IDE plugin [37]. When I started using this plug-in more than one year ago it was very unstable and it was crashing my Eclipse a couple times a day. It was so frustrating to a level that I nearly abandoned Scala. By good luck I've been keeping up and sometime later, Scala IDE plug-in was getting better and better. At the moment there are still some features barely supported such as re-factoring or debugging, nonetheless at least it is stable enough to not complain too much. I heard some good opinions on IntelliJ Scala plugin [38] but because I haven't used it myself I'm not able to give you more information.

For a purpose of a memory and CPU profiling I use Jprofiler [39] and Eclipse MAT (Memory Analyser Tool) [40]. Despite of being dedicated Java tools, they can be used for analysing Scala applications as well. Personally, I found some difficulties while using those tools with Scala, for example they won't let you to analyse parts of your application that use some Scala features that are not available in Java, for instance companion objects and closures. I wouldn't say that it will stop you from using those tools, just the user experience may not be so good as when analysing Java applications.

One family of tools that in my opinion at the moment are hardly usable in Scala are those, which do some sort of code analysis, for instance code coverage [41] and static code analysis [42]. Especially the code coverage tools are those, that I always use quite extensively. I tried to use Eclipse Emma plugin [43] to check ad-hoc a code coverage for a written Scala code and it just didn't work, blowing up with some errors. I have no evidence on other code coverage tools but even if they work somehow, the reports they generate may not be well tuned against Scala code.

Static code analysis tools are even less usable. I evaluated two popular Java tools, pmd [44] that operates on a source code level, and find bugs tool [45], which inspects Java byte code. Both of them report a big number of warnings and errors that are not relevant to a Scala code. That is not a surprise as those tools are optimized to run against Java code.

My overall score to Scala tools is fair(2).

Language extensions

Most of libraries and frameworks are available to Scala due to compatibility with Java, otherwise we wouldn't have too much choice. The two worth to mention native Scala extensions are Lift web framework [46] and Akka actors platform [47]. I have no much experience with Lift web framework so far, so I will leave you alone with it. In terms of Akka, it's a very interesting project lead by Jonas Bonér [48]. In a few words, it's a platform for developing event driven, distributed, scalable and fault tolerant systems adopting Actors concurrency model.

The majority of libraries and frameworks from a Java world, should be possible to use quite smoothly, which among other things, is thanks to implicit conversions [49] in Scala. For example, in Java to convert between two different implementations of a List we would need to call explicitly some custom code that populates an output list by iterating through all elements of an input list. In Scala we also need to write a such code, but the difference is that it is called behind the scene. Please consider the Scala code example below.

//Initialises java.util.ArrayList object with three integers and converts it implicitly to a Scala List.
val newList:List = Arrays.toList(1,2,3)

It is like extending generic language rules for type conversions such as those between numeric types and it really makes it far easier to use external libraries in Scala.

My overall score to Scala language extensions is good (3).

Interoperability

Interoperability between Scala and other programming languages, including Java/C/C++, is very good, mostly because of running on Java virtual machine. What Java may talk to, Scala can do as well. Taking into account support for implicit conversions described above, I would state that Scala is one of the leaders of interoperability among all programming languages.

My overall score to Scala interoperability is excellent (5).

Monitoring and maintenance

For me, the main monitoring tool to analyse production Scala applications is JMX (Java Management Extensions) [50], and again, this is with no surprise thanks to compatibility with JAVA. This tool does its job well when we want to analyse some predefined statistics, but sometimes we need to investigate some aspects of a production application while it's running and JMX can't provide all required data.

To deal with such scenarios, Java provides Java Virtual Machine Tool Interface (JVM TI) [51] that allows for inspecting running Java applications but it can be used for Scala too. Imagine, there is a method inside running Scala application that calculates and returns some data and we want to be informed every time when this method is invoked and what is the value it returns. We can do such things with a btrace [52], command line dynamic monitoring tool from open source world. Eventually if we prefer more sophisticated and powerful platform for monitoring production applications we could choose Dynatrace [53] commercial product. The first one works well with Scala, though not as good as with Java, because it's not a dedicated Scala tool. I haven't tried the second one with Scala but I expect it to work as well, thus it uses the same JVM TI interface as btrace does.

My overall score to Scala monitoring and maintenance is good (3).

Support

Scala community, including forum, mailing list and blogs, is not the biggest in the world, but it's very energetic. My feeling is that most people who are part of a Scala community are very passionate developers, which are always happy to help others to solve theirs problems. It was especially important for me on the beginning of my adventure with Scala.

Next important part of a Scala support are books and here I also found a great position. It's a Programming in Scala book by Marting Oderski, Lex Spoon and Bill Venners. There is big chance that I wouldn't pursue Scala if this book hadn't existed. I'm a such sort of person who usually start learning some new technology by reading some good book. If one doesn't exist, then probably this technology is not worth a penny (at least not yet).

One thing that would definitely help Scala to spread around the world is a global commercial vendor, such as Oracle or IBM. But even if no one ever exist it doesn't mean that Scala can't became a globally used programming language. Nobody should under estimate the power of an open source world wide community.

My overall score to Scala support is good (3).

Scala skills

Even though I don't posses any professional data on a number of Scala developers in the world, I think no one will argue that it is far behind other programming languages such as Java or C/C++. I performed two simple experiments to get some data for you. First I checked for Scala job trends [54] and the number of job offers for Scala developers is barely noticeable but is growing dynamically over last two years. Second I searched for Scala developers on the linkedin.com social site and I got 1709 results. Just to mentioned that querying for Java developers returned 249,589 entries. It doesn't look good, does it? 

One positive aspect I found is that looking for Scala developers we could search for Java developers with Scala interests as well. The learning curve is not massive as Scala follows Java syntax. In such case, at least on experienced Scala developer would help the team adopting new programming language.

My overall score to Scala skills is poor (1).

Summary

To summarise Scala evaluation, this is a scores matrix for all categories (1 - poor, 2 - fair, 3 - good, 4 - very good, 5 - excellent).
  • Programming language - very good (4)
  • Testing - very good (4)
  • Performance - very good (4)
  • Tools - fair (2)
  • Language extensions - good (3)
  • Interoperability - excellent (5)
  • Monitoring and maintenance  - good (3)
  • Support - good (3)
  • Scala skills - poor (1)
The best way to decide on some technology for some project is to ask about the time to market [55], however the problem arises when we stay in front of deciding on some new, not proved yet technology. In that cases, emerging technologies usually are on hiding to nothing, they can hardly win. But sometimes new technologies find a place on a market and I believe that Scala is one of them.

Scala already has beaten Java for developing my research projects and I have no temptation to move back. How about Scala in Enterprise? Here, I'm not so optimistic. Java is a really strong player and being just a better programming language may not be enough. What if Java finally provides functional programming and some better and more concise syntax? What if then Scala disappears from a market and we are in a half way through migrating our applications from Java to Scala? Will we move back to Java? Who will pay for that? Those are the questions that everyone must answer himself.

One advice I can give you myself is to not adapt Scala, just because someone says it's cool, so I want to be cool too. Something that works well in someone's business domain may not work for you. Be reasonable and try to get some experience with Scala first, for instance by getting involved in some open source projects.

Thank you for your attention and I hope you enjoyed it. I invite you to comment on this post and to share your thoughts with others. Daniel.

References
  1. Post on moving from Java to Scala - http://blog.danmachine.com/2009/12/time-to-change-moving-from-java-to.html
  2. Initial post on applying artificial intelligence to betting exchange research - http://blog.danmachine.com/2010/04/betting-ai-artificial-intelligence.html
  3. Betting AI project sources -  http://code.google.com/p/betting-ai/source/browse/#svn%2Ftrunk
  4. Programming in Scala book by Marting Oderski, Lex Spoon and Bill Venners - http://www.goodreads.com/book/show/5680904-programming-in-scala
  5. Scala course by Marting Odersky and Julain Dragos - http://skillsmatter.com/course/scala/object-oriented-meets-functional-an-exploration-of-scala
  6. Betting exchange - http://en.wikipedia.org/wiki/Betting_exchange
  7. CRUD - http://en.wikipedia.org/wiki/Create,_read,_update_and_delete
  8. Imperative programming - http://en.wikipedia.org/wiki/Imperative_programming
  9. Functional programming - http://en.wikipedia.org/wiki/Functional_programming
  10. Lisp programming language - http://en.wikipedia.org/wiki/Lisp_(programming_language)
  11. Erlang programming language- http://en.wikipedia.org/wiki/Erlang_(programming_language)
  12. Esper Complex Event Processing tool - http://http://esper.codehaus.org/
  13. Jackson high-performance JSON parser - http://jackson.codehaus.org/
  14. Clojure programming language - http://clojure.org/
  15. Static typing - http://en.wikipedia.org/wiki/Static_typing#Static_typing
  16. Dynamic typing - http://en.wikipedia.org/wiki/Static_typing#Dynamic_typing
  17. DSL (Domain Specific Language) - http://en.wikipedia.org/wiki/Domain-specific_language
  18. Pattern matching - http://en.wikipedia.org/wiki/Pattern_matching
  19. Tail recursion - http://en.wikipedia.org/wiki/Tail_recursion
  20. Type inference - http://en.wikipedia.org/wiki/Type_inference
  21. Actor model - http://en.wikipedia.org/wiki/Actor_model
  22. Scala 2.8 Release Notes - http://www.scala-lang.org/node/7009
  23. JCP (Java Community Process) - http://jcp.org
  24. Test Driven Development - http://en.wikipedia.org/wiki/Test_driven_development
  25. Junit Test Driven Development tool - http://junit.org/
  26. BDD (Behavior Driven Development) - http://en.wikipedia.org/wiki/Behavior_driven_development
  27. ScalaTest BDD tool - http://www.scalatest.org/getting_started_with_feature_spec
  28. Easyb BDD tool - http://code.google.com/p/easyb/
  29. Auto boxing - http://en.wikipedia.org/wiki/Object_type_(object-oriented_programming)#Autoboxing
  30. Iulian Dragos - http://lamp.epfl.ch/~dragos/
  31. Maven build tool - http://en.wikipedia.org/wiki/Apache_Maven
  32. Maven-scala-plugin - http://scala-tools.org/mvnsites/maven-scala-plugin/
  33. Continuous Integration - http://en.wikipedia.org/wiki/Continuous_integration
  34. Bambo CI tool - http://www.atlassian.com/software/bamboo/
  35. Ant Hill Pro CI tool - http://www.anthillpro.com/html/default.html
  36. Simple Build Tool - http://code.google.com/p/simple-build-tool/
  37. Eclipse Scala IDE - http://www.scala-ide.org/
  38. IntelliJ Scala IDE - http://confluence.jetbrains.net/display/SCA/Getting+Started+with+IntelliJ+IDEA+Scala+Plugin
  39. Jprofiler tool - http://www.ej-technologies.com/products/jprofiler/overview.html
  40. Eclipse Memory Analyser Tool - http://www.eclipse.org/mat/
  41. Code coverage - http://en.wikipedia.org/wiki/Code_coverage
  42. Static code analysis - http://en.wikipedia.org/wiki/Static_code_analysis
  43. Eclipse Emma code coverage plugin - http://www.eclemma.org/
  44. PMD static code analysis tool - http://pmd.sourceforge.net/
  45. Find bugs static code analysis tool - http://findbugs.sourceforge.net/factSheet.html
  46. Lift web framework - http://liftweb.net/
  47. Akka actors platform - http://akka.io/
  48. Jonas Bonér home page - http://jonasboner.com/
  49. Type conversion - http://en.wikipedia.org/wiki/Type_conversion
  50. Java Management Extensions (JMX) - http://en.wikipedia.org/wiki/Java_Management_Extensions
  51. Java Virtual Machine Tool Interface (JVM TI) - http://en.wikipedia.org/wiki/Java_Virtual_Machine_Tools_Interface
  52. Btrace dynamic tracing tool - http://kenai.com/projects/btrace/pages/Home
  53. Dynatrace monitoring production applications tool - http://www.dynatrace.com/en/
  54. Job trends report on Scala jobs - http://www.indeed.com/jobtrends?q=scala&l=
  55. Time to market - http://en.wikipedia.org/wiki/Time_to_market

44 comments:

  1. thank you for this detailed article

    ReplyDelete
  2. I chose Scala language as the main language to develop my systems. Very useful information :)
    Thanks Daniel.

    ReplyDelete
  3. I believe that Scala is not ready for business and in the Future Java will add best feature of Scala to itself.

    ReplyDelete
  4. Please use hyperlinks for the bracketed references ([1] etc.) in your text.
    Having a list of citations is nice, especially for older people used to this "paper" style of referencing, but you really shouldn't force the reader to manually scroll to the bottom of the page instead of providing a link.
    Thanks!

    ReplyDelete
  5. Thanks for spotting it. I just added hyperlinks, I hope it will help.

    ReplyDelete
  6. What about time to market?

    ReplyDelete
  7. Excellent review!

    Although I'm a Clojure user mostly, it is always good to know what is happening in Scala world :)

    I must agree with you: Scala (the same could be applied on Clojure) are probably not ready for the enterprise yet, as they are solving java problems with uncommon approach (functional way), making a quite challenge from developer to get used to that way of thinking.

    As you said, at the end, java will probably pick the best features from both languages...

    ReplyDelete
  8. Hi Daniel,
    Awesome article, thanks for collecting so valuable resources under one roof, hoping myself would go on the same way..

    ReplyDelete
  9. There was a time where there were no Java skills around...

    ReplyDelete
  10. Have no illusions, people - Java will *not* pick the best features of Scala. Vast amount of cruft accumulated in Java over the years makes that simply impossible without breaking most of software written in it. Martin Odersky had realised it once, and that was one of reasons to make Scala's syntax incompatibile.

    Java will add new syntax, indeed, but with even more subtle rules, that you have to know in order to write bug-free code. Java's as a language is a dead end.

    Instead of continuing to wade through this swamp, you're better of trying out Scala, sharing your experience with it and helping spreading the knowledge. While in some cases Java is still going to remain an optimal choice (for some time), Scala is constantly getting more attractive in new domains.

    ReplyDelete
  11. Hi Daniel,
    thanks for very good article. One of very few 'I moved' articles with neutral judgement. I moved about 10 months ago and could not agree more.

    Also give shot to Idea, it works great with Scala

    ReplyDelete
  12. Very useful, thanks for sharing.

    ReplyDelete
  13. One tiny comment : interoperability goes both ways. And while it's easy to see scala interoperability to other languages is very good, the reverse is (imho) much less.

    Calling Scala from C, or even from Java is not at all easy.

    ReplyDelete
  14. Very good read. And thanks for introducing me to Scala over 1 year ago. :)

    Also, Scala is simply more fun to use than Java. Scala is like Ariel Atom, while Java is like Bugatti Veyron. ;)

    ReplyDelete
  15. I also switched to scala about a year ago. I simply cannot turn back anymore to Java. Consider although that I am working in a research company where quick prototyping, the ability to do both extremely flexible code and quick-and-dirty tools is a must. I find myself often programming in Scala to test an idea, just to reimplement it in C++ and assembly as we have real-time constraints.
    I still use Java of course, but I don't think enterprise is such a big obstacle to jump. What Scala needs now are tools. Quality tools.

    ReplyDelete
  16. Hey,

    A quick correction, it's Iulian Dragos[1], not Julian. There are two instances of the typo.

    Best,
    Ismael

    [1] http://lamp.epfl.ch/~dragos/

    ReplyDelete
  17. Hi Ismael,

    Thank you for your spot. It is fixed now.

    Regards.
    Daniel

    ReplyDelete
  18. Excellently written article Daniel, with an unbiased evaluation on several criteria.

    ReplyDelete
  19. having been a Java developer myself for the last 6 years, I decided to try Python(3) 4 months ago, last week I decided to try Scala simply cause I'm more a Java guy ... thanks for all this info, very useful

    ReplyDelete
  20. I am currently using Groovy/Grails. Despite the productiveness boost it delivers compared to Java, I just hate that it is not statically typed and that many errors are not detected by the IDE/compiler. I've looked at Scala and liked it very much. However, for it to be useful to me, I would also need a robust library to interact with the database. Lift seems to be very html-centric (my front ends are in Flex), with a steep learning curve. Querulous from Twitter looks good but it is for an older version of Scala. Hibernate seems to be difficult to use with Scala .... Any suggestions?

    ReplyDelete
  21. Nice article. Agree with most of the points.

    @Eduardo, Lift can be great for doing REST-backed frontends as well. People on the ML have built e.g. apps with a Cappuccino frontend and Lift backend.

    ReplyDelete
  22. @Eduardo: Google "Squeryl" and "Circumflex ORM" -- both work.

    ReplyDelete
  23. @Jeppe and @sorhed: thanks for the tip, I will check them out! And, of course, thanks to Daniel for posting this and hosting this discussion.

    ReplyDelete
  24. Many thanks for the follow up, very useful.

    I would only say that if there was no Scala-plugin for IntelliJ, I would not be using Scala language. It is a must!

    Also, JRebel has proven to be VERY useful, basically if you develop a web application in scala with maven then you can
    export JREBELHOME=$HOME/java/lib/ZeroTurnaround/jrebel-3.5
    then start project with
    mvn jetty:run
    and in separate window mvn scala:cc

    and then development experience is close to RubyOnRails, or PHP - change code and immedialy test it - very very cool

    ReplyDelete
  25. To eduardo: Scala driven Wicket is awesome. Also look at my previous post for tips on making development very productive and dynamic

    ReplyDelete
  26. @Eduardo And there's scalaquery too.

    ReplyDelete
  27. @Daniel: thanks. I get the impression there are quite a few options. But it also seems that none (except Lift) is mainstream with a lot of followers and good docs. I think I will try to learn Lift. I've started learning Scala by reading David Pollak's book, which I found excellent. I am also very impressed at how he (who has rock-star status in the Scala world) actively promotes the language and the framework in web forums, even answering beginner's questions! I heard the rest of the community is also very helpful. I just hope there would be slightly better docs for the database persistence part of Lift. I've read there is Mapper and that the future is Record, but could not find any docs on the latter. The examples I found for Mapper seem to have too much boilerplate compared to Grails/Gorm.

    ReplyDelete
  28. @daniel
    I tried pollack´s book too, same conclusion: it´s really excellent!

    I just finished my bachelor thesis about scala (german) last week. Again, almost the same conclusion as Daniel´s post.

    ReplyDelete
  29. Hi Daniel,

    Excellent review, thanks for sharing your experience !

    Note on code coverage tools : the young "scct" already does marvels (it *just worked* out of the box in a few seconds on my two big projects without any issue, I was just amazed).

    http://mtkopone.github.com/scct/

    Cheers

    ReplyDelete
  30. @Eduardo Datanucleus works reasonably well with scala.

    ReplyDelete
  31. Excellent review, just saw the link on scala-lang
    I do recommend IntelliJ's scala plugin, its working really nice with maven and scalatest support

    ReplyDelete
  32. If you are planning to start a new open source project in Scala I would like to work on one. I have read the entire book Programming in Scala second edition cover to cover. I have started writing small Scala programs.
    Here are my details :
    <>
    gmail id : prathamesh.mone@gmail.com

    ReplyDelete
  33. Hi Prathamesh,

    All enthusiastic developers are welcome to join my betting exchange research project. Please contact me by email: daniel [dot] korzekwa [at] gmail [dot] com to discuss it further.

    Regards.
    Daniel

    ReplyDelete
  34. Excellent review!
    I am evaluating akka, this is very useful.

    ReplyDelete
  35. Hi Daniel!
    Being a java & ruby developer, I've been hearing a lot about scala in the recent past and have been contemplating to use it on more full time basis (Just learning the basics right now). I must say that due to your 2 posts here, my affinity towards scala has been reaffirmed.
    I'll be going through the books/tools you mentioned over a period of time now and maybe post something about my experiences on my blog too.

    ReplyDelete
  36. First, someone up in the comments said that calling Scala from Java is not easy. Well I disagree. I do it in a small part of a Java project, where we have both Java > Scala and Scala > Java communication, and after configuring the maven scala plugin, it works like a charm.

    Next, I mostly agree with you in terms of tools; this is probably the place where Scala still needs to improve the most. That being said, NetBeans (yes, you ignored it =p) has a very nice plugin - usually, refactoring works nicely, for example.

    Finally, regarding enterprise adoption, the scene is not so bad. The scala-lang page has a nice list of Scala users in the enterprise: http://www.scala-lang.org/node/1658 - companies like Twitter, LinkedIn and Foursquare are included. This last one even seems to use a pure Scala solution, with the Lift framework.

    ReplyDelete
  37. I thought clojure was statically typed, too...?

    ReplyDelete
  38. Interesting to note that since publication of this article LinkedIn ratio of Scala/Java has gone from .6% to 2%, Java climbing 3x and Scala 10x.

    Straw analysis

    Java: Demand flat, supply increasing
    Mature/Stagnant Market => Hold/Sell

    Scala: Demand and supply rapidly increasing
    Growth Market => Buy

    ReplyDelete
  39. your article's anniversary is coming up soon;

    update is due? :-)

    ReplyDelete
  40. I think there is not much to update on, which would make me desired to write next 'one year later' post on Scala. I wish I could write, 'yes, one year later, Scala programming language finally entered Enterprise market'. I think next time I post on Scala is when either it vanishes from the market or it achieves amazing success...in three years, maybe more.

    ReplyDelete
  41. Great post man, before I move to Scala I was looking for experience someone has in Scala and you highlighted quite a few things. Thanks

    ReplyDelete
  42. With Scala, you'll think like a mathematician or a player planning a strategy. Even if Java 8 (seen Java 8 lambda) or 9 are out, I find Java are still largely limited on imperative language, it still doesn't remove the boilerplate code altogether and will likely take sometime for Java 8 to adopt in the enterprise.

    As far as I can see JDK7 + Scala 2.10 is the best contender.

    ReplyDelete