If you want to hire an amazing engineer, you better be sure he or she can write amazing code. This can be a difficult skill to assess, particularly if most of your hiring in an organization is done through an HR department, or you’re looking for a programmer with very specific, new skills for complex projects and programs. Codility tests if engineers write correct and efficient code, a skill you want to evaluate early on in each tech recruitment process. We think it’s a fantastic idea, and Codility recently added Scala tests to their roster of languages, based on demand from some big name companies!
Last month, Typesafe’s Grzegorz Kossakowski, who works on the Scala compiler, met with the Codility team in Poland to learn a bit more about the company, and help them out with Scala. He was blown away! So we thought we’d set up a quick Q&A with Greg Jakacki, CEO and founder of Codility to learn a bit more...
Typesafe: What was the inspiration for Codility?
Greg Jakacki: Codility is the product of inspiration and necessity, so talking about inspiration itself would be unfair. Necessity surfaced when, at one point in my career, I had to assess many candidates for programming jobs and my single most common reason for rejecting them was their inability to deliver a correct solution to a simple programming problem. I wanted to speed up, and preferably automate, such assessments.
The inspiration came from the International Olympiad in Informatics (IOI), a worldwide programming contest for high-school students, and the ACM International Collegiate Programming Contest (ICPC), a similar event for universities. These contests are based on assessing the correctness and scalability of programs delivered by the participants. Since the early 1990s, organizers of the Polish IOI elimination rounds had been experimenting with the process of automating such assessments. I was aware of this, and applied such automation in recruitment.
TS: How does Codility work?
GJ: Codility provides hands-on programming tests to help companies hire better programmers. We carefully prepare descriptions of programming tasks that candidates have to solve during a test session. Tasks are developed together with a software suite that subsequently verifies whether the candidate’s solution satisfies all the requirements specified in the task description, i.e. whether the code behaves as prescribed and whether it satisfies the performance constraints. The tasks are formulated in a language-independent way and we admit solutions in 15 programming languages, including our latest addition—Scala!
TS: Recently, people responsible for hiring are questioning the value of test scores. What’s your take on that?
GJ: I can clearly see that mapping human abilities – an intrinsically complex and multidimensional object – onto a linear scale is likely to lose important parts of the overall information. We at Codility don’t claim to have discovered a perfect mapping solution by any means. We are pretty efficient, though, in identifying coding issues that are likely to manifest themselves on the job.
TS: How are you different from similar services, like Interviewstreet?
GJ: After our launch in 2009, several competing services have shown up, either emulating our business model directly or experimenting with adding additional services, such as sourcing or attracting prospective employees through contests.
Codility’s focus remains purely on delivering tests, and we do our best to make sure that we are the best on the planet in this respect. Creating a well-described, fair, clear and automated programming test requires an excellent understanding of the mathematical theory of programming, as well as experience in developing and maintaining concise and informative specifications. We build on 20+ years of experience of Olympiads in Informatics, as well as the experience of our team members in industry and academia, to make sure that our tests are of absolutely top quality, both from the perspective of the test sponsor (usually a recruiter) and the test taker (usually a job candidate). We’ve invested a lot of effort in substantial backstage machinery for manual and automated audits of the tests, cross-checks, tests that improve their accuracy over time, mechanisms that assure fairness of assessment between different programming languages, and more. We work hard to make sure that our tests are open-ended so that they do not restrict the creativity of the person taking the test. After all, some candidates may happen to be smarter than the authors of the testing tasks! We devote a lot of attention to making sure that our automated assessments do not penalize unusual or ingenious solutions.
TS: What kinds of companies are using Codility?
GJ: All kinds of businesses: big and small, start-ups (Square, SlideShare) and corporations (Amazon, Samsung), both with a business focus in software proper and elsewhere (AT&T, Bank of America). We also count academic institutions (University of Oxford, Broad Institute of Harvard and MIT) among our clients. Codility is useful any place programmers are being hired or assessed.
TS: How did the addition of Scala come about?
GJ: Scala support was requested by a large customer (who wishes to remain anonymous). Given the increasing popularity of the language, we decided to release this feature for the benefit of everybody. Enjoy!
TS: Can you give us an example of what a test you might give a candidate interested in a Scala gig might look like?
GJ: We believe in a hands-on approach, so go to http://codility.com/demo/take-sample-test/ and select Scala as the programing language. This is a simple programming problem that can be solved in a few lines of code, but which nevertheless requires a bit of thought about corner cases and performance scalability. One thing to bear in mind is that if you pass this test, it does not automatically mean that you are a great Scala programmer, as there is way more to programming than a Codility test assesses. On the other hand, failing this test suggests that some of your programming skills may be a bit rusty. In such a case, don’t delay: stay fit! Recall your Programming 101, sign up for an online course, take an extra hour and nail this one little problem! You can also check out Codility’s training page on http://codility.com/train/.
