Thursday, August 14, 2014

Functional Programming Principles in Scala

It was a bit of surprise to me to discover a few years ago that Functional Programming (FP) was considered to be(come) mainstream.

FP everywhere, JavaScript, Groovy, Scala and even in Java 8. Not to mention Clojure, the Lisp revival.

Not that I doubt about the benefits of FP, but simply because FP was already mainstream, at least for me, back in the days when I was a student in a French University (Paris, Jussieu), at the beginning of the eighties... Thirty years ago !

Lambda calculus, LF (langage fonctionnel), an FP language implemented on Multics, Lisp, and more precisely LeLisp by Inria, were important parts of my degree course. Recursive programming was a way of life for me and my friends ...

And then after I worked in the real world. No more recursion, no more lists, no more car/cons/append/caddr and the like. Instead Awk, Perl, C, C++, Object Oriented programming.

So when I knew more about Scala in a Paris JUG meeting in april 2010, a language mixing OO and FP, I was puzzled by its potentialities.

I read some books about it  ... but eventually came to the conclusion that Scala did not fit well with my professional context (in a few words: too far from Java, and as such, requiring a collective interest/support/commitment accompanied by a strong professional training).

At the same time, I discovered the Groovy language and found some interesting use cases in the periphery of application development (in short, I started to use Groovy everywhere I used Perl in a Java context, mainly for portable deployment or administration tasks). Groovy is by nature Java friendly (one can use it simply as a better Java) and as such its learning curve is very smooth compared to what is required to practice on a daily basis in Scala.

And I forgot Scala for some years until I started to use the Gatling stress tool, whose DSL is written in Scala. This was the first reminder. The second one was Gitbucket, also written in Scala, which I started to use as an internal Git server. The third reminder was an article of Nicolas Martignole, a well known member of the French Java community, about its transition to Scala.

Finally, the spark was a tweet of Professor Martin Odersky:


I enrolled on the Coursera site, not very sure I will be able to have enough time and energy to reach the end of the course (I remembered the feedback given about it by Nicolas Fränkel some years ago:  First, the courses are quite time-consuming!)

This was my first Mooc experience. It is a seven weeks course, requiring a personal workload of about one day per week (an average because the assignments at the end of the course require more time. Unless you use a solution available on a github repository, but you will not do that because it is nonsensical !).

On Sunday at noon (central european time), the lesson of the week is made available (a series of video and slides) accompanied by an assignment due to at least one week later (it may be more, depending on the assignment). The assignments are well documented programming exercises which can be achieved using a "Scala ready" version of Eclipse Kepler downloadable at the beginning of the course.

Each assignment can be submitted up to five times to an automatic grading system based on a series of ScalaTests, some of them are delivered with the assignment, others are only known by the Coursera grading system. The best score of each assignment is taken into account for the final grade.

There is a forum on the course site where students can help each other or be helped by the course staff.

Optionally, Coursera propose you at the beginning of the course to enroll to its  Signature Track program (I paid $US 49 for this option). The main advantage of Signature Track is to bind your identity to the work you have made during the course. At the end of the course, you obtain an official certificate (available on an permanent URL you can distribute to whoever you want) stating that you have followed the course and obtained at least 60% of the maximum score - above 80% you obtain a certificate with distinction. If you opt for Signature Track, you authenticate yourself via your webcam and a series of keystrokes (which is known to be unique to you) each time you submit an assignment.

This course is known to have good results in terms of achievement. More on this on the feedback about it given by its authors.

I can say that this experience was for me a kind of escape, very refreshing, and I was full of optimistic feelings when I sent back my last assignment...

I cannot really say that it transformed me into a Scala developer (need some daily practice on actual projects to be one of them), but at least, I think that I have the required background to enroll to the other course of EPFL on Coursera "Principles of reactive programming" when a new session will be opened.

Sunday, April 20, 2014

Give context to your Puppet Manifests files using Vagrant

I am using Vagrant & Puppet at work and outside to develop deployment stuff (Puppet modules, native packages builders thanks to FPM).

Vagrant allows you to develop and
test your deployment process
Gitolite was the Git repository I installed at work to begin to share infrastructure code with my co-workers or directly with the infrastructure (a PuppetMaster for the time being).

Lately, in order to scale up our git practice, I installed the excellent Gitbucket to replace our  Gitolite server. 

Aside note: If you don't know Gitbucket and are interested by an internal Git server à la GitHub Enterprise (or à la GitLab), give it a try. It is free, and it is very - very easy to install (a war to drop). Lasts versions support SSH access on non privileged port (this was not the case initially, only http was supported).

Thanks to Gitbucket and it's friendly UI, I started to gitify also my Vagrant environments, and the manifests files accompanying them (it is a very convenient way to share ready to run Vagrant environments). 

My problem was the following: I must use a proxy at work, but not when I work outside. How could I pass this information (use a proxy or not) to the Puppet manifest running on to the guests VM I am provisioning with Vagrant, without having to modify the (now gitted) code of the manifest (a code which must be independent of the current location where I am working - office or outside) ?

I had a good solution right under my eyes since I started to use Vagrant, but I guess I was not yet "ready" to see it ...

In a Vagrantfile, it is possible to add custom facts to the Puppet provisioning system, and thereby to pass context information to the Puppet manifest that is executed within the guests VM.

Example 

config.vm.provision :puppet do |puppet|
        puppet.manifests_path = "manifests"
        puppet.module_path = "/path_to_git_extraction/modules"
        puppet.manifest_file  = "init.pp"
        puppet.facter = {
                "within_my_company_network" => `hostname`.chop.end_with?("acme.com")
        }
end

Then it is very easy to test this fact in the Puppet manifests file to provision or not the proxy:

if ($::within_my_company_network == 'true') {
        opensuse-tools::set-proxy { "proxy":
                proxy_host => "proxy.acme.com",
                proxy_port => "8080",
                no_proxy_domains => ".local, .acme.com",
        }
}

Wednesday, January 1, 2014

Transparent proxy and Maven Archetype plugin under Eclipse Juno

Recently I had a problem installing the Gatling stress tool under Eclipse JunoSR2 on a 12.04 Ubuntu development VM. Being located behind my enterprise proxy, it was impossible for me to download the Gatling archetype catalog, and thus to create Gatling projects within Eclipse.
http://www.squid-cache.org/

My Maven settings.xml file contained the required settings for proxyfication, and as a matter of fact, things worked well outside Eclipse (including archetype catalog download via the proxy).

Inside Eclipse, Maven was OK ... exception made for the component in charge of downloading remote archetypes catalogs. It ignored my proxyfication setup trying constantly to download things directly from the Internet and failing because my computer is on a private network and not natified.

I used the following trick to workaround this:

  • Installation of the squid proxy server on the VM
  • Setup of the squid server as a transparent proxy whose parent proxy is my enterprise proxy
  • Configuration of the iptables in order to redirect to Squid any request made to the 80 port, excluding destinations on my internal network 

Here is the squid.conf file:

http_port 3128 transparent
http_access allow all
cache_peer proxy.mycompany.com parent 8080 0 no-query no-digest
never_direct allow all

And the iptables rules required (starting from a null list):

iptables -t nat -A OUTPUT -p tcp --dport 80 -d 192.168.0.0/16 -j ACCEPT
iptables -t nat -A OUTPUT -p tcp --dport 80 -j REDIRECT --to-port 3128

The first rule aims at bypassing any transparent proxyfication for internal http trafic, the second one magically redirects any http outcoming request to the local squid which delegates any request  to the enterprise proxy.

All of this work with the usual proxyfication setting.