Moving to Persistent

I haven't said much in a while, but of course, things are moving in the EclipseFP world. JPMoresmau announced the 2.20 release of EclipseFP, which in terms of interface gives you a way to manage your local installation of Cabal packages and a better outline of Cabal files. Internally, a lot has changed, as EclipseFP moved from using Scion as backend to BuildWrapper. This big change will hopefully solve most of the memory problems that EclipseFP was experiencing.

Also, I would like to announce that scion-browser is moving from a in-memory database to using SQLite to back up the data. In my tests performance is comparable to the old one (although for big tasks as browsing the entire list of local modules is a bit slower), but doesn't need a big pile of memory to hold the entire database, which I think it's a big win. If you're curious, you can browse the code, but be warned it doesn't work with the current version of EclipseFP.

The change has been possible thanks to the incredible Persistent package of Yesod fame. With that, I hadn't had to write a single line of code to create the database or to insert data. The only problem has been that Persistent doesn't have any way to encode queries with multiples joins. This lead me to write some of the queries by hand (and to refresh my ancient knowledge of SQL).

Hope you like it. As usual, every suggestion to improve EclipseFP or scion-browser is more than welcome :)

EclipseFP 2.1.0 released!

As JP Moresmau points in this blog entry, a new release of EclipseFP is out! It includes bug fixes and new functionality, including my work during the Summer of Code.

These past days JP has been working very hard to fix some bugs my code had, and added support for Unicode characters in Scion Browser (which means that you can browse base-unicode-symbols, for example), so I'm really grateful to him.

Apart from that, the new EclipseFP website is out and contains information about how to install this plug-in your system. If you find any bug, don't forget to tell us so we can improve EclipseFP as much as possible!

As a final note, next Monday 5 of September I'll be speaking about EclipseFP in the beautiful city of A Coruña, in Spain, during the Taller de Programación Funcional.

End of the summer

The Summer of Code has come to its end. It has been three wonderful months, in which I've learnt and hacked a lot. I hope the improvements to EclipseFP will be used and have made it a useful tool for all the Haskell programmers out there :D

One of the tasks I've also done during the summer is building a new webpage for EclipseFP. You can find it in http://eclipsefp.github.com/. In the next days, a new release of EclipseFP with all the changes I've done though the summer will be made available.

I cannot end my summer whithout thanking my mentor, Thomas Schilling, who has been responsive and helped me a lot with the Haskell coding; and JP Moresmau, the current EclipseFP maintainer, who helped me in my first steps diving into the Eclipse Platform. I also have a debt with A O Van Emmenis and Chris Done for the discussions we had about IDEs and Haskell during the last CamHac and the pile of suggestions they gave me.

I hope you have enjoyed the show, and I'll be delighted of hearing how people used EclipseFP, and willing to implemented more features during the next months.

Quick Assist and new builders

This last week of GSoC has been really productive in terms of user interface additions. Apart from completion, which I showed a couple of days ago, EclipseFP now has quick assist support. Quick assist means that for some kinds of errors, a small light bulb is shown in the left side of the editor. When you click it, ways to correct the error are suggested, and if you click on it, automatically applied.

Most of these fixes were already implemented by JP Moresmau, like adding a package to the Cabal file, removing redundant imports or adding a Language pragma to the file. My work in those has been to make them more discoverable and available through the light bulb icon. Apart from those, you will get the ability to add an import for a function that is not found (it even works if you qualify the item, making the new import qualified).

The rest of the work I want to present today was done between CamHac and the end of the last week. I've always felt that Haskell programming was not only writing Haskell code, but also writing code in tools like Alex, Happy or the UU Attribute Grammar system. Those tools allow you to express some constructions better and more concisely, and I thought that it would be nice to have them in EclipseFP.
So now you have syntax higlighting and automatic building of those files.

Last year I was studying at Utrecht University and really liked their Attribute Grammar System. The tool has a lot of options, and now you can configure them inside EclipseFP. Even more, the configuration is saved in the same format that the one used in uuagc-cabal package.

This has been all for the summer. A new EclipseFP release with all the features should be available very soon :D

Autocompletion and more

In the past week I've continued adding features to EclipseFP. For example, now you can edit Alex, Happy or UUAGC files directly in EclipseFP, and you will get syntax highlighting (taking into accounts which zones are Haskell code and which others are non-Haskell). Last weekend I've attend CamHac. That was a wonderful experience to know new people, and also to collect suggestions and improvements for the plug-in. Two of those suggestions have already been implemented.
The first one is a tighter integration with some of the web frameworks available in Haskell-land. In particular, you can create Snap and Yesod projects directly from the New Project wizard in Eclipse, provided the neccessary tools are installed. Also, I've linked some content-types to existing web editors, so you will get HTML editor for a Heist template, CSS editor for a Lucius template and so on.
The other feature that most of the people requested was autocompletion. I didn't implement that before because my knowledge of both Eclipse and Scion was lower, and didn't really know where to start. But after these 3 months I've been able to come to a solution to the problem, and I'm delighted to present the new revamped Haskell code assist for EclipseFP:

In the following days I plan to implement more of the suggestions in my TODO list :D

EclipseFP build error on Ubuntu Oneiric

UPDATE: the bug has been solved in Oneiric, so this PPA is no longer needed. I leave this here only for reference.

If you try to install EclipseFP on a Ubuntu Oneiric (the next release which will have the number 11.10) you may get the following error when scion-browser is built:

ghc: panic! (the 'impossible' happened)
 (GHC version 7.0.3 for x86_64-unknown-linux):
       loadArchive "/usr/lib/ghc-7.0.3/ghc-7.0.3/libHSghc-7.0.3.a": failed

This is due to GHC 7.0.3 being packed in Debian with a stripped libHSghc. I don't know well what that means, but makes impossible to use have ghc as a dependency while using Template Haskell (something scion-browser does). In Debian unstable this has been fixed upgrading to a new GHC version, but in Ubuntu it seems that we will have to wait until the bug is fixed.
In the meanwhile, I've created a PPA archive with the GHC version in Debian (7.0.4-4) and rebuilt the haskell-platform package and its dependencies. This means that you can get a working Haskell Platform with a working version of GHC following the instructions about installing.
The downside is that the PPA only contains the core Haskell packages and the rest of libghc-* packages in Ubuntu are not compatible with these version. So you will have to use cabal install to get any other package in a system using the PPA, even if the library has been packaged in Ubuntu.

SourceGraph on EclipseFP

Eclipse really makes some tasks very very easy. One of them is running a command through a popup menu. So, while I was looking at the list of development tools in the Haskell wiki, I read about SourceGraph, and decided to integrate it into EclipseFP. The idea was easy: to run SourceGraph on the corresponding Cabal file and show the output in the internal browser. Voilà!

Three new features

I haven't written here for almost a month. But that doesn't mean that I hadn't been busy implementing new features for EclipseFP. Today I would like to share them with you.

The first one comprises some refactoring helpers. Now, when in the Eclipse interface you rename or move some file, the program knows that it should change the module name, the references in other modules and the Cabal file if it is referenced. This makes much easier to change the module hierarchy.

The second one is the ability to run unit tests directly from the interface. Because of the changes the Cabal interface for test-suites is being into, only a way to run tests is allowed: you should use test-framework to build them. That was the most complete interface I found, and it provides detailed reports in JUnit-style, that can later be shown in the development environment. So, just specify your tests in the Cabal file, right-click in your project root, select Run As... and then Test-framework test.

The last (but not least) feature I implemented this month has been profiling integration. As easy as you run your executables, you can run them in profile mode (note: this needs -rtsopts to be added to the GHC options in the Cabal file and just runs the executable with +RTS -hT). When the process has finished, the .hp file is saved in your project ans shown graphically, as with hp2ps or hp2pretty, but with the ability to control interactively the degree of grouping.

That's all for today! In the last weeks of the Google Summer of Code I would like to show coverage output from HPC and add some primitive handling of imports in Haskell source files.

Cabal editor, HLint suggestions

The past week I've continued learning about Eclipse and integrating more Haskell tools in it. The final result is that now you can edit simple Cabal files graphically and run HLint on your code automatically. I'll add some screenshots of this:

The Overview page in the graphical Cabal editor has been renamed to Package and now contains almost all general information about a package that you can modify in a Cabal file. In particular, I've added support for a list of tested-with compilers, different build types and a tree to select if you want files from your project inside the data-files property, which you can later access programatically.

You can also configure your Haskell programs via the Executables page. From here, you can change the dependencies, the source directories and the module which will act as main. Extra options for GHC can also be provided from here.

Of course, the Dependencies part uses the information from the Haskell Browser to allow you to navigate through your list of installed packages and read the documentation for them.

The Library page is almost identical to the Executables one. The only important difference is that, instead of a Main list, you get a check-list of Exposed modules.

The main feature from Cabal which is not available here is the use of configuration flags. This is mostly due to the fact that I cannot come with a good user interface for that matter, and haven't found nothing in other similar projects.

The HLint integration is mostly invisible to the programmer until there is some suggestion, which will be shown in the editor and the Problems view as the rest of warnings from the compiler.

If you already have a Haskell project in EclipseFP and want to get this functionality, apart from installing the latest version of EclipseFP from GitHub, you need to add the HLint Builder to the list of builders that will be run automatically.

Last but not least, I spent one day making easier for the user to use the Haskell Browser. As with scion-server, now scion-browser comes bundled with the Eclipse plug-in, and is compiled and installed automatically.

Some videos on Haskell Browser

As I said in the previous post on installing EclipseFP with support for Haskell Browser, I've made three videos showing some of the features I'm most proud of :) I recommend to look at them at fullscreen and change the quality to 720p, because the font size is small and the messages I've written in the editor may be lost.

Installing EclipseFP with Haskell Browser support [very experimental]

In this last week I've been integrating the work I've done in scion-browser into EclipseFP. This means that now you can browse the definitions inside your locally installed packages inside the environment, and also ask to the fabulous Hoogle for information.
However, I don't feel confident to ask for merging my changes into the main branch of EclipseFP yet. That's why I'm posting instructions on how to get an Eclipse with the Haskell Browser so everybody can check by their own and send suggestions and bugs (both of them are very welcome).
So, here are the instructions (they seem very long but I've tried to go step by step so people not used to Eclipse could follow them without problems):

• Install Hoogle in you computer: just open a console and run "cabal install hoogle"
• Hoogle needs some data to work which can be retrieved by running "hoogle data"
• Get a copy of the scion-browser code by running "git clone git://github.com/serras/scion-class-browser.git"
• Build it: "cd scion-class-browser && cabal configure && cabal build"
• In the previous configure step Cabal may tell you that some prerequisites are not installed. If so, install them by running "cabal install <package-name>"
• Download Eclipse. Any version would do, although my recommendation is to get the one for C/C++ developers, because it's a bit smaller than the others
• Download EclipseFP 2011-06-19 archive
• Extract the Eclipse archive you downloaded and run "eclipse" inside the folder that will be created. That is all the installation needed by Eclipse :)
• Once you are in Eclipse (if you are new to it, the first time it will ask you for a workspace, which is the folder used to save configurations and projects), click Help > Install New Software...
• In the window that will appear, click on the Add... button next to the Work with textbox
• You will end into the Add Repository window. Click Archive... and select the EclipseFP 2011-06-19 archive you downloaded. Click OK afterwards
• If you see the message There are no categorized items, uncheck the Group items by category checkbox
• Check FP: Haskell support for Eclipse in the available items list and click Next
• After pressing Next a couple of times more and saying yes to install unsigned software, Eclipse will ask you to restart. Do it
• You will see the next window. Don't panic, it just tells you that we need to configure scion-browser

• After clicking OK, the Scion server (the one providing compilation services to EclipseFP) will start compiling. I recommend you to wait until it finishes

• Once it finished, click Window > Preferences... and navigate to Haskell > Scion, Cabal and Browser page

• For the Browser executable, select the scion-browser you compiled before, which will be in <folder where you cloned the repository>/dist/build/scion-browser/scion-browser
• Click the Open Perspective... button in the upper right corner, select Other... and in the Perspectives window, select Haskell or Haskell Browser
• The next window asking to rebuild the database will be shown. If it doesn't, just close Eclipse and open it again. Of course, answer Yes to the dialog, so the information for the Browser is created

And that's all. When the rebuilding process finishes, you can start using the Browser perspective or the Hoogle view. More information about the features in the next post.

A rapid screenshot

I'll do the usual summary of the week next Sunday, but I cannot resist to show what I just got into my screen. Finally scion-browser and Eclipse are working together and there is a first prototype of a Haskell Browser perspective. It still needs to show documentation comments and Hoogle integration, but it looks nice :)

Click here to see it bigger

Week #2

This week I've been working in several independent tasks for the EclipseFP Browser, both in the Haskell backend and Eclipse frontend.

In the backend, I've tried to speed up a bit the creation of the databases. After some time looking for solutions and trying to build my own one using parallel package, I found the amazing parallel-io, which completely suited my needs. Now the user may use more than one thread for running Hackage and parsing the results back.

Although my initial thought was integrating the browser code directly inside Scion, it seems that library is in the middle of a process of rearchitecture. So my mentor suggested me to create a different executable for browsing tasks, and that was part of my work in the beginning of the week. Being used to the JSON libraries in the Java and Python world, aeson really alleviates a lot of work. The result is that now you can query an executable scion-browser for documentation about the installed packages in your system. The frontend is not very user-friendly, because it's oriented to be consumed by other tools such EclipseFP, but in any case you can consult the entire list of commands here.

The worst part this week has been my continuous fight against linking and Template Haskell-ing. For the console executable I started using editline, but when recompiling some file using Template Haskell, it made GHC crash. Fortunately, this is one of the times when Haskell has more than one library for the same task, and the problem was easily solvable using haskeline.

On the same line, when I was trying to integrate Hoogle, I got lots of problems about undefined references in Hoogle library. At first, they appeared in Template Haskell, so I decided to remove every use of that feature in my code. Thanks to the GHC option -ddump-splices, this was not as utterly boring at it seemed. However, when I finally got rid of that, the linker still showed me those kind of errors. It seems to be related to the fact that the hoogle package builds both a library and an executable and some object files are not correctly put in the archive. The solution for this problem was not using directly the library, but call the hoogle command for searches, and parse its output, which was easy as most of the parsing could be reused from the work in last week.

With all that, I felt it was time to start hacking in the Java side. Grasping all the concepts in Eclipse is taking some time, but I've been reading the great "Eclipse plug-ins" book and have more or less an idea on where to start hacking. Yesterday I used some time to create all the Java counterparts of module, package and declarations information, and the parsing and uparsing of JSON commands.

That's all. The next weeks I'll be working on the Browser in Eclipse. In the meantime, I'm also thinking about the best user interface for Cabal files editing, and I'm willing to hear any suggestion or comment in that topic :)

Request for comments

I didn't mention it on my presentation of the project, but being this a project that tries to enhace the tools available to Haskell developers, and ease the work for coding in that language, I would really like to hear suggestions or critiques from Haskell developers who have used EclipseFP or any other environment to make a better work.

I think the best place to write the suggestions would be in the comment section of this entry. Thank you very much in advance :D

Week #1

In my last post I just introduced a bit the project I will be working on this summer. Of all the features I proposed, I decided to start with the Browser, because it seemed the most orthogonal to the rest of EclipseFP, and a good way to learn about Scion and Eclipse APIs.

I started with a parser for Hoogle files written in Parsec. The information was saved using several custom data types for each kind of element in the database, and serialized to disk using binary. Here I already faced some problems, mostly due to laziness in the parsing, which created a very large memory footprint, so I decided to move to cereal to make the parsing.

After that, my mentor Thomas suggested some improvements for the code. One of it was to use the haskell-src-exts (since now, HSE) package to save information about elements. That turned to be a great idea, because HSE has special datatypes for annotated syntax elements and comes with parsers for some parts of the syntax I could reuse. In that way I could rewrite the parsing of Hoogle files, getting a neater code.

I was still having some problems with memory consumption, so I've been learning about the text and bytestring packages for better string handling. I also changed from automatic derivation of Data.Serialize instances using derive to manual writing of it, in order to implement a lookup tables for common names. That cut the memory footprint a lot, because Haskell type signatures tend to reuse names (such as "a" or "b" for type variables) and a lot was gained by not creating a new instance of Type each time one appeared.

That's all for this week. Until now, I've been making the changes into a separate repository from Scion, but next week I plan to merge my work on it.

Presentation

This is my first post about my Summer of Code working on EclipseFP, adding new features so that Haskell users can have a comfortable working environment inside Eclipse. I've been using EclipseFP for more than a year now (mainly for my University assignments), and I would really like it to have some of the advanced tools that Eclipse gives for Java or C. However, "adding features to EclipseFP" is a very vague statement. So in my proposal and after some mails with my mentor Thomas, I decided to work in 3 more concrete areas.

The first one, and the one I'm currently working on, is to create a Browser similar to the Java one. At least for me, that would mean an increase in productivity, because most of my Haskell programming time is spent changing from the editor to a web browser with Hackage open. With this view, I could browse the list of functions and types in a package or module from inside the environment.

When I was looking for ways to approach this, I found Hoogle. And more importantly, Hoogle maintains a complete index of the documentation of the latest version of every package in Hackage. So I'm going to use that information to provide developers a way to browse, inside the environment, the entire world of Haskell. That also means that when providing some interface for adding dependencies to Cabal projects (see below), we can also show the list of available Hackage packages.

The second feature to implement is some way to run unit tests, and show the statistics about passing and failure, much alike to JUnit tests in Java. The interface is going to be indeed very close to JUnit's integration in Eclipse, but I still have some open problems on getting the list of tests to run. The best option now seems to use Cabal's Test Suite support, although I was not able to find the "connectors" for QuickCheck and HUnit in Hackage.

The third feature I have in mind is to enhace the visual editor for Cabal files. Currently, it allows to change author, name and some other package metadata, but does not handle dependencies, build options or targets. My idea is to create something similar to the Cabal editor of Leksah, but as I said before, enhaced with metainformation I could get from Hackage.

Apart from this, there are lots of places to work. For example, having a database of function names allows to create "automatic imports" as the Java editor does, or provide better autocompletion. Refactoring support would be a great addition to the IDE. But all of this is just to much for a single summer...