The Big Move (to Drupal 7)

WOW! This is the first (real) post on my site, after three (three!) years of stop. A lot of things to do, personal up and downs. But here I’d like to talk about another serious reasons why I haven’t worked to this site for such a long time: I wanted to upgrade the version of Drupal, the CMS both this new and the old site were based on. Still at Drupal 4.7 version, the fact that they have come at 7 (and almost 8), started making me feel like I was walking over eggs (and you should feel the same in similar cyrcumstances).
I already expected this to be painful, so I’ve waited until I had some time and was inspired enough. And yes, first thing to say about is that Drupal has pros and cons and, among them, the upgrade process, despite significant improvement, is still a big thumb down. It took me several months of spare time to face it and below you find some account of this little adventure.

Disclaimer: this is going to be rather technical and if you want to embark on a similar experience, you’ll do it at your own risk. If you don’t even know what Drupal is (or a CMS), don’t bother reading on…

The less painful way: migration and not upgrade

In theory, it should be possible to do something like: upgrade the database, install new Drupal files, run upgrading scripts. Well, that’s just the theory, even the official documentation discourage such utopistic attempt. So, the realistic plan is actually to install a fresh Drupal 7 instance (a.k.a. ‘D7’), with an empty database, and then find a way to populate it with the old contents. Plus, review the site structure and reproduce what you had previosly.

Installing and customising D7

So, the installation of a new Drupal is initially pretty standard: you download the web package, you put it under Apache (which, of course, must have PHP enabled), you create a new MySQL database, you tell Drupal how to access such database and you’re ready to go in few minutes. Now the real job begins.

As in the past, even more than the past, you’ll find soon that Drupal is made of thousand of modules and you’ve to cherry-pick what you need. Nowadawys they have pre-packaged flavours, called distributions, however I could not find anything close to what I wanted, which was spread over many of such bundles. So, I ended up searching modules in the Drupal site and, honestly, this kind of search is very ugly. You can only type keywords, use double quotes for exact phrase searching and use the boolean AND (there’s OR, but it’s the default). Really too little. One thing I miss is annotations of modules scope through a vocabulary (i.e., synonim search and a decent classification taxonomy). User-provided 1-5 star ranking of modules, like you see it in app-stores or e-commerce sites, would be much useful too, given the amount of modules that are often available for doing th same thing, with variable quality and unclear descriptions. At the moment, the only metrics I can base my initial judgment on is the number of downloads.

To make an example, maybe relying on such features would have saved me several hours that I I wasted with the OpenLayers module. I just wanted something to place posts on a Google Map box and then show a map of places my site talks about. OpenLayers goes far beyond that: you can keep location-definition separated from visualisations and configure many aspects of both. Apparently that’s very promising, in practice you’re horribly overwhelmed by the complexity of that. Have a look if you don’t believe me.

Yet another feature I miss is the option to search for a new module straight from my Drupal instance and then tell what to install and having it down to my servers in few minutes, together with the required dependencies. In other words, the same thing you have with many distribution systems (Apt, RPM, Port, you name it). The good news about that is they’re already working on such idea.

Similar considerations apply to templates: if you have a rather precise idea of the template you’re looking for (e.g., styles, colours, boxes, etc), that will make your search even more frustrating than it already is. The fact is the official site just lists the themes, often neither descriptions, nor images are clear enough and the old nice Theme Garden  (a web site to try themes dynamically) has disappeared. In addition to setting it up again, it would greatly help to annotate themes with features like the ones I mentioned (styles, colours and alike, or just ‘similar to’). Of course this would be a great base to implement a ‘similar themes‘ feature. This shouldn’t be difficult to achieve by means of some crowdsourcing and would be a nice project for Drupalists.

Anyway, let’s go ahead. The layout you’re shown once D7 is set-up is pretty nice, especially the quick administration links shown to the admin user. The default theme is nice too, despite I miss so much the old Fancy theme (the author seems to have abandoned it). As you can see, I’ve replaced it with Fervens. Not bad, after all.

The Great Move (of Posts)

As I said, I’ve decided to copy data from the old database to the new one. There are a few options to do that. As it is reasonable, I’ve started from the Drupal docs and its modules. Alas, this confirmed me a sad thing about Drupal and many open source projects (fortunately, not all): the documentation is often shamely poor. Things are explained with a few paragraph tutorial and, after you’ve read it, too often you are left like: OK, this the standard case, what if I have this or that exception? And you eventually give up, after having crawled around for hours in search of a decent answer. It’s through this kind of pain that I’ve learned there are many Drupal modules advertising their ability to support the migration task, but most of them look either badly documented, or feature-poor. For instance, take this table: if I have to rely on it, I conclude that only the module in the first column is good for me (the only one available for D7), but when I follow its link and I try to get a clue of how the hell it works, well, this is the only semblance of documentation I could find. Quite Frustrating.
 

How I came to DIY migration

More on the topic of migration, in my opinion, the only significant exception to the above is Migrate. Powerful, flexible and well-documented. But after having read through I was like: so, I have to code every bit of mapping between the old database and the new one and there is nothing to start with? Conclusion: writing my own module for migration can be more fun, more reliable and, thiking of my profession, not necessarily a one-off effort.

So, I went for that. After all, ‘all’ I had to do is a simple procedure like: query all the nodes in the old DB (via SQL), for each record, create a Drupal node, populate it with data from the record, save. So, I’ve gone again throuhgh the documetation, again that experience of tutorial-like stuff, but fortunately there is fairly good API reference documentation, plus great posts (1, 2, 3) that explain both the basics and advanced features.

By the way, this is what I’ve eventually got. Of course is very dirty write-once-use-once code, but it’s a starting point if you’re interested in developing a similar procedure. Eventually, such code allowed me to import almost all details about old site posts, except a few things. For instance, it is virtually impossible to change a node’s last-update date to a desired value (i.e., the one read from the origin), the date coming from the system is stored instead. I hope this won’t hurt my four users too much.

And then structural items…

I was pretty happy when I finished to import the site posts, cause this is the biggest content type of a site like this. But I had some more work to do: top buttons, links on the left column, taxonomies and all that jazz. For these aspects, I’ve just gone manually, after all, it’s little stuff, which can be reproduced in a couple of hours (i.e., nights…). This is just to say that favouring manual mode to hacking new code can be sensible sometime (OK, you’re not an extreme geek and this is obvious for you…).

While doing such a job, I’ve made a shocking discovering about taxonomies, one of the greatest Drupal features. Details are explained here. In short, it’s no longer possible to do a search based on, let’s say, ‘Computer Science’, and see that results annotated with sub-terms like ‘Bioinformatics’ or ‘Semantic Web’ are included. My workaround for this problem has been views: you create a view where ‘annotated with Semantic Web or Bioinformatics’ is included as filter criterion. Ugly solution, very inflexible and, moreover, as I’ve noticed later on, there is a special, parameterised view, which receives a root term ID + desired depth level and then does what I wanted, it searches a whole taxonomy branch. Drupal views are a really powerful concept. Great, but even more outrageous that they didn’t clearly mention (i.e., with bold big fonts) that this type of view exists.

Another nice thing I’ve added, before coming to writing this post, is some notification mechanism. I wanted something to be promptly informed when someone posts a new comment. Apparently Drupal has plenty of modules for doing that, I’ve tried a couple, none worked for me. I’ve resorted to creating an RSS view that reports the unpublished comments, then I passed it to the brilliant FeedMyInbox service, which now sends me mails about new comments. Smart, isn’t it? 🙂 Ah! You want to be the only one who is accessing this. My simple way to do that was: name the view with some obfuscation code (e.g., comment_7647bhcjhfyt5rd.xml) and tell search engines (via robots.txt) to ignore it. Yes, just that and works!

Would I recommend Drupal?

It has been quite a long work and quite a struggle. I have to admit that, despite having used Drupal for years, when I finished I started wondering if it’s worthwhile to stick with it, rather than looking for something else. There are plenty of blog posts out there on the topic ‘Drupal sucks‘ and as many that defense it. My favourite list of problems includes: upgrade pain (did I already say that?!), poor documentation (and that?!), complexity, lack of a system to rank modules, not using object-oriented PHP, PHP=load-all-each-time=performance concerns.

So, why do I still use Drupal? One reason is that Drupal-to-Drupal migration, no matter how difficult it has been, probably is still less burdening than Drupal-to-Anything-Else. Another is that Drupal has an incredible number of features and modules, which you can cherry-pick when you need this or that. On the other hand, if you just want to write something on the web, you should just open an account on one of the many blog sites out there. I haven’t gone this route yet, cause I want to do a bit more, like learning and keeping myself updated with a CMS that can be very versatile, powerful and useful in many web projects. Which implies experimenting, trying new stuff etc. The environment Drupal is based on (Apache, PHP, MySQL) makes such experimentalistic inclination relative quick to realise. It is also nice that there is a broad community to get help from (and to which you can contribute back).

If you happen to work in the IT field and you happen to need to design and setup non trivial web sites, then Drupal may be right for you, especially if such sites are mostly based on posts, news items or similar elements. Of course, there are alternatives and there are use cases where a CMS like this doesn’t suit very well (although I happen to see surprsing unusual applications). For instance, despite Drupal has some support for Semantic Web technologies, I don’t think you can make something like the DC-THERA Directory with it.

In conclusion, I would still recommend it to those who really need it and have enough guts to deal with it. Now, please enjoy my new site!