RobsHouse.net

How refreshing it is when the US Government does something so right that my chest swells with pride and my heart fills with hope! How exciting it is that Drupal and Acquia can play a role in bringing openness to government. I’m referring to the Open Government Directive (OGD), an instruction from the President of the United States to all federal agencies to drastically change the way that government talks to and shares information with the public.

The OGD is predicated on three principles: transparancy, participation, and collaboration. Here I quote from the President’s website:

Transparency promotes accountability by providing the public with information about what the Government is doing. Participation allows members of the public to contribute ideas and expertise so that their government can make policies with the benefit of information that is widely dispersed in society. Collaboration improves the effectiveness of Government by encouraging partnerships and cooperation within the Federal Government, across levels of government, and between the Government and private institutions.

On top of these principles is a clarification of the baseline position that government agencies are to adopt, in particluar regarding the Freedom of Information Act (FOIA). Government agencies are instructed to presume openess and disclosure.

These instructions also come with teeth. There are concrete steps to be taken, and deadlines. Hard deadlines that are coming up fast. Each agency was given 45 days to identify three sets of data that had never been released before, and to make them available on Data.gov. In the same time agencies are to appoint a highlevel senior official to oversee the “quality and objectivity of, and internal controls over, the Federal spending information publicly disseminated through such public venues as USAspending.gov or other similar websites.” Those are big changes in a short time.

Within 60 days, agencies are to go even further, and launch portals on their websites, http://www.[agency].gov/open, which will keep the public informed of all their activity and efforts pertaining to the OGD. This also pertains to information and policies regarding the FOIA. There are also 60 and 120 day deadlines for the communication of frameworks for making Open Government systemic to the very DNA of government agencies. Changes include a “longer-term comprehensive strategy for Federal spending transparency”.

One particular goal, to be met in the first 45 days, summarizes the attitude towards technology and public participation:

[provide] a forum to share best practices on innovative ideas to promote participation and collaboration, including how to experiment with new technologies, take advantage of the expertise and insight of people both inside and outside the Federal Government, and form high-impact collaborations with researchers, the private sector, and civil society.

This is essentially saying that the government wants to collaborate with the public on the difficult issues facing us, and that modern technologies and tools (such as Drupal’s social publishing and collaboration tools) should be used where appropriate if they further the goal of fostering collaboration.

All of this is like fresh air. I work with open source software specifically because I believe in the value of transparancy, participation, and collaboration. The Drupal project is a shining example of what humans can achieve when they work together in this way. I am also thrilled that Acquia is already deeply involved in helping government agencies realize these goals, and that the tool that many are looking to as an Open Government Directive enabler, is Drupal. Acquia has a new Government JumpStart program, a whitepaper on Social Publising for Government, and an exciting partnership with Carahsoft to guarantee that we can meet the urgent needs of agencies in the throes of change.

Read more about Acquia’s OGD offerings and partnerships. Dana Blankenhorn covered Acquia, Drupal and OGD on ZDNet. Dries and Kieran have both written about Drupal and OGD on Acquia.com.

On October 22, 2009, I gave a keynote presentation at a digital marketing conference in Brussels. After my speech I was interviewed by the organizers of the conference. Here is the video. Note that my grandmother is actually 88, and really is on Facebook.

Yesterday I gave a keynote presentation at the Digital Marketing First 09 trade show in Brussels, Belgium. Drupal was out in full force with four Belgian companies joining forces to make the conference a special Drupal-themed event. There were also a number of other companies present who are using Drupal.

To prepare for the event I made a micro-site that focuses on Drupal and interactive digital marketing (the theme of the conference). It features a directory of the companies that were present and some case studies about how Drupal is the ultimate integration platform for anyone who offers an online service or tool.

The DrupalVillage.be companies that were present:

• Krimson (Acquia partners)
• Connexion
• Coworks
• A Web Factory

Also present were:

• Emakina (Acquia partners)
• Siruna
• TheParkingLot.com (see their Drupal testimonial)

Special thanks to ICanLocalize.com for translating a portion of the micro-site content into Dutch and French.

My presentation was very well attended. The slides are below. The take away for me was that Drupal is a great tool for people doing digital marketing, and Drupal people should be attending marketing conferences (and vice versa).

For the last six months, Scott Reynolds has been keeping a big juicy secret. As the maintainer of the Apache Solr Views module, he knows just how cool the future of Drupal Search is going to be. His module, based on an idea and code from Thomas Seidl, lets you make custom searches against the Solr index the same way you currently make views against the MySQL database. Want to build a search that just includes videos and MP3s, and renders the results as a playlist? Or how about a search that is limited to the current user's images, displayed in a slideshow? How about a block that shows the latest results that contain the phrase "badgers are the new pony"? Well, even if you didn't want a block like that, with Views 3 and Apache Solr Views, you can have it.

Thomas Siedl's brilliant idea was that Views should be able to build "queries" against any data source, not just databases. Earl Miles agreed, and inagurated the Views 3 branch by commiting the patch by Thomas (with great help from Jeff Miccolis and others). With Views 3 I predict you'll be able to build Views using data from Flickr, or from RDF databases using SPARQL, or from the local file system, or from any other data source that has an API.

To test it all out I used the Acquia Drupal Stack to create a new site (I just love the stack's multisite functionality!). I then signed up for a trial Acquia Network subscription because I wanted to get my hands on 30 days of free Acquia Search (it's easier than setting up Solr myself). I then downloaded Views 3 and Apache Solr (DRUPAL-6--2, just for fun. DRUPAL-6--1 works, too). I had to get the Apache Solr Views module from CVS (Scott, make a devel release!). I put these in sites/all/modules so that they'd override the versions in the Acquia Drupal Stack.

The CVS command for getting Apache Solr Views

$ cvs -d:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib \
co -d apachesolr_views contributions/modules/apachesolr_views

I installed Apache Solr manually which means I also needed to get the SolrPhpClient library. Since I have Drush, and since Apache Solr DRUPAL-6--2 has Drush integration, I did it like this:

$ drush solr phpclient

I <3 Drush!

I then used FeedAPI to grab all sorts of content from Planet Drupal. I could have just as well used Drush and the Devel module to generate some content, but lorem ipsum gets mighty boring. Finally I used Drush to run cron and even did a search (from the command line!) to check that the content was in the index.

$ drush cron
$ # wait a few minutes for the search index to commit the changes...
$ drush solr search drupal
node/175 by admin (user/1)
title: Agile and Scrum Videos
 This is likely to become a pretty big collection of videos about Scrum and
other other Agile based managements processes. (Drupal 5, Drupal 6, Drupal 7,
Drupal Planet, Drupal Video) ...

node/1 by admin (user/1)
title: Welcome to your new Acquia Drupal website!
 If you are new to Drupal, follow these steps to set up your web site in
minutes: Step 1 ... , forums, polls, tags, comments, ratings, and more. Acquia
Drupal comes with many modules to power social publishing capabilities on your
site. Hundreds of additional Drupal 6.x compatible modules ...    

The good stuff

Now for the good stuff. When you make a new view in Views 3 you get asked what data source to use. Here you can see that I use the Apache Solr search index as a data source.

Then I added some fields. These are not the same fields that are available to node based views. They are specific to the underlying data source.

I also added a sort so that the results would be displayed according to the search score (keyword relevance).

In order to make this view seem like a "search" screen, it needs a search box, right? You get that by adding a search filter and exposing it. I could add more filters, too, like a filter to limit it to just one content type.

This shouldn't just be a copy of the normal search screen. The results should look different. To that end I told Views to render the results in a table.

Since we want this to be a page view it needs a path, and I went ahead and stuck it in the menu as well.

Finally, I want to be able to use Apache Solr's facet blocks along with the view. This is a three-step process.

1. First I have to enable the facet filters (I enabled the block for Tags - a taxonomy vocabulary I have) at q=admin/settings/apachesolr/enabled-filters
2. Then I have to actually place the blocks at q=admin/build/block
3. Finall, and this is important, I have to pass in an argument to the view to filter by the same "things" that I'm filtering for with the facets. In my example I have the taxonomy facet block enabled, and therefore need to pass in an argument for taxonomy term.

How does it taste?

It tastes great! Feast your eyes on this marvelous search screen.

The keyword search and the facet block interact seamlessly.

An interesting point to note is that there are no database queries used in retrieving the data or displaying it. No complex views query with lots of joins, and no node_load() calls for displaying the results. This method of querying Solr is just as efficient as using the normal Apache Solr search module.

To my mind, Views 3 and Apache Solr Views are the future of Solr search for Drupal. Even though they are both in heavy development, you can try them out and enjoy the great control you have over your search experience. There are many more handlers that need writing, too, so jump into the Apache Solr Views issue queue and help out. Since it all works with Acquia Drupal and Acquia Search, you can easily get up and running using an Acquia subscription. Enjoy!

The Acquia Drupal Stack installer for Mac OS X 10.6 (Snow Leopard) has been released, and it’s pretty cool (works on 10.5 Leopard, too, of course). I’m now using it as my primary development environment, replacing MAMP. In this article I show how you can configure the stack installer to use XDebug for step through debugging. I do so using Komodo IDE, but the steps I follow will help you with Eclipse or any other IDE that supports XDebug. I also install a Firefox plugin called XDebug Helper that makes the whole debugging experience smoothe and sweet.

Acquia Drupal: Commercially supported Drupal from NodeOne.se on Vimeo.

Some time ago I wrote an article that looks deeply at the Drupal path system and shows how easy it is for new developers to hook into a running Drupal system. I explore the idea that this openness and extensibility is a key factor in winning large numbers of developers to work on Drupal, and that this is one of the reasons the project is succeeding. The paper is now available at Acquia, beautifully formatted as a technical whitepaper. I’m very happy with this article and am excited to finally have it available. It’s well worth the short survey you’ll be asked to complete before you can download.

Download the complete whitepaper (Registration required)

In this article I will show you how you can write a tiny bit of code that will reveal new fields and facets for searching with the ApacheSolr module and Acquia Search. Using Acquia Drupal we’ll write an example module that takes the file type from CCK file and image fields and makes them into their own search fields. This results in us being able to filter our search results based on file type. This code fulfils the situation where you want, for example, to find a specific post that has a JPEG image, or all of the posts with PDFs that match a particular keyword.