Return of the bespoke database

Return of the bespoke database


I've mentioned before the recent move among UK charities to become more "professional", which is often translated as "do what the corporates do" (particularly when it comes to IT). One reason for this is the dreaded bespoke friend-of-a-friend database. These "databases" (and I use the term loosely) are often written by a student, with tenuous links to the charity, looking for a final year project and usually in Microsoft Access and they are usually awful to maintain.

Most of the ones I have come across were written before the developer discovered the benefits of modular code or commenting and with little thought to future growth. That's not to say they are all bad, but in the end they don't grow with the charities' needs, and can often suffer badly when it comes to upgrade hardware or--worse--Microsoft Office. At some stage the charity will lose contact with the developer, or the developer will lose some of their belief in doing something like this for nothing.

Faced with this kind of scenario, it's little wonder that charities looking for new membership management databases will end up at the door of the "market leaders". They talk to other charities, attend charity IT events and listen to seminars run by the companies making the software and eventually enquire how much it will cost.

After enquiring about the price they will probably sit down and have a cup of sweet tea

At this point they will probably sit down and have a cup of sweet tea and mutter things like "I didn't know it cost so much". But they will probably go ahead and follow the crowd. They'll end up with a database that fits half their needs, forces them to fit the rest to the developer's needs and ties them into that product for a long time. But it will be better than what they had and their users will love it--for a time.

One size does not fit all

Don't get me wrong: there are proprietary products which will adequately fit the bill and, with some clever data manipulation, you won't have to dance to their tune that much; however, one size does not fit all here. My experience of charity database needs is that they are all different. Sure they all need to have some way of managing donor relationships, and there are some good proprietary products out there (hint to free software developers); however, beyond that the data manipulation needs of a charity may very well differ significantly from their nearest neighbours (or what the business world seems to want to call "competitors"). Some need to record helpline call statistics to give to funders, others may need to manage information that can be given out on those helplines, still others may need to manage relationships for "clients" who are not donors. Some may need to do all three at once or combinations.

CRM may fit some or all of those needs, but there's a reason I put quotes around the word "clients": for a charity the people they seek to serve are rarely seen as clients, and certainly not in the same sense as with a business. And with CRM again you may find that you have to fit your needs to its features. What would be ideal is something which is developed for your needs, has capacity to grow with you, can be altered by anyone you hire (or who volunteers for you) with the adequate skills and does not cost you more money just because you have taken on three more volunteers.

Enter the free bespoke database

Doesn't that sound like a bespoke database? Imagine a database written for you, for which you have the source code and the legal right to change it (or have it changed). Your data is stored in open standards, with long-term use in mind and the number of users is not a licencing but a hardware capacity issue. That's a proper bespoke database.

Instead of being written by a project-hungry student on the only "database tool" they've heard of, it's written by a software developer or software company utilising professional free software database tools, supplied under a free software licence. If you want a maintenance agreement, get one; if you want the developer to change something or write a new module, get them to do it. Does the developer lose out? No. You see, because this is free software, they are free to re-use the same code for other clients under the same terms. In fact, several charities could club together and the developer could be paid to develop the same core functions for each along with slightly different modules as they require. I'm kind of thinking on my feet here, so feel free to jump in and have a discussion on how this could (or couldn't) work.

If the charity loses touch with the developer or needs a quick fix, then the licence permits them to hire (or have volunteer) somebody with sufficient skills to make any changes that they need.

Rallying calls

So here's a rallying call to all the free software developers out there: start looking closer at the needs of the voluntary/non-profit sector, as they could be an excellent client-base. Here's another hint to the non-profits out there: don't just jump on the bandwagon and assume off-the-shelf will be best or is the only option. Working with a developer to produce a bespoke free licenced database may turn out to be the best investment you make. My final request to the charities is not really licence related but that they would--for their own sake--nominate somebody on their side who will "own" the database project. Don't just leave it to the developer, it's your data and it's your database: manage it properly and it will serve you well.

What tools?

After my first draft of this entry, the editor suggested some guidance on what kind of tools free software developers could use to attract VCS clients. Seems like a good idea (I guess that's what editors are for!). Bear in mind this is my opinion, but it is based upon 7 years experience in the sector, much of it writing databases.

The server side

Firstly the staffing levels of charities can fluctuate very quickly; whatever tools are used need to adapt to that without too much fuss. I would suggest a client-server setup would suit most charities, even the small ones. So having a MySQL or Postgres database server somewhere would be a good start. It may seem over the top for a five-user database, but the system will grow with them and--more importantly--won't cost more in licences as they do grow. Such a server could easily be remote hosted or installed on at their premises. Telling them you can reuse that "old" PII/600 as a database server for half a dozen clients will win points. We charity folk are often reluctant to throw stuff away.

The client side

The client-side will depend upon the desktop they are using; however, something that presents entry and search forms in a clear and easy to use manner would suit. For this reason it may be best to avoid something like OpenOffice.org's database: it looks a little unfriendly to the untrained eye. Writing your own database doftware is an option, and Python springs to mind here, particularly if you use a cross-platform widget set like WX. C or C++ will do equally well (again go for a cross-platform widget set) and you could even do it in Java--after all cross platform was what it was designed to do.

The web alternative

Any desktop client must be easy to deploy though. An executable they can copy or install themselves would be ideal. Anything requiring a visit from you would fast become a pain. As an alternative it could be developed as a web-based database. We use one which uses a Perl front-end and a MySQL backend. It's hosted on our Intranet, and thus only requires a web browser on the client side. Deployment therefore becomes a case of giving the user the URL and their login details. I have to say from a development side the web-based option is the most attractive to me--particularly now that AJAX can be utilised to give it that desktop-like instant feedback feel.

The main requirements

In any case, there will be some things it should include:

  • data entry forms;
  • decent cross-table search facilities that are simple to use;
  • data-mining type reporting features.

I once heard that a charity fundraiser will spend half the time trying to get funds and the other half justifying the way they have been used to the funder. Larger funders (like trusts or governments) can demand some in depth reports to ensure their money is being used as they wanted. The charity using this database is likely to want to report on numbers, patterns and trends across a series of date ranges. Ideally the reporting system should provide the user with a way to specify the criteria and then pull off the report. We're not talking Crystal Reports here, just a way for the user to say "show me all the helpline calls from single mothers broken down by where they live for this date range" on a regular basis.

Conclusion

So there you have it a short(ish) summary of the kind of thing that is needed. Some of this already exists in the proprietary world but it's a) expensive and b) not always an exact fit for purpose. Free software can enable charities to make their database and data work for them and enable developers to really make a difference with their products.

Category: 

Comments

volunteer's picture
Submitted by volunteer (not verified) on

I'm enjoying your posts on this topic, as it's becoming a pet subject of mine. I volunteer for a project that has one of those bespoke DBs on M$ and it's reaching its limit. "But MS works" is the cry when any change is suggested, never mind that the computers are falling apart and the client data is grabbable by anyone with a mind for mischief.

Ryan Cartwright's picture

Ah the old resistance to change cry that accompanies so much IT work. :o)

I've encountered my fair share of this and I find it useful to identify your "champion" early on. This will be the person who is giving least resistance to the project and--if you are lucky--can actually see the benefits now. Using this person as a single point of contact will help particualrly if they are in a slightly senior position (without it becoming a dictat of course). Also sell it on better features as well. One of our Access databases was prone to people editing when they thought they were using "find by form" - hence a lot of records would have spurious asterisks in them. The new database prvented that by splitting the search and edit functions. Yes the Access database could have been changed to do that but we had other reasons for changing as well. But for some users the new database was seen as an improvement for that reason alone.

Once you go through this a few times (and deliver on your promises of course) the resistance usually lowers for future projects. As an example I encountered some significant resistance to change when we rolled out the Intranet database I mention above. After the roll-out one of the more vocal users 'phoned to thank me that they were now able to do what previously took three weeks in fifteen minutes (yes the old DB was that bad). For the next project we both worked on I found to be them a considerable ally. I just wish they had sent me an e-mail and I could have had it framed :o)

Author information

Ryan Cartwright's picture

Biography

Ryan Cartwright heads up Equitas IT Solutions who offer fair, quality and free software based solutions to the voluntary and community (non-profit) and SME sectors in the UK. He is a long-term free software user, developer and advocate. You can find him on Twitter and Identi.ca.

Most forwarded

Interview with Dave Mohyla, of DTIDATA

Dave Mohyla is the president and founder of dtidata.com, a hard drive recovery facility based in Tampa, Florida.

TM: Where are you based? What does your company do?
DTI Data recovery is based in South Pasadena, Florida which is a suburb of Tampa. We have been here for over 10 years. We operate a bio-metrically secured class 100 clean room where we perform hard drive recovery on all types of hard disks, from laptop hard drives to multi drive RAID systems.

Anybody up to writing good directory software?

Since the very beginning, directories (of any kind) have had a very central role in the internet. (I have recently grown fond of Free Web Directory. Even Slashdot can be considered a directory: a collection of great news and invaluable user-generated comments. As far as software is concerned, doing a quick search on Google about software directories will return the free (as in freedom) software directories like Savannah, SourceForge, Freshmeat and so on, followed by shareware and freeware sites such as FileBuzz, PCWin Download Center and All Freeware (great if you're looking for shareware and freeware, but definitely less comprehensive than their free-as-in-freedom counterparts).

Interview with Mark Shuttleworth

Mark Shuttleworth is the founder of Thawte, the first Certification Authority to sell public SSL certificates. After selling Thawte to Verisign, Mark moved on to training as an astronaut in Russia and visiting space. Once he got back he founded Ubuntu, the leading GNU/Linux distribution. He agreed on releasing a quick interview to Free Software Magazine.

Is better education the key to finding better software?

I read David Jonathon's article Anybody Up To Writing Good Directory Software? the other day, which got me thinking about software directories in general. As David mentioned, many of the software directories one finds when doing a quick google search are free as in beer, not as in freedom. But what interests me is the software directories that already exist, providing a combination of both free as in beer software, and open source software. Sites such as Freeware Downloads and Shareware Download don't advertise themselves as providing free as in liberty software, but each of them have a good selection of open source software available... if you know where to look.

Most emailed

Free Open Document label templates

If you’ve ever spent hours at work doing mailings, cursed your printer for printing outside the lines on your labels, or moaned “There has got to be a better way to do this,” here’s the solution you’ve been looking for. Working smarter, not harder! Worldlabel.com, a manufacture of labels offers Open Office / Libre Office labels templates for downloading in ODF format which will save you time, effort, and (if you want) make really cool-looking labels

Creating a user-centric site in Drupal

A little while ago, while talking in the #drupal mailing list, I showed my latest creation to one of the core developers there. His reaction was "Wow, I am always surprised what people use Drupal for". His surprise is somehow justified: I did create a site for a bunch of entertainers in Perth, a company set to use Drupal to take over the world with Entertainers.Biz.

Update: since writing this article, I have updated the system so that the whole booking process happens online. I will update the article accordingly!

So, why, why do people and companies develop free software?

More and more people are discovering free software. Many people only do so after weeks, or even months, of using it. I wonder, for example, how many Firefox users actually know how free Firefox really is—many of them realise that you can get it for free, but find it hard to believe that anybody can modify it and even redistribute it legally.

When the discovery is made, the first instinct is to ask: why do they do it? Programming is hard work. Even though most (if not all) programmers are driven by their higher-than-normal IQs and their amazing passion for solving problems, it’s still hard to understand why so many of them would donate so much of their time to creating something that they can’t really show off to anybody but their colleagues or geek friends.

Sure, anybody can buy laptops, and just program. No need to get a full-on lab or spend thousands of dollars in equipment. But... is that the full story?

Fun articles

Santa Claus - the most successful open source project

It dawned on me the other day, as I was shopping for the dozens of gifts it seems I have to buy every December, that Santa Claus is the most successful open source project in history. (Bridget @ Illiterarty would agree with that). Santa Claus is essentially a marketing development that is embodied by everyone who stuffs a sock, gives a gift, hosts a dinner or wishes Merry Christmas over the holiday season.

Most emailed

Editorial

When I first started thinking about Free Software Magazine, I was feeling enthusiastic about the dream. I had Dave, Gianluca, and Alan willing to help me, I had established members of the free software community willing to help me out, I had writers volunteering their time and energy for free, and I had a generous offer from OpenHosting for servers, all before I'd proved myself. There was a sense of excitement in the air, and I thought maybe, just maybe, I could make this work.

Free Software Magazine uses Apollo project management software and CRM for its everyday activities!