Eduforge: eduforge - Criteriafor Selecting Learning Content Management Systems

TUTORIAL 1

CRITERIA FOR SELECTING OPEN SOURCE SOFTWARE FOR ENTERPRISE WIDE APPLICATIONS IN EDUCATION

To improve the tutorial, please add comments at the end of this page. To obtain additional information, please write to Tony Bailetti at bailetti@sce.carleton.ca

ACKNOWLEDGEMENT

Thanks to Femi Oyesanya (Nigeria). He made valuable comments to a previous version of this tutorial.

OBJECTIVE

Upon completion of this tutorial you will know about the criteria used to assess:

enterprise wide applications
open source software for enterprise wide applications

Among the most prevalent enterprise applications in the education industry are: learning content management systems, course management systems, bulletin boards, assessment tools, group collaboration tools, instant messengers, survey tools, helpdesks, library systems, digital content repositories, and personal productivity tools.

TARGET AUDIENCE

This tutorial will benefit technical and non-technical individuals in educational organizations who participate in the selection of enterprise wide applications.

EXCLUDED

This tutorial does not describe the details of the functional criteria typically used to assess enterprise wide applications.

RELEVANCE

To build local capability in open source, technical and non-technical personnel need to know how to evaluate open source software systems. The literature on software system selection describes functional and non-functional requirements typically used to evaluate proprietary systems. This tutorial identifies criteria unique to the assessment of open source systems.

CRITERIA FOR ASSESSING SOFTWARE THAT DELIVERS ENTERPRISE SOLUTIONS

The remainder of this tutorial is organized into two parts:

Criteria for assessing both open source and proprietary software
Criteria for assessing open source software

Criteria for assessing both open source and proprietary software

Typically, educational organizations assess enterprise wide applications based on the following criteria:

Reputation with trusted reference clients
Software performance and reliability as well as functional breadth and depth of the solution offered
Implementation of open standards
User services
Usability (user interface, documentation, and accessibility)
Complexity of installation and upgrade
Availability of third-party application and content add-ons
Total cost of ownership, including the cost of the efforts required to (i) align software application with the objectives of the various learners' programs and the local context, (ii) support the pedagogical models, languages and cultures most appropriate for the target users, and (iii) extend the software�s functionality.

Criteria for assessing open source software

In addition to the criteria identified above, a set of six criteria is used to assess open source software. The criteria are:

Fit between the open source project and the educational organization
Community and commercial support
Expected longevity of the open source project
Development effort
Relationship with the development team

1. Fit between the open source project and the organization adopting it

Does the core mission of the open source project fit the core mission of the organization assessing the software?

The core mission of the open source project affects, at the very least, the architecture of the software, project priorities and the path for software evolution. For example, the mission that drives ATutor is to allow system access to people with disabilities. For this purpose, ATutor developers made significant efforts to comply with the W3C WCAG 1.0 accessibility specifications at the AA+ level. They have also complied with other standards because this made sense for them to fulfill ATutor�s core mission. In contrast, the mission that drives Dokeos and Moodle is a particular learning philosophy, social constructionism. Access to the disabled is not part of Dokeo�s or Moodle�s core missions; learning via social constructionism is.

An organization for which accessibility for the disabled is core to its mission may prefer ATutor over Dokeos and Moodle. Similarly, an organization for which social constructionism is core may prefer Dokeos and Moodle over ATutor.

2. Community and commercial support

Does a community provide prompt and helpful answers to questions about the open source software? Does third party commercial support for the open source software exist?

To test whether or not an active and helpful community exists, post messages on the forums that support the open source software. Prompt and helpful replies to your posts may be evidence that a helpful community exists.

Third party commercial support from local firms and individuals should be available. If it is not available, a plan to develop the appropriate skills should be designed and implemented.

Organizations are advised not to adopt open source software that is not supported by an active and helpful community or commercial firms.

3. Expected longevity

How long will the open source project survive?

Educational organizations need to assess the likelihood that the open source project:

will continue to operate
may change its mission

Organizations should not adopt open source software if there are signs that the project that supports it may cease to exist or may drastically change its mission.

The adoption of open source software that is supported by a small group of people is highly risky.

4. Development effort

How is the development project structured? How large and well established is the project team? Is there clear evidence of ongoing effort to develop the open source software you are considering? How good is the quality of the development team�s test plans? How long has the open source project been around?

Open source development projects are structured in various ways. Project structure affects the characteristics of the end product as well as development time and path of software evolution. Consider the case of open source learning content management systems. We find the following project structures:

software is developed by a team structured around people from various universities all of which operate their own software systems and expect to adopt the new software, e.g. the SAKAI Project (http://sakaiproject.org/).
software is developed by a team at a university and it is used by the university and other universities, e.g. Claroline (http://www.claroline.net/), ILIAS (http://www.ilias.uni-koeln.de/ios/index-e.html), dot.LRN (http://www.dotlrn.org), and LON-CAPA (http://lon-capa.org).
software is developed by a team at a college and it is used by the college and other organizations, e.g. Interact (http://cce-interact.sourceforge.net/about.php), and Manhattan (http://manhattan.sourceforge.net/index.php?menu=1).
software is developed by a team associated with a university�s research center and it is used by organizations other than the university of which the center is a part, e.g. ATutor (http://atutor.ca).
software is developed by a team anchored around core developers who own a business that provides commercial services to software users, e.g. Dokeos (http://www.dokeos.com) and Moodle (http://moodle.com/).
software is developed by a custom software company specialized in e-business applications. E.g., LogiCampus (http://www.logicampus.com/index.php/welcome).

The organization assessing the open source software needs to decide which project structure best fit its needs.

5. Relationship

Can you influence the evolution of the open source software?

Organizations need to assess how well they can influence the evolution of the open source software. For example, it may be easier to influence the evolution of the open source software developed by a team anchored around a commercial enterprise than it is to influence the evolution of the open source software developed by a team anchored around a formal agreement across major universities.

REFERENCES

Blackboard (2004) Form S-1 Registration Statement Filed with the Securities and Exchange Commission on March 5, 2004.

Messerschmitt, David G. (2004) Back to the User. IEEE Software. January/February.

Metcalfe, Randolf (2004) Top Tips for Selecting Open Source Software. OSS Watch. http://www.oss-watch.ac.uk/resources/tips.xml. Accessed August 2, 2004.

PLEASE PROVIDE COMMENTS IN THIS SECTION

I have been working on an open source course management tool for a few months and have been experiencing some success with it. It is currently in use at a small liberal arts college and large university in California as well as the University of New Hampshire. I named it wordcircle and it can be found at http://www.wordcircle.org.