Visualizing OpenCourseware
I’ve been working on this off and on throughout the summer, but haven’t posted anything! It’s probably about time to document the things I’ve been doing.
Context
Earlier this year I performed a review of OpenCourseware courses from three different institutions: MIT, Carnegie Mellon, and The Open University. I evaluated three courses from each of the three institutions.
Process
As I reviewed the courses, I kept a log of features that I recognized as valuable. At the end of my review I codified the results by creating a master list of features, and then organizing the features into five categories:
- Site Structure and Features
- Coruse Structure and Features
- Types of Content
- Efficacy of Content
- Community of Learners
After creating the list, I went back through all nine courses and marked whether or not each feature existed in each course. I found that a binary answer wasn’t always satisfactory. In some cases, the feature existed, but wasn’t complete or fully developed. In this resulting chart, green checks represent “fully present”; black squares represent “partially developed.”
My next step was to assign a relative value for each feature. It was obvious during my review that different institutions valued different features. To serve my purposes, I needed to determine the value that my organization (Center for Distance Education) places on each item in the list. I collaborated with a colleague, Chris Lott, to place a value of 1, 2, or 3 on each item in the list (three being high).
After designating a value for each item on the master list, I entered everything into a spreadsheet and created a series of formulas to calculate relative weights for each of the five categories. As one product of my research, I wanted a way to visually compare the strengths of institutions as well as individual courses, using CDE’s perceived values as a metric. My first thought was to create a treemap (view the treemap attempt), but the data wasn’t as visually intuitive as I’d hoped in this format. After a bit of trial and error, I settled on Bubble Charts as a more effective type of display for this particular data. In the first chart below you’ll see a comparison between the three institutions. In the second chart, a more detailed view comparing all nine courses against the CDE ideal.
Note that the size of the bubble indicates the total number of features employed, while the colors represent the distribution within categories. It’s readily evident that Carnegie Mellon (OLI) and The Open University (OpenLearn) implement more of the features from the list than MIT. It’s also clear that The Open University has placed more emphasis on learning community features than the other two institutions.
Meta
Data Visualizations were created using IBM’s Many Eyes site. I stored some of my pdf documents on Issuu. (This site creates an interesting way to view and expand pdf documents.) For convenience in linking from this blog, the pie chart and bubble charts were uploaded to Flickr.
Fascinating stuff, Carol. This is a very effective way to visualize the results.
I don’ think that this affects your project in any way, but the question of who values what seems to loom pretty large in any sort of comparative evaluation of open courseware implementations. If I were looking for a source for open courseware or myself, I don’t think I’d place much value on whether or not the structure of the courseware allowed for an integrated learning community. I’d probably just want to jump in, learn what I need, and get out. However, an evaluation system that places a high value on integrated learning communities might steer me into a system that may not meet my needs. I wonder if there is some way to correct for this–perhaps by a questionnaire into which you enter your own values and have them weighted accordingly.
As I said, this isn’t an issue for your excellent work on this topic–just a question I had.
Speaking if Issuu–have you ever looked at Scribd? It’s a nice take on the need to publish and print documents while bypassing PDF. I’ve been using it to publish my syllabi. Embeddable docs is the way to go…
Comment by Skip Via — September 22, 2008 @ 3:24 pm
[...] those projects off to other organizations for regular production and operations. In her recent post Visualizing OpenCourseWare, Carol Gering shares some research into characteristics of OpenCourseWare materials from a couple [...]
Pingback by Why OpenCourseWare? : Ruminate — September 23, 2008 @ 6:27 pm
[...] those projects off to other organizations for regular production and operations. In her recent post Visualizing OpenCourseWare, Carol Gering shares some research into characteristics of OpenCourseWare materials from a couple [...]
Pingback by Chris Lott » Blog Archive » Why OpenCourseWare? — September 23, 2008 @ 6:36 pm
Skip, I remember looking at Scribd in the past, but it’s been awhile. I should revisit it…so many cool tools these days. I’m pondering your larger question about customizing the values on course evaluations. More soon.
Comment by carol — September 26, 2008 @ 2:59 pm
[...] Via made an interesting observation in response to my post on Visualizing OpenCourseware: …the question of who values what seems to loom pretty large in any sort of comparative [...]
Pingback by » DIY OpenCourseware Evaluation Carol Gering — September 26, 2008 @ 6:59 pm
[...] Published September 27, 2008 Uncategorized Tags: ocw Carol Gering in her blog described the process she followed to review and visualize OpenCourseware at three [...]
Pingback by Visualizing OpenCourseware « Open Education News — September 27, 2008 @ 3:22 am
[...] Carol Gering introduces a method for comparing OCW implementations (h/t Open Education News). [...]
Pingback by Comparison of OpenCourseWare Implementations on Five Attributes at OCW Blog — September 30, 2008 @ 1:31 pm
I
this diary, Good to see more people writing about this subject.
Comment by youssef mzaoui — May 30, 2010 @ 7:22 pm