Carol Gering

Skip Via made an interesting observation in response to my post on Visualizing OpenCourseware:

…the question of who values what seems to loom pretty large in any sort of comparative evaluation of open courseware implementations. If I were looking for a source for open courseware or myself, I don’t think I’d place much value on whether or not the structure of the courseware allowed for an integrated learning community. I’d probably just want to jump in, learn what I need, and get out.

Since the goal of my research project is to create a specific plan of action for CDE’s contribution to OpenCourseware, my calculations are based on the features that CDE deems valuable (as Skip noted). As a byproduct, however, it seems valuable to create artifacts that might be useful to others. I worked up a streamlined spreadsheet that allows users to identify their own values before rating an OpenCourseware course. See the preview here. The spreadsheet is completed in two steps:

1. Read down the column of features and enter a value of 1, 2, or 3 into the first yellow column to signify how important each feature is to you. (Three is high).
2. Locate an OpenCourseware course that you wish to review, and enter 0, 1, or 2 into each row of the second yellow column. Zero indicates that the feature does not exist in the course, 2 indicates that the feature is fully implemented, and 1 represents a partial implementation.

The spreadsheet will calculate the quality of the OCW course based on the criteria you established with your customized value system. If you’d like a copy of the spreadsheet, leave a request in the comment field, or send me an e-mail (carol.gering [at] uaf.edu).

Addressing the question of why CDE values the Community of Learners so highly… well, that’s a topic for another blog post altogether!

I’ve been working on this off and on throughout the summer, but haven’t posted anything! It’s probably about time to document the things I’ve been doing.

Context

Earlier this year I performed a review of OpenCourseware courses from three different institutions: MIT, Carnegie Mellon, and The Open University. I evaluated three courses from each of the three institutions.

Process

As I reviewed the courses, I kept a log of features that I recognized as valuable. At the end of my review I codified the results by creating a master list of features, and then organizing the features into five categories:

1. Site Structure and Features
2. Coruse Structure and Features
3. Types of Content
4. Efficacy of Content
5. Community of Learners

After creating the list, I went back through all nine courses and marked whether or not each feature existed in each course. I found that a binary answer wasn’t always satisfactory. In some cases, the feature existed, but wasn’t complete or fully developed. In this resulting chart, green checks represent “fully present”; black squares represent “partially developed.”

My next step was to assign a relative value for each feature. It was obvious during my review that different institutions valued different features. To serve my purposes, I needed to determine the value that my organization (Center for Distance Education) places on each item in the list. I collaborated with a colleague, Chris Lott, to place a value of 1, 2, or 3 on each item in the list (three being high).

After designating a value for each item on the master list, I entered everything into a spreadsheet and created a series of formulas to calculate relative weights for each of the five categories. As one product of my research, I wanted a way to visually compare the strengths of institutions as well as individual courses, using CDE’s perceived values as a metric. My first thought was to create a treemap (view the treemap attempt), but the data wasn’t as visually intuitive as I’d hoped in this format. After a bit of trial and error, I settled on Bubble Charts as a more effective type of display for this particular data. In the first chart below you’ll see a comparison between the three institutions. In the second chart, a more detailed view comparing all nine courses against the CDE ideal.

Note that the size of the bubble indicates the total number of features employed, while the colors represent the distribution within categories. It’s readily evident that Carnegie Mellon (OLI) and The Open University (OpenLearn) implement more of the features from the list than MIT. It’s also clear that The Open University has placed more emphasis on learning community features than the other two institutions.

Meta

Data Visualizations were created using IBM’s Many Eyes site. I stored some of my pdf documents on Issuu. (This site creates an interesting way to view and expand pdf documents.) For convenience in linking from this blog, the pie chart and bubble charts were uploaded to Flickr.

I’ve completed my review of nine open courseware courses (three from MIT, three from Carnegie Mellon, and three from the Open University). As expected, I found a great deal of variety—particularly between the different institutions. It seems clear that these schools have very diverse approaches to instruction. I’ve compiled a list of features that I found within the various courses and I’m in the process of coding each course on a master list of features. I would like to create some type of data visualization with the results, but haven’t yet formulated what it will look like. I’ll seek input from colleagues before I link the results here.

I’ve selected nine courses to review for my graduate studies project.  I mentioned in an earlier post that open courseware (OCW) initiatives can be broadly categorized into 3 groups:

1. Those that focus on publishing documents and artifacts from classroom courses (epitomized by MIT)
2. Those that focus on the cognitive science of interaction between students and content (epitomized by OLI at Carnegie Mellon)
3. Those that incorporate social interaction between students as a key element of course design (epitomized by OpenLearn at The Open University)

As outlined in my proposal, I’ve selected 3 courses from each of these groups. I briefly considered selecting the same course (or a rough approximation) in each of the 3 categories, but quickly realized that was impractical. As a compromise, I selected a beginning language course from each of the categories. I selected languages that I’ve never studied, and I selected three different languages (so that anything I learned in one ocw course wouldn’t influence my review of the other two courses). The additional courses were chosen either because they interested me personally, or because they have potential relevance for CDE course development. I looked at each of the courses deeply enough to verify that they contained a significant amount of content.   Here’s the list:

1. Chinese I—MIT
2. French—OLI
3. Spanish: Espacios publicos—OpenLearn
4. Physics of Atmospheres and Oceans—MIT
5. Logic & Proofs—OLI
6. Exploring data: Graphs and numerical summaries—OpenLearn
7. The Nature of Creativity—MIT (Linguistics and Philosophy)
8. Economics—OLI
9. Reading Visual Images—OpenLearn (Social Sciences)

I met with my committee yesterday and received approval on my proposal—now I’m officially beginning the project! As a short, one-sentence description of the study: I’m exploring the implementation of various open courseware projects, with an eye toward a “best fit” for CDE. The final product will be a recommendation document that outlines a practical way for CDE to launch an open courseware project, including software packages, overall design of the learning space, course features, and interaction spaces.I’m particularly interested in course designs that offer significant learning opportunities for individuals. Based on my initial literature review, existing open courseware projects can be divided into 3 major categories:

1. Those that focus on publishing documents and artifacts from classroom courses
2. Those that focus on the cognitive science of interaction between students and content
3. Those that incorporate social interaction between students as a key element of course design

I’m predisposed to believe the second and third types of open courseware are most useful for individuals and self-learners. I’m especially interested in finding a way to implement effective strategies for social interaction in a self-paced, stand-alone study environment.As I begin the project, I’ll post updates here—on Rhetorica—for anyone who’s interested in following my progress.