The field of analytics in higher education is relatively new and descriptions are often imprecise. Different types of analytics, with little in common, are regularly lumped together. At the 1st International Conference on Learning Analytics and Knowledge in 2011, analytics was defined broadly as the "measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs." Analytics comes in many shapes and sizes, depending on: End-users Type of data required Sources of the data captured Purpose or intended goal We propose three categories of analytics in online higher education, each concerned with student performance: Institutional Analytics Engagement Analytics Learning Analytics Identifying and defining the different categories is an important first step in helping educational professionals more quickly focus on the analytics that are of most value to their institutions. Institutional Analytics Institutional analytics is primarily concerned with tracking learners through their educational lifecycle, from enrollment to graduation. The data collected focuses on information such as student profiles (age, address, ethnicity), course selections, pace of program completion, use of support services and graduation rates. Data needed for institutional analytics is often readily available in colleges and universities within the institution’s registration system. The task, then, is to organize the data and identify the metrics that are most important to the institution. The value of the information can be multiplied by linking this data to other systems, such as enrollment and learning management systems, student support applications, and customer relationship management software. Analytics of this type is commonly the purview of the institution’s internal research and business analysts. Much of the data required can be found within the institutions’ registration system, but the value of the information can be multiplied by linking this data to other systems, such as enrollment and learning management systems, student support applications and customer relationship management software. Analytics of this type is commonly the purview of the institution’s internal research and business analysts. Institutions use this information in a variety of ways to: Align recruiting tactics with institutional aid Identify better recruiting practices to improve retention and completion rates Predict high-risk students earlier in order to provide more targeted support External demand for this kind of information is growing as state and regulatory bodies seek to better monitor (and reward) certain types of institutional performance. Similarly, more institutions are distributing information to help students and parents make more informed decisions about programs and schools. Examples of software used to support institutional analytics includes: iDashboards SAP SAS Engagement Analytics Unlike institutional analytics, engagement analytics track student activity within the course environment, which is typically the learning management system (LMS). The information generated can be of value to the institution, students and instructors. But most of the information is designed with the instructor in mind, keeping with the overarching instructional model of higher education. The type information collected in engagement analytics often includes: Number of page views (per page) Contributions by students to discussion threads Which students (and what percentage of the total cohort) have completed the assignments Number of logins This information, as with other types of analytics, is presented in a visual format, often as a dashboard, with its roots in business intelligence software. A well-constructed visual display of data makes interpreting course activity faster and simpler. When used effectively, this information can help instructors and institutions identify students who may need additional support and encouragement, and help determine the most effective type of student support intervention. However, engagement analytics do not necessarily measure learning, per se. What’s measured is student activity, which may or may not signal actual learning. For example, engagement analytics is often used to track student page views. The student’s presence on that particular page within the course site tells us that the student has been exposed to that part of the curriculum. But it doesn’t tell us whether the student understands the curriculum. In fact, it may be that the student inadvertently left the browser window open while searching the Internet. Writer and researcher Stephen Downes, who specializes in online learning, describes the challenge of using engagement analytics this way: "There are different tools for measuring learning engagement, and most of them are quantificational. The obvious ones [measure] page access, time-on-task, successful submission of question results - things like that. Those are suitable for a basic level of assessment. You can tell whether students are actually doing something. That’s important in certain circumstances. But to think that constitutes analytics in any meaningful sense would be a gross oversimplification."[1] Examples of software used for engagement analytics includes Schoology Blackboard Analytics for Learn™ Star Fish Retention Solutions Ellucian Course Signals Learning Analytics Learning analytics measure the student’s actual learning state; what students know, what they don’t know, and why. We propose that the category of learning analytics be reserved for analytics that actually measure changes in a students’ knowledge and skill level, with respect to specific curriculum. The insights generated from true learning analytics support optimization of learning through information, recommendation and personalization. Learning analytics are actionable. Examples of the type of information that can be captured by learning analytics include: What aspects of the course did the student master? Which students are struggling, and with which concepts, topics and problems? What misconceptions about the curriculum are leading to poor performance? What topics require more attention or better presentation? The data for learning analytics is captured through frequent formative and summative assessments. Based on the data generated from a student’s interaction with these assessments, it is now possible, as a result of extensive research at Carnegie Mellon University and elsewhere, to derive strikingly accurate measurements of student knowledge and skills. Learning theory offers explanations of the mechanisms of learning (e.g., the power law of learning, cognitive load, rate of learning, learning decay, etc.) and these cognitive factors can be incorporated into learning analytic models to measure, predict and respond to student performance in the online course. The insights produced by learning analytics can be used to create dashboard-style reports of student performance or to modify a student’s experience in real-time, or both. Learning dashboards give learners, faculty and institutions a visual snapshot of each student’s performance, as it relates to specific learning objectives. The information can also be used in real-time to continuously adapt the instructional activities (e.g., level of difficulty) presented to the learner to match individual needs. Automated recommendations help create a personal path of learning for each student. Practice is personalized so that students receive the right amount of practice, targeted at the right level, for the right topics. Instructors and mentors receive dashboards and alerts to guide more timely and effective interactions and interventions. Instructional design teams use the data to measure efficacy of course materials, and continuously improve the learning experience, saving time and resources. Each of the three types of analytics offers value. And there is inevitably some overlap between the approaches. But learning analytics is the only approach upon which educators can confidently determine the actual state of a student’s learning. It provides a true foundation for new opportunities to improve and optimize learning. [1] Collaboration, Analytics, and the LMS: A Conversation with Stephen Downes. Retrieved February 6, 2014. http://campustechnology.com/newsletters/ctfocus/2010/10/collaboration_analytics_and-the-lms_a-conversation-with-stephen-downes.aspx By John Rinderle, Chief Architect & Dr. Keith Hampson, Managing Director, Client Innovation at Acrobatiq.

Erik Moody, Assistant Professor of Psychology at Marist College, says "I know high-quality online courses with analytics work because we do research on it."  In 2011, Moody was part of a research project that used predictive analytics to help instructors identify at-risk students and intervene early to promote student success. As a result, Marist received the 2013 Campus Technology Innovators Award in the teaching and learning category.   What prompted your first foray into online learning this semester? My interest was spawned from a grant-funded learning analytics research project, the Open Academic Analytics Initiative (OAAI), that we launched in 2011. The OAAI ran a study involving instructors and student at four participating institutions (Marist, Cerritos College, College of the Redwoods, and North Carolina A&T State University). At-risk students were identified using data generated from course management systems and other demographic data. Some of these students were alerted about their at-risk status. Another group received alerts as well as access to additional intervention resources. The basic findings confirmed that even a simple intervention, if provided early, could help students improve their trajectory. The study highlighted important issues that we plan on investigating, such as the timing of an early alert and an observed increase in withdrawal rates. An in-depth description of the study will soon be published in the Journal of Learning Analytics. How did the research project lead to the hybrid Introduction to Psychology course you’re teaching this semester? Intro to Psychology at Marist is a traditional sit-down course meeting twice a week for 75 minutes. I’m supplementing the textbook with Acrobatiq’s Introduction to Psychology for the assessments and analytics. I’m interested in using Acrobatiq’s courseware in a traditional classroom setting.  I believe that the learning experience will be enhanced by using the strategies implemented in the platform — such as numerous and frequent assessment with immediate feedback — in a tradition setting, which allows for personal contact and one-on-one interactions. In other words, you get the best of both worlds. In the future, we may move to a true hybrid, where the class time is cut in half freeing up more time for research, committees, advisement, etc. What’s the feedback so far on the online exercises and assessments? Students seem to like it. Some like the instant feedback on how they’ve done after the checkpoints. Some do very well. For some, it’s still so new. One of the benefits for students of a hybrid course is that it gives them an opportunity to learn in two different environments; it broadens the formats in which they are exposed. From a teaching perspective, what features do you find most beneficial? I like the ‘early and often’ assessments and the fact that students can get instant feedback on the exercises as they complete them, instead of waiting until the quiz is graded. Having an opportunity for them to learn from their mistakes, prior to the assessment, and learn quickly, is very advantageous. Also, as students are doing the assignments, there is a constant stream of emails from them asking me questions. This shows me they are engaged. It seems to have generated a lot more electronic communications outside of the classroom. For me, it’s an indicator that students are engaged and participating. How are you using The Learning Dashboard™? I’m concerned with reaching students early enough to make a difference.  I really like the technology and see great potential. I’m very impressed with the organization of the dashboard. The opportunity to get feedback on student performance early and frequently not only allows a student to identify and address a problem, the online format provides a rich source of data for learning analytics to provide feedback to students and instructors, efficiently and quickly.  It would be nice if the checkpoint outcomes were presented in a format that were easier to review in the classroom. Often struggling students don’t recognize that their efforts are not sufficient until it’s too late to recover from a poor or failing grade. Other students remain in denial until confronted by their instructor. A system that assists an instructor in identifying at-risk students early will speed and facilitate an intervention, which might allow a student to improve their performance resulting in better student outcomes. What would you say to instructors, who care about improving student outcomes, but are reluctant to change what they’ve always done? Everybody is interested, but there’s still a lot they don’t know about the technology. And it’s a change, not the status quo. As educators, we have a responsibility to stay current with different educational resources and approaches. I know high-quality online courses with analytics work because we do research on it.  Anything we can do to offer a great education and free up some time is valuable to tenured professors and adjuncts alike. References: Jayaprakash S., Moody E., Lauría E., Regan J., Baron J., (2014) "Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative", Journal of Learning Analytics (accepted, pending publication) Lauria, E., Moody, E., Jayaprakash, S., Jonnalagadda, N., & Baron, J. (2013). Open Academic  Analytics Initiative: Initial Research Findings. Proceedings of the 3rd international Conference on Learning Analytics and Knowledge. ACM New York, NY, USA, 150-154   doi:10.1145/2460296.2460325  For more on analytics, download Understanding Analytics, the latest paper in the Acrobatiq Insights series or view an excerpt from our recent webinar on the Essentials of Analytics in Online Higher Education.    

Learning management systems have been the dominant technology in online higher ed since the late 1990s. These systems are, by design, as instructionally agnostic as possible. That is, they are designed so as to not impose specific instructional strategies on the educators that adopt them. This fits with the prevailing division of responsibilities and occupational roles in traditional brick and mortar universities, in which course design is primarily a solo activity. The LMS was successful precisely because it didn’t disrupt the status quo. This design also ensured that the software could be used by a diverse set of users, thereby maximizing adoption across an institution - a benefit to both the institutional client and the vendor. Instructionally-agnostic software approach has its place. But over dependence on this approach has and will continue to seriously hamper our efforts to improve the quality and cost-effectiveness of online learning. By relying on instructionally-agnostic software, higher education limits itself to the "correspondence" model of distance education - in which we use software primarily as a cost-effective tool for distributing traditional education experiences - based on print and classroom conventions. We need to complement the agnostic model with software that purposely embodies the best instructional strategies. Instructionally intelligent software does what software does best: it augments and extends our capacity. It enables the educator to do what she can’t currently do given constraints of time, money or skill. Consider, for example, the benefits of providing students with frequent opportunities to apply knowledge and to receive immediate feedback. Building and facilitating courses with hundreds of opportunities for practice and providing feedback, immediate or otherwise, is beyond the capacity of most institutions to support. However, educational software can be designed to offer students frequent opportunities for formative assessments and meaningful feedback within a single course. The software captures and embodies our best understanding of what constitutes an effective learning experience, and puts this knowledge to use in a cost-effective way. When done well, it significantly extends our capacity as educators.

Question: What is Acrobatiq’s take on adaptivity in online courseware? Answer: We view adaptivity as just one application of a broader discipline - data-driven learning.   Critical aspects on our views of data-driven learning fall into three areas: What data do we have (and how do we design learning to collect that data)? How do we model that data? What do with the information coming from our models to optimize learning? What data do we have?  We focus primarily on collecting and modeling learning data, which we distinguish from other forms of data that one might work with in an educational setting Learning Analytics: what are students learning?  What level of mastery against the learning outcomes of the course have they achieved at any moment in time? Engagement Analytics:  what are students doing?  Are they logging in?  When?  What are they clicking?  How engaged do they appear based on their behavior? Institutional Analytics: based on high-level results data, demographic data, and models, what do we know about the effectiveness of our courses, curricula, student support services, etc.? More information on these types of data analytics: Read more: Analytics in Online Higher Education: Three Categories  Watch the webinar: Q&A Panel Webinar: Essentials of Analytics in Online Higher Education How do we model that data? Every student interaction with Acrobatiq activities is recorded, whether the activities are summative checkpoints or practice questions. This data informs our model about changes in learning tied to specific skills or knowledge components related to those activities. By using sophisticated statistical modeling in conjunction with sound principles of cognitive science, we can model their learning on learning objectives and even course-wide competencies.  This allows us both to identify weak points for students and to give us the ability to target specific misconceptions.  We think it is essential to have a precise and nuanced view of a student’s learning state relative to the desired outcomes if we are going to drive actions based on that data.   What do we DO with that data?  (adapt the learning environment amongst other things) Once we have a clear view of a student’s learning state relative to the desired course outcomes, we can leverage that data (often in real time) to improve learning outcomes and to create efficiencies in the teaching process.  Examples of the kinds of things we do with that data include: Learning dashboards for faculty and mentors Allowing teachers to see at a glance where to focus their efforts in the classroom or intervene with individuals or with small groups that need similar instruction Dashboards for learners Giving insight to students on their progress in the course allowing, them to optimize their time spent Recommendations for learners Based on those insights, providing guidance to learners on optimizing their experience by letting them know when it’s time to move on and where to focus their attention Adapt the environment to the learner Using carefully designed questions that synthesize a number of objectives, we present a small scenario to students.  A student who we believe has a good understanding of all of the component objectives may be presented with simple a couple of questions to validate their learning, before being encouraged to move on.  If, on the other hand, we are presenting to a student that has struggled in some component skills, we scaffold that student up front within the context of the scenario by presenting additional questions targeted to their learning needs.  This is different from scaffolding a problem simply based on the student giving a wrong answer.  We optimize the experience to the student need.

  To recruit Dr. Jennifer West, a researcher at Rice University, Duke University had to pay for an entire team of researchers that worked with the Professor, as well as much of their equipment. Dr. West told NPR, "They actually sent architects to Rice (University) who looked at our lab facilities there, then used that information to go back and design the facility that would work for us at Duke."[1] Imagine a university making the same effort to secure the best instructional experience for its online students. Imagine institutions feverishly seeking out the best available digital learning content; recruiting the best instructional designers and the best instructors; the ones that have been proven to improve learning outcomes.  Imagine if the Chief Academic Officer’s career security was dependent on her success in this regard. Given the centrality of teaching and learning to the institution, this shouldn’t sound absurd; the problem, of course, is that it does. [1] Lisa Chow. Duke: 60,000 a Year for College is Actually a Discount, NPR. February 21, 2014. . http://www.npr.org/blogs/money/2014/02/14/277015271/duke-60-000-a-year-for-college-is-actually-a-discount Retrieved May 12 2014.  

We have good reason to be excited about online higher education; it has the potential to reconfigure "how we do" higher education in ways that will improve both the quality and efficiency of learning. However, talk of online education’s current "transformative", "revolutionary" and (worst of all) "disruptive" impact on traditional higher education ignores the fact that one of the core components of online higher education has improved only imperceptibly during last two decades. Online instructional content - the instructional media and activities presented to students - remains largely stuck in the 1990s, barely scratching the potential that we once imagined. The problems with instructional content stem from how courses are created. In most non-profit colleges and universities, the responsibility for the design and development of instructional content continues to fall to under-resourced and typically ill-prepared individual faculty members. The service departments set up in most institutions to support online learning have not substantially changed this fundamental division of labor. Instructional design professionals in these departments - despite their skills - are forced into secondary roles, often pushed toward providing technical training ("How to Set Up Quizzes in Blackboard"), rather than actually working with course instructors to design instruction. The funds available for course development are severely limited to what can be reasonably generated by way of tuition revenue (minus direct expenses) over a few semesters. And incentive systems of traditional colleges and universities make it illogical for faculty to spend excessive time developing instructional content, even if they had the wide range of skills necessary for this kind of work. There are exceptions to this state of affairs, but too few. As a result, online instructional content is often hastily constructed, relies heavily on exposition, and is largely void of the applications that actually take advantage of the unique properties of digital technology. Most discouraging of all, course design is often not based on research about how students actually learn (i.e. the science of learning). The process is neither rigorous, nor ambitious. You’d be hard-pressed to find another sector or institution in which content is both so obviously important and yet given so little attention. (Ironic, given the primacy of evidence-based practice in all other corners of the academy.) A number of changes are required if we are to begin to truly take advantage of potential of instructional content in online higher education. Some of the more obvious, such as team-based design and faculty development have been discussed elsewhere. Below, I offer three additional pieces of the puzzle that will help us get to the next stage of instructional content. One. (Finally) take course design seriously. To recruit Dr. Jennifer West, a researcher at Rice University, Duke University had to pay for an entire team of researchers that worked with Dr. West, as well as much of their equipment. Dr. West told NPR, "They actually sent architects to Rice (University) who looked at our lab facilities there, then used that information to go back and design the facility that would work for us at Duke." Imagine a university making the same effort to secure the best instructional experience for its online students. Imagine institutions feverishly seeking out the best available digital learning experience; recruiting the best instructional designers and the best instructors; the ones that have been proven to improve learning outcomes. Imagine if the Chief Academic Officer’s career security was dependent on her success in this regard. Given the centrality of teaching and learning to the institution, this shouldn’t sound absurd; the problem, of course, is that it does. Two. In the digital world, "design" matters. "Design" here refers to graphic design, industrial design, user experience, and the like. Design has taken on a bigger role in Western societies as a means of mitigating the jarring effect of rapid change. Nowhere is this more obvious than in the realm of personal technology (e.g. apps, tablets, mobile devices). However, online higher education has managed to remain untouched by good design. Its absence in online higher education is a significant inhibitor of the quality of learning: when we move the locus of education from the classroom to the digital environment, we necessarily change the factors that determine the quality of the student’s experience. In the digital environment, design plays a far more important role than it does in the classroom. "Screens" (laptops, smartphones, tablets, etc.) are design-dependent. The quality of design in screen-based environments dramatically influences the end-user’s experience. Well-designed instructional content and interfaces are easier to use and thus, more efficient. But the value of great design goes further: it can, like a great educator, direct the students’ attention to what is most important, increase the amount of time that a user is willing to spend on a particular challenge (i.e. time on task), build on a user’s existing knowledge, provoke a positive emotional response (which can facilitate better learning), and make information memorable and "sticky". Three. We need to know what’s working. A greater commitment to measuring the effectiveness of learning may be the single greatest driver of innovation in higher education in 2014. Better information about learning outcomes in the hands of educators, regulatory bodies, university leadership, employers, and (especially) students will uncover what’s working and what isn’t. But it will also drive innovation by providing both the impetus for change and the data we need to validate new approaches. Despite the relative ease with which we can capture student activity in online education, most institutions have only begun to take advantage of this opportunity. As of 2014, most analytics in higher education - to the extent that it is used for instructional purposes at all - tends to be limited to "engagement" analytics, which tells us what pages, assignments, and so forth that the student has visited. It tells us that the student is "there or not" (i.e. engaged). What it doesn’t tell us is how well the student has mastered specific aspects of the curriculum - which is the realm of true learning analytics. With learning analytics, educators, institutions, and students have the information required to build and test new instructional approaches and to know when they’re working. Great instructional content and learning analytics are natural allies. When tied to instructional content, analytics provides the intelligence that can determine, for example, which content should be served up to students based on their past performance (adaptive), predict the kinds of support that students may need to master specific parts of the curriculum, and help faculty identify the most effective types of instructional content. If the goal of the institution is to help students learn, and the power we assign to these institutions to act as gatekeepers is justified, it is logical (if not a social imperative) that the institution become extremely good at measuring learning. It’s been almost two decades since we got started with online learning. Since then, we’ve made some significant progress. But for a variety of reasons, the level of innovation and quality of course design has fallen behind. It’s time we began to take fuller advantage of the opportunities for improved learning that it affords. Sources Lisa Chow. Duke: 60,000 a Year for College is Actually a Discount, NPR. February 21, 2014. . http://www.npr.org/blogs/money/2014/02/14/277015271/duke-60-000-a-year-for-college-is-actually-a-discount Retrieved May 12 2014. Learning analytics has a quietly subversive quality. Increased measurement of learning outcomes will challenge the existing hierarchy of institutions. Those institutions at the top of the pyramid have nowhere to go but down, and the traditional means by which we’ve measured quality in higher education has worked in their favour. See Lloyd Armstrong. Competitive Higher Education, Changing Higher Education. http://www.changinghighereducation.com/2006/03/competitive_hig.html Retrieved May 12 2014 :: Dr. Keith Hampson is Managing Director, Client Innovation at Acrobatiq, a Carnegie Mellon University venture born out of CMU’s long history in cognitive science, human-computer interaction, and software engineering. @Acrobatiq

Colgate University hosted an event last week. "Innovation + Disruption Symposium". The keynote was none other than Clay Christensen, the Godfather of Disruption Innovation. Following his talk, seven University Presidents of private universities fielded questions from the moderator. Normally, these events are dull. Dreadful, in fact. Few things can  more dampen the likelihood that someone will say something provocative or new than their holding the position of university president. It’s a job for the politically astute. So, at best, these events make for good background noise while you’re working. But a few moments stood out; here’s my favourite. Clay Christensen - having done his bit on disruptive innovation in higher education - was sitting in the audience listening to the Presidents field questions. He rose and noted, half-jokingly, that everyone on the panel disagreed with everything he had just said about the coming disruption in higher education. Laughs ensued. He then asked the panel to imagine that a second panel was  on stage with them. This second panel included the Founder of the Khan Academy, Paul Leblanc of Southern New Hampshire University, the President of Western Governors University, and others that are commonly believed to be leading the changes that are unfolding in higher education. Christensen asked the panel to consider what this second, imaginary panel might say that is different from what he had been hearing. The response? David Oxtoby, President of Pomona College fielded the question: "I actually don’t know any of the people you just mentioned, but . . . " You can’t make this stuff up. I don’t recall ever hearing or reading anything that so succinctly illustrates the existence of different worlds and perspectives within higher education. To fully appreciate the vastness of the space between these worlds, simply consider the context: In 2014 access to quality, low-cost higher education is considered a national imperative. Improving graduation rates are part of the Obama platform. Student debt recently surpassed credit card debt. Think tanks, research groups, philanthropic foundations, and government initiatives are looking for ways to leverage educational technology. Even the mainstream media - after having  long ignored higher education  - are talking about the need for greater innovation in higher education. Yet, the President of a university - having accepted an invitation to an event to talk about disruptive innovation in higher education - has not heard of many of the people behind these changes. Wow. Just wow. I’ve written before about the growing tendency of professionals in higher education to exist blissfully unaware of the goings-on of other parts of the sector. At the time, I was concerned I might have overstated the case.  I’m no longer concerned. Note: Quotes can be misleading and can be easily taken out of context. But my interpretation of the significance of the exchange was based on the President’s full response to the question — not the quote alone  — which strongly suggested to me, and a colleague with whom I conferred  — that the respondent wasn’t able to speak to the work that these "imaginary" panel members were doing. :: :: :: Dr. Keith Hampson is Managing Director, Client Innovation at Acrobatiq, a Carnegie Mellon University venture born out of CMU’s long history in cognitive science, human-computer interaction, and software engineering. @Acrobatiq

Our very own Chief Learning Scientist, Marsha Lovett, is included in a The Trib’s recent article on the Evolution of Higher Education by Debra Erdley. Marsha, who is also Carnegie Mellon’s Director of the Eberly Center for Teaching Excellence and Educational Innovation, and a Teaching Professor in the Department of Psychology, helped us create courses for Western Governors using decades of learning science from Carnegie Mellon University’s Open Learning Initiative (OLI). You can read the full article here. For more resources on learning science view Marsha’s recent webcast, Unlocking Learning through Cognitive Science.

Working with its ad agency, Leo Burnett, Heinz Ketchup changed the criteria of what constitutes good ketchup and, as a result, increased its market share. They defined the product’s relatively thick quality as the new ideal. The difficulty of getting the ketchup out of the bottle became a marker of quality. The criteria we use to evaluate different aspects of our world can have an immense impact. Different criteria encourage different behaviours and outcomes, whether we’re talking about something as banal as ketchup or more substantial matters such as how we evaluate political leaders (e.g. great at speeches, but no experience) or what constitutes great higher education. In most sectors, evaluations strike a balance between the issues of cost and quality. Both concepts can be interpreted in a number of ways, of course: cost can include the price paid, but also the "cost" of accessing the product or service. Quality can focus on the experience of using the product/service, but also the ways in which ownership of the product/service will position the user in social circles. Higher education is unique in this regard. The tendency in this sector is to focus almost exclusively on issues pertaining to quality and to downplay the relevance of cost. This is unfortunate. The primary focus on quality sounds ideal, but it serves to dramatically limit the range of options that higher education offers students. Seeking out innovations that offer new combinations of cost and quality will increase the diversity of solutions available. The objective should be to maximize value; to find new ways to improve the balance between quality and cost.  A wider range of options are needed at various price points. "Free, as in cost-free" Neglect of price is evident in criticism of MOOCs. This comment followed The Economist recent articles on online higher education (See "Creative Destruction" and "The Digital Degree"): "I’m a prof at a mid-sized Canadian university. Plenty of my students have told me that they’ve looked at the MOOC recorded lectures from MIT and Harvard, and they weren’t any better than what we offered." A direct comparison of traditional online higher education and MOOCs based solely on quality is insufficient for determining the significance of this new format and business model — because, as everyone knows, MOOCs are free (or close to free).  The dramatically greater scale (number of end-users) of the MOOC model provides economies which in turn allow for higher quality relative to cost (i.e. value). Consequently, it should be of no solace to our Canadian academic that his courses are as good. The fact that this isn’t obvious reflects the tendency to downplay the relevance of cost. It’s also naive. "She got into a good school . . . " The tendency to only think in terms of quality is also evident in the way in which we compare institutions. Students are told that Yale, Vassar and other selective institutions are "great institutions" that are better than state universities, which  in turn are better than community colleges, and so on. The fact that tuition levels between different institutions can vary by as much as 1500% is downplayed. While everyone understands the importance of relative price on an intellectual level, we continue to reinforce this odd evaluation scheme year after year. In any other sector of the economy, this logic would seem bizarre. This accepted logic and criteria helps to reinforce the tendency of institutions to move in unison toward a singular notion of excellence. Clayton Christensen, and others, have defined this as the Harvard DNA - the guiding North Star of higher education — that encourages universities to gravitate toward the selective research university model. We see this same logic in play when colleges seek to transform themselves into universities, and when mid-tier schools like George Washington University and NYU use dramatic increases in tuition fees to signal "excellence". (See Daniel Luzer’s excellent article, The Prestige Racket.) The narrow definition of value can also suppress innovations in higher education that offer excellent value at far lower prices. Straighterline pioneered a business model that allows students to pay low monthly fees for ACE-accredited courses as they begin their college and university careers. While the company has managed to succeed, their approach challenges orthodoxy in higher education that equates low-prices with bad education. But at these monthly tuition levels, the value is greater than what students would receive from most other education providers. A broader and more fluid evaluation scheme in higher education that considers value — not merely quality — will open the door to new instructional strategies, business models, and types of programs. :: :: Dr. Keith Hampson is Managing Director, Client Innovation at Acrobatiq, a Carnegie Mellon University venture born out of CMU’s long history in cognitive science, human-computer interaction, and software engineering. @Acrobatiq  

"Economies of scale are factors that cause the average cost of producing something to fall as the volume of its output increases." One of the common and more provocative scenarios for MOOCs involves using these courses at multiple institutions. In other words, build once (e.g. by Harvard), use often (multiple institutions). The benefit of this approach, at least potentially, is that by sharing content across multiple institutions we drive down the cost of offering online courses and, second, the relatively high volume of users makes it possible to increase the level of investment in each course. San Jose State University’s attempt to integrate the Udacity course serves as a high profile example. This isn’t a new idea. People have been considering how scale might fit into higher education since the 1990s. It wasn’t until MOOCs that we had a concrete and well-known example to serve as focal point for this discussion. Higher education has not put a great deal of effort into finding ways to reduce costs through scale. While other sectors seized the new economics of digital content storage and distribution, higher education has continued to produce and use digital instructional content locally; a digital cottage industry model, of sorts. For elite institutions or those aspiring to be elite, scale runs counter to one of the trappings of elite status: exclusivity. For others, the highly decentralized organization of the institutions makes institution-wide scale improbable. Another factor is concern amongst decision-makers about the impact of sharing courses on labour market value and autonomy. "Let’s not kid ourselves; administrators at the CSU are beginning a process of replacing faculty with cheap online education." San Jose State University faculty responding to the call to use Michael Sandel’s MOOC as part of their program. Just How Common Is It? Scale is directly tied to the degree to which curriculum is common or shared across institutions.  This component of scale hasn’t, to my knowledge, been sufficiently addressed. If a course is to "scale" there must significant commonalities across institutions of higher education of what they teach and how. It is implied in the Coursera/Udacity/edX business model — but surely never stated outright — that undergraduate curriculum in the US (and beyond) is sufficiently common. Commonality is the premise of the textbook industry, of course. Certain courses, particularly in the first and second year of programs,  can be satisfied by a single textbook across hundreds of institutions. US textbooks are used in other nations, in fact — only modified lightly to refer to local conditions. A highly provocative study by (what was then called) Coopers and Lybrand suggested that 25 individual courses constituted 80% of registrations. The concept of The Long Tail might provide a structured approach to understanding the degree to which courses are common and how this might impact costs. Chris Anderson, The Long Tail Chris Anderson’s "The Long Tail" (2004) contends that the Internet has fundamentally changed the economics of producing and distributing digital products. "Shelf space" on the Internet is virtually infinite and increasingly inexpensive. It’s now financially feasible for vendors to sell a much wider variety of digital products, particularly books, films, and music and other media. Marketing strategy is shifting from a dependence on a relatively limited number of "hits" or "blockbusters"  (e.g. Top 40 radio; New York Times bestseller lists) to serving niches. Anderson argues that consumers have historically purchased "hits" — not because they are indifferent to less popular fare, but because of a lack of choice. But the Internet is now removing the bottleneck between suppliers and consumers. And as search and distribution technologies improve, and costs continue to decrease, Anderson forecasts that the top sellers in a variety of markets will constitute a smaller share of total sales, and the number of different products available will increase dramatically (i.e. further flattening and lengthening of the distribution of sales). While the top-rated television shows often captured more than 50% of the audience in the 1960s, today they represent  only10% to 15% and declining. New business models like Netflix have only amplified the shift. The Endurance of "Hits" and Digital Content in Higher Education Several studies have suggested that Anderson’s original work overstated the degree to which people migrated away from a select number of "hits". Anita Elberse’s analysis, Should You Invest in the Long Tail?  (Harvard Business Review), suggests that the market for "blockbusters" remains largely safe from the onslaught of multiplying niche markets. Despite the changing economics of content authoring and distribution that Anderson describes, the bulk of sales are still found in the "head" and the "tail" is remarkably flat.  Greater variety is not always matched by demand. The degree to which Elberse’s argument invalidates the Long Tail theory is somewhat dependent on where we draw the line between the "head" and the "tail"; that is to say, what we think constitutes a "hit". What’s most useful about her work is that it reminds us that there are important forces at play in each sector that give shape to the distribution of sales (head and tail). The insights from Anderson, and the questions posed about these insights by Elberse, can help us understand and anticipate the factors that might the demand for a more diverse range of content in higher education? Do we have a preference or need for "hits" in higher education? As a starting point for addressing these questions, I offer three issues that may effect the length of the "tail" of content in digital higher ed: Quality (Re)assurance. Do we need assurances from others in the field about the quality of content, and from whom, exactly? There are conventions in place: In traditional textbook publishing, it is conventional to employ currently employed academics from well-known (preferably) institutions of higher education as authors. In OER, we find the use of simple rating systems, such as stars (one-to-five), to crowd-source evaluations. To what degree will the need for assurance from others limit the expansion of the "tail"? Consistent and Coherent Curriculum. To what extent must the content be consistent with the curriculum within the institution and other institutions? Although not to the same degree as K12, higher education is a "system" in which students progress, step-by-step. There are levels into which content must fit. When a student moves from first year to second year, or transfers from one school to another, there is an assumption (hope?) that the first year accounting course at University A is roughly equivalent to the same course at University B.  The Bologna Process is relevant here. Production Quality. How important is it to educators and students that the content that they use meet a minimum standard of production quality? That is to say, at what point does "home-made" content become a liability because it is either difficult to integrate into an LMS, "buggy" (in the case of content embedded in applications), or simply difficult to use for students and instructors? How far along the "tail" will content of sufficient quality be found? As the variety of content increases in the coming years, educators, institutions and publishers may want to pay attention to these and other issues to determine how they go about creating, acquiring and distributing content. Although it’s too early to be certain, my suspicion is that like the markets of music, film, and books, the demand for "hits" in digital edu content will remain surprisingly strong. :: :: Note: A number of people have written about the relationship between The Long Tail and education. You’ll recognize, though, that some of them use the concept of the Long Tail to analyze the diversification of students. That is, the tail gets longer as more people participate in higher education.  Instead, I use the concept to analyze the diversity of educational content. Although the former approach may be of great value, my focus on educational content is more in line with Anderson’s original use of the concept.   Note 2: The concept of "mass customization" overlaps with The Long Tail in important ways. For more on more customization, see "How Technology Can Drive the Next Wave of Mass Customization" (McKinsey) Dr. Keith Hampson is Managing Director, Client Innovation at Acrobatiq, a Carnegie Mellon University venture born out of CMU’s long history in cognitive science, human-computer interaction, and software engineering. @keithhampson & @Acrobatiq