Bill Cerbin is a Professor of Psychology and Director of the Center for Advancing Teaching and Learning  at the University of Wisconsin-La Crosse. He has been teaching undergraduate and graduate courses in general, developmental, educational, cognitive and learning psychology for 30 years.  In June, Bill is co-teaching a faculty seminar, Improving Learning by Design, in a blended format. He is also the author of Exploring How Students Learn, a website providing resources, research findings and theoretical perspectives to help teachers better understand how students learn and inform their teaching. How did you first get into teaching online?   My first experience teaching online was years ago, using email to collect student writing assignments. This predates course management systems. A day before each class period, students emailed responses to several questions related to the reading assignment for that day. Their responses revealed their prior knowledge and misconceptions of the topic, which helped me decide what to do in class the next day - what concepts to highlight, what kind of practice might help to develop an idea. I still use this technique but with newer technology. Fast forward a few years. I currently direct the teaching center on my campus and we offer a three-week intensive online instructor training course. To maintain street credibility among faculty, I took the course in 2009. It was offered online, so participants essentially experienced what it’s like to be a student in an online course. The experience reaffirmed that good teaching and learning, regardless of the format, depend upon careful design. From your work as a Carnegie Scholar with the Carnegie Academy for the Scholarship of Teaching and Learning, are there a few takeaways you can share with instructors? My experience with Carnegie underscored the importance of systematic inquiry into teaching and learning, regardless of the format or delivery mode of a course. The mode, format and context of teaching are always important, but so are differences in the level of students, the discipline, and intended learning outcomes. Systematic inquiry is a tool instructors can use to better understand how and why students learn from instruction, whether that takes place online or face-to-face. I can offer three other Carnegie-inspired ideas that are important for online teaching. 1. Focus first and foremost on learning. Unless you know clearly what you want students to know or to be able to do, you have little basis for deciding how to teach them.     2. Read what others have done before you. In our disciplinary research, we habitually study the field thoroughly and build on previous work. Too often in teaching we try an innovation without any empirical or theoretical rationale. Consequently, we miss the opportunity to learn from our predecessors, and we may produce something that has already been done and does little to advance teaching and learning.   3. Never underestimate the complexity of teaching and learning. There is a fair amount of simplistic thinking among pundits, who advocate for certain pedagogies or best practices in higher education. In their view, all you have to do is adopt the best practice and students will learn. But teaching and learning are inherently problematic.  Two former Carnegie colleagues, Steve Chew at Samford University, and Randy Bass at Georgetown, capture this idea in slightly different ways.  Both perspectives emphasize uncertainty as the norm, and that we need to carefully investigate teaching to determine not just whether it works, but how it supports or does not support learning.  Steve talks about teaching as an ill-structured problem, in which we always face uncertainties about students’ learning. There are multiple factors, such as students’ background knowledge of the subject, their skill and approach to learning, and their motivation that make the outcome of teaching uncertain.  Randy suggests that every time we teach a course it is an experiment. The syllabus is the hypothesis. What are some of the biggest challenges you’ve encountered since you began teaching online?  Technology overload and the conceptual transition from teaching face-to-face to an online format. To the annoyance of my colleagues, I have started using the phrase, Is technology working for us or are we working for technology? Course management systems are still clunky. Updating software, fiddling with glitches, and trying workarounds are time-consuming distractions. Some instructors initially imagine that an online course will be identical to their face-to-face class, with the same structure, rhythm, sequence, and assignments. For instance, an instructor whose customary mode of teaching is to lecture, may simply record the lectures for an online class, and use the same assignments and exams. However, teaching online can and should involve rethinking one’s assumptions and reasons for one’s instructional practices. Why lecture? What should a lecture consist of? What’s its purpose? How long should it be? Should it be students’ first exposure to the material or come later after they have read or engaged in an exercise? What are your creative solutions to those challenges? Perhaps teachers would be better served by attempting small-stakes innovations before adopting large-scale changes. For example, if you like trying out different technological tools, play with them and explore their features before adopting them for a class. More importantly, start with a need or problem you want to solve and then look for a technology that can help you. Teaching online is significantly different than traditional instruction. I recommend that instructors participate in a rigorous course in how to teach online, especially one that is well grounded in instructional design principles. Part of that experience should involve designing a unit or module for a course that can serve as a prototype when it comes time to fully plan and design an online course. A good training experience not only exposes instructors to various techniques, but also addresses underlying assumptions and beliefs about instructional practices and how they support learning. What are some of the authoritative sources of content you seek out (other than textbooks)? I look for: Authors in my field, who publish articles on topics suitable for an undergraduate course. Journals and periodicals that publish articles accessible to undergraduate students. Video and multimedia material from certain sources, e.g., TED Talks, specialized videos by researchers, and peer reviewed learning objects from sources like MERLOT. What would you say to instructors, who care about improving student outcomes, and are looking for a better alternative, but are reluctant to change what they’ve always done? I think most good teachers work to improve their practice. Having tried a lot of major changes over the years, I gradually discovered that it makes more sense to think small—identify an aspect of student learning to focus on rather than adopt an alternative pedagogy wholesale. Change sometimes is disruptive, does not work well initially and may arouse student resistance. However, by focusing on a specific learning problem, one reduces the workload and the risk if change does not work as anticipated. I would strongly advocate approaching the problem as a form of classroom inquiry or research. This will enable you to describe what you did and why, and most importantly, explain how or why students learned or did not learn from the experience. Do you have any suggestions or tips to help online instructors teach more effectively? As an experienced novice, I am still learning and thinking about ways to teach in online and blended formats. I would advocate approaching teaching from a perspective of cognitive empathy. Try to put yourself in the minds of students, who are unfamiliar with both the course structure and the content. How would you like to see online learning evolve? What improvements would you like to see? I am very interested in courseware that can scaffold student learning of foundational knowledge. We know a lot about the benefits of certain learning strategies, such as practice testing and distributed practice. We know about the benefits of targeted feedback. There are examples of pedagogical approaches that incorporate these ideas effectively, e.g., Peer Instruction by Eric Mazur. I would like to see courseware that focuses on the most difficult concepts in particular disciplines, and provides intelligent tutoring and feedback that help students revise misconceptions and deepen their understanding of core concepts.

A few years ago, a major survey asked university leaders if their online programs were profitable - 45% of respondents said they didn’t know. This is both odd and predictable. Odd because online education is not the type of initiative in which cost and revenue are less important - quite the opposite. Institutions typically pursue online learning for "business-like" reasons, such as increasing access, market reach, campus capacity issues, and cost management. At the same time, it’s predictable because defining costs in higher education is a murky process; full of ambiguity. "Profit" is calculated with social responsibility in mind. Most units are budget-based, rather than "cost recovery" or profit centers. University managers are often one step removed from major financial discussions. Many positions don’t require advanced financial literacy. Another factor is that there is simply a remarkable shortage of information about costs in online higher education. Over the next two months we will prepare a set of blog posts, whitepapers and articles that address costs in online higher education. We recognize that this is not a small undertaking; the issue of costs in online education is surprisingly complex and can be approached from a number of angles. Issues pertaining to costs include: Productivity Economies of scale The role of new business models that offer new cost structures Institutional differentiation and specialization Class size and instructional value The role of courseware and its capacity to reduce faculty workload Intellectual property University consortia Freemium and MOOCs Sharing courses to reduce duplication  . . . and more. As a first step, we’re sharing a list of articles and reports that address one or more aspects of the cost issue. The collection includes reports, opinion pieces, articles and news. They come from very different perspectives, from inside and outside of the academy. Please share other resources you’ve come across in the comments section below. Resources: A Starter List How Online Learning Affects Productivity, Cost and Quality in Higher Education: An Environmental Scan and Review of the Literature. 2013. Massive Open Online Forces. The Rise of Online Instruction will Upend the Economics of Higher Education. Economist. February 6, 2014. New Players, Different Game: Understanding the Rise of For-Profit Colleges and Universities by William G. Tierney and Guilbert C. Hentschke. 2007. One Business School is Itself a Case Study in the Economics of Online Education. Goldie Blumenstyk. Chronicle of Higher Education. October 1 2012. MOOCs and Economic Reality. Clay Shirky. Chronicle of HIgher Education. July 8 2013. The Online Evolution: When Technology Meets Tradition in Higher Education. Andrew Norton for Gratton Institute. 2013. The Coming Higher Ed Revolution. Stuart Butler. National Affairs. Winter 2012.  The Scary Economics of Higher Education. William Baldwin. Forbes. January 15 2013.  How Universities are becoming more labour intensive. Alex Usher. Higher Education Strategy. January 7 2014.  UC Strives to Compete in an Era of Free Courses. Alisha Azevedo. October 1 2012. Technology and the Broken Higher Education Cost Model: Insights from the Delta Cost Project. Rita Kirshstein and Jane Wellman. September 5 2012. Twelve Inconvenient Truths About Costs in American Higher Education. Richard Vedder. Center for College Affordability and Productivity. March 2012.  Managing Online Education. Campus Computing Project, 2010.  

Subimal Chatterjee, professor of marketing and associate dean of Binghamton University’s School of Management, has been teaching for more than 20 years. This semester, Dr. Chatterjee is using Acrobatiq’s Statistical Reasoning in a hybrid format for his freshman statistics course. What prompted you to move to an online/hybrid approach to statistics?   We were always unhappy with the way the statistics class was run. It was rated low in satisfaction by both students and instructors. Statistics, like calculus, is a foundation course. If students don’t grasp it, their knowledge deficit will only increase into their junior and senior years. After trying a couple of different approaches (self-paced options), we got it right this semester, with the hybrid format. How is the course organized? All the content is online. We meet each Wednesday to flip the course. We spend classroom time doing the homework. When students come to class, they must have completed the checkpoints for the week. This forces them to complete the reading, and attempt the problems before we meet. What are some of elements of the Statistical Reasoning courseware you find particularly effective? The content is very good and students have no difficulty following along. The system lays out the framework for where students need to go.  The built-in assessments are excellent. Each Wednesday, I will know exactly what area, and what questions, students had problems with. This is so useful to me. I can go into class and say, ‘Out of 50 questions, these 5 questions gave most of you problems. I’m going to start with these 5 questions.’ You’re into the 8th week of the course. What’s been the student feedback? You live and learn. Once you give the students all the materials, they do not want to come to class.  Only 12 students (out of 34) show up every Wednesday.  As we get into the tougher chapters (sampling distribution, for example), I believe that this number will go up.  The feedback from the students who do turn up is overwhelmingly positive - they believe that they are learning.  How’s it going from your perspective? I think some more tweaking will have to be done. To what extent do I force all students to attend the Wednesday classes?  Should I insist on attendance?  Clearly a few students, who are not attending could certainly benefit.  I can see from their checkpoints that they are not grasping the materials. How will you measure success? We’ll gauge student satisfaction with a survey at the end of the course. The final exam is May 17. What do you think are some of the biggest challenges to overcome in moving to an online/hybrid format? Faculty culture. It’s so different than the way faculty are used to doing things. Change is the hardest thing to do. We’re not asking instructors to create their own course.  Just to consume some outstanding new course options to supplement their teaching and solve new problems. Can you share some teaching tips for instructors transitioning from a textbook to courseware? If you use the courseware properly, you don’t need a textbook. Everything is there. Your lectures, your narrative - you don’t need anything more. And it saves time. We only meet once a week, instead of twice, so the time is already reduced by 50 percent. There’s more time for research. If you’re using courseware for the first time, invest the time upfront to know all the modules by the first day of class. Because students are working at their own pace, you can find that the smarter ones are ahead of you. I also recommend analyzing your grade book once a week. Catch those students falling behind and call or email them. When I look at The Learning Dashboard and see that students are having difficulty, I create a short video lecture and push the link to them. You can save a lot of time knowing early on where a student is struggling. Now you have a daily snapshot that assesses students in a way that really helps you and helps individual students. Any other suggestions? For any instructor teaching statistics, I recommend Naked Statistics: Stripping the Dread from the Data by Charles Wheelan. This book is a fun introduction to why we study statistics. How would you like to see online learning evolve? What improvements would you like to see in the future? This is the future of education. The moment we talk about access to education, in five years, due to financial pressures and other developments, this is where we will be. What I sometimes fear is that by trying to broaden access, there might be a temptation to water down content.  That’s why it’s important to have content and assessments that adapt to different students. Not all people are motivated the same way. Or learn the same way. And of course, some people need an instructor to stand in front of them.            

The institution was already a leader in online education when Emma (not her real name) took on the new role at her university as the head of online learning. But the university’s new leadership wanted to take it to the next level and make online education a defining feature of the institution. She was the right person to lead the charge. A member of the faculty, she had recently spent several years as a high-level administrator, building a wide base of support. She was smart and personable. Her first step, like all good diplomats, was "consultation;" to sit down with as many people in the institution as possible. Between meetings, she plowed through the university’s online courses. A Rough Start She was surprised by what she learned during her review. The few well-designed courses were the exception. Not many presented a coherent set of learning outcomes. Assessments were unimaginative; mostly online versions of classroom assignments. Content regularly included slides from classroom lectures, which obviously added little value. When video was used, it was typically just unscripted webcam lectures, which were difficult for many students to follow. (One video apparently resembled an outtake from The Blair Witch Project). The quality of written content was uneven. Materials from publishers were plugged in randomly. Rich media was largely absent and many graphics appeared to be from copyright-protected sources. And despite the capacity of technology to capture and track learner performance, the use of learning analytics was limited. A Classroom Model Her experience was not unique among brick-and-mortar institutions in North America. The design and development of online higher education courses in traditional, non-profit institutions has followed the organizational, financial and distribution methods of traditional classroom education. Online education remains essentially a cottage industry, in which lone instructors are asked to be responsible for the bulk of online course design and development. Consequently, people with limited time, the wrong set of skills, and insufficient compensation build courses. And because the course materials are used for only one course at a single institution, investment is limited to what can be recouped through tuition from a single course delivered for a few semesters. A Way Forward As Emma saw it, in order to improve the quality of course design, faculty needed to spend less time trying to create their own courses from scratch and more time on high-level strategic matters - mapping the curriculum, course-related research, defining learning objectives, and providing meaningful support for learners. She believed a greater percentage of the course content should come from other sources. By relying more heavily on existing course materials, the cost of course development would decline and quality would rise. Her job was to make it easy for faculty to draw on "the best available" instructional resources - whether it came from other universities, consortia, vendors or open education resources (OER).  Faculty would continue to craft their own instructional content and activities, but only where they could produce material that was superior in value to what was available elsewhere. "My Stuff" Emma’s plans ran headlong into what many who work in online higher education learned long ago; there is often considerable pushback from some faculty on the idea of relying more heavily on instructional media and activities from outside sources. "They feel the need to be using their own stuff," as she put it to me. The "my stuff" logic is a by-product of two overlapping origins: Faculty are hired and promoted for being experts. It is this expertise that defines them as "more than teachers" of someone else’s curriculum, and justifies, in part, their relative prestige and autonomy.  The teaching function of the occupation model is thought to flow from the research function. And for this research work to be of value, it must be original. It follows, then, that what they teach should be original, as well. Hence, the pushback. It stems, secondly, from an understandable anxiety that arises from suggesting to faculty that they abdicate part of their current responsibilities to someone else. Many in academia have viewed the rise of educational technology as a threat to their labor market value. This was clearly expressed in the 1990s by critics like David Noble who argued that education technology was a "Trojan horse" that was designed to weaken and ultimately replace faculty labor. " . . .  faculty have much more in common with the historic plight of other skilled workers than they care to acknowledge. Like these others, their activity is being restructured, via the technology, in order to reduce their autonomy, independence, and control over their work and to place workplace knowledge and control as much as possible into the hands of the administration. As in other industries, the technology is being deployed by management primarily to discipline, deskill, and displace labor." (link) More recently, similar sentiments have been expressed about MOOCs. Gianpiero Petriglieri (great name, huh?), associate professor organizational behavior at INSEAD, described MOOCs as a tool of "academic colonialism," whereby more prestigious institutions become the source of curriculum for less prestigious institutions. "It is far more similar to colonialism, that is, disruption brought about by ‘the policy and practice of a power in extending control over weaker people or areas’ and simultaneously increasing its cultural reach and control of resources." (link) In response to the use of MOOCs at San Jose State University, philosophy professors wrote, "Let’s not kid ourselves; the administrators at the CSU are beginning a process of replacing faculty with cheap online education." (link) Of course, most professionals in higher education don’t hold these views. In technology, they see an opportunity to redefine their roles in ways that allow them to focus on higher-level activities, while providing students with the best possible experience. Faculty that have experience teaching online education are aware that course development is highly labor intensive, that compensation is insufficient, and that current practices significantly limits the type of instructional media that we can make available to students. They’ve learned first-hand that new models of instruction - including greater use of shared resources - doesn’t mean that faculty are made redundant; only that they can better focus their time and talents on those parts of the course in which they are most invested. Nevertheless, the state of affairs in online higher education has clung too long to its conventions. Online higher education can offer students much higher quality experiences if we seek out new approaches to course design and development. And in the process, slow down the rapid climb in operating costs and improve learning outcomes. But we need the political will to make it happen. In Part 2, we’ll look at the concept of the "best available;" how it relates to scale and costs in higher education, and what this new approach might entail. Emma, meanwhile, has gone back to the drawing board.              

Jeff Dykes is a science instructor teaching online biology and chemistry at Grays Harbor College and Wenatchee Valley College, in Washington State, and at Charter Oak State College. He has published numerous articles and won a Quality Matters (QM) award for his online chemistry course. How did you first get into teaching online courses? In 2000, I had a sense that teaching would be moving toward online classes and I didn’t want to miss out on this opportunity to be at the forefront of teaching online.  I contacted WashingtonOnline and entered their instructor course for online teaching.  I then sent out resumes and began teaching at several colleges, and developing courses and curriculum. What are some of the biggest challenges you’ve encountered since you began teaching online? In the beginning, I had very slow Internet connections and grading tests was a nightmare.  Just closing out one test and opening another took a long time.  Then multiply that by the number of students in a class - that translated into a great block of time each week just to grade tests.  Also, there’s little drag-and-drop technology in the learning platform.   I would rather see it like a computer desktop, where you can drag and drop files, wherever needed. What are some of the challenges of connecting with your students online? And how have you solved them? When I was in college I took a couple of independent study courses that were an early version of the online courses.  The problem was it was totally up to me when I would do the work, and that causes a lot of trouble.   Now, in my online classes, I try to follow the structure of an in-class course.   I split the course material into weekly sections that have firm assignments and due dates.  To connect with students I have done the typical discussion forum.  However, recently, I have asked students to search the Internet for videos that addresses a certain topic.  Students include the video in their posts and summarize the highlights.   This not only gets students interested in the post but also helps reinforce the current topic. What are you currently using for course content?  My online biology course is from Carnegie Mellon’s Open Learning Initiative. My online chemistry uses an open source chemistry book, and other links to content.  The rest of my online courses use paper textbooks. What are some good examples of peer-reviewed online resources for science instructors? MIT and UCLA Berkeley are two sources of excellent online resources.  They provide video lectures for anyone to view.  These institutions have a history of quality and respect.  Khan Academy has begun to release peer-reviewed material, which is an excellent resource for instructors to direct students to for tutorial help. Have you served as a mentor to other faculty teaching online? I’m a member of the Distance Learning Committee at Charter Oak State College, and review courses and offer tips on improving the class.  I’ve done presentations to faculty groups, and joined online learning groups.  At times, I get calls from other faculty to troubleshoot an online learning platform. Do you have any suggestions or tips to help instructors teach more effectively and connect with students in an online/hybrid environment?   Show up every day to an online class; at least 5 or 6 times a week.   This is an active process.  I make an effort to post an announcement or answer student posts in the discussion forum, so that they know I am present each day. Deal with student’s technical and personal issues, as if they were your own family.  I treat students with respect and give them a break where needed. What would you say to instructors, who care about improving student outcomes, but are reluctant to change? Start with a small change.  Then evaluate the outcome.  Continue to make these small changes, and before long, you have an improved class and student outcomes. How would you like to see online learning evolve? What improvements would you like to see?​ I’d like to see better recording software and interfaces.  I use Tegrity to record PowerPoint lectures; it’s good, but has problems.  I would love to see better and more efficient grading features, such as those found in the Canvas Discussion Forum. For example, when I want to grade a discussion forum or a lab report I can go to my Canvas grade book and select the first student I wish to grade.  The student’s report is brought up (with the ability to annotate or write comments).  I review their work and I enter the points earned.  I can add an additional grading note and then click on the student next in line and repeat the process.   I really like the ability to have the student document linked to the grading all in one screen.      

Every month we ask our Learning Engineers to answer a question or two about their work in developing online learning environments. Most questions come from instructors, who are looking for new ways to engage students in their online, hybrid or flipped courses.  Here Kim Henry and Erin Czerwinski address questions about course content and design. Kim Henry, Learning Engineer and Course Development Manager  How do you develop content for Acrobatiq courseware? Courseware content development is driven by two primary elements:  A set of student-centered, measurable learning objectives that articulate the essential concepts and skills students acquire from engaging with the courseware,  And The Big Picture that represents the subject domain and provides a knowledge structure in which students relate the new concepts and skills being learned. The language used in learning objectives should capture the level at which learning is occurring and at which feedback is relevant. Learning objectives are designed to:  Communicate our intentions clearly to students and to colleagues  Provide a framework for selecting and organizing course content  Guide in decisions about assessment and evaluation methods  Provide a framework for selecting appropriate teaching and learning activities  Give students information for directing their learning efforts and monitoring their own progress Based on A.H. Miller (1987), Course Design for University Lecturers. New York: Nichols Publishing. Also see, C.I. Davidson & S. A. Ambrose (1994), The New Professor’s Handbook: A Guide to Teaching and Research in Engineering and Sciences. Bolton, MA: Anker Publishing Company Inc. How do you determine how many learning objectives are needed in a course? A number of factors may influence the body of learning objectives for a particular course:  Standards set by a professional organization  Advances and/or trends in a field or domain  Role of the course in a sequence and/or curriculum  Expectations based on similar existing courses A small core team of learning engineers, subject matter experts (SMEs), course developers, and other stakeholders engage in a process of gathering all of the relevant information about a course and then synthesize it to determine:  High-level outcomes for the course  Key features of The Big Picture  Scope of the course, including major topics  Learning objectives for the course  Preliminary skills map for the course Once the scope of the course is identified and the learning objectives are written, an organization for the course can be formulated. The organization of Acrobatiq courses is based upon a unit/module structure where units are similar to chapters and modules similar to sections. Modules consist of logically clustered learning objectives. In general, the content development process is focused around the learning objectives and skills map. There are several primary "ingredients" that go into an Acrobatiq course. Formative and Summative ADAPTivities: These are activities that provide students with sufficient practice to support each learning objective’s skills and knowledge components. Formative ADAPTivities include "Learn By Doing" and "Did I Get This?" activities that provide students with opportunities to engage in active learning. Summative ADAPTivities are assessments that may occur at the end modules and/or units.   ADAPTivities are developed to capture and convey information, including "must cover" concepts and skills, as well as common misconceptions. Through various styles of interaction, ADAPTivities provide students with active learning experiences that include hints and targeted feedback. The primary purpose of exposition is to connect ADAPTivities. Generally, exposition is developed in one of two ways:  Exposition from existing content is reconstituted and combined with original exposition written by SMEs.  Original content written primarily by the SMEs on the development team. Erin Czerwinski, Learning Engineer How do you work with subject matter experts in developing course content? Historically, I have worked with subject matter experts (SMEs) to transform materials they developed in one format (paper or PowerPoint, for example) into new, dynamic content delivered through a learning platform. When working with SMEs, who are typically experienced teachers and authors in the subject domain, it’s helpful to be aware of something we call an "expert’s blind spot." SMEs know the subject so well they may not remember how they first struggled to understand and learn the concepts in their field. By asking these faculty experts a series of questions, we help them think from the students’ perspective. This enables us to identify students’ most common misconceptions, and where they struggle with the subject matter. All instructors who teach and design learning materials should be aware of their blind spots. By continually challenging your assumptions about the difficulty of the material you are teaching, you are in a much stronger position to develop course material that speaks directly to your students, and helps them learn faster and better.  What are some sources of instructional content available to all instructors? Wikimedia Commons and Creative Commons are a couple of useful resources. These organizations offer access to search services that will help you find content (images, music and videos) from a variety of sources, and often include licensing or permissions information. Another great source of content is your peers.  Many faculty post domain-specific content they use on the Web. With any content, it is important to check if the materials are copyrighted and what permissions may be required to use it. You can also try creating your own content. For certain media elements, start simple, with something like the animation features in PowerPoint.  Or experiment with screen recording software like Camtasia and others.  Many types of software available for generating media are easier to use than you might think.  And for those technologies that are more difficult, look for free online tutorials. You might find this compilation of open and commercial tools for freelance instructional designers useful. One of the most important things to remember is that multimedia for the sake of having "multimedia" may not be helpful to your students.  Just because students like a video, doesn’t mean they necessarily learn from it.  Create or select videos for the purpose of learning, not entertaining.  And in this age of YouTube, look for short videos; less than three minutes is the general rule. What is most important is whether a video (or any other element) supports a learning objective in your course. What’s the key to writing effective learning objectives? Crafting learning objectives is not a pure science.  There’s a lot of iteration that goes on before we get to a final set of objectives (and the multitude of questions and activities to support them). To develop learning objectives for your course, write a list of questions you want students to be able to answer about the topic. Then think about how you would assess and measure your students’ knowledge. Ask yourself: What do I want my students to know, remember and understand from this unit or module? Does each learning objective explicitly describe what I expect students to know or do?  If not, be prepared to revise it. In writing learning objectives, it’s essential to use the right verbs so you’ll be able to measure whether students can demonstrate their knowledge.  If your learning objective asks students to compare and contrast, it’s important to provide enough information and questions about the topic so students can make multiple comparisons and contrasts. If you find after drafting some activities and questions, that it isn’t necessary or possible for students to compare and contrast, then change the verb to describe the component parts of the topic. Finding the right level of detail is also an iterative process.  Notice the difference in the type of information and level of detail between these two learning objectives from an Anatomy & Physiology course: Identify and describe the organs of the digestive system. Describe the stomach and its functions. You want learning objectives that require a similar level of detail across an entire course.  This helps students know what is expected of them, and helps them feel supported and scaffolded by the material as they work toward meeting the learning objectives.  How do you align content with the learning objectives? Aligning content with the learning objectives is essential for effective teaching and learning. It may sound easy, but again, it’s an iterative process. Here’s another example from the Anatomy and Physiology course: Learning objective: Describe the major functions of the digestive system. Activity question:  Lactose intolerance is caused by insufficient enzyme to digest which kind of organic compound? Answer choices: carbohydrate, protein, lipid, nucleic acid Hint (to help the student answer the question): Lactose is found in milk products, and is also known as "milk sugar." It is quite possible that by answering the activity question, the student is describing some aspect of the digestive system.  It is also possible that the feedback provided for each of the answer choices will focus on digestive functions and tie the question back to the learning objective.  But, without asking the SME, the alignment with the learning objective is not necessarily obvious. The goal is to measure learning as it’s taking place, and measure the right thing. Ideally, I would want to see a question or set of questions that actually asks the students to describe the functions of the digestive system.  If that is too broad for the scope of the course, or not exactly what you need students to be able to do, then the learning objective should be revised to more clearly state what students do need to learn. For help in finding the right, measurable verb for a given learning objective, this simple chart has been invaluable to me. Focus on what your students will be able to do. Look for content and activities that provide practice opportunities for your students to do something with the information they are learning.  This includes making careful use of multimedia and interactive content, and providing different contexts and perspectives on the topics being covered. Starting with student-centered learning objectives that are measurable and at the right level of detail for students is the most important thing you can do when designing learning materials.  If you do that correctly, finding content to support those learning objectives, and your students’ success, becomes a much easier task.              

Following our March Q&A webinar, Acrobatiq Learning Engineers Bill Jerome and Sandy Raysor answer questions about the different types of analytics and how instructors can incorporate more analytics in their courses to impact student learning. Bill Jerome, VP of Product and User Experience What are the main types of analytics in higher education? And what differentiates them from each other? There are many ways to categorize analytics in the education space.  For our purposes, we think of three types of analytics that fall along a spectrum.  They are all useful and can serve practical needs. But they are different in what they mean and what they can do.  They are:  Institutional Analytics (also called Progression Analytics)  Engagement Analytics  Learning Analytics Institutional Analytics Institutions often track their students’ progress towards their degrees.  Identifying parts of the curriculum where otherwise strong students struggle, or courses that often result in students leaving the program, is an important process involving registration, order of courses taken, and course grade data.  It’s worth noting that the inferences drawn from such data analytics are highly correlational.  Knowing a course is a stumbling block does not prescribe a particular action to resolve the problem. Engagement Analytics Engagement data is typically focused on students at a particular moment.  While data from institutional analytics takes time to collect and act upon, engagement analytics records data on student behavior in real-time. For example, by analyzing when students login to online resources or how much time they spend online, patterns can be found. These patterns can lead to predictions about which students might drop out or fail the course. Using the information, institutions can notify faculty or tutors in time to intervene with these struggling students.  Here, the prescribed action is helpful, though it doesn’t point to the underlying cause.  All the system knows is, "this student looks like some other students, who, for unknown reasons, were at risk."  The instructor or tutor isn’t armed with any specific information, other than to check in with the student. Learning Analytics Finally, there’s learning analytics on the causality end of the spectrum.  The goal is to estimate individual student learning, and therefore, draw larger conclusions about groups of students.  Additionally, we can draw conclusions about the effectiveness of the learning resource itself.  By identifying where students are struggling at a topic or concept level, we not only identify students, who are having difficulty, but we can make specific recommendations to fill in the gaps in their learning.  We can also differentiate struggling students who are working hard, from struggling students who aren’t taking advantage of the available resources. This level of detail enables instructors to adapt the type of intervention, e.g., pointing students to resources versus providing additional instruction. In summary, we can think about these three types of analytics in terms of the data each collects and how that data is used. Institutional Analytics ‣  Relatively small amount of data needed for analysis; typically already collected at an institution, such as which courses students enroll in. ‣  Ability to "red flag" courses or sequences in programs that need attention.  Engagement Analytics ‣  While the volume of data needed is much larger, most online systems collect basic metrics similar to online retailers (such as identifying the user and where and when she clicked). ‣  Measures student activity, but not learning. ‣  Groups students to identify at-risk students for human intervention (without identifying why they may be at risk).  Learning Analytics ‣  Captures and reports large amounts of formative and summative data from content instrumented to provide learning information. ‣  Accurately predicts each student’s subject mastery, down to the skill level, based on rich information about the individual’s successes, guesses and failures in interacting with the content. Sandy Raysor, Learning Engineer  What are some practical tips for professors who want to get more learning data from their students’ online activities?  What should instructors look for when assessing ready-made online courseware?  The more online work you can give students the greater opportunity you have to collect data.  Increased amounts of data will give you a clearer picture of where your students are, and where they need additional help.  Decide what type of data you want to capture and then explore tools that collect that specific information. For example, capturing engagement data (tracking the time students spend online) is different than capturing data on how well students are mastering a concept or skill. Just because students spend a lot of time working on material, doesn’t mean that they have actually learned the material. ‣  Look for online activities/programs that give students plenty of practice activities that can be reported back to you with performance metrics. ‣  Acrobatiq courseware is designed with a variety of adaptive, problem-solving activities called "Learn By Doing," "Did I Get This?" and "Checkpoints and Quizzes." ‣  In those areas requiring significant practice for subject mastery, such as math skills, Acrobatiq courseware includes many practice activities related to a single learning objective. If you have the opportunity to develop your own online content and practice activities, be sure that different aspects of the learning objectives (if there are any) are covered in practice. ‣  For example, let’s say that your learning objective is: The student will be able to calculate the volume of a rectangular prism.  There are several different parts to this learning objective that the student must be able to master in order to demonstrate the required skills and proficiency. The student must know what a rectangular prism is. The student must know the equation for doing the calculation. The student must know how to multiply correctly. The student must understand how to do calculations with units correctly. Activities should address all of these skills before the student is asked to tackle the integration of these skills (the learning objective).  Use good distractors so that you have the opportunity to correct misconceptions. Think about common errors that students make. Think about why students make these errors, and what type of information you can give them to correct the misconceptions.    

Whether you’re developing an online, hybrid or flipped course for the first or 50th time, you can benefit from these insights and practical tips from our Learning Engineers Kim Henry and Erin Czerwinski. Combined, they bring 35 years’ experience in designing, developing, implementing and improving online courses, curricula and platforms. Listen to the replay of their recent webinar, Essentials of Effective Online Course Content. How do you develop content for Acrobatiq courseware? Kim Henry: Courseware content development is driven by two primary elements: A set of student-centered, measurable learning objectives that articulate the essential concepts and skills students acquire from engaging with the courseware, And The Big Picture that represents the subject domain and provides a knowledge structure in which students relate the new concepts and skills being learned. The language used in learning objectives should capture the level at which learning is occurring and at which feedback is relevant. Learning objectives are designed to: Communicate our intentions clearly to students and to colleagues Provide a framework for selecting and organizing course content Guide in decisions about assessment and evaluation methods Provide a framework for selecting appropriate teaching and learning activities Give students information for directing their learning efforts and monitoring their own progress Based on A.H. Miller (1987), Course Design for University Lecturers. New York: Nichols Publishing. Also see, C.I. Davidson & S. A. Ambrose (1994), The New Professor’s Handbook: A Guide to Teaching and Research in Engineering and Sciences. Bolton, MA: Anker Publishing Company Inc. How do you determine how many learning objectives are needed in a course? KH: A number of factors may influence the body of learning objectives for a particular course: Standards set by a professional organization Advances and/or trends in a field or domain Role of the course in a sequence and/or curriculum Expectations based on similar existing courses A small core team of learning engineers, subject matter experts (SMEs), course developers, and other stakeholders engage in a process of gathering all of the relevant information about a course and then synthesize it to determine: High-level outcomes for the course Key features of The Big Picture Scope of the course, including major topics Learning objectives for the course Preliminary skills map for the course Once the scope of the course is identified and the learning objectives are written, an organization for the course can be formulated. The organization of Acrobatiq courses is based upon a unit/module structure where units are similar to chapters and modules similar to sections. Modules consist of logically clustered learning objectives. In general, the content development process is focused around the learning objectives and skills map. There are several primary "ingredients" that go into an Acrobatiq course. Formative and Summative ADAPTivities: These are activities that provide students with sufficient practice to support each learning objective’s skills and knowledge components. Formative ADAPTivities include "Learn By Doing" and "Did I Get This?" activities that provide students with opportunities to engage in active learning. Summative ADAPTivities are assessments that may occur at the end modules and/or units. ADAPTivities are developed to capture and convey information, including "must cover" concepts and skills, as well as common misconceptions. Through various styles of interaction, ADAPTivities provide students with active learning experiences that include hints and targeted feedback. The primary purpose of exposition is to connect ADAPTivities. Generally, exposition is developed in one of two ways: Exposition from existing content is reconstituted and combined with original exposition written by SMEs. Original content written primarily by the SMEs on the development team. How do you work with subject matter experts in developing course content? Erin Czerwinski: Historically, I have worked with subject matter experts (SMEs) to transform materials they developed in one format (paper or PowerPoint, for example) into new, dynamic content delivered through a learning platform. When working with SMEs, who are typically experienced teachers and authors in the subject domain, it’s helpful to be aware of something we call an "expert’s blind spot." SMEs know the subject so well they may not remember how they first struggled to understand and learn the concepts in their field. By asking these faculty experts a series of questions, we help them think from the students’ perspective. This enables us to identify students’ most common misconceptions, and where they struggle with the subject matter. All instructors who teach and design learning materials should be aware of their blind spots. By continually challenging your assumptions about the difficulty of the material you are teaching, you are in a much stronger position to develop course material that speaks directly to your students, and helps them learn faster and better. What are some sources of instructional content available to all instructors? EC: Wikimedia Commons and Creative Commons are a couple of useful resources. These organizations offer access to search services that will help you find content (images, music and videos) from a variety of sources, and often include licensing or permissions information. Another great source of content is your peers. Many faculty post domain-specific content they use on the Web. With any content, it is important to check if the materials are copyrighted and what permissions may be required to use it. You can also try creating your own content. For certain media elements, start simple, with something like the animation features in PowerPoint. Or experiment with screen recording software like Camtasia and others. Many types of software available for generating media are easier to use than you might think. And for those technologies that are more difficult, look for free online tutorials. You might find this compilation of open and commercial tools for freelance instructional designers useful. One of the most important things to remember is that multimedia for the sake of having "multimedia" may not be helpful to your students. Just because students like a video, doesn’t mean they necessarily learn from it. Create or select videos for the purpose of learning, not entertaining. And in this age of YouTube, look for short videos; less than three minutes is the general rule. What is most important is whether a video (or any other element) supports a learning objective in your course. What’s the key to writing effective learning objectives? EC: Crafting learning objectives is not a pure science. There’s a lot of iteration that goes on before we get to a final set of objectives (and the multitude of questions and activities to support them). To develop learning objectives for your course, write a list of questions you want students to be able to answer about the topic. Then think about how you would assess and measure your students’ knowledge. Ask yourself: What do I want my students to know, remember and understand from this unit or module? Does each learning objective explicitly describe what I expect students to know or do? If not, be prepared to revise it. In writing learning objectives, it’s essential to use the right verbs so you’ll be able to measure whether students can demonstrate their knowledge. If your learning objective asks students to compare and contrast, it’s important to provide enough information and questions about the topic so students can make multiple comparisons and contrasts. If you find after drafting some activities and questions, that it isn’t necessary or possible for students to compare and contrast, then change the verb to describe the component parts of the topic. Finding the right level of detail is also an iterative process. Notice the difference in the type of information and level of detail between these two learning objectives from an Anatomy & Physiology course: Identify and describe the organs of the digestive system. Describe the stomach and its functions. You want learning objectives that require a similar level of detail across an entire course. This helps students know what is expected of them, and helps them feel supported and scaffolded by the material as they work toward meeting the learning objectives. How do you align content with the learning objectives? EC: Aligning content with the learning objectives is essential for effective teaching and learning. It may sound easy, but again, it’s an iterative process. Here’s another example from the Anatomy and Physiology course: Learning objective: Describe the major functions of the digestive system. Activity question: Lactose intolerance is caused by insufficient enzyme to digest which kind of organic compound? Answer choices: carbohydrate, protein, lipid, nucleic acid Hint (to help the student answer the question): Lactose is found in milk products, and is also known as "milk sugar." It is quite possible that by answering the activity question, the student is describing some aspect of the digestive system. It is also possible that the feedback provided for each of the answer choices will focus on digestive functions and tie the question back to the learning objective. But, without asking the SME, the alignment with the learning objective is not necessarily obvious. The goal is to measure learning as it’s taking place, and measure the right thing. Ideally, I would want to see a question or set of questions that actually asks the students to describe the functions of the digestive system. If that is too broad for the scope of the course, or not exactly what you need students to be able to do, then the learning objective should be revised to more clearly state what students do need to learn. For help in finding the right, measurable verb for a given learning objective, this simple chart has been invaluable to me. Focus on what your students will be able to do. Look for content and activities that provide practice opportunities for your students to do something with the information they are learning. This includes making careful use of multimedia and interactive content, and providing different contexts and perspectives on the topics being covered. Starting with student-centered learning objectives that are measurable and at the right level of detail for students is the most important thing you can do when designing learning materials. If you do that correctly, finding content to support those learning objectives, and your students’ success, becomes a much easier task.              

It’s not uncommon for online courses in higher education to include instructional resources from a wide range of sources. Resources may include digital content from textbooks (e.g. flashcards), images used in campus-based courses, freely available content from the Internet, print or ebooks from publishers, activities pulled from open education resource repositories, and others.  Some of the material is placed within the course environment, some sits outside. The "cut-n-paste" functionality of the Internet has made this "bricolage" approach to course design easy and, therefore, inevitable. However, the bricolage approach has also increased the prevalence of online courses with weak instructional coherence, coordination and consistency. By relying on materials from a wide range of sources - each built by different organizations, to serve different users, and to fit into different contexts - we, inevitably, decrease the degree to which each unit of instructional material aligns with the other materials. In the end, it’s learning outcomes that are compromised. Symptoms of Incoherent Course Design Instructional materials and activities drawn from a variety of sources can differ in a variety of ways that impact instructional quality. Differences include: Level of difficulty. Instructional materials gathered from different sources are designed for students at very different levels of subject mastery and comprehension. Terminology. Different sources often employ different terminology to describe similar information.  While these differences are often small, and may seem inconsequential to subject matter experts, they can easily confuse learners that are new to the curriculum. Pace of instruction. Each instructional element implicitly assumes a certain pace of instruction through which the student will progress through the material. Level of detail/depth. Instructional elements include different amounts of detail. Asking students to move between instructional materials that include different levels of detail may make it more difficult for them to identify what information is essential, and what is not. Organizational principles. Every instructional element is designed to operate in a particular structure and design environments. Pulling items out of one context and dropping them in another adds unintended (and instructionally useless) complexity. Design features. Visual design features, such as use of color and icons, can be used effectively to improve comprehension and ease of use, but they must be applied consistently. Barriers to Coherent Course Design In the classroom setting, the bulk of the instruction is created by and funneled through a single source: the instructor. As a result, instructional coherency tends to occur naturally. In online education a number of factors work against coherency: The ease with which we can find related instructional content on the Internet; Confusing regulations concerning use of copyrighted material on the Internet; The inability of institution staff to produce a wide range of instructional materials at a low cost (due, largely, to the lack of economies of scale in the dominant business model of online education); Insufficient incentives for faculty to dedicate additional time to course design and development, given prevailing compensation and incentive models. The lack of professional development resources for instructors responsible for course design. These inconsistencies make it more difficult for the educator to provide students with coherent and effective learning. The quality of learning can suffer and the need for student support - from the instructor, staff and others - is heightened. Students should be able to focus all of their limited energies on learning, not on trying to understand the different levels, styles, pace, and sequencing in a grab-bag of instructional element. This coherency, in turn, allows the instructor to focus her time on teaching and supporting students, rather than compensating for inconsistent and instructionally incoherent course design. In a well-designed course, the instructional materials are fully integrated and coordinated, pitched at the appropriate level of difficulty, presented to the learner with the ideal amount of detail, and have consistent design features (color, navigation). Each element in a course should be built according to a single, overarching design - coherent, coordinated, and consistent.

Join the conversation on the latest research in the emerging fields of cognitive and learning science in our upcoming webinar with Dr. Marsha Lovett, Director of Carnegie Mellon’s Eberly Center for Teaching Excellence and Acrobatiq’s Chief Learning Scientist. What: Webinar - Unlocking Learning Through Cognitive Science When: Wednesday, April 30, 1:00 to 1:45 pm, EDT Who:  Dr. Marsha Lovett, Director, CMU’s Eberly Center for Teaching Excellence and Chief Learning Scientist at Acrobatiq How: Register now to RSVP During this interactive discussion, Dr. Lovett, co-author of How Learning Works: 7 Research-Based Principles for Smart Teaching, will share insights on: How learning data is used to create powerful feedback loops for students, instructors and course design teams How the science of learning informs both teaching practice and the development of new educational technologies How evidence-based instruction leads to improved learning outcomes Don’t miss this webinar & Q&A with one of the true innovators in educational technology. Bring your questions and colleagues!