Evaluation of your training programs should be occurring at all times. But keep in mind that e-learning evaluation is different; the goal may be the same but your methods will change slightly. When you evaluate your e-learning program, there are two areas to focus on: learner and content. Let’s look at ways to evaluate these areas effectively using your program itself and your delivery system. The initial evaluation of your e-learning program can take place both at the beginning of the program and throughout its lifetime. In the area of content, the first thing you’ll need to do is evaluate the course content. Remember that e-learning is new to your learners and may even be new to the training staff, so discovering how the content is being viewed is critical. You probably conduct "level one" evaluations in your traditional programs, sometimes referred to as "smile sheets" Although this evaluation has a light name, the act should not be taken lightly. The first evaluation is sometimes very valuable. So how do you carry out this task in an e-learning environment? Your LMS may give you the option of putting participants into an evaluation before they get to a test on the material - or immediately after. The best option is to get the evaluation while the audience is "captive". Otherwise, you may not get the evaluation at all. The content of your evaluation can cover traditional bases, that is, did the learner find the content useful and engaging. But instead of talking about an instructor, substitute the system itself. What was the learner’s immediate reaction to the accessibility of the system? Did he or she encounter problems with the navigation or have trouble understanding the instructions? Did the learner encounter long delays while waiting for a video or an activity? If you think about what the learner is seeing and doing at the time he or she is in the course, you’ll be able to come up with an effective evaluation. The learner data, aside from what you can deduce from evaluations, can also be obtained from the system itself. Look at the number of participants you were expecting, or the number of participants who are required to go to certain courses. How many of them accessed the system and the courses? How many of those actually finished the course, took an evaluation, and received a grade? You can also look at initial learner reaction outside of traditional means, especially if e-learning is new to the organization. For example, consider sending out emails to a "test group" of learners in order to get their candid and unstructured feedback. As the e-learning program progresses, consider calling users on a random basis or even having local users attend focus groups. Just because the training intervention is online does not mean that you can’t evaluate it in person. Be creative and remember that many members of your audience are more than willing to help you out. Once you’re past the initial rollout, remember to continue taking initial reaction evaluations -and to continue looking at the results. But what else can you do to ensure effectiveness of content and system when it comes to e-learning? Consider using online evaluations at 30 or 45 days after the course is completed. This technique often works in traditional training, so why not take a look at it for your e-learning program? You can find out if the learners found the content useful. And you can evaluate their reactions after they’ve had time to think about it and take other online courses. Some LMS systems may even "tickle" the user based on his or her login, that is, it may remind them that they need to take an evaluation on a course when they sign in to take another one. As you move forward, continue to look at the user data, as well. Know what your beginning numbers were and compare them to the numbers you’re getting on a monthly basis. You may see small fluctuations in the number of users, but the basic trend you’re looking for is an increase in users over time. If that increase doesn’t occur, dig deeper using more targeted evaluation methods. The best thing to remember with evaluation is that you should use all of the data you collect in order to modify your program - and make it effective for the organization and its learners. Related Posts: 7-Steps to Creating an Effective E-learning Program Part 6: Modification 7-Seven Steps to Creating an Effective E-learning Program Part 7: Regular Monitoring Engaging Participants 7: Evaluating for Engagement 7-Steps to Creating an Effective E-learning Program Part 2: Convert and Create Content Building a Corporate University: Maintenance

"We receive three educations, one from our parents, one from our schoolmasters, and one from the world. The third contradicts all that the first two teach us."~ Charles Baron de Montesquieu (1689-1755) Related Posts:Masters in Finance HQ100% ResponsibilityTraining Alchemy: How the Military Takes Ordinary to ExtraordinaryPersuasion and Influence Are Part of Foundations of LeadershipTraining Alchemy: 5 Disruptive Training Technologies

Now that your e-learning program is up and running, you’ve evaluated it, made changes, and the organization loves it. Can you sit back and relax? Absolutely not. Your e-learning program requires a constant eye on various areas to ensure that nothing goes off track. One tiny flaw can create large problems, and with large problems comes a loss of users. Organizations have made the mistake of letting the e-learning program "ride", only to find that one day no one is using it. Let’s discuss how you can monitor your program at all times. First, continue to monitor user data. This data includes the evaluations that you’ve taken the time to create and integrate. But remember to monitor at a deeper level than the evaluations. Are your user numbers up or down? Are the evaluations themselves changing in trend or tone? For example, a course evaluation that was consistently a 5 on a 5-point scale that moves to a consistent 4 is not exactly failing - but is it as good as it once was? You can also see from this type of data if your audience is growing and becoming more sophisticated. If so, you should consider going back to the modification stage to grow with the audience. Be sure to look at the tests that accompany your online courses, as well. Are certain items being missed or skipped on a regular basis? Those items can point to a poorly written question or a malfunction in the interactivity of the item. Next, be sure to look at your system data. One of the often-forgotten pieces of data for LM Systems is, believe it or not, cost. Once the budget is entered, billing for certain things on an LMS becomes automatic. Be sure that someone is looking at cost for upward trends. For example, the number of users may drop to a point where each one costs more. Or, higher bandwidth use may begin costing the organization more in terms of the overall communications cost. Don’t let the cost get away from you. Another way to look at system data is to find out how many technical troubleshooting calls are made. Whether you are taking calls internally or sending them out to the vendor, you should be able to monitor both number and subject. If users begin showing problems with course access, for example, you may need to tweak your instructions. Whatever the issues, your call information should be able to help you categorize and identify potential opportunities. Also, take a look at the transcripts or records that are being kept by the system. Are learners barely scraping by or are they all making high grades? Are learners being added to the system quick enough to get them going while they are still interested? This information can be monitored on a regular basis and can give you a great deal of insight into your e-learning delivery system. Course data is also an important monitoring point. You can look at evaluations regularly to see if courses are appropriate. But be sure to compare courses to each other. In other words, what common characteristics are shared between the courses that are being used more often? Do you have a high number of users on required courses only, while job or promotion-related courses are being ignored? Remember that a course evaluation is only going to occur when someone takes that course. Use the data that you can mine from your delivery system to make comparisons - and to make changes to course offerings as needed. Finally, don’t forget to monitor the training staff. It’s easy to assume that once the program is up and running the staff will have less to do. This is a common misconception with e-learning programs. As much as we would like the program to run itself, it just won’t happen. Is the administrator spending too much time on clerical tasks, like creating course "shells" or entering course descriptions? If so, consider bringing in an assistant. Are course content designers consistently making changes and taking those changes to the graphic designers? Or is there time to turn a course content designer into a combined content and graphics person? Can your marketing be taken over by someone who resides within the training organization? All of these questions will help you monitor your staff and make changes in order to keep things efficient. An effective e-learning program saves money and time and creates an effective platform to educate the workforce. When you follow these seven steps, you’ll have a guideline to creating the best e-learning program possible. Related Posts:7-Steps to Creating an Effective E-learning Program Part 6: Modification7-Steps to Creating an Effective E-learning Program Part 5: Evaluation7-Steps to Creating an Effective E-learning Program Part 3: Online Learning SystemsLMS: Building the SystemLMS: Training, Marketing, and Technology

Related Posts:Awareness TestDid you know?Math Education: An Inconvenient TruthDeveloping a Culture of Leadership10 YouTube Videos Every Entrepreneur Should Watch

Related Posts:Awareness TestShift HappensMath Education: An Inconvenient TruthDeveloping a Culture of Leadership10 YouTube Videos Every Entrepreneur Should Watch

"Who you become in the future will be a function of the books you read and the people you meet." — Charlie "Tremendous" Jones If you read 15 minutes a day you will read a book a month. That will put you in the top 1% of business people in the developed world. The Book you don’t read won’t help you! Make it a goal to build a great library and leave it as a legacy Related Posts:Training Does Make the Difference7-Steps to Creating an Effective E-learning Program Part 2: Convert and Create Content3 C’s for SuccessTurnover, Turnover, TurnoverWhy is Training Important?

"As the chief learning officer, ask yourself, ‘Are we investing sufficiently in learning to generate the growth and return we need?’ We had a group of graduate students research the topic, plotting long-term shareholder returns against a number of factors. They discovered that investments in training and development were the single best predictor of shareholder return three to five years down the road." —Karen Franse, "Build Your Profits," VARBusiness, January 18, 1999 It was true then (1999) it remains true today. Related Posts:Turnover, Turnover, TurnoverPareto Rule: 80/203 C’s for SuccessWhy is Training Important?Proving the Value of Training

Your e-learning program has rolled out and has come under constant evaluation. What are you doing with the evaluation data? If you’re filing it away, that’s not quite enough. Every piece of data you receive, whether it seems small or not, can help you modify your e-learning program, keep it fresh, and move it into prominence with your organization. When you consider modifying your program, look at three common areas: system, course design and delivery, and marketing. Remember that during evaluation you are looking at user data, evaluation data, and even survey data. When you look at modifications to your system, think about that data in system terms. Are users reporting problems with accessibility? If so, you may need to look at the number of users, the organization’s bandwidth, or its overall technical architecture. Any of these characteristics can show you specific and overall problems with the program. In specific areas, you may find that accessibility problems are caused by a slow running system, malfunctions in video streaming or interactivity, or overall system preparedness. If you determine that there are issues with your system, make a list of the problems. But don’t stop there. Make a list of the solution you’d like to see. For example, if the system is functioning slowly, know what the optimum time should be. Once you have a list of requirements and optimum functions, go back to the vendor or in-house IT department and present it. The next area to look at is course design and delivery. In evaluation, you’ve looked at the number of users per course, how many of those users finish the course, and how many leave the system without a course completion. If the system is functioning well, then you should consider the possibility that the courses are not presented as well as they could be. Remember our discussion on e-learning guidelines. Are the courses too long? Are individual frames covered with text and little interaction? Is the content appropriate for e-learning deployment? Are certain courses used infrequently? If so, consider taking them offline to create space for more critical courses. A training organization must always have a thick skin, but now is the time for the group to look at its output very critically for the benefit of the learners.  The key item to remember about course design and delivery is that changes must be made consistently, especially if courses are required at certain time periods. For example, if the entire organization must go through compliance training each year, consider making changes to the appropriate courses each year. Learners will appreciate the effort and will always be on top of mandated learning. In the course of modifying your program, don’t forget to look at the marketing and message of the program itself. If user numbers are low, think about the message and how it’s being broadcast. Send out surveys to see if learners received marketing information - and what they did with that information. Ask them how they reacted to the message that new training methods were available. Along those lines, find out if learners are being given ample time to complete training - or if they are being pulled back to work before they’ve had time to take courses. The idea here is to also take a critical look at how the program is being marketed - and to be bold in asking for changes to that message. Modification of the e-learning program should be gradual, however. Some organizations have been known to pull large parts of the program offline in order to modify it. This may not be the best way to take care of changes. Determine which areas need the most attention and work on those first. But avoid taking whole pieces of the program offline at all costs. Modifications should be so gradual that the learner notices the small changes each time he or she logs in to the system. Your modification should be a "no shock" plan. Remember to use your evaluation data to make constant modifications to your program. Related Posts:7-Steps to Creating an Effective E-learning Program Part 5: Evaluation7-Seven Steps to Creating an Effective E-learning Program Part 7: Regular Monitoring7-Steps to Creating an Effective E-learning Program Part 4: Implementation7-Steps to Creating an Effective E-learning Program Part 3: Online Learning Systems7-Steps to Creating an Effective E-learning Program Part 2: Convert and Create Content

It’s important to examine the competencies you would like to see in your organization and compare them to the competencies that a professional certificate holder can bring. But when we analyze competencies, we need to look at both expressed and implied competencies. The expressed competencies are those that are stated in the program, and may be technical or functional in nature. The implied competencies are the ones that are not stated by the program and take some inference to determine. In other words, these are the competencies that the certificate holder will most likely have because he or she went through the program successfully. Also keep in mind that you’ll want to look for competencies that are not only technical and functional, but also related to leadership and management. The PRM certification brings numerous expressed competencies to your organization’s collective table. The successful completion of the program means that a person is well versed in risk management theory as well as practice, through the mathematics of risk management. The certificate holder must also have the ability to apply theoretical knowledge, as the exam tests application of knowledge, conduct, and ethics through case studies. The expressed competencies in the PRM program are mainly technical and functional in nature. But let’s analyze the program a little further to look for implied competency for a PRM certificate holder. First, the multiple-choice format of the exams provides us with a picture of someone who is good at making decisions. The fact that the program is self-directed in nature paints a picture of a professional who has drive, determination, and focus. As we’ve mentioned, not all adult learners possess the competency to get through a self-directed study program successfully. Translated into the organizational world, this person will probably have the ability to work well independently. Because the PRM program allows a candidate to self-assess, you’re going to get a professional who is able to address his or her weaknesses. This is not just acknowledgement of a weakness but the ability to correct that weakness through action, study, or other intervention. The choice between online and classroom preparation also ensures that the professional knows his or her own learning style and can apply it quickly and easily to the learning process. Plus, in relation to learning, a successful completion of the PRM shows you that the person is open to being coached, trained, and managed in a positive direction. All of these combined competencies create a great profile for a potential organizational leader. The FRM certification through GARP also carries a high level of technical and functional competency, such as market risk, credit risk, and operational risk. The FRM also tests risk management in investment management. Again, this certification proves a certain ability to take knowledge, synthesize it, and apply it to common situations. In terms of implied competency, the FRM shows us that the certificate holder, because of the elite size of the network, has the ability to see him or herself a part of a larger group. The fact that the person chose the FRM designation shows that he or she is dedicated to the furtherance of the profession and not just interested in his or her own self-interest. It’s also necessary to again consider that a self-directed learner has proven an ability to focus and drive projects through to completion. In regard to the ERP certification through GARP, we can also see a high level of technical and functional competency, including physical energy markets, financial trading instruments, and valuation and structure of energy related transactions. Because there are specific experience requirements to even sit for the exams, we can also assume a higher level of experience in what is certainly a specialized field. On the implied level, the ERP certificate holder not only shows his or her determination and focus, but also a level of devotion to a specialized field. When you add the continuing education requirement to this mix, you’ll also see a professional who is certain that he or she is in the field on a permanent basis. Why are expressed and implied competencies important to your organization? One of the most obvious reasons is that you want to ensure that people in specific positions have a measurable level of functional and technical competency. All of these certification programs prove this competency, as well as experience. But what about bringing in certified professionals as part of your overall leadership "bench" strategy? If you are actively recruiting professionals with the designation, you can be assured that you are going to create a bench of high professional leaders, that is, the ones who can lead and manage functions in their own field. But when you look at the implied competency of these certification programs, you can also be assured that some of those professionals will be high potential leaders, that is, the ones who can be groomed for leadership and management across a spectrum of functional and technical areas. If you are considering certifying existing associates, think about how that will improve your existing "bench". The people who readily enter a certification program are proving that they are dedicated to the profession and dedicated to the organization’s future success. Not only this, they are obviously open to being coached and "schooled" on the most current industry knowledge. Now that we’ve looked at the FRM, PRM, and ERP certifications generally, examined the curricula and exam structures, analyzed recognition and standard, and discovered competencies, the last step is to look at the overall picture of how these things can help your organization. Related Posts:CFA CompetenciesRecognition and Standard of PRM, FRM, and ERPCFA Organizational BenefitsRisk Management: Certification Curriculum and Exam PreparationRisk Management: PRMIA and GARP Certifications

Professional certifications are important to both the individuals that obtain them as well as the organizations in which those individuals work. In the financial field, personnel involved in risk management can obtain several important certifications from two major international groups. In today’s uncertain financial environment, professional certifications can go a long way to calm investors and regulators, as well as restore faith in the financial system in general. But before we look at the organizational and individual benefits of the most common risk management certifications, we should spend some time becoming generally familiar with the certifications and the groups that offer them. There are two major groups offering risk management certifications: the Professional Risk Managers’ International Association (PRMIA) and the Global Association of Risk Professionals (GARP). Both of these organizations show their certifications as widely recognized and accepted, although the organizations approach certifications differently. PRMIA offers the Professional Risk Manager certification, or PRM. PRMIA calls the PRM certification "The Higher Standard in Risk Management" and is very flexible on how professionals prepare for the certification exams. The PRM is essentially a validation of skills that are most likely picked up in every day work in the risk management arena. The certification does stress professional standards and integrity in addition to skills and knowledge. Also, the PRM tests an individual’s ability to not only know best practices but his or her ability to apply those best practices in the appropriate situations. The candidate must be a member of PRMIA in order to sit for the certification exams, and, as in many cases with professional certifications, the candidate with other industry certifications, such as the CFA (Chartered Financial Analyst) may have an easier time attaining the PRM. In the industry at large, hiring managers often use the PRM designation as a measurement for the most desirable risk management skills. GARP offers two major risk management certifications, the FRM, or Financial Risk Manager, and the ERM, or Energy Risk Professional. The FRM, according to GARP, is one of the certifications that is currently desirable to recruiters who are looking to fill senior risk manager positions. There are only around 18,000 FRM’s in the world, which is a small number for a professional certification that is recognized around the world. In order to qualify for the FRM, a professional must have two years related experience and must also be a member of GARP. The ERM certification is obviously for energy industry risk managers, who must also have at least two years experience in the field of energy risk management. These professionals must also be members of GARP. GARP is in the process of creating a continuing education program and requirements for the ERM certification, which will most likely become a requirement in 2010. In the field of risk management, the ERM is one of the only designations that has or is about to have a continuing education requirement. It’s a good idea to have a general feeling of what professionals hold risk management certifications - as well as what industries look for these professionals. The top industries with certified risk managers, and whose recruiters look for certifications, are banking, academics, asset management, and government. There are many other sectors of the financial industry in which you will find certified risk managers. Professionals who hold these certifications also hold a wide variety of positions, from junior through executive levels. The most common jobs held by certified professionals in the field are risk managers, analysts, consultants, accountants, traders, portfolio managers, and even operations managers. What exactly does the professional have to undertake in order to become certified as a risk manager? It depends on the program, but both the PRMIA and GARP certifications are either strictly structured or strictly unstructured in regard to preparation, and both organizations certify only after examination. To obtain a PRM certification, the candidate must take four examinations, either separately within two years or all at once. These exams cover financial theory, financial markets, risk management mathematics, best practices, ethics, conduct, and case studies. PRMIA will help a candidate prepare for the examination through a variety of preparation courses and seminars, but the candidate is not required to "officially" attend any courses. In fact, PRMIA encourages organizations to use the exams separately as ways to test potential job candidates or to test for promotional readiness. As we discussed, a PRM candidate can take the entire battery of tests at one time, or can spread the four out over two years. The GARP FRM certification is broad based, covering market risk, credit risk, operational risk, and risk management in investments. There is only one exam in order to obtain the FRM certification. The ERP certification, on the other hand, requires about 250 hours of study to prepare and is also only one examination. The ERP core competencies include physical energy markets, risk management compliance, financial trading, and valuation of energy transactions. We will discuss specific benefits of these certifications throughout the risk management series, but it’s a good idea to consider why certifications might be important to your organization. A professional designation carries proof of knowledge and competence in specific areas. Your organization can use the certifications as a way to market or prove competence in the field, and can even use the possibility of certification as a way to recruit and retain the best talent. In today’s financial market, a certification may show a higher dedication to ethics and integrity, both on the organizational and individual levels, and this may help to satisfy regulatory agencies and shareholders. The next step in this series is to look more closely at the curriculum and study methods for each of the certifications, and determine why this is important to the organization. Related Posts:Risk Management: Certification Curriculum and Exam PreparationRecognition and Standard of PRM, FRM, and ERPCompetencies for PRM, FRM, and ERP Certificate HoldersCFA Recognition and StandardWhat Is the CFA Designation?