Evidence of effectiveness

This criteria encompasses any evidence of the effectiveness of the instructional media product as judged by those who may have used or evaluated it. Ideally, we should be able to find objective, unbiased information on the effectiveness of the media product in specific educational settings. But this is not always the case, so in lieu of this evidence we may explore with caution alternative measures of overall product quality. Under this criteria, I would suggest any/all of the following items that might qualify as evidence of the overall effectiveness of the product:

testimonials: as with any advertised product, individual claims about the benefit of the product, which are called "testimonials," are a common way for those considering purchase to judge how effective the product is. While testimonials are far from objective, and there can be obvious bias in someone offering praise about a product (especially if they are being paid to do so), these can provide some insight into the product itself. But testimonials should be taken with a large "grain of salt" and consideration should be given to any relationship, financial or otherwise, between the individuals providing the testimonials and the company selling the product.

published research: perhaps representing the highest level of this specific criteria, any published research on a product in a peer-reviewed journal warrants serious consideration. While there can still be bias in this form of evidence, we normally consider research as objective and the recommendations made as valid. That said, some companies have been known to hire someone to conduct "research" on their product and usually this results in a positive recommendation by those so employed. So care must be taken to do our homework when exploring this type of evidence to ensure that the outcomes and results have been reviewed by someone without a vested interest in the product. Otherwise, this is likely the "gold standard" for evidence of effectiveness.

awards: another indicator of the overall quality and perhaps effectiveness of instructional media are awards given by independent agencies who review and recommend products within this category or area. These awards can be from educational institutions, objective sources, or from private sector organizations, and often indicate specific elements of quality in a product. Be sure to review the actual comments or critique of a product that receives an award to ensure an accurate understanding of its view on the product and its evaluation.

other reviews: like awards, individuals or organizations that have reviewed an instructional media product can provide helpful information and insights into the overall effectiveness or quality of that product. The web is full of sites that review and critique products, and instructional media is no exception to this, so a search using Google, Yahoo or Bing for "review" + [product name] usually results in reviews provided by someone other than the company selling the product. As with testimonials, it is essential when reading these type of reviews to explore exactly what criteria were used in the review and ensure no vested interest exists for those conducting the review. Samples of reviews of instructional media products by students in previous EDG/EDT 620 courses are included in this Wiki.

[posted by A. Topper on 10/10/11]