Mediatech Foundation

Mediatech is more than a community technology center. It is also a place where research takes place, on software quality and on how people use the Internet. Here’s the press release from one study that was released last year, on Online Test Prep sites. It was coordinated by Linda Bradway, and designed by Warren Buckleitner.

Above: High School Juniors Test Ten
Versions of Different SAT Tests.

Below: Study authors Linda Bradway and
Warren Buckleitner at the National Press Club.

YONKERS, NY, May 25, 2005 — Consumer Reports WebWatch tests of 10 Web sites that help students prepare for the Scholastic Aptitude Test—a key qualifier for admission to top colleges—show a free site’s services as effective as others costing upwards of $400, while testers found the offerings of many major brands marred by grammatical errors, technical glitches and aggressive advertising tactics.

WebWatch and the Mediatech Foundation, a non-profit organization based in Flemington, N.J., tested 10 sites during summer 2005. Mediatech recruited 20 high school juniors to evaluate 10 sites online: Barron’s Test Prep, Boston Test Prep, Kaplan’s SAT Online Prep, Number2.com, Peterson’s SAT Online Course, PrepMe, SAT Secrets, Test Preparation Program, The Official SAT Online Course, and The Princeton Review.

Taking the SAT has become a rite of passage for more than 1 million college-bound U.S. high-school students each year. In tandem, demand for SAT test preparation services has grown, driven by the competitive nature of college admissions and the ability to take the SAT repeatedly to attain a better score. While the size of the online market is difficult to define, it has been estimated at $50 million by Eduventures Inc., a Boston-based research firm.

Because of the increasing role of the Internet in delivering SAT test prep services to students and families, WebWatch and Mediatech selected the 10 sites with a focus on cost variation and market share to assure a mix. Each of the 10 sites was reviewed by two students during a minimum of five, four-hour sessions. Testers spent more than 400 combined hours reviewing these sites.

Because of the increasing role of the Internet in delivering SAT test prep services to students and families, WebWatch and Mediatech selected the 10 sites with a focus on cost variation and market share to assure a mix. Each of the 10 sites was reviewed by two students during a minimum of five, four-hour sessions. Testers spent more than 400 combined hours reviewing these sites.

Student testers deemed seven of the 10 sites generally effective in what they promised to deliver. In most cases, testers were pleased with their experience, and would recommend this method of study to others.

However, WebWatch observed some troubling trends, particularly regarding the blending of advertising and educational content, aggressive marketing and privacy practices. Download the full report.
Key Findings:

• Number2.com was the only free-of-charge service tested, and it performed exceptionally well against expensive, better-known services such as The Princeton Review and Kaplan’s.
• Testers found the prep service created by The College Board, creator of the SAT itself, had technical glitches and lacked interactive features common on other sites.
• Mistakes in online sample tests – including grammatical problems, questions with no answers, missing sections of text, font problems, or poorly constructed questions – were consistently present in six of the 10 services evaluated.
• One program charging $29.95 for three months contained frequent spelling errors, including words like “whore” instead of “where.” The home page contained 20 non-functional links and no information about the publisher. The site’s Internet Service Provider was traced to Bangkok, Thailand, and e-mail requests for refunds and technical support were not answered.

“Online SAT test prep services are clearly still evolving,” said Warren Buckleitner, Ph.D., the author of the report and a consultant and adviser to Consumer Reports WebWatch. “On one hand, students rated many of the valid sites superior to traditional books or non-individualized SAT classes. On the other hand, there was a wide variation in costs involved, and it was far too easy to find sloppy editorial content, buggy programming and marketing practices that are, at best, questionable.”

Buckleitner is editor of Children’s Technology Review, a periodical covering children’s interactive media. He is also the founder and interim director of Mediatech. The project was funded by Consumer Reports WebWatch at a cost of approximately $33,000.
About Consumer Reports WebWatch
Consumer Reports WebWatch is the leader in investigative reporting on trust and credibility in the online marketplace. WebWatch uses the proven methods of Consumer Reports to produce comprehensive research, breakthrough conferences and serves as a daily resource of unbiased and trustworthy information. Its research agenda includes entire online marketplaces, such as travel, search, health, financial services and more. WebWatch is a project of Consumers Union and is funded by The Pew Charitable Trusts, The John S. and James L. Knight Foundation, and the Open Society Institute. WebWatch’s investigative reports, articles and news are available to the general public at http://www.consumerwebwatch.org/ .

About Consumers Union
Consumers Union, the nonprofit publisher of Consumer Reports, is an expert, independent nonprofit organization whose mission is to work for a fair, just, and safe marketplace for all consumers and to empower consumers to protect themselves. To achieve this mission, we test, inform, and protect. To maintain our independence and impartiality, CU accepts no outside advertising, no free test samples, and has no agenda other than the interests of consumers. CU supports itself through the sale of its information products and services, individual contributions, and a few noncommercial grants. Consumer Reports content can be found online at ConsumerReports.org. Consumers Union’s public policy work can be found online at ConsumersUnion.org.