A handbook for professional education.
Table of Contents
[ Editor's Note: Yes, this promotes one company's wares, and there may be other good vendors as well (several of which are, in fact, mentioned in the article). However, it gives an invaluable overview to the whole issue of monitoring in cyberspace, and the implications for public relations (including customers and investors), sales and marketing, and other important constituencies. Rather than weed it all out, I've allowed the company to toot its own horn. After all, that's why people give out free content, right? It also serves as a great example of viral marketing, since the copyright holder freely gives reprint rights. This version is slightly edited.
This article runs 24 single-spaced pages, as configured by the author. We strongly suggest that you print it out.
Note From CyberAlert: This article is designed as a professional educational program for widespread publication. It is copyright by CyberAlert, Inc., but may be copied and published at will in its entirety or in part, provided the article or excerpt carries appropriate credits for the writer, Amelia Kassel of MarketingBASE (http://www.marketingbase.com) and for CyberAlert, Inc.(http://www.cyberalert.com).
The proper bibliographical citation is: Kassel, A., Internet Monitoring and Clipping: Strategies for Public Relations, Marketing and Competitive Intelligence in http://www.cyberalert.com/whitepaper.html. The copyright owner would appreciate notification of publication. Please send notice to firstname.lastname@example.org. ]
In a few short years, the Internet has evolved into an indispensable marketing tool and the epicenter of business "buzz." At the same time, the Internet has become the favored weapon for angry customers, disaffected employees, and consumer activists to attack corporations and their products. These published criticisms influence attitudes and purchasing decisions of thousands or even millions of people worldwide, threatening a company's reputation or brand image and potentially causing serious financial damage through reduced sales or a battered stock price. In general, the longer negative buzz persists, the greater the damage. And no company or organization is immune.
As a result, monitoring what is said on the Internet about a company and its products has become a fundamental corporate responsibility to manage corporate reputation, protect brand images, track market opinion, correct misinformation, and improve customer service. A well-conceived and well-implemented Internet monitoring strategy provides an early warning system by finding key nuggets of market intelligence or spotting important trends in published articles or criticism.
Like traditional public relations clipping services, Internet monitoring and news retrieval services monitor the contents of major news sources, "clip" those articles containing the company or brand name, and deliver the "clips" enabling their client companies to assess market penetration and effectiveness of specific news releases and public relations programs.
In pre-Internet days, effective market intelligence and reputation management could be achieved by tracking a few thousand publications. On the Internet, there are literally millions of Web sites where crucial market intelligence and criticism can appear, available for viewing by over 100 million wired consumers.
Monitoring the Internet with search engines is less than thorough, lacks the required timeliness, and is extraordinarily labor intensive. Costs of professional hours devoted to manual monitoring are high though hidden in base salaries.
Because of the limitations of search engines, online database services, and Internet-based news aggregators, Internet monitoring and clipping services have emerged. These highly automated services are designed to provide comprehensive and timely "clips" to meet the information needs of public relations, marketing, market research, and competitive intelligence. The services are also used by business planning departments, information managers and librarians, and legal departments for intellectual property infringement monitoring.
Automated Web monitoring services combine the best attributes of search engines (in-depth coverage of the visible Internet), electronic news aggregators (timely coverage of news sources), and traditional clipping services (delivery of all found articles) into a single, automated, timesaving, and productivity-enhancing information retrieval service.
Services for Internet monitoring and clipping vary widely in their capabilities, features and software sophistication. They range from purely manual services that rely on live editors using public search engines to fully automated services using sophisticated, proprietary technologies. Automated services like CyberAlert (http://www.cyberalert.com) deliver more comprehensive and timely results than manual ones.
The key criteria in selecting an Internet monitoring and clipping service are: extent of automation, media coverage and selectivity, search frequency and depth, timeliness of delivery, complexity of search definition, relevance, clip management system, cost and value, and customer service. Internet monitoring and clipping services, such as CyberAlert, are more effective and less expensive than assigning staff to find news and other articles on the Internet about a company and its products. As a business-to-business (B2B) service, an Internet monitoring and clipping service can be an affordable productivity tool for almost any sized company in almost any industry or business. The benefit is superior market intelligence for better-informed business decision-making.
Introduction: What's the Buzz?
Today, the "buzz" about companies and products is on the Internet.
In making buying decisions, wired consumers use the Internet as the primary information resource for product features and specifications, independent product reviews, consumer opinions, and other available information. As a result, the Internet has emerged as a key medium for marketing, public relations, and e-commerce worldwide.
At the same time, the Internet has become the favored weapon for angry customers, disaffected employees, consumer activists, stock traders, and others to bludgeon corporations and their products. Bashing of companies or products, rumors, and hoaxes can turn up almost anywhere on the Internet. These criticisms influence attitudes and purchasing decisions of thousands or even millions of people worldwide, threatening a company reputation or brand image, and potentially causing serious financial damage through reduced sales or a battered stock price.
Often, the criticisms are one-sided, inaccurate, or simply false. Sometimes they are intentionally malicious. And no company is immune.
The net is also becoming a breeding ground for scams of every design. The US Federal Trade Commission (FTC) identified 18,660 instances of potential Internet fraud last year. At the US Security & Exchange Commission, officials receive about 2,000 e-mails a day identifying potential Internet cons.1
These days, smart companies, large and small, use specialized Web monitoring services to learn what others are saying about the company, its products and services; to manage corporate reputation and brand image; to identify trademark and copyright infringements; to track press clippings; and to conduct market research and competitive intelligence.
Web clipping and monitoring services such as CyberAlert 3.0 (http://www.cyberalert.com) capture the most current information from online news sources and Web publications including news syndication services, newspapers, magazines and journals, networks, electronic bulletin boards, and other Web sites. Corporations and other organizations employ the online clipping services to track mentions of companies, brands, executives, and issues. The specialized services can monitor any subject of consequence daily, such as key topics that center on trends, regulations, legal issues, and thought leaders.
Why Monitor the Internet?
"Bad things happen to good companies on the Internet," contends William J. Comcowich, president and CEO of CyberAlert, Inc. in Stratford, CT. "Monitoring what is said on the Internet about a company and its products has become a fundamental corporate responsibility to manage corporate reputation, protect brand images, track market opinion, correct misinformation, and improve customer service. A well-conceived Internet monitoring strategy provides an early warning system by finding key nuggets of market intelligence or spotting important trends or patterns in published articles or criticism," he explains.
According to Roy Lipski, managing director of Infonic, Ltd, (http://www.infonic.co.uk), "In order to adapt correctly and act decisively, a company needs timely feedback. Many of the traditional market research and media monitoring methods are simply too slow," he says.
For decades, corporations and other organizations have utilized traditional clipping services such as Bacon's (http://www.baconsinfo.com) and Burrelle's (http://www.burrelles.com) in the US; Bowden's (http://www.bowdens.com) in Canada; and Durrants (http://www.durrants.co.uk) and Tellex (tellex.press.net) in the UK. Their readers scan thousands of newspapers, business and trade journals, and broadcasts each day, clip pertinent articles, and mail or fax clips to clients. By clipping major publications worldwide, the services help companies measure the effectiveness and reach of corporate public relations activities, provide information on brand image, and glean insight into corporate reputation.
Most traditional print publications now publish on the Internet--often with more and different content than in the print version. "Monitoring online publications is just as important as monitoring print publications--maybe more important because, today, business buzz starts on the Internet, and Web publications wield enormous clout," says Comcowich of CyberAlert. "If you doubt it, reflect on the repercussions of an article in The Drudge Report about the activities of a certain White House intern."
In pre-Internet days, effective market intelligence and reputation management could be achieved by tracking a few thousand publications. On the Internet, it's not just traditional publications that need to be monitored; anyone and everyone has the ability to publish information--accurate or inaccurate--and distribute it worldwide with few, if any, controls.
There are literally millions of Web sites where crucial market intelligence and criticism can appear, available for viewing by over 100 million wired consumers--and the Internet is growing by two million pages a day. NEC Research currently assesses the Web at 1,500 million Web pages, an 88 percent increase from 1998. IDC predicts the number to hit 8 billion in 2002, exceeding the world's population.2
"In today's world, business buzz appears in the most unlikely places on the Internet, spreads rapidly through cyberspace and then finds its way into more traditional publication channels." says Comcowich. "The unhappy customer who once wrote a private letter to the CEO expressing outrage about customer service can now publish that complaint on the Internet, making it available for viewing by over 100 million people worldwide, including attentive reporters from print publications and TV. And, unlike a 'letter to the editor' that appears in one print publication on one day, a complaint on the Internet persists for months, and often is replicated in many other Web sites, news groups, or e-mail discussion lists. With this wide-open publication channel read by millions, it is virtually impossible for corporations to conceal problems."
Case Study: Financial Services
A credit card company in the United States was assailed on the Internet by customers and consumer advocates for offering low promotional interest rates and then suddenly raising rates. To the company, this was the standard way of promoting new accounts with the terms of agreement published on the back of the credit card application. Many customers, however, perceived the offer as an illegal "bait and switch" tactic. Many other customers charged (some with substantial evidence) that the credit card company intentionally delayed crediting payments--and then charged a hefty late fee. A pattern of these complaints persisted over a period of at least four months. Affected customers published at least two different "attack" Web sites. The company did little or nothing to rectify the perceived problems. After a short time, the complaints found their way into feature articles in newspapers, magazines and prime time television news shows--thus gaining a larger audience and greater credence. The buzz started on the Internet, moved to more traditional media, and found its way to the company's bottom line. In the subsequent fiscal quarter, the parent company reported a significant drop in sales and earnings as a result of slowed growth in new credit card holders and increased defections among existing customers. The company's stock price plummeted on the announcement.
Moral of the story:
After the drop in stock price, the company fired the outsourced vendor for payment entry and adopted more consumer-friendly policies. Online criticism abated, sales and profits improved, but, six months later, the stock price had not recovered.
Many disgruntled customers and former employees have set up Web sites to excoriate corporations and brands. The Yahoo! directory lists over 300 "consumer opinion" sites that criticize major corporations including Advanta, Allstate, American Express, AOL, Bell Atlantic, Carnival Cruise Lines, Chase Manhattan, Citibank, First USA, Ford, GM, Home Depot, Macy's, McDonald's, Microsoft, Monsanto, NatWest Bank, Nike, Packard Bell, Prudential Insurance, United Airlines, Wal-Mart, and, yes, Yahoo! itself. The complete list is available at http://dir.yahoo.com/Business_and_Economy/
CompanyEthics.com (http://www.companyethics.com) lists links to sites that use the word "sucks" in the URL. The list includes Abercrombie, Aerojet, Amex, Ameritech, AOL, Ford, GM, Royal Caribbean, and Home Depot.
Protected by the cloak of anonymity and empowered by a worldwide audience, the net-complainers have carried old-fashioned pickets and soapbox tirades into cyberspace, according to Ronald F. Lopez of the law firm Thelen Reid & Priest 3 (http://www.thelenreid.com).
Comcowich estimates that more than 5,000 corporate attack Web sites have been published. Some companies, such as Nike and Microsoft, are targets of multiple Web sites.
Online corporation bashing can be quite sophisticated. A number of sites, for instance, have assembled and published hundreds of complaints about companies and their brands. Others utilize e-mail discussion lists and chat rooms for individuals to vent their anger--often with eloquence. Such well-stated negative opinions can affect attitudes and business decisions of hundreds or thousands of prospects and customers.
Like many of these examples, the vast majority of anti-corporate Web sites originate in customer service problems. Attentive, forthright customer service (and a simple apology) could often prevent customers from creating Web sites that expose corporate faults.
Many anti-corporate sites are extremely well-positioned in search engines being reported in the first 10 to 20 citations, sometimes even ahead of the corporate site. As a result, anti-corporate sites are nearly as "visible" to wired consumers as the corporation's own Web site, giving the angry consumer far greater leverage than in the past.
Consumer Opinion Web Sites
In addition to "basher" and "boycott" sites targeting specific corporations, a new category of "consumer opinion" sites has emerged complaint-specific sites covering a range of industries. Epinions (http://www.epinions.com) avows to help consumers express opinions about products and make better buying decisions. A search engine enables consumers to search a database of complaints on hundreds of products and services. And Epinions pays a royalty to the complainer each time a published complaint is accessed. According to "Forbes," one million reviews and notes have been posted on Epinions since its launch a year ago, about 4,000 per day.4
Epinions is not alone. DooYoo (http://www.dooyoo.co.uk) enables consumers to share their opinions about products and be paid for their opinions. FightBack! (http://www.fightback.com), is the Web site of consumer advocate David Horowitz. National Consumer Complaint Center (http://www.alexanderlaw.com/nccc) enables consumers to complain directly to US federal agencies and is sponsored by a law firm, presumably to spot potential liability cases and class actions suits.
The aptly-named Ecomplaints.com (http://www.ecomplaints.com) was born out of the frustration of Jennifer Biscoe, a former Gartner Group consultant who fruitlessly battled AT&T for a year over a disputed cell phone contract fee. Biscoe touts the site as more than just a place to whine, urging companies to post replies so potential customers can see how they handle problems. So far only 38% of the firms have bothered to defend themselves.5 Airlines occupy six of the 10 top spots of complained-about companies.
Planet Feedback (http://www.planetfeedback.com) provides the name and address of the CEO and the head of customer service for all major US corporations, facilitating consumer complaints. Sucks.com (http://www.sucks.com) is owned by Dan Parisi, who has spent $100,000 to register the domain names of more than 500 of the world's largest corporations plus the sucks.com suffix.6
Many perfectly legitimate sites include message boards for consumer opinions. Amazon.com (http://www.amazon.com) allows consumers to write book reviews and thousands of other sites include similar message boards where consumers and employees can vent. Country-specific and industry-specific complaint sites abound; there are at least six complaint sites and a Usenet news group (alt.flame.airlines) devoted exclusively to the airlines industry.
These sites undoubtedly impact consumer attitudes and behavior. According to a recent survey by Forrester Research, sixty-five percent of community users rate the opinions of other consumers as important or somewhat important influences on their purchasing decisions.7
A comprehensive list of "consumer opinion" sites can be found in the Yahoo! directory (http://dir.yahoo.com/Business_and_Economy/Shopping_and_Services/Consumer_Advocacy_and_Information/Consumer_Opinion/) or enter "consumer opinion" in the search box.
In Usenet, a dozen or more news groups are devoted to consumer complaints. The most prominent are alt.consumers.experiences and misc.consumers.
While "pay for whine" policies in consumer sites may encourage unwarranted kvetching, the consumer opinion sites do provide corporations good insight into customer concerns and criticisms, especially focused on customer service. "I would only hope the companies named in those complaints are monitoring the site to see what folks are saying about them," comments Edward Baig, columnist for "USA Today".8
With a plethora of places where consumers can express their opinions or vent their rage, the Internet also functions as a worldwide electronic grapevine that fosters rumors, innuendos, gossip, hoaxes, intentional misinformation, and stock manipulation. The Internet makes it easy to spread misinformation quickly and widely.
Here are some of the infamous false "urban legends" spread on the Internet:
All not true--but these rumors and hoaxes were seen by tens of thousands of people on the Internet and believed by many--with potentially serious consequences to the brand, corporation, or individual. It is estimated that over three-quarters of Fortune 1000 companies have experienced some type of brand abuse on the Internet.
The Urban Legends Reference Pages (http://www.snopes.com), researched and written by the husband-and-wife team of Barbara and David Mikkelson for the San Fernando Valley Folklore Society, tracks the incidence and content of such urban legends on the Internet.
Pronouncements in Web sites and message boards have become fodder for legal action. Credit Suisse First Boston (CSFB) filed a suit seeking $1 million in damages against a New Jersey man and ten others who posted comments on a Yahoo! Finance message board criticizing one of CSFB's investment research analysts. Raytheon filed and then withdrew a suit against a handful of their own employees alleging unauthorized publication of confidential material on the Internet. Many companies have sued owners of attack sites, seeking removal of the critical material. In most situations, first amendment rights protect the rogue Web sites.
Applications of Internet Monitoring
Not knowing the buzz on the Internet about your company and its products can be costly--disaffected customers, lost opportunities, damaged reputation, lost sales, and reduced profits. Companies must be aware of critical information disseminated throughout cyberspace about their products and product categories. In general, the longer negative buzz persists, the greater the damage. Knowing the buzz is the first step in developing an effective online reputation management, market intelligence, or competitive intelligence program for a corporation or brand.
Public Relations, Market Intelligence and Reputation Management
Like traditional public relations clipping services, Internet monitoring and news retrieval services monitor the contents of major news sources, "clip" those articles containing the company or brand name, and deliver the "clips" enabling companies to assess market penetration and effectiveness of specific news releases and public relations programs.
In pre-Internet days, effective market intelligence and reputation management could be achieved by tracking a few thousand publications. In the conventional press, most articles (with the exception of letters to the editor) were written by professionals, were reviewed by editors, and adhered to well-established journalistic standards. While "bad press" always existed, critical articles maintained decorum.
On the Internet, anyone can be a publisher with little or no journalistic training, no editors, and few guiding standards. Now, there are millions of other Web publications and other Web sites that disseminate information and express opinions not just the few thousand traditional publications. Criticism is much more volatile on the Internet and information tracking has become significantly more difficult.
In the words of Ian Mount, staff writer for eCompany.com (http://www.eCompany.com), "Bad press, scurrilous rumors, and outright lies have always been a fact of life for corporate-image watchers, but the Internet makes it possible to distribute that grief more quickly and widely than ever before. From the standpoint of any company concerned about its online reputation, the Web is a monitoring nightmare."9
Debbie Forward of Cayenne Communication, LLC (http://www.cayennecom.com) explains that she uses an Internet monitoring service, CyberAlert, to keep track of the press her clients receive in Web sites and Usenet News Groups. "CyberAlert saves me a lot of time," she says, "finds more coverage on my clients than I've gotten placed myself, and provides very timely reports." As a result, she says, she is able to concentrate on strategic issues and advising on how to manage potential problems while the automated Web clip service does the grunt work of finding pertinent clips that otherwise would not be identified.
Savvy public relations professionals track the media's coverage for both clippings of news releases and any other information that affects corporate reputation or brand image in traditional publications and throughout the Internet.
Internet monitoring is especially helpful in situations of crisis management. Major news outlets often publish first on the Internet and consumer opinion appears first on the Internet. Tools to monitor the Internet are easily accessible and Internet clips can be delivered the next day or sooner.
Internet monitoring, then, can be a highly productive strategy to measure effectiveness of public relations activities, to assess corporate reputation, to gather market intelligence, to protect brand equity, and to help guide management during crises such as the Bridgestone-Firestone tire recall.
Competitive Intelligence, Business Development, Issues Monitoring
"Competitive intelligence is one of the hottest areas in management today," according to Jonathan Calof, associate professor at the University of Ottawa's Faculty of Administration, and chair of the Canadian Council for Competitive Intelligence. "It is used to identify opportunities and minimize surprises."10
Internet monitoring can gather strategic or tactical intelligence. According to Arik Johnson of AuroraWDC, "Strategic analysis is concerned mainly with ... gaining an understanding of a competitor's future goals, current strategy, assumptions held about itself and the industry, and capabilities. Tactical intelligence...includes competitors' terms of sale, their price policies and the plans they have for changing the way in which they differentiate one or more of their products from yours." 11
Competitive intelligence (CI) on the Internet can include:
Monitoring of broader business, political or social issues, and trends is an equally important component of an Internet intelligence program. This involves clipping of editorials, columns by opinion leaders, and op-ed articles in general news publications, trade journals, and activist Web sites.
"CI's real value is to provide managers with the organizational tool to learn what the competitor will do, not what the competitor has already done," writes Johnson of AuroraWDC9. Internet-based information can deliver that capability.
Internet monitoring can also be useful in counterintelligence that is, defending company secrets by helping assure that employees, ex-employees, or suppliers are not inadvertently or maliciously disseminating confidential company information on the Internet. (Through Internet monitoring, a bank discovered recently that a former employee had revealed the password to a server containing a proprietary set of software tools.) During a panel discussion of public relations experts convened by PR Week (UK), David Phillips, managing director of Internet Reputations Services in Milton Keynes, UK and author of "Managing Reputation in CyberSpace" maintained that companies must guard against increasing porosity -- by which he meant the growing danger of confidential information being leaked out onto the Internet by employees.12 Not surprisingly, a company's own publications and Web sites can also reveal critical information.
Intellectual Property Infringement
"The amount of intellectual property infringement on the Internet is astounding," says CyberAlert's Comcowich. "Software, music, photography, and written copyrighted materials are copied blatantly without payment or attribution and seemingly without guilt."
Corporate and product trademarks are misused and abused sometimes naively, sometimes maliciously. Owners of pornography and other less-than-pristine Web sites use the names of established companies and brands in HTML meta-tags to divert Web traffic to their own sites. Promoters "borrow" Web domain names for spam e-mail campaigns to sell gray market, black market, or unapproved products.
Internet monitoring can identify culprits sites that are pilfering or facilitating the theft of intellectual property. Since most monitoring software is text-based, it is more difficult, but still possible, to monitor for infringements of photography or graphics. Emerging digital fingerprinting software will give corporations enhanced capabilities in spotting infringements within graphic, audio, and video files.
For most companies, marketing on the Internet involves alliances, partnerships and independent dealerships. The Internet is replete with organizations or e-commerce sites selling black market or counterfeit products (e.g., Rolex, Gucci, Hilfiger). Approved distributors and dealers often misrepresent, mis-position, or mis-price products of their manufacturing partners. Comprehensive Internet monitoring can identify Web sites that claim affiliation with your company and can track online activities and promotions of channel partners to assure that they are up-to-date and comply with your company policies for promotion and online sales of your products. It is also possible to monitor consumer opinion on the Web about your existing or potential partners and dealers. Such monitoring can minimize regulatory compliance issues, prevent legal proceedings, and reduce channel conflict.
Value of Internet Monitoring
Since buzz about business generally starts on the Internet, comprehensive Internet monitoring can provide early warning of market intelligence, emerging issues, perceptions, and criticisms.
"View the Internet as a vast, completely uninhibited focus group that can provide insights you'd never get any other way," advises Alan Pell Crawford, a senior counselor at Martin Public Relations in Richmond, VA. "Proper use of the Net allows you to know what your customers are saying about you and, if you are properly prepared, to respond in e-time to...problems."13 But Internet monitoring is much more than an online focus group.
A major sporting goods company used Internet monitoring to uncover an activist group planning a rally to protest the company's employment practices and boycott the company's products. The early "heads up" enabled the company to brief the press in advance. As a result, press coverage of the demonstration was even-handed, reporting the company's well-constructed defense along with the activists' criticism.
Through monitoring Usenet News Groups, an international banking company in the UK became aware of a serious image problem and major dissatisfaction with its account services among university students. The bank addressed the problems at the start of a new term and was able to stem the drop-off in new student accounts.
A pharmaceutical company discovered physicians in e-health message boards advocating unusual and sometimes dangerous uses of the company's anesthesia products.
A financial company found out that its paid "run of site" banner advertisements in a "community" Web site were being placed on personal pages containing pornography.
A soft drink company discovered that a pornography site was using its trade name to attract traffic to its site.
An insurance company discovered that a licensed agent was grossly misrepresenting features of the company's life insurance product on his Web site.
A test equipment manufacturer was able to uncover engineering specifications for a competing product before its launch, and was thereby better prepared to defend the market position of its own product. (The information was published in the competitor's own Web site.)
Monitoring can also help identify inappropriate or improvable business practices.
A business-to-business (B2B) software company was able to identify specific customer dissatisfaction with its new product shortly after launch and provide a software "fix" within days.
A series of Usenet News Group postings tipped off Hewlett Packard that customers mightily resented that HP didn't have a toll-free telephone number for customer support.
Through CyberAlert clips, a manufacturer of computer hardware peripherals learned that an independent dealer was selling refurbished hardware as new and was refusing to honor warranties.
A manufacturer of sporting goods identified more than a dozen Web sites that were promoting its products, but were not authorized dealers.
A manufacturer of upscale watches identified Web sites fraudulently selling goods with its trademark.
According to Amy Jackson of Middleberg Euro (http://www.middleberg.com), acclaimed by Inside PR as the "No. 1 Internet Agency" for 2000, "Web clippings are key building blocks to a good image management strategy."
Internet Monitoring Tools
The volume of information on the Internet is astonishing. Traditional publishers are migrating to the Web in droves. The Internet is growing by two million pages a day. Included in those pages are vast numbers of rumors, innuendo, misinformation, and gossip all kinds of negative information that can severely affect a company's reputation. Because of the Web's potential for dishing up harmful information, companies find it crucial to monitor for detrimental postings or discussions in order to take immediate action--in some cases, legal action--before severe damage occurs.
Search engines and directories are the most evident tool for monitoring the internet.
Competing search engines and directories (e.g., Yahoo!, Lycos, AltaVista, InfoSeek, Go, Google, and Northern Light) monitor only a fraction of the Web. An often-cited 1998 study in Science by Steve Lawrence and C. Lee Giles of the NEC Research Institute14 concludes that only forty percent of the Web was available when using a combination of some of the most well-known search engines of the day. The same authors published in a 1999 study for Nature [http://www.wwwmetrics.com] maintain that search engine coverage had decreased to sixteen percent.15 Google (http://www.google.com), the search engine with the most coverage today, now claims to cover more than 1 billion Web pages, while other major search engines hover between 50 and 500 million pages.16 Competing search engines and directories compile remarkably different results for the same keyword search--varying sometimes up to 50 percent in the number of citations.
A key shortcoming of search engines: they do not sort out NEW documents. If you search today and find 900 documents with your brand name and then search tomorrow and find 901 documents, most search engines provide no reasonable way to find which one of the 901 documents is new. In addition, search engines cannot provide an archive of documents that once existed on the Internet, but have been removed--such as news releases.
Most importantly, the public search engines rarely search the daily news media. Even the most advanced search engines index the Web sites of newspapers, magazines, and television stations only once every six to twelve weeks. Newspapers often remove news stories from their site after a few days. That's one reason why results from public search engines rarely include references to daily newspapers in the list of citations they produce.
Manual monitoring using public search engines alone is less than thorough, lacks the required timeliness, and is extraordinarily labor intensive. The costs of professional hours devoted to searching for citations are high, though well-hidden in base salaries. Finally, manual monitoring with search engines is tedious work at best a task much better suited to software automation.
Database Information Aggregators
Established information aggregators such as Dialog (http://www.dialog.com), Factiva (http://www.factiva.com), and Lexis-Nexis (http://www.lexis-nexis.com) also offer some version of electronic monitoring and clipping services by paid subscription. The information retrieval services provide "clips" from databases containing licensed information from a wide range of publications and other sources.
For close to 30 years, these companies served an extraordinarily valuable purpose by enabling electronic access to the archived information of multiple publishers and database developers. For current news and information monitoring, however, they have significant disadvantages. First, like search engines, many of the publications incorporated into these databases have date lags that can range from one to more than six weeks. A Lexis-Nexis search of the publication Cable World, for example, indicates that articles from the June 19, 2000 issue were uploaded into the database system on July 18, 2000. Using the Lexis-Nexis clipping service puts those monitoring the cable industry behind the times on stories of interest.
The second key drawback: these services lies fail to monitor the Web.
Finally, there is the issue of costs. Today, many newspapers, magazines and journals that supply their copyrighted material to database aggregators also publish their information in their own Web sites on the Internet. Therefore, much of the information accessible through electronic database services at steep subscription fees is also available free of charge on the Internet and potentially accessible through public search engines. On the positive side, the aggregation services add substantial value by incorporating significant database information not readily accessible on the Internet. They also store information for longer periods than most Internet sites.
The online information services such as Dialog and Lexis-Nexis continue to fill an important niche in finding older, more esoteric information. However, they are not the best resource for staying on top of current news and information about corporations, brands, or issues.
Electronic News Services
In the early 1980s, the MIT Media Lab conceived the idea of a personalized daily news service that would deliver news only on specific subjects requested by an individual reader. That visionary concept has been realized in Internet-based customized news services such as NewsEdge (http://www.individual.com), MyYahoo (my.yahoo.com), and MyLycos (my.lycos.com). -- A fairly new entrant is Moreover.com. Like the online information aggregators, the Internet-based news aggregators license copyrighted material from publishers who deliver new material as it is published. The typical online service licenses articles from between 200 to 1,200 news sources. The clients of the customized news service specify the news topics that they would like to receive. Each day, the online news service then automatically delivers to the user the top 10 or so articles on the viewer's selected topics.
Many corporations subscribe to electronic news aggregation services and make them available to key employees on the desktop or through the company's intranet.
These Internet-based news services provide a no-cost or low-cost way for businesses and consumers to monitor key topics in the major news media. Their shortcomings as a reputation management and competitive intelligence tool include limits on the number of news sources that are included, the number of stories that are delivered each day, and the scope of thearticles. The automatically delivered stories are confined to news; the services rarely deliver editorials or other opinion pieces such as product reviews. Stories in regional publications almost never make it into the Internet-based news services.
While Internet-based news aggregation services provide businesses with timely news, these services are unlikely to provide the breadth of monitoring that is required for a comprehensive reputation management, infringement monitoring, or competitive intelligence program.
Internet Monitoring and Clipping Services
Because of the limitations of search engines, online information aggregators, and Internet-based news aggregators, a new class of Internet-based application service provider (ASP) has emerged -- Internet monitoring and clipping services. These highly automated services are designed to provide comprehensive and timely "clips" on customized topics to meet the information needs of public relations, marketing, market research, competitive intelligence, and legal departments for infringement monitoring. The services are also used by business planning departments, information librarians, and other corporate functions.
Web clipping and monitoring services capture the most current information from Web publications, electronic bulletin boards, Web sites, and news sources. Companies can employ these services to track mentions of company activities, brands, and trademarks, as well as those of competitors. It is possible to monitor any subject of consequence daily, such as key topics that center on trends, regulations, legal issues, and thought leaders.
Automated services such as CyberAlert (http://www.cyberalert.com) employ new, proprietary search and filtering technologies to find and capture required data, information, commentary, and intelligence throughout the visible Internet including thousands of Web publications (news syndicators, newspapers, magazines, journals, and online-only publications, also called E-zines or Web-zines); millions of commercial, academic, and government Web sites; and tens of thousands of Usenet News Groups, message boards, and forums.
These services provide the comprehensive, timely coverage required for effective reputation management, competitive intelligence, and infringement monitoring programs. In addition to CyberAlert (http://www.cyberalert.com), monitoring and clipping services include: eWatch (http://www.eWatch.com), a service of PR NewsWire, Bacon's NetClips (http://www.baconsnetclips.com), a service of the old-line print clipping service, Cyveillance (http://www.cyveillance.com), which focuses on infringement monitoring, NetCurrents (http://www.netcurrents.com) and eSleuth (http://www.esleuth.com), which concentrate on financial information on publicly traded stocks. NetCurrents includes analysis; eSleuth does not.
Web clipping companies provide a range of useful benefits that broadly include:
The new Internet monitoring and clipping services differ from public search engines in a number of important ways:
The monitoring services also offer advantages over electronic news aggregators:
Internet monitoring services are not without problems. The spidering and indexing software of the monitoring services often takes a few days or weeks to filter out all archival references on the Internet. As a result, clients may receive older citations during the early stages. This is a temporary problem that usually clears up in the first few weeks.
Some services do not have the software to deploy complex search strings and, therefore, cannot deliver finely grained search results. Some services deliver weekly or monthly results, rather than daily, thus undercutting the timeliness of Internet monitoring. ironically, some Internet monitoring services deliver results in print via US mail ("snail mail") or fax, instead of electronically via the Internet (although many traditionalists still prefer paper-based news clipping services.)
No monitoring and clipping service can guarantee that it will find all articles or postings. (The URL of an article combined with an abstract is called a "citation" by some Internet monitoring and clipping services.) Most Web monitoring services do not deliver articles from Web sites of major news organizations requiring subscriptions, such as the Wall Street Journal. Oftentimes, the services' spidering and indexing software is unable to search Web sites such as the New York Times that block searches by automated robots. Sometimes, the software of Internet monitoring services cannot discriminate substantive articles from unimportant listings, resulting in unwanted "clips" or citations.
Services that monitor Usenet news groups inevitably deliver clips that contain profanity. Monitoring commercial Web sites may result in clips from pornography sites if those sites contain key words in the client's search string. (One medical company conducting a search on the word "stroke" was inundated with citations from pornography sites.)
Despite shortcomings, the services are today's most valuable tool for Internet monitoring, contends Comcowich of CyberAlert. "Automated Internet monitoring services combine the best attributes of search engines (in-depth coverage of the visible Internet), electronic news aggregators (timely coverage of news sources), and traditional clipping services (delivery of all found articles) into a single automated, time-saving, and customized information retrieval service that is both comprehensive and timely."
How Monitoring and Clipping Services Work
To use an Internet monitoring and clipping service, a client establishes an account and gives the customer or technical service department the key words, topics, company, or brand names to be monitored. The service then creates a search strategy (including the areas of the Internet to be searched) and search specification (a "string" of words to be searched, often called a "brief" in Europe). The most advanced Internet monitoring services enable complex Boolean-type search queries such as -- (Monsanto or Pharmacia) and (biotech or genetic or food or seed or farm or agriculture) and not Roundup. Or (AT&T or "Bell Atlantic" or Sprint or MCI) and (marketing or advertis* or "public relations") Or +Orange +mobile -/\Rorange alabama -california -florida -"mini mart" clockwork -juice In the AT&T example, truncation of "advertis*" will pick up various word forms of the root word [Editor's Note: such as advertisement and advertising--taking off the "s" at the end will broaden it to include "advertize"].
The "Orange" example, taken from CyberAlert 3.0, which uses set logic syntax in its search definitions, illustrates how the most powerful software systems can hone in on narrow subject areas while minimizing spurious results. The search is for a mobile phone company in the UK named "Orange." The basic search is for "Orange" and "mobile" two common terms that can be used in myriad contexts, including colors, flavors, refreshments, cities, counties, and retail outlets. In this example, the + sign means "mandatory" so both "Orange" and "mobile" must appear in the article. The -/\R means to ignore the word "orange" if the "o" is lower case. The sign means to ignore any article which contains the words "alabama," "california," "florida," "mini mart," or "clockwork" thus eliminating references to city, county or book names containing the word "orange." The result is a daily list of citations with few unwanted clips. When a client finds that a search specification produces too many irrelevant clips, it can be changed using proximity operators. Proximity capabilities in CyberAlert 3.0 include finding terms within the same line, sentence, paragraph, or Web page.
Each day, the monitoring software indexes the contents of selected Web sites and then searches the index for customer-specified terms. The service delivers a list of citations for new "clips" found during the previous 24 hours that match the client's search terms. The filtering software to deliver only relevant new citations is one of the key value-added features of Internet monitoring services. Each monitoring service uses different technology to identify new clips and assess their relevancy. CyberAlert 3.0 triple-filters each citation to minimize the incidence of irrelevant citations.
Most public search engines, electronic news aggregation services, and Web monitoring services do not allow the complex, finely grained searches of the advanced software in CyberAlert 3.0.
Each citation includes an abstract of the article and a hotlink to the URL of the full-text article. CyberAlert and a few other services also provide clients with access to a password-protected Web site where clients can access their clips. CyberAlert incorporates a clip management system with a database that is fully searchable by keyword, date, and news source. This integrated clip management system enables clients to drill down for more finely grained market intelligence buried in their customized database of clips.
Typically, clients adjust their search criteria a number of times during the first few weeks of service to obtain the most relevant results for their specific needs.
Selecting a Monitoring Service
Services for Internet monitoring and clipping vary widely in their capabilities, features and sophistication. They range from purely manual services that rely on live editors using public search engines to fully-automated services using advanced, proprietary technologies. The automated services deliver more comprehensive and timely results than manual ones.
Key criteria in selecting a Web monitoring service include:
Integrated Internet Monitoring Programs
An automated Internet monitoring and clipping service should be the backbone of a comprehensive corporate Internet monitoring program for reputation management, competitive intelligence, or infringement monitoring. The best of the automated services combine breadth and depth of coverage with timely delivery of citations. With significant advantages in effectiveness and timeliness, the automated services are clearly superior to manual searching with search engines or use of older online information aggregation services such as Dialog or Lexis-Nexis. Internet monitoring and clipping services are usually less expensive than assigning staff to the time-consuming and tedious task of finding news and other articles on the Internet about a company and its products.
However, even the best monitoring and clipping services may need to be supplemented with other services for certain corporate applications. For reputation management, one of the Internet-based news services such as Work.com may provide access to news sources (such as the Dow Jones publications) that may not be covered by an Internet monitoring service. Online database information retrieval systems (e.g., Dialog, Lexis-Nexis) continue to be worthwhile for finding archival material or database information for market research, for example. Lexis-Nexis is still the gold standard for searches involving legal information. An investor relations service such as Company Sleuth (http://www.company.sleuth.com) can be of considerable assistance in monitoring financial message boards such as Silicon Investor, Motley Fool, Yahoo! Finance, and others. At least for the time being, conventional print clipping services such as Bacon's continue to serve a valued role in monitoring print publications.
Independent information professionals, including MarketingBASE and Infonic, or public relations agencies such as Cayenne Communications, Hiron's, or Middleberg Euro, can provide monitoring services using search engines or automated solutions such as CyberAlert. The ideal solution is to deploy CyberAlert or another automated Web monitoring tool for mining the Web (thereby minimizing the amount of time required to gather information) and leave the more complex analytical work to an information professional or public relations consultant.
Comparing Internet Monitoring Services
In an evaluation of Internet monitoring and clipping services that appears in "Web Wise Ways" for Searcher Magazine (http://www.infotoday.com), MarketingBASE found CyberAlert 3.0 more useful than eWatch.17
CyberAlert delivers a daily e-mail report containing new citations from the previous 24 hours. Each new citation has a hotlink to the original source and a key-word-in-context (KWIC) abstract of the article with the key words highlighted. During a two-week trial, CyberAlert was quite reliable, delivering results each day, even on weekends.
CyberAlert uses Boolean and set logic for search definitions and allows for nesting and proximity operators. Proximity capabilities include, for example, finding terms within the same line, sentence, paragraph, or web page.
At the present time, CyberAlert searches 2,700+ Web publications worldwide, with 50 to 100 new Web sites added each week. These cover the major news syndication services, newspapers, magazines, journals and online publications on the Internet. CyberAlert 3.0 groups publications into media categories based on geography or topic of interest. CyberAlert will add sites suggested by customers at no extra cost. Since some Web sites use "robot.txt" files to block automated searching, CyberAlert respects their wishes and avoids those sites.
All CyberAlert customers get access to a private, password-protected Web site that stores every clip in a database from the time a client begins using the service; each clip is saved for the life of the account. Customers can organize the clips into "folders" specified by the customer. Like traditional print news clipping services, CyberAlert can deliver a full-text copy of the clip upon customer request.
MarketingBASE found the clip management system particularly handy; it's no longer necessary to review clips daily--and because the Boolean search engine enables deeper cross-referencing. Within this clip database system, the customer can search at any time for additional key words. It is also possible to sort clips by date or media source. CyberAlert also enables each user to forward a particular citation or even a whole folder of clips to another person.
Similar to CyberAlert, eWatch provides a daily alert and Web access, but offers fewer features and options by comparison, according to MarketingBASE.15 The e-mail alerts do not hotlink to the URL cited. eWatch saves results for only two weeks. At this writing, eWatch does not include a working search function, although the user instructions describe a search capability. eWatch does include some Web services not currently offered by CyberAlert, including monitoring investor relations message boards such as Motley Fool and Silicon Investor. Company Sleuth provides the same services free of charge--but only for public companies.
MarketingBASE was unable to test Cyveillance, which delivers a monthly analytical report, compiled by its editors. The Cyveillance service is more geared to identifying schemes that involve traffic diversion, counterfeit products, and inappropriate product promotion by channel partners.
Among the three services examined, CyberAlert seems to offer the best value and most flexible terms. At the time of the MarketingBASE article, CyberAlert charged $395 per month per search string on a month-to-month agreement with quantity discounts to $295 per month. The eWatch charge is based on numbers of users and is on an annual contract basis. Pricing of Cyveillance begins at $80,000 annually.
Summary and Conclusions
The burgeoning Internet has created a new information and communications revolution. Almost every industry and product category is changing with lightning speed as both dotcom and brick-and-mortar companies enter e-commerce and consumers exploit Web benefits. In the Internet economy, new companies must find novel ways to succeed, and established firms must innovate to survive. The race to market and growth in competition require innovative ways to find, gather, and process information needed for business planning, reputation management, and protection of brand equity.
As a result, automated Internet monitoring services are rapidly becoming an indispensable tool for reputation management, competitive intelligence, and infringement monitoring for most companies.
By combining the best attributes of search engines (in-depth coverage of the visible Internet), electronic news aggregators (timely coverage of news sources), and traditional clipping services (delivery of all found articles) into a single, automated, time-saving, and customized information retrieval service, the automated services are quite clearly the most effective and time-efficient tool now available for Internet monitoring and Web clipping. On the whole, Internet monitoring and clipping services, such as CyberAlert, are more effective and less expensive than assigning staff to find news and other articles about a company and its products on the Internet. With their relatively low costs starting at about $300 per month -- they are an affordable productivity tool for almost any sized company in almost any industry or business, almost anywhere in the world.
Mining information on the Internet is a vital component of a well-designed reputation management and market intelligence program. However, finding information on the Internet is only the first step. For that information to be useful in corporate decision-making, it must be processed stored, organized, evaluated, analyzed and then reported to key decision-makers up to and including the CEO in an actionable form. Well-formed intelligence will lead to better-informed decision-making. For virtually all companies, a cohesive and well-executed program of Internet monitoring and intelligence will undoubtedly lead to better business performance.
1. Fraud on the Internet. Business Week e.Biz Special Report. April 3, 2000. Available at: http://www.businessweek.com/2000/00_14/b3675038.htm. Accessed August 25, 2000.
2. Lake, D. The Web: Growing by 2 million pages a day. The Standard. February 28, 2000. Available at: http://www.thestandard.com/research/metrics/display/0,2799,12329,00.html. Accessed August 23, 2000.
3. Lopez, RF. Corporate strategies for addressing Internet "complaint" sites. Thelen Reid & Priest, LLP, Web site. Available at: http://thelenreid.com/articles/article/art_49_idx.htm. Accessed August 26, 2000.
4. Shoenberger, CR. The opiners, Forbes, September 4, 2000:123 Available at: http://www.forbes.com/forbes/00/0904/6606123a.htm. Accessed August 26, 2000
5. Blakeley, K. The whiners, Forbes, September 4, 2000:122 Available at: http://www.forbes.com/forbes/00/0904/6606122a.htm. Accessed August 26, 2000.
6. Farrell, G. From sour grapes to online whine. USA Today. April 7, 2000. Available at: http://www.usatoday.com/life/cyber/tech/cth701.htm. Accessed: August 10, 2000.
7. Solomon, K. Customer reviews: It's all a matter of opinion. The Standard. July 11, 1999. Accessed August 26, 2000.
8. Baig, E. Appreciating the value of a fine whine. USA Today CyberSpeak. Available at: http://www.usatoday.com/life/cyber/ccarch/cced020.htm. Accessed on August 26, 2000.
9. Mount, I. PR disasters and ways to avoid them. July 2000. eCompany.com Web site. Available at: http://www.ecompany.com/articles/mag/1,1640,6760,00.html. Accessed August 26, 2000
10. Calof, J. Increasing your CIQ: The competitive intelligence edge. Economic & Technology Development Journal of Canada. 1998; Available at: http://www.edco.on.ca/journal/item22.htm. Accessed August 18, 2000.
11. Johnson, A. What is competitive intelligence? AuroraWDC.com Web site. Available at: http://aurorawdc.com/whatisci.htm. Accessed August 24, 2000.
12. http://www.yousuck.com, PR Central, Reputation Management, September/October 1999 Available at: http://www.prcentral.com/repman99/rm_cover_sep99.htm. Accessed: July 2000.
13. Gray, R. PR and the Internet. PR Week (UK). February 11, 2000. Available at: http://www.prweekuk.com/uk/bprac/internet.htm. Accessed August 22, 2000.
15. Lawrence, S., Giles, CL. Accessibility of information on the Web. Nature. 1999; 400:107-109.. Available at: http://www.neci.nec.com/~lawrence/websize.html. Accessed August 26, 2000.
16. Sullivan, D. Search engine sizes. Search Engine Watch. July 7, 2000. Available at: http://www.searchenginewatch.com/reports/sizes.html. Accessed August 12, 2000.
17. Kassel, A. The last word on monitoring and clipping services. Searcher Magazine. September 2000; 8:8, 22-35. Available at: http://www.infotoday.com/searcher/sep00/kassel.htm. Accessed August 26, 2000.
Social networking icons by komodomedia.com.
Site copyright © 2000-2011 by Shel Horowitz