How to Use Secondary Research Surveys To Manage Your Firm: A Primer

 

By Dr. John W. Olmstead

Copyright 1996-2002 by Olmstead & Associates. All Rights Reserved.

The 21st century is presenting law firms with new challenges. The general business economy is in turmoil and law firms are facing new risks and uncertainties. Clients are no longer tolerating arrogance and mediocre services. Clients are holding law firms to higher service standards. In order to prosper in the 21st century, law firms are going to have to drastically change their models for conducting business. Organizational performance, effectiveness, and leadership must rise to higher standards. General management, problem solving, and action taking skills must be enhanced. Firms will have to improve their overall marketing initiatives. This will require that many firms improve their overall management effectiveness and use every management tool available. Law firms will need to identify “best management practices” that can be employed to enhance management effectiveness.

The Search For Best Practices

Managing partners, executive directors, legal administrators, marketing directors, and practice group leaders express common concerns regarding the lack of benchmark data and general information on “best practices.” Law firm management needs answers to questions such as:

Information on best practices can come from either primary research conducted or commissioned by the firm or secondary research. Primary research is often employed in the form of analysis of internal practice data, client surveys, positioning studies, and sometimes to support strategic planning efforts. However, firm management often wants to know what other firms are doing for “benchmarking” purposes and uses secondary survey research for this purpose.

Numerous organizations offer “off-the-shelf” varieties of secondary research surveys. In the area of law firm management, Altman & Weil’s Survey of Law Firm Economics and PricewaterhouseCoopers’ surveys are noteworthy and have been used by law firm management for years as their primary source of benchmark data. Both of these surveys provide detailed benchmark data in areas such as:

Other law firm management consulting firms offer similar surveys. General management benchmark surveys are available from general management consulting firms and general management professional associations. Surveys on specific topics of interest such as law firm financial performance, salaries and compensation, and marketing budgets are also available from national and local bar associations and law firm management and marketing professional associations such as the Association of Legal Administrators and the Legal Marketing Association. The Legal Marketing Association noteworthy “2001 Survey of Law Firm Marketing Budgets” was published this year.

Research Crash Course

Prior to jumping in and reviewing that survey report, you might want to brush up on research fundamentals. Just as the legal, management, and marketing fields have their own rules, procedures, and vocabulary, so does the field of professional research. Standards and ethical guidelines are specified by professional research associations such as Qualitative Research Consultants Association, American Marketing Association, and the Market Research Association. The Legal Marketing Association has published research guidelines for commissioning and conducting market research.

Neither time nor the scope of this article permits a comprehensive treatment of the fundamentals of sound research. Thus, the following key points will provide an outline of important concepts:

KEY POINT #1:What is the purpose of the research? The most common purposes of research are exploration, description, and explanation. The purpose of the original research, as well as your purpose if using secondary research, will determine appropriate research methodology and how existing research should be used. Exploratory studies often use qualitative research designs. Typically, when the purpose of the research involves comprehensive description and explanation, research designs are quantitative-descriptive designs that require either census studies of the entire population using descriptive statistics or random or quota samples using both descriptive and inferential statistics. In such cases the objective is to be able to make inferences and statements about the larger population of interest. In other words, what can you say about a population based upon the results observed in a random or quota sample?

KEY POINT #2: Know the difference between qualitative and quantitative research and appropriate uses for each. Qualitative methods do not use quantity measurement; but rather attempt to measure the “quality” of something. Examples include case studies and focus groups. Quantitative research attempt to measure “quantity.” Quantitative studies would include statistical market surveys, budget surveys, salary surveys, and financial analyses. Quantitative research is the backbone of clinical research and they yield more “scientific” findings than qualitative studies. Qualitative studies are often used as initial exploratory studies and are subsequently supplemented by quantitative studies.

KEY POINT #3: The user of a research study must understand the difference between a population, census and a sample. A research or study population is the larger group of interest while the sample is the group we draw to study. An example of a population would be all law firms in the United States, while a sample would be a subset of the population that is representative of the overall population. In other words a sample is a set of respondents selected from a larger population for the purpose of the survey. A census is a one-by-one count of the entire population.

KEY POINT #4: Do not generalize or make inferences beyond your data. Know the population of the study and whether a census or representative sample was taken. Limit generalizations to the population of the study or to the specific sample if extensive bias or error exist in sample data.

KEY POINT #5: Keep in mind that sample surveys yield accurate results when the following kinds of errors are avoided:

KEY POINT #6: Have a working knowledge of the following statistical concepts:

KEY POINT #7: Know the meaning and the strengths and weaknesses of the mean, median, mode, variance, standard deviation, and variance. Know the dispersion of your data and how it is skewed. When the distribution of the data is mound shaped (bell-shaped curve) the mode, median, and mean are all equal. The mean is always more sensitive to extreme values and when such is the case the distribution will be more skewed (not a normal bell shaped curve). In skewed distributions, the median is usually a more accurate indicator.

KEY POINT #8: Know the characteristics of your data and whether the report is using the right statistics and graphs to describe the data.

KEY POINT #9: Know when a study is so flawed that it is totally worthless.

KEY POINT #10: Have a plan on how the research will be used to assist the firm in formulating management action initiatives.

Tips For Using Secondary Research Surveys

Secondary research surveys must be used with caution. Many of the surveys that are available as off-the-shelf products fail to meet research guidelines and standards. Some are census studies with extremely low response rates. Others are broader national surveys in which samples were used. However, most often these samples were not constructed as random/quota samples and in many cases the sample consisted of clients, prospects, and contacts of the sponsoring organization as opposed to a representative sample of the study population. Many of these samples have been entirely respondent self-select studies. Consequently, many of these studies are potentially biased and contain various degrees of error.

Follow these tips both prior to purchasing and when using secondary research surveys.

TIP #1: Know your objectives and how you will use the research. Is it exploratory in nature and for basic benchmarking or is it intended for comprehensive description or explanation? This will help you determine which studies will satisfy your needs, how rigorous your standards should be, and how to use and interpret these studies.

TIP #2: Before purchasing secondary research surveys learn all that you can about the details of the study. Know the limitations of the study. Ascertain the following:

TIP #3: If you purchase a survey keep Tips 1 and 2, as well as Points 1-10, in mind as you review and interpret the data contained in the survey.

TIP #4: Look for results that matter.

TIP #5: Look for information that you need to know to initiate changes in your management practices and marketing initiatives.

TIP #6: Review the questionnaire to determine how the questions were initially constructed.

TIP #7: Insure that the right statistics are being used to describe the data. (continuous vs categorical data)

TIP #8: Do the findings make sense?

TIP #9: Identify red-flag areas of concern where your firm may need to investigate further.

TIP #10: Formulate a plan for following up on areas requiring additional investigation and possible action.

The use of sound secondary research surveys can be invaluable and can assist firms in their quest for “best practices.” While law firms should strive to use surveys that meet the test of sound research, this is not always possible since no other source of information may be available. In other words – some information may be better than no information at all. In such situations law firms may decide to use research surveys that do not satisfy sound research guidelines. Such information can still be useful for exploratory analysis and when the information will be used for “benchmark” purposes. However, it is important for the firm to keep in mind the limitations of the study.

Many of the national law firm management surveys are designed to provide the information necessary for law firm management to evaluate their firm’s performance relative to comparable law firms. Statistics included in these studies represent broad performance benchmarks against which an individual firm can be measured. Law firms can use this information to compare their firm’s performance with other firms as a whole, as well as with firms of similar size, geographic location, population, practice specialty, etc. Keep in mind that the objective of such comparisons is to identify potential “red flags” that warrant additional investigation. Deviations between your firm’s figures (for any performance measure) and figures in the survey is not necessarily good or bad. It merely runs up the red flag which alerts you investigate further. Information in these surveys should be used as guidelines rather than absolute standards.

It is important that surveys used for exploratory or benchmark purposes not be taken as the gospel. They may not be representative or they may be contaminated with many of the errors mentioned previously. Finally, valid benchmark statistics may not be appropriate goals for your firm. For example, while several national surveys have reported that 2.4% of revenue is the median budget for marketing investments in large law firms, this may not be the appropriate goal. Large CPA firms are spending closer to 10%.

I hope this primer helps you on your journey.

John W. Olmstead, Jr., MBA, Ph.D, CMC, is a Certified Management Consultant and the president of Olmstead & Associates, Legal Management Consultants, based in St. Louis, Missouri. The firm provides practice management, marketing, and technology consulting services to law and other professional service firms to help change and reinvent their practices. Founded in 1984, Olmstead & Associates serves clients across the United States ranging in size from 100 professionals to firms with solo practitioners. Dr. Olmstead is the Editor-in-Chief of “The Lawyers Competitive Edge: The Journal of Law Office Economics and Management,” published by West Group. He also serves as a member of the Legal Marketing Association (LMA) Research Committee. Dr. Olmstead may be contacted via e-mail at jolmstead@olmsteadassoc.com. Additional articles and information is available at the firm’s web site: www.olmsteadassoc.com

Contact Us

  • The best decision when we were considering succession planning was to hire you and your firm. 
  • Curt Tobin
    Tobin & Ramon

Read More Testimonials »