LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT 17 Data collection methods, sources and the measures used are identified. as customer satisfaction ratings) each have strengths and weaknesses associated with them. The purpose of this article is to examine the definition and measurement of performance in logistics research. We begin with a literature review which includes an examination of the various ways in which “performance” has been defined.
Data collection methods, sources, and the measures that have been used are also identified. Next, potential sources of performance data are identified and discussed. Recommendations arising from the review complete the article. Logistics Performance: Definition and Measurement Garland Chow, Trevor D. Heaver and Lennart E. Henriksson International Journal of Physical Distribution & Logistics Management, Vol. 24 No. 1, 1994, pp. 17-28 © MCB University Press, 0960-0035 Literature Review
The contents of five leading logistics journals between 1982 and 1992 (International Journal of Logistics Management, International Journal of Physical Distribution & Logistics Management and its predecessors, Journal of Business Logistics, Logistics and Transportation Review and Transportation Journal) were reviewed for studies addressing performance. A number of studies from other sources were selected for examination on an ad hoc basis. Summary of the Literature The literature may be divided into six categories.
They are: (1) Conceptual works; (2) Performance definition; (3) Performance measurement; (4) “Leading edge” literature; (5) Performance as an outcome variable; (6) Mathematical/economic analyses. The first category is composed of three articles and three volumes. These conceptual works are listed and summarized briefly in Table I. Armitage studied how management accounting techniques could be used to improve productivity analysis in distribution operations[3], while Mentzer and Konrad reviewed performance easurement practices from an efficiency and effectiveness perspective[4]. Rhea and Shrock presented a framework for the development of measures of the effectiveness of distribution customer service programmes[5]. While some empirical work is reported in Byrne and Markham’s volume, a key contribution lies in its conceptual treatment of measurement issues, particularly performance indicators[6]. The volume by La Londe et al. focuses on customer service[7].
This excellent work includes an ambitious survey of shippers, carriers and warehouse executives, and provides several ideas relevant to measuring customer service performance. A Introduction Logistics research may be defined as the systematic and objective search for, and analysis of, information relevant to the identification and solution of any problem in the field of logistics[1]. A great deal of logistics research is conducted around the premiss that a relationship exists between a particular course of action and logistics performance (or effectiveness).
Unfortunately, drawing broad inferences from the work that has been done is frustrating because of the great variety of ways in which performance has been defined in the literature. The definition of performance is a challenge for researchers in any field of management because organizations have multiple and frequently conflicting goals[2]. Some define goals in terms of profits. Others may choose goals such as customer service or sales maximization.
Also difficult are the tasks of selecting and developing adequate measures for the chosen definition. “Hard” measures (such as net income or accounting figures) and “soft” measures (such The authors gratefully acknowledge the financial support provided by the Social Sciences and Humanities Research Council of Canada (SSHRC). An earlier version of this article was presented at the 1993 International Logistics Congress in Toronto, Canada. 18 IJPD & LM 24,1 Table I. Conceptual Articles
Article/book Armitage[3] Mentzer and Konrad[4] Rhea and Shrock[5] Byrne and Markham[6] La Londe, Cooper and Noordewier[7] Nevem Working Group[8] Summary Focuses on the use of management accounting techniques to measure (and improve) efficiency and effectiveness in distribution operations Reviews measurement practices and suggests methods for improvement Defines “physical distribution effectiveness” and discusses some implications for measurement Focus is on quality; a key value of this book lies in its treatment of performance indicators for various dimensions of logistics Focus on customer service and how it may be measured Comprehensive review of performance indicators in logistics particular contribution of the book lies in its emphasis upon the inter-temporal and multi-dimensional nature of customer service. The volume by the Nevem Working Group is a comprehensive description and application of “hard” performance indicators in the logistics setting[8]. A variety of empirical studies comprise the remaining categories. Tables II – VI list the studies by category, and identify the data source and collection method used, how performance is defined, and whether the measures used for performance are soft or hard. Table II.
Empirical Studies: How Performance May Be Measured Article/book Read and Miller[9] Data collection method Mail survey Data source Managers Measures Soft Definition of performance/remarks Quality: (total customer satisfaction, on-time delivery, zero defects, employee awareness of quality importance, reduction of the cost of quality, best-in-class practices, human resource excellence, satisfaction of industry regulations, employee education, other) Vendor performance (criteria developed using brainstorming method): promised lead-time, leadtime variability, fill rate, discrepancies, total purchases Logistical performance: length of promised order cycle times for base-line/in-stock products, manufacturer’s performance in meeting promised delivery dates, fill rate on base-line/in-stock items, advance notice on shipping delays, accuracy of manufacturer in forecasting and committing to estimated shipping dates on contract/project orders, manufacturer’s adherence to special shipping instructions, accuracy in filling orders Study focused upon performance indicators and found significant variation in logistics efficiency depending upon which performance indicators were used Harrington, Lambert and Christopher[10] Archival Firm Hard
Gassenheimer, Sterling and Robicheaux[11] Mail survey Executives Soft Cooper, Browne and Peters[12] Personal interviews European-based Soft companies LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT 19 Table III. Empirical Studies :Which Performance Measures Are Used Article/book Clarke[13] Yavas, Luqmani and Quraeshi[14] Data collection method Mail survey Mail survey Data source Executives Managers Measures Soft Soft Definition of performance/remarks Productivity (study sought to determine which measures used) Efficiency (study provides data on efficiency measure usage, and attempts to correlate this with a measure of purchasing sophistication developed by the authors)
Four empirical studies focus primarily upon how performance could be measured: their content is summarized in Table II. Read and Miller conducted an exploratory study of quality in logistics, using a sample of firms on a consultant’s mailing list[9]. One of the most important findings of their study is “a clear gap between the importance given to the components of logistics quality, and the measures being used” (p. 36). Harrington et al. developed an interesting model for vendor performance evaluation in a logistics context[10]. A brainstorming method was used to identify the various dimensions of vendor performance, and to evaluate the priority that each deserved. The model was subsequently tested in a health-care setting.
Gassenheimer et al. on the other hand, employed factor analysis to identify the dimensions of “performance”[11]. Their analysis revealed five key dimensions: “logistical”, “boundary personnel”, “product/product support” [sic], “flexibility and innovative” [sic] and “inventory assistance” (p. 20). A principal contribution of the paper by Cooper et al. lies in Table IV. Empirical Performance Studies: “Leading-edge” Literature Article/book Bowersox, Daugherty, Droge, Rogers and Wardlow[15] Data collection method Mail survey and interview Data source Firm Measures Soft Definition of performance/remarks Common attributes index (performance antecedents).
Appendices include attempts to associate firm CAI measures with performance; results revealed few associations that were statistically significant Service capabilities of firm (various logistics and support services) Firm’s ability to accommodate special requests: special service requests, programme support, customized service, modification while in logistics system Firm’s ability to accommodate product introduction, product phaseout, product recall and customization of service levels to specific markets or customers Ability of firm’s logistics system to accommodate: supply disruption, production schedule changes, in-stream product modification, services level customization, incentive programmes, special customer requests, product introduction, product phase-out, product recall and computer breakdown Daugherty, Stank and Rogers[17] Daugherty, Sabath and Rogers[19] Mail survey Mail survey Firm Firm Soft Soft Droge and Germain[20] Mail survey Firm Soft Germain[21] Mail survey Firm Soft 20 IJPD & LM 24,1 Table V. Empirical Performance Studies: Performance as an Outcome Variable or Basis for Comparison Data collection method Mail survey Article/book Fawcett and Closs[22]; Fawcett[23] Data source Managers (rated own organization’s performance) Managers (rated own firm’s performance) Key decision makers Managers Measures Soft
Definition of performance/remarks “Competitive position” of firm or profit centre: five perceptual items rating from nontraditional measures of cost and customer service to more traditional measure of growth in sales and growth in return on assets “Operating performance” of specific logistics dimensions: vendor relations, material acquisition lead-time, purchasing method, material invoice, transport management/ comparison between JIT and non-JIT firms “Distribution effectiveness”: adequacy, consistency, accuracy, timeliness, initiative, responsiveness A single logistics function, carrier performance: on-time performance, transit time, rates and tariffs, accuracy, equipment co-ordination, documentation, information, loss and damage “Distribution service performance”/one-item performance measure Perry[24] Site visits, questionnaires and interviews Soft Rhea and Shrock[25, 26] Fawcett[28]; Fawcett and Vellenga[29] Marr[30] Mail survey Soft Mail survey Soft Mail survey Executives Soft ts use of performance indicators for the purpose of measuring logistics efficiency and effectiveness[12]. Two studies, summarized in Table III, report the results of surveys which sought to identify the measures used by decision-makers in assessing logistics performance. Clarke reports how physical distribution productivity was measured by South Carolina distribution executives[13]. Yavas et al. focused a similar study on the purchasing area. They collected data on the usage of efficiency measures in Saudi Arabian firms and found that utilization of measures was associated with purchasing sophistication[14]. Given the different orientations of the two studies, the findings are understandably quite dissimilar.
In Leading-edge Logistics, Bowersox and his colleagues distinguish between emergent, norm and leading-edge organizations on the basis of common attributes index (CAI) scores [15]. However, few significant relationships were reported between CAI scores and publicly-available financial performance measures [15, Appendix E, 16]. Four performance-relevant studies have been found which build on the Leading-edge material. A summary of these studies (along with the original volume) is presented in Table IV. An important limitation of the CAI approach is that it measures potential, that is, services offered rather than actual logistics performance. For example, Daugherty et al. uggest that the ability of the logistics system to accommodate the customization of service levels to specific markets or customers is an important dimension of performance[17]. They measured performance by asking respondents to identify which “expanded services” were offered to customers (e. g. customer billing/collection; freight bill audit and payment and so on). The authors conclude that “firms wishing to improve logistical performance are well advised to concentrate on formalizing selected processes” (p. 57). Because the dependent variable is potential rather than actual service delivery, this conclusion is best regarded as tentative and in need of further mpirical validation. The limitations of outcome variables which may have little or no direct linkage with actual performance are wellknown[18, p. 543]. For, as Cooper and his colleagues noted, “there can be a large gulf between what companies say they do and what they actually do” [12, p. 30]. Daugherty et al. found that firms that could be distinguished as “leading edge” by virtue of several structural characteristics performed better than others[19]. The authors defined performance as “ease of accommodating special requests”, such as sales and marketing incentive programmes. Again, it is potential, LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT 21 Table VI.
Empirical Performance Studies: Mathematical/Economic Analyses Article/book Clarke and Gourdin[31] Kleinsorge, Schary and Tanner[32, 33] Diewert and Smith[37] Data collection method Archival Data source Firm Measures Hard Definition of performance/remarks DEA – efficiency/productivity (also included evaluation of DEA using questionnaire with soft measure) DEA – efficiency Total factor productivity Profitability: also examined order cycle time, variance in order cycle time and order fill rate Archival Archival Firm Firm Previous research Hard Hard Hard Gomes and Mentzer[38] Simulation not actual performance that is being measured. Similar outcome variables were used in Droge and Germain’s study of the effects of formalization[20] and Germain’s study of the effects of customization and standardization[21]. A summary of the studies in which performance was either investigated as an outcome variable or used as a basis for comparison is provided in Table V.
Three studies investigated variables associated with performance in a fairly direct fashion. Fawcett and Closs used a promising application of path analysis to demonstrate linkages between “perceived globalization”, “manufacturing”, “logistics”, and the outcome variable, “competitive position”[22]. In another study using the same database, Fawcett found that firms that emphasized logistics issues at a higher level in the early stages of the co-production decision outperformed their counterparts[23, p. 39]. In his study of firm behaviour and operating performance, Perry conducted a qualitative study utilizing site visits, questionnaires and interviews with managers.
He found differences between just-in-time (JIT) and non-JIT firms in a number of performancerelated areas: vendor relationships, material acquisition leadtimes, purchasing methods and materials inventories[24]. Rhea and Shrock utilized a combination of previous research and interviews to identify six key elements of logistics distribution effectiveness[25,26]. They found that significant differences exist between “effective” and “ineffective” organizations in the predicted direction. A strength of this study lies in its use of “customer” food broker managers to rate the performance of “seller” producers. While the methodology for collecting the data is quite defensible, a limitation arises from the fact that each respondent determined both the overall assessment of four firms as effective or ineffective, and then evaluated ach one on the various elements. The likelihood for a consistency bias is consequently quite high[27]. Fawcett and Vellenga compared the performance of maquiladora and domestic transport operations[28,29]. The survey respondents were managers who rated the performance of transport firms with whom they dealt. The analysis revealed that transport service performance was less in the transnational operating environment. Marr found “management sophistication” to be somewhat helpful in predicting “distribution service performance”, although a limitation in his findings lies in the one-item measure used to measure “level of overall distribution service”[30].
Qualitative variables such as attitudes or perceptions can be included Four studies used mathematical or economic models with hard performance measures, as summarized in Table VI. The papers by Clarke and Gourdin[31] and Kleinsorge et al. [32,33] used data envelopment analysis (DEA) to evaluate the efficiency of logistics practices. Essentially, DEA uses linear programming methods to measure efficient combinations of inputs and outputs for a set of decision-making units (DMUs) and provides relative performance ratings for all the DMUs in the data set[34]. Writers in logistics and other fields have pointed out that the advantages of DEA include the less restrictive requirement for data input in comparison to many other 22 IJPD & LM 24,1 methods of quantitative analysis.
In particular, qualitative variables such as attitudes or perceptions can be included, thereby allowing the injection of essential intangibles into the system analysis. DEA has the ability to isolate variables for measurement while still including the effects of interaction among both input and output variables[33, p. 39; 35, p. 529). This is in marked contrast to “conventional” performance indicator analyses which are only incomplete measures of performance[34]. Disadvantages of DEA include the frequent difficulty of obtaining data [36, p. 443]. It should also be noted that thus far, most applications of DEA have been in the public sector[36, p. 443]. The paper by Diewert and Smith is a promising attempt at applying total factor productivity measurement in a distribution setting[37].
Using data from a large appliance parts distributor in Western Canada, the authors conclude that large productivity gains were possible in the distribution sector of the economy thanks to the computer revolution, which allows a firm to track its purchases and sales of inventory items and to use the latest computer software to minimize inventory holding costs (p. 16). Finally, Gomes and Mentzer employ a simulation to explore the influence of JIT systems on distribution system performance[38]. They found that JIT systems in the physical distribution and materials management contexts were associated with significantly more favourable profit and service results in comparison to non-JIT systems. System-wide JIT, on the other hand, produced favourable service results, but unfavourable profit results (p. 47).
Implications With the exception of the mathematical/economic studies, almost all of the empirical studies utilized soft measures for the outcome variable. Nevertheless, both soft and hard measures are associated with strengths and weaknesses (as a later section of this article discusses in detail). This limits a researcher’s ability to infer the existence of relationships between logistics performance and its antecedents. One limitation common to several of the studies is that the respondents appraised their own performance. This becomes a particular concern in cases where the performance dimension under consideration is better assessed by another source, such as the customer.
Also, very few studies we reviewed adequately captured the multiplicity of goals that must be included in any meaningful evaluation of performance at either the logistics or firm level. To be sure, keeping a study within a feasible domain will often involve limiting the examination to one or more dimensions of “performance”. However, unsupported extrapolations of the findings of such studies to unmeasured dimensions are difficult to justify. For example, a researcher who finds a significant relationship between the utilization of total quality management programmes and customer satisfaction should indicate that the findings of the study do not necessarily generalize to dimensions of performance that were not included in the study.
The predominance of the mail survey as a data collection method in the logistics studies reviewed raises some concern in light of its inherent limitations[39]. To their credit, however, most authors disclosed these limitations to one extent or another. A few offered especially coherent discussions of remedial or assessment measures that had been completed, such as those suggested by Lambert and Harrington[40]. The examination of these studies reveals that an immense variety of operational definitions and measures exists for logistics performance. This is the result of the varying interests of the researchers and the complexity of performance that have been alluded to earlier.
Another source of the variety in performance measures is the domain to which the measure is relevant. Several of the studies measure performance at a logistics activity level (e. g. transport or warehousing). Other studies measure performance at the logistics function level and several attempt to measure the firm’s performance. Distributors in the health-care supply chain are increasingly holding inventory None of these studies examines logistics performance in the context of supply chain management. This is significant. The supply chain comprises all companies that participate in transforming, selling and distributing the product from raw material to the final consumer. It has two implications for research in logistics performance.
First, some research might be oriented towards measurement of supply chain management, that is, performance involving multiple organizations. Second, research oriented to members of a channel should recognize the relationship between channel structure and member functions. For example, distributors in the health-care supply chain are increasingly holding inventory and performing sorting functions traditionally performed by hospitals. This integration has resulted in lower inventory and handling costs in the whole supply chain, although specific members of the chain may exhibit higher inventory and sorting costs than might otherwise be expected.
LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT 23 Unfortunately, the variety of performance measures make it difficult to draw broad inferences from the literature about the relationship between a given logistics practice and performance. Meta-analysis, or aggregating the findings of several studies[41], is frustrated by the use of diverse measures. Differences in findings between one study and another may be attributable solely to the measures used. Figure 1. What Is Logistics Performance? Cost-efficiency Sales growth Profitability Job security and working conditions Keeping promises Low loss and damage “Fair” prices for inputs Flexibility Defining Logistics Performance
Conceptually, logistics performance may be viewed as a subset of the larger notion of firm or organizational performance. The latter has attracted a large volume of diverse research over the years[2,35] and illustrates the futile nature of the search for the “one best way” of defining performance. For example, Gleason and Barnum chose to distinguish between effectiveness and efficiency. They defined effectiveness as “the extent to which an objective has been achieved”, while efficiency was defined as “the degree to which resources have been used economically”[42, p. 380]. Simply put, efficiency is “doing things right”, while effectiveness is “doing the right thing”[42, pp. 3,4].
Sink and his colleagues, on the other hand, defined seven dimensions in order to capture their conception of “what performance means”: they are effectiveness, efficiency, quality, productivity, quality of work life, innovation and profitability/budgetability[43, pp. 266-7]. Social responsibility Customer satisfaction Product availability On-time delivery be surprising that extant literature offers many ideas about the dimensions that ought to be incorporated into a conceptualization of “logistics performance”. One of the best examples is the framework presented by Rhea and Shrock, where physical distribution effectiveness is defined as “the extent to which distribution programmes satisfy customers”[5, p. 35]. They note, however, that other goals remain important under this definition: In taking this orientation, decision makers do not ignore the well-documented need to control costs.
Rather, they incorporate this objective within a customer-oriented managerial philosophy in which it is believed that the longrun goal of profitability is achieved by providing customer need and want satisfaction[44; 5, p. 35]. Care must be taken to incorporate multiple goals… Both of these examples have strengths and weaknesses. A principle strength of Gleason and Barnum’s work is its simplicity. This simplicity may be somewhat deceptive, however: a convincing case could be made about the importance of efficiency as one dimension of effectiveness. As for the dimensions identified by Sink and his colleagues, they are intriguing because they meet the need for a broad, comprehensive framework. However, this comes at the price of some overlap between the various dimensions, as the authors themselves infer[43,p. 268].
In particular, the distinction between performance and effectiveness is somewhat unclear. Given the lack of any universally-accepted definition for performance in the organizational literature, it should not The literature review provided by Rhea and Shrock suggests that care must be taken to incorporate multiple goals in defining performance. In order to begin the task of conceptualizing what the various dimensions of logistics effectiveness might look like, we identified a representative set of answers to the question, “what is logistics performance? ” The results of this exercise are shown in Figure 1. Logistics performance may be defined as the extent to which goals such as those suggested in Figure 1 are achieved.
Figure 1 incorporates various possible dimensions of performance in a single envelope to help highlight the numerous interdependencies and conflicts between the goals. For example, interdependency is likely to exist between employee satisfaction, the quality of customer service and profitability. A conflict would occur if a pay rise for employees were postponed to achieve a shortterm financial improvement – a move which may inhibit the firm’s ability to attract and retain employees who are capable of delivering quality customer service to the benefit of long-run profits. Another example is that a firm may find it advantageous in the short run to make more extensive use of packaging in an environmentallysuboptimal way although it may face more stringent 24
IJPD & LM 24,1 Table VII. Alternative “Hard” Measures for Logistics Performance Measure type Raw financial statistics (e. g. net income, gross sales) Cost statistics (e. g. transport cost, standard labour costs) Advantages/disadvantages Advantages: Often easy and inexpensive to collect, and is likely to be comparable between organizations. Can capture several important dimensions of performance, often with impressive accuracy Disadvantages: May not be comparable between one time interval and another. Accounting methods may limit comparability between organizations. Level of aggregation may be so large that it is difficult to assign responsibility.
Firms may be unwilling to divulge information Advantages: Can be used to evaluate goal attainment in many areas, particularly efficiency and effectiveness Disadvantages: Narrow focus upon individual performance indicators may easily cause faulty analysis or decision-making. Researchers may have trouble getting data. May be incomparable across organizations Input/output measures or “performance indicators” (e. g. number of shipments/ vehicle hour) Quality measures (e. g. order cycle time) regulation, increased paper burden and compliance costs and reduced profits in the long run. Measuring Logistics Performance The preceding discussion argues that performance is multi-dimensional. No one measure will suffice for logistics performance.
Instead, the objective for researchers and managers is to find a set of measures which collectively capture most, if not all, of the performance dimensions thought to be important, over both short- and long-term horizons. For example, in a presentation at the Tenth International Logistics Congress in Toronto, in June 1993, Mr D. Eggleton of Rank-Xerox described the criteria on which his performance is evaluated as employee satisfaction, customer satisfaction, and the company’s rate of return. Many dimensions of logistics performance lend themselves well to hard performance measures. A representative collection of these, along with key advantages and disadvantages, is shown in Table VII. Hard performance measures such as net income or order fill rate are typically impersonal, accurate and easy and inexpensive to collect.
Measures such as net income, and accounting ratios such as return on investment (ROI) are useful ways of capturing profitability, and will often be easy and inexpensive to collect, particularly where logistics is treated as a profit centre. Profitability is a particularly useful goal because it directly reflects the goals of all of the organization’s internal constituent groups to one extent or another, although it may not be a good indicator of the viability of the firm in the long run. Cost accounting measures may also be useful, particularly in evaluating several dimensions of efficiency. The data are often highly accurate and, in many cases, available over long periods of time.
However, these measures are not always comparable between one organization and another. Changes in accounting practices may even inhibit valid comparisons within the same organization over time. There are some difficulties which are common to both raw financial measures and cost accounting data. Because they are often considered confidential, many firms are reluctant to release information to outsiders. Also, in making comparisons between organizations or time periods, variations in standards or accounting methods are a frequent threat[45]. A particular problem to logistics researchers is that in many cases, the level of aggregation is so high that it is difficult to utilize for evaluating subfunctions of the firm.
The use of input-output ratios (also known as productivity or performance indicators) is common in logistics, and has received extensive treatments in textbooks and other literature[8, 46]. Many goals lend themselves well to evaluation using these measures. For example, productivity may be measured utilizing ratios such as number of shipments per vehicle-mile, while a ratio such as percentage orders delivered on time will be helpful in evaluating the quality of service rendered. As the concern of society over environmental issues grows, some firms may find it useful to calculate ratios such as “tons of packaging used/total tonnage shipped”. Again, the limited ability of the researcher to gain access to these data because of their confidential nature is a potential disadvantage. Also, variations in definitions and data
LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT 25 collection procedures will often make it difficult to compare performance indicators between one organization and another. For service measures such as order cycle time or lead time variability, the advantages and disadvantages are essentially the same as those of performance indicators. One limitation common to both is that there are many dimensions of performance which they cannot capture, particularly the extent to which customers are satisfied. The difficulty in capturing customer satisfaction is the underlying reason that hard measures should be supplemented with “soft”, perceptual ones.
Although there are several dimensions of logistics performance which hard measures cannot capture in a meaningful way, customer satisfaction is perhaps the most critical. A set of soft measures, collected using techniques such as the mail survey, telephone interview, or similar method are needed. Besides their usefulness in identifying problems, soft measures may also be called for where available hard measures are not comparable between one organization and another because of differences in accounting standards or similar problems. These measures are subject to the limitations inherent in any self-report, such as consistency bias, and the social desirability problem[27,39]. The difficulties are especially serious for one-item measures[27]. easures are associated with a host of other limitations, the most notable being self-report defects[27] or other forms of bias[39]. An excellent way to improve the validity of soft measures may be found in Churchill’s paradigm for developing better measures of marketing constructs[47]. Essentially, the paradigm comprises a number of carefully-ordered steps by which literature searches, surveys and analysis procedures are linked together in a coherent and justified sequence. First, the domain of the construct is specified. Next, a sample of items is generated. Third, the measure is purified. Reliability and validity of the construct are then assessed. The Churchill paradigm has been employed by a handful of logistics scholars[22], with promising results.
The discussion presented here suggests that although the optimal set of performance measures will depend on the purpose of the research, it will often include a collection of both hard and soft measures. One important criterion to consider in choosing the set is “representativeness”. That is, the set of measures should meaningfully capture those dimensions of performance in which the researcher is interested. The use of only one or two measures of performance is justified for the researcher whose study addresses only customer satisfaction or cost efficiency. However, findings from such studies should not be carelessly extrapolated to include the larger variable, logistics performance. Factor analysis will be an especially useful tool Soft measures may also raise comparability problems.
Suppose that a handful of manufacturers in the same industry report (on a scale of 1 to 5) how well their logistics function achieves on-time delivery goals. The responses may be difficult to interpret if (as will likely be the case), there is variation in the competitive strategy or goals of the firms. This may well influence what is reported by the respondent. As an example, consider two firms with an actual on-time delivery level of 80 per cent. Each firm’s manager is asked to rate performance. If Firm A’s goal is 100 per cent and Firm B’s goal is 80 per cent, it is likely that Firm A’s manager will provide a lower performance rating than Firm B’s.
In other contexts, low scores on “quality of service” may reflect the presence of a cost-minimization strategy which are not meaningfully comparable to those of a firm seeking to differentiate itself by providing superior service. Soft Recommendations This article argues that defining and measuring performance in logistics is a difficult enterprise, for both researchers and managers. The literature review reveals the nature and limitations of the various designs and measures which have been used thus far and, along with subsequent sections, suggests that there are no “easy” ways to address the issues raised. In light of this review, we offer five recommendations which should help improve the quality of future research.
More Efforts to Develop Performance Measures In the short run, we urge researchers to give more detailed information about how they have defined and measured performance, and about the limitations of their study and its findings. Inadequate reporting of the appropriate caveats serves only to mislead readers and frustrate logistical excellence. Special issues of logistics journals devoted to measurement and methodological issues would be most useful. Their leading role would be very likely to generate added interest in logistics research methodology among researchers and students. A more coherent picture of the various dimensions of performance would make it much 26 IJPD & LM 24,1 asier to address subsequent questions such as, “how” should each dimension be measured? ”, and “who should provide data for each measurement? ” Increased attention to the development of valid measures is warranted, particularly the soft ones which capture dimensions of performance which are otherwise left out. The encouraging results of the study by Gassenheimer and his colleagues, in which factor analysis is used, suggest that this will be an especially useful tool for developing valid perceptual or self-report scales[11]. As noted earlier, expanded use of the paradigm suggested by Churchill[47] is very likely to enhance the validity of studies that are conducted, and also facilitate aggregation and comparison among them.
Encouragement of More Innovative Research Designs Studies in which more than one constituency provides data in evaluating performance should be encouraged[48]. Although “rate-your-own-company” studies have a legitimate place in research, they need to be supplemented by data from other constituent groups, especially customers. While we suspect that the mail survey will continue to be an important data-collection method, journal editors should encourage studies in which alternative methods are used. Also, in the short run, journal reviewers and editors should insist that submissions articulate and justify how performance has been defined, and describe the measures that have been used.
In particular, questionnaires should be reproduced verbatim in an appendix, or an address provided where interested researchers may obtain a copy. In this way, a body of time-tested items can be built up in short order, metaanalyses are facilitated, and needless “re-invent the wheel” exercises are avoided. case study that has proven invaluable for many firms is the “logistics performance audit” in which a team systematically evaluates a firm’s logistics function. Development of Contingency Models of Logistics Performance Researchers might do well to explore contingency models of logistics performance[49,50]. To date, their utilization has been sparse, despite their intuitive appeal.
Rather than the “one-best-way” paradigm which is common in so many discussions of logistics research, a contingency model would be based on the supposition that the fit between logistics organization and strategy and the organization’s environment, product line, production technology and size will influence performance outcome variables. The development and testing of such a model might well stimulate research on the question of the primacy of various performance dimensions. For example, the firm which has adopted a cost-minimization strategy may be better served by focusing on cost dimensions of performance than the firm which has chosen to differentiate itself on quality of service. While this assertion may have intuitive appeal, it has little in the way of empirical support thus far. Recognition of the Implications of Supply Chain Management The shift to supply chain management has two implications for logistics performance.
First, the measurement of performance must recognize the particular role of an organization in a supply chain. Comparable stages in different channels may be associated with different functions. Second, consideration should be given to assessing the performance of the supply chain, not just that of individual participants. More Bridge-building between Theory and Practice Is Needed Researchers can and should play a role in making practitioners more familiar with the importance, nature and limitations of research. Practitioners, on the other hand, can offer needed insights into the types of logistics issues that merit greater study and thought, and more generally, what our overall “focus” should be.
Taking advantage of whatever opportunities we receive to engage in dialogue will help ensure that the research we conduct fulfils pragmatic purposes in the long run. References 1. Chow, G. and Henriksson, L. E. , A Critique of Survey Research in Logistics, Working Paper 93-TRA-009, University of British Columbia, Faculty of Commerce and Business Administration, 1993. 2. Hall, R. , Organizations: Structures, Processes and Outcomes, Prentice-Hall, New York and London, 1991, p. 267. Quantitative techniques are potentially important in logistics research Quantitative techniques such as data envelopment analysis and total factor productivity both represent potentially valuable ways to evaluate economic performance, and are potentially important tools in logistics research.
However, these should not be permitted to obscure the potential gains in knowledge that may be facilitated by qualitative studies, most notably the “case study” approach. One particular form of LOGISTICS PERFORMANCE: DEFINITION AND MEASUREMENT 27 3. Armitage, H. M. , “The Use of Management Accounting Techniques to Improve Productivity Analysis in Distribution Operations”, International Journal of Physical Distribution & Materials Management, Vol. 14 No. 1, 1984, pp. 41-51. 4. Mentzer, J. T. and Konrad, B. P. , “An Efficiency/Effectiveness Approach to Logistics Performance Analysis”, Journal of Business Logistics, Vol. 12 No. 1, 1991, pp. 33-61. 5. Rhea, M. J. and Shrock, D. L. , “Measuring the Effectiveness of Physical Distribution Customer Service Programs”, Journal of Business Logistics, Vol. 8 No. , 1987, pp. 31-45. 6. Byrne, P. M. and Markham, W. J. , Improving Quality and Productivity in the Logistics Process, Council of Logistics Management, Oak Brook, IL, 1991. 7. La Londe, B. J. , Cooper, M. C. and Noordewier, T. D. , Customer Service: A Management Perspective, Council of Logistics Management, Oak Brook, IL, 1988. 8. Nevem Working Group, Performance Indicators in Logistics, IFS Publications, Bedford, 1989. 9. Read, W. F. and Miller, M. S. , “The State of Quality in Logistics”, International Journal of Physical Distribution & Logistics Management, Vol. 21 No. 6, 1990, pp. 32-47. 10. Harrington, T. C. , Lambert, D. M. and Christopher, M. “A Methodology for Measuring Vendor Performance”, Journal of Business Logistics, Vol. 12 No. 1, 1991, pp. 83-104. 11. Gassenheimer, J. B. , Sterling, J. U. and Robicheaux, R. A. , “Long-term Channel Member Relationships”, International Journal of Physical Distribution & Logistics Management, Vol. 19 No. 12, 1989, pp. 15-28. 12. Cooper, J. C. , Browne, M. and Peters, M. , “Logistics Performance in Europe: The Challenge of 1992”, International Journal of Logistics Management, Vol. 1 No. 1, 1990, pp. 28-34. 13. Clarke, R. L. , “The Measurement of Physical Distribution Productivity: South Carolina, a Case in Point”, Transportation Journal, Vol. 31 No. , Fall 1991, pp. 14-21. 14. Yavas, U. , Luqmani, M. and Quraeshi, Z. A. , “Purchasing Efficacy in an Arabian Gulf Country”, International Journal of Physical Distribution & Logistics Management, Vol. 19 No. 4, 1989, pp. 20-5. 15. Bowersox, D. J. , Daugherty, P. L. , Droge, C. L. , Rogers, D. S. and Wardlow, D. L. , Leading-edge Logistics: Competitive Positioning for the 1990s, Council of Logistics Management, Oak Brook, IL, 1989. 16. Chow, G. , review of Leading Edge Logistics: Competitive Positioning for the 1990s. Logistics and Transportation Review, Vol. 27 No. 4, December 1991, pp. 384-7. 17. Daugherty, P. J. , Stank, T. P. and Rogers, D. S. “The Impact of Formalization on Warehousing Firms”, International Journal of Logistics Management, Vol. 3 No. 2, 1992, pp. 49-61. 18. Cameron, K. S. , “Effectiveness as Paradox: Consensus and Conflict in Conceptions of Organizational Effectiveness”, Management Science, Vol. 32 No. 5, May 1986, pp. 539-57. 19. Daugherty, P. J. , Sabath, R. E. and Rogers, D. S. , “Competitive Advantage through Customer Responsiveness”, Logistics and Transportation Review, Vol. 28 No. 3, September 1992, pp. 257-71. 20. Droge, C. and Germain, R. , “The Impact of the Centralized Structuring of Logistics Activities on Span of Control, Formalization and Performance”, Journal of the Academy of Marketing Science, Vol. 17 No. 1, 1989, pp. 83-9. 21. Germain, R. “Output Standardisation and Logistical Strategy, Structure and Performance”, International Journal of Physical Distribution & Logistics Management, Vol. 19 No. 1, 1989, pp. 21-9. 22. Fawcett, S. E. and Closs, D. J. , “Coordinated Global Manufacturing, the Logistics/Manufacturing Interaction, and Firm Performance”, Journal of Business Logistics, Vol. 14 No. 1, 1993, pp. 1-25. 23. Fawcett, S. E. , “The Status and Impact of Logistics Issues in the Success of Co-production via Maquiladoras”, International Journal of Logistics Management, Vol. 2 No. 2, 1991, pp. 30-41. 24. Perry, J. H. , “Firm Behavior and Operating Performance in Just-in-Time Logistics Channels”, Journal of Business Logistics, Vol. 9 No. 1, 1988, pp. 19-33. 25. Rhea, M. J. and Shrock, D. L. “Measuring Distribution Effectiveness with Key Informant Reports”, Logistics and Transportation Review, Vol. 23 No. 3, September 1987, pp. 295-306. 26. Rhea, M. J. and Shrock, D. L. , “Physical Distribution Implementation Effectiveness: The Customer Perspective”, Transportation Journal, Vol. 27 No. 1, Fall 1987, pp. 36-42. 27. Podsakoff, P. and Organ, D. W. , “Self-reports in Organizational Research: Problems and Prospects”, Journal of Management, Vol. 12 No. 4, 1986, pp. 531-44. 28. Fawcett, S. E. , “Logistics and Manufacturing Issues in Maquiladora Operations”, International Journal of Physical Distribution & Logistics Management, Vol. 20 No. 4, 1990, pp. 13-21. 29. Fawcett, S. E. and Vellenga, D. B. “Transportation Characteristics and Performance in Maquiladora Operations”, Transportation Journal, Vol. 31 No. 4, Summer 1992, pp. 5-16. 30. Marr, N. E. , “Management Sophistication and Service Performance”, International Journal of Physical Distribution & Logistics Management, Vol. 21 No. 4, 1991, pp. 32-41. 31. Clarke, R. L. and Gourdin, K. N. , “Measuring the Efficiency of the Logistics Process”, Journal of Business Logistics, Vol. 12 No. 2, Fall 1991, pp. 17-33. 32. Kleinsorge, I. K. , Schary, P. B. and Tanner, R. , “Evaluating Logistics Decisions”, International Journal of Physical Distribution & Materials Management, Vol. 19 No. 12, 1989, pp. 2-14. 33. Kleinsorge, I. K. , Schary, P. B. and Tanner, R. “The Shipper-Carrier Partnership: A New Tool for Performance Evaluation”, Journal of Business Logistics, Vol. 12 No. 2, 1991, pp. 35-57. 34. Hensher, D. A. and Waters, W. G. , II, Using Total Factor Productivity and Data Envelopment Analysis for Performance Comparisons among Government Enterprises: Concepts and Issues (unpublished manuscript), Institute of Transport Studies, University of Sydney, Australia, 1993. 28 IJPD & LM 24,1 35. Lewin, A. Y. and Minton, J. W. , “Determining Organizational Effectiveness: Another Look, and an Agenda for Research”, Management Science, Vol. 32 No. 5, May 1986, pp. 514-38. 36. Charnes, A. , Cooper, W. W. and Rhodes, E. “Measuring Efficiency of Decision Making Units”, European Journal of Operational Research, Vol. 2 No. 6, 1978, pp. 429-44. 37. Diewert, W. E. and Smith, A. M. , “Productivity Measurement for a Distribution Firm” (unpublished manuscript), University of British Columbia, Department of Economics, 1992. 38. Gomes, R. and Mentzer, J. T. , “The Influence of Just-inTime Systems on Distribution Channel Performance in the Presence of Environmental Uncertainty”, Transportation Journal, Vol. 30 No. 4, Summer 1991, pp. 36-48. 39. Armstrong, J. S. , Long Range Forecasting (2nd ed. ), Wiley, New York and Chichester, 1985. 40. Lambert, D. M. and Harrington, T. C. “Measuring Nonresponse Bias in Customer Service Mail Surveys”, Journal of Business Logistics, Vol. 11 No. 2, 1990, pp. 5-25. 41. Hunter, J. E. , Schmidt, F. L. and Jackson, G. B. , MetaAnalysis: Cumulating Research Findings across Studies, Sage, New York, NY, 1982. 42. Gleason, J. M. and Barnum, D. T. , “Toward Valid Measures of Public Sector Productivity: Performance Measures in Urban Transit”, Management Science, Vol. 28 No. 4, April 1986, pp. 379-86. 43. Sink, D. S. , Tuttle, T. C. and DeVries, S. J. , “Productivity Measurement and Evaluation: What Is Available? ”, 44. 45. 46. 47. 48. 49. 50. National Productivity Review, Vol. 4 No. 3, 1984, pp. 265387. Lambert, D. M. and Zemke, R. “The Customer Service Component of the Marketing Mix”, Proceedings, Twentieth Annual Conference, National Council of Physical Distribution Management, 1982, pp. 1-24. Dess, G. G. and Robinson, R. B. , Jr, “Measuring Organizational Performance in the Absence of Objective Measures: The Case of the Privately Held Firm and Conglomerate Business Unit”, Strategic Management Journal, Vol. 5, 1984, pp. 265-73. Glaskowski, N. A. , Hudson, D. R. and Ivie, R. M. , Business Logistics (3rd ed. ), Dryden, New York, NY, 1992. Churchill, G. A. , Jr, “A Paradigm for Developing Better Measures of Marketing Constructs”, Journal of Marketing Research, Vol. 16, February 1979, pp. 64-73. Jobson, J. D. and Schneck, R. “Constituent Views of Organizational Effectiveness: Evidence from Police Organizations”, Academy of Management Journal, Vol. 25 No. 1, February 1982, pp. 25-46. Persson, G. , “Organisation Design Strategies for Business Logistics”, International Journal of Physical Distribution & Materials Management, Vol. 8 No. 6, 1978, pp. 287-97. Pfohl, H. C. and Zollner, W. , “Organisation for Logistics: The Contingency Approach”, International Journal of Physical Distribution & Materials Management, Vol. 17 No. 1, 1987, pp. 3-16. Garland Chow is an Associate Professor, Trevor D. Heaver a Professor and Lennart E. Henriksson a Research Associate, all in the Faculty of Commerce and Business Administration, University of British Columbia, Vancouver, Canada.
