IMPLEMENTATION

LOCAL -- MONITORING & EVALUATION

Overview

As stated above in the introductory "Policies for Implementation," TDM projects will be monitored and evaluated for effectiveness:

"Through the use of the California Commute monitor, there must be a consistent and scientifically valid evaluation for each major project."

This policy responds to the legislative mandate that TDM resources -- specifically marketing dollars -- be spent efficiently and effectively. To make that assessment, marketing research will monitor and evaluate TDM marketing projects at the local and regional level.

Standard Methodology

Establishing a standard evaluation methodology to accommodate a wide range of transportation options and marketing activities is essential to the evaluation process. Without standard research methodology, evaluating the effectiveness of similar marketing programs across disparate regions would be virtually impossible.

Local Research

The Local Commuter Monitor is the marketing research program that will measure the effectiveness of corridor-specific marketing projects, as well as broader regional programs. Three times each year, the Local Commuter Monitor will asses commuters' attitudes, perceptions and behaviors. Then, the Monitor will analyze that information in comparison with real-world transit data -- for example, capacity and usage -- collected at the same local level.

Continuous Marketing Research

The Local Commuter Monitor will operate three times each year within each region. This continuous timetable enables the research program to establish a pre-assessment -- or baseline -- of commuter attitudes, perceptions and behaviors regardless of the program's timing. Then, as the marketing program unfolds, the Local Commuter Monitor will track its progress and effectiveness; thereby allowing for mid-course adjustments and more effective program management. At each interval, the Monitor will compare changes in survey research data with changes in the real-world transportation data -- e.g., capacity and usage.

Awareness and Perceptions

The Local Commuter Monitor will evaluate changes in commuters' 1) awareness and 2) perception of transportation modes as well as the marketing activities promoting those modes. For example, a typical Monitor survey would ask a series of questions about that community's bus service. In addition, the same survey would also ask specific questions about that service's promotions.

Commute Behavior

The Local Commuter Monitor will evaluate region-specific and corridor-specific commute behavior -- and assess the effect of transportation marketing programs on that behavior. The Monitor will assess commuter behavior in two ways: 1) through the regular survey research program, commuters will answer a variety of questions about past and present commuting behavior; and 2) real-world transportation data collected at the local level will serve as a separate measurement of commuting behavior.

Effect of Marketing on Commute Behavior

The survey research also will examine the effect of specific marketing programs on commuting behavior. Because it is a critical element, the research will measure the strength of the link, if any, between specific marketing projects and commuters' mode choices.

For example, the Local Commuter Monitor may evaluate a corridor-specific marketing project and determine that a majority of changes in commuting behavior are primarily attributable to external factors having nothing to do with the marketing project itself. That kind of causal relationship will be established only by comprehensive inquiry during the survey research process.

Research Methodology

The Local Commuter Monitor will be executed three times each year in every region. Local transportation planners and managers will have an opportunity to determine which TDM marketing projects and modes will be tested.

Each wave of the Local Commuter Monitor will consist of 900 randomly selected telephone interviews, statistically representative of the population in the local region. Those 900 interviews will consist of one-third randomly sampled all adults; one-third random sample of transportation data bases (rideshare contacts, 800# and information callers, etc.); and one-third panelback interviews from previous Monitor surveys.

This methodology is designed to evaluate the effectiveness of specific marketing programs for local and regional transportation. Analyses of marketing research data (e.g., mode and program awareness, perception and behavior, etc.) and real-world transportation data (e.g., usage and capacity, etc.) are not intended to replace existing methodologies for assessing placement rates and other non-marketing functions.

STATEWIDE -- CALIFORNIA COMMUTE INDEX

In addition to monitoring and evaluating specific local marketing programs, Caltrans will conduct an annual survey -- the California Commute Index -- to monitor changes in the transportation marketplace.

Adhering to the the objectives of the legislative mandate for effective and measurable marketing, the California Commute Index will monitor broad changes in commuters' attitudes and behaviors. In effect, the Commute Index will provide the strategic overview for TDM marketing issues.

This large-scale quantitative survey research will produce statistically significant -- and meaningful -- data in each region.