The Random Sampling/Tracking Tool: A Response to Over-

Surveying

Steve GravesIntel Corporation

Overview Problem Statement Recommended Solution Features User Interface Demo Benefits Concerns Next Steps Possible tool enhancements Beyond Intel Summary

Problem Statement

Perception that too many surveys are deployed to end users from IT – Source: Annual Partnership Excellence Survey with top Intel Execs

Likely causes identified (based on experience as a survey consultant)– Groups tended to survey entire population of interest

instead of a sample (e. g. survey of habits of frequent travelers -- 26,000 people)

– When criteria for a population is only partially available a screening survey is necessary – results in more people surveyed

– Simple random sampling will tend to hit a small percentage of people multiple times

Same Participant

General Population

Study A

Study B

Problem Statement

Effective and efficient process to stop over-surveying does not exist– Previous process relied on incomplete information and

“Guardians” – go between groups– No mechanism to track who has received surveys

– Guardians may be overly cautious

– Meetings used to determine what to do – time consuming and inefficient

– Process is not followed universally

Recommended Solution

Sampling tool containing a repository of Intel employee data– Utilizes data from employee database (non-

confidential)

– Generates random samples of target audience

– Tracks who has been sampled over a given time period – currently 5 months

– Sampled individuals will not be selected again for during this period

Recommended SolutionDatabase perspective

Employee Database Retrieval Database Sampling Database

Sampling DatabaseStorage Database

Tracking Table

Randomly Selected Employees

Recommended SolutionDatabase perspective – previously sampled individuals

removed before Sample is generated

Tracking Table

Generated table based on CriteriaTable that meets criteria and contains no excluded individuals

Recommended Solution

Incorporate into an overall process where researchers are required to utilize the tool and consult with sampling advisors– Advisors can advise researchers on appropriate sampling

techniques – so researchers to survey an entire population (e.g. 3,000 instead of 26,000)

Currently implemented using RAD database programming language

– Tool has been in use for almost 2 years

– Tool is performing reliably and customers are satisfied

– Usage is only through word of mouth

– Approximately 60 samples generated last year

Sampling Tool Features

Current features– Tracking and recording mechanism

– Recording of samples generated by others

– Random Sampling of entire Intel population

– Random sample from group that uses criteria not part of employee database (for example: Employees who use PDAs)

– Random sampling of Sub-Groups (For example: Employees in Folsom California)

– Stratified sampling

User Interface

Demo (Double Click to Start)

Benefits Saves researchers time

–Reduces the need for screening surveys

–No need to re-invent the wheel

–Saves time compared to previous process

Benefits Improvement in sampling process

–Reduce or eliminate “over-sampling”– Delay surveys when target population is not

available– Researchers receive consultation on using

appropriate sample sizes–Distribute sampling more evenly across groups

Concerns May cause sampling bias in some situations

Next Steps Need to develop more sophisticated business rules

around sampling process Currently working to make this an IT wide process

– Early results are promising

Looking at making it an Intel wide process Port to SQL server and have a web based solution

– Current access is through Remote Desktop Connection

Develop strategies to mitigate sampling bias

Possible Future Tool Enhancements

Complex samples through UI – currently requires custom query

Linking to Other database that contains product usage information

Beyond Intel Similar tools could be used in similar situations in other companies or institutions

Summary Problem: Over-surveying by our IT organization

Response: Random Sampling/Tracking Tool in conjunction with business rules to manage survey sampling process

Benefits: Saves researchers time, reduces over-surveying, improves sampling process

Working to make this an IT wide official process and eventually Intel wide

Summary Next steps:

–Link to other databases – Improve business rules –More robust software platform–Strategies to mitigate sampling bias

Similar tools/business rules could be used in other institutions