Using Focus groups to understand organisational practice

1

Using Focus groups to understand

organisational practiceEarly findings

Patrizia Bertini - IS554 - 09/09/10

2

Premises

Research Question: Do all managers and operators involved in data processing activities found their practice on a shared understanding of data and data quality?

As data referred within this research are personal data, how are data protection laws affecting practice and the different departments within the organisations?

Privacy and data

3

Data are an Important assetFor organisations.

Poor quality dataCause: Economic lossesLeads to wrong decisionRise costs.

[Redman 2008]

4

Data quality and Privacy

“OECD Guidelines on the Protection of Privacy and Transborder Flows of Personal Data”:

Part 2 “Personal data should be relevant to the purposes for which they are to be used, and, to the extent necessary for those purposes, should be accurate, complete and kept up-to-date”

5

Data quality and Privacy 2Directive 95/46/EC Article 6 states that Member States shall provide that personal data must be:

(a) processed fairly and lawfully;(b) collected for specified, explicit and legitimate purposes

and not further processed in a way incompatible with those purposes. [...];

(c) adequate, relevant and not excessive in relation to the purposes for which they are collected and/or further processed;

(d) accurate and, where necessary, kept up to date; [...];(e) kept in a form which permits identification of data

subjects for no longer than is necessary for the purposes for which the data were collected [...].

6

Focus groups

“The hallmark of focus groups is the explicit use of the group interaction to produce data and insight that would be less accessible without the interaction found in a group.”

Morgan (1997:12)

7

Focus groups

They are used to:obtain opinions and attitudes;understand the social context;learn participants’ experiences and perceptions;stimulate opinion elaboration.

8

Focus group and interaction Focus group uses interaction to produce data and insights, allowing participants to building on others’ interaction (Morgan 1997, Chiarini et al. 2010).

The interaction reveals: cognitive processes;interpretation of events/concepts;patterns in understanding decision making.

Opinionsopinions are formed by 3 processes:

Compliance (superficial opinion - satisfies expectations)

Identification (answers follows those of other participants)

Internalisation (personal opinion)

(Albrecht et al. 1993)

9

10

Preliminary activities Workshop Post-workshop activities Data collected

Method in practice

Preliminary activities

Participants are asked to select/provide a definition of data quality.

11

Questionnaire

Data Quality - Definitions

Please, select 2 definitions among the following ones which, in your opinion, best describe data

quality. Data quality is...

Conformance to requirements;

Meeting the customer requirements;

Fitness for intended purpose of a product or service;

Excellence achieved by responding to market-led forces;

Providing an efficient and effective service to customers;

Preventing things from going wrong and getting everything right first time, every time;

Involving everyone concerned in every stage, from design to after-sale service;

Data which are free of defects and possess the features needed to make the decision, or

complete the plan;

The extent to which customer information remains reliable and consistent across the organisa-

tion overtime;

Quality data is produced at the source, at the data collection point;

Data are of high quality if they are fit for their intended uses in operations, decision-making, and

planning. Data are fit for use if they are free of defects and possess desired features;

Data quality depends on the processes that produced the data and the context in which data

are used.

If you think that none of the above definitions are appropriate or complete, please, add a personal defini-

tion of data quality based on your own experience:

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Preliminary activity 2

12

Data Quality - Dimensions

According to your experience, select three of the following data quality dimensions as most important and

and rank them from 1 to 3 (being 1 the most important) according to the impact these dimensions have

on your practice and on your organisation’s data quality practice:

Free of error / accuracy

Completeness

Consistency

Timeliness

Believability

Accessibility

Relevancy

Security

Understandability

Concise representation

Ease of manipulation

Appropriate amount of data

Data quality - Management

Are you responsible for data quality within the organisation? If not, who is responsible (please, provide

name, function and department)?

____________________________________________________________________________________________________________________________________________________________________________________

Participants are asked to select and rate 3 dimensions of Data Quality from a list.

How does interaction produce data?

An example from Focus Group 1

13

14

Focus groups dynamics

After analysing participants’ answers, participants have been asked to agree on 2 dimensions.

GDM*: So where was the overlap then, what… so you had which?

[*Group Data Manager]

15

Focus groups dynamicsDPM* : Well the sort of thing around believability and consistency and accuracy all sounded like it was all around a similar theme of knowing what you’ve got, and therefore being able to trust what you’ve got. So the believability and therefore the consistency, and therefore understanding the accuracy all sort of comes under the same umbrella?

GDM: Yeah I suppose.

[* Data Procurement and Relationship Manager]

16


GDA*: Yeah I agree, which is probably why they are my top three because the believability I… coz I looked at, I contemplated completeness and I changed my mind because it doesn’t need to be complete in order to be good data, it can still be very good…

[*Group Data Analysis Manager]

17


GDM: Yeah true it depends how you define believability, that’s a lot harder to define than accuracy.

DPM: Yeah.

GDA: That’s true.

18


GDA: And so that’s why I discounted that one[completeness]. Whereas believability, that’s… if you can’t trust your data then there’s no point having it really...

19


GDM: It’s a lot more subjective. I mean I put down completeness from a [Company] point of view coz all the products we do, we try to do them in a consistent way […] So, but that’s maybe… it depends on the purpose again.

20


DPM: Yeah that was interesting coz you thought that and I was thinking completeness in terms of what fields there were that they were filled, I suppose. I was thinking of attributes […].

21


GDM: Yeah it’s true. I come from a geographical background so I was thinking geographical completeness and you were thinking attribute completeness…

22


DPM: Yeah I was thinking, you know, you don’t have to have all of the attributes but what is there, obviously it’s great if they’re all complete was kind of where I was coming from.

23


DAM: Yeah, and I think I agree with both of you but I think it’s not necessarily paramount, again it’s about understanding what you’ve got […].[…] (if I) can’t trust it, it’s inconsistent there’s no point, and that comes down to the believability of [them all].

24


GDM: So all three of us agree on the consistency then do we?

25


DPM: Do you want to merge your believability into the consistency with the accuracy? It’s negotiation time now. Or vice versa perhaps I dunno, coz if it’s not consistent you don’t believe it.

26


DAM: But he (GDM) said, he said that would be a very difficult one to measure


27

GDM: Also you also have the issue if it’s very interesting data’s often not that believable and if it’s very believable then it’s often not very interesting.DPM: True.DAM: That is true.


28

DAM: Well if the data’s not believable I don’t care how interesting it is [...].


29

GDM: So I suppose the word to use would be reliability rather than believability? Reliability’s not on your list, is that on purpose or is that the translation thing that you… coz there are things you could interpret as reliability?


30

DPM: I think that’s the one we’d all agree on…

DAM: Yeah reliability.

GDM: So it’s a consistency and reliability that we all agree on.

31

Focus group outcomeGroup dynamics produce new data:

Reliability = Emergent data

Initial dimensions Outcome

GDM1) Timeliness2) Consistency3) completeness

Consistency

ReliabilityDAM

1) Accuracy2) Consistency3) Believability

Consistency

ReliabilityDPM

1) Accuracy2) Consistency3) Completeness

Consistency

Reliability

32

Focus group 2Participants

Chief Information Security Officer (CSO)Principal Manager ICT Security (PMICT)Global Privacy Manager (GPM)Regional Head of Security (SEC)

The focus group agreed - less discussions

Interaction did not produced new content

Stronger hierarchical influence

Focus group 2 outcome

33

Initial dimensions Outcome

CSO1) Accuracy2) Completeness 3) Consistency4)

Accuracy

Completeness PMICT

1) Relevancy2) Accuracy 3) Completeness

Accuracy

Completeness GPM

1) Accuracy2) Relevancy 3) Right amount of data

Accuracy

Completeness

SEC1) Accuracy2) Relevancy 3) Completeness

Accuracy

Completeness

Senior Manager’s Influence

34

Focus Group 1 Focus Group 2Lower influence

Facilitate discussions

Promotes interaction

Internalisation

Higher influence

First to answer / dominant

Lead the conversation

IdentificationConstruction of new concepts

Clarification of perspectives

No emergent data

Clear understanding of internal procedures

Focus group Vs InterviewInterview with Data Quality and Supply Manager and Security Manager:

No interaction and no discussion;

No reflection on the topic;

Outcome is a description of practices;

Participants just added content without building new content on previous thoughts;

Superficial Opinions / Compliance35

Data and dynamics

36

Albrecht et Al. 2003

Consequences

Interview Compliance Superficial answers / description of practices

Focus Group 2 Identification Discussion influenced by Senior Manager

Focus group 1 Internalisation Construction of a new dimension

37

Conclusions Focus group:

Induces participants to reflect on the topic individually and before they join the workshop (questionnaire);

Improves/encourages participants’ interaction;

Allow emergent data (participants discuss concepts and discover themselves new concepts)

Help to reveal influence of power and dynamics;

Conclusions

Focus groups allows participants to engage in the co-construction of meaning;

Agreement is a top down process affected by levels of seniority. Need to investigate further the role of power;

Need to think about how many focus group to run... Any thoughts or experiences ? :-)

38

39

ReferencesAlbrecht, T. et al. (1993) “Understanding Communication processes in Focus Groups” in D. L. Morgan (ed.) Successful Focus Groups: Advancing the State of the Art. London: Sage; pp. 51-64.

Chiarini M. , Tremblay A., Hevner R., Bernd D. J., "Focus Groups for Artifact Refinement and Evaluation in Design Research," Communications of the Association for Information Systems: Vol. 26, Article 27.

Morgan D.L. (1997, 2nd Edition) Focus groups as qualitative research. London: Sage.

Redman, T. (2008) Data Driven: Profiting from Your Most Important Business Asset, Harvard Business Press, Cambridge, MA, 2008

Stewart, D. W., P. N. Shamdasani, et al. (2007). Focus groups: Theory and practice, Sage.

Documents

Using Focus groups to understand organisational practice