I agree with Chris. A great deal of theories tested typically at the
between-persons level are based on logic and theorizing at the
within-persons level. Here's a telling example: our department offers an
introductory PhD course that uses Smith and Hitts' book, "Great Minds in
Management: The Process of Theory Development." The first part of that book
deals (presumably though it's debatable) with more micro levels of
management, and even refers to the section as Individuals and Their
Environment. The theories covered include, social cognitive theory, image
theory, theories on organizational fairness, goal setting theory, job
characteristics theory, theories related to organizational commitment,
psychological contract theory, escalation of commitment, and expectancy
theory. An odd assortment yes, but reasonably wide ranging and influential.
If you take those that I listed, most of them have been examined over the
years with numerous between-subjects designs and models that describe
properties of individuals. Yet most or even all of them discuss processes
that involve how people generally react to changes in various aspects of
the workplace (e.g., shocks to decision images, unfair procedures, the
setting of difficult and specific goals, reactions to work conditions,
violations of expectations, etc.), which sounds a lot like Chris'
description that the intervention works for everyone. Not all of them play
out over the span of a week or two, but many of them do (or at least have
components that do).
Clearly, this puts me in the "ESM could be used to test most theories"
camp, but that's hardly surprising. I agree with Marcus though that just
because it can be used, doesn't mean it should be or has been used to
greatest effect. And yes, I would balk at feeling confident over having 32%
of the variance at the within-person level, particularly since that likely
lumps varying degrees of error variance in with it. There's a lot going on
at both the within- and between-person levels for many constructs, and I
would let the nature of the theory being examined dictate the approach
used. My main point here, however, is that you can easily frame your
theoretical arguments at the within-person level for most theories that
matter in our field. Whether you've done so convincingly is a totally
different question, particularly if you are choosing your design based on
the ease of implementation.
-Dan
On Wed, Sep 21, 2016 at 9:37 AM, Christopher Barnes <chris24b@xxxxxx> wrote:
I'm not sure if I agree that our theories are by default at the between
persons level. I guess anything with a clearly stable trait would be. But
aside from that, our theories are often either ambiguous or may even be
implicitly tilted toward within person effects. Many of them have an
implicit assumption that the intervention works for everyone (on average),
which can sounds like a within-persons theory.
Sent from my iPad
On Sep 21, 2016, at 6:20 AM, Butts, Marcus M <mbutts@xxxxxxx> wrote:
Hi ESMers,
I had a thought the other day that is tangentially related to Dan’s last
comment about within-person variance. I’ve seen more and more ESM studies
recently where the theorizing doesn’t match or warrant an ESM design, but
it felt like the authors did an ESM because the sampling is done faster and
with a smaller total sample size. As a Reviewer, I think my default is
that our theorizing usually takes a between person type of angle and the
sampling should match that angle. So, I guess my question to the group is
when do you decide an ESM framing and design is warranted (from a
researcher or reviewer perspective)? I know, personally, I find more and
more people that say, “my ESM worked out great, I had 32% within person
variance in variable X.” But to me, that means 68% of the variance is
still residing at the between person level…and empirical evidence of
within-person variance has little to do with theorizing something as a
within-person versus between-person phenomenon. Thoughts?
Thanks all!
-Marcus
<B636040F-F046-4B54-B846-82400A2A3AFB[20].png>
*M**a**rcus Butts*
Associate Professor
*Department of Management*
Box 19467
701 S. West Street, Suite 212
Arlington, Texas 76019
T 817.272.3855
F 817.272.3122
mbutts@xxxxxxx
From: <dailyworknet-bounce@xxxxxxxxxxxxx> on behalf of Dan Beal <
dbeal@xxxxxx>
Reply-To: "dailyworknet@xxxxxxxxxxxxx" <dailyworknet@xxxxxxxxxxxxx>
Date: Monday, August 29, 2016 at 4:42 PM
To: "dailyworknet@xxxxxxxxxxxxx" <dailyworknet@xxxxxxxxxxxxx>
Subject: [DWK-NET] Re: Welcome to the DWK listserv!
Hi all,
In response to Joel's point about selecting out subsets of items, if
there's reasonable evidence of unidimensionality, then extracting out
smaller subsets of items should theoretically not matter. On the other
hand, if a scale is simply high in internal consistency but there's no good
information about factorial structure, then you're right, extracting a few
items from a larger scale could easily change the nature of the resulting
scores.
Concerning within-person variance, it's interesting to me that it's
sometimes hard to predict which items will vary more or less. The broadly
framed ones seem to be based on more stable perceptions and so they might
be more resistant to daily fluctuations. For example, if responses to the
item are affected by a lot of different behaviors (like "I can trust my
supervisor") then it might require changes on a number of relevant
behaviors to generate a change in response to the broad item. In contrast
though, narrow-domain items (e.g., "asked what is the right thing to do")
just might not happen too often, creating the same level of low daily
variability. I suppose one strategy is to shoot for items that maximize
specificity (in terms of behavior) while maintaining a desirable base rate
of variability. You could come at it from either direction... If the item
is too broad, break it down into more specific behaviors that still seem
likely to fluctuate; if it's too narrow, broaden it without
over-diversifying the set of relevant behaviors. Sounds easy, eh? In any
event, both Joel's point about changing the nature of the construct and
Chris' observation of low daily variance suggest the need to incorporate
some measurement validation work within our ESM papers (or perhaps that's
separate avenue for publication... Development and Validation of Daily
Judgments of Burnout: The DJB Scale).
-DJB (!)
----------------------
Daniel J. Beal
Associate Professor
Department of Management
Pamplin College of Business
Virginia Tech
On Aug 29, 2016, at 4:46 PM, Christopher Barnes <chris24b@xxxxxx> wrote:
I think the use of those 3 items is reasonable. But I bet there would be a
lot of different opinions about that.
Sent from my iPad
On Aug 29, 2016, at 1:41 PM, Koopman, Joel (koopmajo) <
koopmajo@xxxxxxxxxxxxx> wrote:
Hi all:
I hope I am not sending this email twice… I sent this over the weekend,
but I think it might not have gone through? I asked a couple folks and they
said they did not get it.
Anyway, apologies if it did and I’m sending twice, but I wanted to weigh
in on the question with daily ethical leadership. Please see below:
--------------------
Hey folks:
So, I just realized that I have some data that can speak to this also.
Chris, did that study measure all 10 items for Ethical Leadership? We did
this in an ESM study, but only chose 3 items: "my supervisor set an example
of how to do things the right way in terms of ethics" | "my supervisor
asked 'what is the right thing to do' when making decisions" | "my
supervisor disciplined employees who violated ethical standards." This was
just based on us reading each of the items and coming to consensus on which
we thought might be likely to have the most daily variance.
For us, these seemed to work pretty well. Each item had 49% or more of the
variance at the within-level (high of 61% for the third item). And, a scale
created of the three items had 48% of the variance at the within-level. And
so Chris, my question about whether all 10 items were measured, is I wonder
if you're getting variance across items in terms of how much
within-individual variance there is. Perhaps some items ("conducts his/her
personal life in an ethical manner") have very little within variance and
so are dragging the measure down? Alternatively, I think your other
explanation (acquiescence) could also easily explain that.
The broader point, I think, that choosing the items carefully is really
important to get variance. But, are we creating different problems? As we
know, it is common in ESM studies to use only a few items. But I always
wonder when doing this how much of the construct we're actually capturing,
and even further, is it even ethical "leadership" anymore? At what point is
it a different construct altogether?
Dan, I've attached a recent Mayer paper with the 10 items in a nice
Appendix. If it helps, I have some highlighted and some underlined...there
is a distinction sometimes made between "moral manager" and "moral person"
and so I have that distinguished for myself (highlighted are moral manager,
and from those we chose our items). And so, this might also have
implications for item choice as well, in that we cover one subdimension
pretty well, but completely omit the other...Maybe Chris' data would show a
distinction between these in terms of variance if they measured all 10?
J
*From:*dailyworknet-bounce@xxxxxxxxxxxxx [mailto:dailyworknet-bounce@fr
eelists.org <dailyworknet-bounce@xxxxxxxxxxxxx>] *On Behalf Of *Christopher
Barnes
*Sent:* Monday, August 29, 2016 12:52 PM
*To:* dailyworknet@xxxxxxxxxxxxx
*Subject:* [DWK-NET] Re: Welcome to the DWK listserv!
You may be right. At least some of these items are asking participants to
make an inference rather than report a behavior.
I think the more concrete the behavior, the more opportunity for daily
variance. There’s also probably a floor effect on some of these items (like
observing a leader punishing someone) that makes it harder to detect any
variance at all at this level of granularity.
*From:*dailyworknet-bounce@xxxxxxxxxxxxx [mailto:dailyworknet-bounce@fr
eelists.org <dailyworknet-bounce@xxxxxxxxxxxxx>] *On Behalf Of *Gabriel
Rossetti, Allison S - (asgabriel)
*Sent:* Monday, August 29, 2016 9:34 AM
*To:* dailyworknet@xxxxxxxxxxxxx
*Subject:* [DWK-NET] Re: Welcome to the DWK listserv!
Hey Chris—I am wondering if these leadership behaviors are as easily
observable as other leader-based phenomenon? For instance, abusive
supervision seems to be more easily observed by employees throughout the
organization (e.g., I heard my boss yelling at someone down the hall; I saw
my boss today and he looked angry, etc.), but for these, it seems as though
you really have to be having close, daily conversations with that leader to
know if he/she is being ethical or not.
This item seems as though you might get more variance since it is more of
a generic evaluation:
1.
Today, my supervisor can be trusted
*1*
*2*
*3*
*4*
*5*
Just my two cents this sunny Tucson morning. J
Allie
____________________
Allison S. Gabriel, Ph.D.
Assistant Professor of Management and Organizations
Eller College of Management
The University of Arizona
1130 East Helen Street, McClelland Hall 405FF
Tucson, Arizona 85721-0108
Email: asgabriel@xxxxxxxxxxxxxxxxx
Phone: (520) 621-1390
*From:*dailyworknet-bounce@xxxxxxxxxxxxx [mailto:dailyworknet-bounce@fr
eelists.org <dailyworknet-bounce@xxxxxxxxxxxxx>] *On Behalf Of *Christopher
Barnes
*Sent:* Monday, August 29, 2016 9:29 AM
*To:* dailyworknet@xxxxxxxxxxxxx
*Subject:* [DWK-NET] Re: Welcome to the DWK listserv!
Here is the scale:
*Please indicate the frequency, with which your supervisor engaged in each
of the following behaviors today at work,*
*Ranging from 1 = Never to 5 = Five or more times.*
1.
Disciplined an employee who violated ethical standards.
*1*
*2*
*3*
*4*
*5*
1.
Discussed business ethics or values with employees.
*1*
*2*
*3*
*4*
*5*
1.
Set an example of how to do things the right way in terms of ethics.
*1*
*2*
*3*
*4*
*5*
1.
Defined success not just by results but also the way they are obtained.
*1*
*2*
*3*
*4*
*5*
1.
Made fair and balanced decisions.
*1*
*2*
*3*
*4*
*5*
1.
Had the best interests of employees in mind.
*1*
*2*
*3*
*4*
*5*
1.
Listened to what department employees had to say.
*1*
*2*
*3*
*4*
*5*
1.
Asked, “what is the right thing to do?” when making decisions.
*1*
*2*
*3*
*4*
*5*
1.
Today, my supervisor can be trusted
*1*
*2*
*3*
*4*
*5*
1.
Conducted his or her personal life in an ethical manner
*1*
*2*
*3*
*4*
*5*
*From:*dailyworknet-bounce@xxxxxxxxxxxxx [mailto:dailyworknet-bounce@fr
eelists.org <dailyworknet-bounce@xxxxxxxxxxxxx>] *On Behalf Of *Daniel
Beal
*Sent:* Saturday, August 27, 2016 3:41 AM
*To:* dailyworknet@xxxxxxxxxxxxx
*Subject:* [DWK-NET] Re: Welcome to the DWK listserv!
Hey Chris,
Could you give us an example item? I find it much easier to think about
daily or monetary variability by kinda walking through answering the items
in my head.
Thanks,
-Dan
----------------------
Daniel J. Beal
Associate Professor
Department of Management
Pamplin College of Business
Virginia Tech
On Aug 27, 2016, at 12:08 AM, Christopher Barnes <chris24b@xxxxxx> wrote:
I'll add a little detail to Marcus's recap of my part of the conversation.
Some colleagues and I just collected some data to examine daily sleep as a
predictor of daily ethical leadership. I was invited into the project at a
relatively late stage, with data collected by someone in China. We adapted
the wording of the scale to indicate "today".
Only 11.5% of the variance in ethical leadership was at the day level. At
this point, I can't be sure of the reason why, but these are possibilities:
1. This construct really doesn't vary at the day level.
2. The participation rate was nearly 100%. So maybe people felt pressured
to participate, and gave junk data.
3. Maybe better adaption of the scale is needed.
4. Something else.
Anyway, just thought I'd share a heads up. Tread with caution if you want
to examine daily variance in ethical leadership.
Chris
Sent from my iPad
On Aug 26, 2016, at 8:59 PM, Butts, Marcus M <mbutts@xxxxxxx> wrote:
Hi DWK netters:
I’ve finally gotten around to creating a listserv for our Daily Work
Knowledge Collaborative. Feel free to use this listserv as a opportunity
for any thoughtful questions/comments about ESM concepts or methods you
might have. For those that were able to get together at the AOM social, we
had some lively discussions about ESM research. For example, Chris talked
about example leadership constructs that he felt don’t vary much from
day-to-day, Klodiana mused about better integrating daily ESM findings with
more static findings that are in opposition (i.e., variable X is harmful
over time, but is helpful day-to-day), and Joel and I discussed
multi-source missing data in the context of ESM. Those types of
discussions would be great to have occur over this listserv. So, feel free
to post away! Oh, but one caveat. I’d like to ask that we steer clear of
posts such as “I just got an R&R where Reviewer 2 wants us to do…How should
I address that?” Because it is such a small ESM community, there is a good
chance one of us is a Reviewer on your manuscript. If you are going to ask
“advice” questions, please pose them in a way that best maintains the blind
review process.
Thank you all again for being so enthusiastic about this endeavor! I hope
it benefits us as fellow scholars, and helps ESM research and accepted
guidelines/reporting progress further. Below are the members of this closed
listserv, and some obligatory listserv directions are also provided.
Best,
-Marcus
DWK-NET list members:
Chris Barnes
Dan Beal
Marcus Butts
Mike Christian
Charlotte Fritz
Allison Gabriel
Russ Johnson
Joel Koopman
Klodiana Lanaj
Brent Scott
Kristen Shockley
Lauren Simon
John Trougakos
DWK-NET info:
- To post on your mailing list, simply send email to
dailyworknet@xxxxxxxxxxxxx.
- Subscribers can join your list by sending email to
dailyworknet-request@xxxxxxxxxxxxx with 'subscribe' in the Subject field
OR by
visiting your list page at //www.freelists.org/list/dailyworknet.
Requests are then reviewed by the moderator.
- Subscribers can leave your list by sending email to
dailyworknet-request@xxxxxxxxxxxxx with ‘unsubscribe' in the Subject
field OR by
visiting your list page at //www.freelists.org/list/dailyworknet.
- Online, searchable archives of your list are available at
//www.freelists.org/archive/dailyworknet Each list's archive is
automatically updated as new messages come in.
- There's a FAQ at //www.freelists.org/wiki/the_faq that you might
want to let your subscribers in on.
If you have any questions or comments, at any point, please feel free to
contact us by visiting //www.freelists.org/contact.html or by
emailing
staff@xxxxxxxxxxxxx.
-- The FreeLists Team
[image: cid:0447929E-203F-41E0-8979-AC92813FA60B]
*M**a**rcus Butts*
Associate Professor
*Department of Management*
Box 19467
701 S. West Street, Suite 212
Arlington, Texas 76019
T 817.272.3855
F 817.272.3122
mbutts@xxxxxxx
<Mayer, Aquino & Greenbaum (2012, AMJ).pdf>
