Wednesday, November 23, 2005
Ask the Administrator: How to Measure Advisement?
Some measures are fairly easy, if indirect. Does the professor usually show up for office hours? Do we ever see students there? Does the professor show up on in-person registration days? Do we ever get that professor’s signature on change-of-major forms or course-substitution forms?
The problem with these measures, of course, is that they’re vague at best. It’s possible for office hours to consist of social chatter, rather than real advisement. It’s possible that advisement can be erroneous, causing more problems than it solves. Change-of-major and course-sub forms could show active advisement, earlier mistakes, or even new mistakes.
(We also have a group of faculty who get stipends for showing up at in-person advisement sessions during the off-contract times of year. Presence in that group is generally smiled upon, although one could certainly argue that extra pay is its own reward.)
These measures worked tolerably well when advisement and registration were entirely in-person. Now that registration is increasingly on the web, though, measuring advisement is trickier. I certainly don’t want to start snooping through faculty email accounts to see how much they’re helping students that way – the issues there are too many, and too ugly. But without some sort of monitoring, I have no way of knowing which professors are actually carrying the load, and which free-riding. (Confessions of free-riding are few and far between.)
Failing to monitor, effectively, leads to failing to reward. Failing to reward, over time, leads to failure to bother in the first place. I’m concerned that we’re kind of running on historical momentum, force of habit, from the days when everything was in person. That’s fine, for now, but it’s probably not sustainable.
Does anyone out there have a system that effectively rewards faculty for active, distance advisement? How does it work?
Then there's ongoing advisement throughout the semester. I do a great deal more advisement since the advent of email and also have to occasionally explain that we do get breaks from that, too. :-)
This is a great topic to discuss as the traditional model of "in office" hours doesn't work anymore. However, that can be a good thing for those of us who prefer not to measure productively simply by who can spend the most hours in the office when job demands vary by time of year and by position (ie I'm out in the schools a great deal with student teachers and grant teachers).
Thanks for opening up this discussion!
How about asking students when they register for classes to fill out a brief on-line evaluation form that asks questions like:
"What professors have provided helpful advice to you in choosing your classes this semester?"
"If your advisor wanted feedback to help him/her become more helpful to advisees in the future, what would you tell him/her?"
"What questions do you still feel uncertain about?"
"What was the most useful advice your advisor gave you?"
It would also be useful to ask students to fill out this questionnaire again when they log on to find out their grades at the end of the semester. (Kind of retrospective reality check.)
And you could also ask students to respond to the questionnaire again years down the road when they log on to request a transcript to be sent when they are transferring, applying to graduate or professional school, need one for employment, etc.
This isn't just about rewarding evaluation--it's also about giving professors the information they need to improve their advising.
Re. email/distance advising, surely an easy way to do it would be to ask faculty to cc/forward those emails to whoever is in charge of giving them 'credit' for advising? Of course, the down side is, more work for whoever it is reading all the emails.
Or assessment could also be included as part of a year-end review, e.g., having faculty print out representative email exchanges, solicit and provide student letters re. advising, something like that.
Finally, surely gathering certain statistics could help; are advisors assigned to particular students? If so, one could do "outcomes assessment" (ha!) on the students--e.g., do Dr. X's students, the ones whose last names end in A-G, graduate in time? Do they have problems fulfilling requirements in a timely fashion?
Thanks for taking on my question!
I'm getting some good suggestions here from folks, especially from Mathsophie and BitchPhD.
Here's my thinking so far: http://bardiac.blogspot.com/2005/11/advising-documenting-measuring.html
Thanks for the conversation, everyone!
Still, sounds like the kind of thing a grad student with the right data and a good statistics package could make quick work of.
I'm not sure I follow this, would it be so they can evaluate the content of the email? or your response to the student? or just the fact that the emails exist?
It seems like another downside would be the privacy of the student involved, and of the advisor too. I mean I guess all emails to professors are property of the school anyway, but I feel like I'd want to warn students that anything they write to me would also be seen by (whoever). Maybe it would affect what I wrote, as well. Maybe that wouldn't be a bad thing? I don't know.
I guess I'm a little wary of sharing the contents of emails between two people to a third party..
Also, depending on the students and the advisor, the load may be different. I'm great about answering emails. As a result, NO ONE comes to my office hours. I mean, NO ONE. It is at the point where I wonder why I bother. Huge waste of time. Much easier to arrange a meeting with students as needed at a mutually convenient time.
I did some more thinking...