ACET, Inc. Blog

Recently, John Gargani shared a list of 5 words that evaluators use that can be misleading in reports. John’s blog inspired us to identify “pet peeves” we have found in evaluation work. Here’s our own list:

“Problem”

The use of the word “problem” in itself can elicit negative responses and put people on the defensive. When people are defensive, it can be difficult to engage in open dialogue and creative problem solving. Instead of using the word “problem,” Stella recommends using creative solutions to frame the issue to facilitate discussion. For example, Stella suggests using the word “challenge,” as in, “Participants identified some challenges in receiving quality health care…”

“Larger Population”

Joseph highlighted the overuse of questions about a “larger population,” as in, “how do our results compare to the larger population?” In general, evaluation results are obtained from a specific population within a specific context. Even results from “larger populations” are obtained from specific, albeit larger, populations within a specific context (e.g., Minnesotan 3rd through 8th grade students), and not from some generic “larger population.” In order to make valid comparisons, it’s important that evaluators always classify both their population of interest (e.g., South Minneapolis teens, between the ages of 13 and 17, from households with incomes of $25K or less) and the comparison group (e.g., a sample of Minneapolis teens from households with a variety of income levels). As often as possible, evaluators should not use generic “larger populations,” but should be as specific as possible about their comparison groups to enhance the validity and accuracy of their comparisons.

Describing the “Other” Category

Dan pointed out that, when analyzing qualitative data, there might be 5 common themes, but also an “other” theme for comments from a small number of people. When summarizing the results of the qualitative analysis it is important to talk about all of the themes, including describing what “other” indicates. Without describing what “other” indicates, the reader may have no idea as to what the writer is referring and may have to guess what “other” indicates. It is important that evaluators be as clear as possible when writing to ensure that the reader has a clear understanding of the data.

Pronouns

Kirsten’s pet peeve is overuse or abuse of pronouns. Pronoun overuse can detract from the clarity of writing and the overall readability of the document. From a grammatical standpoint, a pronoun always refers to the last used noun; if the last used noun isn’t what the writer intends to refer to, then a pronoun is inappropriate and the writing will not be as clear as the writer intended.

As a rule of thumb, Kirsten alternates noun-pronoun-noun-pronoun. She acknowledges that her method doesn’t always produce highly engaging prose, but it is clear!

Do you have any writing “pet peeves”? If so, please share in the comments below!

ACET Staff

As another school semester comes to an end, the staff at ACET wanted to review some tips for combating public speaking nervousness. Compiled from multiple sources, here are five of our favorites:

Know your audience. In your preparations, make sure that you understand what your audience requires of your speech structurally (for example, 15 minutes plus a question and answer period, or a self-introduction followed by 10 minutes) and their knowledge level (that is, how much of your topic needs explanation). Before you are to give your remarks, take the time to meet with some of the people who will be in your audience. This will strengthen their bond to you, increasing their interest in what you have to say.

Work from an outline, rather than writing down or memorizing an entire speech. Sure, people can and do give speeches that involve reading or speaking directly from a pre-written statement but, at best, this approach can feel unnatural and, at worse, it can distance or cut you off from the people with whom you are trying to communicate. Note your key points and use them as signposts to expand from naturally. This will in turn make your delivery more natural. If you happen to miss or skip a point along the way, that’s okay.

Give your speech a personal touch. You’ve likely heard the saying, “it’s not what you say, it’s how you say it.” Just as it’s good to be cognizant of the structure of your presentation, it’s also important to choose the best tone or mood for your audience. Sharing a related story can help to establish a connection with your audience, helping them to become more engaged, and help you make an easy segue between introducing yourself and your data.

Practice, Practice, Practice. Whether you record your voice, speak in front of a video camera, or rehearse for family, friends, or coworkers, you can gain a lot of strength and confidence from just practicing your remarks and then sitting down with the recording or your audience to review. No matter the format you choose, practicing can help you to refine your abilities.

Relax. Know that it is very natural and human to be nervous when addressing a large group of people, but you can use the jitters that you feel to propel yourself to excellence. Remember to breathe as you speak, and allow yourself to be comfortable with brief silences as you do so, as it can help you combat the use of fillers (“um”, “ah”, “you know”, etc.). Cold liquids are said to constrict the vocal chords, so consider sipping on lukewarm water or hot tea before you speak, and save the caffeine for later.

With these tips and some careful preparation, public speaking does not have to be a daunting experience.

Information was sourced from Forbes, Psychology Today, Monster, Entrenpreneur, timesunion.com, and the University of Leicester for this article.

Mary

Here is a link that highlights a variety of webinar hosts for small businesses.  This is a great place to start if you’re looking to add webinars as an option in your program.

When you want to share information, either hosting a presentation or simply wanting to share work on a document, what resources do you use? We’ve found some particularly useful tools for sharing and presenting information (click on the link attached to the application to view more about that resource). This week we’re posting on tools for sharing documents.

A simple way to share documents that are not confidential is by uploading them to an online space. By using an online space, users are able to access the document anywhere they have internet access.

Google offers a free online sharing application called Google Documents that allows users to not only share documents, but also work on a document at the same time as other users. Once uploaded, users can edit, add, and change information in a document much like they can in Microsoft Word or a similar text editing software. When one user uploads a document, they can share with other users who can view or edit those documents. The original user must have a Google account to upload the document, but any other users do not need a Google account.

Dropbox is also a free project collaboration tool that allows users to upload and store files on multiple spaces of their choice. Just as with Google Documents, users can upload documents, photos, and more. An additional feature that Dropbox offers is the ability to link the application to multiple computers and phones. The program creates folders on each computer or phone that a user wants, allowing a safe space to upload documents without having to go onto a website.

Stay tuned for next week’s post on online resources!

Cassie

Recently, I detailed the issues with describing evaluation and the work we do here at ACET (http://www.acetinc.com/blog/2010/11/22/part-1-%E2%80%9Cso%E2%80%A6what-is-evaluation%E2%80%9D-dealing-with-blank-stares). As I mentioned, often we are asked to include a basic definition for evaluation and to elaborate on what an evaluation looks like.

If you search online for a basic definition of program evaluation, you would likely find something akin to the Wikipedia classification, which states, “a systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency.” http://en.wikipedia.org/wiki/Program_evaluation.

You may also be interested in reviewing the recent expanded definition of program evaluation developed by The joint Committee on Standards for Educational Evaluation recently (The Program Evaluation Standards, 3^rd Ed., 2010). To learn more about the evolution of evaluation definitions, I suggest reading Utilization-Focused Evaluation by Michael Patton, one of the pioneers of evaluation (check out pages 38-39 for his review).

We at ACET have used the following definition for program evaluation: “the process of systematically collecting and analyzing data and using the results to answer questions about a program in order to improve the program’s effectiveness and/or efficiency.” We fully believe that an evaluation should be systematic (objective and specific), results-oriented (just as in education, constructive criticism can be used for good), and should be used to improve the program (we want the program to have optimal results and funding in the future).

If you have additional questions or comments on the variations of program evaluation definitions, I would love to hear from you!

Stella

At ACET, we strongly believe in “capacity building” – but what does that mean? Basically, we believe in creating and implementing evaluation plans that involve our clients. Our clients have input in every step of the evaluation process, which enables us to find the plan that best suits the needs of their program or organization. In turn, they learn how to use the evaluation results to improve programming. Since capacity building is so important to us, we work hard to ensure that we maintain open and transparent communication with our clients.

Stella learned of a video while at the American Evaluation Association conference that summed up precisely what NOT to do when communicating with your clients.

http://www.youtube.com/watch?v=AukrpAoAYY0

The video is amusing, but it brings up some essential key points.

“What does this mean?” Use clear language that clients can understand. Those test results may be second nature to you, but like the man in the video, your client may have no idea what those numbers actually mean.

“What does a score of 35 tell me about my patient?” In our experience, clients don’t want random numbers to throw around; they want solid information that they can use to improve their programming. Be sure to clearly delineate what their results mean and answer any questions they may have.

“I used the scale you developed to measure my patient.” Your client trusts that you have expertise about evaluation – after all, they’re using products you likely designed! – but being able to communicate that knowledge is essential in maintaining the relationship.

The end of the video is also telling. Despite the researcher’s supposed credentials, the client doesn’t want to deal with her anymore. Clients, too, may want to stop working with someone they feel isn’t right for their organization, even if that person or company has years of accolades.

Do you have any thoughts to share about communicating with clients? We’d love to hear them!

Over the winter I read the blog-turned-into-a-novel Julie and Julia, and, since the launch of the ACET blog, I’ve been very interested in the wide variety of blogs that are available on the internet. I currently subscribe to quite a few, including Mrs. Q’s “School Lunch Project,” the Wiktionary “Word of the Day,” and my own guilty pleasure, “Babes Love Baseball.”

I have also discovered that many of our lawmakers have embraced blogging as a way to keep their constituents informed. I was able to find a blog that summarizes the recent activities of Minnesota’s House and blogs for my representatives to Minnesota’s House and Senate. My congressional representative also has a blog, but Senators Franken and Klobuchar don’t (you can subscribe to their email newsletters though).

If you want to subscribe to a blog, look for and click on this image: 
Here are some links if you are interested in learning more:

Daily summary of Minnesota’s House (look for the “Session Daily RSS” button on the right): http://www.house.leg.state.mn.us/hinfo/sdaily.asp

Who Represents You? allows you to find your lawmakers based on your address. Click on the “Member Page” to be redirected to their webpage and sign up for the blog (RSS): http://www.leg.state.mn.us/leg/districtfinder.asp

Senator Franken’s website: http://franken.senate.gov/

Senator Klobuchar’s website: http://klobuchar.senate.gov/

White House blog: http://www.whitehouse.gov/blog

What blogs do you regularly read?

Kirsten

One of my favorite online resources is Andy Goodman’s “free range thinking.” Andy is a nationally-recognized communications specialist and he is passionate about helping people to better communicate with their audience(s). His monthly newsletter reflects that passion.

What I find most appealing about free range thinking is that each newsletter is short and has one (and only one) major point and the information can be utilized immediately. For example, in the October 2008 newsletter, Andy and a guest contributor, Eric Swartz, discuss how to create a unique tagline. And in the April 2009 newsletter, Andy describes how to develop a story bank, a collection of stories about an agency to help spread their message.

Anyone can subscribe to free range thinking and receive the newsletter for free.

I hope you find this resource helpful!

Kirsten