Standardized Testing

[TheMeInTeam]

Now we can state something a little bit different about the GRE, IIRC Owen and Azale both have some horror stories about what they experienced in the "reading" section of it. My experience with the "reading" section of the GRE didn't actually seem to do anything but ask vocabulary questions and finding antonyms and synonyms. While I feel this can be useful, this turns the GRE into nothing more than an "important" bar trivia contest.

It's probably a botched attempt at one's ability to comprehend the intention of writing in context in some cases. When I was still doing it I used to see a lot of questions with regard to the motivations/intentions of characters in a paragraph or two, or questions as to how they likely felt. I know they've changed the structure a few times though so maybe it's different now.

It's why I said quality is crucial to these tests if used in this capacity, and yet it also makes one question the quality of non-standard tests given in schools on any given week where students receive grades and progress based on the outcome, as in my experience the latter isn't any better.

Yeah, but this doesn't work, does it?

They're looking at standardized test performance and not any sort of increase in the quality of education. It seems that the quality is decreasing, even.

You do need a test that requires some degree of quality education in order to perform well on it, yes. If it's not designed in a way that can reliably assess student knowledge and capability to at least some extent, however, the problem lies squarely with the test.

You will never pass a CPA or Medical Board exam and gain a license by accident. You can't do it, you have to have substantial knowledge of the relevant discipline far beyond what a lay person has to be capable of passing them...and if you do pass them your knowledge (and for CPAs, ability) is sufficient to at least be competent in the field (medicine has a practical element also of course that gets tested extensively in addition). I assert that standardized testing is lacking in quality and standards, not in concept.

 

[warpus]

Yeah but i mean, standardized tests seem to test how well students are at learning how to prepare for a standardized test - which doesn't necessarily have to have anything to do with how much they know about the subject matter or the quality of education.

 

[TheMeInTeam]

Yeah but i mean, standardized tests seem to test how well students are at learning how to prepare for a standardized test - which doesn't necessarily have to have anything to do with how much they know about the subject matter or the quality of education.

I don't know what to say in response to this. If you're arguing the test is poor I'd say that is a fair estimation, and that its quality and standards for progression could both use a step up.

Scoring well on the test SHOULD require knowledge of the subjects (and applying it), and thus preparing for it SHOULD involve learning the subject matter. The reason I cited those professional exams is precisely because they're examples of standardized testing that do exactly that, and they do it pretty effectively.

 

[warpus]

I'm arguing that students who perform well on standardized tests might very well just have been conditioned/trained/taught how to perform well on certain types of standardized tests, which would be an indicator of them being good at that specific type of standardized test, but not really in any way on the quality of the education at the school in question.

Seems to me that since funding relies on these tests, schools are likely doing everything in their power to make sure that their students are performing well on these tests - as opposed to actually increasing the quality of the education.

So while doing well on these tests should mean that you know stuff about the subject matter, it doesn't necessarily mean that your school is providing a higher quality education than a school whose students do worse on that test.

 

[Borachio]

That's likely true.

But any sort of testing has drawbacks.

It should be obvious, though, that without some sort of testing, no-one could have any knowledge at all (never mind flawed knowledge) about the quality of education that children are receiving.

 

[downtown]

The reason you tie funding to good performance is that 1) funding is a desirable resource that drives district (and other) decisions and 2) performance is a desirable outcome (as opposed to poor performance, or having a lot of people regardless of the quality of their education).

The idea is that because districts want funding, they will attempt to improve their students' performance. In the current model, there are a few problems, namely that the standardized test isn't particularly high quality and of course that the students are not bound by any similar tangible incentive...both are large noise factors that damage performance.

This is not an especially accurate description of US school funding. I'll get to this in a second.

By extension, one could get a picture of the quality of the teaching given based on the scores, though of course you'll get some variance based on student background and just sampling (some classes are just stronger than others).

That variance, btw, is HUGE, which is why using value-added assessments, or multiple assessments, is so critical.

If you are interested in the topic, you can look at what firms do when hiring. They do screening with standardized tests, and since it's their own $$$ on the line based on their hiring decisions the incentives (and test quality) are a lot tighter, though still imperfect.

I don't know if there is any evidence to suggest this is actually true, tbh.

Aside from simply evaluating writing skill, literature struggles to demonstrate that other methods of assessment than multiple choice provides a material advantage though, no? You can't evaluate certain things that way (writing, spoken language, some aspects of critical thinking, practical physical skills etc), but as a method of assessment as long as you steer clear of multiple true/false (IE poorly written questions) there's not much evidence of other methods doing better.

My graduate school literature was crystal clear on the subject.

The different ways that students process information is tracked on something called Bloom's Taxonomy. Blooms is a pyramid, with lower levels of understanding at the bottom, which lead to higher levels of understanding. Your lower levels are associated with fact memorization, methodology, etc. These lead up to being able to make inferences, comparing, organizing, and then finally into what's called "synthesis" (combining multiple facts, abstract thought, analysis, teaching the concept to others, etc"

It is very very very difficult to measure multiple levels of understanding via a multiple choice assessment. It just doesn't give you enough data. For K12 educators, particularly those who teacher younger children, it is critical to vary your assessments to hit different levels of Blooms, from worksheets, essays, class projects, presentations, multiple-step problems, and more.

It's difficult to standardize a lot of those, which is why we have what we have...it's imperfect, but still a necessary tool, especially since our K12 system is fragmented on every level

 

[jackelgull]

As a student who has just taken the AP Lit exam and the SAT exam, here's a poem I wrote about the whole experience.

Spines arch downwards, pressed
upon by unseen silence,
like pens on paper.

The bell tolls. For whom?
For us. We don't notice. The
silence is draining.

Finally it is the end.
We leave. The silence gives a
round of applause and

gives way to chatter.
How did you do? Oh my god
I totally failed.

The memories of
the silence remain. After
we stream out of the

jail like school building
does the shadow disappear.
Our bones are weary.

Pretty bad eh?

On a more serious note, standardized tests are a pretty good measure of how much mental punishment you can take before your brain gives up, and are as much about that as testing skills. I think that this function justifies their existence. Life is about taking punishment and staying calm under strenuous situations, ain't it? Or at least that is a good skill to learn.

 

[Wrymouth3]

It's about time you learned young literary critic that Hemingway was just writing about finishing the last 1/16th of his Jack Daniels with his Orange Tabby. De Quincey was just writing about how much he didn't brush his teeth and ate poppy. Catullus really meant... well he really meant what he meant.

 

[jackelgull]

It's about time you learned young literary critic that Hemingway was just writing about finishing the last 1/16th of his Jack Daniels with his Orange Tabby. De Quincey was just writing about how much he didn't brush his teeth and ate poppy. Catullus really meant... well he really meant what he meant.

You remind me of a friend who told me that the poem Berry Picking is about first time sex. The teacher told him it was about how experience tainted the poet's child hood memories. My friend told her, "yeah, that's what I said. The poet only realized how bad his first time was after he had more experience."

Needless to say, she wasn't impressed by the answer.

 

[Wrymouth3]

You remind me of a friend who told me that the poem Berry Picking is about first time sex. The teacher told him it was about how experience tainted the poet's child hood memories. My friend told her, "yeah, that's what I said. The poet only realized how bad his first time was after he had more experience."

Needless to say, she wasn't impressed by the answer.

You win sir, I wish I was there when that was stated. The poem certainly suggests this, and we all know poets LOVE to write about lovely times .

 

[TheMeInTeam]

This is not an especially accurate description of US school funding. I'll get to this in a second.

To be clear, at some point during my argument it morphed from "what is currently happening" to more along the lines of what I'd like to see. This is a result for some of the disparity you cite below too. I did not make this demarcation clear in my previous posts, so let me clarify that I'm not (intentionally) asserting this is how it currently works.

That variance, btw, is HUGE, which is why using value-added assessments, or multiple assessments, is so critical.

Are you talking about the test samples, or the student samples? Even with the same school in the same neighborhood with very similar conditions, one class of 30-100 students could easily be markedly better or worse than the previous class. Tests share this "better or worse" potential too, a rather annoying noise factor but that's the reality.

It is very very very difficult to measure multiple levels of understanding via a multiple choice assessment. It just doesn't give you enough data. For K12 educators, particularly those who teacher younger children, it is critical to vary your assessments to hit different levels of Blooms, from worksheets, essays, class projects, presentations, multiple-step problems, and more.

I disagree. What's so difficult about writing 2nd or 3rd order questions that require application of multiple layers of knowledge and making judgments to come up with the answer...aside from a mastery of the content for the writer?

Some of what you said would fit under what I implied as "practical" skills. Presentation skill, writing skill, and generation of working spreadsheet are all actions that generate something...for these kinds of assessments you can easily test the knowledge of them in multiple choice, but you need to see the end product itself to evaluate ability to create them.

However, I'd assert that no current or previous model has captured ability in these departments particularly well in a consistent manner for the general population. As a result, you standardize what you can and move assessment for practical abilities down the line, altering the education incentive structure/ordering a bit. Not every career needs presentations or the ability to build a spreadsheet, and IMO these things are better equipped to be assessed by trained people in the fields that use them than arbitrary (sometimes great, sometimes clueless) teachers k-12.

I wouldn't say that standardized testing is the only measure that should be used, just the best objective one that we can consistently apply to different people to compare them (making some assumptions about the quality of the test).

 

[downtown]

Okay, so before I get to a couple of these...here's how schools in the US are funded, and their relationship with standardized tests. The specifics will vary a little from district to district, but the basics here are basically the same everywhere.

The biggest chunk of school funding comes locally, usually from property taxes. The exact amount will vary, depending on enrollment and the tax rate that the city sets, but this funding is (with a few very specific exceptions), agnostic to quality. The district gets it no matter what.

Next, the district will get supplemental funding from their state. This might come from state lottery money, a state income tax, or something else. This is also not typically tied to test results. How much a district gets from the state (instead of their city) depends a lot on their state, and the demographics of their district. For non-inner city districts, these two combine to make up most of their funding.

Finally, we have the federal govt, who don't really pay for day to day operations or buildings, but for special grants, or programs. One of the largest, and perhaps best known, is Title 1 (which provides funding and programming for high poverty schools), but there are other programs and grants that help pay for ESL instruction, IEP programming, new technology in schools, and others.

Some of these grants are entitlements, but many, especially after NCLB, are not, and MAY be dependent on certain value-added test benchmarks, among other things. The Obama administration also tied a huge amount of grant money towards certain goals and tasks (Race to the Top), some of which included test score benchmarks, but also to lots of other things....changes to how teachers are paid, how tests are given out, how data is kept and recorded, class sizes, etc.

Not that many schools have actually lost funding just because of test scores, because you don't get penalized for single year results (it's a multi-year window), and because you can apply for waivers.

Now, schools CAN more commonly lose funding b/c of test scores in three ways:

1) Most states have an emergency provision, where if a district scores so poorly on tests for X number of years, without improvement, the state can take over the district, fire all the administrators and teachers, and start over. This could result is a changing of district boundaries and enrollment, which impacts funding. Plus, people lose their jobs.

2) Many districts set part of teacher pay (or even administrator pay) to test score goals. There actually isn't very much research that this leads to better student achievement, but lots and lots of places do it anyway.

3) To the extent that families in a district have the means to do so, repeated years of poor scores will cause enrollment to drop, which lowers funding (and, like insurance, can cause a district death spiral).

 

[TheMeInTeam]

2) Many districts set part of teacher pay (or even administrator pay) to test score goals. There actually isn't very much research that this leads to better student achievement, but lots and lots of places do it anyway.

When you say this, are you referring to better student learning/knowledge, or to better performance on the test scores? What incentive do students have to do well on these tests (that said students can perceive)?

 

[downtown]

When you say this, are you referring to better student learning/knowledge, or to better performance on the test scores? What incentive do students have to do well on these tests (that said students can perceive)?

i'm talking about improved test scores.

If students don't pass the tests, they can't graduate high school, and in many states, advance their grade (In Louisiana, where I taught, you can't go to 5th grade without passing the test, regardless of grades). That's a pretty healthy incentive for kids to do well.

A district where everybody passes and the only question is whether you get an 70% or a 90% isn't going to spend money on merit pay.

 

[downtown]

I wish I had a better solution for standardized testing to measure progress and success, but I think that parents should be able to choose which schools they send their kids, rather than forcing them to send them to the closest one. Basically being able to opt out the public school system, and not be taxed for a school that your kids don't attend.

If it is clear that the parents don't give two shi&$ about their kids education, then let the authorities step in and send them to the nearest school and tax the parents for tuition.

School Choice is a really complicated issue (my mom wrote her PhD thesis on it, and is currently researching it in North Carolina now), especially as we find that figuring out the "best" school is very difficult, and the families who could use that choice the most have the hardest time (and are the worst equipped) making it.

I strongly disagree with the idea that you should only be taxed for the district where your kids attend. Everybody in a community, whether they like it or not, is invested in the quality of the local district, even if you don't have kids. It impacts your property values. It impacts your local employment rate and economy. The physical facilities themselves are for community use. Because of that, everybody has to have a stake in paying for them.

It'd be way too expensive otherwise. Urban districts spend 13K per student. That's the tuition of a top tier-in state university. We're going to expect even middle class families to pay that every year for K12?

Property taxes (supplemented by the occasional sales or income tax) probably isn't the more fair way to pay for schools, but the only fair way to do it is to make everybody chip in.

 

[MacAttack]

Day care is expensive and we get to choose that, and I like the flexibility to pull my kid out because they suck.

The property values are tied to those school districts because locals are forced to attend. You missed my point that competition between schools is what would drive the quality up. Schools wouldn't say, aw shucks, we are terrible we will remain terrible and all the kids will be elsewhere. They would put their big boy pants on and fix their inadequacies. If they don't then heads need to roll.