Standardized Testing

[Zkribbler]

If the standardized tests have no direct effects on the test subjects or anyone administering the tests, then they can give useful data. What kind of data depends on the tests.

As they are used today however, they're a complete disgrace. Their only actual purpose seems to be to rob money for the public and give it to test-making companies.

As far as I know the national tests in Norway are at least designed and owned by the government, so there's no direct profit motive involved (unless by people who are interested in keeping their jobs, but I believe they have enough other tasks that this effect is avoided).

 

[nc-1701]

Cool for you.

I'm assuming this is not a job dealing with people.

Actually it is

I think most of the hostility towards standardized testing comes from people who aren't smart enough to do well on tests.

I feel the same. Similarly I feel a lot of the teachers who critisize it are doing so not for valid reasons, but because they are afraid they won't be able to get ahead by blowing administration any more.


The criticism that Narz (and a few others) brings up that tests can't truly measure a person's value is of course true, but it also feels like a red herring. No test purports to do that. Tests can capture and allow comparison of certain knowledge or problem solving skills. Being able to differentiate such is vital for things like college admissions or grading teachers. But remember no single test will get you anywhere in life without a strong follow through, nor will any single test ruin you.

 

[downtown]

If you have rich parents you can just get tons of coaching to do well on these tests. Unfortunately there is likely zero correlation between doing well on them & success if life or even in college (not counting for income & whatnot). The only possible benefit is the discipline to study for them, which of course could be put to more useful ends like learning a 2nd language or useful skill.

I mean, it depends on the test, right?

First, I know it's popular to make fun of test-taking as an actual skill, and it's true, that there a ton of jobs where you'll never have to do it...but there are lots of jobs where you *do*, even in a similar fashion to an ACT/SAT/State Graduation Exam. Anybody that goes to graduate school will need to take a similar exam. Anybody that requires a state license (a teacher, a nurse, etc) will need an exam. Many government jobs require exams. Professional certifications typically require exams. Hell, your drivers license test may even contain multiple choice questions. And that's to say nothing of actual undergraduate courses, should you pursue a vocation that requires a degree.

Now, does the ACT/SAT itself specifically correlate with college success? The research is mixed. It seems like there is a correlation between test scores and graduation rates/GPA, but it isn't an especially strong one. That's why it's not the only data point used in admissions, and in many cases, isn't the most important one. There's also a pretty huge range in ACT/SAT scores that a student can get and still get a college education. A lower score might keep you from an elite school, but you can get a fairly bad one (say, a 19), and with a good GPA and other factors, still get a degree from a Land-Grant university somewhere. Get a 21, and you can go to a top 200 university in the country. Maybe better. If you're scoring lower than a 19, I think its hard to argue you'd be successful in that environment anyway.

But there are lots of other tests. We typically give assessments to late elementary school and middle school students, and I'd argue that those scores matter a lot. Those are testing some foundational skills, that students absolutely need to master, not just for college (or other post-secondary skills), but for their own quality of life. If a kid in late elementary school doesn't have a working vocabulary, it's going to be much harder for them to read later in life. If a kid doesn't learn times tables or get very comfortable with computation, then algebra, or even working life math, will be a real struggle.

The tests that we give here, along with coursework and other factors, can show us if children have mastered these critical, basic skills. You can have elite creativity, but man, if you can't read, or use context clues to figure out a word you don't know, you're going to struggle, and teachers/schools/parents need that data to help figure out how to help.

I think most of the hostility towards standardized testing comes from people who aren't smart enough to do well on tests.

Maybe, but around here, I think it's more resentment of being bored during those tests (which they very easily passed), or resented having to take class-time to help others with those concepts that they picked up very quickly.

 

[MacAttack]

"This video does not exist."

BTW, I'm reasonably sure Galileo wasn't talking about test scores in school, even though part of his life was spent as a teacher.

Fixed! Sorry about that! I quoted Galileo since his quote would be relevant as I figured that progress measurement would be one of the debated topics here.

On some of y'all's comments, I did okay on the standardized tests I took to get into college, yet I did very well in college and have a great career going. Yet, I had a couple of friends who were in the same university, who got nearly perfect SAT scores, yet one barely graduated (no honors) went into the military and the other didn't even make it through, and is actually a UPS driver now. Are we an anomaly?

As a parent, I reward my kids for doing well in those tests, but I give them larger rewards for doing well in their actual school work.

I am of the camp that the foundation of a successful future is rooted on the family unit's ability to raise their children right with the aid of the school system, not the other way around.

I wish I had a better solution for standardized testing to measure progress and success, but I think that parents should be able to choose which schools they send their kids, rather than forcing them to send them to the closest one. Basically being able to opt out the public school system, and not be taxed for a school that your kids don't attend.

If it is clear that the parents don't give two shi&$ about their kids education, then let the authorities step in and send them to the nearest school and tax the parents for tuition.

 

[TheMeInTeam]

Standardized tests are used because they have demonstrable predictive value. They are far from perfect, but last I checked still pick out future strong performers more accurately than alternatives.

I disagree on putting them in the hands of the department of education though. The incentive for the people running it to make sure the kids get the right coverage isn't any better than what we have now. Even if you push aside any political indoctrination (towards either party, likely both over long periods), Joe state department employee cares about as much for the average student as John private test company employee. That's not going to solve anything other than centralizing the information source.

A few things could help the standardized test process:

- Tie them to student grades so students have real incentive to do well.
- Vetting, not sure who does it though. I don't trust any one organization alone to not make discriminatory tests down one of many potential wrong paths.
- If you tie funding to these, the incentive to teach to the test is obvious. The education system would have to be built around that reality, or you have to tie funding to something else. It's naïve to the point of being inane to decide salaries and school funding based on standardized test performance and then expect schools to do anything except focus on preparing students for the tests.

 

[warpus]

It seems to me that the problem is that institutions attempt to make everything revolve around standardized tests. That seems to be a big mistake. Use them effectively, but don't clobber the kids over the heads with it.

 

[TheMeInTeam]

It seems to me that the problem is that institutions attempt to make everything revolve around standardized tests. That seems to be a big mistake. Use them effectively, but don't clobber the kids over the heads with it.

If you make funding depend on the results, you will never get a different practice.

That forces you to either fund based on other criteria, or attempt to develop standardized tests that are rigorous enough to be used in place of standard school tests. Both have significant potential downside.

 

[warpus]

Why not fund based on other criteria? Tying funding to standardized test results seems to lead to mainly negative results - for the students that is.

Why should a district which gets better scores deserve more money anyway? I'd think that the district with worse results should get more money - if anything, obviously they need more help rather than less.

It's not the kids' educations they have in mind when implementing these things at all..

 

[TheMeInTeam]

Why not fund based on other criteria? Tying funding to standardized test results seems to lead to mainly negative results - for the students that is.

Why should a district which gets better scores deserve more money anyway? I'd think that the district with worse results should get more money - if anything, obviously they need more help rather than less.

It's not the kids' educations they have in mind when implementing these things at all..

If you give more money to poor performance, you create incentive to have poor performance. Schools would certainly take advantage of that and the result would be even worse than now.

If the goal is to create a better environment for the students (one would hope that's the goal), it would make more sense to create incentives for schools to behave in their interests and make them stronger performers. The students themselves aren't getting the funding no matter what you do, so you want to fund something that helps them actually learn.

That's why I said you'd either need a rigorous standardized test that determines their grades and progression through each year of school, or you come up with an alternative measure of their success. I am not aware of a better alternative measure, right now coming up with questions that cover each of their essential subjects (and having the same score mean the same thing regardless of where it is taken or how it is taught) is the best/most evidence driven method we have available to my knowledge. Do you know of a better one? Certainly, paying schools for failure isn't going to cause schools to stop failing.

 

[warpus]

If you give more money to poor performance, you create incentive to have poor performance.

Hmm I did not consider that schools would act like that..

Why not just tie funding to the number of students in the district? Why does it make sense to tie funding to performance in the first place?

 

[Narz]

Actually it is

So whats the job?

Hell, your drivers license test may even contain multiple choice questions.

Well anyone too lazy to pass the driver's exam probably shouldn't be allowed to drive I agree with that but answering these questions has nothing to do with how well you are going to drive (are most driving infractions due to people not knowing what double yellow lines mean or the speed limit or knowing exactly what they mean & not caring?)

I'm not saying these tests don't have any value whatsoever in determining what people know. They might be ok assessment tools for certain types of understanding but tests are terrible teaching tools. You learn how to cram information & then forget it, which is really bad practice for life & also certainly doesn't help you love the material (the more test-days however, the fewer days the teacher has to engage with his/her students).

Of course its alot easier to score these types of tests (by computer) rather than essays & reports which is why they're used.

 

[Narz]

Standardized tests are used because they have demonstrable predictive value.

Good schools won't accept those with low test scores therefore they have strong predictive value over whether you get into good schools. If you made shoe size the criteria that too would have high predictive value.

 

[downtown]

They might be ok assessment tools for certain types of understanding but tests are terrible teaching tools. You learn how to cram information & then forget it, which is really bad practice for life & also certainly doesn't help you love the material (the more test-days however, the fewer days the teacher has to engage with his/her students).

Of course its alot easier to score these types of tests (by computer) rather than essays & reports which is why they're used.

Well, the test isn't supposed to be a teaching tool. The test is to assess what was already taught. Assessments are 100% absolutely vital to instruction, although ONLY using fill in the bubble, multiple choice, scantron assessments is going to cause problems.

A bubble-sheet isn't perfect, but it can be a useful tool. If a teacher is removing every other kind of internal assessment in favor of drilling with bubble sheets, that's an instructional problem (and a big one!), not necessarily a problem with the test.

Good schools won't accept those with low test scores therefore they have strong predictive value over whether you get into good schools. If you made shoe size the criteria that too would have high predictive value.

Predictive value for freshman GPA and graduation rate, not predictive value for where you go to school.

 

[Narz]

Well seeing as tests are an integral part of school it makes sense test scores would determine how well you do on future tests. You base education around something else & that can become a predictor.

 

[MacAttack]

Hmm I did not consider that schools would act like that..

Why not just tie funding to the number of students in the district? Why does it make sense to tie funding to performance in the first place?

They already do that. If you look at US national and local Gov't statistics, they count funding on a per child basis, that way districts in large urban areas like Houston or Dallas can be compared to rural areas like West Texas.

On you second question, I truly believe that the government does not have a better idea. Seems to be one of the few things that GW Bush and BH Obama agreed (well signed off) on.

I like the idea to tie the test to grades and perhaps for school funding, let families choose where they send their kids to school. Using the principles of economics can be a powerful force.

 

[downtown]

No district has 100% of their funding dependent on test results in the US, FWIW. The degree that "funding" is a function of test results varies WILDLY from state to state and district to district.

 

[Terxpahseyton]

@nc-1701

Thanks for the illustration. Maybe I was too pre-occupied with hating on the tests. I admit you make it sound like something that could do some good.

 

[TheMeInTeam]

Hmm I did not consider that schools would act like that..

Why not just tie funding to the number of students in the district? Why does it make sense to tie funding to performance in the first place?

You'd certainly see some interesting district territory wars!

The reason you tie funding to good performance is that 1) funding is a desirable resource that drives district (and other) decisions and 2) performance is a desirable outcome (as opposed to poor performance, or having a lot of people regardless of the quality of their education).

The idea is that because districts want funding, they will attempt to improve their students' performance. In the current model, there are a few problems, namely that the standardized test isn't particularly high quality and of course that the students are not bound by any similar tangible incentive...both are large noise factors that damage performance.

I'm not saying these tests don't have any value whatsoever in determining what people know. They might be ok assessment tools for certain types of understanding but tests are terrible teaching tools. You learn how to cram information & then forget it, which is really bad practice for life & also certainly doesn't help you love the material (the more test-days however, the fewer days the teacher has to engage with his/her students).

Tests are not teaching tools. Tests are assessment tools. Their purpose is to ascertain the knowledge of the person taking it, and a standardized one ensures that this evaluation is consistent between people and places.

A well-made standardized test on core disciplines should be able to assess the knowledge of the students on those disciplines with some reliability. By extension, one could get a picture of the quality of the teaching given based on the scores, though of course you'll get some variance based on student background and just sampling (some classes are just stronger than others). The tests themselves, however, are not teaching devices. They are a framework to evaluate the quality of teaching. It is absolutely critical that the concepts they test are reflective of useful knowledge and skills to the students, or they are junk.

Good schools won't accept those with low test scores therefore they have strong predictive value over whether you get into good schools. If you made shoe size the criteria that too would have high predictive value.

I am talking about standardized testing in the general sense. Well designed tests have (imperfect) predictive value, as in they give some picture of "how likely is this person to be good at doing x" that is better than alternative objective measures. I am a lot more interested in relative performance after graduation than in the "good schools". A lot of the "good schools" really means "schools for people with money", which then gives you connections to other people with lots of money and an insider advantage. However, how do these students perform on average in the work force or on other standardized tests against other students from other schools?

The gist of the matter is that if you have a good standardized test, out of 200 people taking it where 100 get 80% or higher and the other 100 get 40% or less, the 80% crowd will, on average, succeed much more often (far more of that crowd performs well later).

If you are interested in the topic, you can look at what firms do when hiring. They do screening with standardized tests, and since it's their own $$$ on the line based on their hiring decisions the incentives (and test quality) are a lot tighter, though still imperfect.

Well, the test isn't supposed to be a teaching tool. The test is to assess what was already taught. Assessments are 100% absolutely vital to instruction, although ONLY using fill in the bubble, multiple choice, scantron assessments is going to cause problems.

Aside from simply evaluating writing skill, literature struggles to demonstrate that other methods of assessment than multiple choice provides a material advantage though, no? You can't evaluate certain things that way (writing, spoken language, some aspects of critical thinking, practical physical skills etc), but as a method of assessment as long as you steer clear of multiple true/false (IE poorly written questions) there's not much evidence of other methods doing better.

I like the idea to tie the test to grades and perhaps for school funding, let families choose where they send their kids to school. Using the principles of economics can be a powerful force.

The natural choice would then be to select the school that best prepares the students for the standardized assessments. It's not necessarily bad, but the quality of that test becomes absolutely crucial, more so than it is now.

Though this is not unlike quite a few current standardized professional exams, such as CPA, CMA, medical board, actuary, law, etc. Anybody with an accounting degree and a CPA license can practice, and there's a lot at stake for them. But that sucker isn't easy and they take it very seriously, so you can have pretty solid assurance of a CPA's *capability* to do his/her work, though of course this depends on their willingness to do it properly and not deliberately mess with things .

 

[Wrymouth3]

Now we can state something a little bit different about the GRE, IIRC Owen and Azale both have some horror stories about what they experienced in the "reading" section of it. My experience with the "reading" section of the GRE didn't actually seem to do anything but ask vocabulary questions and finding antonyms and synonyms. While I feel this can be useful, this turns the GRE into nothing more than an "important" bar trivia contest.

 

[warpus]

performance is a desirable outcome (as opposed to poor performance, or having a lot of people regardless of the quality of their education).

Yeah, but this doesn't work, does it?

They're looking at standardized test performance and not any sort of increase in the quality of education. It seems that the quality is decreasing, even.