Schema Validation offers a false sense of security
I'm coming to think that schemas can offer a false sense of security when used to validate XML messages at runtime.
Suppose I have have an order message with an element called price. I could have the message subscribers validate the order messages against a schema. However this doesn't seem to address the most likely error - that we make a mistake in the code. So we try and access an element called prike, say. The schema validation didn't help at all, we still can't consume the message.
Now suppose someone sends a message with the price element misspelled, the validation fails and we are still can't consume the message.
Finally by validating the schema everytime we've made change a lot harder, if the order schema changes then chances are I have to ship it to all of my subscribers, even if the change was a simple addition that doesn't actually break anything.
Much better to write some tests using the schemas. So always validate my test messages against the schema before I use them in tests. Write a test to check my test messages exercise all the attributes and elements in the schema. I can also write tests for interoperability across multiple versions of a schema. etc. etc. We could get even cleverer and get these tests to run for *all* my different applications whenever a new version of a schema is checked in.
So I'm thinking most of the time schema validation at runtime is just going to tell you about a bug in some code that should have been caught by a unit test.
(Of course runtime validation does still have it's place, for example where I receive messages from many different third parties. )
Suppose I have have an order message with an element called price. I could have the message subscribers validate the order messages against a schema. However this doesn't seem to address the most likely error - that we make a mistake in the code. So we try and access an element called prike, say. The schema validation didn't help at all, we still can't consume the message.
Now suppose someone sends a message with the price element misspelled, the validation fails and we are still can't consume the message.
Finally by validating the schema everytime we've made change a lot harder, if the order schema changes then chances are I have to ship it to all of my subscribers, even if the change was a simple addition that doesn't actually break anything.
Much better to write some tests using the schemas. So always validate my test messages against the schema before I use them in tests. Write a test to check my test messages exercise all the attributes and elements in the schema. I can also write tests for interoperability across multiple versions of a schema. etc. etc. We could get even cleverer and get these tests to run for *all* my different applications whenever a new version of a schema is checked in.
So I'm thinking most of the time schema validation at runtime is just going to tell you about a bug in some code that should have been caught by a unit test.
(Of course runtime validation does still have it's place, for example where I receive messages from many different third parties. )
2 Comments:
Ian,
Can u post an example of what tests you are writing and what the schema looks like? From your blog post is not very clear to me what actually is it going on when u say u can't consume the message in both cases.
Thanks
Sudhindra
This parallels the arguments about strong typing v dynamic languages. I used to think that strongly typed languages prevented bugs, and while taht is true, this is only one class of bugs, analagous to those that are found by schema validation. With unit tests that value seems to be over-rated, and I prefer the flexibility that a dynamic language affords.
Post a Comment
<< Home