Noise-to-Data (N-to-D)
For some random and possibly chaotic inputs or outputs, states, activities, (or indeed anything) to be considered “Data” would require some structuring. This is what Ackoff identified as “symbology”—it is some collection of, well, knowledge about an applicable symbol set: what it is (existence), what it is supposed to represent (analogy), what it looks like (ontology, appearance, structure), what coding scheme is used to represent it (language, number system, etc.), where it is stored...
To an encrypted string of digital signals, an (effective) symbology might be some keying mechanism that renders the otherwise incoherent string coherent [1]. To “random” movement of particles in a medium, such as molecules in a gas, the symbology might be some stochastic model against which the otherwise random movements are measured and, indeed, the measurements themselves.
There are two forms of knowledge necessary for the N-to-D transformation: the symbology and the process for applying it (see The Intrinsic Recursion of Knowledge). We could also argue that we would also need to know why we are bothering to apply this symbology to bake the noise into data, though this is/should be/might be somewhat dealt with by the “higher” levels of the (N)DIKW pyramid. The fact that we might need access to higher levels of (N)DIKW knowledge in order to process lower levels of (N)DIKW knowledge is another consideration, of course; it means the levels cannot stand on their own. It is also means they are self-referential.
Data-to-Information (D-to-I)
It is easy to see that this transformation requires some structure be applied to the Data and also requires a process to apply it. Much of the effort expended in the world of computing is devoted to designing, building, testing, and deploying such structures and processes. In general, data does not spontaneously structure itself into information [2]—it must be shoehorned by some process into the structure. But to do that requires a priori knowledge of the both the structure and the shoehorning process; these are both different (though related) knowledge elements than the knowledge in the input Data and output Information.
Information-to-Knowledge (I-to-K)
As we move up the pyramid, the transformation becomes a little more amorphous and, to my mind, somewhat anthropomorphic. Ackoff considered “Knowledge” (as he envisaged it) as helping to answer the “why?” questions. Indeed that “why?” question permeates all of this. Why would we even want to try to identify data out of noise or information out of data? Of course, it is to try to answer the question that we presume the Information to Knowledge helps identify. From a purely human perspective, if we don’t go through a data collection and information structuring activity, not only do we not have the answer, we likely don’t have the question. Or not a “good” question anyway [3].
This definition of “knowledge” has its problems for me. It seems too restrictive, too compartmentalized. As I think I’ve already shown, even getting from noise to data requires that we already know what we have to do to process it. The moment we say we have to “know” we are saying we need to acquire knowledge and demonstrates the recursive nature of knowledge—it always folds back in on itself. Whenever we try to pin it down, we always find we need some other knowledge to identify or define it.
At this level Ackoff and others consider the question-to-be-answered to be somewhere else. Indeed, in a practical sense, we synthesize data out of noise and process it into information so that we might know (acquire the knowledge of) how to solve something that is usually outside of the (N)DIK… system to date. But that question-to-be-answered is itself knowledge which we must acquire or already have acquired.
If your head is hurting at the moment, I sympathize and I know (sic) the feeling.
FOOTNOTES
[1] Ah! But compared to what? A text message? A JPG photograph? What? We would need to know that!
[2] That said, techniques of affinity analysis can clump data by similarities to point to informational structures. But, of course, the analysis method (aka. process knowledge) does need knowledge of whatever characteristics are to be used to determine if there are similarities, right? And we’re back at knowledge again.
However, there are physical mechanisms that “appear” to spontaneously generate structure out of otherwise chaotic systems. I will talk about these later.
[3] I will try to offer an answer the meta question “what is a ‘good’ question?” later.