Twin Sons of Different Mothers – part 1

What is so similar, coming from such different backgrounds? I will first briefly describe the parallels in our work, then highlight the importance of this resemblance.

NW unnumbered 150

DFCS: Cognitive Load in a System of Systems

This network diagram is the tool I developed from my martial arts background to frame my thinking. By tracking a training dyad’s iterative and interactive journey through this miniature world, I defined 4 Performance Dimensions that I called Diversity, Fluency, Conformity, and Speed (DFCS). I demonstrated that individuals distribute their cognitive resources among these four Dimensions quite differently, showing cognitive approaches (strategies) to a problem each was most comfortable with. The currency I saw participants exchanging in a strongly economic fashion was Cognitive Load (CL). I defined action options according to their cost to a particular individual in this currency on a scale I called CIAO (Continue, Initiate, Accommodate, Overload), which I later expanded to RAICO (Repeat, Accommodate, Initiate, Counter-Initiate, Overload). Including events of Overload was important because there were at least three distinct variations seen, and the specific processes of Recovery from Overload told a lot about the person. This much clearly gave a view of individual differences—a field of great interest to me as an educator. One of the last additions in this original burst of creative energy was the concept of “Interactive Differences”–a result of two people becoming a “System of Systems” in the emergence of their unique one-time dynamic. We do indeed present different faces depending on who we are interacting with, influenced by many circumstances.

Boyd’s VRHI

Boyd’s four Attributes are very parallel to mine, though he was not trying to quantify them as I have.

  • “…variety/rapidity/harmony/initiative (and their interaction) seem to be key qualities that permit one to shape and adapt to an ever-changing environment.”
  • “It is advantageous to possess a variety of responses that can be applied rapidly to gain sustenance, avoid danger, and diminish adversary’s capacity for independent action.”
  • “To shape and adapt one cannot be passive; instead one must take the initiative.” (all from Patterns of Conflict, slide 13, highlighting his)

The First Three: Quite Similar

Variety, Rapidity, and Harmony, we closely agree upon. For Variety, I use Diversity. For Rapidity, I use Fluency (ability to switch effortlessly and quickly from one action/tactic to another). Where Boyd used Harmony, I have variously used Cooperation or Conformity. I could also see Coherence working there.

The Fourth Quality/Quantity

I did not define Initiative as the fourth quality/measured quantity. I chose Speed—literally the clock-time passing “in the background” as each exchange of action takes place. It is the baseline unit against which we can measure the “stretching or compressing of time” that Boyd refers to 1. This helps to clarify that Fluency refers to something other than clockspeed. In the personal combat domain, how quickly, in outside-world terms, you can deliver a hit, especially the first one, has to have significance.

“Seize initiative at the outset by attacking enemy with an ever-shifting kaleidoscope of (strategic) moves and diversions in order to upset his actions and unsettle his plans thereby psychologically unbalance him and keep initiative throughout.” (Patterns of Conflict, slide 53, highlighting his)

So where is the Initiative?

Since my model world allows me to quantify my four Performance Dimensions, I can generate from them a cumulative score called Initiative. Basically, Initiative is the commodity we acquire by good investment of the currency I call Cognitive Load. Very early in the learning of the LinSao activity, people quickly realize how much effort it costs to mentally push aside the immediate threat, making room for the alternative of Transitioning the activity. When a Transition is performed, the other person is given a short-term crisis that skyrockets his/her Cognitive Load 2 until coped with. This gives the first person a short relief in which to produce another Transition, putting the second person under even greater pressure. Accumulating Initiative in this way transmutes it into the Gold Standard of any contest: Dominance. In Boyd’s words:

“He who is willing and able to take the initiative to exploit variety, rapidity, and harmony–as the basis to create as well as adapt to the more indistinct – more irregular – quicker changes of rhythm and pattern, yet shape the focus and direction of effort–survives and dominates” (Patterns of Conflict, slide 174, his highlighting removed, mine added).

My Deepest Respect

Boyd’s work greatly impresses me. I had to “go miniature” with my LinSao activity over about a decade to define my 4 Performance Dimensions, their interactions and implications. He worked out his 4 Attributes and other factors in the “way-too-big-and-complex” real world of aircraft and ground combat. I have read numerous researchers over the years who have tried to do something like that with modest activities like soccer games, producing nothing like the striking results. Kudos, Colonel!

Getting the Ideas Across

In the decades of Boyd’s own effort, and the nearly two decades since, of efforts of various writers, Boyd’s ideas are still not generally understood. Boyd took increasing numbers of hours of verbal massaging/pounding to draw an audience into a mental space where they could hear his thoughts–“create among his audience a way of thinking, a thought process” 3.. My general experience in 40 years of teaching is that getting people to do something once does more than hours of talking at them. Fourteen years prior to my hearing of Boyd or his work:

  • an anonymous War College instructor told me: “In two minutes you have shown me what we hope our cadets will learn in four years.”
  • an anonymous researcher commented on my conference presentation: “Fighter pilots would LOVE this.”

What I have come to understand in just the past month is that they were talking about Boyd’s work, and the very mixed esteem 4 Coram and Osinga describe it receiving. The latter explains why both persons refused to give me introductions, or to discuss the subject further. The whole topic was a “hot potato”.

Boyd did historic work in describing the re-engineering 5 of combat aviation, then expanding the concepts requiring that re-engineering, into a general theory of combat. But, as Coram and other authors point out, it is too easily over-simplified and misunderstood. To date, it has been expressed only in verbal/written form. An intellectual comprehension is not preparation to apply it–for many, no reason to trust it. Application requires a skill set, which can only be developed by doing something. “Think-aloud” or pencil and paper exercises are not the same as doing, and if doing requires the risk of lives and millions of dollars of equipment, there will not be much doing to develop the understanding that leads to trusting an idea.

I Have Not Duplicated Boyd’s Work

By coming from a different direction, I offer a “lab exercise” to match with reading/lecture/discussion, that allows two people to experience the complexity that Boyd described, instead of having to imagine it. By gradually increasing the pressure of time, complexity and consequence 6 on each other, in a matter of seconds two people get a literal feel of what Boyd was talking about. They get to:

  • “Exploit operational and technical features to: Generate a rapidly changing environment”
  • “Inhibit an adversaries [sic] capacity to adapt to such an environment” (Both: Fast Transients, slide 22)

They also get to value the effect more highly by being on the receiving end of it some of the time.

Initiative is not an easy thing to inspire in trainees. Boyd described it as an “Internal drive to think and take action without being urged. (Patterns of Conflict, slide 145). A lasting motivation might be developed by repeatedly experiencing the reward of doing so, and the disappointment of being overrun (in an exercise) by not doing it. An embodied understanding of the tangible value of taking Initiative through frequently and Fluently expressing Diversity in a Coherent manner is both undeniable and memorable.

Boyd expressed a concrete value for such partnered learning, to Leadership and Communication:

“A similar implicit orientation for commanders and subordinates alike will allow them to: Diminish the friction…” (Organic Design for Command and Control, slide 23, highlighting his).

Distilled Essence

Osinga wrote:

“He developed the ability to see air combat as a contest of moves and countermoves in time, a contest in which a repertoire of moves and the agility to transition from one to another quickly and accurately in regard to the opponent’s options was essential. He managed to develop the intellectual and analytical toolkit to translate his insights from practice into better weapon systems.” 7

The LinSao Activity appears to distill the essence of what Boyd wanted us to understand into a do-anywhere-anytime activity at the speed of a knife fight.

copyright Herbert N. Maier, Ph.D. 2015

Thanks again to Chuck Spinney for making John Boyd’s briefings available here.

Your responses are welcome.
1 In Boyd terms: stretching the other’s time and compressing one’s own (examples: “Patterns of Conflict”, screens # 8, 79, 87, 153,178,185)

2 In Boyd’s terms: increases friction, (Patterns of Conflict, slide 43), “drives him bananas” (Coram, “Boyd”, p. 332)

3 Osinga, Frans (2007). Science, Strategy and War. New York: Routledge. page 7.

4 Boyd did not make it any easier. “He deliberately embarrassed the leadership of the US military…” Osinga, ibid. page 50.

5 From engineering of hardware to “engineering of cognition” (my phrase) versus “cognitive engineering” (engineering of systems to support cognition).

6 Fundamentals of “Naturalistic Decision Making

7 Osinga, ibid. page 28. Highlights mine.

It Can Only Enrich Both

It can only enrich both efforts to find that two separate lines of inquiry have reached the same conclusions. This remains my impression after reading Coram’s book (2002) and starting on Richards’ “Certain to Win” (2004).CertToWinCover I will be spending some time carefully interpreting two independent sets of terminology, influenced by two different backgrounds, including pathways through differing sets of domains. But I feel safe in speculating that we are looking at the same basic ideas like “two sides of the same coin”.

Boyd and I both talk in terms of depriving the opponent of the time he needs:

  • Boyd: “operate at a faster tempo, …inside his adversary’s time scale.” (Coram, 2002, p. 327)
  • Maier: giving them too much puzzle to solve while also limiting their time.

Note that these expressions were published the same year, with no awareness of each other:

  • “they aim to attack the ability of the other side to make effective decisions under conditions of danger, fear, and uncertainty and to increase our ability to function well under these conditions.” (Richards, 2004, p. 10)
  • Results also show a clear dynamic balancing the total load managed by each of 10 dyads, defining opponents as a system, pushing each other toward cognitive overload and failure.” (Maier, 2004, p. 562)

The basic commodity in each model is explicitly:

  • Boyd: Time
  • Maier: Cognitive Load

but they both amount to limits of human processing leading up to a decision. They both involve domains explored famously by Endsley (SA) and Klein (NDM), as well as Roscoe, Damos, Schvaneveldt, Battig and many others.

And we have not been alone. Richards (2004) points out that:

  • “Although Peters [“Thriving on Chaos”] did not mention Boyd by name, their conclusions were too similar to be simple coincidence” (p. 10),
  • “a doctrine of war and a car manufacturing system [Toyota] turning out to be brothers under the skin…” (p. 10),
  • “Honda, however,chose to attack [Yamaha] through speed and agility” (p. 26).

To me, it is no coincidence that all these events would appear synchronously as computation power began to explode, making possible explicit exploration of systems and complexity. The birth and growth of organizations like Society for Chaos theory in Psychology and Life Sciences, and System Dynamics Society, where a century earlier, pencil-and-paper mathematics recoiled in horror at the very thought that non-linearity was the rule in this world, rather than the exception.

Active intelligent agents struggling to undo each other. Boyd started from a pair of jet fighters, I started from a pair of unarmed fighters—not as different as might seem. I got delayed by being an unconnected civilian, but now I think I see a way to join the discussion. I expect my contribution to be a formalized procedure for both evaluating and training of individuals and dyadic teams in:

  1. increasing this ability to make decisions under pressure,
  2. driving an opponent promptly to one of of three defined states of failure,
  3. recovering from a failure state in time to avoid, or at least postpone, total devastation.

Copyright Herbert N. Maier, 2015

Who is John Boyd?

Image

…and why is he talking about my work???

My son is Air Force, so I had heard of the OODA loop several times and thought it boring—too simple! I ran into this book 51dU1i1+AiL._AA160_at a used book store. A quick glance through the chapter titled “OODA Loop” convinced me that it sounded too simple because others had made it simple by linearizing it.

Over the next couple days, I have read that chapter four times, both marking it up & taking notes. That gives it 5 stars on my scale. Then a quick Google search brought up 5 blogs and pages (example) that should give me starting points. I’ll be spending some time with this, and writing up points I think I understand.

What genuinely struck me on the first reading was that, decades before me, he was describing the same four qualities as I had described in my dissertation (2004) and later conference papers. The difference is that I defined them as measurable Performance Dimensions. That will be my first topic, because I’m feeling a bit like Mandelbrot when he asked “How is it you have my illustrations up before my lecture?”

As a late-life grad student, I was not interested in starting a faculty line, or really even a corporate career, so I continued designing and teaching. I had always felt like my work was obvious enough that others had done much of it, though I could not find evidence of that. I see here that I did not find Boyd’s work in academic forums for the simple reason, as Coram says, he never really wrote anything. Thank you, Robert Coram, and Chuck Spinney, for offering me a thread!

Copyright Herbert N. Maier, 2015

 

Situation Awareness

Real human function is most notable for its high adaptability, flexibility and complexity. At our best, we naturally sift through what would be TeraBytes of data in moments, acting on the very few bits that really matter. Solving most real problems involves pattern recognition (often involving expert field knowledge), saying “Something odd here!” early enough to do something about it.

Situation Awareness (SA) is a current label for being alert to surroundings and having some responses in mind. SA became a funded research field first in combat and civilian aviation because a single pilot or air traffic controller (ATC) may have split-second responsibility for millions of dollars and hundreds of lives. But it extends to many other occupations and many situations of daily life—wherever consequences of a mis-judgment are unacceptable.

Stanley Roscoe’s psychology work during WWII led to his definition of SA1 as an ability to:

  • Attend to multiple information sources,
  • Evaluate alternatives
  • Establish priorities,
  • Estimate probable outcomes for different courses of action,
  • Work on the highest current urgency without losing sight of the bigger picture,
  • Reorder priorities as situations change
  • Act decisively in the face of indecision in others.

This definition can be applied equally well to any field of activity. It could even be said to define an effective human being.

Mica Endsley’s work in information systems design led her to a definition2 that specifically leaves out decision and action as being separate from awareness. Her definition specifies 3 levels of processing:

  • Level 1: Perception of the Elements in the Environment,
  • Level 2: Comprehension of the situation based on a synthesis of disjointed Level-1 elements in light of operator goals,
  • Level 3: Projection of future actions of those Elements, at least in the very near term.

Contrasting but valid definitions like these are good examples of Adult Learning, in being the primary stage of defining a problem out of a complex problem space. Both definitions involve command of a complex system of rules, selective management of attention and working memory, and making quick accurate predictions. Tracking multiple information streams and multiple response alternatives under changing priorities places high demands on executive function, as well as expert field knowledge to support pattern recognition.

Many of the domains where expertise has been traditionally studied, such as chess, music or physics, tend to emphasize focused rather than distributed attention. Our approach at TactiCog (sm) is to study, and prepare people for, situations requiring distributed attention in high-speed, high-consequence, highly-interactive situations.

Three further points are important in supporting the validity of our approach:

  • “experts are more likely to consider other players’ perspectives in their decision making,”3
  • “an expert commander has a mental model of the tactical situation that differs in measureable ways from that of a novice,” 4

And of particular interest:

  • “it is not immediately obvious why a psychomotor task should be related to SA…”5

Seems pretty obvious to us at TactiCog (sm).

1 Roscoe, S. N. (1997). Predicting Human Performance.Quebec: Helio Press, Inc.

2 Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64.

3 Lipshitz, R., & Shaul, O. B. (1997). Schemata and mental models in recognition-primed decision making. In C. E. Zsambok & G. Klein (Eds.), Naturalistic Decision Making (pp. 293-303).Mahwah, NJ:Lawrence Erlbaum Associates.

4 Serfaty, D., MacMillan, J., Entin, E. E., & Entin, E. B. (1997). The decision making expertise of battle commanders. In C. E. Zsambok & G. Klein (Eds.), Naturalistic Decision Making (pp. 23-246). Mahwah, NJ:Lawrence Erlbaum Associates.

5 Endsley, M. R., & Bolstad, C. A. (1994). Individual differences in pilot situation awareness. International Journal of Aviation Psychology, 4(3), 241-264.

Naturalistic Decision Making

The unorthodox question is: “What do expert decision makers do naturally?”

Naturalistic approaches to psychological phenomena1,2 work to model decision making and situation awareness in real-world, as contrasted with theoretically-conceived, activities. Traditional research has long found human decision making inferior to formalized rational decision models such as Bayesian analysis or multi-attribute utility theory. Then we are urged to be rational creatures, which we very basically are not.

NDM’s (Naturalistic Decision Making) position is that real experts, working under time pressure and unreliable information, with changing goals and situations, are simply not represented by traditional laboratory models. “In the Wild”, they tend to make no comparisons of alternatives, quickly generating a workable first option based on similarity to prior situations (the experience factor), fixing or replacing it only if quick mental simulation shows significant flaws.

Experts recognize prototypical situations using cues3 and patterns that simplified laboratory experiments remove from simplified laboratory testing. Drillings and Serfaty3 refer to Daniel’s 1979 work demonstrating individual expertise as being at least as important as quantity of information, in that “the best players made as good decisions with 20% of the ground truth as did the worst players with 80%…” (p. 78). Endsley, who specializes in Situation Awareness had similar findings.

Klein’s Recognition-Primed Decision (RPD) model began with fire-ground commanders at fires4 , and developed further with military field commanders. This model has been tested in activities ranging from anesthesiology, nuclear power plant operation, software design, offshore drilling and jury deliberations to highway design. This diversification is important in showing that the process is universally human rather than domain-specific.

The NDM approach has generated more than 7 world conferences. The first one5 listed 8 factors of interest:

  1. Ill-structured problems
  2. Uncertain dynamic environments
  3. Shifting, ill-defined, or competing goals
  4. Action/feedback loops
  5. Time stress
  6. High stakes
  7. Multiple players
  8. Organizational goals and norms.

This list shows that uncertainty is the most general factor, since most of the factors contribute to it. It is this uncertainty, and its impact on individual interpretations of the problem they face that link NDM to Adult Learning, and also motivates simplification of a problem in most laboratory research.

A major point contributed by NDM is the argument that psychological research cannot approach real-world problems without incorporating options, uncertainty, real stakes, and change. It also strongly argues that starting from theory without first observing what naturally occurs, and trying to describe it, is fruitless.

We agree.

The training model developed at TactiCog(sm) does not start from abstract theory—it is drawn from combat-proven real-world training procedures with a combat-proven history. Our model exhibits points 3-8 of this list, and generates points 1 & 2 at the level of the participants’ experience of it.

1 Klein, G. A., Orasanu, J., Calderwood, R., & Zsambook, C. E. (Eds.). (1993). Decision Making in Action: Models and Methods. Norwood, NJ: Ablex Publishing Corporation.

2 Endsley, M. R., & Bolstad, C. A. (1994). Individual differences in pilot situation awareness. International Journal of Aviation Psychology, 4(3), 241-264.

3 Drillings, M., & Serfaty, D. (1997). Naturalistic decision making in command and control. In C. E. Zsambok & G. Klein (Eds.), Naturalistic Decision Making (pp. 71-80). Mahwah, NJ:LawrenceErlbaum Associates.

4 Klein, G. (1997). Making Decisions in Natural Environments (Special Report 31): Research and Advanced Concepts Office, U.S. Army Research Institute for the Behavioral and Social Sciences.

5 Orasanu, J., & Connolly, T. (1993). The reinvention of decision making. In G. A. Klein & J. Orasanu & R. Calderwood & C. E. Zsambook (Eds.), Decision Making in Action: Models and Methods (pp. 3-20).Norwood, NJ: Ablex Publishing Corporation.

Adult Learning

Mature adult cognition is contextualized, with one foot in the abstract/conceptual and the other firmly in the concrete limitations of daily life. It is not handicapped by the signature adolescent demand that “it should be like…”. Living in, rather than denying, the strictures and complexities of the real world challenges us with frequent selection of a problem (often ill-defined, even contradictory) out of a multi-dimensional problem space. This is unfortunately a primary task that formal testing and traditional education do not prepare us for.

Selection or definition, followed by periodic re-definition, of a problem relates to Endsley’s Level-2 SA in its synthesizing of pattern from perceived elements and Level-3 SA in its projection of future actions. It is similarly reminiscent of several of Roscoe’s points (at the same link). In real life, there are frequently multiple ‘correct’ or variously satisfactory answers. This can be because they all lead to a similar-enough state, or because the simple fact of change is enough to unstick things.

As measures of adulthood, Tennant and Pogson1 wrote of the development of tolerance for contradiction and ambiguity. Similarly, in the domain of military decision making (where adulthood is essential), Schmitt and Klein2 wrote of tolerance of uncertainty. Both adult education and military training respect the importance of feedback, and/or incomplete/unreliable information. Tennant and Pogson wrote:

“Ambiguity, poor feedback, unclear problem boundaries, the vagaries of the relationships we have with others, and many other factors all combine to constitute the very loose framework of our adult experience of intelligent action in the everyday world.”ibid., p. 33

“Army Lt. Col. John Mowchan believes the United Statesis better served by not setting thresholds for response [to a cyber attack]. Maintaining ambiguity [italics added], he says, gives U.S. officials greater flexibility and keeps potential enemies guessing. He also cautions that, because the lines between various threat sources have increasingly blurred, deciding on the right response has become more problematic.” 3  Hedging is a primary tactic between individuals as between other entities.

By reading across the three areas, it can be seen that many major points of adult learning are shared with Situation Awareness and Naturalistic Decision Making.

[As an aside: you will find very different uses of the term “Adult Learning”. A very different use is at: http://en.wikipedia.org/wiki/Adult_education .]

1 Tennant, M., & Pogson, P. (1995). Learning and Change in the Adult Years: A Developmental Perspective. San Franscisco: Jossey-Bass.

2 Schmitt, J. F. M., USMCR, & Klein, G. A. (1996). Fighting in the fog: dealing with battlefield uncertainty. Marine Corps Gazette, 62-69.

3 Merzlak, P. (October, 2001). Editor’s Page. U. S. Naval Institute Proceedings, 4.

Tactical Timing

Part of reducing the cost, effort or load of an action is developing a sense of timing. Aside from literal experience in a specific field, timing practice is all around us, from sliding through a door before it closes, to catching the rhythm of a string of stop lights. A very annoying example is the person who takes pleasure in cutting off every sentence you try to say. Timing ties in with SA (situation awareness) in using the low-demand part of a situational cycle to update information. People will develop habits to avoid having to pay attention, such as heading for the shoulder on a road no matter why traffic has slowed down. Investing in a habit of scanning at a greater radius (say, of 2-3 cars ahead and behind) pays off in giving time to make an intelligent decision instead of trusting a default (often over-) reaction.

Models and model systems are all the rage right now, especially if they can have a cool acronym and be marketed. But eternal, useful ones are all around us in “public domain”. Driving in traffic (hang up and LEARN something) has plentiful variety of situation and pacing.

Learning to notice what level of cycle is critical to our interaction is valuable. In music terms, we might be most interested in the pacing per beat, per measure or per coda. Every bum-bum-bum might be too fast to even track, while biddy-bum, biddy-bum might be workable. Or we might note that the Other cycles back to a favorite tactic every third, fifth or eighth time around, or presents a signature lead-in that tells us when it’s coming. It’s all about the Δt.

What is Tactical Cognition? – A Brief

“Thinking that interacts with another person, whether in competition or cooperation, is tactical.”

Tactic is interactive. It expects feedback loops and a sense of “mirror looking at mirror”, realizing that the puzzle is not a passive one, but another intelligence with a will that might be antagonistic. Tactic seeks advantage and tries to anticipate.

Functionally, cognition is thinking at a data-processing level. It is not interested in emotion or motivation. Cognitive Science developed in tandem with its sister, Computer Science. In trying to create a mechanical brain, people developed a conceptual model of what they felt they could mimic—linear processing of data. Since it satisficed (a good-enough link), or was close enough to be useful, it also became an approach to study the brain/mind.

See for more: TactiCog (sm)

Adult Learning vs. Child Learning

A major category of posts here will be on the topic of Adult Learning. This topic will underlie most others. There are different thoughts on this distinction. I want to start off being clear on the definitions that will be used here, and why this is important. These are unfortunate labels as they are misleading—the difference has nothing to do with age, though it has some connection to developmental level. People vary widely in what physical age developmental stages occur. We must keep our focus on the essential attribute even though the common label sends us astray.

“Child” learning tends to consist of defined problems where a particular “right” answer is desired. The great majority of our education nowadays is designed this way because it is easier to run in a factory-style setting, even if the students are physically adults. Easier is usually the keyword here. If learning, interaction and communication are kept in this modality, vital aspects of cognitive function will fail to develop for lack of exercise. They are probably not damaged, just lying latent, waiting on a call to action.

“Adult” learning also really has nothing to do with age. Adult learning starts with a problem-space in which the first action is to define a problem. Much of the play of children is actually of this type as they create games and scenarios of their own, turning a cardboard box into whatever they need at the moment. In the real world, two people producing two approaches are often actually solving two different problems. In a team situation, this can enrich the discussion and strengthen the efforts. In selecting between competing proposals such as construction projects, investment strategies or political elections, it can add enough to the demands of evaluation to just throw up the hands and pick something. In presenting competing proposals, it can lead to layers of glitter, pyrotechnics, back-room machinations and obfuscation as well as clear compare/contrast evaluations. Whether we lean toward clarity or not, these all enter the realm of tactic.

As listeners, we must practice skills of managing uncertainty and high consequence on-the-fly, or in current jargon “in real-time”.