Should smart technologies be transparent when used?


“[D]eep learning systems are ‘already pushing their way into real-world applications. Some help drive services inside Google and other Internet giants, helping to identify faces in photos, recognize commands spoken into smartphones, and so much more’. If deep learning networks systematically classify the world’s patterns in ways that are at variance with our ordinary human classifications, and if those networks are lodged in the workings of the technology that organizes and shapes our cognitive lives, then those lives will be organized and shaped by those variant classifications.
However, surely we will notice this divergence, I hear you say. It is here that a second point becomes relevant. What if the networks in question are fluidly and expertly integrated into our everyday activities, such that they are transparent in use? Imagine such networks operating as part of a cognitive-assistant-style wearable that classifies situations and transmits the results via an optical head-mounted display… such behaviour-guiding technology, even though it enhances cognitive performance, and even though it is operating in cognition central, could be transparent. On some occasions, no doubt, its variant classifications of the world would lead to mismatches to which the human user will be sensitive before anything detrimental occurs. However, it seems just as likely that subtle changes in one’s engagement with the world—changes that, for example, have potentially damaging social consequences for how one classifies others…
The final aspect of this worrying scenario comes to light once one realizes that [such technologies] are at least on the way to being correctly treated as genuine parts of the user’s own cognitive architecture… that unconsciously guide my behaviour… be part of what I unconsciously believe to be the case, and thus presumably will have the same status as my more familiar, internally realized unconscious beliefs when it comes to any moral judgments that are made about my resulting thoughts and actions.”
– Michael Wheeler

When some technology is used by a skilled person without difficulty, the person is no longer consciously aware of the technology. It disappears from her awareness in the same way that when in the flow of writing, she doesn’t notice the pen in her hand. This kind of lack of active awareness of the technology in use is labelled transparency.

How does technology enable scientific discovery?


“We should notice the force, effect, and consequences of inventions, which are nowhere more conspicuous than in those three which were unknown to the ancients… printing, gun powder, and the compass. For these three have changed the appearance and state of the whole world… innumerable changes have been thence derived, so that no empire… appears to have exercised a greater power and influence on human affairs than these mechanical  discoveries.”
– Francis Bacon

The idea that science is a “theory producing machine” came to be challenged by philosophers of science such as Karl Popper and Thomas Kuhn. A growing, if still controversial, view sees science as situated within social, political and constructivist contexts.  Even amongst scientific realists (and I tentatively include myself in that camp), it is recognised that science itself is technologically embodied:

“Without instruments and laboratories, there was no science.” (p. 7)


How do we free our relationship with technology?


“Thus we shall never experience our relationship to the essence of technology so long as we merely conceive and push forward the technological, put up with it, or evade it. Everywhere we remain unfree and chained to technology, whether we passionately affirm or deny it. But we are delivered over to it in the worst possible way when we regard it as something neutral; for this conception of it, to which today we particularly like to do homage, makes us utterly blind to the essence of technology.”
– Martin Heidegger

A prevalent misunderstanding is that technology is a tool, a means to an end and neutral in value. Heidegger wants to free us from this misunderstanding, to really bring to attention our attitude and relationship towards the technology that we have at hand – in use. That we accept and use technology without this attentiveness, without often being aware that we are using it at all, is what makes us unfree.

Essays and Stories by S. P. Razavi