The greatest risk of AI isn’t that machines become more human.
It’s that humans begin living more like AI.
I had dinner with Sato Toshiaki from Glico. Naturally, the conversation drifted toward AI.
We talked about what AI means for design, creativity, and the future of work. One thing he said stayed with me. His concern wasn’t simply about what AI can generate. It was about the mindset designers bring when they create. Technology changes, but the mindset behind design determines whether it serves people or merely optimizes them.
Last week, I was reading Don Norman’s latest essay, Moving from Human to Humanity, Design to Action. Although they approached the problem from different directions, they were asking remarkably similar questions.
Don argues that design can no longer focus only on the individual user. It must consider humanity, culture, ecosystems, and the long-term consequences of our actions.
Reading his essay, I realized my own concern was slightly different — but connected. We often ask whether AI products are useful, efficient, or profitable. I found myself asking a different question:
What kind of humans are these systems quietly encouraging us to become?
The wrong question about AI
Everyone is asking whether AI will become more human. I think a more urgent question is whether humans are becoming more like AI.
Instagram, LinkedIn, Facebook, X, YouTube, and Duolingo, etc., are increasingly impacting our lives around external scores. Likes, rankings, followers, performance reviews, health and sleep scores, and daily streaks constantly tell us how well we are doing — and whether we are improving.
These measurements can be useful. But when every activity becomes something to track, compare, and optimize, we begin to evaluate ourselves in the same way an AI system is evaluated: through externally defined goals, continuous feedback, and the pressure to achieve a higher score.
The danger isn’t just that AI might someday copy human behavior. It’s that humans could gradually give up our lives and sense of purpose and start living by the logic of machines.
AI lives by external Evaluation
Modern AI learns through external evaluation. It has no intrinsic sense of purpose or satisfaction. Instead, it improves by optimizing objectives defined by others. Engineers specify a goal, measure the quality of its output using loss functions, reward models, benchmarks, or human feedback, and the AI continually adjusts itself to achieve a higher score.
Increasingly, our lives are following the same pattern.
Social platforms such as TikTok, Instagram, YouTube, LinkedIn, Facebook, and X reward us with likes, views, followers, shares, and engagement metrics. At work, performance reviews, KPIs, and productivity dashboards evaluate our value. Financial institutions assign credit scores. Health apps measure sleep quality, heart rate, calories, and daily activity. Language-learning apps encourage us to maintain streaks. Nearly every digital service presents a number, badge, ranking, or score that tells us how well we are doing.
None of these systems is inherently harmful. Many provide useful feedback and can motivate positive habits. The danger arises when external evaluation becomes the primary way we understand our own worth. We begin to optimize our behavior not because it is meaningful to us, but because it improves a metric designed by someone else.
In doing so, we begin to resemble the very systems we created. Like AI, we learn to pursue objectives defined outside ourselves, slowly replacing intrinsic motivation with external optimization.
The greatest risk of the AI age is not that artificial intelligence learns to think like humans. It is that humans learn to live like artificial intelligence.
Japan has a Word for What Cannot Be Measured
Modern life encourages us to believe that everything important can be measured. We count likes, followers, salaries, credit scores, productivity, sleep quality, and daily streaks. The more data we collect, the more we assume we understand ourselves.
But here in Japan, there is a word for something that exists beyond measurement.
It is called ikigai.
Outside Japan, ikigai is often translated as “your reason for living” or reduced to a career diagram about finding the intersection of passion, talent, purpose, and income. While that framework may be useful for career planning, it misses something essential.
At its heart, ikigai is much simpler ~ and much deeper.
It is the quiet joy of doing something because it brings meaning to your own life, regardless of whether anyone notices, rewards, or even understands it.
It can be watching cherry blossoms bloom for a few brief days each spring. It can be recognizing the species of a butterfly during a morning walk. It can be carefully preparing a cup of coffee, tending a small garden, drawing for hours, practicing calligraphy, cooking for your family, or becoming completely absorbed in learning something simply because your curiosity leads you there.
None of these moments requires applause. None need followers, rankings, salaries, or performance reviews. They are valuable because the experience itself is valuable.
This is what makes ikigai fundamentally different from the way artificial intelligence learns.
AI requires external objectives to improve. Ikigai requires none. It exists independently of recognition, achievement, or measurable success. It is a reminder that some of the most meaningful parts of being human cannot be optimized because they were never meant to be measured in the first place.
How the West Turned Ikigai into a Career Diagram
If you search for ikigai online, you will almost certainly encounter a colorful Venn diagram with four overlapping circles:
What you love
What you are good at
What the world needs
What you can be paid for
It is an elegant framework for thinking about careers, purpose, and professional fulfillment. For many people, it has been a helpful tool for reflecting on work and life decisions.
The problem is not the framework itself.
The problem is assuming that this is what ikigai has always meant in Japan.
Traditional Japanese understandings of ikigai are far broader and often far more ordinary. They do not require that your joy become your profession, contribute to society in a measurable way, or generate income. A retiree tending a vegetable garden, someone quietly sketching every evening, a child collecting insects, or an elderly person enjoying a morning walk can all experience ikigai without asking whether those activities are productive or economically valuable.
This does not mean that work, contribution, or recognition are unimportant. Many people find ikigai through their profession, their family, or serving others. But these outcomes are not prerequisites.
Payment, usefulness, recognition, and social contribution may emerge from ikigai, but they do not define it.
In its traditional sense, ikigai begins in a much quieter place. It begins with the simple experience of finding meaning in the act itself rather than in its evaluation. Whether anyone notices, applauds, or rewards it is secondary. The joy exists before the recognition ever arrives.
The Deeper Japanese Meaning: Small Joys Without Permission
One of the reasons ikigai can be difficult to explain is that it often reveals itself through life’s smallest moments rather than its biggest achievements.
It doesn’t ask for permission. It doesn’t wait for recognition. It doesn’t need to become productive or profitable before it becomes meaningful.
A child chasing butterflies.
A quiet morning spent making coffee.
Watching cherry blossoms bloom for just a few days each spring.
Learning the names of birds simply because they fascinate you.
Drawing, gardening, fishing, walking, reading, or practicing a craft long after everyone else has stopped paying attention.
These moments may seem insignificant to the outside world, yet they can become deeply significant to the person experiencing them. This is one of the quiet philosophies woven throughout much of Japanese culture.
You can see this sensibility reflected in many of Japan’s most beloved creative works.
Hayao Miyazaki’s My Neighbor Totoro is remembered not for epic battles or grand ambitions, but for its celebration of childhood curiosity, quiet wonder, and the extraordinary hidden within ordinary life. Much of its emotional power comes from slowing down long enough to notice the world.
The Pokémon series began with creator Satoshi Tajiri’s childhood fascination with collecting insects. That simple joy of exploring nature and discovering small living creatures eventually became one of the world’s most successful entertainment franchises. What began as a deeply personal curiosity grew into something shared by millions.
Animal Crossing offers another perspective. Rather than rewarding players through constant competition or high-stakes objectives, it invites them to enjoy everyday rhythms — arranging a room, planting flowers, talking to neighbors, fishing by a river, or simply watching the seasons change. Progress exists, but it is not the point. The experience itself is.
These works resonate around the world because they remind us of something many adults gradually forget: not every meaningful experience needs to be optimized, measured, or justified.
Perhaps this is one of Japan’s quietest contributions to the age of artificial intelligence.
Some of the most valuable parts of being human begin as small joys that require no permission, no audience, and no score. They matter simply because they matter to the person experiencing them.
That, perhaps, is one of the deepest meanings of ikigai.
The Paradox: What Cannot Be Measured Can Still Create Value
Modern society often assumes that value begins with recognition.
We work to earn a promotion. We create to gain followers. We study for grades. We optimize for performance reviews, engagement metrics, or financial rewards. Success is expected to come first, with meaning arriving afterward.
Ikigai suggests the opposite.
Meaning does not begin with recognition. It begins with genuine interest.
When we care deeply about something, we return to it — not because someone tells us to, but because the activity itself is rewarding. We notice details others overlook. We practice voluntarily. We repeat without being asked. Over months and years, curiosity becomes knowledge. Knowledge becomes skill. Skill becomes craftsmanship.
Only then do the things society can measure begin to appear.
A beautifully crafted product.
A work of art.
A scientific discovery.
A successful business.
A beloved game.
A cultural movement.
Economic value, recognition, and contribution often emerge from this process, but they are its outcomes — not its starting point.
This distinction is easy to overlook because society mostly celebrates the visible results. We admire the finished film, not the child’s fascination with insects that inspired Pokémon. We celebrate a world-famous animation studio, not Hayao Miyazaki quietly observing clouds, forests, children, and everyday life. We praise innovation while forgetting the years of curiosity, play, and patient craftsmanship that made it possible.
In other words, what society measures is often the final chapter of a much longer story.
The story begins with something no algorithm can quantify: the quiet satisfaction of doing something because it matters to you.
This is the paradox of ikigai.
What cannot be measured can still become the source of immense cultural, social, and economic value.
External value can grow from inner fulfillment. But external value is not what defines that fulfillment.
The Ikigai Risk
Much of the public conversation about artificial intelligence focuses on one question: Will AI replace human work? This is an important question. But another question we question ourselves less often.
What if humans begin living like AI?
Artificial intelligence improves through external evaluation. It receives feedback, adjusts its behavior, and optimizes toward objectives defined by others. It has no intrinsic sense of purpose. Its success is determined entirely by how well it performs against external criteria.
Increasingly, our lives are beginning to follow the same pattern.
We optimize our social media posts for likes and engagement. We build our careers around performance reviews and KPIs. We monitor our sleep, exercise, productivity, and finances through dashboards and scores. Every platform promises that one more improvement, one higher ranking, or one longer streak will make us better.
None of these measurements are inherently bad.
Grades help us learn. Salaries recognize our work. Analytics help organizations improve. Health metrics can encourage healthier habits. Feedback has always been part of human growth.
The danger begins when these measurements stop serving life and start defining it.
When every decision is filtered through what can be measured, compared, and optimized, we gradually lose touch with the parts of ourselves that never needed a score in the first place.
Curiosity that exists simply to explore.
Play that has no productive outcome.
Conversations with no agenda.
Time spent watching the seasons change.
Making something because the process itself is rewarding.
These experiences rarely improve a dashboard, yet they are often the moments that make life feel meaningful.
The greatest risk of the AI age may not be that machines learn to think like humans.
It may be that humans slowly forget how to live as humans.
When our sense of worth depends entirely on external evaluation, we become trapped in an endless cycle of optimization, always pursuing the next score but never asking a more important question:
What would I still choose to do if nobody were measuring me?
That question lies at the heart of ikigai.
From Human-Centered Design to Humanity-Centered Action
If the greatest risk of the AI age is that humans begin living like machines, then we need a new way to think about design.
For decades, Human-Centered Design has asked an important question:
How do we design products that serve people?
Don Norman argues that this question is no longer enough.
In “Design for a Better World”, he proposes expanding Human-Centered Design into Humanity-Centered Action
It represents two fundamental shifts.
The first expands our responsibility.
Human-Centered Design focuses on the individual user. Humanity-Centered Design asks us to consider everyone affected by our decisions — communities, cultures, future generations, and the ecosystems that sustain life.
The second expands our accountability.
A design is not successful simply because it is elegant or usable. It must also be judged by what happens after it enters the real world. A product changes behavior. It shapes organizations. It influences society. Its consequences matter as much as its intentions.
This way of thinking becomes especially important in the age of artificial intelligence.
Suppose an AI system makes employees 30 percent more productive. On paper, it is a success. But what if it also reduces curiosity? What if it discourages learning because the AI always provides the answer? What if it weakens craftsmanship because people no longer develop the underlying skills?
What if relationships become increasingly transactional because algorithms optimize every interaction?
What if every moment of life becomes another score to improve?Traditional product metrics may celebrate these systems because they increase efficiency, engagement, or profit.
Humanity-Centered Action asks a different question.
What kind of humans are we helping to create?
That question may become one of the defining design challenges of the AI era.
Perhaps the next evolution of Humanity-Centered Action is not simply protecting humanity as a society, but protecting the conditions that allow humanity to flourish from within.
Not only clean air, healthy ecosystems, and thriving communities.
But also curiosity.
Agency.
Play.
Craftsmanship.
Contemplation.
Relationships.
Unmeasured time.
Intrinsic motivation.
And what Japan has long called ikigai. These qualities rarely appear on a dashboard. They are difficult to quantify, difficult to optimize, and often invisible to traditional business metrics.
Yet they are precisely the conditions from which creativity, innovation, culture, and meaningful lives emerge. Perhaps the challenge of the AI age is not deciding what machines should become.
It is deciding which parts of humanity should never be reduced to a metric.
Then Don’s HCA+ becomes:
- Industrial era → machines measured production.
- Information era → computers measured information.
- AI era → algorithms measure human behavior.
What Should We Optimize?
Artificial intelligence has become remarkably good at optimizing measurable outcomes. It can maximize efficiency, engagement, accuracy, productivity, and profit at a scale never before possible.
The question is no longer whether we can optimize.
The question is what deserves to be optimized — and what deserves to be protected instead?
If we want AI to strengthen humanity rather than gradually reshape it into another optimization problem, then the principles guiding its design should extend beyond technical performance.
Preserve Human Agency
AI should expand human choice, not quietly replace it. The purpose of AI should be to help people think, create, and decide more effectively — not to remove meaningful decisions simply because automation is possible.
Don’t Turn Everything into a Score
Not everything that matters can be measured. Likes, rankings, streaks, dashboards, productivity metrics, and engagement scores can all provide useful feedback. But they should remain tools, not definitions of personal worth.
Some of the most meaningful parts of life lose their meaning the moment they become performance metrics.
Protect Meaningful Effort
Efficiency is not always the same as progress. Many of the experiences that shape us — learning a language, mastering an instrument, writing a book, raising a child, or becoming skilled at a craft — require effort.
When AI removes every challenge, it can also remove the satisfaction that comes from overcoming it.
Support Creation, Not Just Consumption
AI should help people express ideas, explore possibilities, and create things they could not have created alone.
Its greatest contribution should not be generating endless streams of content for passive consumption, but empowering more people to become creators, thinkers, artists, researchers, and problem solvers.
Leave Space for Curiosity
Not every interaction should be optimized toward the fastest answer. Discovery often begins with wandering, experimenting, making mistakes, asking unexpected questions, or following curiosity without knowing where it will lead.
Innovation rarely follows a perfectly optimized path.
Evaluate Human Consequences, Not Just Product Metrics
The success of AI should not be judged solely by efficiency, engagement, revenue, or productivity.
We should also ask what it is doing to people.
Does it strengthen or weaken attention?
Does it increase or reduce autonomy?
Does it encourage learning or passive dependence?
Does it deepen relationships or replace them?
Does it cultivate craftsmanship or make expertise disposable?
Does it help people flourish — or merely perform better on measurable metrics?
These questions are harder to quantify, but they may ultimately be the most important ones. Artificial intelligence will continue becoming more capable. The real challenge is ensuring that, as our machines become more intelligent, our definition of human progress does not become smaller.
Technology should optimize what humans produce. It should never reduce what it means to be human.
When does measurement stop helping us and start defining us?
Reference: https://donnorman1.substack.com/p/moving-from-human-to-humanity-design
