Human behavior is All You Need

A child learns to pick up a glass without ever being taught the physics of friction. A carpenter knows when a screw is tight enough by the resistance in their hand. A surgeon adjusts the pressure of an instrument based on changes too subtle to explain.

We rarely think about how much intelligence goes into these actions. Humans constantly observe the world, make decisions, act, and adjust to what happens next. Much of this knowledge is acquired through experience, and much of it is difficult to put into words.

Artificial intelligence has learned an extraordinary amount from what humans have written, drawn, photographed, and recorded. Models can reason through complex problems, generate software, and interpret the world around them. Yet many of the things people find effortless remain remarkably difficult for machines.

Why?

We believe part of the answer lies in the difference between observing the world and experiencing it.

A video of someone tying a knot shows the movement of their hands. It doesn't necessarily reveal how tightly they're pulling the rope, when they feel it slipping, or why they adjust their grip. The finished knot tells us even less about the decisions that produced it.

The information is there when the action happens, but much of it disappears before we ever have the chance to learn from it.

This points to a larger problem. Human behavior contains an enormous amount of knowledge about the world, but we have only begun to capture and understand it.

We believe that learning to systematically capture, represent, and model human behavior will play an important role in the next generation of artificial intelligence.

This is the work we are beginning at Humanity Labs.

Intelligence is more than what we can explain

For much of the history of machine learning, progress has depended on finding new ways to turn human knowledge and experience into training data.

The internet made an extraordinary amount of written knowledge accessible. Images and videos provided models with a richer understanding of the visual world. Human feedback and increasingly sophisticated training environments have helped models learn how to reason and complete tasks.

But there is an important distinction between what humans know and what humans can explicitly communicate.

Consider learning to ride a bicycle. Someone can explain how to balance, steer, and pedal, but those instructions are not enough to reproduce the skill. A person develops the ability through repeated interaction with the world, continuously correcting their movements based on feedback.

The same principle appears throughout human activity. We learn how objects behave by manipulating them. We develop intuition through experience. We recognize mistakes, adapt our strategies, and gradually become better at accomplishing goals.

Much of this understanding is expressed through our actions and interactions rather than language.

If we want machines to develop more of these capabilities, we need ways to study not only what humans produce, but how they interact with the world to produce it.

Why we are starting with robotics

Robotics makes this gap particularly clear.

A person can walk into an unfamiliar kitchen, pick up a fragile object, open a stubborn drawer, or manipulate something they have never encountered before. These actions involve perception, planning, motor control, physical intuition, and constant feedback.

Robots have made remarkable progress in learning from human demonstrations. Vision-language-action models can connect visual observations and instructions to physical actions, allowing a single model to perform increasingly diverse tasks.

Still, reliable physical manipulation remains difficult, particularly in unfamiliar environments and situations involving contact, uncertainty, and precise control.

One reason is that much of the information relevant to manipulation is difficult to obtain from conventional demonstrations.

A camera can record the trajectory of a hand, but not necessarily the forces exerted by individual fingers. It can show an object moving, but not directly reveal the tactile feedback that caused someone to change their grip. It can capture a successful action without explaining what made it successful.

These distinctions matter when the objective is not merely to recognize an action, but to reproduce it reliably. They also matter for models of the physical world, which must learn how actions, forces, objects, and environments interact.

We believe there is an opportunity to make human physical experience substantially more useful for training robots and models of the physical world.

Making human experience learnable

The world already produces an enormous amount of human behavioral data.

People work in factories, prepare meals, repair equipment, perform surgery, assemble products, and interact with physical environments every day. Cameras and other devices capture fragments of this activity.

But collecting observations is not the same as capturing the information necessary to understand an action.

To learn from an interaction, we may need to know what a person was trying to accomplish, what they perceived, how they moved, what physical feedback they received, and whether the action succeeded.

Some of this information can be reconstructed from existing recordings. Other parts require additional measurements. Capturing everything with specialized equipment is expensive and difficult to scale. Capturing only what is convenient risks losing the information that makes an experience valuable.

We believe the opportunity lies in bridging these approaches.

High-fidelity measurements can provide a deeper understanding of human interactions, while models trained on those measurements may help recover useful information from simpler, more abundant observations.

If successful, a relatively small amount of richly measured experience could improve the usefulness of much larger collections of human activity, without requiring every interaction to be recorded with specialized hardware.

Turning that information into valuable training data remains an open research problem. We must determine which signals matter, how to capture or reconstruct them accurately, and how to represent them in ways that improve model capabilities.

Our goal is not simply to build the largest archive of human activity. It is to develop the scalable systems that turn human experience into intelligence machines can learn from.

We envision building this infrastructure through research, scalable collection methods, and partnerships with organizations where valuable experiences already occur. Over time, better measurements could improve our models, and better models could make increasingly abundant sources of human data useful for learning.

The future of Humanity Labs

The importance of human behavior extends far beyond robotics. Human experience shapes how we learn, make decisions, interact, and adapt. A deeper understanding of these processes could have applications across artificial intelligence, brain-computer interfaces, healthcare, human-computer interaction, education, simulation, and scientific research. Each field presents different challenges, but all could benefit from understanding aspects of human behavior that existing datasets only partially capture.

We envision Humanity Labs as a research lab building the data, models, and infrastructure to make human behavior useful for AI across industries. Just as language, images, and interactive feedback have expanded what machines can learn, we believe human experience represents another vast, largely untapped source of intelligence.

The world is full of intelligence that has never been written down. We want to make more of it learnable. If you are interested in our mission, you can contact us at mridhanbalaji21@gmail.com.