The gap between how technology companies market their products and how those products are actually used is often vast, but two recent developments have made this disconnect impossible to ignore. A new independent research project called the AI Observatory has begun to fill a critical transparency gap, revealing that the sensitive, personal, and ethically ambiguous ways people interact with AI are far more prevalent than the industry’s own usage reports suggest. At the same time, Flock Safety, the police-tech giant operating a network of roughly 120,000 automatic license plate readers across the United States, has been forced to introduce new safeguards to prevent its own system from being used for stalking and other illegitimate purposes. Together, these stories underscore a critical inflection point for the technology industry, moving beyond the utopian promises of innovation to a more complex and necessary conversation about transparency, accountability, and the real-world consequences of design decisions.
The AI Observatory Reveals a Hidden Spectrum of User Behavior
For years, the public and policymakers have relied on self-reported data from major AI companies to understand how models like ChatGPT, Gemini, and Claude are being used. These official reports tend to paint a picture of a productive, professional tool, emphasizing applications in coding, writing, and business analysis. The AI Observatory, a research project designed to provide an independent and more granular view of user behavior, is challenging that narrative. Its analysis shows that many more sensitive behaviors are occurring than are captured in reports from major AI companies, which focus more on work than on personal use.
What the AI Observatory researchers found is a landscape of AI usage that is far more intimate, experimental, and risky than the corporate data suggests. People are turning to these models for advice on sensitive health issues, navigating complex relationship problems, engaging in political debate, and participating in forms of social roleplay that blur the lines between entertainment and emotional support. These are areas where the stakes of a biased, harmful, or simply incorrect output are highest, yet they are precisely the areas that are most likely to be underreported or sanitized in official metrics.
What Is the AI Observatory?
The AI Observatory is an independent research project that analyzes aggregate interaction data to identify patterns of AI use that escape standard telemetry and corporate reporting. Its goal is to provide a more accurate, transparent, and publicly accessible account of how artificial intelligence is actually being integrated into daily life, including its most sensitive and controversial applications.
Anthropic, Gemini, and ChatGPT Serve Very Different Roles
The AI Observatory’s research also uncovers significant differences in how people use various models. These findings suggest that users are not treating AI models as interchangeable commodities, but are selecting them based on specific strengths and perceived personalities. This divergence has profound implications for safety training, content moderation, and the competitive landscape.
- Anthropic’s Claude became the go-to tool for coding and technical software development. Users seeking a reliable, precise, and logically structured assistant for debugging, code generation, and architectural planning consistently gravitated toward this model. This suggests Anthropic has successfully carved out a niche in the high-stakes, high-precision world of software engineering.
- Google’s Gemini saw a disproportionate share of social and roleplay interactions. Users were more likely to turn to Gemini for conversational scenarios, creative storytelling, and fantasy roleplay. This indicates a perceived strength in Gemini’s ability to handle free-form, imaginative, and socially complex interactions, even if those interactions sometimes veer into sensitive territory.
- OpenAI’s ChatGPT became the dominant platform for homework assistance and educational support. Students across all age groups are using ChatGPT as a primary tool for understanding concepts, completing assignments, and generating ideas. This finding has massive implications for the future of education, raising questions about academic integrity, learning outcomes, and the very definition of original work.
These distinctions are not just academic curiosities. They are a roadmap to the areas where each company must invest heavily in safety and alignment research. If a model is heavily used for roleplay, its safety fine-tuning needs to be robust against manipulation, grooming, and the generation of harmful content in a narrative context. If a model is the primary source of homework help, it must be held to an exceptionally high standard of factual accuracy and pedagogical soundness.
Flock Safety Curbs Misuse, but Critics Question the System’s Foundation
While the AI Observatory grapples with the invisible behaviors of users, Flock Safety is dealing with the tangible consequences of its own technology’s design. The police-tech giant, known for its vast network of automatic license plate readers (ALPRs), recently announced changes to its platform. These updates are meant to prevent officers from using the system for illegal or illegitimate purposes, including stalking.
The announcement is a tacit admission of a deeply ingrained problem. The ability to search for a specific license plate across a city or state over a given time period, without a warrant or specific probable cause, creates an obvious and potent vector for abuse. Flock’s network, which captures data on every passing vehicle, not just those connected to a crime, creates a comprehensive digital grid that can retroactively reconstruct the movements of any individual
How Does Flock Safety’s License Plate Reader Network Work?
Flock Safety operates a network of over 120,000 automatic license plate readers installed across the United States. These cameras capture images of every passing vehicle, recording the license plate number, location, date, and time. This data is stored on a centralized, searchable platform accessible to law enforcement agencies, allowing them to track the movements of a specific vehicle over time and across different jurisdictions. The system is designed to solve crimes, but its broad, indiscriminate data collection has raised significant civil liberties concerns.
The Flawed Defense of Indiscriminate Surveillance
In the wake of the announced changes, several arguments have emerged defending Flock Safety’s system. The core of the defense is a simple, powerful appeal to public safety: if these cameras help solve crime, is it such a big deal? On a good day, they might help catch a kidnapper. On an ordinary day, they are simply snapping pictures of cars that nobody will ever look at. This line of reasoning, however, misses the more important question entirely.
The problem is not with the goal of solving crime. The problem is with the kind of crime-fighting system Flock Safety has chosen to build. Its network works the way it does not because of technological inevitability, but because of specific, deliberate decisions about what information to collect, who can search it, how long to keep it, and how widely to share it. These decisions set the terms of the bargain between security and civil liberties. By choosing to collect data on every car, rather than only those flagged as stolen or wanted in connection with a crime, Flock has built a system of mass surveillance that is inherently prone to abuse and function creep.
Defenders of the system often ignore the fact that the potential for stalking and harassment is not a bug in the software; it is a feature of a system that stores comprehensive location data on millions of innocent people. The reforms announced by Flock are a step in the right direction, but they are a band-aid on a structural wound. A narrower Flock system, one that only stored data on hotlisted vehicles, that required a warrant for historical searches, and that was subject to independent audits, would look very different from the one currently in place. The fact that the company has to actively curb misuse is evidence that the system, as designed, is too powerful and too permissive.
Beyond Good Intentions: The Demand for Responsible Technology
What connects the AI Observatory’s findings with Flock Safety’s predicament is a fundamental convergence on the theme of accountability. For years, technology companies have operated with a wide degree of latitude, asking for forgiveness rather than permission. The AI Observatory represents a new wave of independent oversight, using data to hold the AI industry accountable for understanding its true societal impact. It is a direct challenge to the idea that companies can be trusted to self-report their own shortcomings.
Similarly, the pressure on Flock Safety is a sign that the public and the courts are no longer willing to accept the simple trade-off of freedom for security. The question is no longer just “does it work?” but “for whom does it work, and at what cost to our collective privacy and civil liberties?” The changes Flock is implementing are a direct response to the realization that without external accountability, internal incentives will always drift toward expanding surveillance and reducing oversight.
Both stories highlight the limits of self-regulation. Companies have a financial incentive to focus on the positive, productive, and profitable uses of their technology. The negative externalities, whether they are the sensitive mental health conversations happening in an unregulated AI space or the potential for a police officer to stalk a partner using a license plate database, are often ignored until they become public scandals. The role of independent researchers, watchdog journalists, and informed public debate has never been more critical.
The AI Observatory’s data is a roadmap to the areas that need the most rigorous safety work. For CEOs and product leaders, it is a clear signal that the “sensitive use” categories are not edge cases; they are core use cases that require dedicated safety teams, robust red-teaming, and transparent reporting. For companies like Flock, the challenge is to prove that surveillance can be precise and accountable rather than indiscriminate and opaque. The next generation of technology will be defined not by the problems it claims to solve, but by the systems of accountability it builds to ensure it does not create new ones in the process. The conversation is no longer about what technology can do, but what it should do, and who gets to decide the terms of that bargain.
