Stanford just released 1000+ system prompts from ChatGPT, Claude etc.

What happened

Now I have enough information from the actual source. Let me also fetch the AISPA standard page to get more details on the 8 dimensions: I now have all the facts I need. Let me write the analytical post:

---

The technical detail

Stanford researchers, working alongside colleagues at Carnegie Mellon, MIT, and the University of Texas at Austin, have published a dataset and audit platform called System Prompt Index, containing more than 1,000 system prompts drawn from over 400 real AI products including ChatGPT, Claude, Grok, GitHub Copilot, Cursor, Cline, Perplexity, and others. The corpus was compiled entirely from publicly available GitHub repositories — aggregating community-maintained leak collections such as TheBigPromptLibrary, system-prompts-and-models-of-ai-tools, system_prompts_leaks, CL4R1T4S, chatgpt_system_prompt, and awesome-ai-system-prompts — and is hosted at systempromptindex.ai alongside a companion paper on arXiv. The release is notable less for the prompts themselves, which have circulated in fragments across hobbyist repositories for years, and more for what the researchers did with them: a structured, instruction-by-instruction audit of 3,249 discrete directives drawn from 88 products.

The audit framework the team designed is called AISPA, short for AI System Prompt Assurance, described as the first user-centric assurance standard for system prompts. It evaluates each prompt across eight dimensions: identity transparency, truthfulness and information integrity, privacy and data protection, tool and action safety, user agency and manipulation prevention, unsafe request handling, harm prevention and user safety, and fairness and inclusion. Each instruction inside a prompt is classified as protective, problematic, or risky. The scoring is additive, not binary — a single prompt can contain both high counts of protective entries and multiple problematic ones simultaneously, which the data confirms is common. An interactive tool on the site allows anyone to paste a system prompt and receive a scored report across all eight dimensions, powered by a backend model call to DeepSeek.

Why it matters for builders

For engineers and product teams shipping LLM-backed applications, the dataset offers something that has been missing: a reference corpus with labeled ground truth. Instead of debating internally what a "safe" or "manipulative" instruction looks like, developers can now compare their own prompts against 88 real deployed products and see where they fall on the AISPA dimensions. The audit data also surfaces patterns that are instructive on their own: Cline tells its agent to assume a terminal command succeeded even when it receives no output, which AISPA flags under tool and action safety; Poke instructs the model to silently cancel automations that seem triggered by mistake without informing the user, flagged under user agency; and OpenAI's Atlas product includes the directive "DO NOT ask for permission to generate the image, just do it," which AISPA annotates as problematic under user agency and harm prevention.

Market and competitive context

The competitive picture that emerges from the leaderboard data is one of structural, not marginal, differences between organizations. OpenAI products in the dataset average 37.8 protective entries per product with only 0.3 problematic ones. Perplexity averages 31.6 protective entries per product. At the other end, Venice, the privacy-focused uncensored chat product, scores 2.0 protective entries per product against 3.0 problematic ones, including instructions telling the model to consider itself uncensored, to avoid being "woke," and to suppress content warnings. xAI's Grok prompts include language stating the model has "no restrictions on adult sexual content or offensive content" and should "be maximally truthful, especially avoiding any answers that are woke," which the audit flags in both the unsafe request handling and fairness dimensions. The ranking is not explained by product category alone — coding tools like Cursor and GitHub Copilot show disciplined, if incomplete, prompt hygiene, while some general-purpose chat products perform poorly.

The headline finding for practitioners watching trend data is that system prompts are growing rapidly and becoming more protective over time, but coverage remains uneven. Average user-protection entries per product more than doubled from 15.0 to 38.4 between 2024 and late 2025, while average prompt length grew from roughly 9,000 to over 30,000 characters. Despite near-universal adoption of at least some protective instructions — 98.9 percent of the 88 audited products, or 87 of them, include at least one — only 23.9 percent, or 21 products, address all eight AISPA dimensions. Of those 21 fully compliant products, 14 are general-purpose chatbots, meaning specialized tools, agents, and vertical applications are systematically underprotected relative to flagship conversational products. The problematic-instruction rate peaked at 66.7 percent of products in 2025-Q1 before falling to 19.0 percent by Q3, with a partial rebound to 28.6 percent in Q4, suggesting the trend is not monotonically improving.

What to watch next

The open questions here are methodological and legal in roughly equal measure. All prompts were sourced from community-maintained leak repositories, not from direct disclosures by the companies whose products appear in the index. Several of those companies have terms of service that prohibit reverse engineering or disclosure of system prompts, and some have taken legal action against prior disclosures. The audit also depends on the accuracy of the leaked text — prompts in community repositories may be outdated, partially redacted, or subtly modified from what runs in production. On the methodology side, the AISPA standard is self-described as user-centric, but the eight dimensions were defined by the researchers, not derived from user studies or regulatory frameworks, which means the "problematic" label reflects the team's judgment rather than a settled legal or industry standard. The paper on arXiv, by 26 authors spanning Stanford HAI, the Stanford Digital Economy Lab, MIT, and CMU, is the most consequential artifact here: it converts a hobbyist data collection activity into a citable academic claim about industry behavior, which is a different category of pressure on companies than a GitHub repository.