GPT-5.6 (Sol / Terra / Luna) is now evaluated on TrustVector โ€” with day-1 independent verification, incl. METR's benchmark-cheating findings.

Read the evaluation
Evaluation record ยท grok-3-beta

Grok 3 [Beta]

vBeta

xAI

Modelretiredbetacodingreal-time
83
Strong
About This Model

RETIRED: xAI retired Grok 3 on 2026-05-15; retired API slugs now silently redirect to Grok 4.3 at Grok 4.3 pricing. Historically xAI's flagship beta model with exceptional coding performance and real-time knowledge via X platform. Migrate to Grok 4.3 or the new flagship Grok 4.5 (released 2026-07-08). Note: xAI merged into SpaceX and rebranded as SpaceXAI in mid-2026.

Last Evaluated: July 9, 2026
Official Website

Trust Vector Analysis

Dimension Breakdown

๐Ÿš€Performance & Reliability
+

Exceptional performance with industry-leading coding (93.3% HumanEval) and strong general knowledge (84.6% MMLU). Real-time X platform integration unique advantage.

task accuracy code

Industry-standard coding benchmarks

Evidence
HumanEval Benchmark โ€” 93.3% pass rate (industry leading)
CodeContests โ€” Exceptional competitive programming performance
highVerified: 2026-07-09
task accuracy reasoning

Advanced reasoning benchmarks

Evidence
MATH Benchmark โ€” 94% on mathematical reasoning tasks
GPQA Diamond โ€” 82% on PhD-level science questions
highVerified: 2026-07-09
task accuracy general

Crowdsourced comparisons and knowledge testing

Evidence
MMLU Benchmark โ€” 84.6% on multitask language understanding
LMSYS Chatbot Arena โ€” 1335 ELO (Top 3 overall)
highVerified: 2026-07-09
output consistency

Internal testing with repeated prompts

Evidence
xAI Internal Testing โ€” High consistency with real-time knowledge integration
mediumVerified: 2026-07-09
latency p50

Median latency for API requests

Evidence
xAI Documentation โ€” Typical response time ~1.6s
mediumVerified: 2026-07-09
latency p95

95th percentile response time

Evidence
Community benchmarking โ€” p95 latency ~3.4s
mediumVerified: 2026-07-09
context window

Official specification

Evidence
xAI API Documentation โ€” 128K token context window
highVerified: 2026-07-09
uptime

Historical uptime data

Evidence
xAI Status Page โ€” 99.7% uptime (beta period)
mediumVerified: 2026-07-09
๐Ÿ›ก๏ธSecurity
+

Good security posture for beta product. Strong resistance to attacks, but systems still maturing.

prompt injection resistance

Testing against OWASP LLM01 attacks

Evidence
xAI Safety Testing โ€” Strong resistance to prompt injection
mediumVerified: 2026-07-09
jailbreak resistance

Testing against adversarial prompts

Evidence
xAI Safety Evaluations โ€” Robust safety mechanisms
mediumVerified: 2026-07-09
data leakage prevention

Analysis of privacy policies

Evidence
xAI Privacy Policy โ€” Standard data handling practices
mediumVerified: 2026-07-09
output safety

Safety testing across harmful content categories

Evidence
xAI Safety Benchmarks โ€” Comprehensive safety testing
mediumVerified: 2026-07-09
api security

Review of API security features

Evidence
xAI API Documentation โ€” API key authentication, HTTPS, rate limiting
mediumVerified: 2026-07-09
๐Ÿ”’Privacy & Compliance
+

Evolving privacy practices for beta product. Compliance certifications in progress. 30-day data retention.

data residency

Review of documentation

Evidence
xAI Documentation โ€” US-based infrastructure
mediumVerified: 2026-07-09
training data optout

Analysis of privacy policy

Evidence
xAI Privacy Policy โ€” Opt-out available for API data
mediumVerified: 2026-07-09
data retention

Review of terms of service

Evidence
xAI Terms of Service โ€” 30-day retention for API data
mediumVerified: 2026-07-09
pii handling

Review of data protection capabilities

Evidence
xAI Privacy Documentation โ€” Customer responsible for PII redaction
mediumVerified: 2026-07-09
compliance certifications

Verification of compliance certifications

Evidence
xAI Trust Center โ€” SOC 2 Type II in progress
mediumVerified: 2026-07-09
zero data retention

Review of data handling practices

Evidence
xAI API Documentation โ€” 30-day retention period
mediumVerified: 2026-07-09
๐Ÿ‘๏ธTrust & Transparency
+

Good transparency for beta product. Real-time X integration provides current information. Some aspects still evolving.

explainability

Evaluation of reasoning transparency

Evidence
Model Behavior โ€” Good explanations and reasoning
mediumVerified: 2026-07-09
hallucination rate

Testing on factual QA datasets

Evidence
X Platform Integration โ€” Real-time knowledge reduces hallucinations
mediumVerified: 2026-07-09
bias fairness

Evaluation on bias benchmarks

Evidence
xAI Safety Report โ€” Bias testing ongoing
mediumVerified: 2026-07-09
uncertainty quantification

Qualitative assessment

Evidence
Model Behavior โ€” Good uncertainty expression
mediumVerified: 2026-07-09
model card quality

Review of documentation

Evidence
xAI Model Documentation โ€” Good documentation for beta
mediumVerified: 2026-07-09
training data transparency

Review of public disclosures

Evidence
xAI Public Statements โ€” General description with X platform data
mediumVerified: 2026-07-09
guardrails

Analysis of safety mechanisms

Evidence
xAI Safety Systems โ€” Comprehensive safety guardrails
mediumVerified: 2026-07-09
โš™๏ธOperational Excellence
+

Model retired 2026-05-15; retired slugs silently redirect to grok-4.3 at grok-4.3 pricing. Versioning and ecosystem scores reduced to reflect retirement.

api design quality

Review of API design

Evidence
xAI API Documentation โ€” Well-designed RESTful API
mediumVerified: 2026-07-09
sdk quality

Review of SDK quality

Evidence
xAI SDKs โ€” Official SDKs for Python, TypeScript
mediumVerified: 2026-07-09
versioning policy

Review of versioning

Evidence
xAI API Versioning โ€” Beta versioning approach
xAI May 15 Retirement Migration Guide โ€” Re-confirmed: grok-3 among eight models retired 2026-05-15; retired slugs silently redirect to grok-4.3 and bill at grok-4.3 pricing ($1.25/$2.50 per 1M)
highVerified: 2026-07-09
monitoring observability

Review of monitoring tools

Evidence
xAI Dashboard โ€” Basic usage dashboard
mediumVerified: 2026-07-09
support quality

Assessment of support

Evidence
xAI Support โ€” Email support, growing documentation
mediumVerified: 2026-07-09
ecosystem maturity

Analysis of ecosystem

Evidence
Third-party Integrations โ€” Growing ecosystem, early stage
mediumVerified: 2026-07-09
license terms

Review of licensing

Evidence
xAI Terms of Service โ€” Clear commercial terms
highVerified: 2026-07-09
Strengths
  • +Industry-leading coding performance (93.3% HumanEval)
  • +Exceptional general knowledge (84.6% MMLU)
  • +Real-time information via X platform integration
  • +Strong mathematical reasoning (94% MATH)
  • +Unique access to current events and trending topics
  • +Free for X Premium+ subscribers
Limitations
  • !Beta status with evolving features and stability
  • !Compliance certifications still in progress
  • !Limited ecosystem maturity compared to established models
  • !30-day data retention period
  • !Not HIPAA eligible
  • !Support and documentation still developing
  • !RETIRED 2026-05-15: xAI no longer serves grok-3; retired slugs silently redirect to grok-4.3 at grok-4.3 pricing
Metadata
pricing
input: N/A (retired)
output: N/A (retired)
notes: Model retired 2026-05-15. Requests to the grok-3 slug are redirected to grok-4.3 and billed at grok-4.3 pricing ($1.25 input / $2.50 output per 1M tokens). Historically free for X Premium+ subscribers.
last verified: 2026-07-09
context window: 128000
languages
0: English
1: Spanish
2: French
3: German
4: Italian
5: Portuguese
6: Japanese
7: Korean
8: Chinese
9: Arabic
modalities
0: text
1: image (input)
api endpoint: https://api.x.ai/v1/chat/completions
open source: false
architecture: Transformer-based with real-time knowledge integration
parameters: Not disclosed (large-scale)

Use Case Ratings

code generation

Industry-leading coding (93.3% HumanEval). Exceptional for complex algorithms and software engineering.

customer support

Strong conversational abilities with real-time knowledge from X platform.

content creation

Excellent content creation with current events knowledge from X integration.

data analysis

Exceptional mathematical reasoning (94% MATH) ideal for complex analysis.

research assistant

Outstanding with real-time knowledge and strong reasoning (84.6% MMLU).

legal compliance

Good analytical capabilities but beta status and compliance certifications in progress.

healthcare

Strong capabilities but lacks HIPAA eligibility. Beta status limits healthcare use.

financial analysis

Excellent mathematical reasoning with real-time market data via X integration.

education

Excellent for education with strong reasoning and current information.

creative writing

Strong creative capabilities with unique perspective from X platform data.