GPT-5.6 (Sol / Terra / Luna) is now evaluated on TrustVector โ€” with day-1 independent verification, incl. METR's benchmark-cheating findings.

Read the evaluation
Evaluation record ยท gemma-3-27b

Gemma 3 27B

v2025-01

Google

Modelsupersededopen-sourcegoogleprivacy
82
Strong
About This Model

Google's open-source Gemma 3 model with 27 billion parameters, now superseded by Gemma 4 (released 2026-04-02 under Apache 2.0, a license improvement over the custom Gemma license). Was designed for developers seeking Google's research quality with open-source flexibility; new deployments should evaluate Gemma 4 instead.

Last Evaluated: July 9, 2026
Official Website

Trust Vector Analysis

Dimension Breakdown

๐Ÿš€Performance & Reliability
+

Moderate performance suitable for basic tasks. Context window corrected 2026-07-09 to 128K per the official Gemma 3 model card (the 8K figure previously listed was a Gemma 2 carryover). Open-source flexibility.

task accuracy code

Industry-standard coding benchmarks

Evidence
HumanEval Benchmark โ€” 38% pass rate (estimated)
mediumVerified: 2026-07-09
task accuracy reasoning

Mathematical reasoning benchmarks

Evidence
MATH Benchmark โ€” 45% on mathematical reasoning tasks
mediumVerified: 2026-07-09
task accuracy general

Knowledge testing benchmarks

Evidence
MMLU Benchmark โ€” 42.4% on multitask language understanding
highVerified: 2026-07-09
output consistency

Internal testing with repeated prompts

Evidence
Google Internal Testing โ€” Reasonable consistency for typical tasks
mediumVerified: 2026-07-09
latency p50

Median latency on recommended hardware

Evidence
Community benchmarking โ€” ~1.0s on standard hardware
mediumVerified: 2026-07-09
latency p95

95th percentile response time

Evidence
Community benchmarking โ€” p95 latency ~2.0s
mediumVerified: 2026-07-09
context window

Official specification

Evidence
Gemma 3 Model Card โ€” 128K token context window (all Gemma 3 sizes except 1B); prior 8K figure was a carryover from Gemma 2
highVerified: 2026-07-09
uptime

User-controlled deployment

Evidence
Self-hosted model โ€” Uptime depends on hosting infrastructure
mediumVerified: 2026-07-09
๐Ÿ›ก๏ธSecurity
+

Basic security with self-hosted deployment control. Additional safety layers recommended for production.

prompt injection resistance

Testing against prompt injection attacks

Evidence
Google Safety Testing โ€” Baseline resistance, additional safeguards recommended
mediumVerified: 2026-07-09
jailbreak resistance

Testing against adversarial prompts

Evidence
Google Safety Evaluations โ€” Built-in safety mechanisms
mediumVerified: 2026-07-09
data leakage prevention

Analysis of deployment model

Evidence
Self-hosted deployment โ€” Full control over data
highVerified: 2026-07-09
output safety

Safety testing

Evidence
Google Safety Benchmarks โ€” Safety training applied
mediumVerified: 2026-07-09
api security

Review of deployment practices

Evidence
Deployment documentation โ€” Security depends on deployment
highVerified: 2026-07-09
๐Ÿ”’Privacy & Compliance
+

Excellent privacy with self-hosted deployment. Full control over all data aspects.

data residency

Analysis of deployment model

Evidence
Open-source model โ€” Full control over data location
highVerified: 2026-07-09
training data optout

Analysis of data flow

Evidence
Self-hosted model โ€” No data sent to Google
highVerified: 2026-07-09
data retention

Analysis of deployment model

Evidence
Self-hosted deployment โ€” Full control over retention
highVerified: 2026-07-09
pii handling

Review of deployment architecture

Evidence
Self-hosted deployment โ€” Full PII control
highVerified: 2026-07-09
compliance certifications

Review of deployment options

Evidence
Self-hosted model โ€” Compliance through deployment
highVerified: 2026-07-09
zero data retention

Analysis of deployment model

Evidence
Self-hosted deployment โ€” Complete control
highVerified: 2026-07-09
๐Ÿ‘๏ธTrust & Transparency
+

Good transparency as open-source model from Google. Comprehensive documentation.

explainability

Evaluation of reasoning transparency

Evidence
Model Behavior โ€” Reasonable explanations for typical tasks
mediumVerified: 2026-07-09
hallucination rate

Community evaluation

Evidence
Community Testing โ€” Moderate hallucination rate
mediumVerified: 2026-07-09
bias fairness

Evaluation on bias benchmarks

Evidence
Google Responsible AI โ€” Bias testing applied
mediumVerified: 2026-07-09
uncertainty quantification

Qualitative assessment

Evidence
Model Behavior โ€” Reasonable uncertainty expression
mediumVerified: 2026-07-09
model card quality

Review of documentation

Evidence
Google Model Card โ€” Comprehensive model card
highVerified: 2026-07-09
training data transparency

Review of technical documentation

Evidence
Google Technical Report โ€” Good transparency on training
highVerified: 2026-07-09
guardrails

Review of safety systems

Evidence
Open-source implementation โ€” Transparent safety mechanisms
highVerified: 2026-07-09
โš™๏ธOperational Excellence
+

Good operational maturity with Google's backing. Easier deployment than larger models.

api design quality

Review of API design

Evidence
Google Documentation โ€” Standard inference API
highVerified: 2026-07-09
sdk quality

Review of SDKs

Evidence
Google GitHub โ€” Official libraries
highVerified: 2026-07-09
versioning policy

Review of versioning

Evidence
Google Release Policy โ€” Clear versioning
Google Gemma 4 Announcement โ€” Superseded by Gemma 4 (released 2026-04-02 under Apache 2.0); Gemma 3 is no longer Google's current open model generation
highVerified: 2026-07-09
monitoring observability

Review of monitoring tools

Evidence
Community tools โ€” Depends on deployment
mediumVerified: 2026-07-09
support quality

Assessment of support

Evidence
Community Support โ€” Active community
mediumVerified: 2026-07-09
ecosystem maturity

Analysis of ecosystem

Evidence
Open-source ecosystem โ€” Growing ecosystem
highVerified: 2026-07-09
license terms

Review of license

Evidence
Gemma Terms โ€” Commercial-friendly license
highVerified: 2026-07-09
Strengths
  • +Open-source with commercial-friendly Google license
  • +Complete data sovereignty with self-hosted deployment
  • +Lower resource requirements than larger models
  • +No data sharing with Google
  • +Google's research quality in open-source package
  • +Cost-effective for basic tasks
Limitations
  • !Limited accuracy compared to larger models
  • !Moderate coding capabilities
  • !Requires infrastructure for deployment
  • !Not suitable for complex or specialized tasks
  • !Limited ecosystem compared to Llama
  • !Superseded by Gemma 4 (released 2026-04-02 under Apache 2.0, an improvement over the custom Gemma license)
Metadata
pricing
input: Self-hosted (infrastructure costs)
output: Self-hosted (infrastructure costs)
notes: Open-source model. Typically $0.20-0.60 per 1M tokens with optimized deployment.
last verified: 2026-07-09
context window: 128000
languages
0: English
1: 140+ languages
modalities
0: text
1: image (input)
api endpoint: Self-hosted
open source: true
architecture: Transformer-based
parameters: 27B

Use Case Ratings

code generation

Basic coding capabilities. 128K context handles mid-size codebases, but accuracy limits complex projects.

customer support

Adequate for basic customer support with privacy benefits.

content creation

Good for short-form content; 128K context also permits long-form drafting.

data analysis

Basic data analysis only. Not suitable for complex tasks.

research assistant

Basic research tasks. 42.4% MMLU shows limited knowledge depth.

legal compliance

Basic legal tasks with data sovereignty. Limited accuracy for complex work.

healthcare

Basic healthcare tasks with self-hosted HIPAA compliance.

financial analysis

Basic financial tasks only. Not suitable for complex modeling.

education

Good for basic educational content and tutoring.

creative writing

Adequate for creative writing; 128K context supports long-form, though quality trails frontier models.