High-Performance AI with 89% Human Performance Now Accessible:Google's Gemma

Hello, today we will discuss Gemma, recently released by Google. Gemma is an advanced, lightweight open model suite developed based on the core technology and research of Google’s Gemini project. Developed in collaboration between Google DeepMind and various Google departments, Gemma is inspired by the Latin word for ‘gem’, including Gemini.

In the journey to broaden the horizons of Artificial Intelligence (AI), Large Language Models (LLMs) have been at the forefront. These intelligent systems, capable of generating text and engaging in conversations at a human-like level, have showcased the astonishing potential in this field. However, the true potential of these technologies has been explored within a limited scope as they were not made publicly available. In that sense, the arrival of Google DeepMind’s latest creation, Gemma, marks a new turning point.

Gemma’s open model approach as an open innovation platform allows developers and researchers worldwide to freely conduct experiments and innovations in AI technology. This, unlike the previous closed or restricted AI models, enables participation and collaboration within the vast open-source community, thereby accelerating the development of AI applications.

Visit Google Gemma Blog

Gemma utilizes a transformer-based architecture and training methods similar to DeepMind’s primary Gemini model. It has been trained on up to 6 trillion text tokens, including web documents, mathematical texts, and codes. In short, Gemma is a high-performance AI trained with supervised learning and human feedback for function enhancement in areas such as conversation, following instructions, and coding. With human performance estimated at 89%, the capabilities of Gemma are highly anticipated. The key features that distinguish Gemma from existing AI models include:

Availability in Various Model Sizes: Gemma is offered in two sizes, 2B (2 billion parameters) and 7B (7 billion parameters), allowing users to select the optimal model based on their needs and resources. This flexibility is particularly beneficial for users in resource-constrained environments.
Advanced Architecture and Training Methodology: Gemma employs a transformer-based architecture, leveraging advanced training techniques derived from DeepMind’s powerful Gemini model. This enables the model to exhibit impressive capabilities in tasks such as text generation, common sense reasoning, and coding.
Trained on Massive Data: Gemma has been trained on up to 6 trillion text tokens, covering a wide range of text types including web documents, mathematical problems, and code snippets. This extensive training data allows the model to acquire a broad range of knowledge and respond to various types of queries.
Emphasis on Responsible AI Use: Gemma emphasizes responsible AI usage, undergoing rigorous testing for safety, fairness, and robustness. Efforts have been made to minimize the impact of harmful or biased content through data filtering and fine-tuning processes.
Cross-Platform Compatibility and Accessibility: Gemma is designed for use across various platforms and devices, and can be easily scaled through services such as Google Cloud’s Vertex AI. This allows users to easily deploy and utilize the model in their environments.

Gemma’s performance tests show impressive results:

Math: Gemma excels in mathematical reasoning tests like GSM8K and MATH, surpassing models such as Codex and Anthropic’s Claude by over 10 points.

Coding: Despite not being specifically trained for coding, Gemma matches or exceeds the performance of Codex in programming benchmarks like MBPP.

Conversation: In human preference tests, Gemma demonstrates strong conversational abilities with a 7% win rate over Anthropic’s Mistral-51.7B.

Reasoning: In tasks requiring reasoning, such as ARC and Winogrande, Gemma outperforms other 7B models by 5-10 points.

While work remains to improve AI’s reliability, interpretability, and controllability, Gemma represents a significant leap forward in open-source NLP, potentially igniting new advancements across the AI community:

Accessibility: Gemma lowers the barriers for organizations previously faced with high computing/data costs for training their own LLMs, making cutting-edge NLP more accessible.

New Applications: By open-sourcing pre-trained and fine-tuned checkpoints, DeepMind facilitates the development of useful applications in fields like education, science, and accessibility.

Customization: Developers can further customize Gemma for industry-specific or domain-specific applications through continued training on proprietary data.

Research: Open models like Gemma enhance the transparency and auditability of current NLP systems, illuminating future research directions.

Innovation: The use of a powerful foundational model like Gemma is expected to accelerate advancements in areas such as bias mitigation, factualness, and AI safety.

Despite potential risks, DeepMind’s technical and ethical efforts suggest that the benefits of Gemma outweigh the potential hazards. Gemma brings us one step closer to AI that benefits all, presenting new business models even to small and startup companies.

DeepMind hopes that by making Gemma’s capabilities available to everyone through open sourcing, it will further drive responsible AI development for social good.

I wish you all a great day.

Visit Google Gemma Blog