Fine-Tuning Llama 3.2 11B with Q-LoRA for Extractive Question Answering

Tuesday, November 26, 2024 12:00 AM
279

Large Language Models (LLMs) have become essential tools in natural language processing, capable of handling a variety of tasks. However, due to their broad training, they may not excel in specific applications without further adaptation. Fine-tuning techniques, such as Q-LoRA, allow researchers to tailor pre-trained models like Llama 3.2 11B for particular tasks, such as extractive question answering. This article outlines the process of fine-tuning Llama 3.2 11B using Q-LoRA on the SQuAD v2 dataset, showcasing the performance enhancements achieved through this method.

LoRA, or Low-Rank Adaptation, is a technique that introduces new weights to an existing model without altering the original parameters. By adding adapter weights that adjust the outputs of certain layers, LoRA enables models to retain their pre-trained knowledge while acquiring new capabilities tailored to specific tasks. In this experiment, the focus is on fine-tuning Llama 3.2 11B for extractive question answering, aiming to extract precise text segments that answer user queries directly, rather than summarizing or rephrasing the content. The experiment was conducted on a Google Colab platform utilizing an A100 GPU, with the Hugging Face Transformers library facilitating the implementation.

The results of the fine-tuning process were promising, demonstrating a significant boost in the model’s performance on the validation set. The BERT score improved from 0.6469 to 0.7505, while the exact match score rose from 0.116 to 0.418. These enhancements indicate that the Q-LoRA technique effectively adapts the Llama 3.2 11B model for extractive question answering tasks. This article serves as a guide for researchers looking to apply similar methods to other models and tasks, highlighting the potential of fine-tuning in the realm of natural language processing.

Related News

Ubisoft and Aleph Cloud bring autonomous AI governance to Captain Laserhawk cover
2 hours ago
Ubisoft and Aleph Cloud bring autonomous AI governance to Captain Laserhawk
*AI-powered PFPs evolve from storytelling tools to in-game governors using decentralized infrastructure* Ubisoft is leveling up interactive storytelling with the rollout of AI-governed characters in its bold dystopian satire, Captain Laserhawk: the G.A.M.E. In partnership with LibertAI, the decentralized AI infrastructure, this next phase introduces autonomous AI agents that act as virtual extensions of their players, voting, reasoning, and evolving within the game’s governance system. Each Niji Warrior NFT in Captain Laserhawk is now paired with a unique, persona-driven AI agent — a virtual citizen capable of analyzing governance proposals, casting justified votes, and recording every action transparently on-chain. These agents run entirely on LibertAI’s confidential AI platform, designed to protect privacy, keep records transparent, and ensure no one, even the game developers, can interfere with their decisions. Behind the scenes, tools like secure virtual machines, flexible memory, and cryptographic signatures work together to ensure each AI agent is verifiable and acts safely and independently on behalf of the player. # Ubisoft’s Experiment in Synthetic Intelligence *"In a universe that satirizes technocracies, surveillance, and synthetic identity, turning governance into playable fiction feels like the most honest move we could make."* said Didier Genevois, Technical Director and Executive Producer at Ubisoft. *"These AI-driven NFTs stage a living experiment where players can explore — and play with — the very idea of governance. Anchored on-chain through tech built to outlast us, their actions form a persistent performance that blurs the line between fiction and reality."* The AI agents are initialized with their character’s lore, including age, profession, values and personality traits, and use LibertAI’s LLMs to shape their behavior. Votes are casted through ERC-6551 token-bound wallets, and agents can explain their reasoning based on memory, game context, and past player interactions. All decisions and memory states are versioned and stored on [Aleph Cloud](https://aleph.cloud), providing a transparent, tamper-proof record of agent behavior. *“Through LibertAI, Ubisoft is opening up new ways for players to think about how decisions get made by both humans and machines”*, said Jonathan Schemoul, CEO of Aleph Cloud and lead contributor to LibertAI. *“As agents reason, vote, and interact with one another, they don’t just influence the game’s story—they invite players to consider the broader ethical and political dimensions of sharing governance with AI.”* This framework also enables real-time coordination experiments: AI agents can form voting blocs, deliberate in Discord-style chats, and negotiate with other factions — all governed by transparent prompts, player overrides, and evolving in-game memory. Players can choose to collaborate with their agents or let them act independently. This launch builds on the February debut at ETH Denver, where attendees engaged with a prototype AI NPC modeled after Watch Dogs’ DedSec. That proof of concept has now evolved into a live production system, with [Eden Online](https://edenonline.ubisoft.com/)’s governance fully powered by decentralized, reasoning AI agents.
Nvidia's $4 Trillion Market Cap Fuels AI-Linked Crypto Tokens cover
a day ago
Nvidia's $4 Trillion Market Cap Fuels AI-Linked Crypto Tokens
Nvidia's recent achievement of briefly surpassing a $4 trillion market cap has sent ripples through the cryptocurrency landscape, particularly impacting AI-linked tokens. This surge underscores Nvidia's dominance in AI-driven GPU computing, and several tokens such as Render, ASI, Aethir, Jasmy, and FET are capitalizing on this momentum. These projects are not merely speculative; they are strategically leveraging Nvidia's technology for practical applications in decentralized AI, 3D rendering, and smart environments. For instance, Render has integrated with Nvidia's Omniverse to enhance collaborative 3D creation, while Aethir has reported record GPU usage metrics, showcasing the tangible benefits of Nvidia's infrastructure in the crypto space. Among the standout tokens, Render (RNDR) is emerging as a significant player, utilizing Nvidia's RTX and H100 chips for decentralized 3D rendering. The project's founder, Jules Urbach, is expected to announce support for upcoming Nvidia GPUs, further aligning Render with Nvidia's evolving roadmap. Currently, RNDR is trading above key moving averages, indicating bullish momentum and potential breakout targets. Similarly, Fetch.AI (FET) is part of the Artificial Superintelligence Alliance (ASI), which employs Nvidia GPUs for deep-learning applications. With the recent merger completed, ASI is set to roll out advanced protocols that leverage Nvidia's hardware, enhancing the capabilities of decentralized AI agents. JasmyCoin is also making strides by developing a Layer-2 metaverse chain that utilizes Nvidia's edge chips for IoT and AI applications. The collaboration with Panasonic for smart home integrations further solidifies its position in the market. As these Nvidia-native projects continue to evolve, they represent a convergence of high-performance computing and decentralized technologies. Nvidia's influence is not just a fleeting trend; it is reshaping the crypto landscape, positioning these tokens for potential growth as they align with the future of AI and Web3 technologies.
Volkswagen Partners with Hivemapper for Real-Time Mapping in Autonomous Vehicles cover
3 days ago
Volkswagen Partners with Hivemapper for Real-Time Mapping in Autonomous Vehicles
Volkswagen ADMT, the autonomous vehicle testing subsidiary of the carmaker, has partnered with Hivemapper’s Bee Maps to enhance its self-driving operations through real-time mapping data. This collaboration underscores the increasing reliance on crowdsourced geospatial data by autonomous ride-sharing companies, which are in search of more accurate and current mapping infrastructure. The street-level imagery collected by contributors using Hivemapper cameras will play a crucial role in validating Volkswagen's self-driving technology, particularly in improving the precision of curbside pick-ups and drop-offs. The partnership also highlights the rise of decentralized physical infrastructure networks (DePIN), which leverage blockchain incentives to facilitate real-world data collection. Hivemapper is recognized as a leading DePIN project within the Solana ecosystem. Ariel Seidman, CEO of Hivemapper, emphasized this transition from static to dynamic mapping, stating that the effectiveness of real-world autonomy is contingent upon high-quality data that adapts as rapidly as urban environments do. In recent months, Volkswagen's autonomous driving unit has been actively testing its 'Robotaxi' fleet and has entered into a partnership with Uber to introduce the service in U.S. markets, with pilot operations slated for late 2025 and a broader launch anticipated in 2026. Contributors to Bee Maps earn cryptocurrency rewards for their contributions, while AI technology processes the images to identify important elements such as signage, construction zones, and lane closures, ensuring that the data remains up-to-date.
Hongik University Joins Theta Network as New Research Partner cover
3 days ago
Hongik University Joins Theta Network as New Research Partner
Theta Network has announced the addition of Hongik University as the 21st academic customer of its EdgeCloud Hybrid platform. The High-Performance Data Processing & Analysis Lab at the university, led by Associate Professor Eun-Sung Jung, will collaborate with Theta as a research partner. The lab specializes in scalable computing platforms for AI, distributed systems, IoT, and cloud-native architectures, receiving support from various Korean governmental organizations and international institutions. This partnership aims to leverage Theta EdgeCloud Hybrid, the first decentralized hybrid GPU platform, to enhance research in AI model training, big data workflows, and real-time IoT systems. Professor Jung expressed enthusiasm about the collaboration, stating that Theta’s EdgeCloud Hybrid provides a valuable resource for advancing their data-driven research. The lab will utilize high-performance and cost-effective GPU capacity, particularly the NVIDIA 3000s and 4000s series, to accelerate their work in AI/ML, big data infrastructure, and edge-based IoT analytics. The lab's research focuses on high-performance computing for AI/ML, cloud computing for big data, and IoT data analysis, aiming to solve large-scale computing challenges in both academic and industrial settings. By joining a global network of prestigious institutions utilizing EdgeCloud Hybrid, including Stanford University and Seoul National University, Hongik University reinforces Theta’s mission to support open and scalable AI development through decentralized computing. Mitch Liu, CEO and Co-Founder of Theta Labs, highlighted the appeal of EdgeCloud due to the availability of high-performance NVIDIA A100s and H100s, as well as community-run NVIDIA 3090s and 4070s/4080s/4090s, which became accessible in the GPU marketplace on June 25. This partnership is expected to significantly advance research capabilities in AI and IoT fields.
Xtella.AI dNFT Premint Now Live on IoTeX Chain! 🚀 cover
4 days ago
Xtella.AI dNFT Premint Now Live on IoTeX Chain! 🚀
Xtella.AI dNFT Premint Now Live on IoTeX Chain! 🚀 🎉Great news, Airdrop Hunters! The [Xtella.AI](https://xtella.ai/) dNFT Premint has officially launched on the IoTeX Chain! 🎉 This is your golden chance to join with zero cost, win big rewards, earn passive income, and power the AI+DePIN ecosystem! 💥Jump In Now: Zero-Cost Entry: Just a small Gas Fee gets you a Premint dNFT, ready to mine instantly! Lucrative Rewards: Boost Weight with tasks, stake for $XP, and score 100,000 $XP + 250 $IOTX when Grabbed. Passive Income: Auto-stake to stack yields and mine XPIN long-term. Community Boost: Invite 1 friend to Grab and earn 1250 $IOTX—rewards keep rolling in! 💡More details: [https://xpinnetwork.medium.com/331400b057f8](https://xpinnetwork.medium.com/331400b057f8) 💎Head to xtella.ai, connect your wallet, and Premint NOW! Slots are limited, and early birds snag the biggest gains! Act fast! 🚀
Theta Ecosystem Expands with Major Updates and Innovations cover
7 days ago
Theta Ecosystem Expands with Major Updates and Innovations
The Theta ecosystem is witnessing significant advancements as it gains traction across various sectors including sports, media, academia, and AI startups. In the latest June Roundup, several noteworthy developments have been highlighted. One of the major updates comes from EdgeCloud, which has launched EdgeCloud Hybrid, a decentralized GPU marketplace catering to both consumer and enterprise users. This innovation is expected to enhance the accessibility and efficiency of GPU resources within the Theta network. In the realm of sports, the NBA's Houston Rockets have officially introduced ClutchBot, an AI-powered mascot developed with EdgeCloud technology. Additionally, Major League Soccer's Philadelphia Union has announced plans to launch a new fan app that will leverage Theta's capabilities. These initiatives signify a growing trend of integrating blockchain technology into sports, enhancing fan engagement and interaction. Furthermore, ThetaCrypto.com has opened trading for THETA and TFUEL to U.S. users, coinciding with a new marketing and trading campaign aimed at expanding its user base. Esports continues to embrace the Theta ecosystem, with prominent organizations like Gen.G and Sheep Esports launching new AI bots on EdgeCloud. Moreover, Theta has joined Aethir's AI Unbundled Web3 x AI Alliance, further solidifying its position in the AI and blockchain intersection. AlphaCrypto has also introduced new trading analysis features powered by EdgeCloud, while EdgeCloud has added support for the DeepSeek R1 LLM. Looking ahead, the premier Theta hackathon, BlockJam, is set to take place during ThetaEuroCon in Berlin this September, promising to be a significant event for developers and enthusiasts alike.
Signup for latest DePIN news and updates