What happens when you compress Al's heavier parts with a new centrifugal blade? You save so much storage, memory, GPU stacks and compute gas-tanks, of course. Besides boosting speed, cutting inference latency and expanding compatibility for small devices and edge-networks. But how does this 'squeeze' affect accuracy, error compensation and application-ease? And what about all the other grinders that are attacking the same problem? Like SLMs, GPTQ and QuIP? Recently, Yandex Research, IST Austria, NeuralMagic, and KAUST announced that they have developed what they call 'two innovative compression methods for large language models'. It was also claimed that, when combined, these methods allow for a reduction in model size by up to 8 times while preserving response quality by 95 per cent. Compressed models like Llama 2 13B can run on I GPU instead of 4- they added. So how does it all work and does it address the issues we mentioned earlier?
We compress it all in this interview with Artem Babenko, Head of Yandex Research. He oversees scientific research at Yandex and the company's engagement with the international scientific community. He also supervises a team of approximately 30 researchers engaged in various areas of computer science. According to Artem, his main achievements are his scientific contributions in three key areas: neural networks for image search, high-dimensional vector compression, and fast search across massive databases containing billions of records. Who better than he to explain the ambition, ingredients and final taste of compression? Let's press those buttons.
Can you explain additive quantization and PV-tuning in simpler terms- for a layman? Is it similar to model pruning?
Denne historien er fra October 2024-utgaven av DataQuest.
Start din 7-dagers gratis prøveperiode på Magzter GOLD for å få tilgang til tusenvis av utvalgte premiumhistorier og 9000+ magasiner og aviser.
Allerede abonnent ? Logg på
Denne historien er fra October 2024-utgaven av DataQuest.
Start din 7-dagers gratis prøveperiode på Magzter GOLD for å få tilgang til tusenvis av utvalgte premiumhistorier og 9000+ magasiner og aviser.
Allerede abonnent? Logg på
Data center trends 2025: Innovation in powering and cooling AI racks, managing energy consumption
Vertiv anticipates increased innovation and integration for high-density computing, regulatory scrutiny around AI, focus on sustainability and cyber security efforts.
The Switzerland of Data and AI a weather-check
It is both remarkable and challenging when crucial R&D work - with global connotations- starts happening in Indian labs. An example- from a Cloud and AI player.
Prioritizing the Impossible: How Genpact's SOC Tackles Cyber Threats
Genpact's Vice President of Global Information Security, Faizul Mufti, discusses the company's multi-layered approach to cybersecurity. By prioritizing incident response, leveraging threat intelligence, and fostering a culture of security, Genpact aims to stay ahead of evolving cyber threats and protect its sensitive data.
Connected, Protected and Sustainable: Cisco's Vision for the Network of Tomorrow
As organizations face the demands of hybrid work, multi-cloud environments, and sustainable IT, networking is no longer just an infrastructure component― it's the backbone of digital transformation. In this exclusive conversation, Brink Sanders, Vice President of Global Networking Sales at Cisco, discusses how Cisco is helping enterprises navigate these challenges with forwardthinking solutions that blend innovation, security, and adaptability.
Multi-Cloud Mastery: Lessons from a CIO on Choosing the Right Cloud for the Job
In an exclusive interview, Prosenjit Sengupta, Group Chief Digital and Information Officer (CDIO), shares insights on ITC's cloud journey, discussing the challenges, benefits, and best practices of leveraging Azure, AWS, and Google Cloud.
Secure and Simplify SD-Branch Networks
Secure and seamless branch network management is critical in today's IoT-driven, cloud-centric world. A unified SD-Branch solution offers the much-needed convergence of networking and security to tackle modern-day challenges.
Is Your Multi-Cloud Environment Zero-Trust Ready?
Tony Velleca, CEO of CyberProof, a UST company, shared his insights on the challenges of implementing zero-trust security in multi-cloud environments. As organizations navigate the complexities of managing multiple cloud providers, Velleca emphasized the importance of achieving unified visibility, policy enforcement, and continuous monitoring to protect sensitive data and prevent cyber threats.
The Toxic Cloud Trilogy
As Indian organisations increasingly migrate to the cloud, they face a complex and evolving security landscape. Ari Eitan, Director of Research at Tenable, highlights the \"Toxic Cloud Trilogy\" - a convergence of publicly exposed workloads, critical vulnerabilities, and over-privileged identities - as a critical threat to Indian organisations. In this interview, Eitan discusses the unique vulnerabilities of cloud environments, systemic issues preventing organisations from fully addressing weaknesses and practical steps Indian organisations can take to mitigate these risks.
Networking Redefined: The Cisco Vision for a Connected Future
At the sidelines of Cisco Live 2024 in Melbourne, Raymond Janse van Rensburg, Vice President of Networking and Solutions Engineering for APJC at Cisco, engaged in an insightful conversation with Minu Sirsalewala, Executive Editor of Dataquest. During the interaction, he shared about the transformative trends shaping the networking and communication landscape. From AI-driven automation to the evolving dynamics of IoT, Cisco's strategies aim to redefine enterprise connectivity, security, and operational resilience.
HPCL's AI Evolution: From Generative Models to Secure Cloud Integration
Jayant Gupta, Executive Director of Information Systems at Hindustan Petroleum Corporation Limited (HPCL), shares insights on the company's strategic approach to leveraging generative AI, managing hybrid cloud environments, and staying ahead of advancements in emerging technologies.
