Daily AI Newsletter: Finding GPT-4’s mistakes with GPT-4

In today’s email:

🕵🏻‍♂️ Amazon Is Investigating Perplexity Over Claims of Scraping Abuse
☠️ Zuckerberg disses closed-source AI competitors as trying to ‘create God’
👀 When Your Building Super Is an A.I. Bot
🧰 10 new AI-powered tools and resources. Make sure to check the online version for the full list of tools.

Finding GPT-4’s mistakes with GPT-4

CriticGPT is a specialized model developed from GPT-4, designed to identify and critique errors in responses given by ChatGPT. The model was trained using Reinforcement Learning from Human Feedback (RLHF), a method where AI trainers rate various responses. This training involved exposing CriticGPT to numerous erroneous responses purposely inserted by trainers, who then documented the inaccuracies as if they were real mistakes. Through these exercises, CriticGPT not only learns to spot bugs but also aids trainers by enhancing their ability to identify subtle errors that might otherwise be missed in the nuanced outputs of advanced AI systems like ChatGPT.

The implementation of CriticGPT has proven to be beneficial in improving the quality of critiques. The combined efforts of humans and CriticGPT in spotting errors are more effective than those of individuals working alone. The critiques produced by this collaboration are more thorough, have fewer incorrect bug identifications (hallucinations), and are generally preferred by trainers. Notably, this collaboration has shown a preference rate of over 60% in scenarios involving naturally occurring mistakes, demonstrating the practical utility of incorporating AI-driven critique into the RLHF process.

Despite its successes, CriticGPT faces challenges, particularly with complex responses and distributed errors that are not confined to a single point. The current model has limitations in dealing with long and intricate tasks, suggesting that future enhancements and more sophisticated methods are necessary. Moving forward, the goal is to refine these tools to better support trainers in handling increasingly complex AI outputs, ensuring that the AI systems remain aligned and effective in real-world applications.

Copper is up 70%

Discover the Zonia Advantage

World Copper’s (OTC: WCUFF | TSX.V: WCU) Zonia project is leading the industry in next-generation copper production, and is poised to meet surging global copper demand. By leveraging simple metallurgy and proven SX-EW processing in a prime location, Zonia stands to achieve cash flow up to 4x faster than typical projects.

World Copper is one of the very few projects that can start producing before the end of this decade, just in time to capture the upcoming copper price surge.

LEARN MORE

(OTC: WCUFF | TSX.V: WCU)

Meta’s LLM Compiler is the latest AI breakthrough to change the way we code

Meta has introduced the Meta Large Language Model (LLM) Compiler, a pioneering suite of open-source models designed to enhance code optimization and revolutionize compiler design. This tool leverages a massive dataset of 546 billion tokens of LLVM-IR and assembly code to enable a deeper understanding and manipulation of compiler intermediate representations, assembly language, and optimization techniques. According to the researchers, the LLM Compiler achieves significant improvements in code size reduction and efficiency, boasting a 77% success rate in optimizing potential compared to autotuning searches.

The capabilities of the LLM Compiler extend to disassembly tasks, where it demonstrates a 45% success rate in converting x86_64 and ARM assembly back into LLVM-IR, which is crucial for reverse engineering and legacy code maintenance. Core contributor Chris Cummins highlighted the availability of the compiler in two model sizes and its effectiveness, suggesting a broad potential for AI-driven enhancements in code and compiler optimization that were previously unexplored.

By releasing the LLM Compiler under a permissive commercial license, Meta aims to facilitate widespread adoption and innovation in the software development community. This initiative could accelerate the integration of AI capabilities in programming, potentially reshaping the skills required for software engineers and compiler designers. As the field evolves, it will be intriguing to see how the LLM Compiler influences software development practices and the broader landscape of AI-driven optimization.

Amazon Is Investigating Perplexity Over Claims of Scraping Abuse

Amazon's cloud division, AWS, has initiated an investigation into Perplexity AI over allegations of web scraping. This inquiry follows discoveries that Perplexity, supported by the Jeff Bezos family fund and Nvidia and valued at $3 billion, may have scraped content from websites despite prohibitions set by the Robots Exclusion Protocol. Although this protocol is not legally enforceable, AWS requires adherence to it according to their terms of service, which prohibits any illegal activities and mandates compliance with all applicable laws.

WIRED's investigations have revealed that Perplexity's AI-driven search chatbot might have engaged in scraping and plagiarism, including accessing Condé Nast's properties despite explicit blocks via robots.txt. Perplexity reportedly used an unpublished IP linked to an AWS server to conduct these activities. In response to these allegations, Perplexity CEO Aravind Srinivas has refuted the implications, stating that there is a "fundamental misunderstanding" of how their systems and the internet operate. Meanwhile, Condé Nast and other media outlets have detected unauthorized access by Perplexity's IP addresses, indicating potential violations of web standards.

Responding to AWS's scrutiny, Perplexity spokesperson Sara Platnick stated that their operations hadn't changed in light of the investigation and claimed that their PerplexityBot generally respects robots.txt directives. However, she noted an exception when users manually input specific URLs, arguing that such actions do not constitute unauthorized crawling since it mimics a user’s direct interaction with a website. This stance aligns with Perplexity's assertion of compliance with AWS's terms of service, even as industry voices like Digital Content Next's CEO Jason Kint express concerns over potential misuse of copyrighted digital content.