Daily AI Newsletter: Impressive new paper: Seeing the World through Your Eyes

In today’s email:

👁️ Seeing the World Through Your Eyes, An Impressive new paper
🔥 Google introduces a virtual Try-O.
🎥 Zero-shot Text-guided Video-to-video Translation
🛠 Various AI-related tools and platforms, including My Mind, Obviously AI, Nesh, SCRAP, ChatShitGPT, AIArtTalentShow, Bard, and more.

Highlights💡

Impressive new paper: Seeing the World through Your Eyes [Link]

The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like. By imaging the eyes of a moving person, we can collect multiple views of a scene outside the camera's direct line of sight through the reflections in the eyes. In this paper, we reconstruct a 3D scene beyond the camera's line of sight using portrait images containing eye reflections.
This task is challenging due to 1) the difficulty of accurately estimating eye pose and 2) the entangled texture of the eye iris and the scene reflections. Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer's eye iris texture. We further propose a simple regularization prior on the iris texture pattern to improve reconstruction quality. Through various experiments on synthetic and real-world captures featuring people with varied eye colors, we demonstrate the feasibility of our approach to recover 3D scenes using eye reflections.

Render a Video: Zero-shot Text-guided Video-to-video Translation [Link][Paper]

Advanced text-to-image diffusion models have demonstrated their capability to produce top-tier images. Yet, their application to the realm of video has faced difficulties in ensuring continuity across the frames, posing a significant hurdle. This research introduces a groundbreaking zero-shot text-guided video-to-video translation mechanism to modify image models for videos. The mechanism is comprised of two segments: keyframe translation and full video translation.

In the first segment, a modified diffusion model is employed to create keyframes. To ensure consistency in shapes, textures, and colors, we apply hierarchical cross-frame constraints. Following this, the second segment extends these keyframes to the remaining frames using a method of temporal-aware patch matching and frame blending.

Our mechanism is able to achieve consistency in global style and local texture temporally while also being cost-effective as it requires no re-training or optimization. Furthermore, it is compatible with current image diffusion techniques, thereby leveraging their benefits. This includes the customization of specific subjects using LoRA and the incorporation of additional spatial guidance through ControlNet.

The thorough experimental findings provide evidence of the superior performance of our proposed mechanism compared to existing methodologies in the production of high-quality and time-consistent videos.

Learn AI like it’s 2023 [Link]

AI won’t take your job. Someone using AI will. Best time to level up? Yesterday. Second best? Right now.

Luckily, there’s Brilliant — the interactive app that makes it easy to master concepts in math, data, and computer science in just minutes a day.

Here’s how:

They have thousands of lessons on tons of topics — from AI and neural networks to data science.
They break down complex concepts into digestible building blocks that stick.
Their interactive style keeps you engaged, so it’s easy to build a daily habit.

Join over 10 million people around the world and start building skills in minutes a day. You can try everything Brilliant has to offer for free for a full 30 days. Plus, right now you can get an exclusive 20% off an annual premium membership.

Google introduces a virtual Try-On [Link]

See clothes on a wide range of models and use new types of filters to better refine your options.

Google is introducing two new features to enhance the online clothes shopping experience. The first feature is a virtual try-on for apparel, which utilizes generative AI to show clothes on real models of various sizes, skin tones, body types, and hair types. This allows users to see how the clothes will look on them before making a purchase. Users can select the "Try On" option on Google Search and choose a model that represents them best. Initially available for women's tops from brands like Anthropologie, Everlane, H&M, and LOFT, the feature will expand to include more brands and items over time, including men's tops.

The second feature is guided refinements, which helps users find the perfect piece by allowing them to refine their search based on criteria like color, style, and pattern. Using machine learning and visual matching algorithms, the feature presents options from various online stores, providing a wider range of choices. This feature is available for tops initially and can be accessed within product listings.

Google believes that these advancements, powered by AI, will make online clothes shopping more convenient and enjoyable for users. They plan to continue leveraging advanced technologies like AI to improve the online shopping experience further. [Related Paper from Google]