Tracking Everything Everywhere All at Once

In today’s email:

🔥 Impressive new paper: Track Everything Everywhere All at Once
🔓 Google Generative AI Platform is now open to everyone.
🤯 Generative AI is more biased than humans
🛠 Various AI-related tools and platforms, including MusicGen, Translate Audio, Leapp AI, AI Spend, AcademicGPT, InboxGPT, GPT Engineer, PaperBrain, Whisper Web, and more.

Highlights💡

Tracking Everything Everywhere All at Once [Link]

The paper introduces OmniMotion, a new method for accurately estimating dense and long-range motion in videos. Traditional optical flow and particle video tracking algorithms face challenges in tracking through occlusions and maintaining consistent motion trajectories over time due to their limited temporal windows. In contrast, our approach offers a complete and globally consistent motion representation.

OmniMotion employs a quasi-3D canonical volume to represent the video and performs pixel-wise tracking using mappings between local and canonical space. This representation enables us to ensure global consistency, track through occlusions, and model various types of camera and object motion.

Through extensive evaluations of the TAP-Vid benchmark and real-world footage, we demonstrate that our approach significantly outperforms previous state-of-the-art methods both quantitatively and qualitatively.

BREAKING: Google opens its Generative AI Platform to all. [Link]

Google Cloud has announced the general availability of Generative AI support on Vertex AI, offering tools and models for developers to build custom AI applications. The update includes access to Google's text model PaLM 2, Embeddings API for text, and other models in Model Garden. In addition, Vertex AI provides tools for model tuning, deployment, and enterprise-grade data governance. Some of the companies already leveraging these capabilities include GA Telesis, GitLab, and Canva. The announcement also extends access to Enterprise Search on Generative AI App Builder, facilitating the creation of custom chatbots and search engines.

Humans Are Biased. Generative Ai Is Even Worse [Link]

A detailed analysis revealed that Stable Diffusion amplifies racial and gender disparities in the images it generates. For instance, high-paying jobs were typically associated with lighter-skinned subjects, while those with darker skin tones were commonly linked with low-paying jobs. The model also tends to generate more images of men compared to women. These biases could result in unfair treatment and the perpetuation of stereotypes. Stable Diffusion's maker, StabilityAI, has acknowledged these biases and is planning to develop open-source models trained on diverse datasets. Industry leaders like Adobe and Canva are already using text-to-image AI in various applications, and estimates suggest that generative AI could soon be responsible for producing a significant proportion of internet content, potentially exacerbating these bias issues.

The AI image generation model, Stable Diffusion, is reportedly perpetuating existing gender and racial biases. Analysis shows that the generated images overrepresent men and lighter-skinned individuals in higher-income roles, while women and darker-skinned individuals are overrepresented in lower-income roles. For example, in reality, 34% of US judges are women, but the model generated images of women for the keyword "judge" only about 3% of the time. Moreover, people with darker skin tones appeared in 70% of the images for "fast-food worker," even though 70% of actual fast-food workers in the US are white. The model was also found to be significantly misrepresenting women and people of color in multiple fields. This bias in the AI model can reinforce harmful stereotypes and pose significant barriers for underrepresented groups. AI tools have previously been criticized for misidentifying the gender of people in photos, particularly misgendering women of color.