Daily item co-occurrence embeddings are paired with next-day behavior without future-behavior leakage. Multi-day click-through rate (CTR) training therefore mixes versions whose directions and norms may differ for the same item. We compare Word2Vec and an item-graph XSimGCL adaptation across initialization, inheritance, post-processing, training constraints, and downstream augmentation. The inventory contains 48 configurations, including 32 common-protocol CTR evaluations. Observed metrics are reported as relative percentage changes from the corresponding Word2Vec baseline (0%), with raw values and exact dataset sizes withheld. Three findings emerge. First, multiple upstream interventions improve compatibility and CTR. ID-init and orthogonal alignment increase cosine by 51.2% and 93.4%, and GAUC by 2.2% and 2.6%, relative to ordinary Word2Vec. Retrieval and geometry remain insufficient downstream surrogates. Second, warm starts perform strongly but accumulate scale. The day-matched norm increase over Word2Vec reaches 102.5% after eight days; daily global rescaling limits it to 8.2% and attains the highest observed GAUC increase, 4.0%. Third, CTR learning through a dimensionality-reducing encoder further adapts to residual drift. Against dimension-matched random-projection, SVD, and untrained-network controls, trained encoders attain the highest cross-day cosine for four of six upstream methods. The other two rank third and second in train-to-test cosine. These industrial results support complementary roles for upstream stabilization, inherited-scale management, and downstream task learning.
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent 等 1137 位作者
In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.