Resources

Presentation materials and surveys used during the study sessions are available to the public. Note that the contents are mainly written in Japanese.

Additionally, we are updating the Overview of Japanese LLMs.

• 2024-05-26
  • LLM-jp Status Report (Kurohashi)
  • Development of a Japanese LLM with 100B Parameters (Omi)
  • Overview of R&D for Generative AI at Databricks (Yayoi)
  • Multilingual LLM: Data Construction, Fine-tuning, and LLM-based Evaluation (Peter)
  • Corpus Construction Working Group (Kawahara)
  • Model Building Working Group (Suzuki)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine) (To be uploaded)
  • Multi-modal Working Group (Okazaki)
• 2024-03-26
  • LLM-jp Status Report（Kurohashi）
  • Geniac Program Initiatives at ABEJA (Hattori)
  • Estimated timing of transition from Prompt Tuning to Fine Tuning（Kubo）
  • An experiment to reduce halucinations produced by ichikara-instruction, Less NE experiments (Sekine)
  • Evaluation and Tuning Working Group (Miyao)
  • Corpus Construction Working Group (Kawahara)
  • Safety Working Group (Sekine)
  • Model Building Working Group (Suzuki)
• 2024-01-22
  • Self-improvement loops from observational data, Grounding LLMs with causal inference methods (Sannai)
  • Comparative analysis of LLM human evaluation and GPT evaluation using ichikara-instruction (Sekine)
  • Kotoba Technologies voice-based model and Mamba architecture (Kojima and Kasai)
  • Large language model, Swallow (TokyoTech LLM members)
  • Safety Working Group (Sekine)
  • Corpus Construction Working Group (Kawahara)
  • Evaluation and Tuning Working Group (Miyao)
  • Model Building Working Group (Suzuki)
• 2023-11-29
  • LLM-jp Status Report (Kurohashi)
  • LLM Security Survey and Japanese Data (Suzuki)
  • Development of 13 Billion Parameter Japanese Pre-Learning Model for Business Domains and Latest Information (Omi)
  • Potential and Progress of Large-scale Language Models in Medicine (Kodera)
  • Corpus Construction Working Group (Kawahara)
  • Model Building Working Group (Suzuki)
  • Evaluation and Tuning Working Group (Miyao)
  • Safety Working Group (Sekine)
• 2023-10-18
  • Training and applying Vision & Language model with Heron (Inoue)
  • Technical overview of Japanese LLM “ELYZA-japanese-Llama-2-7b” based on “Llama 2” from Meta (Nakamura)
  • Technical overview of pre-training for PLaMo-13B (Mikami)
  • LLM models for robotics (Kawaharazuka)
• 2023-09-04
  • A survey for instruction tuning for Model imitation explained (Mizuki)
  • PEFT: LazyLoRA (Wu)
  • Overview of R&D for LLM at Stability AI Japan (Lee)
  • Generative AI, construction and copyright (Kakinuma)
  • Construction of a large-scale bilingual English/Japanese language model “Weblab-10B” (Kojima)
• 2023-07-20
  • ACL2023 participation announcement (Kodama) (Yamada) (Ueda) (Deguchi)
  • LLM Peripheral technology (June/July 2023) (Tsuruoka)
• 2023-06-19
  • Overview of the model project for CyberAgent (Ishigami)
  • Tips for training of T5 in Japanese (Nishitoba)
  • Participation report for the ABCI Grand challenge (Sakaguchi)
  • Research and development of large-scale language models at NICT (Torizawa)​
  • Future Cooperation for NII：Domain adaptation for Biomedical (Aizawa)
• 2023-05-15
  • PaLM2 Technical Report - Thesis Introduction (Okazaki)
  • Toward the co-evolution of Japanese basic models and language understanding benchmarks (Kawahara)
  • Survey about LLM（Kawahara, Sugawara, and Kurita)