NTT Advances AI Accuracy, Security & Cost at ICML 2025

Conference showcased the most innovative global breakthroughs in AI and ML fundamental research and development

News Highlights:

• NTT Research and NTT R&D scientists presented 12 papers at ICML 2025, one of the world’s most prestigious conferences on AI and machine learning.
• Three papers co-authored by NTT Research Physics of AI Group scientists included new findings on why algorithms designed to improve LLM accuracy and factual associations can negatively impact models’ factual recall and reasoning abilities.
• Eight papers co-authored by NTT R&D scientists present included proposal of world’s first “Portable Tuning” technology to reduce retraining cost and improve sustainability of generative AI.

NTT Research, Inc. and NTT R&D, divisions of NTT (TYO:9432), researchers presented twelve papers at the forty-second International Conference on Machine Learning (ICML), held July 13-19, 2025 in Vancouver. ICML 2025 is a leading global conference “dedicated to the advancement of the branch of artificial intelligence known as machine learning,” including for applications such as machine vision, computational biology, speech recognition and robotics.

The three accepted papers co-authored by members of NTT Research’s Physics of Artificial Intelligence (PAI) Group advanced our understanding of large language model (LLM) accuracy, machine learning interpretability and the neural mechanisms of short-term memory in recurrent neural networks (RNNs).

In the paper “Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing,”¹ researchers explored the question of why Knowledge Editing (KE) algorithms—algorithms that alter LLMs’ weights to fix incorrect, outdated or unwanted factual associations—can negatively impact models’ factual recall accuracy and reasoning abilities. Researchers demonstrated that a phenomenon they call “representation shattering” is responsible for this degradation in models’ factual recall and reasoning performance, showing that KE “inadvertently affects representations of entities beyond the targeted one, distorting relevant structures that allow a model to infer unseen knowledge about an entity.”

In “Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models,”² researchers revealed a fundamental limitation in Sparse Autoencoders (SAEs), dictionary learning frameworks for improved machine learning interpretability: severe instability that undermines their reliability and their usefulness as an interpretability tool. The researchers presented a solution, Archetypal SAEs (and a variant, Relaxed Archetypal SAEs), that significantly enhances SAE stability.

In “Dynamical Phases of Short-Term Memory Mechanisms in RNNs,”³ researchers explored the largely unknown role of neural mechanisms in short-term memory. Researchers’ work provided new insights into these short-term memory mechanisms proposing experimentally testable predictions for further systems neuroscience study.

“As a company, NTT has committed to developing AI technologies that enable sustainable development, respect human autonomy, ensure fairness and openness and protect security and privacy,” said NTT Research PAI Group Leader Hidenori Tanaka. “Realizing those positive outcomes begins with the exploration of AI ML at a fundamental level through scientific examination. I am proud of the work being performed by the NTT Research PAI Group and our colleagues at NTT R&D, and the opportunity to share our findings with the rest of the AI academic community at ICML 2025 was a great privilege.”

In addition to the PAI Group’s accepted papers, eight papers presented at ICML 2025 were co-authored by scientists affiliated with NTT R&D laboratories in Japan.

One paper, “Portable Reward Tuning: Towards Reusable Fine-Tuning Across Different Pretrained Models,”⁴ presented the world’s first “Portable Tuning” technology that eliminates the need to retrain specialized AI models when updating underlying foundation models—significantly reducing retraining costs and improving the sustainability of customized generative AI.

Another paper, “Plausible Token Amplification for Improving Accuracy of Differentially Private In-Context Learning Based on Implicit Bayesian Inference,”⁵ proposed Plausible Token Amplification (PTA), becoming the first to theoretically reveal how adding “noise” based on differential privacy (a method to mitigate data leakage) degrades the accuracy of LLMs. This technique is expected to promote the use of LLMs in fields using personal user-related data, like healthcare, government and finance.

And in, “K²IE: Kernel Method-based Kernel Intensity Estimators for Inhomogeneous Poisson Processes”⁶ researchers demonstrated a major improvement in the computational efficiency of large-data-set Poisson processes, which are used to analyze and forecast event patterns in space and time, from posts on social media platforms SNS to disease outbreaks.

The five other accepted papers authored or co-authored by NTT R&D scientists included:

• “Positive-Unlabeled AUC Maximization Under Covariate Shift”⁷
• “Natural Perturbations for Black-Box Training of Neural Networks by Zeroth-Order Optimization”⁸
• “Learning to Generate Projections for Reducing Dimensionality of Heterogeneous Linear Programming Problems”⁹
• “Guided Zeroth-Order Methods for Stochastic Non-Convex Problems with Decision-Dependent Distributions”¹⁰
• “Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines”¹¹

Additionally, NTT R&D researchers presented the poster about model merging, “Linear Mode Connectivity between Multiple Models modulo Permutation Symmetries.”¹² Previous studies have proposed methods for merging two pretrained models by leveraging the permutation symmetry inherent in neural networks to construct functionally equivalent but distinct models. This study extended these methods to merge three or more models simultaneously, demonstrating that the performance of the merged model improves as the number of models being merged increases.

NTT is committed to combining trust, integrity and innovation to empower businesses and communities with cutting-edge AI solutions that drive efficiency, safety and progress globally. Established in April 2025, the NTT Research PAI Group is studying the nature of AI, aiming to “address similarities between biological and artificial intelligences, further unravel the complexities of AI mechanisms and build trust that leads to more harmonious fusion of human and AI collaboration.”

For more information about how NTT is innovating AI for a better future, please visit: https://www.global.ntt/innovation/artificial-intelligence

For more information about the NTT Research PAI Group, visit: https://ntt-research.com/pai-group

Index

1. Nishi, K., Ramesh, R., Okawa, M., Khona, M., Tanaka, H., & Lubana, E. S. (2024). Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing. ArXiv. https://arxiv.org/abs/2410.17194
2. Fel, T., Lubana, E. S., Prince, J. S., Kowal, M., Boutin, V., Papadimitriou, I., Wang, B., Wattenberg, M., Ba, D., & Konkle, T. (2025). Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models. ArXiv. https://arxiv.org/abs/2502.12892
3. Kurtkaya, B., Dinc, F., Yuksekgonul, M., Cirakman, E., Schnitzer, M., Yemez, Y., Tanaka, H., Yuan, P., & Miolane, N. (2025). Dynamical phases of short-term memory mechanisms in RNNs. ArXiv. https://arxiv.org/abs/2502.17433
4. Chijiwa, D., Hasegawa, T., Nishida, K., Saito, K., & Takeuchi, S. (2025). Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models. ArXiv. https://arxiv.org/abs/2502.12776
5. Yamasaki, Y., Niwa, K., Chijiwa, D., Fukami, T., Miura, T. (2025). Plausible Token Amplification for Improving Accuracy of Differentially Private In-Context Learning Based on Implicit Bayesian Inference. OpenReview. https://openreview.net/forum?id=skAjaAEuA2
6. Kin, H., Iwata, T., Fujino, A. (2025). K²IE: Kernel Method-based Kernel Intensity Estimators for Inhomogeneous Poisson Processes. OpenReview. https://openreview.net/forum?id=XKuTFM93mG
7. Kumagai, A., Iwata, T., Takahashi, H., Nishiyama, T., Adachi, K., Fujiwara, Y. (2025). Positive-Unlabeled AUC Maximization Under Covariate Shift. OpenReview. https://openreview.net/forum?id=HQEPgICjBS
8. Sawada, H., Aoyama, K., Hikima, Y. (2025). Natural Perturbations for Black-Box Training of Neural Networks by Zeroth-Order Optimization. OpenReview. https://openreview.net/forum?id=ULAQ9GmJlo
9. Iwata, T., Sakaue, S. (2025). Learning to Generate Projections for Reducing Dimensionality of Heterogeneous Linear Programming Problems. OpenReview. https://openreview.net/forum?id=LnoTEuVhud
10. Hikima, Y., Sawada, H., Fujino, A. (2025). Guided Zeroth-Order Methods for Stochastic Non-Convex Problems with Decision-Dependent Distributions. OpenReview. https://openreview.net/forum?id=cRmuEY7jhb
11. Sonoda, S., Hashimoto, Y., Ishikawa, I., Ikeda, M. (2025). Deep Ridgelet Transform and Unified Universality Theorem for Deep and Shallow Joint-Group-Equivariant Machines. OpenReview. https://openreview.net/forum?id=JKsxKPXXUd
12. Ito, A., Yamada, M., Kumagai, A. (2025). Linear Mode Connectivity between Multiple Models modulo Permutation Symmetries. OpenReview. https://openreview.net/forum?id=qaJuLzY6iL