发现信息
- 作者:Mebrahatom Yihdego Mekonen, Abrham Mamu Abebe, Gebrekiros Gebreyesus Gebremariam, Yemane Hailay Nega, Hailemariam Brhane Bitew, Kbrom Tekulu Desta, Abrha Tareke Araya
- 年份:2026
- 发表源:Discover Artificial Intelligence
- OpenAlex ID:https://openalex.org/W7225693897
- DOI:https://doi.org/10.1007/s44163-026-02726-7
- arXiv:待核验
- 检索主题:recursive-improvement
- OpenAlex 引用数快照:0(2026-11-12)
摘要线索
Transformers have evolved from a sequence-to-sequence architecture for machine translation into a general computational framework underlying large language, vision, speech, multimodal, scientific, and decision-making systems. This critical narrative review synthesizes the field along four connected axes: architectural evolution and attention efficiency; neural scaling laws across parameters, data, compute, and inference-time computation; mechanisms and evaluation debates surrounding in-context learning and emergent reasoning; and the expansion of Transformer systems across modalities and application domains. We move beyond a paper-by-paper chronology by comparing architectural choices under explicit constraints of context length, memory bandwidth, training compute, inference latency, data availability, and task structure. Particular attention is given to sparse and IO-aware attention, multi-query and grouped-query attention, mixture-of-experts routing, multimodal integration, and Transformer–state-space hybrids. We reconcile the Kaplan and Chinchilla scaling formulations with later replication and inference-aware analyses, and distinguish training-time scaling from test-time compute scaling. For in-context learning, we synthesize evidence concerning induction heads, implicit task inference, pre-training data composition, and the debate over metric-dependent emergence. The review also broadens application coverage to speech and audio processing, UAVs and autonomous systems, cybersecurity and intrusion detection, communications and networking, and time-series forecasting, in addition to scientific discovery, software engineering, healthcare, and education. Quantitative comparison tables are restricted to studies reporting results on common datasets and metrics, while explicitly noting differences in training protocols and computational budgets. We conclude that progress is increasingly determined not by parameter count alone, but by joint optimization of architecture, data quality, inference compute, hardware efficiency, and deployment constraints.
核验与筛选
- 当前状态:待筛选、去重和一手来源核验。
- OpenAlex 元数据和引用数只用于发现,不作为正式引文或质量判断。
- 记录正式版本、venue、方法细节、实验协议和局限后,才迁入稳定论文卡片。