EfficientNet

Uses your system voice; voice and rate in settings.

Snapshot

≈ 4 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

Family of computer vision models

EfficientNet is a family of convolutional neural networks (CNNs) for computer vision published by researchers at Google AI in 2019.[1] Its key innovation is compound scaling, which uniformly scales all dimensions of depth, width, and resolution using a single parameter.

EfficientNet models have been adopted in various computer vision tasks, including image classification, object detection, and segmentation.

Compound scaling

edit edit source

EfficientNet introduces compound scaling, which, instead of scaling one dimension of the network at a time, such as depth (number of layers), width (number of channels), or resolution (input image size), uses a compound coefficient ϕ to scale all three dimensions simultaneously. Specifically, given a baseline network, the depth, width, and resolution are scaled according to the following equations:[1]depth multiplier: d=αϕwidth multiplier: w=βϕresolution multiplier: r=γϕsubject to α⋅β2⋅γ2≈2 and α≥1,β≥1,γ≥1. The α⋅β2⋅γ2≈2 condition is such that increasing ϕ by a factor of ϕ0 would increase the total FLOPs of running the network on an image approximately 2ϕ0 times. The hyperparameters α, β, and γ are determined by a small grid search. The original paper suggested 1.2, 1.1, and 1.15, respectively.

Architecturally, they optimized the choice of modules by neural architecture search (NAS), and found that the inverted bottleneck convolution (which they called MBConv) used in MobileNet worked well.

The EfficientNet family is a stack of MBConv layers, with shapes determined by the compound scaling. The original publication consisted of 8 models, from EfficientNet-B0 to EfficientNet-B7, with increasing model size and accuracy. EfficientNet-B0 is the baseline network, and subsequent models are obtained by scaling the baseline network by increasing ϕ.

Variants

edit edit source

EfficientNet has been adapted for fast inference on edge TPUs[2] and centralized TPU or GPU clusters by NAS.[3]

EfficientNet V2 was published in June 2021. The architecture was improved by further NAS search with more types of convolutional layers.[4] It also introduced a training method, which progressively increases image size during training, and uses regularization techniques like dropout, RandAugment,[5] and Mixup.[6] The authors claim this approach mitigates accuracy drops often associated with progressive resizing.

See also

edit edit source

References

edit edit source
  1. ^ a b Tan, Mingxing; Le, Quoc V. (2020-09-11), EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks, arXiv:1905.11946
  2. ^ "EfficientNet-EdgeTPU: Creating Accelerator-Optimized Neural Networks with AutoML". research.google. August 6, 2019. Retrieved 2024-10-18.
  3. ^ Li, Sheng; Tan, Mingxing; Pang, Ruoming; Li, Andrew; Cheng, Liqun; Le, Quoc; Jouppi, Norman P. (2021-02-10), Searching for Fast Model Families on Datacenter Accelerators, arXiv:2102.05610
  4. ^ Tan, Mingxing; Le, Quoc V. (2021-06-23), EfficientNetV2: Smaller Models and Faster Training, arXiv:2104.00298
  5. ^ Cubuk, Ekin D.; Zoph, Barret; Shlens, Jonathon; Le, Quoc V. (2020). "Randaugment: Practical Automated Data Augmentation With a Reduced Search Space": 702–703. arXiv:1909.13719. {{cite journal}}: Cite journal requires |journal= (help)
  6. ^ Zhang, Hongyi; Cisse, Moustapha; Dauphin, Yann N.; Lopez-Paz, David (2018-04-27), mixup: Beyond Empirical Risk Minimization, arXiv:1710.09412
edit edit source Google AI Computer
programs Other Machine
learning Generative
AI See also Differentiable computingGeneral Hardware Software libraries

Text derived from https://en.wikipedia.org/w/index.php?oldid=1309177911 under CC BY-SA 4.0; modifications have been made.

References

  1. ↩Wikipedia “EfficientNet” revision 1309177911 · retrieved 2026-10-20 publication date (×1) · confidence 0.95 revision URL