Artificial Intelligence in Musculoskeletal Imaging: Current Status and Future Directions
Soterios Gyftopoulos1,2, Dana Lin1, Florian Knoll1, Ankur M. Doshi1 ... Show all
Share Share
+ Affiliations:
Citation: American Journal of Roentgenology: 1-8. 10.2214/AJR.19.21117
AbstractFull TextReferencesPDFPDF PlusAdd to FavoritesPermissionsDownload Citation
ABSTRACT :
OBJECTIVE. The objective of this article is to show how artificial intelligence (AI) has impacted different components of the imaging value chain thus far as well as to describe its potential future uses.

CONCLUSION. The use of AI has the potential to greatly enhance every component of the imaging value chain. From assessing the appropriateness of imaging orders to helping predict patients at risk for fracture, AI can increase the value that musculoskeletal imagers provide to their patients and to referring clinicians by improving image quality, patient centricity, imaging efficiency, and diagnostic accuracy.

Keywords: artificial intelligence, deep learning, fast MRI, machine learning, MRI, musculoskeletal imaging

Supported in part by grant R01 EB024532 from the National Institutes of Health for F. Knoll's machine learning research.

Acknowledgment

Previous sectionNext section
We thank Daniel Sodickson for his guidance and contributions to this manuscript.

References

Previous section
1. Harkey P, Duszak R Jr, Gyftopoulos S, Rosen-krantz AB. Who refers musculoskeletal extremity imaging examinations to radiologists? AJR 2018; 210:834–841 [Abstract] [Google Scholar]
2. Doshi AM, Moore WH, Kim DC, et al. Informatics solutions for driving an effective and efficient radiology practice. RadioGraphics 2018; 38:1810–1822 [Crossref] [Medline] [Google Scholar]
3. Poole DL, Mackworth AK, Goebel R. Computational intelligence and knowledge. In: Computational intelligence: a logical approach. New York, NY: Oxford University Press, 1998: 1–22 [Google Scholar]
4. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521:436–444 [Crossref] [Medline] [Google Scholar]
5. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, vol. 1. Lake Tahoe, NV: Curran Associates, 2012:1097–1105 [Google Scholar]
6. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yullie AL. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 2018; 40:834–848 [Crossref] [Medline] [Google Scholar]
7. Dosovitskiy A, Fischer P, Ilg E, et al. FlowNet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Washington, DC: IEEE, 2015:2758–2766 [Crossref] [Google Scholar]
8. Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016; 529:484–489 [Crossref] [Medline] [Google Scholar]
9. Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 1958; 65:386–408 [Crossref] [Medline] [Google Scholar]
10. Pinkus A. Approximation theory of the MLP model in neural networks. Acta Numer 1999; 8:143–195 [Crossref] [Google Scholar]
11. Cybenko G. Approximation by superpositions of sigmoidal function. MCSS 1989; 2:303–314 [Google Scholar]
12. Poggio T, Mhaskar H, Rosasco L, Miranda B, Liao Q. Why and when can deep–but not shallow–networks avoid the curse of dimensionality: a review. International Journal of Automation and Computing 2017; 14:503–519 [Crossref] [Google Scholar]
13. Lowe DG. Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece: IEEE, 1999:1150–1157 [Crossref] [Google Scholar]
14. Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995; 20:273–297 [Crossref] [Google Scholar]
15. Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1986; 323:6533–6536 [Crossref] [Google Scholar]
16. American College of Radiology (ACR). Appropriateness criteria. ACR website. acsearch.acr.org/list. Published 2019. Accessed April 1, 2019 [Google Scholar]
17. Lakhani P, Prater AB, Hutson RK, et al. Machine learning in radiology: applications beyond image interpretation. J Am Coll Radiol 2018; 15:350–359 [Crossref] [Medline] [Google Scholar]
18. Lee YH. Efficiency improvement in a busy radiology practice: determination of musculoskeletal magnetic resonance imaging protocol using deep-learning convolutional neural networks. J Digit Imaging 2018; 31:604–610 [Crossref] [Medline] [Google Scholar]
19. Trivedi H, Mesterhazy J, Laguna B, Vu T, Sohn JH. Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations using IBM Watson's natural language processing algorithm. J Digit Imaging 2018; 31:245–251 [Crossref] [Medline] [Google Scholar]
20. Kohli M, Dreyer KJ, Geis JR. Rethinking radiology informatics. AJR 2015; 204:716–720 [Abstract] [Google Scholar]
21. Mieloszyk RJ, Rosenbaum JI, Hall CS, Raghavan UN, Bhargava P. The financial burden of missed appointments: uncaptured revenue due to outpatient no-shows in radiology. Curr Probl Diagn Radiol 2018; 47:285–286 [Crossref] [Medline] [Google Scholar]
22. Harvey HB, Liu C, Ai J, et al. Predicting no-shows in radiology using regression modeling of data available in the electronic medical record. J Am Coll Radiol 2017; 14:1303–1309 [Crossref] [Medline] [Google Scholar]
23. Kurasawa H, Hayashi K, Fujino A, et al. Machine-learning-based prediction of a missed scheduled clinical appointment by patients with diabetes. J Diabetes Sci Technol 2016; 10:730–736 [Crossref] [Medline] [Google Scholar]
24. Torres O, Rothberg MB, Garb J, Ogunneye O, Onyema J, Higgins T. Risk factor model to predict a missed clinic appointment in an urban, academic, and underserved setting. Popul Health Manag 2015; 18:131–136 [Crossref] [Medline] [Google Scholar]
25. Sodickson DK, Manning WJ. Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays. Magn Reson Med 1997; 38:591–603 [Crossref] [Medline] [Google Scholar]
26. Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. SENSE: sensitivity encoding for fast MRI. Magn Reson Med 1999; 42:952–962 [Crossref] [Medline] [Google Scholar]
27. Griswold MA, Jakob PM, Heidemann RM, et al. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 2002; 47:1202–1210 [Crossref] [Medline] [Google Scholar]
28. Lustig M, Donoho D, Pauly JM. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn Reson Med 2007; 58:1182–1195 [Crossref] [Medline] [Google Scholar]
29. Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018; 79:3055–3071 [Crossref] [Medline] [Google Scholar]
30. Knoll F, Hammernik K, Kobler E, Pock T, Recht MP, Sodickson DK. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn Reson Med 2019; 81:116–128 [Crossref] [Medline] [Google Scholar]
31. Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging 2018; 37:491–503 [Crossref] [Medline] [Google Scholar]
32. Wang S, Su Z, Ying L. Accelerating magnetic resonance imaging via deep learning. In: Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI). Prague, Czech Republic: Institute of Electrical and Electronics Engineers, 2016:514–517 [Crossref] [Google Scholar]
33. Knoll F, Hammernik A, Garwood K, et al. Accelerated knee imaging using a deep learning based reconstruction. In: Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM). Honolulu, HI: ISMRM, 2017:645 [Google Scholar]
34. Pruessmann KP, Weiger M, Börnert P, Boesiger P. Advances in sensitivity encoding with arbitrary k-space trajectories. Magn Reson Med 2001; 46:638–651 [Crossref] [Medline] [Google Scholar]
35. Knoll F, Bredies K, Pock T, Stollberger R. Second order total generalized variation (TGV) for MRI. Magn Reson Med 2011; 65:480–491 [Crossref] [Medline] [Google Scholar]
36. Ravishankar S, Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging 2011; 30:1028–1041 [Crossref] [Medline] [Google Scholar]
37. Subhas N, Pursyko CP, Polster JM, et al. Dose reduction with dedicated CT metal artifact reduction algorithm: CT phantom study. AJR 2018; 210:593–600 [Abstract] [Google Scholar]
38. Subhas N, Polster JM, Obuchowski NA, et al. Imaging of arthroplasties: improved image quality and lesion detection with iterative metal artifact reduction, a new CT metal artifact reduction technique. AJR 2016; 207:378–385 [Abstract] [Google Scholar]
39. Cross N, DeBerry J, Ortiz D, et al. Diagnostic quality of machine learning algorithm for optimization of low dose computed tomography. cdn.ymaws.com/siim.org/resource/resmgr/siim2017/abstracts/posters-Cross.pdf. Published 2017. Accessed April 1, 2019 [Google Scholar]
40. McDonald RJ, Schwartz KM, Eckel LJ, et al. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad Radiol 2015; 22:1191–1198 [Crossref] [Medline] [Google Scholar]
41. Wang T, Iankoulski A. Intelligent tools for a productive radiologist workflow: how machine learning enriches hanging protocols. GE Healthcare website. www3.gehealthcare.com.sg/~/media/downloads/asean/healthcare_it/rdiology%20solutions/radiology%20solutions%20additional%20resources/smart_hanging_protocol_white_paper_doc1388817_july_2013_kl.pdf. Published 2013. Accessed April 1, 2019 [Google Scholar]
42. Roth HR, Yao J, Lu L, Stieger J, Burns JE, Summers RM. Detection of sclerotic spine metastases via random aggregation of deep convolutional neural network classifications. In: Yao J, Glocker B, Klinder T, Li S, eds. Recent advances in computational methods and clinical applications for spine imaging. Basel, Switzerland: Springer International Publishing, 2015:3–12 [Crossref] [Google Scholar]
43. Golan D, Donner Y, Mansi C, Jaremko J, Ramachandran M. Fully automating Graf's method for DDH diagnosis using deep convolutional neural networks. In: Carneiro G, Mateus D, Peter L, et al., eds. Deep learning and data labeling for medical applications. Basel, Switzerland: Springer, 2016:130–141 [Crossref] [Google Scholar]
44. Shun Miao, Wang ZJ, Rui Liao. A CNN regression approach for real-time 2D/3D registration. IEEE Trans Med Imaging 2016; 35:1352–1363 [Crossref] [Medline] [Google Scholar]
45. Štern D, Payer C, Lepetit V, Urschler M. Automated age estimation from hand MRI volumes using deep learning. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens, Greece: Springer, 2016:194–202 [Google Scholar]
46. Jamaludin A, Lootus M, Kadir T, et al.; Genodisc Consortium. ISSLS Prize in Bioengineering Science 2017: automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur Spine J 2017; 26:1374–1383 [Crossref] [Medline] [Google Scholar]
47. Chung SW, Han SS, Lee JW, et al. Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 2018; 89:468–473 [Crossref] [Medline] [Google Scholar]
48. Olczak J, Fahlberg N, Maki A, et al. Artificial intelligence for analyzing orthopedic trauma radio-graphs. Acta Orthop 2017; 88:581–586 [Crossref] [Medline] [Google Scholar]
49. Roth HR, Wang Y, Yao J, Lu L, Burns JE, Summers RM. Deep convolutional networks for automated detection of posterior-element fractures on spine CT. arXiv website. arxiv.org/abs/1602.00020. Submitted January 29, 2016. Accessed April 4, 2019 [Google Scholar]
50. Burns JE, Yao J, Summers RM. Vertebral body compression fractures and bone density: automated detection and classification on CT images. Radiology 2017; 284:788–797 [Crossref] [Medline] [Google Scholar]
51. Xue Y, Zhang R, Deng Y, Chen K, Jiang T. A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis. PLoS One 2017; 12:e0178992 [Crossref] [Medline] [Google Scholar]
52. Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S. Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 2018; 8:1727 [Crossref] [Medline] [Google Scholar]
53. Antony J, McGuinness K, Connor NEO, Moran K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. arXiv website. arxiv.org/abs/1602.00020. Submitted September 8, 2016. Accessed April 4, 2019 [Google Scholar]
54. Antony J, McGuinness K, Moran K, O'Connor NE. Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. New York, NY: Springer, 2017 [Crossref] [Google Scholar]
55. Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 2018; 287:313–322 [Crossref] [Medline] [Google Scholar]
56. Lee H, Tajmir S, Lee J, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging 2017; 30:427–441 [Crossref] [Medline] [Google Scholar]
57. Kim JR, Shim WH, Yoon HM, et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR 2017; 209:1374–1380 [Abstract] [Google Scholar]
58. Spampinato C, Palazzo S, Giordano D, Aldinucci M, Leonardi R. Deep learning for automated skeletal bone age assessment in X-ray images. Med Image Anal 2017; 36:41–51 [Crossref] [Medline] [Google Scholar]
59. Tajmir SH, Lee H, Shailam R, et al. Artificial intelligence-assisted interpretation of bone age radiographs improves accuracy and decreases variability. Skeletal Radiol 2019; 48:275–283 [Crossref] [Medline] [Google Scholar]
60. Yang CC, Nagarajan MB, Huber MB, et al. Improving bone strength prediction in human proximal femur specimens through geometrical characterization of trabecular bone microarchitecture and support vector regression. J Electron Imaging 2014; 23:013013 [Crossref] [Medline] [Google Scholar]
61. Huber MB, Lancianese SL, Nagarajan MB, Ikpot IZ, Lerner AL, Wismuller A. Prediction of biomechanical properties of trabecular bone in MR images with geometric features and support vector regression. IEEE Trans Biomed Eng 2011; 58:1820–1826 [Crossref] [Medline] [Google Scholar]
62. Sharma GB, Robertson DD, Laney DA, Gambello MJ, Terk M. Machine learning based analytics of micro-MRI trabecular bone microarchitecture and texture in type 1 Gaucher disease. J Biomech 2016; 49:1961–1968 [Crossref] [Medline] [Google Scholar]
63. Ferizi U, Besser H, Hysi P, et al. Artificial intelligence applied to osteoporosis: a performance comparison of machine learning algorithms in predicting fragility fractures from MRI data. J Magn Reson Imaging 2018; 49:1029–1038 [Crossref] [Medline] [Google Scholar]
64. Pedoia V, Majumdar S, Link TM. Segmentation of joint and musculoskeletal tissue in the study of arthritis. MAGMA 2016; 29:207–221 [Crossref] [Medline] [Google Scholar]
65. Gassman EE, Powell SM, Kallemeyn NA, et al. Automated bony region identification using artificial neural networks: reliability and validation measurements. Skeletal Radiol 2008; 37:313–319 [Crossref] [Medline] [Google Scholar]
66. Zhou Z, Zhao G, Kijowski R, Liu F. Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med 2018; 80:2759–2770 [Crossref] [Medline] [Google Scholar]
67. Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med 2018; 79:2379–2391 [Crossref] [Medline] [Google Scholar]
68. Norman B, Pedoia V, Majumdar S. Use of 2D UNet convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology 2018; 288:177–185 [Crossref] [Medline] [Google Scholar]
69. Deniz CM, Xiang S, Hallyburton RS, et al. Segmentation of the proximal femur from MR images using deep convolutional neural networks. Sci Rep 2018; 8:16485 [Crossref] [Medline] [Google Scholar]
70. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology 2016; 278:563–577 [Crossref] [Medline] [Google Scholar]
71. McBee MP, Awan OA, Colucci AT, et al. Deep learning in radiology. Acad Radiol 2018; 25:1472–1480 [Crossref] [Medline] [Google Scholar]
72. Yin P, Mao N, Zhao C, et al. Comparison of radiomics machine-learning classifiers and feature selection for differentiation of sacral chordoma and sacral giant cell tumour based on 3D computed tomography features. Eur Radiol 2019; 29:1841–1847 [Crossref] [Medline] [Google Scholar]
Address correspondence to S. Gyftopoulos (Soterios.Gyftopoulos@nyumc.org).



Read More: https://www.ajronline.org/doi/abs/10.2214/AJR.19.21117