AI – is four subfields: machine learning, natural language processing, artificial neural networks, computer vision
Hashimoto 2018 [Daniel A et al. “Artificial Intelligence in Surgery: Promises and Perils.” Annals of surgery vol. 268,1 (2018): 70-76. doi:10.1097/SLA.0000000000002693]
Artificial intelligence (AI) can be loosely defined as the study of algorithms that give machines the ability to reason and perform cognitive functions such as problem solving, object and word recognition, and decision-making.1 Previously thought to be science fiction, AI has increasingly become the topic of both popular and academic literature as years of research have finally built to thresholds of knowledge that have rapidly generated practical applications, such as International Business Machine’s (Armonk, NY, USA) Watson and Tesla’s (Palo Alto, CA, USA) autopilot.2 Stories of man-versus-machine, such as that of John Henry working to death to outperform the steam-powered hammer3 , demonstrate how machines have long been feared yet ultimately both accepted and eagerly anticipated. Society proceeded to integrate simple machines into human workflow, and the resulting Industrial Revolution yielded a massive shift in productivity and quality of life. Similarly, AI has inspired awe and struck fear in people who now face a technology that can not only outperform but also potentially outthink its creators. With the Information Age, a shift in workflow and productivity similar to that of the Industrial Revolution has begun; and surgery stands to gain from the current explosion of information technology. However, as with many emerging technologies, the true promise of AI can be lost in its hype.4 It is, therefore, important for surgeons to have a foundation of knowledge of AI to understand how it may impact healthcare and to consider ways in which they may interact with this technology. This review provides an introduction to AI by highlighting four core subfields – 1) machine learning, 2) natural language processing, 3) artificial neural networks, 4) computer vision – their limitations, and future implications for surgeons. Subfields in AI AI’s roots are found across multiple fields, including robotics, philosophy, psychology, linguistics, and statistics.5 Major advances in computer science, such as improvements in processing speed and power, have functioned as a catalyst to allow for the base technologies required for the advent of AI. The growing popularity of AI across many different industries has attracted venture capital investment up to $5 billion in 2016 alone.6 Much of the current attention on AI has focused on the four core subfields introduced below. Machine Learning Machine learning (ML) enables machines to learn and make predictions by recognizing patterns. Traditional computer programs are explicitly programmed with a desired behavior (e.g. when the user clicks an icon, a new program opens). ML allows a computer to utilize partial labelling of the data (supervised learning) or the structure detected in the data itself (unsupervised learning) to explain or make predictions about the data without explicit programming (Figure 1). Supervised learning is useful for training a ML algorithm to predict a known result or outcome while unsupervised learning is useful in searching for patterns within data.7 A third category within machine learning is reinforcement learning, where a program attempts to accomplish a task (e.g. driving a car, inferring medical decisions) while learning from its own successes and mistakes.8 One can conceptualize reinforcement learning as the computer science equivalent of operant conditioning9 and is useful for automated tuning of predictions or actions, such as controlling an artificial pancreas system to fine tune the measurement and delivery of insulin to diabetic patients.10 ML is particularly useful for identifying subtle patterns in large datasets – patterns that may be imperceptible to humans performing manual analyses – by employing techniques that allow for more indirect and complex non-linear relationships and multivariate effects than conventional statistical analysis.11,12 ML has outperformed logistic regression for prediction of surgical site infections (SSI) by building non-linear models that incorporate multiple data sources, including diagnoses, treatments, and laboratory values.13 Furthermore, multiple algorithms working together (ensemble ML) can be used to calculate predictions at accuracy levels thought to be unattainable with conventional statistics.14 For example, by analyzing patterns of diagnostic and therapeutic data (including surgical resection) in the Surveillance, Epidemiology and End Results (SEER) cancer registry and comparing data to Medicare claims, ensemble ML with random forests, neural networks, and lasso regression was able to predict patient lung cancer staging by using International Classification of Diseases (ICD)-9 claims data alone with 93% sensitivity, 92% specificity, and 93% accuracy, outperforming a decision tree approach based on clinical guidelines alone (53% sensitivity, 89% specificity, 72% accuracy).15 Natural Language Processing Natural language processing (NLP) is a subfield that emphasizes building a computer’s ability to understand human language and is crucial for large scale analyses of content such as electronic medical record (EMR) data, especially physicians’ narrative documentation. To achieve human-level understanding of language, successful NLP systems must expand beyond simple word recognition to incorporate semantics and syntax into their analyses.16 Rather than relying on codified classifications such as ICD codes, NLP enables machines to infer meaning and sentiment from unstructured data (e.g. prose written in the history of present illness or in a physician’s assessment and plan). NLP allows clinicians to write more naturally rather than having to input specific text sequences or select from menus to allow a computer to recognize the data. NLP has been utilized for large scale database analysis of the EMR to detect adverse events and postoperative complications from physician documentation17, 18, and many EMR systems now incorporate NLP – for example, to achieve automated claims coding – into their underlying software architecture to improve workflow or billing.19 In surgical patients, NLP has been used to automatically comb through EMRs to identify words and phrases in operative reports and progress notes that predicted anastomotic leak after colorectal resections. Many of its predictions reflected simple clinical knowledge that a surgeon would have (e.g. operation type and difficulty), but the algorithm was also able to adjust predictive weights of phrases describing patients (e.g. irritated, tired) relative to the postoperative day to achieve predictions of leak with a sensitivity of 100% and specificity of 72%.20 The ability of algorithms to self-correct can increase the utility of their predictions as datasets grow to become more representative of a patient population. Artificial Neural Networks Artificial neural networks, a subfield of ML, are inspired by biological nervous systems and have become of paramount importance in many AI applications. Neural networks process signals in layers of simple computational units (neurons); connections between neurons are then parameterized via weights that change as the network learns different input-output maps corresponding to tasks such as pattern/image recognition and data classification (Figure 2).7 Deep learning networks are neural networks comprised of many layers and are able to learn more complex and subtle patterns than simple one or two-layer neural networks.21 Clinically, ANNs have significantly outperformed more traditional risk prediction approaches. For example, an ANN’s sensitivity (89%) and specificity (96%) outperformed APACHE II sensitivity (80%) and specificity (85%) for prediction of pancreatitis severity six hours after admission.22 By using clinical variables such as patient history, medications, blood pressure, and length of stay, ANNs, in combination with other ML approaches, have yielded predictions of in-hospital mortality after open abdominal aortic aneurysm repair with sensitivity of 87%, specificity of 96.1%, and accuracy of 95.4%.23 Computer Vision Computer vision describes machine understanding of images and videos, and significant advances have resulted in machines achieving human-level capabilities in areas such as object and scene recognition.24 Important healthcare-related work in computer vision includes image acquisition and interpretation in axial imaging with applications including computer-aided diagnosis, image-guided surgery, and virtual colonoscopy.25 Initially influenced by statistical signal processing, the field has recently shifted significantly towards more data-intensive ML approaches, such as neural networks,26 with adaptation into new applications. Utilizing ML approaches, current work in computer vision is focusing on higher level concepts such as image-based analysis of patient cohorts, longitudinal studies, and inference of more subtle conditions such as decision-making in surgery. For example, real-time analysis of laparoscopic video has yielded 92.8% accuracy in automated identification of the steps of a sleeve gastrectomy and noted missing or unexpected steps.27 With one minute of high-definition surgical video estimated to contain 25 times the amount of data found in a high-resolution computed tomography image28, video could contain a wealth of actionable data.29, 30 Thus, while predictive video analysis is in its infancy, such work provides proofof-concept that AI can be leveraged to process massive amounts of surgical data to identify or predict adverse events in real-time for intraoperative clinical decision support (Figure 3).