人工智能在神經(jīng)病理學(xué)中的應(yīng)用：基于深度學(xué)習(xí)的tau蛋白病變評估

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

HHSPublicAccess

Authormanuscript

LabInvest.Authormanuscript;availableinPMC2020November24.

Publishedinfinaleditedformas:

LabInvest.2019July;99(7):1019–1029.doi:10.1038/s41374-019-0202-4.

Artificialintelligenceinneuropathology:deeplearning-basedassessmentoftauopathy

MaximSignaevsky1,2,3,MarcelPrastawa1,4,KurtFarrell1,2,3,NabilTabish1,2,3,ElenaBaldwin1,2,3,NataliaHan1,2,3,MeganA.Iida1,2,3,JohnKoll1,4,ClareBryce1,2,3,DushyantPurohit1,2,5,VahramHaroutunian5,6,AnnC.McKee7,8,9,10,11,ThorD.Stein8,9,10,11,Charles

L.WhiteIII12,JamieWalker12,TimothyE.Richardson12,RussellHanson1,2,3,MichaelJ.Donovan1,4,CarlosCordon-Cardo1,4,JackZeineh1,4,GerardoFernandez1,4,JohnF.Crary1,2,3

1DepartmentofPathology,IcahnSchoolofMedicineatMountSinai,NewYork,NY10029,USA

2DepartmentofNeuroscience,IcahnSchoolofMedicineatMountSinai,NewYork,NY10029,USA

3RonaldM.LoebCenterforAlzheimer’sDisease,IcahnSchoolofMedicineatMountSinai,NewYork,NY10029,USA

4CenterforComputationalandSystemsPathology,IcahnSchoolofMedicineatMountSinai,NewYork,NY10025,USA

5DepartmentsofPsychiatryandNeuroscience,IcahnSchoolofMedicineatMountSinai,NewYork,NY10029,USA

6J.JamesPetersVAMedicalCenter,Bronx,NY,USA

7DepartmentofNeurology,BostonUniversitySchoolofMedicine,Boston,MA02118,USA

8DepartmentofPathology,BostonUniversitySchoolofMedicine,Boston,MA02118,USA

9Alzheimer’sDiseaseCenter,CTEProgram,BostonUniversitySchoolofMedicine,Boston,MA02118,USA

10MentalIllnessResearch,EducationandClinicalCenter,JamesJ.PetersVABostonHealthcareSystem,Boston,MA02130,USA

11DepartmentofVeteranAffairsMedicalCenter,Bedford,MA01730,USA

12NeuropathologyLaboratory,DepartmentofPathology,UTSouthwesternMedicalCenter,Dallas,TX75390,USA

Abstract

Accumulationofabnormaltauinneurofibrillarytangles(NFT)occursinAlzheimerdisease(AD)andaspectrumoftauopathies.Thesetauopathieshavediverseandoverlappingmorphological

JohnF.Crary,

john.crary@.

ConflictofinterestTheauthorsdeclarethattheyhavenoconflictofinterest.

Publisher’snote:SpringerNatureremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.

LabInvest.Authormanuscript;availableinPMC2020November24.

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Signaevskyetal. Page2

phenotypesthatobscureclassificationandquantitativeassessments.Recently,powerfulmachinelearning-basedapproacheshaveemerged,allowingtherecognitionandquantificationofpathologicalchangesfromdigitalimages.Here,weapplieddeeplearningtotheneuropathologicalassessmentofNFTinpostmortemhumanbraintissuetodevelopaclassifiercapableofrecognizingandquantifyingtauburden.Thehistopathologicalmaterialwasderivedfrom22autopsybrainsfrompatientswithtauopathies.Weusedacustomweb-basedinformaticsplatformintegratedwithanin-houseinformationmanagementsystemtomanagewholeslideimages(WSI)andhumanexpertannotationsasgroundtruth.Weutilizedfullyannotatedregionstotrainadeeplearningfullyconvolutionalneuralnetwork(FCN)implementedinPyTorchagainstthehumanexpertannotations.WefoundthatthedeeplearningframeworkiscapableofidentifyingandquantifyingNFTwitharangeofstainingintensitiesanddiversemorphologies.WithourFCNmodel,weachievedhighprecisionandrecallinnaiveWSIsemanticsegmentation,correctlyidentifyingtangleobjectsusingaSegNetmodeltrainedfor200epochs.OurFCNisefficientandwellsuitedforthepracticalapplicationofWSIswithaverageprocessingtimesof45minperWSIperGPU,enablingreliableandreproduciblelarge-scaledetectionoftangles.Wemeasuredperformanceontestdataof50pre-annotatedregionsoneightnaiveWSIacrossvarioustauopathies,resultingintherecall,precision,andanF1scoreof0.92,0.72,and0.81,respectively.MachinelearningisausefultoolforcomplexpathologicalassessmentofADandothertauopathies.Usingdeeplearningclassifiers,wehavethepotentialtointegratecell-andregion-specificannotationswithclinical,genetic,andmoleculardata,providingunbiaseddataforclinicopathologicalcorrelationsthatwillenhanceourknowledgeoftheneurodegeneration.

Introduction

Tau-relatedneurodegenerativedisorders,thetauopathies,compriseaheterogeneousgroupofdisorderswithaclinicalspectrumthatincludesprimarymotorsymptoms,movementdisorder,psychiatricdysfunction,andcognitiveimpairment[1].Histomorphologically,tauopathiesarecharacterizedbyintracellulardepositionofhyperphosphorylatedtauprotein.Variousisoformcompositions,morphology,andanatomicaldistributionsofintracellulartaurepresentdistinctdiagnosticfeaturesoftauopathies[1–3].Howpathologicaltaucausesneuronaldysfunctionanddegenerationisunclear.Severalmechanismshavebeenimplicated,includingbothgeneticandenvironmentalriskfactors,butmostcasesareidiopathic[1,3–5].Sporadictauopathies,suchasthevastmajorityofAlzheimerdisease(AD)andprogressivesupranuclearpalsy(PSP)cases,areassociatedwithcommongeneticriskalleles[1,3].Rarehighlypenetrantmutationsinthemicrotubule-associatedproteintaugeneareassociatedwithsomeformsoffrontotemporallobardegeneration[6].

Environmentalfactors,suchastraumaticbraininjuryinthecaseofchronictraumaticencephalopathy(CTE)orputativeneurotoxins,havealsobeenimplicated[7,8].

Pathologicalchangesintaumetabolismandpost-translationalmodificationsresultintheaccumulationoftoxicformsofmisfoldedtauaggregatesinneuronsandglialcellsinvariousbrainregions.Thesemisfoldedaggregatesareassociatedwithlossoffunctionandultimatelycelldeath[1,2].

Pathologicaltauformsinclusionsinneuronsandgliawithhistomorphologicallydistinguishablefeatures.Inneurons,thesetaketheformoftheclassicalflame-shaped

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Signaevskyetal. Page3

intracellularneurofibrillarytangles(NFTs),granularpre-NFTs,extracellular“ghost”tangles,ringtangles,andglobosetangles,amongothers[9].Inglia,thereisaspectrumofcharacteristichistomorphologicalformsthatarecommonlyassociatedwithspecificdiseases,includingglialplaquesofcorticobasaldegeneration,tuftedastrocytesofPSP,globularastroglialinclusionsinglobularglialtauopathy,ramifiedastrocytesofPickdisease,andthorn-shapedastrocytesaswellasgranularfuzzyastrocytesofaging-relatedtauastrogliopathy[9–11].Onerecentlyproposedclassificationschemecodifiessevenprimarytauopathies,andtwosecondarytauopathiesundertheumbrellaofneurodegenerativediseases,eachwithauniqueconstellationofregionalvulnerabilityandhistomorphologyoftauaggregatesthatdefinethem[1,2].Pathologicalaccumulationofhyperphosphorylatedtauisalsodescribedinvariousinfectious/post-infectious,metabolic,genetic/chromosomal,neoplastic/hamartomatous,andmyopathicdiseases[12].Giventhecomplexityandmorphologicaloverlap,diagnosingthesediseasesisachallengeforneuropathologists,andcommandsahighdegreeofexpertise.

Microscopicanalysisofstainedpostmortemsectionsbyatrainedexpertremainstheonlymodalityofconfirmatorydiagnosisoftauopathies.Despitethecontinuouseffortandimprovementsinthefield,theanalysesrequiredfordefinitivediagnosisandsubtypingofneurodegenerativediseasesremainhighlytime-andcost-consumingandaresubjecttoasubstantialdegreeofinter-andintra-observervariability,thuslackingoverallaccuracyandprecision.ThegoldstandardforhistomorphologicalassessmentoftauburdenandprogressioninAlzheimer’sdiseaseistheBraakstagingsystem,whichfocusesonthehierarchicalsequenceoftauaccumulation,butnotaquantitativemeasurementoftauburden,althoughdistributionandqualitativeNFTandthreaddensityarecorrelatedinthisstagingsystem[13].Despitethislimitation,theBraakstagingsystemhasbeenwidelyacceptedandadoptedfordecadesforitssimplicityandrobustness.Recentinterestindifferentialsemi-quantitativeassessmentoftauburdeninADisexemplifiedintheworkofJellinger[14].

Further,variousstagesofintracellularpathologictauaccumulationaredescribed(e.g.,pre-tangles,matureNFTs,andso-called“ghost”tangles—theremnantsofthetaufibrillaryscaffoldafterneuronalcelldeath;Fig.1).TheBraakstagingapproachdoesnotaddressthesefeatures,andthusinherentlylacksgranularityandquantification.Atthesametime,thefieldofdiagnosticneuropathologyisfacingchallengesrelatedtotheoveralllackofaccuracy,demandedbytheever-evolvingresearchandhealthcarestandards,anddiscrepancieswithclinicopathologicalcorrelations,witharecognizedneedtoaddresstheseissues[15].

Recently,therehasbeenanincreasinginterestindevelopingcomputationalmethodstoassistthepathologistinhistologicalanalysisviadigitalmicroscopicwholeslideimages(WSI).

Thisisprimarilyintendedtoreducethehumanerrorrateandbringaboutuniformityandaccuracyinpathologicaldiagnosis[16].Oneoftheapproachesthathasbeenanticipatedandsoughtafterfornearlyhalfacenturyisartificialintelligence(AI)[17,18].ThemostadvancedAI,calleddeeplearning(DL),isnowusedforcomplextaskssuchasspeechrecognition,languagetranslation,andimagerecognitionandinterpretation[19–21].Litjensetal.provideacomprehensivesurveyofpublishedstudiesontheuseofAI/DLinmedicalimageanalysisincludingWSIinpathology[17].Althoughmachinelearning-basedmethodshavehadlimitedapplicationindiagnosticpathologytodate,duetothevariabilityof

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Signaevskyetal. Page4

laboratorystandardsandoutcomes,andlackofreliablecomputer-backedplatforms,advanceshavebeenmaderecently.Therelevanceandpotentialofautomatedclassificationalgorithmsinsurgicalpathologyareexemplifiedbyitsapplicationtothehistologicgradingandprogressionofbreastandprostatecancer[17,22,23].Theseendeavorspaveawaytowardincreaseduseofmachinelearningforimprovingstratification,characterization,andquantificationformanyotherdiseaseprocesses,includingtheneuropathologicalassessmentoftauopathiesandADcohorts.Todate,nodatasetsderivedfromtheapplicationofmachine-basedlearningtoneurodegenerativediseaseareavailable.

WeaimedtodevelopandtestanovelDLalgorithmusingconvolutionalneuralnetworks

[20]thatwouldbeabletorecognize,classify,andquantifydiagnosticelementsoftauopathiesonWSIofpostmortemhumanbraintissuespecimensfrompatientswithtau-associatedneurodegenerativeconditionsinordertobetterstratifypatientsforclinicalandothercorrelativestudies(Fig.2).Inthisstudy,wefocusedonthedevelopment,validation,andtestingoftheDLalgorithmsforrecognitionandquantificationofNFTinanarrayoftauopathies.Thiswillallowustoapplythesetrainednetworksforlargerdisease-specificcohortsandtogeneratequantitativedataforclinicopathologicalcorrelations,aswellasformolecularandgeneticstudies,andenablefurtherdiagnosticandtherapeuticstrategies.

Materialsandmethods

Casematerial

De-identifiedautopsybraintissueswereobtainedfrom22representativeindividualswithAD,primaryage-relatedtauopathy(PART),PSP,andCTE[24](Table1).Thiscohortwasaconveniencesampleselectedbytheinvestigators.Weusedthefollowingselectioncriteria:

(i)clinical/pathological:well-characterizedclinicalcase,representativeofavarietyofpathognomonicdiagnostichistomorphologicalfeatures,andwithminimalorabsentneuropathologicalcomorbidities;(ii)technical:adequatelystainedtissuewithminimalornoartifacts.

Immunohistochemistry

Weusedstandardhistologicalcoronalsectionsfromformalin-fixedparaffin-embedded(FFPE)postmortembraintissue,representinghippocampalformationanddorsolateralprefrontalcortex.ForPARTandADcases,theimmunohistochemistry(IHC)ofallcaseswasperformedattheUniversityofTexasSouthwestern(UTSW)usinganti-phosphorylatedtauantibodies(AT8,Invitrogen,Waltham,MA)at1:200dilutionusingaLeicaBondIIIautomatedimmunostainer(LeicaMicrosystems,BuffaloGrove,IL).PSPandCTEcaseswereimmunostainedattheNeuropathologyResearchCoreatMountSinaiwithanti-phosphorylatedtauantibodies(AT8,Invitrogen)at1:2000dilutionusingaVentanaautostainer(RocheDiagnostics,Rotkreuz,Switzerland).

Slidedigitization

AllsectionsweredigitizedtoobtaindigitalWSI.ForPSPandCTE,WSIwereacquiredusingtheUltraFastScannerDigitalPathologySlideScanner(Philips,Amsterdam,Netherlands),whichscanshistologicalsamplesmountedonstandardglassslidesatx40

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Signaevskyetal. Page5

magnification(0.25μm/pixel)andsavesthemintheproprietaryiSyntaxformat.ForPARTandADcases,allslideswerescannedusinganAperioCSimagescanner(LeicaMicrosystems)atx20magnification(0.50μm/pixel)andsavedin.svsformat.AllimagesinproprietaryformatswerethenconvertedintoaGeoTIFFandstoredontheserverbehindthehospitalfirewallforinteractivedisplayovertheintranet.

Pathologicalannotations

WSIwereuploadedtothePreciseInformaticsPlatform(PIP)developedbytheCenterforComputationalandSystemsPathologyatMountSinai(MP,JK,JZ,andGF),whichallowsforthemanagementofthousandsofimageswithpathologistannotations.AuthorspreviouslyhaveappliedmachinelearningtoprostatecancerforGleasongrading[23,25],anditiscurrentlybeingusedinourCLIA-approvedlaboratory.Inaddition,PIPenablesgraphicsprocessingunit(GPU)-acceleratedDLforrapidvalidationandvisualizationofhowDLclassifiersperformindifferentscenarios(brainregions,celltypes,andstaining).

AnnotationsweregeneratedusingthePIPcollaborativeweb-baseduserinterfaceforoutlining(Fig.3).AnNFTwasoperationalizedasanobject,i.e.“foreground”,withcytoplasmicfinegranular,coarsegranular,orfibrillary/condensedAT8immunopositivitymorphologicallyconsistentwithaneuronbasedonthehistologicalcontext.Inaddition,extracellularAT8-positivestructuresmorphologicallyconsistentwiththeneuronalsomatodendriticcompartmentwerecountedasghosttangles.Partialneuriteslackingconnectiontothesomaorhillockwereexcluded.OtherAT8-positivestructuresincludingneuropilthreads,neuropilgranules/grains,andambiguousnon-neuronalphospho-taustainingwerecategorizedas“background”.Thetotalnumberof22WSIwasdividedinto14fortrainingandvalidation(modelselection),with8reservedasatestsetforperformanceevaluation.

Weconductedaconcordancestudytoassesstheinter-raterreliabilityusingacustominterfacewithinthePIPplatform.Atotalof471uniquepatchesofmixedhumanexpert-annotatedgroundtruthNFTsandAI-detectedfalsepositiveswereindependentlyassessedbythreeneuropathologists(MS,JFC,orCB)andcomparedusingaFleiss’kappastatistic.

Fullyconvolutionalnetwork(FCN)trainingandmodelselection

ThetrainingdatasetconsistedofWSIofsectionsfrom14subjects(Table1).Intotal,178representativerectangularregionsofinterests(ROI)wereselectedbytheinvestigatorsforanalysis.ThecriteriaforROIwereasfollows:(1)arepresentativecorticalareawithanadequateIHCofdiagnosticquality,(2)arepresentativevarietyofrecognizabledistincthistologicalAT8-stainedelements,and(3)intacttissuewithoutdetachmentorlargetissuefolds.AllNFTformswerecomputedtogether.ThetotalnumberofAT8-positiveNFTsofvariousmorphologiesrangingfrompre-tanglestomatureNFTsandghosttanglesusedforfullyconvolutionalneuralnetworktrainingandmodelselectionwas2221.Wefurtherextractedimagepatchesofsize512×512pixelsatx20bypartitioningtheROIs.Thetotalnumberofpatcheswas3177,comprising2414fromAperioscannedPARTandADWSIs,aswellas763fromPhilipsscannedCTEandPSPWSIs(Fig.4).Wefurtherassigned200patchesfromthisdatasettothevalidationset(formodelselection),withtheremainderusedfortraininganeuralnetworkclassifier.

Signaevskyetal. Page6

Fordeepconvolutionalneuralnetworkgeneration,weusedamodifiedversionofthefullyconvolutionalSegNetarchitecture(Fig.5)[26].Weusedthreespatialscales(numberofblockscontainingmultipleconvolutionlayersfollowedbyapoolinglayer)inthenetworktomodelthevisualcontextforNFT.Weightparametersfortheneuralnetworkaretheminimaforthepixel-wisebinarycrossentropyloss.Specifically,givenasetoftrainingexampleimagesIwithassociatedgroundtruthlabelsy,theFCNwithweightswgeneratespixelprobabilityateachlocationxforNFTobjectsaspj(x)=FCN(Ij(x),w),yieldingthe

followinglossfunction:

NL(w)=?∑

j=1

?

?∑··yj(x)logpi(x)+1?yj(x)log1?pi(x)

x

Thisdifferentiablelossfunctionisminimizedusingstochasticgradientdescent,whichperformsgradientupdatesonsmallbatchesofimages.Asetofgradientupdateiterationsthatutilizethecompletesetoftrainingimages,comprisingmultiplesmallbatches,iscalledanepoch.EachupdateiterationcanbecomputedefficientlyinparallelusingcommodityGPUhardware.WeusedthePyTorchsoftwarepackage(

)forbuildingourneuralnetworkmodel[27].

FCNtesting

WeappliedthetrainedSegNettoasetofeightnaiveWSIs,capturingarangeofscannerandstainingvariabilities.Forthese,weusedoneWSIforAD,twoforPART,threeforPSP,andtwoforCTE(Table1).ThetotalnumberoffullyannotatedrectangularROIoneightnaiveslidesrepresentingvariousnosologieswas50.ThetotalnumberofAT8positivestructuresamongthevariousmorphologieswas618.Positivefeatureswereusedtointerrogatenetworkperformance.

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Results

First,wecomputedtheoptimalweightparametersforNFTdetectionin200epochs.Networkweightswereupdatedtoreducetrainingloss,and,ateachupdate,wecomputedthelossonvalidationdata,whichisseparatefromtrainingdata,toensurethatperformancebetweenthetwodoesnotdiverge.Incaseswheretraininglossisreducedandvalidationlossisincreased,theestimatednetworkweightswillresultinamodeloverfittedtothetrainingset.Itwillperformwellontrainingdatabutwillhavesuboptimalperformanceonnoveldata.Weperformeddataaugmentationateachepochonarandomsubsetoftrainingsamples,whichincludescontrastshiftandgeometricchanges(flipsandrotations).Thisaugmentationstepprovidesaricherexampleforournetworkandreducesthelikelihoodofoverfitting.ThetrainingprocessforadeepneuralnetworkfordetectingNFTbyoptimizationofthecross-entropylossfunctionisshown(Fig.6).Optimizationwasperformedusingstochasticgradientdescentonthetrainingdata;selectingthemodelthatminimizestheseparatevalidationdataensuresthatthenetworkmodelcanbegeneralizedandappliedtounseenWSI.Theseresultsindicatethatournetworkweightsareoptimalandarenotoverfittedtothetrainingdata.

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Signaevskyetal. Page7

Thenetworkachieveshighsensitivityforbothvalidationandtestdata,withalowerprecisiononthetestsetcomparedtothevalidationsetasourcurrentnetworkgeneratesmorefalsepositivesinthenaivetestWSIs(Figs.7,8).Onvalidation,wehaveachievedrecall,precision,andF1scoreof0.91,0.80,and0.86,respectively.Ontesting,weachievedoverallrecall,precision,andF1scoreof0.92,0.72,and0.81,respectively.TheoverallFCNperformancewashigherinthehigh-tau-burdenAD/PARTcohortcomparedtothelow-to-moderatetauburdenPSP/CTEcohort.TheFCNwastrainedusingdatawhereAD/PARTishigherinproportion(Table2).TheFleiss’kappaforinter-raterreliabilitybetweenneuropathologistsdeterminedonacollectionofpatchesconsistingofamixofnetwork-definedfalsepositivesandtruepositiveswas0.78(p-value<0.0001)(Table3).

WetrainedandtestedourFCNonvariousstainingconditions.Thetruepositive(TP),falsepositive(FP),andfalsenegative(FN)valuesinhighbackgroundWSI(AperioscannedADandPARTcasesfromUTSW)were329,98,and14,respectively.TheTP,FP,andFNvaluesinlowbackgroundWSI(PhilipsscannedPSPandCTEcasesfromMSSM)were244,122,and45,respectively.OverallFCNperformancerepresentedwithanF1scorewashigherinthehigh-backgroundhigh-tau-burdenAD/PARTcohort(0.85)comparedtothelow-backgroundandlow-to-moderatetau-burdenPSP/CTEcohort(0.75)(Table4).

Theobjectdetectiontimeforasinglewholeslideimagerangedfrom10minto2h(averaging45min)usingoneNVIDIATitanXpGPU,withperformancedependingonthedigitalscanresolutionandmagnification.FullyautomaticdetectionofNFTsatthisperformancelevelwillenablelarge-scaleanalysisofWSI.

Discussion

Inthisstudy,wepresentanovelmachinelearning-basedmethodusingautomatedqualitativeandquantitativeassessmentofNFTonIHC-stainedpreparations.Thevalueofareproducible,rapid,andunbiasedapproachtoaugmentlabor-intensivemanualcountingofhistopathologicalfeaturesiswellrecognized.ImplementationofDLisacompellingcomputationaltoolthatcanaddressthisgap.DLenablestherapiddevelopmentofnewalgorithmsandtoolsbutrequiresthecreationofcomputationalinfrastructureandlargeneuropathologicaldatasetscontainingrichlyvariedhigh-qualityannotations.ThisisgreatlyfacilitatedbyacollaborativeannotationplatformthatutilizespowerfulGPUhardwareandrapidfeedbackfromcomputationalalgorithms.Wehaveachievedasignificantmilestonebydevelopingaweb-basedplatformfordatamanagement,visualexploration,objectoutlining,multi-userreview,andevaluationofDLalgorithmresultsinWSI.OurNFTclassifiercurrentlytakesanaverageof45minto1htocomputationallyidentifyandcountNFTonanentireWSI,illustratingthefeasibilityofapplyingthisapproachtolargedatasets.Toourknowledge,thisisamongthefirstframeworksavailableforbuildingandevaluatingDLalgorithmsusinglarge-scaleimagedatainneuropathology.

Ourlong-termgoalistodevelopacomprehensiveplatformthatcanbeutilizedacrosscontexts(e.g.,basicresearchlaboratories,brainbanks,andclinicalneuropathologylaboratories)withvariabilityinsamplingprotocols,tissuesectionquality,stainingmethodology,andpathologicalfeatures.Hence,inourcurrentstudy,severalstepswere

AuthorManuscript

AuthorManuscript

AuthorManuscript

AuthorManuscript

Signaevskyetal. Page8

takentoincreasetheadaptabilityoftheneuralnetwork.Weusedmultiplebrainregions,aspectrumofdifferenttau-relateddiseases,avarietyofstainingconditions,andimagesacquiredontwodifferentslide-scanningplatforms.Thesestepshavelaidthegroundworktoprovideahighlyadaptableandrobusttangleclassifierforuseonimmunohistochemicallystainedsectionsthatcanbereadilyintegratedintoexistingclinicalneuropathologyandresearch.

DLalgorithmsarebasedonconceptsdevelopedinthe1940sandhavestartedbeingusedinmedicalimagingonlyrecently.UseofthesealgorithmsarebecomingpracticalduetothedevelopmentofGPUhardwareandtheyhavebeensuccessfullyappliedtosolvevariousimageclassification,detection,andsegmentationtasks[17,20].SeveralgroupsareapplyingsimilarAItechnologiestohistopathologyandhavecomparedthemtohumanexperts.Forexample,indermatologyandophthalmology,DLalgorithmswereabletooutperformahumanexpert[28,29].TherecentBreAstCancerHistologyimagesGrandChallengedemonstratedthatAIisabletopushforwardthestate-of-the-artaccuracy(87%)[22].

AnotherstudybyEstevaetal.utilizedapre-trainedGoogleNetInceptionv3CNNwith

~1.28millionpubliclyavailableimagesofskincancer[28].Thechallenge,however,liesintheacquisitionofasufficientnumberofrelevantgroundtruthexpertannotations.Further,evenwhenabodyofdataisannotatedbydomainexperts,labelnoisefromintra-andinter-observervariabilitycallingpresentsasignificantlimitingfactorindevelopingthealgorithms,andthereforearigorousqualitycontrolandexpertconsensusareneededfortrainingsets.Thus,publishedstudiesdemonstratethepromiseofAIinaidinganexpertinmakingmoreefficientdiagnoses.

OurSegNetfullyconvolutionalneuralnetworkhasreachedpracticallyusefullevelsofperformancebutcouldbeimproved.GiventhatwefocusedonNFT,performancewillbeenhancedwithlargerandmorevariedannotatedtrainingdatathatcaptureawiderrangeofneuropathologies(e.g.,amyloidplaques,Lewybodies,cerebrovasculardisease,etc.),stainingparameters,andanatomicalregions/sub-regions.Thelimitationsaremainlyattributedtofalsepositives,manyofwhichrepresenttauaccumulationinglialcells(datanotshown).WealsoobservedbetternetworkperformanceinAperio-scannedslides(ADandPARTcases),possiblyduetothelargeramountofannotationdatacomparedtoPhilips-scannedslides(CTEandPSPcases).Whilethenetworkperformanceismorerobustinnosologiesthatcontributedmoreannotationstothetrainingdataset,thiscanbeovercomebyincreasingthetotalnumberofgroundtruthannotationsandsaturatingthelearningcurve.

Infuturedisease-specificstudies,weplantouseexpandedneuroanatomicalsamplingpertinenttotargeteddiseaseentities.Forexample,itmaybehelpfultodifferentiateNFTfromdifferentbrainregionsordifferentsegmentsofoneregion,e.g.NFTsofhippocampusproperpyramidalneuronsandofdentategyrusgranuleneurons.Also,wecombinedallNFTsintoasinglecategory;however,thedifferentiationofpre-NFT,intracellularNFT,andghostNFTmayhelpimprovetheperformanceandprovidemoregranulardata.Finally,wefocusedourstudyonIHCstains,butabnormaltauandothe