标签: With 87205

WithPYNQandMachineLearningApplicationsTableofContentsForeword.............................................................xviiAcknowledgements.....................................................xix1)Introduction............................................................11.1)WhyShouldIbeInterested?.............................................21.2)TheEvolutionofXilinxSoCs—VeryBriefly!...............................31.3)DesignMethods......................................................41.4)HowtoUsethisBook..................................................61.4.1)OrganisationoftheBook.........................................61.4.2)FurtherSourcesofInformation.....................................71.4.3)SuggestionsforBeginners.........................................71.5)WhatNext?..........................................................81.6)References...........................................................8PartA:GettingtoKnowZynqMPSoC.................................92)FPGAs,Zynq,andZynqMPSoC!........................................112.1)TechnologyTimeline.................................................122.2)TheZynq-7000SoC..................................................122.2.1)ZynqArchitectureandFeatures...................................122.2.2)ZynqDevices..................................................142.2.3)ZynqUseModels..............................................142.3)TheXilinxZynqMPSoC..............................................152.3.1)TheReleaseofZynqMPSoC.....................................152.3.2)ZynqMPSoCArchitectureandFeatures............................162.4)FPGAs............................................................182.4.1)WhatisanFPGA?.............................................182.4.2)TheDevelopmentofFPGAs......................................182.4.3)TheModernFPGAArchitecture:UltraScale+........................212.5)ComparisonandDiscussion:FPGAvZynqvZynqMPSoC...................242.5.1)Architectures..................................................242.5.2)PowerConsumptionandPerformance..............................252.5.3)EmbeddedSystemsImplementation................................272.5.4)Applications...................................................282.6)ChapterSummary....................................................292.7)References...........................................................293)AnOverviewoftheZynqMPSoCArchitecture.............................313.1)ZynqMPSoCDeviceFamilies..........................................313.1.1)ArmVersusXilinxDocumentation.................................333.2)ProcessingSystem....................................................333.2.1)ApplicationProcessingUnit.......................................353.2.2)Real-TimeProcessingUnit.......................................373.2.3)GraphicsProcessingUnit.........................................383.2.4)Connectivity...................................................393.3)PlatformManagement.................................................423.3.1)PowerModes..................................................423.3.2)PowerDomains................................................433.3.3)PlatformManagementUnit.......................................443.4)ProgrammableLogic..................................................453.4.1)TheLogicFabric...............................................463.4.2)StorageandSignalProcessingResources.............................473.4.3)PLPeripherals.................................................483.4.4)VideoCodecUnit..............................................483.4.5)GeneralPurposeInput/Output....................................483.4.6)High-SpeedConnectivity........................................493.4.7)JTAGInterface................................................493.5)ProcessingSystemandProgrammableLogicInterfaces........................493.5.1)TheArmAMBAOpenStandard..................................503.5.2)PStoPLInterconnectsandInterfaces...............................523.5.3)EMIOInterfaces...............................................543.5.4)OtherPStoPLInterfaces........................................553.6)SecurityandConfiguration.............................................553.6.1)ConfigurationSecurityUnit......................................563.7)ChapterReview......................................................583.8)References..........................................................584)DesignTools&MethodsfortheZynqMPSoC.............................634.1)AnatomyofaZynqMPSoCDesign......................................634.1.1)ZynqMPSoCDevelopmentBoards................................644.1.2)TheHardwareSystem...........................................654.1.3)TheSoftwareStack.............................................684.2)TheDesignProcess...................................................694.2.1)RequirementsandSpecification...................................694.2.2)ProjectDecisionsandPlanning....................................704.2.3)DesignFlow..................................................714.2.4)Testing......................................................724.3)GettingReady:DesignToolsandDevelopmentSystemSetup.................724.3.1)HostSystem..................................................724.3.2)XilinxSoftwareComponents.....................................734.3.3)HardwareRequirements.........................................804.4)DevelopmentBoardsandSupportingResources.............................814.5)ResourcesandSupport................................................824.5.1)InformationaboutZynqMPSoCDevices...........................824.5.2)SoftwareToolsSupport..........................................834.5.3)SupportforSystemDesign.......................................844.6)TheWiderEcosystem.................................................844.6.1)HardwareSystemDesignToolsandComponents.....................854.6.2)SoftwareSystemDesignToolsandComponents......................874.6.3)AlgorithmDevelopmentandInterfacing............................894.6.4)HardwareandPeripherals........................................894.7)ChapterSummary....................................................914.8)References..........................................................915)CandidateApplicationAreasforZynqMPSoC.............................975.1)WhatMakesmySystema‘ZynqMPSoCSystem’?..........................975.2)ApplicationAreasfortheZynqMPSoC(andRFSoC).......................995.3)Drones............................................................1005.3.1)Flying......................................................1005.3.2)VideoProcessing..............................................1025.3.3)Sensing......................................................1025.3.4)CommunicationsandNavigation..................................1035.3.5)AdvancedDrones..............................................1045.4)SmartandAutonomousVehicles........................................1055.4.1)WhatDoes‘ADAS’Cover?......................................1065.4.2)SensingRequirementsandImplementation..........................1065.4.3)AutonomousVehicles..........................................1085.4.4)ConnectivityandSecurity.......................................1085.5)InterfacingtotheRFAnalogueWorld—theEvolutiontoZynqRFSoC........1095.5.1)TheZynqRFSoCDeviceFamilyComparedtoMPSoC...............1105.5.2)RFSampling:RF-ADCsandRF-DACs,andSingleChipSDR.........1135.5.3)DirectSamplingZynqRFSoCDataConverterSubsystem..............1145.5.4)4GLTEMulticarrierSolutions...................................1165.5.5)5GMobileandWirelessImplementations..........................1175.5.6)5GmmWaveImplementationsusingIFArchitectures.................1175.6)ChapterSummary...................................................1195.7)References..........................................................119PartB:TheZynqMPSoCArchitectureinDetail......................1236)TheApplicationProcessingUnit.........................................1256.1)TheCortex-A53MPCoreProcessor.....................................1256.2)FundamentalsofArmv8-A............................................1266.2.1)64-Bit/32-BitExecutionStates...................................1276.2.2)InstructionSetsandProgrammingLanguages........................1286.2.3)ExceptionLevels..............................................1296.2.4)ProcessorModes..............................................1316.2.5)Armv8SecurityModel..........................................1336.3)SignalsandInterfaces.................................................1376.3.1)TheMasterMemoryInterface....................................1386.3.2)ACPSlaveInterface............................................1396.3.3)CrossTriggerInterface(CTI)....................................1416.3.4)AdvancedPeripheralBus(APB)Debug............................1426.3.5)GICInterface.................................................1426.3.6)TraceInterface................................................1426.3.7)OtherSignals.................................................1436.4)MemoryManagementUnit............................................1446.5)MemorySystems....................................................1446.5.1)Level1MemorySystem........................................1456.5.2)Level2MemorySystem........................................1466.6)ProcessorExtensions.................................................1466.6.1)Cryptography(Crypto).........................................1466.6.2)NEONMediaProcessingEngine.................................1466.6.3)VectorFloatingPointUnit......................................1486.7)Interrupts..........................................................1486.7.1)GICv2InterruptTypes.........................................1496.7.2)GICv2InterruptPriorityandStates...............................1506.7.3)GIC-400Overview............................................1506.8)PowerManagement..................................................1536.8.1)PowerDomains...............................................1536.8.2)PowerModes.................................................1546.9)SystemVirtualisation.................................................1556.9.1)AddressTranslationinaVirtualEnvironment.......................1576.9.2)InterruptVirtualisation.........................................1576.10)ChapterReview.....................................................1596.11)References.........................................................1597)TheReal-TimeProcessingUnit.........................................1617.1)Introduction........................................................1617.1.1)WhatisReal-TimeProcessing?..................................1617.1.2)WhyaDifferentArchitectureforReal-TimeProcessing?..............1637.2)Overview..........................................................1637.3)DeterminismandResponsiveness.......................................1657.3.1)TightlyCoupledMemories......................................1657.3.2)InterruptSystem..............................................1667.4)SafetywithintheRPU................................................1707.4.1)MemoryProtection............................................1717.4.2)Split/LockModes.............................................1747.5)ChapterReview.....................................................1767.6)References.........................................................1768)SecurityinZynqMPSoC................................................1798.1)InformationAssuranceforConfigurationSecurity..........................1808.1.1)ConfigurationSecurityUnitIntroduction...........................1818.1.2)CryptoBlocks.................................................1838.1.3)KeyManagement..............................................1908.2)Anti-Tampering.....................................................2008.2.1)Monitoring...................................................2018.2.2)Response....................................................2028.2.3)Precautions...................................................2048.3)SecurityThroughIsolation.............................................2058.3.1)IsolatingSoftwarewithVirtualisationandArmv8.....................2068.3.2)ExtendingIsolationtoaCompleteSystem..........................2098.3.3)IsolationSummary.............................................2138.4)ChapterSummary...................................................2138.5)References..........................................................2149)SafetyFeatures&Techniques...........................................2179.1)AnIntroductiontoSafety..............................................2179.1.1)SafetyandDependability........................................2189.1.2)SafetyApplicationExamples.....................................2199.2)FunctionalSafety....................................................2199.2.1)WhatisFunctionalSafety?.......................................2199.2.2)Errors,FaultsandFailures!......................................2209.2.3)HandlingofFaults,ErrorsandFailures.............................2219.2.4)FunctionalSafetyStandards......................................2259.3)DesignPrinciplesandArchitecturalSupport..............................2279.3.1)Redundancy..................................................2279.3.2)Diversity.....................................................2309.3.3)IsolationDesignFlow..........................................2329.3.4)Real-TimeProcessors:Dual-LockstepMode........................2339.3.5)CCFsandSystemMonitoring....................................2349.3.6)Error-CorrectingCode(ECC)Memory............................2359.3.7)FaultInjectionandTesting......................................2369.4)ChapterSummary...................................................2379.5)References..........................................................23710)PlatformManagementFeatures.........................................23910.1)PowerModes.......................................................24010.1.1)BatteryPoweredMode.........................................24010.1.2)Low-PowerMode.............................................24110.1.3)Full-PowerMode.............................................24110.1.4)Deep-SleepMode.............................................24110.2)PowerDomains.....................................................24210.2.1)BatteryPowerDomain.........................................24310.2.2)Low-PowerDomain...........................................24410.2.3)Full-PowerDomain...........................................24510.2.4)PLPowerDomain............................................24610.2.5)OtherPowerDomains.........................................24710.3)PlatformManagementUnit...........................................24710.3.1)PMUProcessor...............................................24910.3.2)InterfacesandAXIInterconnect..................................25010.3.3)LocalandGlobalRegisters......................................25110.3.4)ProgrammableIntervalTimers...................................25110.3.5)Interrupts....................................................25110.4)ErrorManagement..................................................25310.4.1)ErrorStatusandEnableRegisters.................................25310.4.2)InterruptErrorRegisters........................................25410.4.3)ErrorHandling...............................................25410.5)PMUFirmware.....................................................25510.5.1)CompositionofthePMUFirmware...............................25510.5.2)DefaultModules..............................................25610.5.3)ExecutionofthePMUFirmware.................................25710.5.4)CustomModules..............................................25810.6)ChapterReview.....................................................25810.7)References.........................................................259PartC:ZynqMPSoCSystemsDevelopment.........................26111)HardwareSystemDevelopment..........................................26311.1)HeterogeneousComputingwithZynqMPSoC............................26411.2)HardwareSystemOverview...........................................26611.2.1)InterfacesandSignals...........................................26611.2.2)TheInterconnect..............................................26911.2.3)Memories....................................................27211.3)PLInterfacing......................................................27311.3.1)Snooping....................................................27411.3.2)AXICoherencyExtension(ACE)Interface.........................27411.3.3)TheAXIFIFOInterface........................................27611.3.4)PL-FPDAXIMasters..........................................27711.3.5)PL-LPDAXIMaster..........................................27711.3.6)PL-PSAXISlaves.............................................27811.3.7)SelectingaPLInterface.........................................27811.4)TheInterruptSystem.................................................28011.4.1)InterruptSystemOverview.......................................28011.4.2)TypesofInterrupts.............................................28111.4.3)InterruptPrioritisation,States,andHandling........................28311.4.4)TheInter-ProcessorInterrupt....................................28411.5)Memories..........................................................28511.5.1)TheGlobalAddressSpace.......................................28511.5.2)On-ChipMemory.............................................28611.5.3)DDRMemoryInterface.........................................28711.6)DataMovementFundamentals.........................................28911.6.1)DirectMemoryAccess..........................................28911.6.2)TheAXIInterconnect..........................................29111.6.3)DMAControllersinthePS......................................29111.6.4)SimpleAXICommunication.....................................29311.6.5)TheAXIDMA...............................................29411.6.6)TheAXIVideoDMA..........................................29611.7)ChapterReview.....................................................29811.8)References..........................................................29812)SoftwareStacks........................................................30112.1)Bare-MetalSoftwareStack.............................................30112.1.1)WhatdoesBare-Metalgiveus?...................................30212.1.2)CStandardLibraries...........................................30312.1.3)StandaloneLibraries............................................30412.1.4)StandaloneDrivers.............................................30512.1.5)Howcanweuseit?............................................30512.2)FreeRTOSSoftwareStack............................................30712.2.1)WhatdoesFreeRTOSgiveus?...................................30712.2.2)Tasks.......................................................30812.2.3)Inter-taskSynchronisation......................................30912.2.4)OtherUtilities................................................31112.2.5)Howcanweuseit?............................................31212.3)LinuxSoftwareStack.................................................31312.3.1)WhatdoesLinuxgiveus?.......................................31412.3.2)TheKernelSpaceandUserSpaceDivide...........................31512.3.3)MemoryManagement..........................................31712.3.4)DeviceDrivers................................................31912.3.5)Processmanagement...........................................32212.3.6)Inter-ProcessCommunication...................................32412.3.7)Howcanweuseit?............................................32512.3.8)MultimediaSoftwareStack......................................32512.4)ChapterReview.....................................................32612.5)References.........................................................32713)MultiprocessorDevelopment............................................33113.1)IntroductiontoHeterogeneousProcessing................................33213.1.1)ProcessorSetsonZynqMPSoC..................................33213.1.2)SplittingSoftwareTasks........................................33213.1.3)HeterogeneousComputingConcepts..............................33413.2)SymmetricMultiprocessingwithLinux..................................33613.3)AsymmetricMultiprocessing...........................................33913.3.1)UnsupervisedAMPwithOpenAMP..............................33913.3.2)SupervisedAMPwiththeXenHypervisor..........................34413.4)AHybridExampleforSoftwareDefinedRadioApplications.................34813.5)ChapterSummary...................................................35013.6)References.........................................................35114)SystemBooting.......................................................35314.1)IntroductiontoSystemBooting........................................35314.2)Non-SecureBootProcess.............................................35614.2.1)BootProcessOverview.........................................35614.2.2)BootMediaOptions...........................................36014.2.3)BootImageSearch.............................................36114.2.4)BootImageFormat............................................36114.2.5)RecapwithaLinux/OpenAMPUse-Case...........................36314.3)SecureBootProcess..................................................36414.3.1)SecureBootFundamentals.......................................36414.3.2)SecureBootCryptography.......................................36514.3.3)SecureBootImageFormat.......................................36914.4)PracticalNon-SecureDeviceConfiguration...............................37014.4.1)GeneratingBootImages........................................37014.4.2)ProgrammingtheBootImages...................................37114.5)PracticalSecureDeviceConfiguration....................................37114.5.1)GeneratingBootImages........................................37214.5.2)DeviceConfigurationforSecureBoot..............................37414.5.3)MaintainingtheChainofTrustBeyondtheFSBL....................37414.6)ChapterSummary...................................................37514.7)References..........................................................375PartD:SystemDesignwithXilinxSDxDevelopmentEnvironment......37715)IntroductiontoSystemDesignwithSDx..................................37915.1)MotivationforusingSDx..............................................37915.2)AboutSDx.........................................................38115.2.1)TheDevelopmentEnvironment..................................38115.2.2)IntroducingtheFull-SystemOptimisingCompiler....................38315.2.3)DataMotionNetworks.........................................38315.2.4)DirectingtheSDxTool-Chain...................................38615.2.5)SDxAPI.....................................................38915.2.6)SDxTool-ChainSettings.......................................39015.3)TheDesignFlow....................................................39215.3.1)SDxPlatform.................................................39415.3.2)SelectingInitialCandidateFunctionsforHardware...................39515.3.3)EstimatingSystemPerformance..................................39615.3.4)OptimisingSystemPerformance..................................39615.3.5)AnalysingSystemPerformance...................................39715.4)SDxProjectHierarchy................................................39815.5)ChapterReview.....................................................40215.6)References.........................................................40316)SystemProfilingandAccelerationwithSDx...............................40516.1)SystemProfiling....................................................40516.1.1)Software-OnlySystemProfiling..................................40516.1.2)EstimatingRelativePerformance.................................41116.1.3)MeasuringSystemPerformance..................................41216.2)SoftwareAccelerationusingProgrammableLogic..........................41416.2.1)IPCoreOptimisation..........................................41416.2.2)OptimisingIPCoreIntegration..................................41616.3)ChapterReview.....................................................42116.4)References.........................................................42217)ReusingExistingIPinSDx.............................................42317.1)CreatingaC-CallableLibrary..........................................42317.1.1)IPCoreRequirements..........................................42417.1.2)IPConfigurationParameters.....................................42517.1.3)FunctionDefinition...........................................42517.1.4)FunctionArgumentMapping....................................42617.1.5)ThesdslibUtility..............................................42717.1.6)LibraryHeaderFile............................................42817.2)UsingaC-CallableLibrary............................................42917.3)ChapterReview.....................................................43017.4)References.........................................................43018)DebuggingandPerformanceMonitoringwithSDx.........................43118.1)SystemEmulation...................................................43218.2)SoftwareDebugging.................................................43418.3)HardwareDebugging................................................43618.4)PerformanceMonitoring..............................................43918.4.1)EventTracing................................................43918.4.2)AXIPerformanceMonitor......................................44118.4.3)ProcessingSystemPerformanceMonitoring.........................44418.5)ChapterReview.....................................................44518.6)References.........................................................44619)CustomSDxPlatforms.................................................44719.1)WhymightwewanttodevelopacustomSDxPlatform?.....................44719.2)SDxPlatformStructureandComponents.................................44819.2.1)MetadataFiles................................................45019.2.2)SampleApplications............................................45019.3)SDxPlatformHardwareComponent.....................................45119.3.1)DevelopingthePlatformHardwareDesign..........................45119.3.2)PlatformHardwareMetadataFile.................................45319.4)SDxPlatformSoftwareComponent.....................................45719.4.1)BootFiles....................................................45719.4.2)PetaLinuxTools...............................................46519.4.3)BootImageFormatFiles........................................46519.4.4)Libraries.....................................................46619.5)SDxPlatformSampleApplications......................................46719.6)ThesdspfmUtility...................................................46919.7)Pre-BuiltHardware..................................................47119.8)TestingandUsingaCustomSDxPlatform................................47319.9)ChapterReview.....................................................47419.10)References..........................................................474PartE:PYNQ,andMachineLearningApplications....................47920)DeepLearning........................................................48120.1)MachineLearningisEverywhere(orwillbe!)..............................48120.1.1)MachineLearningversusDeepLearning...........................48120.1.2)MachineLearningOverview.....................................48320.1.3)EstablishedApplications........................................48520.1.4)EmergingApplications.........................................48620.1.5)PlatformsforMachineLearning..................................48920.2)CommonNeuralNetworkArchitectures..................................48920.2.1)TheNeuron..................................................48920.2.2)MultilayerPerceptron..........................................49120.2.3)ConvolutionalNeuralNetworks(CNNs)...........................49320.2.4)RecurrentNeuralNetworks(RNNs)...............................49520.3)TrainingNeuralNetworks.............................................49720.3.1)LossFunctions...............................................49820.3.2)Backpropagation..............................................49920.3.3)StochasticGradientDescent(SGD)...............................50120.3.4)Regularisation................................................50220.4)DeepLearningToolsandFrameworks...................................50320.5)TheNeedforAcceleration............................................50420.5.1)CloudandEdgeApplications....................................50420.5.2)CompressionTechniques.......................................50520.6)ChapterSummary...................................................50520.7)References.........................................................50621)ReducedPrecisionNeuralNetworks......................................50921.1)ReducedPrecisionNeuralNetworks.....................................50921.1.1)Whatis‘ReducedPrecision’?.....................................51021.1.2)Motivation...................................................51121.1.3)TheImpactonScaling.........................................51121.1.4)NeuralNetworkImplementationOptions..........................51321.2)IntroducingFINN-R.................................................51421.2.1)OriginsofFINN-R............................................51421.2.2)ReducedPrecisionOptimisations.................................51421.2.3)ArchitecturalOptimisations.....................................51421.2.4)FINN-RDesignFlow..........................................51521.2.5)WorkingwithFINN-R.........................................51621.3)NeuralNetworksonProgrammableLogic................................51621.3.1)ANeuroninProgrammableLogic................................51721.3.2)CreatingLayers...............................................51821.3.3)DesignMethodsandTools......................................52021.4)FINN-RCaseStudy:AnInferenceAcceleratorforZynqMPSoC.............52121.5)ChapterSummary...................................................52221.6)References.........................................................52322)PYNQ...............................................................52522.1)IntroducingPYNQ..................................................52522.1.1)PYNQasaFramework.........................................52522.1.2)WhatisPYNQ?..............................................52622.1.3)WhyPython?.................................................52922.1.4)WhatPYNQIsNot!...........................................53022.2)AnExamplePYNQApplication........................................53122.2.1)PythonNotebook(SoftwareOnly)................................53422.2.2)PYNQNotebook(HardwareAccelerated)..........................53422.2.3)SystemArchitecture(HardwareAcceleratedVersion)..................53422.2.4)KeyObservations..............................................53722.3)ComponentsofaPYNQSystem........................................53722.3.1)PYNQImage.................................................53722.3.2)FileHandling.................................................53822.3.3)Overlays.....................................................53922.3.4)JupyterNotebooks.............................................54222.3.5)FilesandInstallation...........................................54422.3.6)AdvancedUserCustomisation....................................54522.4)ThePYNQProject...................................................54622.4.1)PYNQGoals.................................................54622.4.2)PYNQInfluences..............................................54722.5)Resources..........................................................54922.5.1)HowDoIGetPYNQ?.........................................54922.5.2)Documentation...............................................54922.5.3)FilesandSourceCode..........................................55022.5.4)Hardware....................................................55022.5.5)CommunityResources..........................................55022.6)ChapterSummary...................................................55122.7)References..........................................................55223)CaseStudy:PrototypingCNNsonZynqforSpaceApplications,UsingPYNQ..55523.1)DesignConcept.....................................................55623.2)CloudDetectionwithNeuralNetworks...................................55623.2.1)AlgorithmSelection............................................55623.2.2)AlgorithmDevelopment........................................55823.3)HardwareSelection..................................................55823.3.1)SelectingtheFPGA............................................55823.3.2)DiscoveringPYNQ............................................56023.4)CloudDetectionwithPYNQ..........................................56023.4.1)DevelopingaPrototypewithPYNQ...............................56023.4.2)Results......................................................56123.5)ReflectionsandRuminations...........................................56223.6)References.........................................................562PartF:AndFinally................................................56524)AcademicCaseStudies.................................................56724.1)EmbeddedComputerVision...........................................56724.1.1)AdvancedDriverAssistanceSystems..............................57124.2)SpaceflightSystems..................................................57324.3)MachineLearning...................................................57424.4)ChapterSummary...................................................57824.5)References.........................................................578Postscript.........................................................581Appendix:BSD3-ClauseLicence........................................583ListofAcronyms......................................................585Index................................................................605