人工智能:機(jī)器學(xué)習(xí)概念簡(jiǎn)介

Introduction

to

AI/ML

ConceptsA

bicycle

for

the

mindJustin

JeffressDeveloper

Advocate@SleepyDeveloperSurajSubramanianDeveloperAdvocate@subramenWhat

is

AI?Artificial

IntelligenceArtificial

IntelligenceRefers

to

the

simulation

of

humanintelligenceArtificialIntelligenceMimicking

theintelligence

orbehavioral

pattern

of

humans

orany

otherliving

entity.What

is

ML?Machine

LearningMachine

LearningEnables

computers

to

learn

from

dataArtificialIntelligenceMachineLearningA

techniqueby

which

a

computercan

“l(fā)earn”

fromdata

withoutusing

acomplex

set

of

rules.Mainly

based

on

training

a

modelfrom

datasetsInnovationsImageNetLargest

dataset

of

annotated

imagesCreated

in

2009

@

Stanford

UniversityCreators:

Fei-Fei

Li

&

Jia

Deng14

million

images22

thousand

categories

of

imagesLarge

Scale

Visual

Recognition

ChallengeImageNet’s

Yearly

AI

Challenge

to

inspire

and

reward

innovationCompetition

to

achieve

highest

accuracy

onthe

taskDriven

rapid

advancesComputer

visionDeep

learningMany

moreAlexNetWinner,

winner

chicken

dinnerConvolutionalNeural

NetworkDemonstrated

feasibility

deep

CNNs

end-to-end15.3%

top-5

error

rate!Enabled

further

innovation!

(VGGNet,

GoogLENet,

ResNet,

etc.)AlexNetAlexNetBlock

DiagramWhat

is

Deep

Learning?Deep

LearningPattern

Recognition

&

Feature

extraction

w/

multi-layer

neural

networksArtificialIntelligenceMachineLearningDeepLearningA

technique

to

perform

machinelearning

inspired

by

our

brain’sown

network

of

neurons.Deep

Neural

NetworksInspired

by

the

human

brainHidden

layer

1

Hidden

layer

2Input

LayerHidden

layer

3Output

LayerAI/ML

and

Deep

LearningUnderstanding

how

each

subset

fits

into

the

overall

pictureArtificialIntelligenceMachineLearningDeepLearningMimicking

theintelligence

orbehavioral

pattern

ofhumansor

any

otherliving

entity.Atechnique

by

which

acomputercan

“l(fā)earn”

from

datawithout

using

acomplex

set

ofrules.

Mainly

based

on

trainingamodel

from

datasetsA

technique

to

performmachinelearning

inspired

byour

brain’s

own

network

ofneurons.Deep

Learning

@

MetaIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIIf

you

use

Meta

Products,

DL

is

in

your

lifeNews

Feed

PersonalizationImage

andVideo

RecognitionLanguage

TranslationSpam

and

Fake

News

DetectionPredictive

AnalyticsHow

Deep

Learning

is

used

at

MetaInstagram’s

Explore

recommendersystem/blog/powered-by-ai-instagrams-explore-recommender-system/IntroductionDeep

LearningIntro

to

PyTorchGenerative

AICase

Study:

DisneyAnimated

face

detection/pytorch/how-disney-uses-pytorch-for-animated-character-recognition-a1722a182627IntroductionDeep

LearningIntro

to

PyTorchGenerative

AICase

Study:

DisneyIntroductionDeep

LearningIntro

to

PyTorchGenerative

AINon-human

facial

detection

presents

new

challenges/pytorch/how-disney-uses-pytorch-for-animated-character-recognition-a1722a182627Case

Study:

Blue

River

TechSelf-driving

automated

weed

eliminating

tractors!/pytorch/ai-for-ag-production-machine-learning-for-agriculture-e8cfdb9849a1IntroductionDeep

LearningIntro

to

PyTorchGenerative

AICase

Study:

Blue

River

TechWeed

detection

models/pytorch/ai-for-ag-production-machine-learning-for-agriculture-e8cfdb9849a1IntroductionDeep

LearningIntro

to

PyTorchGenerative

AICase

StudiesRed

=

Weed;Green

!=

Weed/pytorch/ai-for-ag-production-machine-learning-for-agriculture-e8cfdb9849a1IntroductionDeep

LearningIntro

to

PyTorchGenerative

AICheck

out

more

case

studiesGain

inspiration

for

your

AI/ML

projects/community-storiesAdvertising

&MarketingAgricultureAutonomous

DrivingEducationFinanceHealthcareInsuranceMedia

&

EntertainmentMedicalMiningRetailTechnologyTravelIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIPyTorchOpen-source

library

to

build

and

train

modelsBased

on

the

Torch

LibraryDeveloped

by

Facebook’s

AI

Research

LabReleased

in

2016Programming

interface

for

building

and

training

NeuralNetworksIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIPyTorchWell-known

domain-specific

librariesTorchTextTorchVisionTorchAudioIntroductionDeep

LearningIntro

to

PyTorchGenerative

AITypical

ML

Pipeline

with

PyTorchUnderstanding

the

processIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIGetting

started

with

PyTorchIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIUseful

resourcesLearn

thebasics:/tutorials/beginner/basics/intro.htmlQuickstart:/tutorials/beginner/basics/quickstart_tutorial.htmlWorkshop:

Identify

Objects

with

TorchVisionIdentify

objects

with

TorchVisionIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIIs

there

a

traffic

light

in

this

image?Identify

objects

with

TorchVisionIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIIs

there

a

traffic

light

in

this

image?Typical

pipeline

for

object

detectionIdentifying

objects

in

images

with

TorchVisionIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIHow

do

computers

see

images?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIDo

Androids

Dream

of

Electric

Sheep?How

do

computers

see

images?Ever

openan

image

in

a

text

editor?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsTensors:

Multi-dimensional

data

structuresScalarVectorMatrixTensor121

23

41

23

41IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsTensors:

Multi-dimensional

data

structuresScalarVectorMatrixTensorRank

0

TensorRank

1

TensorRank

2

TensorRank

3

Tensor121

23

41

23

41IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsTensors:

Multi-dimensional

data

structuresScalarVectorMatrixTensorRank

0

TensorRank

1

TensorRank

2

TensorRank

3

Tensor121

23

41

23

411234123412341234IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIRank

4

Tensor/tutorials/beginner/basics/tensorqs_tutorial.html/tutorials/beginner/introyt/tensors_deeper_tutorial.htmlWorkshop

key

concepts/tutorials/beginner/basics/tensorqs_tutorial.htmlImage

TensorsImage

tensors

are

typically

rank

3tensorsdim0:

number

of

channels

(3for

anRGBimage)dim1:

heightoftheimagedim2:

widthof

the

imageIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AITorchvision/vision/stable/index.htmlLibrary

for

Image

and

Video:datasetsmodels

(pretrained

and

untrained)transformationsWorkshop

key

conceptsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIBatchingImage1Image2Image3Image4Image5GPUWorkshop

key

conceptsBatchingImage1CPUQUEUEImage2Image3Image4Image5Image6IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsCPUQUEUEImage7Image8Image9Image10Image11GPUBatchingImage1Image2Image3Image4Image5Image6IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIGPUWorkshop

key

conceptsBatchingImage1Image2Image3Image4Image5Image1Image1Image1Image1Image1Batch

of

6

imagesIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsCPUQUEUEImage7Image8Image9Image10Image11GPUBatchingImage1Image2Image3Image4Image5Image6Image7Image7Image7Image7Image12Image8Image8Image8Image8Image18Image9Image9Image9Image9Image24Image10Image10Image10Image10Image30Image11Image11Image11Image11Image36IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIPretrained

ModelsYou

will

use

fasterrcnn_resnet50_fpn

for

the

labThe

name

refers

to

the

neural

architectures

used

in

themodel.Resnet50

is

a

popular

model

that

extracts

useful

information

from

an

imagetensorFaster

RCNN

is

an

object-detection

architecture

that

uses

Resnet’s

extractedfeatures

to

identify

objectsin

an

imageThe

model

has

been

trained

on

the

COCO

academicdatasetTorchvision

contains

several

more

pretrained

models

for

different

use

casesWorkshop

key

conceptsFast

R-CNN/pdf/1506.01497.pdfIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsCOCO

datasetCOCO

dataset

contains

many

common

objects.Models

trained

on

COCO

predict

the

class

ofthe

object

asaninteger.We

then

look

up

the

integer

to

find

out

the

object

itrepresentsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIModel

InferenceProcess

of

generating

a

prediction

from

inputsInPyTorch,

as

simple

asprediction

=model(input)If

inputis

a

batch

of

Nsamples,

outputis

a

batch

of

N

predictionsEach

prediction

is

a

listof

the

objects

detected

in

the

image,

andhow

confidentthemodel

isaboutthedetectedobjectWorkshop

key

conceptsPost

processingOutput...0:

[Pizza,

1]1:[P0e:p[pPeizrzoan,i,11]6]2:

[C1:he[Peesep,pe1r]oni,16]...2:

[C0:he[Peiszez,a1,

]1]IntroductionDeep

LearningIntro

to

PyTorchGenerative

AI1:

[Pepperoni,

16]2:

[Cheese,

1]...Use

TorchVision

to

identify

objectsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIFollow

the

steps

at

your

own

pace45

MIN11:15AM/fbsamples/mit-dl-workshophttps://discord.gg/uNRcgFVWWorkshop

wrap-upIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWe

learntImage

loading

and

manipulation

in

Python

and

PyTorchLoading

pretrained

models

with

TorchvisionBatch

processing

in

deeplearning

modelsInference

andpost-processing

with

object

detection

modelsGenerative

AIWhat

is

Generative

AI?What

is

a

modality?Input

vs

Output

ModalitiesGenerative

AI

can

be

segmented

by

modalityTextAudioImages

-

2DVideos

–

2D3D

assets–

static3D

assets-movementTextLines

of

codeEssays,

chatbots,

conversationAudioCleanedupaudioSongs

/

instrumentalpiecesVoice

RenderingsImages

-

2DVideos

–

2D3D

assets–

static3D

assets-movementInput

modalitiesOutputmodalitiesIntroductionDeep

LearningIntro

to

PyTorchGenerative

AINotablePlayersInnovators

in

the

generative

AI

spaceDALL-E2StableDiffusionIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIRefik

Anadol

StudiosIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIUsing

data

as

pigments

to

generate

a

new

artformRefik

Anadol

StudiosCheck

out

the

interviewhttps:///watch?v=yjPv2ltMt-EIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop:

Video

Synopsis

GeneratorCreate

a

text

summary

of

a

videoEasily

create

cliff’s

notes

for

videos!Art

&

AI/ML

collaborate

increative

ways,

like

how

the

Refik

Anadol

Studio

is

powered

by

PyTorch.

Watch

Refik

and

Christian

B.

talk

with

Developer

Advocates

Suraj

Subramanian

and

Justin

Jeffress

about

how

theStudio

uses

PyTorch

to

turn

data

into

pigments…IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIAnatomy

of

the

video

summarizerFrom

video

to

text

summaryIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsExtract

audio

from

videoFFMPEG

is

a

suite

oflibraries

and

programsfor

handling

video,

audio,

other

multimediafiles,

and

streams.It

isacommand-line

tool,

but

canalso

becalled

from

python

notebooks

by

prefixing

anexclamation

mark

(!)!ffmpeg

-i

input.mp4

output.aviIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsAutomatic

Speech

RecognitionBuilding

modelswith

PyTorch

isfun!Building

modelswith

PyTorch

is

fun!IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

key

conceptsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIText

SummarizationProduce

a

concise

and

accurate

summary

of

the

input

textEarlier

NLP

architectures

used

recurrent

neural

networks

(RNNs).

ModernNLPmodels

aretransformer-basedSummarization

models

are

general

language

models

that

have

been

fine-tuned

forsummary

generation

using

datasets

like

CNN

Dailymail,

Amazon

reviewsetc.Typically,

models

have

limits

on

the

input

length

i.e.

the

number

of

tokensconstituting

theinput

fedto

themodelWorkshop

key

conceptsTokenizationSplitting

a

largebody

of

textinto

smaller

pieces(tokens)Tokens

can

be

words,

phrases

or

even

whole

sentencesTokenization

helps

to

make

the

text

more

manageable

and

easier

to

process.IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIBuildyour

video

synopsis

generatorIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIFollow

the

steps

at

your

own

pace60

MIN/fbsamples/mit-dl-workshop/blob/main/video-summarizer/exercise.ipynbWe

learntFFMPEG

for

audio

extractionAutomatic

speech

recognitionNLP

concepts

(tokenization,

summarization)Whisper

and

Huggingface

APIsPandas

DataFramesIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop

wrap-upHow

might

you

use

the

summarizer?We

used

it

on

the

recording

of

this

workshop!IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIHow

might

you

use

the

summarizer?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWe

used

it

on

the

recording

of

this

workshop!When

dealing

with

generative

AI,

you

have

different

modalities.An

input

modality

could

be

text,

it

could

beaudio.

It

couldbeimages,

videos,

3D

assets.

Generative

AI

is

a

type

of

artificialintelligence

that

is

being

made

available

to

third

parties

to

be

ableto

play

with.

Rafik

Anadol

Studios

is

using

generative

AI

to

createart

from

people's

brainwaves.

We're

going

to

go

through

aworkshop

on

how

to

create

a

video

synopsis

generator

with

AI.We're

going

to

be

using

two

different

AI

ML

models

to

achieve

thistask.

And

so

you'll

learn

some

more

details

as

we

go

along.

Onceyou've

done

this,

you'll

actually

have

the

necessary

components

to

be

able

to

do

whatever

video

you

want

to.

Python

is

aprogramming

language.

It

can

be

used

to

generate

videosummaries

and

other

types

ofdata.FeedbackIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIIthelps

us

improve

our

contenthttps://forms.gle/fYp6LdCcdufTRczc7Generative

AI

(cont.)OpenAIIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIText

completion,

image

and

code

generation;

Oh

my!chatGPTVirtual

writing

assistantIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIchatGPTYou

can

change

the

writing

style

with

a

simple

prompt!IntroductionDeep

LearningIntro

to

PyTorchGenerative

AINot

trained

onanything

post

2021Don’t

worry

you

can

fill

in

the

gapsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AINot

trained

onanything

post

2021Don’t

worry

you

can

fill

in

the

gapsIntroductionDeep

LearningIntro

to

PyTorchGenerative

AINot

trained

onanything

post

2021Don’t

worry

you

can

fill

in

the

gapsNeed

help

coding?Should

I

go

to

Stack

Overflow

or

chatGPT?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AINeed

help

coding?Do

it

manually

using

recursionIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIOther

things

to

tryPoetryIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIWorkshop:

Generative

AI

as

a

creative

partnerGet

your

OpenAI

API

KeyHow

do

I

get

one?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIGet

your

OpenAI

API

KeyHow

do

I

get

one?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIGet

your

OpenAI

API

KeyHow

do

I

get

one?IntroductionDeep

LearningIntro

to

PyTorchGenerative

AIGet

your

OpenAI

API

KeyHow

do

I

get

one?Save

this

in

a

file

somewhereIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIPart

1:

Create

your

Open

AI

KeyIntroductionDeep

LearningIntro

to

PyTorchGenerative

AICreate

an

account,

save

your

API

key,

and

write

a

story5

MIN/api/https://discord.gg/uNRcgFVWPart

2:

Personal

Assistant

with

openAIIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIPart

2:

Personal

Assistant

with

openAIIntroductionDeep

LearningIntro

to

PyTorchGenerative

AIPar