AWS - Compute - SAM

• SAM Serverless Application Model 无服务器应用模型
  • AWS SAM
  • Benefits of using AWS SAM
    • SAM 模板特有资源类型
      • AWS::Serverless::Function
      • AWS::Serverless::Api
      • AWS::Serverless::SimpleTable
  • SAM code
  • example
    • 使用SAM模板管理Lambda环境变量
    • 用CloudFormation部署SAM模板
    • Deploying a Hello World application
    • Process Amazon S3 events
    • SAM + Lambda

---

SAM Serverless Application Model 无服务器应用模型

无服务器（Serverless）

• 看上去似乎是「没有服务器」的意思，但其实=是「云原生」（Cloud Native）。
• 换句话说，就是「无服务器」的服务，一定是「云原生」的。
• 虽然用户也可以自建机房，也能抽象出一个平台式的服务，看上去好像也不用去管理服务器，但实际上这只是把服务器管理的职责从研发团队转移到了运维团队，所有的成本、风险、经济效应，甚至到遇到问题之后的调试方式和研发的思路，并没有实质的变化。也正因如此，这类服务一般不叫「无服务器」，最多也就被叫「PaaS」（Platform-as-a-Service，平台即服务）。
• 真正要做到无服务器，就意味着管理服务器的工作，服务器运行、运维的成本，必须完全转移到第三方——即云服务商。只有这样，才能做到非常精细的按需付费，并且形成与原来完全不同的研发思路。

云研发

• 既然服务器运行、运维都由云服务商负责，那么整套体系就必然完全架设在云上，并且对用户来说它是不透明的。
• 也就是说，它是云原生的，系统的整个生命周期都完全在云上。
• 云的巨大弹性和底层优化，使得模拟一个类似环境来做研发、测试变得困难，大家只能都在云上去做研发。
• 最好的情况当然是，我们在浏览器或者远程桌面上开一个 IDE，直接在云上做开发，然后云上做测试，提交代码到云上的仓库，并且通过云上的流水线做部署。
• 可现实是，大部分的研发还是发生在客户端。大家还是希望有个简单的办法，让我在本地机器，用自己喜欢的 IDE 和编辑器，快速地写代码、做测试，而不是必须连到云上来操作。
• 这时候，我们就需要一个工具，它应该要能在本地模拟一套无服务器的研发环境。最好足够简单、便捷，也不辜负我们使用无服务器的初衷。针对这个需求，AWS 推出了 Serverless Application Model（SAM）。

---

AWS SAM

• an open-source framework
  • 是一个开源的模型，
  • https://github.com/awslabs/serverless-application-model

• to build serverless applications on AWS.

• A serverless application
  • use AWS SAM to define the serverless applications.
  • 用无服务器应用模型 部署 无服务器应用
  • a combination of Lambda functions, event sources, and other resources (such as APIs, databases) that work together to perform tasks.

• AWS 无服务器应用模型 AWS Serverless Application Model

• AWS 2016年11月发布

• 结合AWS自动运维相关的服务如AWS CloudFormation 和AWS CodePipeline，统一管理多种资源，实现无服务器应用的 持续集成和部署。

• 把这些服务资源方便地管理起来

• an open-source framework to build serverless applications on AWS.
  • A serverless application is a combination of Lambda functions, event sources, and other resources that work together to perform tasks.
  • Note that a serverless application is more than just a Lambda function—it can include additional resources such as APIs, databases, and event source mappings

AWS SAM consists of the following components:

1. AWS SAM template specification.
   • 一套对 CloudFormation 模板格式的扩展，让我们可以更抽象地来定义无服务器资源。
   • use this specification to define the serverless application.
   • simple and clean syntax to describe the functions, APIs, permissions, configurations, and events that make up a serverless application.
   • use an AWS SAM template file to operate on a single, deployable, versioned entity that’s your serverless application.
2. AWS SAM command line interface (AWS SAM CLI).
   • 一个命令行工具，帮助我们在本地搭建无服务器计算环境，即 API Gateway 和 Lambda。
   • use this tool to build serverless applications that are defined by AWS SAM templates.
   • The CLI provides commands enable you to
     • verify that AWS SAM template files are written according to the specification,
     • invoke Lambda functions locally,
     • step-through debug Lambda functions,
     • package and deploy serverless applications to the AWS Cloud, and so on.

• 基于以yaml格式编写的模板使用各种AWS资源构建应用程序。

• 实质上是一个AWS CloudFormation 的扩展

  • 基于AWS CloudFormation 并且为无服务器做了优化，
  • 简化了无服务器资源的管理，增加了无服务器相关的新资源类型。
  • AWS CloudFormation标准模板语法比较复杂，SAM模板提供了一套简化的语法
  • SAM 基于 CloudFormation，所以也是支持YAML 和JSON两种格式。

# 模板格式的版本号和Transform声明。
# Transform声明告诉 CloudFormation这是一个 SAM 模板，需要转换成标准模板再执行。
# 它的值取固定值，这里是 AWS::Serverless-2016-10-31，告诉CloudFormation这个模板里的声明都是无服务器应用的描述，以及进行相应的转换。
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

# 具体资源的声明：
Resources:
  GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: s3 file location
      Handler: index.gethtml
      Runtime: nodejs4.3
      Policies: AmazonDynamoDBReadOnlyAccess
      Events:
        GetHtml:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: ANY
  ListTable:
    Type: AWS::Serverless::SimpleTable

---

Benefits of using AWS SAM

Because AWS SAM integrates with other AWS services, creating serverless applications with AWS SAM:

Single-deployment configuration

• easy to organize related components and resources, and operate on a single stack.
• use AWS SAM to share configuration (such as memory and timeouts) between resources, and deploy all related resources together as a single, versioned entity.

Extension of AWS CloudFormation

• Because AWS SAM is an extension of AWS CloudFormation, you get the reliable deployment capabilities of AWS CloudFormation.
• You can define resources by using AWS CloudFormation in your AWS SAM template.
• You can use the full suite of resources, intrinsic functions, and other template features that are available in AWS CloudFormation.

Built-in best practices

• You can use AWS SAM to define and deploy your infrastructure as config.
• This makes it possible for you to use and enforce best practices such as code reviews.
• Also, with a few lines of configuration, enable safe deployments through CodeDeploy, and enable tracing by using AWS X-Ray.

Local debugging and testing

• The AWS SAM CLI lets you locally build, test, and debug serverless applications that are defined by AWS SAM templates.
• The CLI provides a Lambda-like execution environment locally.
• It helps you catch issues upfront by providing parity with the actual Lambda execution environment. To step through and debug your code to understand what the code is doing,
• use AWS SAM with AWS toolkits like the AWS Toolkit for JetBrains, PyCharm, IntelliJ, Visual Studio Code. This tightens the feedback loop by making it possible for you to find and troubleshoot issues that you might run into in the cloud.

Deep integration with development tools

• You can use AWS SAM with a suite of AWS tools for building serverless applications. You can discover new applications in the AWS Serverless Application Repository. For authoring, testing, and debugging AWS SAM–based serverless applications,
• use the AWS Cloud9 IDE. To build a deployment pipeline for your serverless applications,
• use CodeBuild, CodeDeploy, and CodePipeline.
• use AWS CodeStar to get started with a project structure, code repository, and a CI/CD pipeline that’s automatically configured for you. To deploy your serverless application,
• use the Jenkins plugin.

---

SAM 模板特有资源类型

SAM 模板 7个特有资源类型

• AWS::Serverless::Function
• AWS::Serverless::Api
• AWS::Serverless::SimpleTable
• …
• 这是当前Transform版本为 AWS::Serverless-2016-10-31所支持的特有资源类型。
• 将来还有可能增加更多资源类型，升级换用相应的Transform版本号。

AWS::Serverless::Function

• Lambda函数，
• 模板中包括Lambda函数所有的属性，如Handler、运行时、代码地址、描述等等。
• Events用来声明事件源，同一函数可以支持多个事件源。
  • AWS Lambda 是事件驱动的无服务器函数服务，所以事件源也是部署Lambda函数的重要属性。
  • 事件源可以有很多种，大体分为3类：
    • 数据状态变化，如S3对象的新增、删除。
    • 请求端点，这里主要指的是通过 API Gateway 暴露为对外服务的 HTTP API 接口。
    • 资源状态变化，如EC2实例的启动、停止等状态。
    • 具体产生的事件源来自这些服务：S3、SNS、Kinesis、DynamoDB、Schedule、CloudWatchEvent、AlexaSkill。
    • 各事件源的各种事件及属性全部支持。
• Policies 声明IAM策略。
• Environment 可以声明环境变量，可用于传递给 Lambda函数。
  • Lambda环境变量
  • Lambda环境变量是可以动态传递给我们的函数的键值对，比如IAM的验证凭据，API的密钥等等。
  • Lambda环境变量是Lambda服务本身的功能，在无服务器应用模型SAM里，我们可以方便地把环境变量管理起来。
  • 在SAM模板中以 Parameters 节点来声明环境变量。
  • 可以通过标准环境变量API使用，如 Node.js 的process.env 或 Python的os.environ，即Lambda的环境变量会添加到Node.js 的process.env 里，方便咱们开发时使用。
• 另外还有 Tags声明标签，这是 AWS 管理资源的通用功能，比如用于资源分组，账单和成本分解等。

AWS::Serverless::Api

• API Gateway，关于API的详细定义，在 DefinitionUri 指定的swagger.yml里。
• 其余的属性不多，主要是：
• StageName 阶段名称、
• CacheClusterEnabled 是否启用API Gateway的缓存，
• 以及 CacheClusterSize缓存的容量。
• 最后的Variables是传递给API 的参数，比如阶段参数，也是用来灵活部署的。

AWS::Serverless::SimpleTable

• 用于创建 DynamoDb 表。
• 需要声明主键、类型，以及配置的容量规模。

---

SAM code

$ pip install --upgrade awscli
$ pip install --upgrade aws-sam-cli
$ sam --version


$ tree
SAM-Tutorial
├── afterAllowTraffic.js
├── beforeAllowTraffic.js
├── myFunction.js
└── template.yml


$ sam package \
    --template-file template.yml \
    --output-template-file package.yml \
    --s3-bucket YOUR_BUCKET
# Uploading to c2fe022d92156eeb459d45f02a766921  4113 / 4113.0  (100.00%)
# Successfully packaged artifacts and wrote output template to file package.yml.



$ sam deploy \
  --template-file package.yml \
  --stack-name my-date-time-app2 \
  --capabilities CAPABILITY_IAM
    # Deploying with following values
    # ===============================
    # Stack name                 : my-date-time-app2
    # Region                     : None
    # Confirm changeset          : False
    # Deployment s3 bucket       : None
    # Capabilities               : ["CAPABILITY_IAM"]
    # Parameter overrides        : {}

$ aws lambda invoke \
  --function arn:aws:lambda:region:xxxx:function:my-date-time-app2-myFunction-1S9F \
  --payload "{"option": "date", "period": "today"}" out.txt
# {
#     "StatusCode": 200,
#     "ExecutedVersion": "$LATEST"
# }


# update
# sam build and sam deploy can be leveraged to deploy changes for testing
$ sam build & sam deploy


SAM init

---

example

使用SAM模板管理Lambda环境变量

https://github.com/xfsnow/serverless/tree/master/sam/parameters

Parameters:
# 声明了一个Lambda环境变量，变量名是MyEnvironment。
# 属性类型是字符串，默认值是testing，可取值是testing、staging和 prod。
# 描述, 说明它们具体的使用情况。
  MyEnvironment:
    Type: String
    Default: testing
    AllowedValues:
      - testing
      - staging
      - prod
    Description: Environment of this stack of resources

# 然后在声明Lambda函数时在 Variables 段声明一个环境变量S3_BUCKET
# 它的值使用CloudFormatioin内置函数 !Ref 读取SAM模板中的环境变量MyEnvironment。
ApiHelloFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs6.10
      Environment:
        Variables:
          S3_BUCKET: !Ref MyEnvironment

# 相应地，在Lambda函数代码中，index.js 中上述这段就可以通过全局变量process.env 获取到S3_BUCKET这个环境变量值了。
# var bucketName = process.env.S3_BUCKET;

---

用CloudFormation部署SAM模板

# 把文件上传到S3并打包成可用于 CloudFormation 的模板文件。
aws cloudformation package \
    --template-file parameters.yaml \
    --s3-bucket <bucket-name> \
    --output-template-file packaged_parameters.yaml

# –parameter-overrides MyEnvironment=prod 表示部署时为 CloudFormation 的模板参数指定值为 prod。
aws cloudformation deploy \
   --template-file output_parameters.yaml \
   --stack-name parameters \
   --capabilities CAPABILITY_IAM \
   --parameter-overrides MyEnvironment=prod

# 顺利地话，会看到逐渐输出的返回结果。
Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - lambdaProxy


# 这时到 CloudFormation 的控制台已经创建出一个 lambdaProxy，整个过程大约持续 1 到 2 分钟。
# 然后到 API Gateway 控制台，可以看到创建出的 lambdaProxy 的 API
# 点击其 Stages 下的 Prod 链接，可以看到形如下面的调用 URL:
# Invoke URL: https://xxxxxxxxx.execute-api.my-region.amazonaws.com/Prod

# 点击它，打开一个新窗口，显示
{“bucketName”:”prod”}
# 表示已经部署成功。


# 再执行一次 aws cloudformation deploy，把 MyEnvironment 参数变成 testing
aws cloudformation deploy \
   --template-file output_parameters.yaml \
   --stack-name parameters \
   --capabilities CAPABILITY_IAM \
   --parameter-overrides MyEnvironment=testing
# 等待执行完毕后，刷新刚才的调用 URL，可以看到内容已经更新成了
{“bucketName”:”testing”}
# 这个例子演示了我们在SAM模板中定义的环境变量在具体部署时可以灵活赋成不同的值，然后部署出相应的效果。

---

Deploying a Hello World application

This application implements a basic API backend. It consists of an Amazon API Gateway endpoint and an AWS Lambda function.

• When you send a GET request to the API Gateway endpoint, the Lambda function is invoked.
• This function returns a hello world message.

# Prerequisites
# Creating an AWS account.
# Configuring AWS Identity and Access Management (IAM) permissions.

# Installing Docker. prerequisite only for testing your application locally.

# Installing Homebrew.
# Installing the AWS SAM command line interface (CLI).
# Check the version
sam --version command.

# If you select the Image package type, having an Amazon Elastic Container Registry (Amazon ECR) repository URI to perform a deployment.


# ------------------ Step 1 - Download a sample application
sam init
# This command creates a directory with the name that you provided as the project name. The contents of the project directory are similar to the following:
 sam-app/
   ├── README.md
   ├── events/
   │   └── event.json
   ├── hello_world/
   │   ├── __init__.py
   │   ├── app.py            #Contains your AWS Lambda handler logic.
   │   └── requirements.txt  #Contains any Python dependencies the application requires, used for sam build
   ├── template.yaml         #Contains the AWS SAM template defining your application's AWS resources.
   └── tests/
       └── unit/
           ├── __init__.py
           └── test_handler.p


# ------------------ Step 2 - Build your application
cd sam-app
sam build
# sam build command builds any dependencies that your application has,
# and copies your application source code to folders under .aws-sam/build to be zipped and uploaded to Lambda.
# You can see the following top-level tree under .aws-sam:
 .aws_sam/
   └── build/
       ├── HelloWorldFunction/ # directory contains app.py file, third-party dependencies that app uses.
       └── template.yaml


# ------------------ Step 3 - Deploy your application
sam deploy --guided
# This command deploys your application to the AWS Cloud.
# It takes the deployment artifacts that you build with the sam build command, packages and uploads them to an Amazon Simple Storage Service (Amazon S3) bucket that the AWS SAM CLI creates, and deploys the application using AWS CloudFormation.
# In the output of the sam deploy command, you can see the changes being made to your AWS CloudFormation stack.


# ------------------ Step 4 - test your application
# If your application created an HTTP endpoint, the outputs that sam deploy generates also show you the endpoint URL for your test application. You can use curl to send a request to your application using that endpoint URL. For example:
curl https://<restapiid>.execute-api.us-east-1.amazonaws.com/Prod/hello/
#  {"message": "hello world"}


# ------------------ Step 4: (Optional) Test your application locally
# When you're developing your application, you might find it useful to test locally. The AWS SAM CLI provides the sam local command to run your application using Docker containers that simulate the execution environment of Lambda. There are two options to do this:
# Host your API locally
sam local start-api
curl https://127.0.0.1:3000/hello

# Invoke your Lambda function directly
sam local invoke "HelloWorldFunction" -e events/event.json

Process Amazon S3 events

# Step 1: Initialize the application
# download the sample application
# consists of an AWS SAM template and application code.

sam init \
  --location https://github.com/aws-samples/cookiecutter-aws-sam-s3-rekognition-dynamodb-python \
  --no-input

# Review the contents of the directory that the command created (aws_sam_ocr/):
# template.yaml – Defines three AWS resources that the Amazon S3 application needs: a Lambda function, an Amazon S3 bucket, and a DynamoDB table. The template also defines the mappings and permissions between these resources.
# src/ directory – Contains the Amazon S3 application code.
# SampleEvent.json – The sample event source, which is used for local testing.



# Step 2: Package the application
# create a Lambda deployment package, which you use to deploy the application to the AWS Cloud.
# This deployment creates the necessary AWS resources and permissions that are required to test the application locally.

# Create an S3 bucket where to save the packaged code.
aws s3 mb s3://bucketname

# Create the deployment package
sam package \
    --template-file template.yaml \
    --output-template-file packaged.yaml \  # specify the new template file
    --s3-bucket bucketname



# Step 3: Deploy the application
# test the application by invoking it in the AWS Cloud.

# To deploy the serverless application to the AWS Cloud
sam deploy \
    --template-file packaged.yaml \
    --stack-name aws-sam-ocr \
    --capabilities CAPABILITY_IAM \   # allows AWS CloudFormation to create an IAM role.
    --region us-east-1



# Step 4: test the serverless application in the AWS Cloud
# Upload an image to the Amazon S3 bucket that you created for this sample application.
# Open the DynamoDB console and find the table that was created. See the table for results returned by Amazon Rekognition.
# Verify that the DynamoDB table contains new records that contain text that Amazon Rekognition found in the uploaded image.



# Step 4: Test the application locally
# retrieve the names of the AWS resources that were created by AWS CloudFormation.

# Retrieve the Amazon S3 key name and bucket name from AWS CloudFormation.
# Modify the SampleEvent.json file by replacing the values for the object key, bucket name, and bucket ARN.

# Retrieve the DynamoDB table name. This name is used for the following sam local invoke command.

# generate a sample Amazon S3 event and invoke the Lambda function:
TABLE_NAME="Table name obtained from AWS CloudFormation console"

sam local invoke --event SampleEvent.json

# The TABLE_NAME= portion sets the DynamoDB table name.
# The --event parameter specifies the file that contains the test event message to pass to the Lambda function.
#  now verify that the expected DynamoDB records were created, based on the results returned by Amazon Rekognition.

---

SAM + Lambda

code:

from __future__ import print_function

import json
import urllib
import boto3

print('Loading function')

s3 = boto3.client('s3')

def lambda_handler(event, context):
    # Get the object from the event and show its content type
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.unquote_plus(event['Records'][0]['s3']['object']['key'].encode('utf8'))

    try:
        response = s3.get_object(Bucket=bucket, Key=key)
        print("CONTENT TYPE: " + response['ContentType'])
        return response['ContentType']

    except Exception as e:
        print(e)
        print('Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket))
        raise e

resource:

• S3 的 bucket: test4lambda
• key: lambda-test.txt

test

sam local generate-event s3 \
  --region us-east-1 \
  --bucket test4lambda \
  --key lambda-test.txt > event.json

生成如下用例 event.json

{
    "Records": [
        {
            "eventVersion": "2.0",
            "eventName": "ObjectCreated:Put",
            "eventTime": "1970-01-01T00:00:00.000Z",
            "userIdentity": {
                "principalId": "EXAMPLE"
            },
            "eventSource": "aws:s3",
            "requestParameters": {
                "sourceIPAddress": "127.0.0.1"
            },
            "s3": {
                "configurationId": "testConfigRule",
                "object": {
                    "eTag": "0123456789abcdef0123456789abcdef",
                    "key": "lambda-test.txt",
                    "sequencer": "0A1B2C3D4E5F678901",
                    "size": 1024
                },
                "bucket": {
                    "ownerIdentity": {
                        "principalId": "EXAMPLE"
                    },
                    "name": "test4lambda",
                    "arn": "arn:aws:s3:::test4lambda"
                },
                "s3SchemaVersion": "1.0"
            },
            "responseElements": {
                "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnGH",
                "x-amz-request-id": "EXAMPLE123456789"
            },
            "awsRegion": "us-east-1"
        }
    ]
}

编写 yaml 配置文件

• 到 lambda 后台导出函数
• 下载 AWS SAM 文件

得到的配置文件如下

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An Amazon S3 trigger that retrieves metadata for the object that has been updated.
Resources:

  myTest1:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: lambda_function.lambda_handler
      Runtime: python2.7
      CodeUri: .
      Description: An Amazon S3 trigger that retrieves metadata for the object that has been updated.
      MemorySize: 128
      Timeout: 3
      Role: 'arn:aws:iam::851829110870:role/service-role/lambdaTest'
      Events:
        BucketEvent1:
          Type: S3
          Properties:
            Bucket:
              Ref: Bucket1
            Events:
              - 's3:ObjectCreated:*'
      Tags:
        'lambda-console:blueprint': s3-get-object-python

  Bucket1:
    Type: 'AWS::S3::Bucket'

运行

sam local invoke myTest1 \
  -e event.json \
  -t myTest1.yaml

# "time out after 3 seconds"
# 解决方法: 在 myTest1.yaml 找到 Timeout 字段, 改成 30

# 再次运行, 成功输出 "text/plain", 也就是 python 代码那句 print 语句的结果

.